The Prime Minister has given aged care operators an ultimatum to adhere to the restrictions on visitors recommended by the national cabinet and not go further, saying they could face a mandatory system.