Change the locks.
How terrible for you to not be able to control your cats' safety. I'd be afraid to leave the house.
What on earth is it to them that you keep your cats in, where they are SAFE and will live longer healthier lives?
Put inside bolts on all the doors, so you only have to change the lock on one door. Bolt the others when you aren't home.