First, if you don't want 2 cages, I don't think you want a male mouse in with 3 females. Average mouse litter is 11, but can go up to 25. Then, all the males you have you'll probably have to keep in
separate cages cause males fight other males... I am currently breeding mice (couple of litters) and I have 13 cages devoted to my mice only.
Then, a male mouse will kill babies of a female if they are put together when she's about to give birth. If introduced early in the pregnancy, you can trick him into thinking that the babies are his, but if you put them together when she's about to give birth, he'll kill the babies so the female will mate with him and have babies of his own.
This said, I suggest that you put the little guy in a separate cage, or, if you really can't have 2 mice cages, then try to find him a good home.