The problem with keeping males and females together is that you'll have a LOT of babies. Also, the females won't have any rest between litters. This shortens the life span of your females. If what you are planning is to breed for feeders (or to sell to petstores), then I'd say that a male per every 6-7 females would be ok... but... remember that you shouldn't keep more than 1 male together... so having a huge mixed colony in a 40g is not a good idea. If you want a large colony in the 40g, then I suggest you to just keep a big colony of females in it, and keep a smaller tank or 2 with the breeders. Remember that you have to know what you will be doing with the babies before you start breeding. To know how many mice you can fit in a 40g, check out this page, just put the dimensions of your cage and it will give you the # of mice that can fit there.
http://www.geocities.com/funchy_crunchy/mcage.htm#calc