I've kept mice for a few years, and am looking to bring my first rats home this Saturday, but have done endless studying up on rats for the last 6 months.
Rats are by far the more intelligent and social, so males and females alike can (and need to) be kept in same sex pairs or groups, and they need a lot more room and stimulation. They also have very specific health issues like tumors and being prone to respiratory infections.
All of these things makes them potentially much more expensive to keep, since more care must be made in their husbandry to reduce the threat of life threatening respiratory infections--open ventilation in the cage, non-aromatic low dust bedding, frequent cleaning to avoid ammonia build up, very specific hard to obtain healthy diet--and effort to keep them mentally stimulated and safe in their explorations--rat proofing their play area, investing in toys and mazes--though there are ways to be frugal about it, such as building your own cage and making your own toys.
They are probably the most expensive of pocket pets to keep properly, and some would argue that occasionally, they are more expensive than cats or dogs, but that can be debatable depending on your level of dedication and how sickly your individual rat might be.
Most of us get our first rats through pet stores, but by educating yourself, you'll see it is by far the worst option for getting them, since less care is made by the pet stores to insure the health and temperament of your new rat, and often young female rats will come home to you already pregnant because no care was made in separating sexes before purchase. Plus, many (not all) pet stores (*cough* petsmart *cough*) obtain their rats from rat mills which are every bit as awful as puppy mills.
You can hedge your bets in getting a healthier, more friendly rat by going through a private breeder where they are bred for both, but getting them from a rat rescue is another fabulous option. I've found many resources for obtaining a rat through grassroots communities online such as this one. Still, many great pets have come from pet stores, and sometimes that's the only option for finding a seller.
Moving on, rats are easily as intelligent as cats, and some dogs. This means they can bond with you personally and be taught tricks like coming when called, turning in circles, running a maze etc...though some individuals are smarter than others and can learn better. Females are silly and very active and entertaining. Males are lazy lap rats who enjoy good snuggles.
As for mice, while female mice do great in same sex colonies, male mice must be kept alone. They will kill other males, and endlessly breed with other females, neither of which is a good option. Males are super stinky especially compared to their size, and so in addition to separate containers for each of them, you'll also need to have an area where their strong smell won't be an issue.
They are sweet and trusting and literally like to ride around in your pocket if they are well socialized, which most of the time only requires patience, though poor breeding can mean you might obtain a mouse that is more nippy and skittish than usual. Best bet is to let the mouse choose you. If they run up to you and are inquisitive, they're a good choice.
They are super cheap to keep, as long as you don't let breeding go out of control. They do fine in tanks as long as you are careful about over crowding. My only reoccurring cost was aspen bedding and a bag of mouse block. I briefly kept 12 mice in a 20 gallon tank. I personally didn't go more than that because that seemed to be the 'break point' of difficulty in keeping it clean and the mice starting to seem a little stressed for crowding. I later decided 8 was the best number for that size tank.
So basically, rats are much more expensive and time consuming, but much more emotionally rewarding because of their level of devotion and interaction. Mice, not so interactive or time consuming, and definitely not as expensive. They are precious though, and very worthwhile in their own way.
This is probably more than you cared to know, but there ya go!