Sounds like you need to start potty training.
When you go to pick them up, hold them over newspaper and say "go potty". Then carry them around for a little bit. In about 15 mins go back over to the newspaper, hold them over it and say "go potty". If they go, make a huge deal of it, getting all excited. If they don't, just quietly take them away and bring 'em back in a few minutes and try again. Pretty soon they will associate "go potty" with the act.
It took Toby less then a week to be potty trained. I trained him when he was about 3 months old. He's not perfect and he does have accidents but for the most part, he's great. Babies have to go more often than adult birds so take that into consideration also.