The tank is a 55-gallon then, right? That's kinda small for five goldfish...even if they're fancies. But you seem to be keeping up on water changes so that's good .
Have you tested the water? Maybe it's a nitrate problem. Also, what kind of filtration do you have? Goldfish need pretty strong filters...multiple HOBs or a good canister filter is the way to go.
This can be a pretty annoying thing to get rid of. But to start you'll need to find the source of the problem which is probably too much light (try leaving the tank light off and make sure it's not near a window) or improper filtration/not enough water changes.