Eating alfalfa hay shouldn't hurt them, the only issue is that it has higher levels of calcuim and protien and lower fiber. The high calcuim content can lead to urine sludge, the high protien can cause them to gain more weight. But if the alfalfa is mixed in with timathy and they're getting both, they should be okay.
Are you sure the flower is from alfalfa though? From my experience, if there is any kind of weeds or anything in the field when they harvest the hay, they don't seperate it out. I've gotten bales of hay with some dried up dandelion leaves in it. I've also found some dried up dead grasshoppers in the hay.
What kind of hay are you feeding them?