Yes, as soon as they stop eating it is a serious problem. Rabbits are designed to have something moving through their systems almost 24/7. That's why they need unlimited amounts of hay. Longer than an hour or two is a concern, especially if they are also lethargic -- it means their GI tract has already begun to shut down.
We are as gods to the beasts of the fields. We order the time o' their birth and the time o' their death. Between times, we ha' a duty. - Terry Pratchett.
"Men have forgotten this truth", said the fox, "But you must not forget it. You become responsible, forever, for what you have tamed. - Antoine de Saint-Exupéry