I'm still fairly new to fish keeping but I'm pretty sure Betta fish need around 2 gallons (don't quote me on that, I may be wrong). If I am right though, he could be suffering from stress through the lack of swimming space.
In a small tank, ammonia would build up real quick, maybe in a day which would mean your fish could be suffering from ammonia poisoning. You could try doing a 15% water change every day to see if that perks him up.
Bettas are warm water fish I think, too and I think optimum temperature for them is around 78 degrees (far.) which - off the top of my head - is around 25 degrees Celsius.
Make sure, when you do a water change, that you use a product like 'Tapsafe' to get the chlorine and/or chloramine out - if you havent done this before, the chlorine and/or chloramine might be 'burning' or poisoning your fish.
Like I said, I'm still fairly new to this myself but that's just my 2 cents.
Hope everything gets resolved!
edit: Sorry, I didn't see that you already had him in a 2 gallon tank!