It sounds like you don't have a UVB bulb, first of all, so definitely get one asap. If it's warm enough, bringing him outside for some sunlight will work in the meantime. UVB bulbs can be bought at pet stores and you should leave it on during the day
Also, you said that it's warm enough without any heating but I would actually offer a heat lamp anyway. That way your tank has a temperature gradient (a warm side and a cool side) and your turtle can regulate his body temperature better
The last thing I would do is test the water. You can buy water test kits at most pet stores (in the fish section) or you can bring in a water sample for the store to test (it's usually free). Check to make sure you don't have any ammonia or nitrites. New aquariums are prone to spikes and this can cause a lot of stress for the animals. The fact that he isn't swimming in the water much is telling you that something's wrong with the water (wrong temperature, ammonia spikes, etc.) or that he is trying to get UVB (and is therefore basking more than usual).
Oh, and try some frozen and/or live foods if you can. Some turtles just don't like pellets (and they're not as healthy). Bloodworms, brine shrimp, krill, earthworms, feeder minnows, snails, ghost shrimp, blackworms, mealworms, and crickets are some foods you could try. Don't worry about him not liking plant matter--they usually don't eat much of it when they're young.
How often is he yawning? Turtles do yawn but constant yawning is usually a sign that the turtle is getting sick or that something's wrong with the water.