Well, first off, it's not unusual for them to go a week or two without food at first. I actually had one betta that went a month without eating anything! I kept trying and one day he decided to eat, haha.
Is the tank cycled or are you doing full water changes?
The filter might be too strong, yeah. Bettas don't like strong currents (although some enjoy playing in them!). What kind of filter do you have? Some of them can be turned down. If you're running something like a Whisper filter, consider replacing it with a smaller, quieter filter. One intended for 5-gallons would be perfect (a lot of 5-gallons come with filters rated for 10-gallons). The Azoo Palm filters (I think that's what they're called!) are pretty good for small tanks.
If you think lack of oxygen is the problem, make sure there are plenty of decorations in the main tank. That way he can rest on them if needed

. Also try live plants: they'll provide oxygen, hiding places, and resting spots. They'll even help out your tank's cycle. I'd recommend sticking to small, hardy species like java ferns, water sprite, and anubius nana.
What brand of pellets are you feeding? Popular brands tend to be HBH Betta Bites, Hikari Betta Bio-Gold, and Atison's. I would try those. You could also try frozen foods (freeze-dried foods are actually rather unhealthy and should be given as treats only, you don't want him to get used to eating them) such as brine shrimp and mysis shrimp. As a last resort, you could also try
live bloodworms, blackworms, brine shrimp, pinhead crickets, fruit flies, and/or earthworms. I say last resort because most people don't want to feed those foods all the time. If you don't mind, go for it

.
If none of those ideas work you might just have to wait and see what happens. It's not uncommon for bettas to be ill when you buy them so you may end up having to treat him for something.