What you're describing is pretty normal guinea pig behavior. You can give them veggie treats whenever you come by the cage, and attempt to move as slowly as possible, but to some extent most guinea pigs will always be shy of hands in their cage. They're prey animals!
We are as gods to the beasts of the fields. We order the time o' their birth and the time o' their death. Between times, we ha' a duty. - Terry Pratchett.
"Men have forgotten this truth", said the fox, "But you must not forget it. You become responsible, forever, for what you have tamed. - Antoine de Saint-Exupéry