Petting him while he's growling is only reinforcing the idea to him that growling is an acceptable behavior. Also, you are presenting yourself as a threat, since every time you're around, his food disappears. But take heart, it's a fixable problem.
I would start by hand feeding him all his meals. He doesn't get one kibble that he doesn't gently take out of your hand, and preferably only after he obeys a command. This is part of a technique known as "NILIF" or "Nothing In Life Is Free". It's also a great opportunity to work on any other training you want to do with him. So for the next meal, don't just pour the food in the bowl from a cup. Take a couple of kibbles and have him sit. Once he sits, then give him those kibbles. Lather, rinse, repeat, varying the commands so he doesn't get bored and frustrated. I would do this for at least a couple of weeks.
Once you have established that all food comes directly from you, move to putting the kibble in his bowl one handful at a time, so that you are establishing that hands in the bowl means more goodies for him instead of a threat or competition. This should also extend over a period of a few weeks.
When he is comfortable with hands in his food dish giving him food, go pack to pouring his food into his dish all at once, make him do a sit/stay or down/stay before he can start eating. Have a tasty treat ready, like a bit of cooked chicken breast or beef jerky if he doesn't have a beef or poultry allergy. Walk away at first, then come back, casually toss a bit of the treat into his bowl and walk away again...but only if he's not growling at you. Again, you don't want to reinforce the idea that growling over food is good.
All praise should come when he's happy and relaxed and taking food nicely from your hand. Good luck, keep us updated and don't be afraid to ask any other questions! Do you have any pictures?