Just because there weren't signs up doesn't mean there aren't leash laws, but since you say there's very rarely anyone there besides you, I can see why your dogs were off-leash.
But, just because they weren't showing any signs of aggression doesn't mean he doesn't have any right to be scared. Maybe he has had bad experiences with dogs in the past, maybe he's not a dog lover, maybe he thought they were mean, or maybe, like me, he's not too fond of dogs he doesn't know running up to him. Whatever the cause, I agree it was rude of him to yell at your dogs, but he had every right to be scared.