This will sound awful, but my father was talking to an English breeder, and for some reason labs, specificially chocolates came up, and appearently, in some circles, if a chocolate happens to be born, they are put down immediately, because they are 'crazy' . . . I would take that to mean more hyper and unfocused. This was years ago, and the breeder in question bred for field work . . . but . . . well, that's what I know on the topic. I've heard that silver poodles are supposed to be flaky . .. there are alot of those breed/color myths . . . I suspect they have some truth in them, but only because the original lines bred to get that color had a temperment issue . . . I doubt its the COLOR that does it. So, I would suspect that there are plenty of just fine chocolate labs out there, whatever this guy thought. But if you had a line where there were lots of chocolates, bred only to chocolates, for a long time, then the 'craziness' might come out.
As for all breeding for color being byb, that might be a bit much. Yellow and Chocolate are legitimate lab colors, and assuming conformation and temperment are good, I see no reason not to breed a yellow to a yellow rather than a black . . . after all, if we didn't do that, all dogs would end up one of the dominant colors, except for occasionally throws (black is dominant, so two black dogs bed together, or a black and yellow, can produce a yellow, because they can carry yellow). I don't know about you, but a world of black, brindle and merle dogs would get pretty old (those being the genes I know to dominant for color, I'm sure there are few more). Breeding purely for color, like breeding purely for certain conformation traits (as opposed to good solid basic build, with the breed traits tacked on) it of course, very unwise.