Mountain Men and Trappers had beards. Imagine how germ ridden they were. My wife's uncle has a big beard and I've often wondered how clean it is. It never looks clean. You'd think with such a level of filth that they would all stink.
When I was a kid, I spent a ton of time in a local swamp. I inspected dead animals regularly and was far enough from home that washing my hands was usually forgotten until supper time. I held just about any insects I could find, ditto turtles and snakes. General illnesses were rare for me. So how bad can a dirty beard be? Mountain Men were very hardy, maybe our germaphobio is out of hand. Exposure to germs strengthens the immune system.