We had really reduced fertility in all the rabbits - not just the bucks - when they had syphilis aka 'vent disease'. Not sure if that's what your buck has, though. Syphilis is usually noticed as small sores and scabs on their faces or genital areas. The swollen and the clump of hair is different.
Your three basic options for any sort of rabbit distress is environmental, bacteria or virus. If it's environmental, it could be something along the line of urine scald. Does he have a hard surface to pee on that may splash back on him? Maybe a dirty area he can stick his bum on? If he has a sore area, maybe there's a wetter cooler area where he's trying to cool the burn, although if it's wetter and cooler because of urine or feces, well, that's not gonna help things in the long run even if it may briefly feel cooler.
If it's bacterial, then there's several different anti-bacterial medicines for bunnies. Even the basic Triple Antibiotic in your medicine cabinet may be of use? Check which version can be used for rabbits, there's some online discussion about several different varieties of neosporin. We occasionally use Triple Antibiotic along with Ivermectin and Pen G penicillin. The last one is an injection and a cure for syphilis. Although, we don't know what exactly is wrong with the rabbit. It may be more than one thing with the swelling a separate issue from the infertility.
If it's viral, there's not much that can be done except for alleviate the symptoms although there are some vaccines for rabbits, but I don't know where you'd get them other than a vet? I've not had much experience with viral problems in rabbits.
However, as a caveat to all of the above, I am NOT a veterinary, I don't even play on on TV so this advice is not professional advice, heed it at your own risk.
