It is safe to say slow breeding bucks in my breeds of choice were just not bred, and went in a pot instead, for many generations.
There is no question that it is important to select for important traits. The hard part is starting with less than ideal stock and trying to get enough offspring to select from. Especially when you have bucks that just aren't interested in breeding. I had that problem with Champagne d' Argent rabbits, the bucks were so laid back, they had no interest in breeding. It wasn't weather related at all, and they weren't too young. I had purchased a group of the rabbits, beautiful, sweet, easy to work with, good conformation, in good condition. . .but the boys had no interest at all in the girls. None of the tricks I tried worked, not changing light conditions, nutrition, swapping pens with the girls, time of day. . .the boys got along with the girls just fine, they just didn't want to breed. No way to select for more aggressive breeding in the males when you can't get offspring.
With the Angora rabbits, especially the Satin Angoras, replacement stock can be very hard to come by, so when a buck doesn't want to breed, it's a major issue. Young males can quickly be frightened by an aggressive doe, it's hard for them to overcome that fear.
My husband wanted a nice uniform herd of black based Silver Fox rabbits, but the starter herd he purchased had some blue in the background. One beautiful big blue buck was born, grew spectacularly, fabulous conformation and personality. He kept the buck, but didn't really intend to use him, since he didn't want blue kits. But when the other bucks didn't want to breed, that blue buck was always ready to go--with quick, efficient breedings every time. He definitely had the right stuff as far as breeding went, but he was the wrong color. Which leads to another issue, often the rabbit with the 'right' color or whatever trait you deem important--doesn't want to breed. Leaving you with another you have to use that breeds well, but may transmit other traits you are not looking for.
My friend has solved this problem with a colony set up in an old chicken barn. Even the reluctant breeders, male and female, started breeding. She ended up with more rabbits than she imagined possible. Does that never conceived had litters, bucks that never wanted to breed sired kits. By stretching out on the cool concrete floor in the summer, bucks avoided the whole heat sterility issue, so she has litters all summer long.
Generally here, bucks not wanting to breed isn't a heat issue. Summer sterility is just that, the bucks breed, have multiple fall-offs, but the does don't conceive. I have a few bucks that are anxious to breed no matter the time of year, and have good aim, so they catch the doe early in the chase. They are treasured. Other bucks are more clueless, don't line up properly, and exhaust themselves in a position that will never breed the doe. Trying to reposition them properly just confuses them and they stop trying. And still other bucks just don't care, and just lay there and ignore the doe. When you are trying to breed for specific colors, patterns, or fiber types, and you end up with a single buck of that trait, and it is maddening when they don't perform to continue the trait.
It is so much simpler to select for one trait, such as meat conformation, and breed strictly for that one trait. Genetically, you can make tremendous progress in a short time that way. When you are trying to select for many lesser traits at the same time, progress gets stalled, as you are selecting animals as breeders that may not have the best growth rate, or the best personality, or the best breeding skills, simply because they have a color you want, or a fiber/fur type, or a pattern that interests you.
With sheep, there are breeders that select only the fastest growing, best-looking ram lambs from the spring crop to be their fall breeders, no adult rams are used. Only the biggest and best of that year's crop get a chance to breed, and only the most fertile will succeed, thus making quick progress in breeding for those traits.
My herd is separated into a half-dozen mini-herds, as some colors genetically do not combine well with other colors, and having a variety of colors of angora to spin of a very soft texture that doesn't mat on the rabbit is my goal. Satin Angoras are separate from English Angoras. The harlequins are their own mini herd with the fawn rabbits (the recessive to harlequin). The chinchillas (including ermine, which is chin plus the non-extension fawn gene, which I use as my 'white' fiber) are their own herd--as chin, like harlequin, can mess with other more dominant colors. Non-agouti black and chocolate are their own herd, trying to breed for darker, more uniform color on the hairshaft. . .you get the idea. So even though I have a fair number of rabbits, each little group only has a buck or two to choose from. And that is where trouble can begin, when you only have one buck to choose from, and he just doesn't wanna perform. Or he tries, but the doe never settles.
The quick solution is to go back and make a priority of strong breeding traits, get a herd that breeds well, and then start selecting for more minor traits such as fiber/fur and color. But it's hard to discard many years of breeding for the fiber in the process, the best fibered bucks are rarely the best breeders, and some color patterns have associated health issues (like broken and megacolon) that may impede selection of the best bucks. It is so hard to get a harlequin with the proper pattern, with the proper color changes on the face, ears, legs, etc., and the proper banding--that when one getting even close to that ideal is born, you want to use it as a breeder. But what are the chances that it will be a good and lusty breeder, and pass on those gorgeous pattern traits?