So even though the doe is not letting them feed at five weeks, they are not ready to wean?
You
can wean them now, and they will likely be okay as long as they are kept on the same diet they've been on. In the wild, kits are sometimes weaned at 4 weeks so that the dam can raise another litter. However, a lot of people who raise rabbits like to wait for at least another 3-4 weeks, to wean at at least 7-8 weeks, for the general health and well-being of the bunnies.
I've weaned litters anywhere from 4 weeks to 12 weeks. They generally all do just fine; I haven't noticed any more problems with weaning enteritis or behavior depending on the age. In my barn, bunnies which were going to get weaning enteritis seemed to do so whether they were still with their dam or not. At least in my experience, it has more to do with genetics than anything (assuming they're being fed the same thing as the dam has been eating, which is quite important).
But... in my meat rabbits, which are the ones for which I have the most consistent data, the growth rates are
definitely affected by weaning age, at least in the short term. We have done some unintended experiments as a result of breeding meat pen bunnies for 4-Hers. Until recently, the rule in Alaska 4-H was that each 4-Her had to own their meat pen rabbits by the time the bunnies were 6 weeks old, so that there was time to actually raise the rabbits (meat pens are typically three 10 week old bunnies). So, I'd sell 6-week-old bunnies to other 4-Hers, and my kids would keep others for their own meat pens. In a meat pen, the fryers are supposed to be matched in terms of color, which for us often means that the pens are made up of bunnies from more than litter. What we saw was that the bunnies sold at 6 weeks hit a distinct growth plateau after they were moved to a different barn, which meant they were smaller and less well-conditioned than the ones we kept. Those were left with their dams till the Fair, and never experienced a plateau, so they were bigger and in better flesh condition than the ones weaned at 6 weeks, even though they were all from the same several litters.
We know that the 6-week-weaned bunnies were taken care of well, and were kept on the same type of pellets that we use. We also know that some of those bunnies that weren't sold at auction eventually did catch up with the 10-week-weaned bunnies, so they weren't stunted forever. It could be that it was the change of barn that stressed them, but in my own barn, I also generally see a growth plateau when I wean rabbits. Interestingly, it doesn't seem to be directly related to age at weaning.
Here are charts of growth rates of two rabbits left with their dams as long as the charts cover (they're both does since I wouldn't leave bucks past 12 weeks). There are slight variations in the rates, but no distinct plateaus:
Below are three charts from rabbits born on 6/12 and weaned on 8/29, at 11 weeks of age. You have to pay attention to the dates since some charts cover much more time than others (the first chart doesn't extend past the plateau), but you can see that there is a distinct break in growth rate at exactly the time of weaning: