It's been a long time since I had rabbits in cages, but I tended to leave them with mom for a bit longer than four weeks, unless there was a reason to do otherwise. It's always been my feeling that there is less chance of weaning enteritis if weaning is a process rather than an event.
I'm not even sure it is the sudden change in diet that can occasionally cause problems. It may be due to stress. This can be minimized by moving mom instead of the kits, but you can't do this if you want to wean one of two at a time... and a lot of people prefer to do this for the doe's sake, to minimize that chance of mastitis. Mint and sage are good herbs for drying up the milk supply, provided your rabbits are accustomed to greens.
Try observing the kits for a day or two to help determine if they are already weaned. Are they still trying to crawl under mom to nurse? Is she acting irritated with them? (Some does love having their older babies with them; others not so much.)
Another consideration is whether or not the doe has been rebred. Most people like to give the does a break from the company of youngsters before they kindle again.
As with so many questions about rabbit husbandry, there is no "one right answer". As you gain experience with your rabbits, you will find what works best for you.