I'd bet you're right that it was stress causing the panting; the shivering could be due to the same thing. 50s in the day and 20s in the night shouldn't be too hard on him, even if he was in a bit warmer spot before. Those aren't extreme temperatures for a rabbit, but the added stress of constant moving, in and out and into different cages might tip him over the edge. Rabbits need a home base to feel secure, and he's probably pretty confused right now. If it was me, I'd put him where he's going to be, and leave him to adjust.
Cover the cage with a blanket, sure. But ever since a brooder lamp short burned down our greenhouse with our chicks and bunnies in it - it was horrible - I've stayed away from any electric heat in my barn.
But we've found that even our smallest bunnies do great if we give them a nesting box (wood or even cardboard in a pinch) stuffed with hay or straw. We typically get down to -25F at least a couple of times a year, and have wide temperature swings (it was 39F the night before last and dropped to -2F overnight). But the rabbits can adjust on their own by burrowing into the hay or sitting on top of the box, as they wish. Just be sure to keep adding bedding material, especially if it's hay, since they'll eat their blanket. I'd go out of my way to get him situated where he'll be, with as few changes as possible, at least for a time while he gets settled in, though. Young rabbits are pretty sturdy, but in general, rabbits don't like change, at all.
