I think you described it perfectly.
I am supercharged. I can't really lay into it until 3rd gear. At that point you are already moving along, and the boost is manageable. Same with downshifting with purpose, like passing or avoidance, the boost is there when you want it, and you expect it so it's manageable.
Having said all that, I got 3000 miles out of my last rear tire. I mean 3000 smiles. I had a blast.