I do the same thing - Layers! Tee shirt, then long sleeves, then sweater/light jacket, then heavy jacket. I even have insulated Jean's if need be. I remember one time in Oregon riding the Victory - 80 degrees one 1 side of the mountain and 50 on the other, literally just a couple miles apart.
Yup, layering is the key. Though, no matter how warm the temperatures are, we are always wearing a lighter, (moisture wicking?), long sleeve shirt and pants for sun protection.