HTTPS has a lot of opportunity for tuning on the server side that can significantly impact the speed at which your content is delivered.

Ilya Grigorik collected the current best practices on istlsfastyet.com and has published his excellent Velocity Conference talk on the topic to YouTube.