One thing that's pretty cool to use are those chemical reaction hand-warmer things that stay warm for hours. You can sometimes find em on sale in bulk for about $20 for 50 pairs of em.
If you wanna warm up your fingers you just rip off the plastic pouch and stick one in a pocket...
When you're playing you can slip your hand into your pocket between songs and it's kinda like running your fingers under warm water without getting em all wet & sticky.