SVG Explorer is an open source tool which allows you to view SVG files as thumbnails in Explorer.
The program requires installation, and it's important to make sure you get the right version for your operating system (32 or 64-bit), but there's no adware or other hassles - just a few DLLs.
You don't have to reboot to see the results. Just open an Explorer window, browse to a folder containing SVGs, select a Medium/ Large/ Extra Large Icon view and watch as the thumbnails appear.
This all worked just fine for us, but if you have problems, SVG Explorer can be uninstalled via the Programs and Features applet, which should return your system to normal.
It may only be version 0.1.1, but SVG Explorer worked well for us, rendering all our test images without difficulty.