Sometimes, you don't want to carefully sort your files; you just want to delete them, and be sure they really are gone for good. For those times, the simplest, most bare-bones options is SDelete, a free command line utility directly from Microsoft's Windows Sysinternals.
You can use this 81KB download to delete specific files, but you can also have it wipe all free space on your hard drive to securely remove all traces of old files. After running it with the -c ("Clean free space") option, your drive should be pretty much impervious to attempts to recover deleted files (as long as they're not in the Recycle Bin awaiting easy retrieval, of course).
SDelete can work well as a scheduled task, periodically cleaning your drive. Because it is so minimal, you'll have to set up that task manually, using the Task Scheduler built into Windows.