I've been working in the testing team for a software project for the past month and a bit. And you know what? I'd forgotten just how hard it can be to work in this field. On the face of it, the role of a tester is a pretty simple one: you take the spec and the product, you devise a set of test plans from the spec, and then you run the tests to check that the product to check that it conforms to the spec, and if there's a problem you chuck it back at the developers. The reality isn't that simple, though.

Problem one is to define a set of tests that the business is happy with, but which can be carried out in the time available. Practically speaking, in the average project you can't test every last possible scenario - which means that right at the start you're into the process of trying to define a set of tests that's sufficiently detailed to provice an acceptable level of confidence that the product has been thoroughly tested whilst fitting into the time and resources available. The harder job, of course, is to get across to the business types that getting the testing done this century introduces a risk because you're not testing every last possible eventuality.

Problem two is that testing, by its very nature, comes at the end of the project. After all, you can't test a product or component until it's been built. This means that when the design schedule slips, then the production schedule slips, you can be pretty sure that the launch date will stay put. The result is, of course, that testing starts late but generally still gets lumbered with the original deadline - thus squeezing the schedules.

Then you get good old "scope creep". During the design phase it's realised that a feature was missed from the original spec, or that some existing component that was going to be used isn't going to work, so a new one has to be built/bought. These get designed and implemented, but when the original guesswork was done on the testing schedule (and it usually is guesswork - after all, you can't have more than a vague idea about test schedules when a product is merely some scrawls on a flipchart) they were obviously not considered.

And then regression testing comes along and bites you on the arse. If you don't know what regression testing is, think of the old adage: the act of fixing a bug often results in one or more new bugs being introduced. Regression testing is the act of re-running all your old tests on the new revision of the product to make sure the old stuff still works before you run your new tests to ensure the new stuff is OK. If testing new features was simply a case of testing the new stuff, life would be easy - but that just ain't the case.

The scariest bit about testing, though, is the stuff that makes you go "What the f...?!!?!?". Only a few hours ago I was doing some performance testing of a database module, running tests with an increasing level of concurrency. So I ran a single user, then two concurrent (simulated) users, then five, then 10, then 15 ... the figures were looking very encouraging. "Oh, what the hell", I thought, and cranked it up to 50 users ... at which point the wheels fell off. SQL Server started screaming about deadlock and rolling back my transactions as it saw fit (which is what it's meant to do when it spots a deadlock situation), but as far as I could see the query it was consistently failing on was atomic and wasn't even touching the table that the error message said it was trying to get a lock on. After half an hour of head-scratching on the part of my scarily bright PhD-wielding colleague and me, we realised that the apparently atomic INSERT statement was actually causing a trigger to fire, which was executing a SELECT statement on a table that something already had a lock on, and that this latter lock was an exclusive one when it should have been shared.

If you're designing a power station and you have millions of pounds, you stand a chance of being able to test everything thoroughly, and to a high degree of confidence. The same cannot be said for the average IT project in the average company (or even in a large company, as one might imply from the fact that the recently released Office 2007 service pack is a 300MB+ download). Even if the schedules don't slip, and the scope doesn't creep, and no new bugs are introduced by fixing old ones, you've got a hard enough job - which means that as soon as anything goes wrong, testing grows in complexity. And chuck in the wacky stuff that only happens with a westerly wind and when you're using a beige keyboard, it's tempting to think: "Why am I bothering, I'll never find all the bonkers problems".

So spare a thought for those nice people who sit doing the apparently mundane job of clicking things, typing things and seeing what happens as a result. It's a whole lot more complex and stressful than it looks.