A lot of really good research has been published about the Conficker worm, its many forms, infection vectors and speculation as to what it’s going to do next. But what seems to be missing is the operational side of fighting Conficker. What signs would you expect to see, how do you really fight it and what can you possibly do to prevent it?
Here’s a fictional case study that may be of help.
Day one: Why is my account getting locked out?
You come in to the office thinking it's just another day in the security trenches, but when you try to log on to the network you notice your account is locked out. OK, no big deal, although a little odd. You could call the help desk but you reset it yourself and then head for a cup of coffee when you realise a number of employees are wondering around mumbling about being locked out and complaining about a huge help desk queue.
OK, now something is smelling fishy. You start to check around and realise that accounts are being locked out all over the network. Time for your operational team to start doing some homework.
But where do you start? What if you don’t have central log monitoring or your intrusion detection system doesn’t see anything? One place to start: go to your directory service server logs and see which computer the source of the account lockouts is coming from. In Active Directory, you can use a free Microsoft tool called ‘Log Parser’ to search multiple logs at once.
Chances are that the logs will be overwhelmingly huge because of all the lockouts, but if you determine the main log entry in the Security log that is related to this issue (like event id 644, User Account Locked Out), use Log Parser to pull out all the relevant events and look at how many the source computers are generating. If there are three instances, this most likely is normal activity. If there are thousands, then this might be your culprit.
Maybe not though! Some computers may have an errant process running that generates that type of activity. The lesson here: make sure you use gentle language when describing the issue because accusing people of having infected computers without gathering facts can cause trust issues amongst staff.
If you can identify a starting point for the virus, make sure you grab log files and put them somewhere safe (application, security and system - not only for the infected computers, but for the directory service servers as well). At some point when things calm down, you’ll want these for forensic analysis.
Day 2 was spent dealing with more of the same. You’ve combed log files. You have identified that systems were in fact infected by the Conficker virus. And you have been cleaning and chasing it all over your network, even on systems that were patched and had up to date anti-virus software. You are frustrated and exhausted as are the rest of the staff that are trying to clean this stuff up!
Day three: OK, now we have a process to clean it
Before you tear all of your hair out, you do some more research and realise that running your antivirus software alone may not clean the virus.
Conficker has five known variants and can utilise the various versions to help itself spread fast and via different vectors. Looking at the research done by the Conficker Work Group, you find that the major antivirus vendors have published a variety of repair tools that may clean the virus better than anti-virus alone. Furthermore, you start reading the fine print in the Microsoft Knowledge Base article on Conficker and you see that there is a Group Policy Object (GPO) that can be used to help stop the propagation of the worm.
Now what? First up: get the GPO in place as soon as possible and make sure you enforce it to all Organisational Units. This is not the right time to block inheritance of a policy! Next, you need to test how best to clean an infected machine. Typically, running a couple of tools (one being a full scan of the anti-virus) to find and clean the infection, then rebooting, then running the anti-virus again (full scan), along with having the GPO in place, will help clean and stop the spreading. Start with systems that are showing the account lockout traffic in your logs to focus efforts on known infected machines.
Day five: Oh man. This is the worst day ever!
You see that the number of infected machines is much smaller, but the account lockouts are more frequent. Be patient! Keep scanning and cleaning. It will get better, but it takes persistence.
Day eight: Things are looking better
Finally, things start to calm down and you only see a few computers infected every day. It starts to sink in that this may never fully go away, so you start looking ahead and how to keep the monitoring process ongoing.
Day ten: Keep scanning!
You implement procedures for your operational staff to scan the logs and look for infected machines every couple of hours. Staff should be dispatched to clean machines as soon as they are discovered.
Today: Hold your breath!
But not for too long! See instead if you can focus on doing some forensics and post-mortem analysis to see what you can do better next time. Some things to consider:
• How was the incident handled? Was it clear that an incident manager was designated to be in charge?
• Do you have an incident handling policy? Was it followed? Did it work?
• How was communication to: staff involved in the incident, other IT groups, management, and end users?
• How easy was it to contain computers with the virus? Is your network architecture able to shut down parts of the network easily to isolate virus-infected segments?
Of course, an ounce of prevention is worth a pound of cure. Here are some thoughts about how to prevent this from running rampant in your organization.
First and foremost, make sure you are patched. It only takes one computer in to get infected before Conficker will let loose everywhere using its other infection vectors. Secondly, try to have a spyware tool in use with your anti-virus tools. The results can be mixed (or non-existent) if you don’t have both in play.
And as with anything, have your incident response procedures up to date and vetted. If you don’t have these processes in place, then you may get too many incident managers running the show and soon become counter-productive. Communication can get muddled, and worse, you may not have enough communication.
Westphal is a 16 year information technology professional with specific experience in the area of information security. She is currently employed as the CISO of the Arizona Department of Economic Security. Skilled in troubleshooting and process analysis, specific expertise in security areas includes forensics, operating system and network security, intrusion detection, incident handling, vulnerability analysis and policy development. Wetphal has been a CISSP since 2001 and a CISA since 2008. You can reach her at [email protected]