#141492 - 2005-06-12 10:28 AM
OT: Spyware removal tools not up to the job
|
NTDOC
Administrator
Registered: 2000-07-28
Posts: 11625
Loc: CA
|
Prompted by a discussion I had with Al_Po and having been wanting to do this sort of test for a long time now, I finally got around to doing it.
In order to test the power of some of the current leaders in the field of Spyware removal I ran the following test.
Using VMware Workstation 5 I created a Windows XP Pro installation.
I gave the system 2.2GB of diskspace and 800MB of RAM to use.
1. No Service Packs installed
2. No Antivirus installed
3. No Microsoft hotfixes installed
The reason I chose the above setup is that in my opinion I think that setup more closely follows how many home users are setup.
Granted some may be on Windows XP Home Edition, but for this testing I don't feel the Pro version adds any benefits to help prevent spyware.
I then proceeded to start surfing the Web and I allowed popups to run, I allowed software to install by "accidentally" clicking on links. In the real World it might take a home user a few weeks to get the system as infected as I had it, but eventually by fairly normal surfing all Windows users will get spyware if they don't have methods or software to prevent it.
Within a few hours the system was taking about 5 minutes to even finish logging in and was using 100% of the CPU (when the system was first installed and doing nothing the CPU usage was about 0 to 1% ) RAM was consuming 600+MB and paging to the swap file.
On day 2 of this test the system blue screened twice and I had to manually intervene by setting up a script to loop through some of the programs and keep killing them while I cleaned it up some. I got the system stable again by removing some spyware manually (couldn't be avoided). Then started the system back up normally and started surfing again.
It got to the point that many of the spyware programs were fighting each other to take control of IE. It got to the point that .html files were no longer associated to run with IE and would prompt you what to do with links. Again I had to manually intervene to get IE working again. Okay, started browsing again, but not very well. Finally a program named clicksearchclick.com became the dominant application that took over all browser requests. I'm not sure what the real intent of this site or method is. Even if you click on the bogus links that it gives you when you search, it runs another search and never goes to the intended real link. If you said, okay I'll buy some of "what ever" and click on it, you still just keep getting more useless search results. Don't understand the ROI for clicksearchclick.com with this method.
Then a program named Desktop Search got onto the system and it was GAME OVER... That little baby would not go away no matter what. I continued attempting to get more spyware onto the system but it was evident that the system was so bogged down now that it was useless trying to get more spyware to install.
Okay, time to start using some of the Top rated spyware removers and see where we get. Let me say right off that I assumed that none of them alone would be able to remove all of the spyware. I was soon to be amazed and find that not only could none of them remove all of it alone, but all of the programs used together were not able to remove all of it.
NOTE! I had to map a drive to the root of the test system and copy over the removal tools and then remove the map. Internet explorer was no longer capable of downloading any removal tools on it's own.
This is something to be quite aware of for some users. You might need to talk someone through getting the appropriate removal tools via a command-line connection of FTP.
Originally I had wanted to let each program try and remove stuff then replace the VMware image with a backup of the spware infected image, but since finding that using all the spyware apps without success there was no reason to proceed further in that direction.
Here are the results from each of the tested programs.
- PROGRAM: Spybot Search & Destroy
Version: 14
Defs: June, 10, 2005
Database: 24,627 potential threats covered
First pass and reboot request: 24 minutes 48 seconds
304 Items fixed
26 could not be fixed
Allowed Spybot to autorun after reboot
Second pass and reboot request:
57 Items fixed
3 could not be fixed
Third (auto launch did not auto launch)
Third pass and reboot request (manually kicked off 3rd scan)
57 Items fixed
4 could not be fixed
Fourth pass and reboot request:
55 Items fixed
3 could not be fixed
At this point it would appear senseless to continue allowing Spybot to
autolaunch and scan after each reboot. Spybot does not appear to be able to
automatically remove the following items without further assistance from the user.
- PROGRAM: Ad-AwareSE
Version: 1.06r1
Defs: May, 31, 2005
First pass and reboot request:
Found: 499 problems in 8 minutes 30 seconds
Second pass and reboot request:
113 Objects found
Third pass and reboot request:
165 Objects found
Fourth pass and reboot request:
144 Objects found
At this point it would appear senseless to continue allowing Ad-AwareSE to
autolaunch and scan after each reboot. Ad-AwareSE does not appear to be able to
automatically remove the following items without further assistance from the user.
- PROGRAM: Microsoft Antispyware
Version: beta 1
Defs: #5725
First pass and reboot request:
Found: 35 problems in 1 minutes 42 seconds
Second pass and reboot request:
12 Items found
Third pass and reboot request:
10 Items found
At this point it would appear senseless to continue allowing Microsoft Antispyware to
autolaunch and scan after each reboot. Microsoft Antispyware does not appear to be able to
automatically remove the following items without further assistance from the user.
- PROGRAM: Webroot Spy Sweeper
Version: 4.0.3 (Build 363)
Defs: 494
Database: 93,577 potential threats covered
First pass and reboot request:
Found: 813 problems in 2 minutes 53 seconds
Second pass and reboot request:
Found: 14 problems
Third pass and reboot request:
Found: 14 problems
Fourth pass and reboot request:
Found: 163 problems
At this point it would appear senseless to continue allowing Webroot Spy Sweeper to
autolaunch and scan after each reboot. Webroot Spy Sweeper does not appear to be able to
automatically remove the following items without further assistance from the user.
- PROGRAM: EWIDO
Version: 2.1
Defs: #1242 June 11, 2005
Database: 152,781 potential threats covered
First pass and reboot request:
Found: 813 problems in 2 minutes 53 seconds
Second pass and reboot request:
Found: 202 problems
Third pass and reboot request:
Found: 66 problems
At this point it would appear senseless to continue allowing EWIDO to
autolaunch and scan after each reboot. EWIDO does not appear to be able to
automatically remove the following items without further assistance from the user.
Well bottom line is that a normal computer user would NOT be able to remove all the spyware from their system. The average home user has trouble running and using many basic applications, let alone using the advanced features of many applications.
Suggestions for all of the makers of these Spyware Removal applications:
On a heavily infected system all applications run very slowly and the CPU is often at 100% utilization due to all the auto-launched applications consuming the CPU. Having multiple "I agree", click next buttons takes way too long to get through and is quite annoying.
1. Remove or minimize the amount of screens required to get the program installed and running.
2. Have 2 install methods. a.) I'm a new computer user. Make all choices for me. Install the application and remove the spyware without asking me ANY other questions. (many of the questions often asked are not known by inexperienced users). b.) I'm an experienced user please give me all the advanced features and don't run anything automatically, I'll take care of it.
3. Microsoft's spyware removal tool is one of the worst tested about asking the user what to do, and if you don't read the dialog box carefully you can easily allow the spyware to be blocked in the ON position.
4. Some of the spyware removal tools did not give the user the option to reboot immediately which in some cases allowed some spyware to spawn and install more stuff.
5. Overall all the Spyware Removal tools need a much more "beginner" method of taking educated guesses for the inexperienced users and attempting to remove the spyware without getting into an annoying loop themselves.
Manual Cleanup Required
After giving all of the tools a chance to clean up the system I had to manually intervene and clean it up.
01. Reboot into safe mode.
02. Using personal knowledge gained by many years of working on Windows systems. Sorted and verified version fields of DLL files and removed those that either had no information or I knew were not valid files. Note! not all EXE and DLL files that don't have a resouce field are bogus but many are.
03. Scan the registry for all startup locations and removed bogus junk.
04. Modified permissions on all the main startup locations in the REGISTRY so that not even the Administrator can add or edit entries.
05. Scan and cleaned up bogus POLICY entries in the registry that were used to prevent the user from running or modifying many things inside Windows.
06. Went into each of the user profiles and deleted all the contents of the TEMP, IE TEMP, IE HISTORY folders.
07. Had to go get and use XNET to remove the Desktop Search which had installed itself as a service and was using SVCHOST (a valid network application) to run it'self.
08. Desktop Search had also created a SERVICES32 key under HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Services32 I didn't know this was a valid entry in Windows but the system seemed to use it as a method to keep it installed and running.
09. Removed many .ini files in the %windir%\system32 folder which were designed to rename seemingly non threatening files to EXE and DLL files and randomize spawning of new applications on the fly.
10. Deleted all IE cookies as well (not that critical but helps to prevent the spyware removal tools from flagging them)
11. Disabled the Windows Restore feature and then verified the files were deleted from there which had copies of many of the bogus files.
12. Not sure if it was a result of spyware or not, but ran into a couple of folders that would not allow me to take ownership on and change permissions, so I ran a full CHKDSK C: /F and then I was able to take back ownership and change permissions so I could remove the bogus folders.
Restared the system and ran all the Spyware Removal tools again. The first one Ad-AwareSE found 8 entries in a recycle bin for another user account. I had it remove them. Then ran each one and all 5 of them then came up with a clean system.
So the results of my testing though not througly documented shows that none of the top rated Spyware Removal applications are capable of cleaning up all systems. This shows that an experienced Windows Analyst is still often needed to physically review and cleanup some systems and so far can not be left to an automated piece of software.
I'm sure that it is possible for some of them to clean up less infected systems, but I also know that there are systems out there that are even more infected with different things then mine had that they would not be able to remove either. I've had to manually clean up many systems that were brought in to me because the user was unable to clean it up with the tools either on their own.
I will say thought that over-all the programs tested do perform an amazing job of attempting to cleanup up something that is quite difficult to document due to ever changing and self spawning randomly generated new applications. However it also seems that the culprit programmers behind some of these scum spyware applications are using more advanced methods then they used to use in order to bypass these cleanup tools.
The use of the free tools from http://www.systeminternals.com was quite useful in tracking down some of the methods used by some of these spyware apps.
|
|
Top
|
|
|
|
|
#141493 - 2005-06-12 02:54 PM
Re: OT: Spyware removal tools not up to the job
|
mole
Getting the hang of it
Registered: 2003-01-01
Posts: 80
Loc: Indian Head, Maryland, USA
|
Doc,
This is interesting and many thanks for the research. I too have had similar experiences, though not in a "controlled" situation such as your VM, it was as always pulling these things out from a system at work a user had got screwed up or a "harry homeownser" I was doing a good deed for.
Since you limit the topic to spyware tools like Adaware and Spybot and didn't cover quasi-spyware related tools and/or virus removal tools such as S-t-i-n-g-e-r (ugh horrible choice to rename to) or MS's MRT, I'll only throw in my two cents with respect to two of the tools you covered that I use most, Adaware and Spybot.
1) If you know the system is really messed up its best to start from safe mode. Then go into task manager and try to kill any suspect looking thing if you can before continuing. Then also try to stop any unnesessary services.
2) I have found it is possible to run both of these spyware removal apps from a network share or just plain copied into a directory to make a stab at getting a system under control again. Agreeably this defeats some of the utilities in each, but as you pointed out stepping through the install prompts and setup is tedious on a system clocking near 100% of its CPUs to crapware.
3) As I think you and others have posted here and the tools have on their respective websites there are also silent command line switches that can do a scan or auto remove without stepping through the dialogs.
4) As you found, it becomes necessary to resport to manual pulling of some stubborn spyware. I especially enjoy typing a for-in-do loop in a spyware directory and calling regsvr32 to unregister the offending *.dll's.
Bottom line I guess is as you point out, none of the current removal tools are up to the job on theri own and especially when used "out-of-the-box" in the hands of "Harry homeowner". It does take experience for a really messed up system to come clean.
I recall one system belonging to a friend of my wife's that I called the "Petri Dish", it had everything growing in it, even some classics. It took 3 hours to clean, update and get AV, firewall and everything working again. Would have been easier to format and reinstall, but I wanted to see for myself if it could be done.
mole
_________________________
mole
Who is John Galt?
|
|
Top
|
|
|
|
|
#141494 - 2005-06-12 05:46 PM
Re: OT: Spyware removal tools not up to the job
|
masken
MM club member
Registered: 2000-11-27
Posts: 1222
Loc: Gothenburg, Sweden
|
Read an article describing that on an average, 25% of the spyware isn't handled with the tools avaiable today.
Just recompiling old viruses will fool most antiviruses... *brr*
The biggest step towards security (besides having a patched system - btw; WSUS went sharp a few days ago), is to run all Internet-facing applications (browser, Office, IM-apps etc) with lower privilegues, EVEN if you "have to" be logged on as local admin. There are several good ways of achieving this, either through local policies, or with third party tools. Have a look here if you haven't already read these articles (especially Part 2):
Browsing the Web and Reading E-mail Safely as an Administrator
Browsing the Web and Reading E-mail Safely as an Administrator, Part 2
Third party tool (freeware):
RunAsAdmin
..even though these tools are somewhat a "reversed security" approach, they do solve security problems for alot of admins (the one's who should be as well as the one's who shouldn't). In the next release of Windows, Users are finally gonna be Users by default, and not Admins.
Edited by masken (2005-06-12 05:50 PM)
_________________________
The tart is out there
|
|
Top
|
|
|
|
|
#141495 - 2005-06-12 11:03 PM
Re: OT: Spyware removal tools not up to the job
|
NTDOC
Administrator
Registered: 2000-07-28
Posts: 11625
Loc: CA
|
Thanks Mole and Masken for the replies.
The intent of my post was not to show alternative methods of removal or as Mole asked - ( nothing to do with flat out viruses and worms )
Hopefully all the Admins of this board are quite capable of cleaning up a system one way or another. I was wanting to point out that the tools are not up to the task for a "HOME USER" which most are targeted at and only recently have some started targeting at the IT Professional.
Giving a home user the tip to stop "suspicious" looking processes with Taskmanager is semi futile. Most of the more advanced spyware will just spawn a new process and most home users and many Admins don't know what looks "suspicious" in the first place. I've worked with many other Admins and most are great guys, but knowing what processes should or should not be running was not a strong point for probably more then half of them.
The average home user will typically only run or click what you put right in their face. No searching for options etc...
That said, since some are now starting to target the IT Professional - they're not up to the task either. As briefly mentioned I also at the end used the advanced options and methods to remove the problematic spyware and none of the applications were able to remove them.
Mole, yes I agree that going into SAFE MODE right off the bat is probably the best approach to start cleaning up, however NONE of the products tell you that or offer to walk the user through that type of cleanup. All of them attempt to keep cleaning while using the users rights of a normal logon.
There are some great sites out there with good information for removing stubborn spyware.
Al was somewhat skeptical that I would be able to remove NAIL.EXE from the system so it took me a while to find it, but finally found a site and downloaded it and installed it.
It was not too hard to remove Al 
Some of the spyware loaded as system drivers which makes it difficult to remove even in SAFE MODE unless you pay attention.
Hopefully maybe some of the makers of these programs will continue to make their programs better for the average user.
If you do want more information on Spyware this site seems to have some of the best information on the subject that I could find.
Spyware Warrior - Waging the war against spyware
They have a very active forum on the subject and a lot of good links and other information on the subject. This would be good for an "above average" home user, but the typical home user wouldn't know or even want to spend the time reading this, if they even could. A typical heavily infected system often can't get to a specific site anyways and is frustrated just trying to run the other applications while these darn popups keep happening without them even launching IE on their own.
As a note often brought up by Microsoft and others which is true. If the user has the latest service pack and all critical updates applied and is running a current and updated antivirus solution then the spyware infestation possibilities are greatly reduced. Running as a limited user really helps even more.
Maybe the http://spywarewarrior.com/ people will read this posting as well as some of the makers of these Spyware Removal tools.
|
|
Top
|
|
|
|
|
#141496 - 2005-06-13 03:38 AM
Re: OT: Spyware removal tools not up to the job
|
Allen
KiX Supporter
Registered: 2003-04-19
Posts: 4562
Loc: USA
|
I have to agree with you Doc, the tools that are available are no where near thorough and home/nonpower users will never get rid of everything with just a tool. That being said, I usually can get rid of "most" things with a combination of, Ewido, Counterspy, and HijackThis, and then make sure all hotfixes and service packs are installed.
Nail.exe is an interesting one. It has different versions which do different things. I think you may have mis-understood what I meant about that one . I've actually been able to get rid of it with a lot of tinkering, but I've not been able to figure out what program installs it.
The spyware that I had the nightmare with was one that installed itself as a system "boot" driver, which made it next to impossible stop, even in safe mode. Starting up in the command console was about the only way to disable it, but upon reboot, some other process was watching it and immediately recreated the file and added the process. Before I wasted any more time on that system, I wiped the box, so that one was one I never did figure out. I'd actually like to find it again to take the fight on again  .
|
|
Top
|
|
|
|
|
#141498 - 2005-07-23 04:30 AM
Re: OT: Spyware removal tools not up to the job
|
NTDOC
Administrator
Registered: 2000-07-28
Posts: 11625
Loc: CA
|
As an FYI - Microsoft updated their Antispyware beta build on July 18.
Quote:
Overview
Windows AntiSpyware (Beta) is a security technology that helps protect Windows users from spyware and other potentially unwanted software. Known spyware on your PC can be detected and removed. This helps reduce negative effects caused by spyware including slow PC performance, annoying pop-up ads, unwanted changes to Internet settings, and unauthorized use of your private information. Continuous protection improves Internet browsing safety by guarding over fifty (50) ways spyware can enter your PC.
Participants in the worldwide SpyNet™ community play a key role in determining which suspicious programs are classified as spyware. Microsoft researchers quickly develop methods to counteract these threats, and updates are automatically downloaded to your PC so you stay up to date.
Beta 1 Versions
Since releasing Windows AntiSpyware (Beta) on January 6, 2005 (Build 1.0.501), we have continued to receive feedback from customers. We introduced a beta refresh on February 16, 2005 (Build 1.0.509) which enhanced some of the real-time protection agents, added new threat categories, and improved stability and performance.
An updated beta refresh released on June 23, 2005 (Build 1.0.613), introduced enhancements to the detection and removal capabilities, including improved Winsock LSP removal capabilities and support for long descriptions of categorized software. In addition, we have also extended the Windows AntiSpyware beta expiration date to December 31, 2005.
The latest beta refresh, build 1.0.615, addresses issues pertaining to how Windows AntiSpyware (Beta) provides information to the user about processes running on a PC, solves an issue regarding the delivery of new anti-spyware signatures for some customers and provides new signature updates to help protect against recently identified spyware.
Existing users of the beta (Builds 1.0.501, 1.0.509, 1.0.613, and 1.0.614) will receive a software update that includes the new beta refresh. The new beta refresh is also available for download through this site.
Microsoft would like to encourage all Windows AntiSpyware (beta) users to download and install the new update (Build 1.0.615).
The user must be an administrator to install this application. To check the version number, click About Microsoft AntiSpyware… on the Help menu.
Important Notes
English only:
The current beta is intended only for English language versions of the Microsoft Windows operating system. It should not be deployed on non-English versions.
Beta Support Policy:
This is pre-release (beta) software distributed for feedback and testing purposes. Microsoft does not provide technical support for beta releases (see below for information about how to gain access to newsgroups). If Windows AntiSpyware (Beta) is causing an issue with your system, we recommend removing it by using Add or Remove Programs and even using System Restore if the problem persists.
Access to Newsgroups:
Although formal support is not offered for this beta, we have provided newsgroups to help get your questions answered.
Microsoft® Windows AntiSpyware (Beta) (Build 1.0.615)
|
|
Top
|
|
|
|
#141499 - 2005-07-23 04:43 AM
Re: OT: Spyware removal tools not up to the job
|
Kdyer
KiX Supporter
Registered: 2001-01-03
Posts: 6241
Loc: Tigard, OR
|
Doc,
This was talked about on http://msfn.org as well.
Kent
|
|
Top
|
|
|
|
|
#141500 - 2005-08-17 07:37 PM
Re: OT: Spyware removal tools not up to the job
|
NTDOC
Administrator
Registered: 2000-07-28
Posts: 11625
Loc: CA
|
Wow, hopefully this site is only experiencing higher than average traffic and is not down long term. This was one of the best resources on the Net that I sent home users to to help them clean up their systems.
http://spywarewarrior.com/
|
|
Top
|
|
|
|
Moderator: Arend_, Allen, Jochen, Radimus, Glenn Barnas, ShaneEP, Ruud van Velsen, Mart
|
0 registered
and 1574 anonymous users online.
|
|
|
