A novel defense mechanism against web crawler intrusion

<p>Web robots also known as crawlers or spiders are used by search engines, hackers and spammers to gather information about web pages. Timely detection and prevention of unwanted crawlers increases privacy and security of websites. In this research, a novel method to identify web crawlers is proposed to prevent unwanted crawler to access websites. The proposed method suggests a five-factor identification process to detect unwanted crawlers. This study provides the pretest and posttest results along with a systematic evaluation of web pages with the proposed identification technique versus web pages without the proposed identification process. An experiment was performed with repeated measures for two groups with each group containing ninety web pages. The outputs of the logistic regression analysis of treatment and control groups confirm the novel five-factor identification process as an effective mechanism to prevent unwanted web crawlers. This study concluded that the proposed five distinct identifier process is a very effective technique as demonstrated by a successful outcome.</p>

Name thesis:degree_name

Doctor of Philosophy (PhD)

Level thesis:degree_level

Open Access Dissertation

Discipline thesis:degree_discipline

College of Technology

Year dc:date.available

2013

Author dc:creator

• Aghamohammadi, Alireza

Contributors dc:contributor

• Dr. Ali Eydgahi, Ph.D., (Chair)
• Dr. Daniel Fields, Ph.D.
• Dr. Huei Lee, Ph.D.

• privacy
• computer crimes
• security systems
• world wide web
• hackers
• Digital Communications and Networking
• Information Security

Repository record dc:identifier

https://commons.emich.edu/theses/544

OAI identifier oai:identifier

oai:commons.emich.edu:theses-1917