Web Bot – Definition And Kinds Of Web Bots

A bot does what a users does with his/her browser but more!

Basically a web bot is a program that does things a user could do with his browser (Firefox, Safari, Internet Explorer, Opera, etc.) but much, much more.

Browsing the internet the regular way is fun but has certain limitations. Limitations like that someone has to actually browse and use the browser. Hmm what does that mean? Well sometimes you want to check things automatically – like if you have been out-bided by someone on ebay or if someone has written a comment on a blog-post you find interesting. All those things are very time consuming and the time adds up quickly just for those kind of repetitive tasks.

A web bot is the perfect solutions for this matter of doing repetitive tasks. The web bot can do the same things that you can do with your browser – just a lot faster.

Ok – but what exactly is a web bot?

A web bot basically is ……  just another computer program that is either running on a server or on your computer. The program mimics the browsers actions and does things that you would do online – just a lot faster and automatic – as written and mentioned above.

This is the reason why a web bot is also often called a bot program.

Considering the primary function of a web bot – a web bot can be classified in the following categories.

Web Bot Kinds

Spider

A spider such as the Google spider or yahoo slurp is a program that visits webpages – analyses its content and then saves, indexes the content by considering its relevancy in its’own database. If the spider finds link to other pages and websites – it will note them down to visit analyze them later on also. This functionality is very basic – and in reality a lot more complex. In it’s simplest form however – writing a spider is not hard to do and can be done pretty quickly. (check out the web bot code library to write your own spider – easy and quick)

Scraper/Crawler

A crawler and scraper which are synonyms for each other are very, very similar to spiders. They also crawl web pages and analyze the content. But instead of just sorting and indexing it – they often extract a specific part of the content or all the content for later use in either another program or for publishing it on another webpage. If these kinds of bots are legal is topic of another post.

PokerBot

Bots the play poker on the users behalf. Pokernetworks hate them and try everything they can to avoid them.

eMail Bot

Are bots that can read and sent emails for the user. The bot that is reading emails can be used to do certain actions once an email with a specific subject or content is being received.

Example:

• We could create a bot that check the online status of a webpage when we sent the bot an email message having the subject – “Check availability of domain.com”. The bot then checks if the website is online. If it is offline the user gets a message to his mobilepone.

On the other way around a bot could also be used to send email to a list of email newsletter subscribers or to his administrator. IMPORTANT a bot must not be used to send email spam!

FTP Bot

A bot that can use the FTP (FileTransferProtocol) to upload and download files from and to a server.

Example:

• This kind of bot could also be used to synchronize files between 2 servers.
• In combination with an email bot – a bot could receive an attachment and upload it to a server. This could function as a workaround for smartphones that cannot FTP files to a server – instead the send an email and the bot does the ftp-ing for them.

UseNet Bot

Has the capability of using the usenet as information resource.

Example:

• A bot that scans the usenet on company information/news
• A bot that scans the usenet for copyright infringements.

Things a bot must be able to do independent from it’s kind

Additional to the specific functions above – each web bot should be able to manage (store,read, delete) cookies and to emulate online forms (login-forms, etc.).

Well these are the basic functions of web bots. Each functionality will be covered in more details in another posts or tutorial.

What coding languages you may use to write web bots can be read about in the Web Bot Languages post.

Share:

2 Responses to “What is a web bot?”