An internet bot or web robot is a software application or script that performs repetitive tasks such as crawling or fetching data from a website to be analyzed.
More than 50% of internet web traffic is generated by bots.
Different types of bots might include:
Search bots – Systematically crawl the internet to index websites for search engines.
Social bots – Can be used to communicate autonomously on social media platforms. It is estimated that 9-15% of Twitter accounts are social bots.
Commercial bots – This automated bot scours websites looking for bargains.
Chat bots – can be used by to conduct an online conversation via text in lieu of communicating with a live agent.
Spam bots – these are applications meant to collect and spread data. They can register addresses in your analytics in hopes that site owners visit the websites or scour your website to collect email addresses.
Bots can wreak havoc on your website and web hosting.
- Create false data in your web analytics such as deceptive traffic and high bounce rates
- Cause a spike in memory usage and tie up resources
- Affect your website loading time
- Scrape data from your website to gain the competitive edge
- Cause site owners to upgrade hosting packages when unnecessary
How to block bad bots in WordPress
1. Install WordFence
WordFence is a free plugin that can be downloaded through your WordPress plugin library (Plugins->Add New).
In addition, this WordPress security plugin provides a firewall and malware scanner that keeps your website safe from malicious attacks.
WordFence already has a database of malicious bots and IP’s. However not all spam blockers will ever be 100%
This WordPress security plugin also includes a firewall which filters specific applications and constantly monitors for suspicious traffic.
2. Install WP Statistics
In your WordPress dashboard visit Statistics->Top Visitors Today to identify IP’s that have an unusual amount of hits to your website.
You can also cross check the IP on multiple websites to see if the IP is part of any known blacklists.
3. Block IP through WordFence or .htaccess
To block an IP through WordFence, you can select WordFence->All Options->Tool Options and enter the IP in the field “List of comma separated IP addresses to ignore”.
If you prefer to block the IP through your .htaccess file, you can add the following entry:
Deny from 100.100.100.100 (replace the IP with the one you want to block).
4. Rename your login page
Brute force attack bots work relentlessly to access your sites login page since the standard address is always your domain + /wp-admin.
These attacks can exhaust memory usage on your hosting package which can slow down your website or trigger your hosting to shut your website down.
Installing a plugin such as the plugin Rename wp-login.php which will allow you to rename your admin path to whatever you want.
IE yourdomain.com/{whateveryouwant}
5. Update your Robots.txt file
A robots.txt file lives in the root directory of your website and sets the rules for search engine crawlers.
Some search engine crawlers such as Google’s (googlebot) or Bings (bingbot) can sometimes be aggressive when crawling your website and overload your hosting resources.
Through your robots.txt file you can set the rules on which bots are allowed to crawl your site, their frequency or deny them completely.
The following directive below tells Bing to crawl your website slower or 1 page every 10 seconds.
The directive below blocks Baidu spiders entirely.
Learning how to manage your robots.txt is very helpful in regulating good bots such as a Google search crawler vs bad bots trying to access your website.
Final Words
As your website becomes more popular and receives more traffic, the more of a target your website will become.
Before that happens, it’s best practice to protect your website before it’s too late.
If you are unsure of how to setup the above, just hire a web development expert and have some peace of mind.
