Use Mod_Security to Block the IP Address of Scrapers and Script Kiddies

How closely do you watch your logs, and how much do you care about scrapers or script-kiddies constantly trying to poke their way through holes in your website? If you’re as anal-obsessive about it as I am, probably a lot – and if you do watch your logs, you’ve probably seen those scrapers or script kiddies pounding away at URL’s that don’t exist on your site. For example, if your website is built on Drupal, you may see repeated attempts to access /wp-login.php which is the default login URL for WordPress. There is no earthly reason for anyone to be accessing that URL, on a non-Wordpress website and obviously anyone hitting that URL has bad intentions, so why not block the attempt? And, if they keep trying, why not block their IP address at the firewall? BLOCK ‘EM, I always say!

Secured with Mod SecurityTo know what invalid URL’s are being hit at your website you need to watch your logs. If you run a Drupal website, you can see most invalid URL attempts in your dBlog (/admin/reports/dblog), but to get the full picture you should probably be looking at your Apache error_log and /log/messages logs.

You can download these logs to your computer and view them offline but that’s cumbersome to do every 10 minutes.  A better way might be to SSH into your server and do a simple TAIL, to watch the log-hits live, as they happen. For example:

 tail -f /var/log/secure -f /var/log/messages -f /usr/local/apache/logs/error_log

If you start using custom Mod Security rules  and CSF to block things (as I am about to explain), it’s even more important to watch your logs closely because it’s very easy to create a custom rule that inadvertently blocks innocent users.  To help avoid that I usually keep an SSH Terminal window open on my screen all day, so i can watch all of my fancy custom Mod_Security rule blocking as well as other Apache errors “live” as they occur.

Blocking access to specific URLs with Mod Security

You will need Mod_Security installed and running on your webserver, which you can install with Easy Apache, or manually from downloading it from modsecurity.org.  If your server is on a shared host, it may already be installed, or, it may not be possible to use, so check with your hosting company.

After you have Mod_Security installed and dialed-in (which could take some time), you can begin creating your own custom rules.

To get started, create a file for your custom rules. This is just a simple text file where you will keep your custom rules. I named mine “custom_Mod_Security_rules”. You will have to upload this file to your default Mod_Sec rules directory (or wherever you specify).

You will have to add your new rules file to your Mod Security configuration file by adding the following line:

 Include /mod_security/custom_Mod_Security_rules

Be sure to use the path to where you uploaded your custom rule file and the filename you are using.

At this point you can restart Apache to make sure nothing is broken so far. You haven’t actually created a new Mod_Sec rule yet, but it’s good to test up to this point.

Assuming your server doesn’t fall over, you are now ready to start adding your custom rules to the “custom_mod_security_rules” file that you have created. For this example we will create a rule that blocks all attempts to log-in to WordPress. If you have a Drupal website, nobody has any business trying to brute-force WordPress logins so let’s block them! If you also have a WordPress install on your server I will show you below how to disable the rule for the WordPress blog, so that it only runs against your non-Wordpress sites. If you primarily run a WordPress site, you can change “wp-login.php” to whatever path or URL that you want to block.

To block the path “wp-login.php”, simply add these three lines to your custom Mod_Security rules file:

## block wordpress login attempts
 SecRule REQUEST_URI "wp-login.php" \
 "id:'9990227',severity:'3',msg:'Bad url - wp-login.php'"

The first line is a comment describing what the rule does
The second line tells Mod_Security to look at the requested URI (page URL) and match on “wp-login.php”
The third line assigns an ID to the rule, in this case 9990227. You can use any ID number that you want but they should be unique for each rule, and be sure that they don’t duplicate any of the default rules that come with Mod Security.  The rest of line 3 sets the ‘severity’, and sets the message that will appear in the logs, in this case “Bad url – wp-login.php” – you can enter anything here but be sure to wrap the text of the message with single quotes (apostrophe) and make sure it ends with the double-quote that corresponds the the double quote at the beginning of line 3.

Save or upload your updated custom_Mod_Security_rules file to your sever and restart Apache. If Apache does not restart, remove the 3 lines and restart again, or comment-out the “Include /mod_security/custom_Mod_Security_rules” that you added to your Mod_Security configuration file and figure out what went wrong (probably a missing ” or other syntax error). Assuming Apache restarts with no issues, all attempts to hit /wp-login.php should now be blocked and re-directed to an error page before it even hits the PHP preprocessor. Yay!

This is all fine and dandy, but what if you have a WordPress site on your server? You will need access to wp-login.php if you ever want to log-in, so you will have to disable this custom rule for WordPress. To disable the rule, just add the following lines to your Mod_Security configuration file and restart Apache:

<Directory /home/account_name/public_html/my_blog/> 
SecRuleRemoveById 9990227 
</Directory>

The first line is the path to your WordPress installation (you will have to use your actual path). The second line tells Mod Security to ignore your custom rule id “9990227”, and the third line ends the directive.

If all went well, you should now be blocking any attempts at /wp-login.php except in your WordPress installation. But at this point there is nothing to stop the evildoer from continuing to try all day long, or trying to probe other potential vulnerabilities on your server – for that, you will need to block their IP address at the firewall.

Use Mod_Security and CSF to block the IP address

Now that you have Mod_Security configured with your custom rule, you can use CSF to block the offending IP address at the firewall after a configurable number of attempts to hit that URL from the same IP.

First, you will need CSF (Configserver Security & Firewall) which is an awesome (and FREE!) firewall and IDS (Intrusion Detection System) for Linux that you can download here: http://configserver.com/cp/csf.html
After you have CSF installed and running, find the following setting in your CSF configuration file:

LF_MODSEC =

The LF_MODSEC setting value is the number of Mod_Security ‘hits’ or blocks required before adding the offending IP to be blocked by the firewall. Setting LF_MODSEC to 5 would block the IP address after 5 MOD_SECURITY blocks within the time period set in LF_INTERVAL, which is set at one-hour (3600 seconds) by default.

Enter your value for LF_MODSEC, let’s say, 5, to begin, restart CSF, and watch your logs. If all went well, any time your custom rule is triggered by the same IP, more than the times you set in LF_MODSEC, that IP will be blocked at the firewall, forcing the bad-guy to go play somewhere else!

Now start looking for other invalid URL’s to create and, don’t stop at URL’s. You can create custom Mod_Security rules to block by browser agent-id’s, referrers, and several other things. Look through the default Mod_Security rules to get an idea of what you can do with it – or, feel free to post a comment for tips on what types of things I block with Mod_Security.

One final tip:

It is very easy to get yourself blocked by CSF while testing your Mod_Security rules. Be sure to whitelist your IP in CSF by added it to the CSF.ALLOW file. You can also whitelist yourself from Mod Security, but doing so makes it impossible to test your custom rules.

7 thoughts on “Use Mod_Security to Block the IP Address of Scrapers and Script Kiddies

  1. Hi Randy pretty nice article you have here. I started using it on my server but the office has grown since then so is there a way to disable a rule by IP?

    I have tried using:

    SecRule REMOTE_ADDR “^X.X.X.X$” “nolog,allow,id:999022712”
    SecAction “phase:2,pass,nolog,id:9990227123”

    But then apache wont restart.

    Any help appreciated,
    Andy

  2. Hi Rand

    I have tried below to whitelist ip but it still gives warning msg,currently modsecurity is detectiononly mode

    SecRule REMOTE_ADDR “^^X.X.X.X$” phase:1,t:none,nolog,allow,id:999999995,ctl:ruleEngine=off,ctl:auditEngine=Off

    1. it sounds like you have the SecRuleEngine DetectionOnly command set which is good for testing, but it does not actually do any blocking – it sounds like that is what the warning is telling you..
      You also have ruleEngine=off in that rule which also effectively disables that rule, if I remember correctly.

      1. yes ,currently it is in non-blocking but it can be blocked when I active blocking mode.

        Also can you please help me how to add multiple IPs &

        Below is what I am trying to pass through mod security,Actually I wanted to pass all text area posting through application forms

        Below is for passing all field name contain word “Text”,but not working.
        SecRule ARGS:/Text/ “.*” phase:1,t:none,nolog,allow,id:999999801,ctl:ruleEngine=off,ctl:auditEngine=On

  3. I like to get their ip, run it through whois, then block their entire network (so they don’t get a retry via DHCP)

    <>
    SecRuleEngine On
    #replace 1.2.3.4 with your own ip address
    SetEnvIfNoCase Remote_Addr ^1.2.3.4$ MODSEC_ENABLE=Off
    SecRule REQUEST_URI “@pmFromFile /path/to/keyword.txt” “id:1001,log,exec:/path/to/modsec.cmd”

    <>
    java WhoisBlock %REMOTE_ADDR% “cmd /c echo deny from %%CIDRADDR%% & netsh ipsec static add filter filterlist=BlockNet srcaddr=%%NETADDR%% srcmask=%%CIDRBITS%% dstaddr=Me” 1>>apachelogsmodsec.log 2>&1

    <>

    // runs an offending ip through whois server and returns cidr mask hi lo, etc depending on what your OS or router wants

Leave a Reply

Your email address will not be published.