Tag: command

Writing OSSEC Custom Rules, Decoders and Alerts

OSSEC (http://www.ossec.net) is an open-source host-based intrusion detection system (HIDS). OSSEC can be used to monitor your local files and logs to check for intrusions, alert you of rootkit installation and do file integrity checking. OSSEC is a wonderful tool because it is highly customizable. By default, OSSEC monitors many of the programs commonly installed on a machine, but its real power comes from the ability of system administrators to customize OSSEC. By writing custom rules and decoders, you can allow OSSEC to parse through non-standard log files and generate alerts based on custom criteria. This allows OSSEC to monitor custom applications and provide intrusion detection services that might otherwise not be available, or would have to be developed on a per-application basis.

OSSEC rules are based on log file parsing. The log files that OSSEC monitors are specified in the /var/log/ossec/etc/ossec.conf file in the following format:

  <!-- Files to monitor (localfiles) -->

  <localfile>
    <log_format>syslog</log_format>
    <location>/var/log/messages</location>
  </localfile>

  <localfile>
    <log_format>syslog</log_format>
    <location>/var/log/secure</location>
  </localfile>

  <localfile>
    <log_format>syslog</log_format>
    <location>/var/log/maillog</location>
  </localfile>

  <localfile>
    <log_format>apache</log_format>
    <location>/var/log/httpd/error_log</location>
  </localfile>

  <localfile>
    <log_format>apache</log_format>
    <location>/var/log/httpd/access_log</location>
  </localfile>

Each file that is monitored depends on a “decoder” which is a regular expression used to parse up the pieces of the log file to extract fields such as the source IP, the time, and the actual log message. The decoders are stored in /var/ossec/etc/decoder.xml. The following is an extract of the SSH decoder portion of the decoder.xml logfile:

<decoder name="sshd">
  <program_name>^sshd</program_name>
</decoder>

<decoder name="sshd-success">
  <parent>sshd</parent>
  <prematch>^Accepted</prematch>
  <regex offset="after_prematch">^ \S+ for (\S+) from (\S+) port </regex>
  <order>user, srcip</order>
  <fts>name, user, location</fts>
</decoder>

<decoder name="ssh-denied">
  <parent>sshd</parent>
  <prematch>^User \S+ from </prematch>
  <regex offset="after_parent">^User (\S+) from (\S+) </regex>
  <order>user, srcip</order>
</decoder>

You can see that the decoder.xml file is used to parse through the log file using regular expression pattern matching. This means that you can add additional files to the list of those which OSSEC is checking if you would like. You’ll also note that the XML rules in decoder.xml are nested so that you can use the <parent> tag to nest rules. A rule with a “parent” will only attempt matching if the parent rule matched successfully. Using the order and its statements you can populate OSSEC’s predefined variables with portions of the log file. The following variables are supported:

  • location
  • hostname
  • log_tag
  • srcip
  • dstip
  • srcport
  • dstport
  • protocol
  • action
  • user
  • dstuser
  • id
  • command
  • url
  • data

Supposing you have a log file produced by an application that isn’t covered by the default decoders you could write your own decoder and parsing rules. Unfortunately OSSEC only supports logs in the formats syslog, snort-full, snort-fast, squid, iis, eventlog, mysql_log, postgresql_log, nmapg or apache. Therefore any custom logging you write must conform to one of these formats. Syslog is probably the easiest to use as it is designed to handle any one line log entry.

Let us suppose we have a custom PHP based application that resides in /var/www/html/custom. Our application will write Apache format logs to a file called ‘alert.log’ in the ‘logs/’ application subdirectory. This program has the following lines in example.php:

<?php

$id = $_GET['id'];
$logfile = 'logs/alert.log';
if (! is_numeric($_GET['id'])) {
	$timestamp = date("Y-m-d H:m:s ");
	$ip = $_SERVER['REMOTE_ADDR'];
	$log = fopen($logfile, 'a');
	$message = $timestamp . $ip . ' PHP app Attempt at non-numeric input (possible attack) detected!' . "\n";
	fwrite($log, $message);
}

?>

This would write a log file to /var/www/html/custom/logs/alert.log in the format:

2009-10-13 11:10:36 192.168.97.1 PHP app Attempt at non-numeric input (possible attack) detected!

Once we have this application log set up we need to adjust our OSSEC configuration so that it reads the new log file. The following change needs to be done in both agent and server’s ossec.conf file. We can add the following lines to our /var/ossec/etc/ossec.conf file to enable OSSEC to read this new log file:

  <localfile>
    <log_format>syslog</log_format>
    <location>/var/www/html/custom/logs/alert.log</location>
  </localfile>

Once OSSEC is monitoring this file (this will require us to restart OSSEC) we’ll need an appropriate decoder. Make this change on ossec server. Add the following in /var/ossec/etc/local_decoder.xml. By default ossec reads only 2 decoder files: decoder.xml and local_decoder.xml. decoder.xml can be overwritten during upgrades, so add all the custom decoders in local_decoder.xml. Writing a decoder for this format would be quite simple. It would appear as:

<!-- Custom decoder for example -->
<decoder name="php-app">
  <prematch>^\d\d\d\d-\d\d-\d\d \d\d:\d\d:\d\d</prematch>
</decoder>

<decoder name="php-app-alert">
  <parent>php-app</parent>
  <regex offset="after_parent">^ (\d+.\d+.\d+.\d+) PHP app</regex>
  <order>srcip</order>
</decoder>

What we’re doing here is telling OSSEC how to extract IP information from the log. All the strings in the regex portion of the new decoder can be assigned, in order, to options listed in the order tag. You can define each of OSSEC’s possible variables and tell OSSEC how to identify them in the logs using the decoder.

Once we have our decoder we can write custom rules based on the log file. This is to be done on ossec server. There are two ways to create custom rules for OSSEC. The first is to alter the ossec.conf configuration file and add a new rule file to the list. The second is to simply append your rules to the local-rules.xml rules file. Either one works, but the second makes upgrading to newer versions of OSSEC a little cleaner.

We’ll add the following group to our local-rules.xml file, found in the rules directory under the OSSEC installation root:

<group name="syslog,php-app,">
  <rule id="110000" level="0">
    <decoded_as>php-app</decoded_as>
    <description>PHP custom app group.</description>
  </rule>

  <rule id="110001" level="10">
    <if_sid>110000</if_sid>
    <srcip>127.0.0.1</srcip>
    <match>attack</match>
    <description>Possible attack from localhost?!?</description>
  </rule>
</group>

You’ll notice that we have two rules. Because rules can be nested it is usually helpful to subdivide them into small, hierarchical pieces. In this case, we have one rule that serves as a catch-all for our custom application alerts. After that, we can write rules for any number of circumstances and have these rules only checked if the parent rule is matched. This rule will fire if an entry is written into the custom alert.log that contains the source IP of 127.0.0.1. The rule id is extremely important in this definition. OSSEC reserves rule id’s above 100,000 for custom rules. It is useful to develop a schema for your new rules, for instance allocating each 1000 above 100,000 for a generic, catch-all rule and writing child rules in that space. This helps to avoid the hassle of having intermingled rule numbers and aids in long term maintenance.

To clarify the case above, there are two rules. The first rule will only fire if a log entry is “decoded_as” or matches the decoder for “php-app.” If this decoder is used then rule 110,000 will be triggered. The second rule is only checked if rule 110,000 is triggered as specified in the if_sid tag. This rule will only be triggered if the source ip, specified in the srcip tag, is equal to ‘127.0.0.1.’ If this is the case then the rule will do a string match for the word “attack” in the log entry. If this match is successful the rule will trigger at level 10 as specified in the rule tag. This will cause an OSSEC alert to be logged with the associated description. OSSEC by default also attempts to e-mail alerts with level 7 or higher to recipients specified in the ossec.conf file. As you can see, with the addition of the decoder and these rules we’ve allowed OSSEC to read our custom format logfile.

While this example may seem straightforward writing your own decoders and rules can be maddening. Because OSSEC will not dynamically load the XML files defining your decoders, rules, or files to watch, you must restart the program to propagate changes. This can be a real hassle when you’re debugging new XML rules or decoders. To alleviate the problem of constantly restarting the server you can use the program ossec-logtest found in the bin directory of the OSSEC installation root. This is present on the ossec server. This program allows you to paste, or type, one line of a log file into the input then traces the decoders and rules that the line matches like so:

# bin/ossec-logtest 
2009/10/13 13:30:25 ossec-testrule: INFO: Started (pid: 14330).
ossec-testrule: Type one log per line.

2009-10-13 12:10:09 127.0.0.1 PHP app Attempt to attack the host!


**Phase 1: Completed pre-decoding.
       full event: '2009-10-13 12:10:09 127.0.0.1 PHP app Attempt to attack the host!'
       hostname: 'webdev'
       program_name: '(null)'
       log: '2009-10-13 12:10:09 127.0.0.1 PHP app Attempt to attack the host!'

**Phase 2: Completed decoding.
       decoder: 'php-app'
       srcip: '127.0.0.1'

**Phase 3: Completed filtering (rules).
       Rule id: '110001'
       Level: '10'
       Description: 'Possible attack from localhost?!?'
**Alert to be generated.

Note that this program will not reload changes, but you can quit ossec-logtest, make changes to any of the XML files then restart it to test your changes. Using ossec-logtest is invaluable when trying to create new rules as it saves you the hassle of restarting the server and the hassle of actually triggering events for which you want to generate alerts.

In case you get an issue like “No decoders match” in ossec-logtest, please check your decoder file. It might be most probably due to some syntax error in your decoder xml.

After your testing in logtest is successful, restart your ossec agent and master with

/var/ossec/bin/ossec-control restart

After this your alerting system will be in place for your custom logs just like other alerts.

Getting hard disk drive serial number at the Linux terminal

You can get the serial number through :

1) hdparm:

hdparm -I /dev/sda | grep Serial

2) sginfo is a part of sg3-utils package:

sginfo -a /dev/sda | grep Serial

3) sdparm command :

sdparm -i /dev/sda | grep 'vendor specific'

4) lshw:

lshw -class disk -class storage | grep serial

How to delete a specific line from a text file in command line on Linux?

sed -i '4d' ./file

Here, -i means edit the file in-place. d is the command to “delete the pattern space; immediately start next cycle”. 4 means the 4th line.

Remove the last line:

sed '$d' filename.txt

Remove all empty lines:

sed '/^$/d' ./file

or

sed '/./!d' ./file

Remove lines from 7 to 9:

sed '7,9d' ./file

Remove the line matching by a regular expression REGULAR:

sed '/REGULAR/d' ./file

For a simple example, remove the lines containing “oops”:

sed '/oops/d' ./file

Why does xargs strip quotes from input?

Why does xargs strip quotes from input text?

Here is a simplified example:

echo "/Place/='http://www.google.com'" | xargs echo

outputs

/Place/=http://www.google.com

Solution 1:

From the xargs manual:

If you want an input argument to contain blanks or horizontal tabs, enclose it in double quotes or apostrophes. If the argument contains a double quote character ("), you must enclose the argument in apostrophes. Conversely, if the argument contains an apostrophe ('), you must enclose the argument in double quotes. You can also put a backslash (\) in front of a character to tell xargs to ignore any special meaning the character may have (for example, white space characters, or quotes).

This means you can escape quotes if the quotes are quoted themselves:

$ echo "/Place/=\'http://www.google.com\'" | xargs echo
/Place/='http://www.google.com'

 

Solution 2:

if you want xargs to ignore quotes one of the good soultion can be the use of xargs flag xargs -0

Directly from Man page OPTIONS

OPTIONS -0, –null

Input items are terminated by a null character instead of by whitespace, and the quotes and backslash are not special (every character is taken literally). Disables the end of file string, which is treated like any other argument. Useful when input items might contain white space, quote marks, or backslashes. The GNU find -print0 option produces input suitable for this mode.

I’ve checked on a GNU system that setting the delimiter to a specific value (like a newline) with -doption (and not just -0) would also cause xargs not to treat the quotes etc specially.

How can I recall the argument of the previous bash command?

Problem:

Is there a way in Bash to recall the argument of the previous command?

I usually do vi file.c followed by gcc file.c.

Is there a way in Bash to recall the argument of the previous command?

Solution:

You can use $_ or !$ to recall the last argument of the previous command.

Also Alt + . can be used to recall the last argument of any of the previous commands.

CHECKING THE DEFAULT SELINUX CONTEXT

Use the matchpathcon command to check if files and directories have the correct SELinux context. From the matchpathcon(8) manual page: “matchpathcon queries the system policy and outputs the default security context associated with the file path”. The following example demonstrates using the matchpathcon command to verify that files in /var/www/html/ directory are labeled correctly:
  1. As the Linux root user, run the touch /var/www/html/file{1,2,3} command to create three files (file1, file2, and file3). These files inherit the httpd_sys_content_t type from the /var/www/html/ directory:
    ~]# touch /var/www/html/file{1,2,3} ~]# ls -Z /var/www/html/ -rw-r--r-- root root unconfined_u:object_r:httpd_sys_content_t:s0 file1 -rw-r--r-- root root unconfined_u:object_r:httpd_sys_content_t:s0 file2 -rw-r--r-- root root unconfined_u:object_r:httpd_sys_content_t:s0 file3
  2. As the Linux root user, run the chcon -t samba_share_t /var/www/html/file1command to change the file1 type to samba_share_t. Note that the Apache HTTP Server cannot read files or directories labeled with the samba_share_t type.
  3. The matchpathcon -V option compares the current SELinux context to the correct, default context in SELinux policy. Run the matchpathcon -V /var/www/html/*command to check all files in the /var/www/html/ directory:
    ~]$ matchpathcon -V /var/www/html/* /var/www/html/file1 has context unconfined_u:object_r:samba_share_t:s0, should be system_u:object_r:httpd_sys_content_t:s0 /var/www/html/file2 verified. /var/www/html/file3 verified.
The following output from the matchpathcon command explains that file1 is labeled with the samba_share_t type, but should be labeled with the httpd_sys_content_ttype:
/var/www/html/file1 has context unconfined_u:object_r:samba_share_t:s0, should be system_u:object_r:httpd_sys_content_t:s0
To resolve the label problem and allow the Apache HTTP Server access to file1, as the Linux root user, run the restorecon -v /var/www/html/file1 command:
~]# restorecon -v /var/www/html/file1 restorecon reset /var/www/html/file1 context unconfined_u:object_r:samba_share_t:s0->system_u:object_r:httpd_sys_content_t:s0

Less: Quickly Jump to Line Number in Large File

If the file is open you can type:

  • 100g to go to the 100th line.
  • 50p to go to 50% into the file.
  • 100P to go to the line containing 100th byte.

You can use these from terminal by adding + in front of them:

less +100g bigfile.txt

iptables: delete specific rule

You can list the existing iptables rule along with their line number with:

iptables -L –line-numbers

The output will be as follows:

Chain INPUT (policy ACCEPT)
num target prot opt source destination
1 KUBE-SERVICES all -- anywhere anywhere /* kubernetes service portals */
2 KUBE-FIREWALL all -- anywhere anywhere
3 ACCEPT udp -- anywhere anywhere udp dpt:domain
4 ACCEPT tcp -- anywhere anywhere tcp dpt:domain
5 ACCEPT udp -- anywhere anywhere udp dpt:bootps
6 ACCEPT tcp -- anywhere anywhere tcp dpt:bootps

Then you can delete the rule with the line-number  3 with:
iptables -D INPUT 3