Security Onion Delete or Reset Snort Data

We are building a portable IDS that we take from location to location to assess different legs of the network.  The concept was to build the box, test it out in the office, configure and apply upgrades the perform some maintenance.

Snort

As for snort, security onion makes sure disk use is below 90% with an hourly cron job, but if you need to delete all data, right now, so you can change networks or image a disk you are out of luck with that cron.  I’ve run the contents manually using:

1
/usr/local/sbin/nsm_sensor_clean --force-yes

That would recover some disk space, but not for our purposes.

According to the Security Onion FAQ, pcaps are stored in /nsm/sensor_data/NAME_OF_SENSOR/dailylogs/ and you can verify their disk usage with du -hs.  Ours was 293G.  You can delete these files by replacing NAME_OF_SENSOR with your sensor name and issuing the following command as root:

1
rm -rf /nsm/sensor_data/NAME_OF_SENSOR/dailylogs/*

MySQL

Something a little more tricky.  For whatever reason, security onion or snort stores data for each day and interface in their own set of tables.  This is a pain to clean by hand so don’t try.  First find out if this is an issue by logging in to mysl from shell using:

1
mysql -uroot

There is no root password.  You can find out the disk size of each of your MySQL databases using this:

1
2
3
4
5
6
7
8
9
SELECT
   table_schema, count(*) TABLES,
   concat(round(sum(table_rows)/1000000,2),'M')
   rows,concat(round(sum(data_length)/(1024*1024*1024),2),'G')
   DATA,concat(round(sum(index_length)/(1024*1024*1024),2),'G')
   idx,concat(round(sum(data_length+index_length)/(1024*1024*1024),2),'G')
   total_size,round(sum(index_length)/sum(data_length),2) idxfrac
FROM
   information_schema.TABLES group by table_schema;

I can see that my big databases are snorby and securityonion_db. You can find out which are the big tables if you like using this:

1
2
3
4
SELECT table_name,
           round(((data_length + index_length) / (1024*1024)),2) as "MBytes"
FROM information_schema.tables
WHERE table_schema = "securityonion_db";

That will give you some file sizes so we know the effects of our commands. Also, you can check the entire directory like so.

1
du -hs /var/lib/mysql

Mine was 1.4G – not acceptable for taking an image.  Now you can purge the sguil data from security onion using a provided script at /usr/local/bin/sguil-db-purge.  My suggestion to you is to copy this file into your home directory and call it just “db-purge” so you don’t get confused.

In order for this to work, you must find the lines:

1
2
DAYSTOKEEP=365
source /etc/nsm/securityonion.conf

….and change them to:

1
2
source /etc/nsm/securityonion.conf
DAYSTOKEEP=0

First, if you don’t change the order, you’ll cause the securityonion.conf script to override the variable.  Changing it to 0 will delete all the sguil archives. Now run that script:

1
/usr/local/bin/sguil-db-purge

That’s good.  It got my /var/lib/mysql down to 678M, but still have more work to do. There’s still a matter of snorby.  However for me this made up only about 250MB and I wasn’t able to get it to work so I left it.  But the suggestion on the list was to run the following command, but it just caused an error for me.

1
bundle exec rake snorby:hard_reset

That’s good enough for me.  You could keep on going if you want by moving into the /nsm directory and deleting logs for things like httpry. I’m prepared to move along at this point with a mostly fresh install, already configured, and ready for imaging.