Creating a network intrusion detector

This guide provides a walkthrough on setting up a Raspberry PI to run the OpenCanary software along with the additional elements to make it more usable in my use.

If you are looking for a quick(er) install then check out this shorter page, if you want more detail then carry on here.

If you are asking what any of this means, then you might have to wait for me to add a section on background and related links. This should follow soon, but don’t hold your breath.

Main steps: I will add hyperlinks to these sections later (I hope)

1. Get and setup a Raspberry Pi
2. Install the OpenCanary software and pre-requisites
3. Install additional protocols
4. Configuration and Setup
5. Adding reporting system
6. Additional system monitoring

Get and setup a Raspberry Pi

Once the PI is setup then run the commands to do a system update:

sudo apt update
sudo apt upgrade

Any prompts for space use, just say yes.

Once these are done the system should be on the latest version.

Now you can do a few basic config changes to make the Pi more suitable for being an Open Canary.

Changing the MAC address

You can skip this stage, but if you do then anyone looking on the network will see the mac address of a Raspberry Pi and will know that this is not what it appears to be.

So decide what you want the Pi to look like and find a mac address of a suitable device. In my case I was planning on making it look like a Synology NAS. So I searched online to find sample mac addresses for these devices. I found a number that start 00:11:32:xx:xx:xx and so chose that as a basis and randomly picked the lower 3 bytes.

Then use the command:

sudo nano /boot/cmdline.txt

You can then edit the command line so that it ends as:

console=serial0,115200 console=tty1 root=/dev/mmcblk0p7 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait fbcon=map:10 fbcon=font:VGA8x8 quiet splash plymouth.ignore-serial-consoles smsc95xx.macaddr=00:11:32:11:22:33

Note this might be slightly different depending on you Pi’s configuration, and you should only need to change the last part (smsc95xx onwards) to add/update the mac address accordingly to the one you previously identified.

Change the Raspberry Pi name

All very well changing the mac address, but if the Pi still announces itself as RaspberryPi then it’s going to be clear it’s not a Synology.

I used the GUI and just changed the name of the machine to something more reasonable “myserver” or similar.

Frist edit the /etc/host file:

sudo nano /etc/hosts

Change the last entry from 127.0.1.1 raspberrypi to 127.0.1.1 myserver.

And then change the /etc/hostname file:

sudo nano /etc/hostname

and replace the raspberrypi with the myserver name.

Once these are both done, you can reboot to let the changes apply.

sudo reboot now

Install the OpenCanary software and pre-requisites

Now we will install the latest features needed.

sudo apt-get install git python-virtualenv python-pip python-dev libssl-dev libffi-dev

Install a virtual environment

Although not needed, it is best to run the OpenCanary in a virtual environment as it helps keep dependencies easier to manage.

Note that the OpenCanary itself is supposed to work with Python 2 and Python 3, but as Python 2 will soon stop getting updates I decided to go with Python 3. This did cause some issues later when I tried using some of the more advanced notification methods, but I worked around that anyway.

From within the virtual environment

pip install –upgrade pip setuptools

TIP – note this is two hyphens not one as it can appear on some screens.

Clone and install the git repository

git clone https://github.com/thinkst/opencanary
cd opencanary
sudo python setup.py install

This will take some time and you will get some warnings about depreciated calls, in my case I went away and made some tea, so I can’t say how long.

Next we need to set up a link and update a couple file as it seems to get confused on some of the install.

sudo link /usr/local/bin/twistd /home/pi/canary-env/bin/twistd
sudo link /usr/local/bin/twistd /home/pi/opencanary/bin/twistd
cp /home/pi/opencanary/build/scripts-3.7/opencanary.tac /home/pi/canary-env/bin/opencanary.tac
sudo cp ./build/scripts-3.7/opencanary.tac /usr/local/bin/opencanary.tac

Initial configuration

According to the guides, it should create a config file, but for some reason mine didn’t seem to work, so I had to create my own one. The following is an example that should work and does include a modification to write to two log files. This I use later in this guide.

Create the opencanary.conf file with:

sudo nano /home/pi/opencanary/opencanary.conf

then add the following to it.

{
     "device.node_id": "opencanary-1",
     "git.enabled": false,
     "git.port" : 9418,
     "ftp.enabled": true,
     "ftp.port": 21,
     "ftp.banner": "NAME OF THE FTP SITE YOU WANT",
     "http.banner": "Apache/2.2.22 (Ubuntu)",
     "http.enabled": true,
     "http.port": 80,
     "http.skin": "nasLogin",
     "httpproxy.enabled" : false,
     "httpproxy.port": 8080,
     "httpproxy.skin": "squid",
     "logger": {
         "class": "PyLogger",
         "kwargs": {
             "formatters": {
                 "plain": {
                     "format": "%(message)s"
                 },
                 "syslog_rfc": {
                     "format": "opencanaryd[%(process)-5s:%(thread)d]: %(name)s %(levelname)-5s %(message)s"
                 }
             },
             "handlers": {
                 "console": {
                     "class": "logging.StreamHandler",
                     "stream": "ext://sys.stdout"
                 },
                 "file": {
                     "class": "logging.FileHandler",
                     "filename": "/var/tmp/opencanary.log"
                 },
                 "file2": {
                     "class": "logging.FileHandler",
                     "filename": "/var/tmp/opencanary-tmp.log"
                 }
             }
         }
      },
     "portscan.enabled": true,
     "portscan.logfile":"/var/log/kern.log",
     "portscan.synrate": 5,
     "portscan.nmaposrate": 5,
     "portscan.lorate": 3,
     "smb.auditfile": "/var/log/samba-audit.log",
     "smb.enabled": false,
     "mysql.enabled": false,
     "mysql.port": 3306,
     "mysql.banner": "5.5.43-0ubuntu0.14.04.1",
     "ssh.enabled": false,
     "ssh.port": 22,
     "ssh.version": "SSH-2.0-OpenSSH_5.1p1 Debian-4",
     "redis.enabled": false,
     "redis.port": 6379,
     "rdp.enabled": true,
     "rdp.port": 3389,
     "sip.enabled": false,
     "sip.port": 5060,
     "snmp.enabled": true,
     "snmp.port": 161,
     "ntp.enabled": true,
     "ntp.port": 123,
     "tftp.enabled": true,
     "tftp.port": 69,
     "tcpbanner.maxnum":10,
     "tcpbanner.enabled": false,
     "tcpbanner_1.enabled": false,
     "tcpbanner_1.port": 8001,
     "tcpbanner_1.datareceivedbanner": "",
     "tcpbanner_1.initbanner": "",
     "tcpbanner_1.alertstring.enabled": false,
     "tcpbanner_1.alertstring": "",
     "tcpbanner_1.keep_alive.enabled": false,
     "tcpbanner_1.keep_alive_secret": "",
     "tcpbanner_1.keep_alive_probes": 11,
     "tcpbanner_1.keep_alive_interval":300,
     "tcpbanner_1.keep_alive_idle": 300,
     "telnet.enabled": false,
     "telnet.port": 23,
     "telnet.banner": "",
     "telnet.honeycreds": [
         {
             "username": "admin",
             "password": "$pbkdf2-sha512$19000$bG1NaY3xvjdGyBlj7N37Xw$dGrmBqqWa1okTCpN3QEmeo9j5DuV2u1EuVFD8Di0GxNiM64To5O/Y66f7UASvnQr8.LCzqTm6awC8Kj/aGKvwA"
         },
         {
             "username": "admin",
             "password": "admin1"
         }
     ],
     "mssql.enabled": false,
     "mssql.version": "2012",
     "mssql.port":1433,
     "vnc.enabled": true,
     "vnc.port":5000
 }

You should now be able to start the OpenCanary in development mode, so you can see the output directly.

If you then use a tool to scan ports, you should see the raspberry PI is offering a number of open ports, the exact number and types will depend on what you have configured in the above config file.

Making it Autostart

Now we have the detection software running, it’s time to make it automatically start on powering up the Pi.

You can skip this stage if you are going to always start it manually, but for me I like this being automatic.

First Step is to make a service file that provides the details of how to start the opencanaryd as a service, along with the virtual environment and relevant paths.

sudo nano /etc/systemd/system/opencanary.service

Then put the following code into it. If you have changed any paths then make sure you update this accordingly.

[Unit]
Description=OpenCanary honeypot
After=syslog.target
After=network.target

[Service]
User=root
Restart=always
Environment=VIRTUAL_ENV=/home/pi/canary-env/
Environment=PATH=$VIRTUAL_ENV/bin:/usr/bin:$PATH
WorkingDirectory=/home/pi/opencanary
ExecStart=/home/pi/opencanary/bin/opencanaryd --dev

[Install]
WantedBy=multi-user.target

Now we enable and start the service manually and check it’s status.

sudo systemctl enable opencanary.service
sudo systemctl start opencanary.service
systemctl status opencanary.service

If all is well this should show that the service is up and running with no errors.

You can now reboot. Once it’s up and running you can re-check if the service started with:

systemctl status opencanary.service

Something to watch for. If for some reason your Pi shuts down incorrectly (such as a power failure), then the the canary will still restart, but wont be running properly. To fix this, go into the /home/pi/opencanary directory and delete the file twistd.pid. Then reboot and it should be working normally.

Clearly we don’t get the displays on screen now, so this is where the logs come in. For me and this project I split the logs and use the file in /var/tmp/opencanary-tmp.log as a temporary file and this one will get emptied often. For security I do this from a normal user (Pi) and not root so we will need to open the access on this file:

sudo chmod 666 /var/tmp/opencanary-tmp.log

For testing empty the file using something simple like:

echo “start” > /var/tmp/opencanary-tmp.log

Trigger some event (using fing or similar) and then check the file. It should then contain just the new events. Meanwhile the main file /var/tmp/opencanary.log will have a full record of all the events that have happened.

Making it visible

The Opencanary does seem to come with some tools for notification, but I seemed to run into too many issues making these work, so this is where I deviated from the existing approaches and decided to write a simple python script to do the reporting and displaying. The following is very much a simple tool, and I do expect to update it in the near future as part of a follow on project. However that said this works for me now and has been running for a couple of months with no issues.

mkdir /home/pi/opencanary/logtool
nano /home/pi/opencanary/logtool/log-reader.py

 
import smtplib
import string
import os
from time import strftime
import sys

#Settings
TO_ADDRESS = 'DestinationEmail@email.com'
SMTP_SERVER = 'smtp.gmail.com'
 
SMTP_USERNAME = "sourceEmail@email.com"
SMTP_PASSWORD = "xxxxxxxxxx"
SUBJECT = "OpenCanary Alert"

EMAILTEMPLATE = """From: OpenCanary RPi 
Subject: OpenCanary Alert from RPi

New Events
"""

NEWLINE = "\n\r"
SOURCELOGFILE = '/var/tmp/opencanary-tmp.log'
 
# very basic code to send a simple email to the defined recipient
def  SendEmail(emailText):
    emailSent = False
    try:
        emailMessage = EMAILTEMPLATE + emailText
        server = smtplib.SMTP(SMTP_SERVER,587)
        server.ehlo()
        server.starttls()
        server.login(SMTP_USERNAME, SMTP_PASSWORD)
        server.sendmail(SMTP_USERNAME, TO_ADDRESS, emailMessage)
        server.quit()
        emailSent = True
    except:
        print("Error sending Emails - Log not Emptied")
    return emailSent


 
# utility to find the value for a given source parameter.
# sample of event string elements is
# , "node_id": "opencanary-1", "src_host": "192.168.0.11", "src_port": "37284"}
# the string to check should be of the format "src_port"

def findParam(sourceEvent,checkString):
    result = ""
    fullCheckString = '"'+checkString+'": "'
    startChar = sourceEvent.find(fullCheckString)
    if(startChar>0):
        startChar +=len(fullCheckString)
        endChar =sourceEvent.find('"',startChar)
        if(endChar>0):
            result = sourceEvent[startChar:endChar]
        else:
            print("no matching end \n")
    return result
 
#basic parser for each line of text to see if it is one of the whitelisted events that do not need reporting
def CheckLine (sourceEvent):
     sendTheEmail = True 
     print("checking line > {}\n",sourceEvent) 

     sourceIP        = findParam(sourceEvent,"src_host") 
     destinationPort = findParam(sourceEvent,"dst_port") 
     sourcePort      = findParam(sourceEvent,"src_port") 
     print("source IP: {}   destination port: {} \n".format(sourceIP,destinationPort)) 
     #better code would be to use a config file, but for now lets just add some simple cases 
     if(sourceIP =="127.0.0.1"):        
         if(destinationPort=="631"):        
             #local port on Rpi  doing a regular check of the printer port        
             sendTheEmail = False       
     else:     
         if(sourceIP == "192.168.0.13"):         
             if(destinationPort == "445") or (destinationPort == "139" ):             
                 #mac mini doing a regular port check on these 2 ports             
                 sendTheEmail = False     
         else:         
             if(sourceIP == "192.168.0.15"):             
                 if(destinationPort == "139"):                 
                      #main PC on wired network                 
                      sendTheEmail = False 
     displayCommand = "{0}:{1} > {2}  ".format (sourceIP,sourcePort,destinationPort)
     if (sendTheEmail):     
         displayCommand += '\033[31;40m UNKNOWN \033[37;40m\n' 
     else:     
         displayCommand += '\033[32;40m Ignored \033[37;40m\n' 
     
     f = open("/dev/tty1", "w")
     f.write(displayCommand)
     f.close() 
     return sendTheEmail

#main code starts here
localText = ""
 

file2 = open(SOURCELOGFILE,'r')
count  =0
for line in file2:
    if (CheckLine(line.strip())==True):
        count +=1
        localText += "Event {}: {}".format(count,line.strip())
        localText += NEWLINE
    else:
        print("ignoring line\n\r")
file2.close

if (count >0):
    emailsSent= SendEmail (localText)
    if(emailsSent):
        #this is a bit crude but acts as a simple emptying of the source file
        # Only clear the log if the email was sent
        # if not the log will remain and next time it will re-try
        file2 = open(SOURCELOGFILE,'w')
        file2.writelines([])
        file2.close

NOTE: The above code has been updated to remove a possible code injection vulnerability. This is also I think easlier to read, so I hope this helps everyone.

Update 2 (11/07/2021): this code now correctly does the line terminations. Further there is extra code added to ensure the log is not cleared if there are any errors while sending the emails.
Updates thanks to comments from readers.

To test this as is, generate a few intrusions (use the Fing tool again) and check that there are errors in the /var/tmp/opencanary-tmp.log file and then you can run the python script by:

python3 log-reader.py

It should then provide a number of lines of output reporting the intrusions and give a source IP and destination Port summary. If you running in a window still then this will be all you see, if however you are running full screen console then you should also see some summary lines with red (and maybe green) reports.

If you are lucky and entered your email details correct then you may also find you are receiving emails with this summary in it.

Now if we make this python script automatically run every minute then the pi will be set up to run and report automatically.

crontab -e

select 1 (nano) if you are prompted

then at the end of the file add the line

* * * * * python /home/pi/opencanary/logtool/log-reader.py

restart the Pi making sure it is in console mode and once up and running, send a few intrusions (fing again) and within a minute you will get the screen updated showing the summary of the alerts, similarly you will get the emails sent to you.

After running for a while you will find there are some devices on your network that do regularly like to connect to the Pi, this is where the filtering I put in stops me getting emails for regular (known) events but still sends me the messages for new (unexpected) events. On the console this will show the expected items in green (and not emailed) and unknown ones in Red (and will send the email). On a big screen this may seem very wasteful of the screen real estate, but in my instance I have also added a small Ada Fruit 2.2″ screen to the Pi and set this up as my console output. This then allows me to leave the Pi on and running with a simple screen showing any intrusions.

This shows it runng and attached to the monitor arm of my main screen. Note there is also a DHT11 temperature monitor sensor attached, which I also use. That’s another page for another day.

References

A lot of the reference material for this came from the post by Michael Van Delft which can be found at https://xo.tc/installing-opencanary-on-a-raspberry-pi.html I tried using this as it was but I was getting a few too many issued, probably due to my preference for Python3. But all the same many thanks go out to him for the earlier work.

The original Opencanary information comes itself from the Thinkst site itself. If anyone is looking for a more commercial offering then the full Canary systems would be a great place to go. Thinkst main commercial site.