wottsecurity / agent Goto Github PK

View Code? Open in Web Editor NEW

22.0 7.0 12.0 6.42 MB

Simple security audit for linux systems

Home Page: https://wott.io/

License: MIT License

Dockerfile 0.41% Python 96.99% Shell 2.60%

tls security cis-benchmark

agent's Introduction

WoTT Agent

What is this?

TL;DR: WoTT provides seamless security audit of linux nodes

Our goal is to improve the security posture of your servers and devices. This is our open source agent that connects to the WoTT Dashboard.

Here are some of the things that WoTT will check for:

Continuously analyzing your system for known vulnerabilities
- I.e. a CVE scan of your installed system packages
Auditing your services to ensure they are configured securely
- E.g. making sure your SSH daemon doesn't allow root logins
Making it easy to configure your firewall
Ensuring that you don't have any insecure services running
- E.g. rsh and telnet

In addition to this, we also provide:

A cryptographic identity to each node, that can be used for access control using Mutual TLS (mTLS).
A simple credential management tool to help you remove hard coded credentials and API keys from your system

For more details and installation instructions, please see our Getting Started Guide.

You can also browser our Use Cases for more inspiration.

Supported operating systems

Linux Distribution	Version	Comment
Ubuntu	16.04, 18.04
Ubuntu Core	16, 18	Only works with Snap version
Debian/Raspbian	Jessie (8), Stretch (9), Buster (10)
Amazon Linux	2	CVE scannning not available.

Alternative runtime environments

Due to technical limitations in both Docker and the Snap package, the WoTT agent is unable to perform a full security audit in these environments. For best result, use the Debian package.

agent's People

Contributors

Stargazers

Watchers

Forkers

slilo a-martynovich malinoff greatfruitomsk eugenosm fshmcallister zaiddabaeen swipswaps ealmonte32 librechain sierrasecurity

agent's Issues

Add 20 unit tests

Create 20 unit tests for the agent.

Pack newer version of python-iptables in a deb package

Push it to our PPA. Make sure it is installed instead of the older version from Stretch.

Avoid running multiple same operations at the same time

Running send_ping and fetch_credentials at the same time is fine; running two send_pings or two fetch_credentials is not.

Devise a way to reliably lock a cross-process object and unlock it.

An object could be a "lock file" in /var/lock named smth like wott-agent.ping.lock or wott-agent.creds.lock containing the PID of the process locking it. The actual locking can be done with fcntl.lockf.

There's a ready to use library for that which can be used, but it needs to be tested first.

Question: what happens if a process holding the lock dies? How do we recover from that?

Capture running process

We ant to be able to get a list of the running processes of the device to assess if any rouge process are running.

Create post installation for deb package

The Debian package should do the following things:

Run wott-agent to initiate the package
Print out a post-installation message

Post installation message

Thank you for installing the wott-agent. Your device's unique ID is: [device id]

To unleash the value of WoTT, and to gain insight into the security of your device(s), sign up for an account at http://dash.wott.io.

You can claim your device in the dashboard by visiting:
[device claim URL]

For more information on how to get started, visit wott.io/getting-started

Replace CFSSL on for CSR generation with cryptography

To cut back on requirements, and to simplify the CSR process, we should replace CFSSL with cryptography on the agent.

Enable the ability to "force run" the wott-agent to report in data.

It is sometime useful to "force push" status for a device when you are experimenting. Hence, even after we've converted the agent from a cronjob (or timer) in #35, we still need to be able to run a CLI command to speed things up.

Auto-bump deb version

If CircleCI is building master branch it should run the script which bumps the version before building deb package.
The script should parse the latest debian/changelog record looking for commit hash. If found, it should create another record with all commits messages since the said commit, a bumped version and the latest commit hash.

wott-agent (<RELEASE_VERSION>+ppa<COUNTER>) stable; urgency=medium

  Built from commit <hashhashhash>.
  * Commit one <hashhashhash>
  * Commit two <hashhashhash>
  <...>

 -- WoTT <[email protected]>  Sat, 1 Apr 2019 00:00:00 +0000

Add check to see if the firewall is enabled

One of the security assessments we want to do is to check if the firewall on the device is enabled. There are a few ways to do this, but I would recommend using python-iptables to asses the firewall.

In the first iteration, i just want this function to return something like:

{'firewall_enabled': Bool}

Later on, we want to expand this functionality with more detailed analysis.

Switch signing CA to EC256

Google Cloud will not work with higher bit CA.

Firewall status not being updated due to execution order

If you make a firewall change, the agent must run twice for the changes to be picked up in the interface. This is likely because we collect firewall status before we apply them.

Add ipv6 argument to block_networks/ports

Subtask of WoTTsecurity/api#210

Delete credentials from the device

If a credential is deleted from the dashboard (or made inaccessible to the device), it needs to be cleaned up by the agent to ensure that we're not "leaking" creadentials.

Consolidate Python and Deb version

As it stands right now, we have two different versions of the agent (in the same repo):

The Python version
The Debian package version

let's consolidate the two are coupled right now anyway.

Add ability to intercept syscalls

In order to assess the threat level of the device, we need to be able to extract the syscalls. If we first are able to establish the baseline of commands during normal operations, and then all of the sudden nc is being executed on the device, this would be highly suspicious. Perhaps it is being used for troubleshooting, but it should raise a warning.

There are a few tools we could potentially do this, including:

SELinux
Seccom-bpf
Apparmor

Fix missing dependency

I got the following error upon just installing the agent on a new device:

$ sudo apt-get install wott-agent
[...]
Selecting previously unselected package python3-urllib3.
Preparing to unpack .../06-python3-urllib3_1.19.1-1_all.deb ...
Unpacking python3-urllib3 (1.19.1-1) ...
Selecting previously unselected package python3-requests.
Preparing to unpack .../07-python3-requests_2.12.4-1_all.deb ...
Unpacking python3-requests (2.12.4-1) ...
Selecting previously unselected package python3-sh.
Preparing to unpack .../08-python3-sh_1.11-1_all.deb ...
Unpacking python3-sh (1.11-1) ...
Selecting previously unselected package python3-tz.
Preparing to unpack .../09-python3-tz_2016.7-0.3_all.deb ...
Unpacking python3-tz (2016.7-0.3) ...
Selecting previously unselected package ghostunnel.
Preparing to unpack .../10-ghostunnel_1.4.0+nmu2_armhf.deb ...
Unpacking ghostunnel (1.4.0+nmu2) ...
Selecting previously unselected package wott-agent.
Preparing to unpack .../11-wott-agent_0.1.4.5_all.deb ...
Unpacking wott-agent (0.1.4.5) ...
Setting up python3-sh (1.11-1) ...
Setting up python3-iptables (0.11.0-4) ...
Setting up python3-certifi (2016.2.28-1) ...
Setting up python3-psutil (5.0.1-1) ...
Setting up python3-netifaces (0.10.4-0.1+b1) ...
Setting up python3-chardet (2.3.0-2) ...
Setting up ghostunnel (1.4.0+nmu2) ...
Processing triggers for man-db (2.7.6.1-2) ...
Setting up python3-openssl (16.2.0-1) ...
Setting up python3-urllib3 (1.19.1-1) ...
Setting up python3-tz (2016.7-0.3) ...
Setting up python3-requests (2.12.4-1) ...
Setting up wott-agent (0.1.4.5) ...
Created symlink /etc/systemd/system/timers.target.wants/wott-agent.timer → /lib/systemd/system/wott-agent.timer.
wott-agent.service is a disabled or a static unit, not starting it.
Traceback (most recent call last):
  File "/usr/bin/wott-agent", line 11, in <module>
    load_entry_point('wott-agent==0.1.2', 'console_scripts', 'wott-agent')()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 561, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2631, in load_entry_point
    return ep.load()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2291, in load
    return self.resolve()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2297, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/lib/python3/dist-packages/agent/__init__.py", line 10, in <module>
    from agent import journal_helper
  File "/usr/lib/python3/dist-packages/agent/journal_helper.py", line 1, in <module>
    from systemd import journal
ImportError: No module named 'systemd'
Traceback (most recent call last):
  File "/usr/bin/wott-agent", line 11, in <module>
    load_entry_point('wott-agent==0.1.2', 'console_scripts', 'wott-agent')()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 561, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2631, in load_entry_point
    return ep.load()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2291, in load
    return self.resolve()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2297, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/lib/python3/dist-packages/agent/__init__.py", line 10, in <module>
    from agent import journal_helper
  File "/usr/lib/python3/dist-packages/agent/journal_helper.py", line 1, in <module>
    from systemd import journal
ImportError: No module named 'systemd'

Good job installing the wott-agent and welcome to the community of security conscious hardware devs!

You're one step closer to building a secure device. Your device's unique ID is:

To unleash the value of WoTT, and to gain insight into the security of your device(s), connect your device at https://dash.wott.io.

You can claim your device in the dashboard by visiting:

For more information on how to get started, visit wott.io/getting-started or drop us a line at [email protected].

Regards,
The WoTT Team

Send ping after bootstrap

Currently if you run agent for the first time it will generate device cert, print out claim URL and exit. And if you claim the device and try to see its info (dash.wott.io/devices/) you'll get 404 because no ping was yet sent and thus no info.

Ping should be sent if bootstrapping, too.

Outline how to add TPM support

If the device comes with a TPM, we should use it to store the key. There are many parts to this ticket, but let's start scoping it out.

Open questions:

What TPM devices will we support?
How do we integrate?
What libraries do we use?

nmap needs a timeout

Sometimes it takes ages for nmap to scan. It takes it 2 minutes on Pi Zero. Not sure why, but we can set a timeout by adding --host-timeout=<seconds> option.

Add uptime reporting

It would be helpful to understand the uptime of a device as part of the assessment. Hence we should make this data accessible.

Allow override of default encryption bits

We should provide a sensible default for encryption keys (EC256), but it also makes sense to have the ability to override this if needed.

Don't use iptc.easy for port/network blocking

Deleting a rule fails in 50% of my test runs because it can't find a rule to delete.

Use a "non-easy" API for that:

import iptc

tbl = iptc.Table('filter')
ch = iptc.Chain(tbl, 'INPUT')
for r in ch.rules:
    for m in r.matches:
        print(m.get_all_parameters())
        if m.comment == 'added by WoTT':
            print('deleting')
            ch.delete_rule(r)
            break

Upgrade to the next released `python-iptables` package version when ready

For now we decided to switch to its trunk version because we need iptc.easy interface that isn't release yet.

Block by ports AND hosts

Instead of blocking listening ports from any host we should block them selectively.
New format for security_helper.block_ports():
[{host, proto, port}]
This matches the format returned by security_helper.nmap_scan().

Sometimes show errors at the end of deb package installation

Sometimes at the end of deb wott-agent installation, daemon hangs with errors but installation finished and device_id/claim tocken created and "wellcome" displayed.

possible this would be a solution: #110

Setting up wott-agent (0.1.5.4) ...
Created symlink /etc/systemd/system/wott-agent -> /lib/systemd/system/wott-agent.service.
Created symlink /etc/systemd/system/multi-user.target.wants/wott-agent.service -> /lib/systemd/system/wott-agent.service.
Traceback (most recent call last):
  File "/usr/bin/wott-agent", line 11, in <module>
    load_entry_point('wott-agent==0.1.5', 'console_scripts', 'wott-agent')()
  File "/usr/lib/python3/dist-packages/agent/__main__.py", line 47, in main
    run(ping=False, debug=args.debug, dev=args.dev)
  File "/usr/lib/python3/dist-packages/agent/__init__.py", line 558, in run
    send_ping(debug=debug, dev=dev)
  File "/usr/lib/python3/dist-packages/agent/__init__.py", line 279, in send_ping
    security_helper.block_networks(pong.get('block_networks', []))
  File "/usr/lib/python3/dist-packages/agent/security_helper.py", line 212, in block_networks
    update_iptables(TABLE, OUTPUT_CHAIN, rules)
  File "/usr/lib/python3/dist-packages/agent/security_helper.py", line 169, in update_iptables
    iptc_helper.add_rule(table, chain, r, ipv6=ipv6)
  File "/usr/lib/python3/dist-packages/agent/iptc_helper.py", line 83, in add_rule
    iptc_chain = _iptc_getchain(table, chain, ipv6)
  File "/usr/lib/python3/dist-packages/agent/iptc_helper.py", line 422, in _iptc_getchain
    
  File "/usr/lib/python3/dist-packages/agent/iptc_helper.py", line 414, in _iptc_gettable
    iptc_table.commit()
  File "/usr/lib/python3/dist-packages/iptc/ip4tc.py", line 1561, in commit
    raise IPTCError("can't commit: %s" % (self.strerror()))
iptc.ip4tc.IPTCError: can't commit: b'Resource temporarily unavailable'

Good job installing the wott-agent and welcome to the community of security conscious hardware devs!

You're one step closer to building a secure device. Your device's unique ID is: No certificate found on disk.
Got WoTT ID: 662567da674f45f6b2bc55e657fbd6b3.d.wott.local
Generating certificate...
Submitting CSR...
Got Claim Token: 5fd9b5b3-9325-40a9-8ffd-cfdddc526a4e
Claim your device: https://dash.wott.io/claim-device?device_id=662567da674f45f6b2bc55e657fbd6b3.d.wott.local&claim_token=5fd9b5b3-9325-40a9-8ffd-cfdddc526a4e
Writing certificate and key to disk...
Writing config...

This occurs not on ever installation, so needed a number of remove/install cycles to cause

Version errors

Subtask of #96

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.5/json/__init__.py", line 230, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib/python3.5/json/encoder.py", line 198, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python3.5/json/encoder.py", line 256, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib/python3.5/json/encoder.py", line 179, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: wott-agent 0.1.5.0-7d20ed2 (/usr/lib/python3/dist-packages) is not JSON serializable

pi@raspberrypi:~ $ sudo dpkg -l | grep agent
ii  wott-agent                      0.1.5.284~7d20ed2            all          Let's Encrypt for IoT (with more bells and whistles).

Build number didn't get included into the Python version. Probably because debhelper ignores all env vars and needs them to be specified manually. Also, the tilde character transformed into dash. The deb version is fine.
The error which caused the traceback can be fixed by simply using str(__version__).

Add distribution information

In order to better understand a fleet, we need to know what operating system/distro a given device is running. lsb_release -a is a good example of this:

$ lsb_release -a
No LSB modules are available.
Distributor ID: Raspbian
Description:    Raspbian GNU/Linux 9.8 (stretch)
Release:        9.8
Codename:       stretch

We want to show something similar. However, we need to keep some things in mind:

We need to support Ubuntu Core
We need to support Balena/Resin

Receive credentials from the server

Subtask of WoTTsecurity/api#86
Requires #89

Schedule periodic GET request to /api/.../creds every 15 minutes. Write the result to a file.

Credentials are in the form of [{"name": ..., "key": ..., "value": ...}, ...]. Agent should write {key: value, ...} to a file named {name}.json (grouping credentials by name).

Switch default key to 256 (P-256) from 384.

384 bit EC isn't widely accepted (for instance, Google Cloud does not support it). Hence, in order to provide better comparability, let's drop it to 256.

Add check for logins (failed and successful) in the last hour

We want to collect the number of failed logins in the last hour. Here's how to do this using journalctl:

$ journalctl -l SYSLOG_FACILITY=10 --priority=5 --since "1 hour ago" --no-pager
-- Logs begin at Fri 2019-03-08 00:05:21 PST, end at Fri 2019-03-08 03:31:14 PST. --
Mar 08 03:31:12 wott0 sshd[4698]: pam_unix(sshd:auth): check pass; user unknown
Mar 08 03:31:12 wott0 sshd[4698]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=192.168.x.y

In the above example, the result should be 1.

Please note that this needs to work within the wott-agent-snap, so using the Python module rather than trying to read this from disk is required.

Don't display post-installation instruction if claimed

Let's differentiate between a claimed and unclaimed device in the post-installation instructions. If the device has already been claimed, there is no need to display the message about claiming.

Fix URL for claiming device.

The URL is incorrect for claiming a device right now:
https://api.wott.io/v0.2/claim-device?device-id=abc

We need to be able to send the user to dash.wott.io as that is the user-facing endpoint.

Send list of installed packages to API

In order to assess the security state of the installed packages, we need to submit the list upstream. This is of course a rather complicated task, so we need to make some assumptions to limit the scope. In the initial version, let's only worry about Debian/Raspbian.

For instance, we can extract all the installed packages, along with their versions from Raspbian by running:

$ sudo dpkg -l
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                                                                                  Version                                         Architecture                                    Description
+++-=====================================================================================-===============================================-===============================================-===============================================================================================================================================================================
ii  adduser                                                                               3.115                                           all                                             add and remove users and groups
ii  alsa-utils                                                                            1.1.3-1                                         armhf                                           Utilities for configuring and using ALSA
ii  apparmor                                                                              2.11.0-3+deb9u2                                 armhf                                           user-space parser utility for AppArmor
ii  apt                                                                                   1.4.9                                           armhf                                           commandline package manager
ii  apt-listchanges                                                                       3.10                                            all                                             package change history notification tool
[...]

This is of course not very easily parseable, but serves the purpose of the kind of data we want to send upstream.

Treat 0.0.0.0 and ::: as "any address"

Blocking these addresses explicitly doesn't do anything. So if this address needs to be blocked, just remove it from the rule.

Fix argparse helpers

We need to add helpers to argparse such that the format looks user friendly.

Here's a snippet from the official documentation:

$ python myprogram.py --help
usage: myprogram.py [-h] [--foo FOO]

optional arguments:
 -h, --help  show this help message and exit
 --foo FOO   foo help

Fix linting of code base

The linting is currently failing due to:

$ flake8 ./ --exclude=venv --ignore=E501                                                                                                                                  ./agent/__init__.py:76:5: E722 do not use bare 'except'

Let's fix this.

Rewrite Agent to a service because we need the ability to a) serve an API and b) to be able to spawn sub-processes.

A quick look at this StackOverflow post covers the basics on how to best do this.

Check password against known default passwords.

We should:

Build a list of default passwords (such as pi:raspberry)
Check that list against the system
Report that in the dashboard

Write simple web app example that uses auth backend.

We have a simple webapp example for mTLS, but we'd need one for auth too.

Let's use a similar structure as the last one but let's use this simple HTTP Basic Auth example for Flask to hook it up with credentials generated by the agent.

Don't filter out port block rules and WOTT_LOG_DROP

Create native debian packaging

There are benefits with the Snap packaging platform. However, it also comes with a lot of drawbacks. To move faster, we should move to native Debian packaging as it gives us less restrictions.

Here are some requirements:

We need to be able to build these packages in a build farm
We need to provide multiple architectures

One solution is Ubuntu's Launchpad with their PPAs. Another alternative is packagecloud.

Make agent "runtime aware"

We currently support four different kind of runtime environments:

Debian package
Python package
Docker
Ubuntu Snap

Because we cannot do all operations in all these different environments, we need to first be able to detect what environment we are operating in. Once we have done this, we need to disable certain functionalities, such as port scanning and password checks).

We also need to report the runtime environment upstream, just like the version.

Monitor network connections

In order to determine suspicious network activities, we need to be able to sample the network connections from a given device. In a perfect world, we'd be able to monitor all network connections, but this is unrealistic, so sampling is the next best thing.

A proof of concept illustrating this would be to say run netstat every 60 seconds (with a jitter) and record these transactions. We can then log the connections and submit them upstream for processing.

Make CircleCI run tests

Currently CircleCI just runs the linting (#14). I want to also make sure that it runs the tests.

To do this, we need to:

Add tests (d'oh).
Add the test executioner to .circleci/config.yml. There's an example here.

Publish version of agent as part of payload

For debugging and troubleshooting reasons, we want to know what version a given device is running from the dashboard. Let's submit this as part of the payload and display it under the "Software" tab.

Add arg parser to agent for invoking functions

We have a number of sub-commands in our snap. With the move to native deb packaging, we lose these.

As such, the best solution is likely to introduce an arg parser to the client, such that these commands can be invoked.

Fix out-of-entropy error

OpenSSL throws getrandom() initialization failed.. Possible solution was found here:

I've installed rng-tools and it solved the problem.

Desciption of rng-tools:

The rngd daemon acts as a bridge between a Hardware TRNG (true random number generator) such as the ones in some Intel/AMD/VIA chipsets, and the kernel's PRNG (pseudo-random number generator).
This is an unofficial version of rng-tools which has been extensively modified to add multithreading and a lot of new functionality.

Add it as a dependency of the deb package. Check that it solves the problem.

Fix systemd timer in deb package

After installing the new debian package, we are missing the systemd timer:

$ sudo  systemctl list-timers
NEXT                         LEFT          LAST                         PASSED       UNIT                         ACTIVATES
Tue 2019-04-16 19:03:51 BST  4h 19min left Tue 2019-04-16 11:55:45 BST  2h 48min ago snapd.refresh.timer          snapd.refresh.service
Wed 2019-04-17 05:06:48 BST  14h left      Tue 2019-04-16 13:37:45 BST  1h 6min ago  apt-daily.timer              apt-daily.service
Wed 2019-04-17 06:48:56 BST  16h left      Tue 2019-04-16 06:59:10 BST  7h ago       apt-daily-upgrade.timer      apt-daily-upgrade.service
Wed 2019-04-17 11:28:45 BST  20h left      Tue 2019-04-16 11:28:45 BST  3h 15min ago systemd-tmpfiles-clean.timer systemd-tmpfiles-clean.service

4 timers listed.
Pass --all to see loaded but inactive timers, too.

Separate out hourly and daily reports

There are some payloads that we want to send hourly, and some daily. Currently we only have hourly reports, so we need to refactor this a bit.

Check for default credentials

Let's start building out a basic default credentials check. The easiest way to do this is likely to build out a simple check against /etc/shadows for pre-determined hashes.

For instance, we can build out a database of hashes and check them like this:

pi@wott0:~ $ sudo grep "pi:$6$D12eVhKX$00kKcOd8ExXk0ZruVWRQnukJi4CEW7Jg7DAgf3E6umxe4PQn7ac4X4TobozWbBIthsUM26EA7ZY4Ypvv63H121" /etc/shadow 
pi:$6$D12eVhKX$00kKcOd8ExXk0ZruVWRQnukJi4CEW7Jg7DAgf3E6umxe4PQn7ac4X4TobozWbBIthsUM26EA7ZY4Ypvv63H121:17709:0:99999:7:::