google / container-explorer Goto Github PK

License: Apache License 2.0

Go 98.31% Shell 1.69%

container-explorer's Introduction

Container Explorer

Container Explorer (container-explorer) is a tool to explore containers of a disk image. Container Explorer supports exploring containers managed using containerd and docker container runtimes. Container Explorer attempts to provide the familiar output generated by tools like ctr and docker.

Container Explorer provides the following functionalities:

Exploring namespaces
Exploring containers
Exploring images
Exploring snapshots
Exploring contents
Mounting containers
Support JSON output

You can build the Container Explorer using the instruction at Build Container Explorer.

If you don't want to build, the binaries are available on https://github.com/google/container-explorer/releases.

Usage

The figure below shows the output of the container-explorer --help command.

NAME:
   container-explorer - A standalone utility to explore container details
 
USAGE:
   container-explorer [global options] command [command options] [arguments...]
 
VERSION:
   0.0.2
 
DESCRIPTION:
   A standalone utility to explore container details.
  
  Container explorer supports exploring containers managed using containerd and
  docker. The utility also supports exploring containers created and managed
  using Kubernetes.
  
 
COMMANDS:
   list, ls              Lists container related information
   info                  show internal information
   mount                 mount a container to a mount point
   mount-all, mount_all  mount all containers
   help, h               Shows a list of commands or help for one command
 
GLOBAL OPTIONS:
   --debug                                   enable debug messages
   --containerd-root value, -c value         specify containerd root directory
   --image-root value, -i value              specify mount point for a disk image
   --metadata-file value, -m value           specify the path to containerd metadata file i.e. meta.db
   --snapshot-metadata-file value, -s value  specify the path to containerd snapshot metadata file i.e. metadata.db.
   --namespace value, -n value               specify container namespace (default: "default")
   --docker-managed                          specify docker manages standalone or Kubernetes containers
   --docker-root value                       specify docker root directory. This is only used with flag --docker-managed
   --support-container-data value            a yaml file containing information about support containers
   --output value                            output format in json, table. Default is table (default: "table")
   --help, -h                                show help
   --version, -v                             print the version

Container Explorer helps you explore containers on a mounted disk image. Let's assume we have a clone of the Google Kubernetes Engine (GKE) node attached on a forensic VM as /dev/sdb.

List the disk partition table.

sudo fdisk -l /dev/sdb

The output of the fdisk command.

Disk /dev/sdb: 10 GiB, 10737418240 bytes, 20971520 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 7C818738-EDF0-B246-960D-0E7EE8655B06

Device     Start      End  Sectors  Size Type
/dev/sdb1  8704000 20971486 12267487  5.8G Linux filesystem
/dev/sdb2    20480    53247    32768   16M ChromeOS kernel
/dev/sdb3  4509696  8703999  4194304    2G ChromeOS root fs
/dev/sdb4    53248    86015    32768   16M ChromeOS kernel
/dev/sdb5   315392  4509695  4194304    2G ChromeOS root fs
/dev/sdb6    16448    16448        1  512B ChromeOS kernel
/dev/sdb7    16449    16449        1  512B ChromeOS root fs
/dev/sdb8    86016   118783    32768   16M Linux filesystem
/dev/sdb9    16450    16450        1  512B ChromeOS reserved
/dev/sdb10   16451    16451        1  512B ChromeOS reserved
/dev/sdb11      64    16447    16384    8M BIOS boot
/dev/sdb12  249856   315391    65536   32M EFI System

Mount the /dev/sdb1 as read-only disk on mount point /mnt/case.
```
sudo mount -o ro,noload,noexec /dev/sdb1 /mnt/case
```

Use container-explorer to explore the mounted image.

sudo ce -i /mnt/case --support-container-data supportcontainer.yaml list containers

Mount an individual container or all containers

Mount a container to mount point /mnt/container.

sudo ce -i /mnt/case –support-container-data supportcontainer.yaml -n k8s.io mount f3c910583a81e7441e2cbd209b72afa4740e676ff8d82f2c74fdc5c78e179c10 /container

Mount all containers to mount point /mnt/container. Mounting all containers will create sub-directories using container ID as directory name.

sudo ce -i /mnt/case –support-container-data supportcontainer.yaml mount-all /mnt/container

List the mounted containers within /mnt/container/.

sudo ls -l /mnt/container

The output of the command.

drwxr-xr-x 1 root root 4096 Feb  5 08:55 3544209cfda893703458d7d0a6a65970bfb46e9be6a60faa1e4e9d0adae11b55
drwxr-xr-x 1 root root 4096 Feb  5 08:54 3646fe81507be0510e9191d7e34adbeb751e7ecd86f7e1657289968828c5c8e3
drwxr-xr-x 1 root root 4096 Feb  5 08:54 68a04caa81f9a4265e53a83b50874faca5a7c8400ee0c064d40d81cde6f03b86
drwxr-xr-x 1 root root 4096 Feb  5 09:14 6f68aeae9c0288c2412f793d3a7b85efac189786ed8da2bdce9f88d39827fb80
drwxr-xr-x 1 root root 4096 Feb  5 08:55 7227972ec83761790a65c137239c48817a26b8ad85be74b1ecf751656a2a61be
drwxr-xr-x 1 root root 4096 Feb  5 09:13 cc9bc4f6c6b35b8a3616d8b4586741d8dc148c62b394d276dfab7572ee5aa542
drwxr-xr-x 1 root root 4096 Feb  5 09:13 d3d1ff8c4ef39acbdf0a44bee6c326786309e408942d6a2d42cbaa1661bac77f
drwxr-xr-x 1 root root 4096 Feb  5 08:54 f3c910583a81e7441e2cbd209b72afa4740e676ff8d82f2c74fdc5c78e179c10

Use your favorite forensic tool to process mounted containers.

Mounting Disk Image

Let's assume you have a GKE node disk image as clone-gke-wp-cluster-default-pool-b4e5d97b-btxm.img.

List the partition table.

sudo fdisk -l clone-gke-wp-cluster-default-pool-b4e5d97b-btxm.img

The output of the fdisk -l command.

Disk clone-gke-wp-cluster-default-pool-b4e5d97b-btxm.img: 10 GiB, 10737418240 bytes, 20971520 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 7C818738-EDF0-B246-960D-0E7EE8655B06

Device                                                  Start      End  Sectors  Size Type
clone-gke-wp-cluster-default-pool-b4e5d97b-btxm.img1  8704000 20971486 12267487  5.8G Linux filesystem
clone-gke-wp-cluster-default-pool-b4e5d97b-btxm.img2    20480    53247    32768   16M ChromeOS kernel
clone-gke-wp-cluster-default-pool-b4e5d97b-btxm.img3  4509696  8703999  4194304    2G ChromeOS root fs
clone-gke-wp-cluster-default-pool-b4e5d97b-btxm.img4    53248    86015    32768   16M ChromeOS kernel
clone-gke-wp-cluster-default-pool-b4e5d97b-btxm.img5   315392  4509695  4194304    2G ChromeOS root fs
clone-gke-wp-cluster-default-pool-b4e5d97b-btxm.img6    16448    16448        1  512B ChromeOS kernel
clone-gke-wp-cluster-default-pool-b4e5d97b-btxm.img7    16449    16449        1  512B ChromeOS root fs
clone-gke-wp-cluster-default-pool-b4e5d97b-btxm.img8    86016   118783    32768   16M Linux filesystem
clone-gke-wp-cluster-default-pool-b4e5d97b-btxm.img9    16450    16450        1  512B ChromeOS reserved
clone-gke-wp-cluster-default-pool-b4e5d97b-btxm.img10   16451    16451        1  512B ChromeOS reserved
clone-gke-wp-cluster-default-pool-b4e5d97b-btxm.img11      64    16447    16384    8M BIOS boot
clone-gke-wp-cluster-default-pool-b4e5d97b-btxm.img12  249856   315391    65536   32M EFI System

Mount the first partition (Linux Filesystem)

sudo mount -o ro,noload,noexec,offset=$((8704000*512)) clone-gke-wp-cluster-default-pool-b4e5d97b-btxm.img /mnt/case

Docker Containers

Container Explorer supports exploring Docker managed containers. Use --docker-managed global flag to explore Docker containers.

sudo ce -i /mnt/case --support-container-data supportcontainer.yaml --docker-managed list containers

Container Explorer supports the following operation on Docker containers:

Listing containers
Listing images
Mounting an individual container
Mounting all containers
Excluding containers by image, hostname, and labels

Excluding Containers

When a GKE cluster is created, several containers are created to support the Kubernetes. These clusters are used to support Kubernetes only and may not be interesting for the investigation.

The Kubernetes support containers are hidden by default when the global flag --support-container-data=supportcontainer.yaml is used.

The supportcontainer.yaml contains the commonly known hostname, image, and labels used to identify the support containers.

When --support-container-data is used, the list and mount-all commands automatically ignores the known support containers where applicable. You can use --show-support-containers and --mount-support-containers to display and mount the support containers.

Filtering Containers

Container Explorer supports filtering containers using the labels. This is particularly handy while reviewing GKE containers. Filter supports comma separated key/value pairs. The filter --filter io.cri-containerd.kind=container lists containerd containers.

The command below shows containers in pod namespace default.

/opt/container-explorer/bin/ce -i /mnt list containers --filter io.cri-containerd.kind=container,io.kubernetes.pod.namespace=default

Installing Container Explorer

Follow the steps below to install a pre-compiled Container Explorer on Linux systems.

Download setup script setup.sh which is located at https://github.com/google/container-explorer/blob/main/script/setup.sh
```
wget https://raw.githubusercontent.com/google/container-explorer/main/script/setup.sh
```
Run the script with root privileges.
```
sudo bash setup.sh install
```
Container Explorer files will be created at /opt/container-explorer
Run Container Explorer
```
/opt/container-explorer/bin/ce -h
```
Note: supportcontainer.yaml is located at /opt/container-explorer/etc/supportcontainer.yaml

Build Container Explorer

Follow the steps below to compile the Container Explorer.

Verify Golang version is 1.20 or above
```
go version
```

Clone Container Explorer github project

git clone https://github.com/google/container-explorer

Compile the code

cd container-explorer
go build -ldflags '-s -w' -o $HOME/ce cmd/main.go

Run container-explorer
```
$HOME/ce -h
```

container-explorer's People

Contributors

Stargazers

Watchers

Forkers

leozz37 scottdef rgayon isabella232 ductnn jaguar13 dandugula 0xcpu dfjxs

container-explorer's Issues

mount uses the wrong order of lowerdirs for overlayfs

Ahoi @roshanmaskey !
First of all: Thank your for your super cool tool! Came in really handy as i am currently working on a master thesis about Kubernetes forensics. :)

I've evaluated your tool for post-mortem forensics by using an exported image from a saved virtual machine which is a worker node in a Kubernetes cluster, where i deployed a pod that pulls a specially crafted container that downloads two more files when active.

When the container runs and DownloadFiles.sh has been executed, the directory /home/SampleFiles/ contains the following files:

# ls -la /home/SampleFiles/
total 8336
drwxr-xr-x 1 root root    4096 Nov 17 17:59 .
drwxr-xr-x 1 root root    4096 Nov 15 23:58 ..
-rwxr-xr-x 1 root root    1089 Nov 15 23:50 DownloadFiles.sh
-rwxr-xr-x 1 root root 2155399 Nov 15 21:09 File01.2MB.tmp
-rwxr-xr-x 1 root root 2097180 Nov 15 21:09 File02.2MB.tmp
-rw-r--r-- 1 root root 2155399 Nov 17 17:59 File03.2MB.tmp
-rw-r--r-- 1 root root 2097180 Nov 17 17:59 File04.2MB.tmp

The used image for the container is: docker.io/pr3l14t0r/forensics:craftedfiles

I've now stumbled across a problem with the implementation of the container mount options.

Context

I can see the contents of that container by manually digging through the snapshotter fs directories by using Autopsy or whatever tool which is capable of parsing the exported disk image.

Now i want to try to mount the container using your tool.

This is how the mount options for the container look like when you run findmnt on the worker node when live:

Target                                                                                                                                                        Source                 FSType     Options
------                                                                                                                                                        ------                 ------     -------
/run/containerd/io.containerd.runtime.v2.task/k8s.io/2df8c9bf881593eea104ed0cce6234582aef7f26a13fdf034ca1fdd6abf60b46/rootfs                                  overlay                overlay    rw,relatime,lowerdir=/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/44/fs:/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/43/fs:/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/42/fs:/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/41/fs:/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/40/fs:/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/39/fs,upperdir=/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/45/fs,workdir=/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/45/work,xino=off

Notice that the commited layers are mounted from last to first, so 44-39 being the lowerdirs and 45 being the upper- and work-dir reflecting the live file system mounted into the container.

So.. the "lowest" lowerdir - the first layer of a container image - needs to be specified last in the lowerdir= parameter.

The Issue

When mounting this container with container-explorer, the following (debug) output is given:

container-explorer --debug --image-root /tmp/investigation -n k8s.io mount 2df8c9bf881593eea104ed0cce6234582aef7f26a13fdf034ca1fdd6abf60b46 /mnt/container

DEBU[0000] user mount command                            containerid=2df8c9bf881593eea104ed0cce6234582aef7f26a13fdf034ca1fdd6abf60b46 mountpoint=/mnt/container namespace=k8s.io
DEBU[0000] container environment                         container-root=/var/lib/containerd image-root=/tmp/investigation manifest-file=
DEBU[0000] updated metadata file                         path=/tmp/investigation/var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db
DEBU[0000] snapshot information for container 2df8c9bf881593eea104ed0cce6234582aef7f26a13fdf034ca1fdd6abf60b46  id=2df8c9bf881593eea104ed0cce6234582aef7f26a13fdf034ca1fdd6abf60b46 image="sha256:84e90d9540723be712a71a8ed21f082847b89f444d9c9c8a9d2696682d1f9848" snapshotkey=2df8c9bf881593eea104ed0cce6234582aef7f26a13fdf034ca1fdd6abf60b46 snapshotter=overlayfs
DEBU[0000] container root directory                      path=/tmp/investigation/var/lib/containerd
DEBU[0000] snapshotter root directory                    path=/tmp/investigation/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs
DEBU[0000] snapshotter database file                     path=/tmp/investigation/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/metadata.db
DEBU[0000] snapshotter root directory                    path=/tmp/investigation/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs
DEBU[0000] overlay directories                           lowerdir="/tmp/investigation/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/39/fs:/tmp/investigation/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/40/fs:/tmp/investigation/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/41/fs:/tmp/investigation/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/42/fs:/tmp/investigation/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/43/fs:/tmp/investigation/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/44/fs" upperdir=/tmp/investigation/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/45/fs workdir=/tmp/investigation/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/45/work
DEBU[0000] mount command options: [-t overlay overlay -o ro,lowerdir=/tmp/investigation/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/39/fs:/tmp/investigation/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/40/fs:/tmp/investigation/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/41/fs:/tmp/investigation/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/42/fs:/tmp/investigation/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/43/fs:/tmp/investigation/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/44/fs:/tmp/investigation/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/45/fs /mnt/container]
root@73e111c726f1:/go#

You see in the mount options that the fs directories are mounted as lowerdirs from 39-45, which is actually the wrong order.
The first commited layer of an image would overwrite everything that the other layers are doing to it.

This is proven by the follwing command:

root@73e111c726f1:/go# ls -la /mnt/container/home/SampleFiles/
total 8
drwxr-xr-x 2 root root 4096 Nov 15 23:58 .
drwxr-xr-x 1 root root 4096 Apr 15  2020 ..
root@73e111c726f1:/go#

The files shown above are not listed as they got deleted. They are present in layer 45 but not in 39, thus 39 will remove the files.

Solution

If you manually change the order in the mount command, everything is fine again:

# create mountpoint
mkdir /mnt/container

# mount dirs
mount -t overlay overlay -o ro,lowerdir=/tmp/investigation/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/45/fs:/tmp/investigation/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/44/fs:/tmp/investigation/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/43/fs:/tmp/investigation/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/42/fs:/tmp/investigation/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/41/fs:/tmp/investigation/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/40/fs:/tmp/investigation/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/39/fs /mnt/container

# List mount directory
ls -la /mnt/container/home/SampleFiles/

total 8336
drwxr-xr-x 1 root root    4096 Nov 17 17:59 .
drwxr-xr-x 1 root root    4096 Nov 15 23:58 ..
-rwxr-xr-x 1 root root    1089 Nov 15 23:50 DownloadFiles.sh
-rwxr-xr-x 1 root root 2155399 Nov 15 21:09 File01.2MB.tmp
-rwxr-xr-x 1 root root 2097180 Nov 15 21:09 File02.2MB.tmp
-rw-r--r-- 1 root root 2155399 Nov 17 17:59 File03.2MB.tmp
-rw-r--r-- 1 root root 2097180 Nov 17 17:59 File04.2MB.tmp

So long story short: You need to reverse the order of lowerdirs that you pass to mount. :)

Unfortunately i am not that skilled in Go and need the current time heavily to finish my thesis.. Otherwise i'd try to come up with a PR, so please have mercy.. :D

Kind regards,
pr3l14t0r

PS: If needed i can export the /var/lib/containerd directory of my exported image for you to test. But you can reproduce this with any other "once deployed" container that has an image using multiple layers.

README says binaries available but are not

Can't compile binary on a fresh Ubuntu bionic

There is probably some go relevant setup required in the README

# go build -ldflags '-s -w' -o $HOME/container-explorer cmd/main.go
cmd/main.go:22:2: cannot find package "github.com/google/container-explorer/cmd/commands" in any of:
        /usr/lib/go-1.10/src/github.com/google/container-explorer/cmd/commands (from $GOROOT)
        /home/user/go/src/github.com/google/container-explorer/cmd/commands (from $GOPATH)
cmd/main.go:23:2: cannot find package "github.com/sirupsen/logrus" in any of:
        /usr/lib/go-1.10/src/github.com/sirupsen/logrus (from $GOROOT)
        /home/user/go/src/github.com/sirupsen/logrus (from $GOPATH)
cmd/main.go:24:2: cannot find package "github.com/urfave/cli" in any of:
        /usr/lib/go-1.10/src/github.com/urfave/cli (from $GOROOT)
        /home/user/go/src/github.com/urfave/cli (from $GOPATH)
        ```

Filtering containers based on label

Add filtering to the list containers output using label key-value e.g.

ce -i /mnt/container list containers --filter "io.kubernets.pod.namespace=mynamespace"

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.