Trainbot watches a piece of train track, detects passing trains, and stitches together images of them. Should work with any video4linux USB cam, or Raspberry Pi camera v3 modules.

Frontend: https://trains.jo-m.ch/

It also contains some packages which might be useful for other purposes:

• pkg/pmatch: Image patch matching
• pkg/ransac: RANSAC algorithm implementation

The binaries are currently built and tested on X86_64 and a Raspberry Pi 4 B.

The computer vision used in trainbot is fairly naive and simple. There is no camera calibration, image stabilization, undistortion, perspective mapping, or "real" object tracking. This allows us to stay away from complex dependencies like OpenCV, and keeps the computational requirements low. All processing happens on CPU.

The assumptions are (there might be more implicit ones):

1. Trains only appear in a (manually) pre-cropped region.
2. The camera is stable and the image does not move around in any direction.
3. There are no large fast brightness changes.
4. Trains have a given min and max speed.
5. We are looking at the tracks more or less perpendicularly in the chosen image crop region.
6. Trains are coming from one direction at a time, crossings are not handled properly.
7. Trains have a constant acceleration (might be 0) and do not stop and turn around while in front of the camera.

There is a helper Makefile which calls the standard Go build tools and an arm64 cross build inside Docker.

# list
ffmpeg -f v4l2 -list_formats all -i /dev/video2
v4l2-ctl --all --device /dev/video2

# exposure
v4l2-ctl -c exposure_auto=3 --device /dev/video2

# autofocus
v4l2-ctl -c focus_auto=1 --device /dev/video2

# fixed
v4l2-ctl -c focus_auto=0 --device /dev/video2
v4l2-ctl -c focus_absolute=0 --device /dev/video2
v4l2-ctl -c focus_absolute=1023 --device /dev/video2

ffplay -f video4linux2 -framerate 30 -video_size 3264x2448 -pixel_format mjpeg /dev/video2
ffplay -f video4linux2 -framerate 30 -video_size 1920x1080 -pixel_format mjpeg /dev/video2

ffmpeg -f v4l2 -framerate 30 -video_size 3264x2448 -pixel_format mjpeg -i /dev/video2 output.avi

# setup
sudo apt-get install libcamera0 libcamera-apps-lite
sudo apt install -y vlc

# grab frame
# https://www.raspberrypi.com/documentation/computers/camera_software.html#libcamera-and-libcamera-apps
libcamera-jpeg -o out.jpg -t 1 --width 4608 --height 2592 --rotation 180 --autofocus-mode=manual --lens-position=2
libcamera-jpeg -o out.jpg -t 1 --width 2304 --height 1296 --rotation 180 --autofocus-mode=manual --lens-position=4.5 --roi 0.25,0.5,0.5,0.5

# record video
DATE=$(date +'%F_%H-%M-%S'); libcamera-vid -o $DATE.h264 --save-pts $DATE.txt --width 1080 --height 720 --rotation 180 --autofocus-mode=manual --lens-position=0 -t 0

# stream through network
libcamera-vid -t 0 --inline --nopreview --width 4608 --height 2592 --rotation 180 --codec mjpeg --framerate 5 --listen -o tcp://0.0.0.0:8080 --autofocus-mode=manual --lens-position=0 --roi 0.25,0.5,0.5,0.5
# on localhost
ffplay http://pi4:8080/video.mjpeg

# manually record video for test cases
libcamera-vid \
   --verbose=1 \
   --timeout=0 \
   --inline \
   --nopreview \
   --width 350 --height 280 \
   --roi 0.368924,0.532407,0.151910,0.216049 \
   --mode=2304:1296:12:P \
   --framerate 30 \
   --autofocus-mode=manual --lens-position=0.000000 \
   --rotation=180 \
   -o vid.h264 --save-pts vid-timestamps.txt

mkvmerge -o test.mkv --timecodes 0:vid-timestamps.txt vid.h264

sudo usermod -a -G video pi

# confighelper
./confighelper-arm64 --log-pretty --input=picam3 --listen-addr=0.0.0.0:8080

The current production deployment is in a Tmux session...

source ./env

while true; do \
./trainbot-arm64 \
   --log-pretty --log-level=debug \
   --input picam3 \
   --camera-format-fourcc=MJPG \
   -X 850 -Y 690 -W 350 -H 280 \
   --px-per-m=42 \
   --enable-upload; \
done

Download latest data from Raspberry Pi:

ssh "$TRAINBOT_DEPLOY_TARGET_SSH_HOST" sqlite3 data/db.sqlite3
.backup data/db.sqlite3.bak
# Ctrl+D
rsync --verbose --archive --rsh=ssh "$TRAINBOT_DEPLOY_TARGET_SSH_HOST:data/" data/
rm data/db.sqlite3-shm data/db.sqlite3-wal
mv data/db.sqlite3.bak data/db.sqlite3

Images and database are uploaded to a web server via FTP. The frontend served as a static HTML/JS bundle from the same server. All database access happens in the browser via sql.js.

• Zerolog is used as logging framework
• "Library" code uses panic(), "application" code use log.Panic()...

• Also create GIFs
• Test in snow/bad weather
• Clean up and document build system
• Calculate length
• Write GIFs again
• Store metadata in db
• Sync pics and db up to bucket
• Static web frontend serving train sightings
• Github button link
• Filter view (longest, fastest, ...)
• Fix stale relative timestamps
• Improve train detail view
• Store filter state in URL
• Show "showing X of Y" counter somewhere
• Auto dark mode
• Nicer dark mode colors
• Logo/Favicon
• Delete data after upload
• Clean up frontend db/blob path handling
• Add favorites feature
• (Maybe) Search in multiple horizontal slices for robustness
• Stats
• Correct for changing exposure, improve stitching seams
• Add machine learning to classify trains (MobileNet, EfficientNet, https://mediapipe-studio.webapps.google.com/demo/image_classifier)
• Create some screenshots
• Better deployment setup, remove hardcoded stuff, document deployment
• Deploy to Raspberry Pi via gokrazy
• Add run/deploy instructions to README (including confighelper)