360(2 pi) panoramic photo camera kit.

It uses simplest panoramic camera model, which including 2 fisheye camera (fov > 180), and computing unite to generate panoramic image from two circular images. This kit uses one Raspberry Pi W (RPi0W) and one Raspberry Pi Zero (RPi0). The RPi0 acts as USB gadget, ready to recieve the commands from RPi0W. The RPi0W performs as the master pi, which hosts the HTML Web interface, requests image from slave Pi and generates panoramic image from two circular images. The kit offered the USB and power connection between two Pis, so no other usb cables needed.

This project hacked the RPi-Web-Cam-Interface a bit. The mjpeg was used to show the circular images on one html page (panoindex.php, cam_slave_pic.php and cam_slave_pic_new.php) and take the photo via html request.

The panoramic generation algorithm is not very straightforward, here is the link which introdcued how to map a circular image into panoramic image. This paper also introduced the several APIs, however, I wrote this algorithm in python by myself just for better understanding. The 360 panoramic image was generated by adding two 180 pano image side by side and use a filter to smooth the boundray. In a short word, the algorithm mapped the circular image into sphere (3D x,y,z) and then mapped the sphere into panoramic image. For the better understanding, the world map is an excellent example, it maps the earth (3D sphere in the space) into 2D world map, and this map is panomaric image.

In this section, the dual RPis camera system will be set up. Dual RPis system communicated via USB Ethernet.

The following steps will show how to setup the slave pi (RPi0) and master pi (RPi0W).

1. Follow the instructions to set up the USBGadget. The full instruction can be found in here. In short, here is the steps copied from that instruction.

   For this method, alongside your Pi Zero, MicroUSB cable and MicroSD card, only an additional computer is required, which can be running Windows (with Bonjour, iTunes or Quicktime installed), Mac OS or Linux (with Avahi Daemon installed, for example Ubuntu has it built in).

   • Flash Raspbian full or Raspbian Lite (recommended) onto the SD card

   • Once Raspbian is flashed, open up the boot partition (in Windows Explorer, Finder etc) and add to the bottom of the config.txt file dtoverlay=dwc2 on a new line, then save the file.

   • If using a recent release of raspbian (Dec 2016 onwards), then create a new file simply called ssh in the SD card as well. By default SSH is now disabled so this is required to enable it. Remember - Make sure your file doesn't have an extension (like .txt etc)!

   • Finally, open up the cmdline.txt. Be careful with this file, it is very picky with its formatting! Each parameter is seperated by a single space (it does not use newlines). Insert modules-load=dwc2,g_ether after rootwait. To compare, an edited version of the cmdline.txt file at the time of writing, can be found here.

   • That's it, eject the SD card from your computer, put it in your Raspberry Pi Zero and connect it via USB to your computer. It will take up to 90s to boot up (shorter on subsequent boots). It should then appear as a USB Ethernet device. You can SSH into it using raspberrypi.local as the address.

2. If you used Lite OS, there were several packages should be installed before the next step. As follows

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install git
sudo apt-get install apache2
sudo apt-get install php

After these steps, enable **Camera Interface** by _sudo raspi-config_, and then reboot the system.

3. Install the rpi cam web interface, better to follow the instructions here. When you install the rpi web cam interface, there will be a step for you to choose some configurations. One of them is to set up the install directory. The default directory is html, change it to picam. The benefit is that the default could be used for other projects. And I hard coded the image directory to picam. So for time saving sake, change html to picam.

• TP: connect the slave pi to the PC, open a browser, input raspberrypi.local/picam if shown the live stream, it means it works.

4. Change some image settings, such as resolutions to Max View, turn off the exposure mode and white ballance, rotate the image if necessary.

5. Link the mjpeg to the picam. Log in to the slave pi and direct to /var/www/picam/ and input the following command

sudo ln -s /dev/shm/mjpeg/cam.jpg cam.jpg

Back to /var/www and the run the following to change the permission.

sudo chown -R www-data:www-data picam

The slave pi is setup and ready to use.

1. Prepare the Micro SD card. The first step is similar to the slave pi.

   • Flash Raspbian full or Raspbian Lite (recommended) onto the SD card

   • If using a recent release of raspbian (Dec 2016 onwards), then create a new file simply called ssh in the SD card as well. By default SSH is now disabled so this is required to enable it. Remember - Make sure your file doesn't have an extension (like .txt etc)!

Still in the boot partition, add the file wpa_supplicant.conf and add the following lines and replace SSID and PASSWORD with the ones for your network:

    ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
    update_config=1
    network={
        ssid="SSID"
        psk="PASSWORD"
    }

2. Setup USBNet

Boot the master RPi and add the following line at the end of the file /etc/network/interfaces

    auto lo
    iface lo inet loopback

    iface eth0 inet manual

    allow-hotplug wlan0
    iface wlan0 inet manual
        wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf

Reboot the mater pi.

3. Install the RPi Web Cam Interface. The steps is as same as the slave pi. If you used Lite OS, there were several packages should be installed before the next step. As follows

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install git
sudo apt-get install apache2
sudo apt-get install php

After these steps, enable **Camera Interface** by _sudo raspi-config_, and then reboot the system. 

Install the rpi cam web interface, better to follow the instructions [here](https://elinux.org/RPi-Cam-Web-Interface). When you install the rpi web cam interface, there will be a step for you to choose some configurations. One of them is to set up the install directory. The default directory is _html_, change it to _picam_. The benefit is that the default could be used for other projects. And I hard coded the image directory to _picam_. So for time saving sake, change **html** to **picam**. 

* _**TP**: Power up RPi0W and then wait for about 1 min, open a browser, check the IP and input the http://rpi-ip/picam. If shown the live stream, it worded._

1. Follow the order of install.list to install the needed packages in RPi0W. In short

sudo apt-get install python-opencv
sudo apt-get install libopencv-dev
sudo apt-get install python-pip
sudo apt-get install python-numpy
sudo apt-get install libjpeg8-dev
sudo pip install pillow
sudo pip install requests
sudo pip install pathlib

2. Clone this git and test

• git clone https://github.com/rpicamera/360pano.git

• TP: Test the regression code. Run the following line. If no error message and it generates the pano image named regression.png as follows, it worked.

python regression.py

1. Introduction of 2PiView kit

• 2PiView main frame offered the build in connection of USB and Power shareing. The pin is used to do the physical connection. The following shows the frame. Also it is used for the camera stand.

• The outside stand protect the frame and let it stand on the surface. The set up overlook is as follows.

2. Setup steps

Please watch this video.

3. Test the connection

• connect the power to the RPi0 (slave pi), and connect to RPi0W using ssh

• ping raspberrypi.local, if the ip address was start from 169. then it works

1. copy files

In this step, the root access is needed (sudo ). Here assume the RPi-Web-Cam-Interface install directory is picam.

• go to ~/360pano/picam and run

  sudo sh ./filecopy.sh
  

• go to /var/www/picam create the directory named img and the directory names imghelp

• copy the world map if you would like to, run:

  sudo cp ~/360pano/img/panowm.jpg /var/www/picam/img/panowm.jpg
  

• go to /var/www and run the command:

      sudo chown -R www-data:www-data picam
  

• go to directory and test the python setting by

      sudo python genpano.py
  

wait for it finished, go to img and check is there a new file. If passed the test, it means it worked.

2. Initial the system

Because each fisheye camera module may not be the same. So the circular image may not located on the default place. In this step, using html interface, the configure file will be generated and saved to the local.

• Open a browser and url: _ip/picam/crophelper.php and click the crop button. The program will shoot two photos and show them on the page.

• Input the value in the text box to move the red circle such that the circule could match the boundary of the cirular image.

• Click save button to save the configuration

3. Try the whole system

After initial the system, it should generate the pano image. Next step is to generate the 3 maps from the setting by

```
    sudo python xymapbuilder.py
```

This will generate the maps for the remapping, make the code run faster. The following is the pages for taking and download the pano photos.

• The address for taking photos is : ip/picam/panoindex.php

• The address for download photos is: ip/picam/downloadpano.php

In download page, it gives the option to preview the image with VR. So if you had a VR head set and a Smart Phone, you could try with it. The default pano image I put here is the world map.

• The address for preview photos is: ip/picam/previewpano.php

Create the pano from the my 2PiView kit using Python code, detail in genpano.py. And here is the example

This is the view from the phone.

Here is my 2PiView kit: