curtiswest / pepi Goto Github PK

With the addition of multi-camera servers in v3.0 comes the ability to stream from these multi-camera servers.

The present implementation of the user interface only displays one of these streams (the first in the streams_url reply from the server).

Ideally, there would be a way to display all of the cameras connected to a server in that servers panel. This shouldn't require any backend work -- after all, the stream URLs are already being accessed, but they're discarded before being used. Implementation will require modifying the Flask templates setup.html to display these streams.

It seems there is some issue in capturing to an RGB array while PiCamera is also recording from the video port. This seems to stem from the use_video_port= parameter or from using the resize= parameter, causing an ENOMEM error when attempting to get a RGB capture.

Capturing to YUV and converting to RGB works perfectly (but is considerably slower than capturing to RGB directly), so capturing & recording simultaneously is clearly not an issue.

The PiCamera documentation mentions some issues about RGB captures that cause a memory error:

RGB captures from the still port do not work at the full resolution of the camera (they result in an out of memory error).
Either use YUV captures, or capture from the video port if you require full resolution.

However, it seems this bug still happens even when capturing from the video port as suggested, hence the possibility that the resizer component added when using resize= throws a wrench in the mix somewhere?

The MJPEG stream implementation provided in the Server package occasionally refuses to display an image. It appears that this does not stem from any issue with the camera, as capturing a full-res image still works.

Pressing 'stop' in the browser displaying the stream tends to cause the last frame to appear, but of course stops the stream. Refreshing after this does not restart the stream. This means that it probably has something to do with how the images are streamed over the HTTP response.

The existing design of PEPI as of v3.0 has the requirement that servers and cameras (in Python, MetaServer and MetaCamera) be implemented in the same language. This requirement stems from the fact that the camera object's still() method is called directly from the server.

This issue proposes that the MetaCamera object be replaced with an Thrift service defined in pepi.thrift.

Benefits:

• Adding a Thrift server for a camera would allow the still() method to be called over Thrift, which would inherently provide cross-language support.
• It is consistent with the existing design of abstracting away components of the system from the caller
• A server could support cameras that were available over the network.
• This would provide a single source of truth for what a camera is, rather than only providing a Python object implementation.
• Further to the above point, it would mean a single test suite for contracts could be implemented in Python by accessing objects from other languages over Thrift. If the wrong values are received in Python, then we can infer that the implementation is incorrect in that language.

Drawbacks:

• The distinction between a Camera and a CameraServer is muddied as they both act as servers.
• This could introduce unacceptable latency/overhead to the system for synchronised captures.
• Server's need to know whether they're dealing with a local or remote camera, and therefore where that camera is accessible at. Accessing all over Apache Thrift may be an OK design, but breaks existing compatibility with 3.0, so perhaps there will be a way to pass different types of cameras to a server.

The current implementation of PEPI does not support camera configuration.

If multiple cameras are used, it may be advantageous to be able to configure their parameters, e.g. white balance, exposure, ISO, etc. to achieve a consistent capture.

This enhancement proposes an extension of the CameraServer defined in pepi.thrift called ConfigurableCameraServer. This would take the form of:

service ConfigurableCameraServer extends CameraServer {
    i32 iso(1:i32 value),
    i32 white_balance_temp(1:i32 value),
    float shutter_speed(1:float value)
}

There will need to be some thoughtful consideration put into deciding which operations to include in this new extension. Additionally, will need to look into some mechanism whereby server access will fallback to a normal CameraServer if ConfigurableCameraServer is not implemented. Such a mechanism may already be available in Thrift?

The user interface will need to be updated to accept these configuration functions.