• Setup
• Usage
• Render a local file on a mesh.
• Render camera record on a mesh.

• To use this plugin first Clone this repository in the Plugins/Simbotic directory of your Engine:

git clone [email protected]:Simbotic/Streamer.git

• After clone the repository, recompile the engine using make .

• To use the plugin on a project, update your .uproject adding the UE4 Gstreamer Plugin as an additional dependency on .uproject:

...,
"Modules": [
	{
		"Name": "GST_Test",
		"Type": "Runtime",
		"LoadingPhase": "Default",
		"AdditionalDependencies": [
			"Engine"
		]
	}
],
"Plugins": [
	{
		"Name": "GStreamer",
		"Enabled": true
	},
],
...

• Add, in the constructors of the classes of your project: Source/PROJECT_NAME.Target.cs and Source/PROJECT_NAMEEditor.Target.cs the following:

bUseUnityBuild = false;
bUsePCHFiles = false;

On Blueprints the Blueprint hierarchy you need:

• GstPipeline.
• 2 GstAppSrc these components will be named as GstAppSrcRGB and GstAppSrcDepth.
• 2 scene Capture Component 2D these components will be named as SceneCaptureRGB and SceneCaptureDepth.

• Pipeline Name will be set as robot.

• Pipeline config will set as:

  appsrc name=sensor_rgb caps=video/x-raw,width=512,height=512,format=BGRx, framerate=10/1 ! videoconvert ! queue2 ! fpsdisplaysink

• Pipeline Config File will be empty.

• Pipeline Use File will be unchecked.

• Pipeline Autostart will be checked.

The components GstAppSrc are named as GstAppSrcRGB and GstAppSrcDepth these components are related with Scene Capture Components.

• Pipeline Name will be set as robot.
• App Src Name it will be set as sensor_rgb and sensor_depth respectively.
• App Src Enabled will be set as checked.
• Src Video it will be set as SceneCaptureRGB and SceneCaptureDepth.
• Src Klv Will Set as empty.

on Scene Capture component go to Post Process Volume/Rendering/Features/Post process Materials/Array.

create a new array, on choose select asset reference, then add the respectively Material M_SensorBGRA and M_SensorDepth.

Modify tab Scene Capture.

• On Primitive Render Mode select Render Scene Primitives .
• On Capture Source select Final Color (LDR) in RGB .

Modify tab Scene Capture.

• On Capture Source select Final Color (LDR) in RGB .

We will build a BP where we will add the following components.

• GstPipeline.
• GstAppSink.
• Plane (mesh).

We need to add the name and the config to the pipeline.

robot

filesrc location=/Absolute/Path/To/Your/File.mp4 ! decodebin ! videoconvert ! video/x-raw,format=(string)RGBA ! videoconvert ! appsink name=sink

We need to add the pipeline name that we defined before robot and the appsink that is set at the end of the pipeline config, it's called sink.

We need to add a variable, it type will be Material Interface, we will name it as Material.

We will drag the GstAppSink component to the event Graph, from it pin we will generate a Get Texture Module, then using the variable we created before, generate a module type Create Dynamic Material Instance, Finally drag the Plane from BP components that we created at the begin, we will set a material on the plane.

Create a material (we will name it as (GstRender), inside of it, create a Texture Sample plug in the Base color, then add a Texture Object an convert it on a parameter, connect this parameter to the Texture input of the Texture Sample, Create A Texture Coordinate and plug in the UVs input, finally, create a constant it's value will be 0, plug in metallic and specular inputs.

Return to the BP that we create at the begin, and set Variable Material and the plane material, with the material we created.

We will combine the configurations that we set on Render a local file on a mesh., and gstappsrc, and the biggest change will be set on the Pipeline Config.

On our BP we will add a gstappsrc (use the configuration of gstappsrc), add 2 sceneCapture, and name then as SceneCaptureRGB and SceneCaptureDepth (configuration for the SceneCapture is the same that we set on SceneCapture).

Our pipeline config will looks like this.

appsrc name=sensor_rgb caps=video/x-raw,width=512,height=512,format=BGRx,framerate=10/1 ! videoconvert ! queue2! videoconvert ! video/x-raw,format=(string)RGBA ! videoconvert ! appsink name=sink