The Gst-nvstreamdemux plugin demultiplexes batched frames into separate buffers. It creates a separate Gst buffer for each frame in the batch. It does not copy video frames. Each Gst buffer contains a pointer to the corresponding frame in the batch. The plugin pushes unbatched Gst buffer objects downstream of the pad corresponding to each frame source. The plugin obtains this information through the NvDsBatchMeta attached to Gst-nvstreammux. Raw buffer timestamps (PTS) of individual frames are also appended back to the Gst buffer. Since there is no frame copy, the input Gst Buffer will not be returned upstream immediately. When all non-batch Gst Buffer objects decoupled from the input batch Gst Buffer are returned to the decoupler by downstream components, the input batch Gst buffer is returned upstream. Even if Gst-nvstreammux scales the buffer, the demuxer does not scale the buffer back to the original resolution of the source.

input and output

• enter

• Gst buffer (batch)

• NvDsBatchMeta

• Other Meta

• Control parameters

• none

• output

• Gst buffer (non-bulk, single source)

• Meta associated with each Gst buffer source

This plugin can be tested with the following pipeline:

use case 1

• Use nvstreammux to mix two video sources together.

• The output of muxer is configured as nvinfer with batch-size=2.

• After nvinfer, we use nvstreamdemux to display the content of video sources 0 and 1 on two separate windows, and the inferred output for each overlay using the nvdsosd plugin.

Pipeline (x86):

gst-launch-1.0 -e nvstreammux name=mux batch-size=2 width=1920 height=1080 ! nvinfer config-file-path=/opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_infer_primary.txt batch-size=2
! nvstreamdemux name=demux filesrc location=/ opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! queue !
mux.sink_0 filesrc location=/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.h264 ! h264parse ! nvv4l2decoder !
queue ! mux.sink_1 demux.src_0 ! "video/x-raw(memory:NVMM), format=NV12" ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM), format=RGBA" ! nvdsosd !
nvvideoconvert ! nveglglessink demux.src_1 ! queue ! "video/x-raw(memory:NVMM), format=NV12" ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM), format=RGBA" ! nvdsosd ! nveglglessink

Pipeline (Jetson):

gst-launch-1.0 -e nvstreammux name=mux batch-size=2 width=1920 height=1080 ! nvinfer config-file-path=/opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_infer_primary.txt batch-size=2  !
nvstreamdemux name=demux filesrc location=/opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! queue !
mux.sink_0 filesrc location=/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.h264 ! h264parse ! nvv4l2decoder ! queue ! mux.sink_1 demux.src_0 ! "video/x-raw(memory:NVMM), format=NV12" !
queue ! nvvideoconvert ! "video/x-raw(memory:NVMM), format=RGBA" ! nvdsosd ! nvvideoconvert ! nv3dsink demux.src_1 ! queue ! "video/x-raw(memory:NVMM), format=NV12" ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM), format=RGBA" ! nv3dsink

use case 2

• Use nvstreammux to mix two video sources together.

• The output of muxer is configured as nvinfer with batch-size=2.

• After nvinfer, we use nvstreamdemux to write the content of video source 0 to a file along with the inferred output overwritten using the nvdsosd plugin.

• The content of the demux after video source 1 is displayed directly on the screen using the nveglglessink plugin

Pipeline (x86):

gst-launch-1.0 -e nvstreammux name=mux batch-size=2 width=1920 height=1080 ! nvinfer config-file-path=/opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_infer_primary.txt batch-size=2  ! nvstreamdemux name=demux filesrc location=/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.h264 ! h264parse ! nvv4l2decoder ! queue ! mux.sink_0 filesrc location=/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.h264 ! h264parse ! nvv4l2decoder ! queue ! mux.sink_1 demux.src_0 ! "video/x-raw(memory:NVMM), format=NV12" ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM), format=RGBA" ! nvdsosd ! nvvideoconvert ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=./out.mp4 demux.src_1 ! queue ! nveglglessink

Pipeline (Jetson):

gst-launch-1.0 -e nvstreammux name=mux batch-size=2 width=1920 height=1080 ! nvinfer config-file-path=/opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_infer_primary.txt batch-size=2  ! nvstreamdemux name=demux filesrc location=/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.h264 ! h264parse ! nvv4l2decoder ! queue ! mux.sink_0 filesrc location=/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.h264 ! h264parse ! nvv4l2decoder ! queue ! mux.sink_1 demux.src_0 ! "video/x-raw(memory:NVMM), format=NV12" ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM), format=RGBA" ! nvdsosd ! nvvideoconvert ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=./out.mp4 demux.src_1 ! queue ! nv3dsink

use case 3

Use case 3 demonstrates displaying two streams in two separate windows.

Use nvstreammux to mix two video sources together.

The output of muxer is configured as nvinfer with batch-size=2.

After nvinfer, we use nvstreamdemux to display the content of video sources 0 and 1 on two separate windows.

Pipeline (x86):

gst-launch-1.0 -e nvstreammux name=mux batch-size=2 width=1920 height=1080 ! nvinfer config-file-path=/opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_infer_primary.txt batch-size=2  ! nvstreamdemux name=demux filesrc location=/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.h264 ! h264parse ! nvv4l2decoder ! queue ! mux.sink_0 filesrc location=/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.h264 ! h264parse ! nvv4l2decoder ! queue ! mux.sink_1 demux.src_0 ! queue ! nvvideoconvert ! nveglglessink demux.src_1 ! queue ! nveglglessink

Pipeline (Jetson):

gst-launch-1.0 -e nvstreammux name=mux batch-size=2 width=1920 height=1080 ! nvinfer config-file-path=/opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_infer_primary.txt batch-size=2  ! nvstreamdemux name=demux filesrc location=/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.h264 ! h264parse ! nvv4l2decoder ! queue ! mux.sink_0 filesrc location=/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.h264 ! h264parse ! nvv4l2decoder ! queue ! mux.sink_1 demux.src_0 ! queue ! nv3dsink demux.src_1 ! queue ! nv3dsink