Skip to content

Instantly share code, notes, and snippets.

@awesomebytes

awesomebytes/gstreamer_gscam_tests.md

Created January 19, 2019 11:10
Show Gist options
  • Save awesomebytes/1ee65b9a1da2c15f010dfe64959c4199 to your computer and use it in GitHub Desktop.
Save awesomebytes/1ee65b9a1da2c15f010dfe64959c4199 to your computer and use it in GitHub Desktop.
Download ZIP
Gstreamer and gscam tests on Pepper
Raw
gstreamer_gscam_tests.md

Gstreamer tests

Offer video streams to mahcine 192.168.1.18.

Needs AutonomousLife disabled on boot to access top camera (video).

From Paco Dupont post:

gst-launch-0.10 -v v4l2src device=/dev/video0 ! 'video/x-raw-yuv,width=640, height=480,framerate=30/1' ! ffmpegcolorspace ! jpegenc ! rtpjpegpay ! udpsink host=192.168.1.18 port=3000


gst-launch-0.10 -v v4l2src device=/dev/video1 ! 'video/x-raw-yuv,width=640, height=480,framerate=30/1' ! ffmpegcolorspace ! jpegenc ! rtpjpegpay ! udpsink host=192.168.1.18 port=3001

On the client:

gst-launch-0.10 -v udpsrc port=3001 ! application/x-rtp, encoding-name=JPEG,payload=26 ! rtpjpegdepay ! jpegdec ! autovideosink

Moving to gstreamer 1.10:

WARNING: erroneous pipeline: no element "ffmpegcolorspace"

ffmpegcolorspace became videoconvert.

WARNING: erroneous pipeline: no element "jpegenc"

Wait how did we compile gst-plugin-goods?

/data/home/nao/gentoo/var/tmp/portage/media-libs/gst-plugins-good-1.14.4/work/gst-plugins-good-1.14.4/configure --prefix=/tmp/gentoo/usr --build=i686-pc-linux-gnu --host=i686-pc-linux-gnu --mandir=/tmp/gentoo/usr/share/man --infodir=/tmp/gentoo/usr/share/info --datadir=/tmp/gentoo/usr/share --sysconfdir=/tmp/gentoo/etc --localstatedir=/tmp/gentoo/var/lib --disable-dependency-tracking --disable-silent-rules --docdir=/tmp/gentoo/usr/share/doc/gst-plugins-good-1.14.4 --htmldir=/tmp/gentoo/usr/share/doc/gst-plugins-good-1.14.4/html --libdir=/tmp/gentoo/usr/lib --with-package-name=Gentoo GStreamer ebuild --with-package-origin=https://www.gentoo.org --disable-directsound --disable-waveform --disable-oss --disable-oss4 --disable-osx_audio --disable-osx_video --disable-gst_v4l2 --disable-x --disable-aalib --disable-cairo --disable-flac --disable-gdk_pixbuf --disable-gtk3 --disable-jack --disable-jpeg --disable-lame --disable-libcaca --disable-libdv --disable-libpng --disable-mpg123 --disable-pulse --disable-dv1394 --disable-qt --disable-shout2 --disable-soup --disable-speex --disable-taglib --disable-twolame --disable-vpx --disable-wavpack --disable-zlib --disable-bz2 --enable-orc --disable-maintainer-mode --enable-nls --enable-bz2 --enable-zlib --disable-examples --with-default-audiosink=autoaudiosink --with-default-visualizer=goom

Oh damn, not good. I could try adding EXTRA_ECONF flags, but... This got ugly.

Found how it works, in Gentoo the jpeg plugin and friends are separated in other packages.

Made a binhost to make things faster:

# build computer (12min)
time quickpkg "*/*"

# copy to binserver
scp -r * [email protected]:/volume1/web/pepper_os/2019/packages

# add to client /tmp/gentoo/etc/portage/make.conf
EMERGE_DEFAULT_OPTS="--jobs 4 --load-average 2"
PORTAGE_BINHOST="http://192.168.1.8/pepper_os/2019/packages"
# Can also add 
FEATURES="getbinpkg"

# on installs
--getbinpkgonly (-G)
gst-launch-1.0 -v v4l2src device=/dev/video0 ! 'video/x-raw,width=640, height=480,framerate=30/1' ! videoconvert ! jpegenc ! rtpjpegpay ! udpsink host=192.168.1.72 port=3000


gst-launch-1.0 -v v4l2src device=/dev/video1 ! 'video/x-raw,width=640, height=480,framerate=30/1' ! videoconvert ! jpegenc ! rtpjpegpay ! udpsink host=192.168.1.18 port=3001


# bigger resolution didnt work... neither modifying fps
gst-launch-1.0 -v v4l2src device=/dev/video0 ! 'video/x-raw,width=1280, height=960,framerate=30/1' ! videoconvert ! jpegenc ! rtpjpegpay ! udpsink host=192.168.1.72 port=3000

x264 didnt work for me. Crazy cpu usage anyways (600%)

To check fps transmitted thru the network:

sudo apt-get install gstreamer1.0-plugins-bad

gst-launch-1.0 -v udpsrc port=3000 ! application/x-rtp, encoding-name=JPEG,payload=26 ! rtpjpegdepay ! jpegdec ! fpsdisplaysink

Nice document with a lot of gstreamer examples: https://github.com/intel/gstreamer-media-SDK/blob/master/README.USAGE

Short datasheet of the top and bottom cameras: https://media.digikey.com/pdf/Data%20Sheets/OmniVision%20PDFs/OV5640_PB_3-4-11.pdf

Pepper official doc about cameras: http://doc.aldebaran.com/2-5/family/pepper_technical/video_2D_pep.html

What does the system itself say?

v4l2-ctl --device=/dev/video0 --all
Driver Info (not using libv4l2):
    Driver name   : unicorn
    Card type     : unicorn
    Bus info      : PCIe:0000:01:00.0
    Driver version: 0.0.106
    Capabilities  : 0x85200001
        Video Capture
        Read/Write
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps   : 0x05200001
        Video Capture
        Read/Write
        Streaming
        Extended Pix Format
Priority: 0
Video input : 0 (input 0: ok)
Video Standard = 0x00000000
Format Video Capture:
    Width/Height  : 320/240
    Pixel Format  : 'YUYV'
    Field         : None
    Bytes per Line: 640
    Size Image    : 153600
    Colorspace    : Unknown (00000000)
    Flags         : 
Streaming Parameters Video Capture:
    Frames per second: invalid (0/1)
    Read buffers     : 0
                     brightness (int)    : min=0 max=255 step=1 default=0 value=0 flags=slider
                       contrast (int)    : min=0 max=255 step=1 default=32 value=32 flags=slider, volatile
                     saturation (int)    : min=0 max=255 step=1 default=64 value=64 flags=slider
                            hue (int)    : min=-180 max=180 step=1 default=0 value=0 flags=slider
        white_balance_automatic (bool)   : default=1 value=1
               do_white_balance (int)    : min=0 max=4096 step=1 default=2048 value=1024 flags=read-only, volatile
                    red_balance (int)    : min=0 max=4096 step=1 default=2048 value=1024 flags=read-only, slider, volatile
                   blue_balance (int)    : min=0 max=4096 step=1 default=2048 value=1024 flags=read-only, slider, volatile
                       exposure (int)    : min=0 max=65535 step=1 default=0 value=980 flags=inactive, volatile
                 gain_automatic (bool)   : default=1 value=1
                           gain (int)    : min=0 max=1024 step=1 default=32 value=248 flags=volatile
                horizontal_flip (bool)   : default=0 value=0
                  vertical_flip (bool)   : default=0 value=1
               background_color (int)    : min=0 max=16777215 step=1 default=0 value=101 flags=read-only, volatile
v4l2-ctl --device=/dev/video1 --all
Driver Info (not using libv4l2):
    Driver name   : unicorn
    Card type     : unicorn
    Bus info      : PCIe:0000:01:00.0
    Driver version: 0.0.106
    Capabilities  : 0x85200001
        Video Capture
        Read/Write
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps   : 0x05200001
        Video Capture
        Read/Write
        Streaming
        Extended Pix Format
Priority: 0
Video input : 1 (input 1: ok)
Video Standard = 0x00000000
Format Video Capture:
    Width/Height  : 320/240
    Pixel Format  : 'YUYV'
    Field         : None
    Bytes per Line: 640
    Size Image    : 153600
    Colorspace    : Unknown (00000000)
    Flags         : 
Streaming Parameters Video Capture:
    Frames per second: invalid (0/1)
    Read buffers     : 0
                     brightness (int)    : min=0 max=255 step=1 default=0 value=0 flags=slider
                       contrast (int)    : min=0 max=255 step=1 default=32 value=32 flags=slider, volatile
                     saturation (int)    : min=0 max=255 step=1 default=64 value=64 flags=slider
                            hue (int)    : min=-180 max=180 step=1 default=0 value=0 flags=slider
        white_balance_automatic (bool)   : default=1 value=1
               do_white_balance (int)    : min=0 max=4096 step=1 default=2048 value=1024 flags=read-only, volatile
                    red_balance (int)    : min=0 max=4096 step=1 default=2048 value=1024 flags=read-only, slider, volatile
                   blue_balance (int)    : min=0 max=4096 step=1 default=2048 value=1024 flags=read-only, slider, volatile
                       exposure (int)    : min=0 max=65535 step=1 default=0 value=980 flags=inactive, volatile
                 gain_automatic (bool)   : default=1 value=1
                           gain (int)    : min=0 max=1024 step=1 default=32 value=32 flags=volatile
                horizontal_flip (bool)   : default=0 value=1
                  vertical_flip (bool)   : default=0 value=0
               background_color (int)    : min=0 max=16777215 step=1 default=0 value=0 flags=read-only, volatile

After setting another resolution, the line Width/Height changes to the last resolution set.

Also v4l2-ctl --list-formats-ext should give the available framerates and resolutions, but it doesnt.

Playing with the FPS doesn't get me any difference.

<launch>
  <!-- This launchfile should bring up a node that broadcasts a ros image
       transport on /webcam/image_raw -->

  <arg name="DEVICE" default="/dev/video0"/>
  <!-- The GStreamer framerate needs to be an integral fraction -->
  <arg name="FPS" default="30/1"/>
  <arg name="PUBLISH_FRAME" default="false"/>
  <arg name="GST10" default="true"/>

  <node ns="v4l" name="gscam_driver_v4l" pkg="gscam" type="gscam" output="screen">
    <param name="camera_name" value="default"/>
    <param name="camera_info_url" value="package://gscam/examples/uncalibrated_parameters.ini"/>
    <param unless="$(arg GST10)" name="gscam_config" value="v4l2src device=$(arg DEVICE) ! video/x-raw-rgb,framerate=$(arg FPS) ! ffmpegcolorspace"/>
    <param if="$(arg GST10)" name="gscam_config" value="v4l2src device=$(arg DEVICE) ! video/x-raw,width=640,height=480,framerate=30/1 ! videoconvert"/>
    <param name="frame_id" value="/v4l_frame"/>
    <param name="sync_sink" value="true"/>
  </node>

  <node if="$(arg PUBLISH_FRAME)" name="v4l_transform" pkg="tf" type="static_transform_publisher" args="1 2 3 0 -3.141 0 /world /v4l_frame 10"/>
</launch>

gscam 1 camera 40% CPU 24fps. doing nothing, 45% with a subscriber. naoqi-service 40% CPU (is doing nothing)

Old driver (note that in theory it can provide up to 30fps but I don't really believe it): naoqi-driver 11% CPU (doing nothing is 5%) naoqi-service 55% CPU (15+11=26%cpu)

With both cameras, top and bottom

naoqi-driver 18% naoqi-service 67% (18+27=45%cpu)

CONCLUSION: NOT VIABLE TO USE GSCAM for video Better to stick to naoqi-driver, or maybe have a lightweight version of it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment