H.264 support about turbovnc HOT 10 OPEN

I'm currently working, on a somewhat low priority, on adding NVENC support to the TurboVNC Benchmark Tools so I can get an idea of how the "naive" approach-- simply invoking NVENC from within TurboVNC on the entire framebuffer-- might perform. Working with NVENC is somewhat difficult, given that the convenience classes for it aren't open source. Thus, it's necessary to access the low-level functions directly. I haven't been able to return to that effort in recent months due to other pressing concerns, but I hope to get back to it before the end of the year.

If the naive approach seems like it's worth pursuing, then it may be possible to get that into TurboVNC 2.2. If the naive approach isn't worth pursuing, then in all likelihood, this will get punted until a TurboVNC Wayland compositor is developed, unless an organization steps up to fund the necessary architectural modifications to VGL and TurboVNC (likely to be a rather large and expensive effort.)

from turbovnc.

After discussing with nVidia, it seems that Proposal 3 above is possible, but PBO handles cannot be passed between processes. Assuming that VirtualGL is creating the PBO and shipping it to TurboVNC for compression, it will be necessary for VirtualGL to create a CUDA pointer from the PBO, create a CUDA IPC handle from the CUDA pointer, and ship the CUDA IPC handle to the X server (via some as-yet-to-be-defined X extension, probably one that is not "official" but just used within VirtualGL and TurboVNC.) TurboVNC would then invoke NvENC on the memory region pointed to by the CUDA IPC handle.

from turbovnc.

I think H.264 would be a very valuable feature. Using Nvidia's GameStream or whatever it's called I was able to stream my desktop to my Nvidia shield tablet with virtually no delay.
Before that I tried Windows Remote Destop, TeamViewer and tons of VNC clients and they all introdued significantly higher latencies which made steaming fps games and such completely unenjoyable. Audio/video sync was also a major issue for me because my speakers were still hooked up directly to my PC. With Nvidia GameStream all these problems went away.

I would imagine performance-wise this would only make sense if you have the hardware encoders/decoders on both ends, but I think Nvidia has started added it to all their GPUs starting 4 years ago and AMD apparently has their own version now.

It would be really amazing to be able to do that on Linux as well.

Btw have you made any progress?

from turbovnc.

Unfortunately not much. As an independent open source developer who makes money only through support and funded development contracts, I’m constantly having to formulate strategies for the product that will maintain its high quality and usability standards and create meaningful improvements for users while still driving business my way in the form of funded development of new features. As such, I have to be careful not to give away so much free milk that no one wants to buy the cow. I committed long ago to do the initial analysis on the H.264 project using General Fund money, but I haven’t managed to find the time to get back to it. Unfortunately some corporate players have, in recent years, started aggressively trying to take customers away from me by cutting into the market space that I busted my butt to build, so now I’m busting my butt yet again to reposition TurboVNC 3.0 so it can compete with these corporate players. I hope to get back to more long-term pursuits like this and Wayland support, etc., once I’m out of survival mode.

from turbovnc.

Also, you mention the latency problems with other VNC solutions, but TurboVNC has specifically implemented RFB extensions that should improve that situation. Have you tried TurboVNC as it exists today?

from turbovnc.

I don't think I've tried TurboVNC yet. Back then, I was on Windows and now I'm on Fedora... which means I'm on Wayland. :/

from turbovnc.

It means you're on Wayland for the root display, unless you configure it otherwise (Fedora still supports the traditional mode, just not by default), but that doesn't prevent you from using an X proxy such as TurboVNC, since it is not dependent on the root display.

from turbovnc.

Hi. I am working on a Raspberry based hardware IP KVM project and I have written my own VNC server implementing JPEG compression. Now I implement H264 and have already made a server for video encoding with low latency, it remains only to wrap the H264 frames to VNC protocol. What is the current situation with H264 in the TurboVNC client side? I would be happy if this client could support H264 with my project.

from turbovnc.

@mdevaev H.264 has not been implemented yet in TurboVNC and won't make it into the upcoming 3.0 release. Which specification did you use for RFB-encoding the H.264 framebuffer updates? I wasn't able to find a definitive specification for an H.264 RFB encoding type.

from turbovnc.

At the moment, none. It seems that we (if you decide to support it) can use whatever we want. I would start from the registered encoding with code 20. But maybe it would be great to have some kind of service pseudo-encodings to regulate the bitrate like for the Tight JPEG.

Before implementing the support myself, I decided to investigate the issue and ask you about your plans.

from turbovnc.