Jerks in HLS when coverted from UDP. about x9k3 HOT 40 CLOSED

from x9k3.

I really need the stream if you want me to help you. That's just not enough information for me to diagnose your issue. A couple things jumped out at me though,

• your segment size is over 7 seconds, I strongly suggest 2 second segments. If you did not set the seconds to 7, that means your gop size is very large. I like iframes every second, I would have them at least every 2 seconds. I would start with this.
• What is your datagram size on the UDP stream? It needs to be 1316.
• crank up your udp receive buffer max with sysctl like this
  sysctl -w net.core.rmem_max = 26214400

from x9k3.

I did a bunch of UDP tuning in the new_reader package back in February, that's what x9k3 uses to read UDP. Update new_reader, that will probably help a lot. Sorry, I just remember that.

pypy3 -mpip install --upgrade new_reader
or
python3 -mpip install --upgrade new_reader

A guy was using really high bitrate video over UDP and we spent a good bit of time tuning up new_reader specifically for UDP.
Try this first.

from x9k3.

I just tested it over UDP and I get perfect playback, and my bitrate is a bit higher.

Input #0, mpegts, from 'seg3.ts':
  Duration: 00:00:02.00, start: 93541.004622, bitrate: 2582 kb/s
  Program 1 
  Stream #0:0[0x1e1]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, progressive), 960x540 [SAR 1:1 DAR 16:9], 29.97 fps, 29.97 tbr, 90k tbn
  Stream #0:1[0x1e2](und): Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 97 kb/s
  Stream #0:2[0x1e3]: Data: scte_35
  Stream #0:3[0x1e4]: Data: timed_id3 (ID3  / 0x20334449)

Do the stuff I mentioned above and let me know what happens.

from x9k3.

Talk to me Goose, let me know what's happening. We'll figure it out.

from x9k3.

Talk to me Goose, let me know what's happening. We'll figure it out.

Hey Buddy,
Thanks
"crank up your udp receive buffer max with sysctl like this
sysctl -w net.core.rmem_max = 26214400"

this point helped to play smooth stream. but running for long time(approx more than 24hrs) creates the same issue again .

from x9k3.

Hi mate,
Thanks for your help and support.

I am writing on behalf of gaurav.

After per your suggestion, we have reduced the chunk duration from 6 to 2 secs and set udp receive buffer max with sysctl as below :
sysctl -w net.core.rmem_max = 26214400

Post that HLS(.m3u8) stream is running fine but running for long duration (approx more than 24hrs) creates the same issue again. Is it because it exceeds the udp rx buffer size..just suspecting ?

Few Asks:

• How can we get the info that IDR frame are inserted every 2 secs during UDP to HLS translation ?
• How the GOP calaculation is done and how to check what is the GOP structure and GOP Length currently as part of .ts post translation ?
• How GPU acceleration can be added to this workflow ?
• How DRM can be integrated with X9K3 workflow ?

from x9k3.

You can detect UDP receive buffer issues (on Linux) with netstat -su In the output, under "Udp:" look at the number of "receive buffer errors". Ideally it is 0, suggesting that's not the problem. If > 0, that might be historical so run netstat -su repeatedly (e.g., use watch) to see if it's increasing.

from x9k3.

Yeah, I am familiar with netstat, and I don't usually need to adjust my receive buffer, but y'all didn't give me much to work with here as far being able diagnose your issue.

Did you upgrade new_reader? There are a lot of UDP specific updates. That is should help a lot .

How can we get the info that IDR frame are inserted every 2 secs during UDP to HLS translation ?

I wrote a tool called iframes, it should already be installed with x9k3,

iframes udp://1.2.3.4:5000

it will spit out a list of where the iframes are located in terms of PTS.

 iframes /home/a/mpegts/pcrvid.ts
14648.092267
14648.342511
14648.509344
14648.7596
14649.009844
14649.2601
14649.426933

or you can make a shell script like this

#!/bin/sh 

ffprobe -loglevel error -skip_frame nokey -select_streams v:0 -show_entries frame=pts_time $1 | grep pts | awk -F"=" '{print $2,","}'

How the GOP calaculation is done and how to check what is the GOP structure and GOP Length currently as part of .ts post translation ?

The is the simple explanation.
Group Of Pictures GOP size is how far apart your iframes are. if you do 30 frames a second and want an iframe every 2 seconds use a GOP of 60.

30 frames * 2 seconds = GOP size of 60

make sense?

How GPU acceleration can be added to this workflow ?

• That is outside the scope of x9k3 and I don't see why you would need it with x9k3, it doesn't use a lot of CPU.

  How DRM can be integrated with X9K3 workflow ?

• I have no idea, I'm not a big fan of DRM.

from x9k3.

How are y'all x9k3? what command line switches?
Are you using --live and --delete?
If not --live your manifest with be a mile long after 24 hours.

from x9k3.

How are y'all x9k3? what command line switches? Are you using --live and --delete? If not --live your manifest with be a mile long after 24 hours.

x9k3 -i udp://@236.1.2.8:5000 -o /var/www/html/X9k3_hls_scte35/ -t 2 -T x_cue -w 6 -l -d

we are using this command and we can see jerks after long run.

from x9k3.

Yeah, I am familiar with netstat, and I don't usually need to adjust my receive buffer, but y'all didn't give me much to work with here as far being able diagnose your issue.

I'm not with those guys. I'm just trying to help them. My other thought was timestamp rollover. This could make sense if the "after 24 hours" was actually 26.51 hours since their clock restarted.

from x9k3.

Rollover can be as pain in the ass, that's for sure, but I've had x9k3 running for months at a time without a problem. Each time it writes a segment it can reset the clock to anything because a lot of times the ads have wildly differing PTS, and I add DISCONTINUITY tags on every ad break. Even if there was a rollover problem, maybe one segment would be off.

@gauravv5 I need to see it to help you. Let's try this, After it's been running for 24 hours,
Install m3ufu and then run it against your x9k3 index.m3u8 file.
You are using pypy3 aren't you? It run's threefive, the SCTE35 decoder, 4 times faster than python3.

pypy3 -mpip install m3ufu

OR

python3 -mpip install m3ufu

run

m3ufu -i /var/www/html/X9k3_hls_scte35/index.m3u8 -o x9k3_test.ts

let it run for two minutes and then ctrl-C to kill it and put x9k3_test.ts somewhere I can download it.

from x9k3.

Run top and see what the CPU usage for x9k3 looks like, it should be low.

That's my x9k3, it's at 2.3%.
the pypy3 job is something else I have running.

from x9k3.

Hi @futzu , I used python3 for installing x9k3. i will install m3ufu and share ts file.
Thanks

from x9k3.

from x9k3.

uneven duration when segment duration is set at 4 sec.

from x9k3.

https://we.tl/t-LrgKCPRG8t you can download this file from here

from x9k3.

The glitches and jerks are generally called video artifacts, and I got them when used x9k3 and streamed the video over UDP. I ran the command

printf  26214400 > /proc/sys/net/core/rmem_max

and then again streamed over UDP to x9k3 and it played perfectly.
That's what you had said, but that the artifacts returned after 24 hours.

Here are your options:

• The encoding on that video is shit, you could try re-encoding it before segmenting.

• You could use ffmpeg or something else to download the stream and then pipe it to x9k3 so you don't have to write it to disk .

For example:

ffmpeg -copyts -i udp://1.2.3.4:3535 -map 0 -c copy - | x9k3 -d  -w 6

• I would try setting a cron job as root like every hour to set the rmem

Something like:

35   *   *   *   *   printf  26214400 > /proc/sys/net/core/rmem_max

and see if that helps.

from x9k3.

hi @futzu,
when using above command suggested by you .

ffmpeg -copyts -i udp://1.2.3.4:3535 -map 0 -c copy - | x9k3 -d -w 6

we are getting this error:-

Input #0, mpegts, from 'udp://236.1.2.8:5000':
Duration: N/A, start: 21535.037978, bitrate: N/A
Program 1
Metadata:
service_name : MService001
service_provider: Medialooks
Stream #0:0[0x400]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(top first), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 50 tbr, 90k tbn, 50 tbc
Stream #0:1[0x401]: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 128 kb/s
Stream #0:2[0x402]: Data: scte_35
[NULL @ 0x12b9060] Unable to find a suitable output format for 'pipe:'
pipe:: Invalid argument

No Stream Found.

so we have used
ffmpeg -copyts -i udp://236.1.2.8:5000 -map 0 -c copy -f mpegts udp://236.1.2.8:6000

and ingested output udp of feed to x9k3 running two different instances.

we can still see the video artifacts in the same.

Thanks for your extended support

from x9k3.

I'm sorry I gave you an incomplete command for ffmpeg
since your piping it to stdout, ffmpeg cannot infer the output format , you have to add -f mpegts

ffmpeg -copyts -i udp://1.2.3.4:3535 -map 0 -c copy -f mpegts - | x9k3 -d -w 6

ffmpeg udp options https://ffmpeg.org/ffmpeg-protocols.html#udp

from x9k3.

@futzu , Thanks for the complete command.
what I have noticed that when SCTE comes in and out . We can see 1-2 sec loading of the video or we can call it buffer.

Except that part stream looks to run fine.

from x9k3.

@futzu i have recored logs from player when stream stucks i.e scte-35 is injested.

[log] > [stream-controller]: Loaded fragment 826 of level 0
transmuxer-interface.ts:379 [log] > [mp4-remuxer]: ISGenerated flag reset
transmuxer-interface.ts:379 [log] > [mp4-remuxer]: initPTS & initDTS reset
transmuxer-interface.ts:379 [warn] > MPEG-TS PMT found at 16732 after unknown PID '1024'. Backtracking to sync byte @0 to parse all TS packets.
r.onWorkerMessage @ transmuxer-interface.ts:379
onwmsg @ transmuxer-interface.ts:88
transmuxer-interface.ts:379 [warn] > AVC: 178 ms (16022dts) hole between fragments detected, filling it
r.onWorkerMessage @ transmuxer-interface.ts:379
onwmsg @ transmuxer-interface.ts:88
transmuxer-interface.ts:379 [log] > Video: First PTS/DTS adjusted: 85601/85561, delta: 178 ms
base-stream-controller.ts:1745 [log] > [stream-controller]: FRAG_LOADING->PARSING
stream-controller.ts:1250 [log] > [stream-controller]: Init audio buffer, container:audio/mpeg, codecs[selected/level/parsed]=[//]
stream-controller.ts:1261 [log] > [stream-controller]: Init video buffer, container:video/mp4, codecs[level/parsed]=[/avc1.640028]
audio-stream-controller.ts:128 [log] > [audio-stream-controller]: InitPTS for cc: 97 found from main: 2831945453
transmuxer-interface.ts:379 [log] > [transmuxer.ts]: Flushed fragment 826 of level 0
base-stream-controller.ts:1745 [log] > [stream-controller]: PARSING->PARSED
base-stream-controller.ts:556 [log] > [stream-controller]: Buffered main sn: 826 of level 0 (frag:[85.601-86.959] > buffer:[38.602-48.096][48.182-65.044][65.092-73.516][73.601-86.705])
base-stream-controller.ts:1745 [log] > [stream-controller]: PARSED->IDLE
base-playlist-controller.ts:148 [log] > [level-controller]: live playlist 0 REFRESHED 829--1
base-playlist-controller.ts:261 [log] > [level-controller]: reload live playlist 0 in 2325 ms
stream-controller.ts:631 [log] > [stream-controller]: Level 0 loaded [810,829][part-829--1], cc [94, 97] duration:46.18306666666668
buffer-controller.ts:692 [log] > [buffer-controller]: Updating Media Source duration to 94.279
App.js:118 onProgress {playedSeconds: 84.827584, played: 0.8997499338842273, loadedSeconds: 86.705388, loaded: 0.9196674411993895}
base-stream-controller.ts:727 [log] > [stream-controller]: Loading fragment 827 cc: 97 of [810-829] level: 0, target: 86.959
base-stream-controller.ts:1745 [log] > [stream-controller]: IDLE->FRAG_LOADING
App.js:118 onProgress {playedSeconds: 85.83442, played: 0.9104292504663458, loadedSeconds: 86.705388, loaded: 0.9196674411993895}
base-stream-controller.ts:386 [log] > [stream-controller]: Loaded fragment 827 of level 0
transmuxer-interface.ts:379 [warn] > AVC: 20 ms (1831dts) hole between fragments detected, filling it
r.onWorkerMessage @ transmuxer-interface.ts:379
onwmsg @ transmuxer-interface.ts:88
transmuxer-interface.ts:379 [log] > Video: First PTS/DTS adjusted: 86959/86919, delta: 20 ms
base-stream-controller.ts:1745 [log] > [stream-controller]: FRAG_LOADING->PARSING

from x9k3.

I really don't think the problem is x9k3,
correlation is not causation, meaing the SCTE-35 is where the ads splice in and there are a lot of other things going on.
x9k3 doesnt do any alterations of anything in the video stream, it just slices.
My best guess is that the javascript is struggling with bad discontinuity from the stream. The PTS, PMT, and PAT often change with the ad, that's why x9k3 adds the DISCONTINUITY tags, The player is supposed to reset on discontinuity. I think it's just poor encoding.

transmuxer-interface.ts:379 [warn] > MPEG-TS PMT found at 16732 after unknown PID '1024'. Backtracking to sync byte @0 to parse all TS packets.

That's not hls.js is it?

from x9k3.

Try this

python3 -mpip install --upgrade threefive

then when you run x9k3, turn on Shulga mode with -S
run the same command but with a -S like x9k3 -d -w 6 -S

(Shulga mode is named after a guy named Shulga who had h264 video but mpeg2 iframes.)

from x9k3.

Without Shulga mode:

#EXT-X-X9K3-VERSION:0.1.89
#EXTINF:3.650022,
seg0.ts
#EXTINF:4.000000,
seg1.ts
#EXTINF:2.600000,
seg2.ts
#EXTINF:1.880000,
seg3.ts
#EXT-X-DISCONTINUITY
#EXT-X-CUE-OUT:20.0
#EXTINF:7.560000,
seg4.ts
#EXT-X-CUE-OUT-CONT:7.560000/20.0
#EXTINF:3.960000,
seg5.ts
#EXT-X-CUE-OUT-CONT:11.520000/20.0
#EXTINF:4.000000,
seg6.ts
#EXT-X-CUE-OUT-CONT:15.520000/20.0
#EXTINF:2.600000,
seg7.ts
#EXT-X-CUE-OUT-CONT:18.120000/20.0
#EXTINF:1.880000,
seg8.ts
#EXT-X-DISCONTINUITY
#EXT-X-CUE-OUT:20.0
#EXTINF:7.520000,
seg9.ts
#EXT-X-CUE-OUT-CONT:7.520000/20.0
#EXTINF:3.960000,
seg10.ts

With Shulga mode:

#EXT-X-X9K3-VERSION:0.1.89
#EXTINF:2.016000,
seg0.ts
#EXTINF:2.154022,
seg1.ts
#EXTINF:2.080000,
seg2.ts
#EXTINF:2.000000,
seg3.ts
#EXTINF:2.000000,
seg4.ts
#EXTINF:1.880000,
seg5.ts
#EXT-X-DISCONTINUITY
#EXT-X-CUE-OUT:20.0
#EXTINF:2.040000,
seg6.ts
#EXT-X-CUE-OUT-CONT:2.040000/20.0
#EXTINF:2.080000,
seg7.ts
#EXT-X-CUE-OUT-CONT:4.120000/20.0
#EXTINF:2.000000,
seg8.ts
#EXT-X-CUE-OUT-CONT:6.120000/20.0
#EXTINF:2.000000,
seg9.ts
#EXT-X-CUE-OUT-CONT:8.120000/20.0
#EXTINF:2.000000,
seg10.ts
#EXT-X-CUE-OUT-CONT:10.120000/20.0
#EXTINF:2.109978,

from x9k3.

can you please brief what Shulga mode do?
Because I can see repeated artifacts in output stream when using shulga.

from x9k3.

(Shulga mode is named after a guy named Shulga who had h264 video but mpeg2 iframes.)

Look man, it's not x9k3.We got the udp sorted by adjusting your system settings, and it worked fine. It works fine for me. Like I said, x9k3 just slices the video, it does not alter anything.
The encoding is poor and you're transcoding in javascript to mp4, these are less than ideal conditions.
x9k3 is not the issue.

from x9k3.

Thanks Adrian mate,

We appreciate your help on the subject and we are working to improve the encoding part so that DVB-Compliant stream will ingest into the X9K3 workflow solution.

Thanks much for your support.

Regards,
Manu

from x9k3.

I wish I had a better answer for you, but I don't.

from x9k3.

No problem Adrian mate.

We really appreciate your help and support.
We are now developing master.m3u8 from index.m3u8 and from there we will take .ts segments and add gpu acceleration into the same; meaning transcoding using ffmpeg

from x9k3.

HI @futzu, we need your assistance we are trying to transcode ts genrated by x9k3 to multi birate ts.
and creating our own multi bitrate index.m3u8. But we are facing Audiopts overlap issue. which has led us to ask you for help.

I would share my development if needed to check

Any help would be appreciated.

Thanks

from x9k3.

That's going to be fault of the encoding.
All x9k3 does is slice the stream, it does not change anything about the video or audio.

I'll tell you this, audio is often out of sync with video, I see it a lot in mpegts. x9k3 is not going to fix that, or make it happen either.

from x9k3.

Hi @futzu,

X9K3 works fine with the DVB-Complaint input stream and able to create single profile HLS manifest(index.m3u8)

Problem we are facing when we are trying to create a multi-bitrate profiles(1080p, 720p, 576p, 480p, 320p) from the .ts chunks created from X9K3 (UDP to HLS translation) as part of the index.m3u8

We are trying to pickup those .ts chunks for creating master manifest (master.m3u8) along with multi-bitrate profiles, with that we are facing pts alignment problem as mentioned by gaurav

we need your help if you can assist us on the same..

from x9k3.

That is not an x9k3 issue man.
If you want to sort out your setup, that's what I do for a living.

from x9k3.

Hi @futzu mate,
I know it's not an X9K3 issue. we want to finalise our solution like umzz

Please share your email id and contact details so that we can discuss on the same from commercial perspective

from x9k3.

@futzu mate,
How much your solution will cost to us..?

As i have mentioned, our solution works fine with single profile & manifest. The problem is with multi-bitrate along with master.m3u8

from x9k3.

I'm sorry, I keep forgetting to reply to you. I am kind of busy right now, probably for the next month.
I just saw you're doing it from the chunks,if you are using that video you showed me, that's probably not going to go well.
What's cool about threefive, (the SCTE-35parser ) of x9k3 is that even if ffmpeg changes the stream type, it can still parse it.

What are you using to encode the video? If you are using ffmpeg, make sure start with ffmpeg -copyts and set the GOP to 30 with -g30, that will give you an iframe every second. Iframes will solve a lot of problems.

Here's what I would try re-encode with ffmpeg and pipe it to x9k3
ffmpeg -copyts -i udp://example.com -map 0 ..... -g30 -f mpegts - | x9k3

(don't set the input for x9k3 if you pipe to it)
Re-encode the video and audio and see if that works.

See if that works better when you encode the chunks

from x9k3.

Now the above is just a way to help you figure out where the problem may be.

Another idea, do what you're doing now, but just make two variants and keep the very close in scale, if you scale video up or down more than twice it's size, it can be problematic. Just to test , just encode one variant from the x9k3 chunks and keep it very close in scale. Like if it's 960 , do a variant at 768, and see if that works better.

from x9k3.

No response for 3 weeks so I am closing this.

from x9k3.