I'm using ffmpeg to create media playlists. After that I wonder if there is a way to create a master playlist using this lib gluing all those media playlists varying them on bandwidth

When an HLS object is cloned via JSON.parse(JSON.stringify(manifest)), the resulting object cannot be stringified. The issue lies in

1. programDateTime field - This gets converted into string. Need to be converted back to Date before stringify
2. IV - The buffer gets converted to object. Should be converted back to buffer before stringify
   It would be great if the stringify function can handle this internally more efficiently

Here is the sample code to demonstrate this

const hls   =  require("hls-parser")
var input = "#EXTM3U \n" + 
            "#EXT-X-VERSION:5 \n" + 
            "#EXT-X-MEDIA-SEQUENCE:0 \n" + 
            "#EXT-X-TARGETDURATION:4 \n" + 
            "#EXT-X-PROGRAM-DATE-TIME:2021-08-02T07:24:22.997000+00:00 \n" + 
            "#EXT-X-KEY:METHOD=AES-128,URI=\"enc.key\"" + ",IV=0x0000000000000000000000000000044A \n" + 
            "#EXTINF:4, \n" + 
            "0.ts \n" + 
            "#EXTINF:4, \n" + 
            "1.ts \n" + 
            "#EXTINF:4, \n" + 
            "2.ts";
var manifest  = hls.parse(input);
var manifest2 = JSON.parse(JSON.stringify(manifest));
var output = hls.stringify(manifest2);

so that we can do parse()/stringify() only with relative URIs that should not be resolved in parse/stringfy time but in download time.

Upon serialize() call

The multiple markers are unsupported when stringify try to write an m3u8 file.
Redundant item will be thrown.
You should add the #EXT-X-CUE-OUT & #EXT-X-CUE-IN in ALLOW_REDUNDANCY array.

for the competing LHLS proposal, it would be helpful for hls-parser to be able to parse/serialize EXT-X-PREFETCH tags.

https://github.com/video-dev/hlsjs-rfcs/blob/lhls-spec/proposals/0001-lhls.md

Below is an example

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:4
#EXT-X-MEDIA-SEQUENCE:109791
#EXT-X-SERVER-CONTROL:CAN-BLOCK-RELOAD=YES,PART-HOLD-BACK=3.12
#EXT-X-PART-INF:PART-TARGET=1.04
#EXTINF:3.96,
l_5797_63028341_109791.ts
#EXT-X-PART:DURATION=1,URI="l_5797_63032301_109792_0.ts"
#EXT-X-PART:DURATION=1,URI="l_5797_63032301_109792_1.ts"
#EXT-X-PART:DURATION=1,URI="l_5797_63032301_109792_2.ts"
#EXT-X-PART:DURATION=1.04,URI="l_5797_63032301_109792_3.ts"
#EXTINF:3.96,
l_5797_63032301_109792.ts
#EXT-X-PART:DURATION=1,URI="l_5797_63036261_109793_0.ts"
#EXT-X-PART:DURATION=1,URI="l_5797_63036261_109793_1.ts"
#EXT-X-PART:DURATION=1,URI="l_5797_63036261_109793_2.ts"
#EXT-X-PART:DURATION=1.04,URI="l_5797_63036261_109793_3.ts"

If encryption is enabled but no IV is provided, the IV is the corresponds to it's segment number (the default behaviour for HLS).
see also:
rfc8216
HLS spec rev.23

Right now the IV is undefined

HLS.parse takes a text input, however for very large playlists having to load the entire playlist to memory (a string variable) can be very memory intensive. It would be great to support a buffered input so that not the entire playlist has to be loaded to memory all at once. Supporting buffered output for hls.stringify will also allow to have a buffered reader/writer for the entire parse & stringify pipeline.

The hls-parser is somewhat opinionated about constructs which turn up in 'real world' scenario's, such as the BipBop hello world HLS stream. To demonstrate:

/*
 * videojs-contrib-hls project suggests the following 'Hello, world' type HLS example:
 * https://d2zihajmogu5jn.cloudfront.net/bipbop-advanced/bipbop_16x9_variant.m3u8
 *
 * It's essentially the NTSC test signal with a few tracks of varying qualities,
 * a few audio tracks, subtitles and so on
 *
 * Note the main tracks are loops of the test signal to get to a 30min duration.
 *
 * ...
 *
 * hls-parser *should* be able to parse & stringify this
 */

const fetch = require('node-fetch');
const HLSParser = require('hls-parser');

// test the master playlist
/*
 * It fails because, it turns out subtitles *can* be forced
 */
const masterPlaylist = fetch('https://d2zihajmogu5jn.cloudfront.net/bipbop-advanced/bipbop_16x9_variant.m3u8')
    .then(response => response.text())
    .then(HLSParser.parse)
    .then(HLSParser.stringify)
    .then(() => console.log('Master playlist: PASS'))
    .catch(error => console.log('Master playlist: FAIL\n', error));

// test the gear1 variant referenced from the master playlist
/*
 * It fails becasue it is a loop of a segment and hls-parser is opinionated about this
 * ... similarly it does not like re-used keys/IVs in sequences such as IV_A, IV_B, IV_A (which is admittedly strange but still 'valid')
 */
const gear1 = fetch('https://d2zihajmogu5jn.cloudfront.net/bipbop-advanced/gear1/prog_index.m3u8')
    .then(response => response.text())
    .then(HLSParser.parse)
    .then(HLSParser.stringify)
    .then(() => console.log('Gear1 channel playlist: PASS'))
    .catch(error => console.log('Gear1 channel playlist: FAIL\n', error));

I guess the question is: should hls-parser relax some of its assertions to cope with this?

Hello,

I have multiple noImplicitAny errors when building my project, despites having the following config:

{
    "skipLibCheck": true,
    "noImplicitAny": true
}

It is probably related to the issue: microsoft/TypeScript#15363

It looks like .ts file should not be shipped in the release.

Thanks

When I've multiple rendition (for example multiple codec for the same language) stringify return the manifest with a EXT-X-STREAM-INF between EXT-X-MEDIA.

For example :

#EXTM3U
#EXT-X-VERSION:4
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio_aacl",NAME="English",DEFAULT=YES,AUTOSELECT=YES,LANGUAGE="eng",CHANNELS="2",URI="eng_aacl.m3u8"
#EXT-X-STREAM-INF:BANDWIDTH=8277095,AVERAGE-BANDWIDTH=6670088,CODECS="avc1.640028,mp4a.40.5",RESOLUTION=1920x1080,FRAME-RATE=29.970,AUDIO="audio_aacl"
1080p.m3u8
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio_aac",NAME="English",DEFAULT=YES,AUTOSELECT=YES,LANGUAGE="eng",CHANNELS="2",URI="eng_aac.m3u8"
#EXT-X-STREAM-INF:BANDWIDTH=8417895,AVERAGE-BANDWIDTH=6810888,CODECS="avc1.640028,mp4a.40.2",RESOLUTION=1920x1080,FRAME-RATE=29.970,AUDIO="eng_aac"
1080p.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=5235630,AVERAGE-BANDWIDTH=3951068,CODECS="avc1.4D401F,mp4a.40.2",RESOLUTION=1280x720,FRAME-RATE=29.970,AUDIO="audio_aacl"
720p.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=5094830,AVERAGE-BANDWIDTH=3810268,CODECS="avc1.4D401F,mp4a.40.5",RESOLUTION=1280x720,FRAME-RATE=29.970,AUDIO="eng_aac"
720p.m3u8

It's appear to be a design issue in the code : https://github.com/kuu/hls-parser/blob/3d37ad8ef3a4836357e30764c8c43010cb8ed0a4/stringify.ts#L126C1-L126C1

You should first create the renditions and then the EXT-X-STREAM-INF

for testing:
https://developer.apple.com/streaming/examples/

I tried to use the parser in the browser, but I get errors about Buffer not being available (ReferenceError ...)
Is this library intended to be only used with ode.js?

When I replace an existing segment with a new segment with a different key parameters, the replaced segment has correct key information. However, the original segments which follow replaced segment have wrong keys.

Sample code to reproduce this issue

const hls   =  require("hls-parser")
const axios = require("axios")
axios.get("https://k7q5a5e5.ssl.hwcdn.net/files/company/5e2f4a02199f0c33444d9403/assets/videos/5e311e889ba3f225c86c0b52/vod/5e311e889ba3f225c86c0b52_Layer_1.m3u8")
.then(function(manifest_data) {
   var media_manifest = hls.parse(manifest_data.data);
   var temp = JSON.parse(JSON.stringify(media_manifest.segments[100]));
   temp.key.uri="dummy.enc";
   media_manifest.segments.splice(100, 1, temp);
   console.log(hls.stringify(media_manifest))
});

Per the revision 23 of the HLS spec, multiple EXT-X-MAP's can be contained in a media playlist as long as it comes after a DISCONTINUITY tag.

This effects fragmented mp4s or fmp4 when you are compositing different videos together. A new map file containing the videos PMT data needs to be inserted after the DISCONTINUITY flag.

As of right now hls-parser throws an error for Redundant Item which prevents these composited playlists from being loaded.

'#EXT-X-MAP' needs to be added to the end of the ALLOW_REDUNDANCY constant in utils.js.

When trying to output in HLS-V4 format, executing the following code causes an error.

const HLS = require('hls-parser');

const { MediaPlaylist, Segment } = HLS.types;
const segments = [];
Items.forEach((item) => {
  segments.push(
    new Segment({
      uri: item.uri,
      duration: item.duration,
      byterange: {
        length: item.byterangeLength,
        offset: item.byterangeOffset
      },
      mediaSequenceNumber: item.mediaSequenceNumber,
      discontinuitySequence: 0
    })
  )
});

const playlist = new MediaPlaylist({
  version: 4,
  targetDuration: 10,
  mediaSequenceBase: 0,
  playlistType: 'VOD',
  segments: segments
});

console.log(HLS.stringify(playlist))

-----error log-----

{
    "errorMessage": "Invalid Playlist : Redundant item (test01.ts)",
    "errorType": "Error",
    "stackTrace": [
        "Object.INVALIDPLAYLIST (/var/task/node_modules/hls-parser/utils.js:44:9)",
        "LineArray.push (/var/task/node_modules/hls-parser/stringify.js:34:15)",
        "buildSegment (/var/task/node_modules/hls-parser/stringify.js:218:9)",
        "buildMediaPlaylist (/var/task/node_modules/hls-parser/stringify.js:191:5)",
        "Object.stringify (/var/task/node_modules/hls-parser/stringify.js:280:5)",
        "module.exports.handler (/var/task/index.js:89:19)",
        "<anonymous>",
        "process._tickDomainCallback (internal/process/next_tick.js:228:7)"
    ]
}

I do not know if coding is bad or hls-parser is faulty. Please tell me how to resolve.
If byterange is not written, no error occurs and it works normally.

Hello this library has been a life-saver for me generating M3U8 files, but I've noticed they sometimes fail depending on the consumer, namely iOS/Safari based viewers.
Not that I believe this library's implementation that betrays the spec, but rather Apple is strict in ways they do not advertise (i.e. it is Apple who is wrong, but we can accommodate)

The problem

Take two example M3U8 files:

(These files are slight modifications of sample files provided here: https://ottverse.com/free-hls-m3u8-test-urls/ -> http://demo.unified-streaming.com/video/tears-of-steel/tears-of-steel.ism/.m3u8)

success.m3u8

#EXTM3U
#EXT-X-VERSION:7
#EXT-X-TARGETDURATION:5
#EXT-X-PLAYLIST-TYPE:VOD
#EXTINF:4
http://demo.unified-streaming.com/video/tears-of-steel/tears-of-steel.ism/tears-of-steel-audio_eng=128002-video_eng=1001000-1.ts
#EXT-X-DISCONTINUITY
#EXTINF:4
#EXT-X-BYTERANGE:126712@0
http://demo.unified-streaming.com/video/tears-of-steel/tears-of-steel.ism/tears-of-steel-audio_eng=128002-video_eng=1001000-2.ts
#EXTINF:4
http://demo.unified-streaming.com/video/tears-of-steel/tears-of-steel.ism/tears-of-steel-audio_eng=128002-video_eng=1001000-3.ts
#EXT-X-ENDLIST

and fail.m3u8

#EXTM3U
#EXT-X-VERSION:7
#EXT-X-TARGETDURATION:5
#EXT-X-PLAYLIST-TYPE:VOD
#EXTINF:4
http://demo.unified-streaming.com/video/tears-of-steel/tears-of-steel.ism/tears-of-steel-audio_eng=128002-video_eng=1001000-1.ts
#EXT-X-BYTERANGE:126712@0
#EXT-X-DISCONTINUITY
#EXTINF:4
http://demo.unified-streaming.com/video/tears-of-steel/tears-of-steel.ism/tears-of-steel-audio_eng=128002-video_eng=1001000-2.ts
#EXTINF:4
http://demo.unified-streaming.com/video/tears-of-steel/tears-of-steel.ism/tears-of-steel-audio_eng=128002-video_eng=1001000-3.ts
#EXT-X-ENDLIST

Notice that:

1. The second segment has both a byte range (this is really the whole file, but proves a point here) and a discontinuity (again not actually necessary but proves a point).
2. The second file is supposedly exactly the same, but the byte range is now at the start of the segment

Try loading the files in Safari (note Safari actually matters, I am not able to reproduce on Chrome, or other players like hls.js who are probably less strict)
A local serving package like https://www.npmjs.com/package/serve could come in handy

The first file is successful and plays a 12 second snippet, the second fails to load.

From my testing, it is the positioning of EXT-X-BYTERANGE before EXT-X-DISCONTINUITY (and potentially before EXTINF) that causes this issue

How it impacts this package

The current stringify logic writes EXT-X-BYTERANGE before anything else https://github.com/kuu/hls-parser/blob/master/stringify.js#L296 recreating the issue discussed above.

As stated before I believe this is allowed by spec, but some interpreters do not handle it well, so it is easy enough to move that render to later in the buildSegment method.

I'm happy to submit a PR with this suggested change as well, but wanted to open the discussion for what the real problem is, and potential ways to fix it.

Hello 👋

First of all, thanks for this lib which helped quite a bit so far 🎉

Do you @kuu have any plans to support the new versions described here : https://tools.ietf.org/html/draft-pantos-hls-rfc8216bis-08 ?
Latest version appears to be 10 and I would need it to parse the video-range tag.

Regards

If you parse and then stringify the following

#EXTM3U
#EXT-X-VERSION:6
#EXT-X-TARGETDURATION:5
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-MAP:URI="high_1.mp4",BYTERANGE="824@0"
#EXTINF:4.805,
#EXT-X-BYTERANGE:1531172@976
high_1.mp4
#EXT-X-ENDLIST

Instead of getting the same as the above, you'd get something like

#EXTM3U
#EXT-X-VERSION:6
#EXT-X-TARGETDURATION:5
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-MAP:URI="high_1.mp4",BYTERANGE=[object Object]
#EXTINF:4.805,
#EXT-X-BYTERANGE:1531172@976
high_1.mp4
#EXT-X-ENDLIST

Notice the JS objects as [object Object]

This is because the code doesn't try to stringy the byterange object of a map item:

Should probably be abstracted into a function and reused by both.

Hi,

I've experienced the following issue during parse:
CLOSED-CAPTIONS attribute MUST match the value of the GROUP-ID attribute of an EXT-X-MEDIA tag

The playlist has the following variant:
#EXT-X-STREAM-INF:BANDWIDTH=202000,AVERAGE-BANDWIDTH=184000,CODECS="mp4a.40.2,avc1.42C01E",RESOLUTION=192x108,FRAME-RATE=25,AUDIO="audio-aacl-64",CLOSED-CAPTIONS=NONE

But there's no closed caption group provided in the playlist.

However, I then checked the specification and found this:
The value can be either a quoted-string or an enumerated-string with the value NONE.

So it seems that the parse method does not honor that

Hello. I see "license": "MIT", in package.json but I can't find any actual license file in the project. The one condition in the MIT License is:

The above copyright notice and this permission notice (including the next paragraph) shall be included in all copies or substantial portions of the Software.

But there are no copyright notices in the project, and the permission notice is not included in any portion of the software.

Hi,
What about parsing and dealing with ad markers segments ?
i.e.

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:8
#EXT-X-PLAYLIST-TYPE:VOD
#EXTINF:8.008,
1.ts
#EXT-X-CUE-OUT:15.000
#EXTINF:8.008,
2.ts
#EXTINF:8.008,
3.ts
#EXT-X-CUE-IN
#EXTINF:8.008,
4.ts
#EXT-X-ENDLIST

Thanks

Sample:

#EXTM3U
#EXT-X-VERSION:6
#EXT-X-TARGETDURATION:4
#EXT-X-MAP:URI="init.mp4"
#EXT-X-KEY:METHOD=SAMPLE-AES,URI="skd://SOME_SKD",KEYFORMATVERSIONS="1",KEYFORMAT="com.apple.streamingkeydelivery"
#EXT-X-KEY:METHOD=SAMPLE-AES,URI="XXXXX"
#EXTINF:2.000,
127920.m4s

After parsing I see one key in the segment

For example, the Variant defines score as optional, however, the type does not reflect that:

Just have to say great library so far! Been using it to generate playlist.
I don't have a fix right off the top of my head after glancing at the code but I noticed that I can't add Redundant Streams to the playlist variants when I call HLS.stringify(). My quick fix was to comment out https://github.com/kuu/hls-parser/blob/master/stringify.js#L18 but I'm pretty sure this would cause more problems then it would fix.

Let me know if their is any way I can help.

Great work though, thanks!

Hi,

thanks for this awesome library. I want to use it for a new project I have but it seems that the last version published on NPM isn't that actual state of the lib.

In my project I need both Typescript and marker support. I found that @types/hls-parser project doesn't includes markers and if I use npm install [email protected]:kuu/hls-parser.git I get the right types for typescript but I can't compile anymore.

Let me know if I can help in any way.

Thanks

Hi @kuu,

I came across the following issue but I can't see what's wrong with parsing the playlist:

Error: Invalid Playlist : The EXT-X-MEDIA-SEQUENCE tag MUST appear before the first Media Segment in the Playlist.

#EXTM3U
# NeuLion Adaptive Streaming Server Version: 4.2.20181107.22941 
#EXT-X-PROGRAM-DATE-TIME:2018-12-05T23:41:56Z
#EXT-X-TARGETDURATION:10
#EXT-X-MEDIA-SEQUENCE:1116587
#EXTINF:10,
3y7ZyvdpWu_hd_800_20181205234156_042.ts
#EXTINF:10,
3y7ZyvdpWu_hd_800_20181205234206_042.ts
#EXTINF:10,
3y7ZyvdpWu_hd_800_20181205234216_042.ts
#EXTINF:10,
3y7ZyvdpWu_hd_800_20181205234226_042.ts
#EXTINF:10,
3y7ZyvdpWu_hd_800_20181205234236_042.ts
#EXTINF:10,
3y7ZyvdpWu_hd_800_20181205234246_042.ts

As far as i can tell the EXT-X-MEDIA-SEQUENCE appears before the first segment.

Might need an additional check for dateRange.start on this line for parsing EXT-X-DATERANGE tags with no START-DATE attribute.

Although this is a required field based on the RFC, I would expect these invalid tags to be skipped rather than throw an error if strictMode is disabled

The RFC is slightly ambiguous as it includes an example of the tag without this attribute https://tools.ietf.org/html/rfc8216#section-8.10

Currently we're seeing TypeError: Cannot read property 'getTime' of undefined errors for tags without START-DATE

object ({AUDIO: number, VIDEO: number, SUBTITLES: number, CLOSED-CAPTIONS: number})

This is a build problem with TypeScript (when trying to add types) and will be an exception in run time.