CBC News Videos Not Downloading Normally - Missing XML (404 Error) about youtube-dl HOT 4 OPEN

This looks like, and is, the problem reported in yt-dlp/yt-dlp#9534.

#30839 "if you were actually using yt-dlp ..."

However yt-dl also has the problem and we should be able to use the same solution. After revising a WIP PR from several years ago that was never merged to include some of the changes from the yt-dlp PR:

python -m youtube_dl -vF 'https://www.cbc.ca/player/play/1.7159486'
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-vF', u'https://www.cbc.ca/player/play/1.7159486']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Git HEAD: 8cd66b76f
[debug] Python 2.7.15 (CPython i686 32bit) - Linux-6.1.0-18-686-pae-i686-with-debian-12.5 - OpenSSL 1.1.1a  20 Nov 2018 - glibc 2.1.3
[debug] exe versions: ffmpeg 5.1.4-0, ffprobe 5.1.4-0
[debug] Proxy map: {}
[cbc.ca:player] 1.7159486: Downloading webpage
[ThePlatform] 2324188227929: Downloading SMIL data
[ThePlatform] 2324188227929: Downloading m3u8 information
[ThePlatform] 2324188227929: Downloading JSON metadata
[info] Available formats for 2324188227929:
format code                extension  resolution note
hls-program_audio-English  mp4        audio only [eng] 
hls-422                    mp4        320x180     422k , avc1.4d400d, 30.0fps, video only
hls-580                    mp4        320x180     580k , avc1.640029, 30.0fps, video only
hls-910                    mp4        640x360     910k , avc1.640029, 30.0fps, video only
hls-1350                   mp4        864x486    1350k , avc1.640029, 30.0fps, video only
hls-2120                   mp4        864x486    2120k , avc1.640029, 30.0fps, video only
hls-5640                   mp4        1920x1080  5640k , avc1.4d4028, 30.0fps, video only (best)
$ 

Rather than passing the ID from the URL directly to the video host, we have to extract a media ID from the hydration JSON in the video page.

from youtube-dl.

Yes, this is the exact same problem. I'll be updating the PR at yt-dlp in a bit, the same changes should be usable here as well.

from youtube-dl.

I noticed that the video host URLs used by the site contain the query parameter formats=M3U,MPEG4,MP3 (used in the above test patch, too) rather than the olde worlde formats=MPEG4,FLV,MP3 in the extractor(s).

Maybe this is ignored, or maybe it'll start to cause a problem later if ThePlatform withdraws some level of FLV support.

from youtube-dl.

IIRC, formats=M3U,MPEG4,MP3 preferentially downloads the m3u manifest if available (which contains the information for streaming the individual chunks), whereas formats=MPEG4,FLV,MP3 preferentially downloads an MP4 file as a whole. For streaming, downloading it in chunks makes more sense - plus it offers the benefit that most of the M3U files have multiple streams specified for different bandwidths, allowing for the media player to chose the appropriate stream based on the current resolution and available bandwidth. For downloading, however, it makes more sense to download the MP4 as a whole, since that generally has better performance on in this case, since it avoids all the overhead of downloading often 100+ chunks (with associated I/O) and assembling them all back together afterwards.

from youtube-dl.