clitic / vsd Goto Github PK

View Code? Open in Web Editor NEW

181.0 4.0 19.0 4.54 MB

Download video streams served over HTTP from websites, DASH (.mpd) and HLS (.m3u8) playlists

License: Apache License 2.0

Rust 95.29% Jupyter Notebook 1.12% C++ 3.25% C 0.34%

hls m3u8 m3u8-downloader downloader video dash mpd

vsd's People

Contributors

Stargazers

Watchers

Forkers

thisisdevang cenker373 geekportuga burtay saikrishnaketha kirinse limangox kousei-friend-a remme123 krzemienski wcork mvasuki1 qwqawawow parkour-labs trulyinfinite dreamjz jasonrm jcallicoat denji

vsd's Issues

Unable to pass cookies

GET /live_record_g/STAR_LIFE_1494_250_320x240_v1_FMP4/record_5215226.m3u8 HTTP/1.1
User-Agent: Mozilla/5.0 (Linux; Android 12) StarTimesON/6.8-1
Accept: /
Range: bytes=0-
Connection: close
Host: cloudflare-video.gslb.startimestv.com
Cookie: CloudFront-Signature=AbrjlHHK2m1TxUqCncwe17wljFVDwc8Yga~~QJIxx8z3YQJ8Wwr4dmg5OiuGV29KWyrgbYX05jSrGGkEXxyXg4KBMIj2SHxR9~~UgCxAWagLWCv0eU3qagsomPYXuLmhctJZjOgnDQ3AQ4mGruNNY5c~~HTgkiGCZCYg0NwiJAnEJj9lbn-yH93XZ8hJxLUmJKthTko9V09sd3uCSWcc3MnU1ueFvzQd5lHNwoIRP~~nq~~ZycweekmwSmx9sZFu~~SvLg8rHXrcxbjK-pDIt6cE6GepGUNasRK0UeJvfk-tXcwxSCVcxlmDOjeNIzjASPwv3qaJ318HcjLu0JiVL8F3C70Q__;CloudFront-Policy=eyJTdGF0ZW1lbnQiOiBbeyJSZXNvdXJjZSI6Imh0dHA6Ly8qLnN0YXJ0aW1lc3R2LmNvbS9saXZlX3JlY29yKiIsIkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTY5NjIzNjc3OX19fV19;CloudFront-Key-Pair-Id=APKAIPKYIDIQPHMZ2WKA

I have tried to pass cookies but still not working got error

vsd save --set-cookie "CloudFront-Signature=AbrjlHHK2m1TxUqCncwe17wljFVDwc8Yga~~QJIxx8z3YQJ8Wwr4dmg5OiuGV29KWyrgbYX05jSrGGkEXxyXg4KBMIj2SHxR9~~UgCxAWagLWCv0eU3qagsomPYXuLmhctJZjOgnDQ3AQ4mGruNNY5c~~HTgkiGCZCYg0NwiJAnEJj9lbn-yH93XZ8hJxLUmJKthTko9V09sd3uCSWcc3MnU1ueFvzQd5lHNwoIRP~~nq~~ZycweekmwSmx9sZFu~~SvLg8rHXrcxbjK-pDIt6cE6GepGUNasRK0UeJvfk-tXcwxSCVcxlmDOjeNIzjASPwv3qaJ318HcjLu0JiVL8F3C70Q__;CloudFront-Policy=eyJTdGF0ZW1lbnQiOiBbeyJSZXNvdXJjZSI6Imh0dHA6Ly8qLnN0YXJ0aW1lc3R2LmNvbS9saXZlX3JlY29yKiIsIkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTY5NjIzNjc3OX19fV19;CloudFront-Key-Pair-Id=APKAIPKYIDIQPHMZ2WKA" "http://cloudflare-video.gslb.startimestv.com/live_record_g/STAR_LIFE_1494_250_320x240_v1_FMP4/record_5215226.m3u8" output_file.mp4 error: builder error: relative URL without a #base

-o option not working with Twitch streams

Sample code:

$ vsd save 'https://d1m7jfoe9zdc1j.cloudfront.net/2e8eaeb3b5b742695290_nba2k_49250912077_1694040317/chunked/index-dvr.m3u8' -o nba.ts

File will still be saved as vsd_undefined_index-dvr_m3u8.ts.

Cannot decode PSSH box data as valid widevine data.

I try to download mpd url but I get this error

error: Cannot decode PSSH box data as valid widevine data.

DecodeError {
description: "invalid wire type: Varint (expected LengthDelimited)",
stack: [
(
"WidevinePsshData",
"key_ids",
),
],
}.

How to add custom header like cookies or referer in CLI version?

error when using the `vsd --capture` behind cloudflare warp connection

description

specific sites are blocked in my country. i need to use cloudflare warp to unblock the targeted sites. but the problem occurs when using the vsd within the cloudflare warp connection.

command

vsd https://www.youtube.com/watch?v=vT1HI_-dJwA --capture

command vsd https://test-streams.mux.dev/x36xhzz/x36xhzz.m3u8 -o video.mp4 working perfectly. the problems only occur in the capture & collect feature.

error

tech stack

vsd

google-chrome

ffmpeg

cargo

os

wsl 2 with ubuntu 20 distro within windows 11

another

also, gui version of the google-chrome usually works within wsl 2

Not able to select all the streams (multi period mpd)

Hello! I came across your downloader while searching for a tool that can support multi-period MPD links. However, I am facing an issue where I cannot select all the video and audio streams simultaneously. For instance, to get the best video stream (1080p), I need to select three ID's and combine them into a single output. The same is applicable for audio streams. Can you add a feature that allows me to select all streams, including all 1080p and audio streams, and merge them into a single output for both video and audio?

As an example, here's an MPD link -> https://pubads.g.doubleclick.net/ondemand/dash/content/2615619/vid/10150065538386527/GRQ/streams/0bca5579-f7df-40f3-ae3a-7f846e1b0094/manifest.mpd that requires a VPN to access due to geoblocking If needed,I can provide VPN and site credentials to access this link from Belgium. Please let me know if you require any further information. Thank you in advance for your assistance.

Hi! Hope to add batch download.

CENC

I see CENC is supported:

https://github.com/clitic/vsd#features

but I dont see any examples. If I have a key, how do I supply it?

what's going to do after the chrome window opens in collect mode?

description

how to download the video after the chrome window is opened? i've played the video and opened the dev tools, and nothing happened. am i missing a step?

So many issues ...

Support for canceling the captured requests

Several servers, like akamai, will block all requests after the first request. This problem will make the second request from vsd be blocked (the first request happens on the website or browser, and the second request happens inside the vsd).

Can we add a new parameter like --block-captured-requests to tell vsd to don't let the captured requests continue/pass to prevent duplicate requests?

vsd/src/chrome.rs

Line 152 in d234217

RequestInterceptionDecision::Continue

I can make a PR for this simple request (still learning rust😭. come from javascript & golang).

Cannot install under macOS due to missing rustfmt

I'm trying to install this, but I'm getting an error:

➜ cargo install vsd
    Updating crates.io index
  Downloaded vsd v0.3.0
  Downloaded 1 crate (63.2 KB) in 1.43s
  Installing vsd v0.3.0
    Updating crates.io index
  Downloaded errno v0.3.8
  Downloaded anstyle-query v1.0.2
  Downloaded darling_core v0.14.4
  Downloaded anstream v0.6.11
  Downloaded convert_case v0.4.0
  Downloaded darling_macro v0.14.4
  Downloaded fixedbitset v0.4.2
  Downloaded byteorder v1.5.0
  Downloaded cpufeatures v0.2.12
  Downloaded darling_macro v0.20.5
  Downloaded derive_builder_macro v0.12.0
  Downloaded untrusted v0.9.0
  Downloaded try-lock v0.2.5
  Downloaded equivalent v1.0.1
  Downloaded futures-sink v0.3.30
  Downloaded clap_lex v0.6.0
  Downloaded system-configuration-sys v0.5.0
  Downloaded time-core v0.1.2
  Downloaded anstyle v1.0.5
  Downloaded utf-8 v0.7.6
  Downloaded xattr v1.3.1
  Downloaded form_urlencoded v1.2.1
  Downloaded glob v0.3.1
  Downloaded socks v0.3.4
  Downloaded data-encoding v2.5.0
  Downloaded terminal_size v0.3.0
  Downloaded ctrlc v3.4.2
  Downloaded thiserror v1.0.56
  Downloaded socket2 v0.5.5
  Downloaded either v1.9.0
  Downloaded tempfile v3.9.0
  Downloaded tracing-core v0.1.32
  Downloaded unicode-ident v1.0.12
  Downloaded unicode-bidi v0.3.15
  Downloaded tungstenite v0.21.0
  Downloaded unicode-segmentation v1.10.1
  Downloaded tokio-util v0.7.10
  Downloaded ureq v2.9.1
  Downloaded time v0.3.31
  Downloaded nix v0.27.1
  Downloaded regex v1.10.3
  Downloaded regex-automata v0.4.5
  Downloaded system-configuration v0.5.1
  Downloaded darling_core v0.20.5
  Downloaded bitflags v2.4.2
  Downloaded tokio v1.35.1
  Downloaded threadpool v1.8.1
  Downloaded anyhow v1.0.79
  Downloaded cc v1.0.83
  Downloaded bytes v1.5.0
  Downloaded signal-hook v0.3.17
  Downloaded prost-build v0.11.9
  Downloaded base64 v0.21.7
  Downloaded serde_derive v1.0.196
  Downloaded security-framework v2.9.2
  Downloaded indexmap v2.2.1
  Downloaded typenum v1.17.0
  Downloaded memchr v2.7.1
  Downloaded crossterm v0.25.0
  Downloaded clap_builder v4.4.18
  Downloaded serde_with v3.6.0
  Downloaded chrono v0.4.33
  Downloaded rustls v0.21.10
  Downloaded rustix v0.38.30
  Downloaded idna v0.5.0
  Downloaded libc v0.2.152
  Downloaded rustls-webpki v0.101.7
  Downloaded encoding_rs v0.8.33
  Downloaded mp4decrypt v0.4.2
  Downloaded kdam_derive v0.1.0
  Downloaded petgraph v0.6.4
  Downloaded hyper v0.14.28
  Downloaded m3u8-rs v5.0.5
  Downloaded ring v0.17.7
  Downloaded vsd-mp4 v0.1.0
  Downloaded headless_chrome v1.0.9
  Downloaded reqwest v0.11.23
  Downloaded auto_generate_cdp v0.4.4
  Downloaded h2 v0.3.24
  Downloaded futures-util v0.3.30
  Downloaded serde_json v1.0.113
  Downloaded hashbrown v0.14.3
  Downloaded mio v0.8.10
  Downloaded http v1.0.0
  Downloaded kdam v0.4.1
  Downloaded http v0.2.11
  Downloaded requestty v0.5.0
  Downloaded serde v1.0.196
  Downloaded proc-macro2 v1.0.78
  Downloaded prettyplease v0.1.25
  Downloaded num-traits v0.2.17
  Downloaded unicode-width v0.1.11
  Downloaded prost-derive v0.11.9
  Downloaded flate2 v1.0.28
  Downloaded ryu v1.0.16
  Downloaded prost-types v0.11.9
  Downloaded prost v0.11.9
  Downloaded parking_lot_core v0.9.9
  Downloaded once_cell v1.19.0
  Downloaded native-tls v0.2.11
  Downloaded jobserver v0.1.27
  Downloaded getrandom v0.2.12
  Downloaded clap v4.4.18
  Downloaded log v0.4.20
  Downloaded ipnet v2.9.0
  Downloaded iana-time-zone v0.1.59
  Downloaded aho-corasick v1.1.2
  Downloaded smawk v0.3.2
  Downloaded slab v0.4.9
  Downloaded signal-hook-registry v1.4.1
  Downloaded signal-hook-mio v0.2.3
  Downloaded shell-words v1.1.0
  Downloaded sha1 v0.10.6
  Downloaded sct v0.7.1
  Downloaded powerfmt v0.2.0
  Downloaded percent-encoding v2.3.1
  Downloaded multimap v0.8.3
  Downloaded futures-io v0.3.30
  Downloaded regex-syntax v0.8.2
  Downloaded clap_derive v4.4.7
  Downloaded derive_builder_core v0.12.0
  Downloaded darling v0.14.4
  Downloaded security-framework-sys v2.9.1
  Downloaded rustix v0.37.27
  Downloaded bento4-src v0.1.2
  Downloaded hyper-tls v0.5.0
  Downloaded httpdate v1.0.3
  Downloaded futures-channel v0.3.30
  Downloaded syn v2.0.48
  Downloaded deranged v0.3.11
  Downloaded anstyle-parse v0.2.3
  Downloaded core-foundation-sys v0.8.6
  Downloaded core-foundation v0.9.4
  Downloaded darling v0.20.5
  Downloaded cookie v0.17.0
  Downloaded webpki-roots v0.25.3
  Downloaded itoa v1.0.10
  Downloaded http-body v0.4.6
  Downloaded derive_builder v0.12.0
  Downloaded quick-xml v0.28.2
  Downloaded itertools v0.10.5
  Downloaded url v2.5.0
  Downloaded fs-err v2.11.0
  Downloaded futures-core v0.3.30
  Downloaded tracing v0.1.40
  Downloaded fastrand v2.0.1
  Downloaded textwrap v0.15.2
  Downloaded quote v1.0.35
  Downloaded home v0.5.9
  Downloaded which v4.4.2
  Downloaded lock_api v0.4.11
  Downloaded requestty-ui v0.5.0
  Downloaded which v5.0.0
  Downloaded pin-project-lite v0.2.13
  Downloaded futures-task v0.3.30
  Downloaded time-macros v0.2.16
  Downloaded spin v0.9.8
  Downloaded serde_with_macros v3.6.0
  Downloaded smallvec v1.13.1
  Downloaded thiserror-impl v1.0.56
  Downloaded terminal_size v0.2.6
  Downloaded tokio-native-tls v0.3.1
  Downloaded unicode-linebreak v0.1.5
  Downloaded 163 crates (17.8 MB) in 2.31s (largest was `ring` at 4.1 MB)
   Compiling libc v0.2.152
   Compiling proc-macro2 v1.0.78
   Compiling unicode-ident v1.0.12
   Compiling autocfg v1.1.0
   Compiling rustix v0.38.30
   Compiling version_check v0.9.4
   Compiling serde v1.0.196
   Compiling cfg-if v1.0.0
   Compiling syn v1.0.109
   Compiling log v0.4.20
   Compiling memchr v2.7.1
   Compiling bitflags v2.4.2
   Compiling bytes v1.5.0
   Compiling itoa v1.0.10
   Compiling typenum v1.17.0
   Compiling generic-array v0.14.7
   Compiling strsim v0.10.0
   Compiling core-foundation-sys v0.8.6
   Compiling fnv v1.0.7
   Compiling ident_case v1.0.1
   Compiling anyhow v1.0.79
   Compiling bitflags v1.3.2
   Compiling pin-project-lite v0.2.13
   Compiling tinyvec_macros v0.1.1
   Compiling tinyvec v1.6.0
   Compiling either v1.9.0
   Compiling serde_json v1.0.113
   Compiling spin v0.9.8
   Compiling errno v0.3.8
   Compiling jobserver v0.1.27
   Compiling mio v0.8.10
   Compiling num_cpus v1.16.0
   Compiling cc v1.0.83
   Compiling unicode-normalization v0.1.22
   Compiling socket2 v0.5.5
   Compiling quote v1.0.35
   Compiling getrandom v0.2.12
   Compiling syn v2.0.48
   Compiling once_cell v1.19.0
   Compiling ring v0.17.7
   Compiling untrusted v0.9.0
   Compiling unicode-bidi v0.3.15
   Compiling tokio v1.35.1
   Compiling itertools v0.10.5
   Compiling slab v0.4.9
   Compiling crc32fast v1.3.2
   Compiling futures-core v0.3.30
   Compiling percent-encoding v2.3.1
   Compiling crypto-common v0.1.6
   Compiling thiserror v1.0.56
   Compiling rustls v0.21.10
   Compiling httparse v1.8.0
   Compiling fastrand v2.0.1
   Compiling adler v1.0.2
   Compiling aho-corasick v1.1.2
   Compiling darling_core v0.14.4
   Compiling tempfile v3.9.0
   Compiling miniz_oxide v0.7.1
   Compiling idna v0.5.0
   Compiling form_urlencoded v1.2.1
   Compiling tracing-core v0.1.32
   Compiling core-foundation v0.9.4
   Compiling lock_api v0.4.11
   Compiling futures-task v0.3.30
   Compiling signal-hook v0.3.17
   Compiling futures-io v0.3.30
   Compiling prost-derive v0.11.9
   Compiling prettyplease v0.1.25
   Compiling serde_derive v1.0.196
   Compiling thiserror-impl v1.0.56
   Compiling heck v0.4.1
   Compiling hashbrown v0.14.3
   Compiling base64 v0.21.7
   Compiling time-core v0.1.2
   Compiling pin-utils v0.1.0
   Compiling byteorder v1.5.0
   Compiling parking_lot_core v0.9.9
   Compiling powerfmt v0.2.0
   Compiling regex-syntax v0.8.2
   Compiling equivalent v1.0.1
   Compiling indexmap v2.2.1
   Compiling deranged v0.3.11
   Compiling socks v0.3.4
   Compiling futures-util v0.3.30
   Compiling time-macros v0.2.16
   Compiling prost v0.11.9
   Compiling darling_macro v0.14.4
   Compiling tracing v0.1.40
   Compiling regex-automata v0.4.5
   Compiling flate2 v1.0.28
   Compiling url v2.5.0
   Compiling darling_core v0.20.5
   Compiling http v0.2.11
   Compiling security-framework-sys v2.9.1
   Compiling signal-hook-registry v1.4.1
   Compiling num-traits v0.2.17
   Compiling ryu v1.0.16
   Compiling futures-sink v0.3.30
   Compiling io-lifetimes v1.0.11
   Compiling auto_generate_cdp v0.4.4
   Compiling scopeguard v1.2.0
   Compiling smallvec v1.13.1
   Compiling webpki-roots v0.25.3
   Compiling home v0.5.9
   Compiling native-tls v0.2.11
   Compiling fixedbitset v0.4.2
   Compiling time v0.3.31
   Compiling which v4.4.2
   Compiling petgraph v0.6.4
   Compiling tokio-util v0.7.10
   Compiling darling_macro v0.20.5
   Compiling security-framework v2.9.2
   Compiling regex v1.10.3
   Compiling rand_core v0.6.4
   Compiling darling v0.14.4
   Compiling prost-types v0.11.9
   Compiling block-buffer v0.10.4
   Compiling block-padding v0.3.3
   Compiling cpufeatures v0.2.12
   Compiling cookie v0.16.2
   Compiling convert_case v0.4.0
   Compiling ppv-lite86 v0.2.17
   Compiling try-lock v0.2.5
   Compiling multimap v0.8.3
   Compiling rustix v0.37.27
   Compiling minimal-lexical v0.2.1
   Compiling system-configuration-sys v0.5.0
   Compiling lazy_static v1.4.0
   Compiling utf8parse v0.2.1
   Compiling anstyle-parse v0.2.3
   Compiling prost-build v0.11.9
   Compiling nom v7.1.3
   Compiling want v0.3.1
   Compiling rand_chacha v0.3.1
   Compiling inout v0.1.3
   Compiling derive_builder_core v0.12.0
   Compiling digest v0.10.7
   Compiling darling v0.20.5
   Compiling signal-hook-mio v0.2.3
   Compiling h2 v0.3.24
   Compiling parking_lot v0.12.1
   Compiling http-body v0.4.6
   Compiling futures-channel v0.3.30
   Compiling idna v0.3.0
   Compiling bento4-src v0.1.2
   Compiling iana-time-zone v0.1.59
   Compiling fs-err v2.11.0
   Compiling unicode-linebreak v0.1.5
   Compiling anstyle v1.0.5
   Compiling httpdate v1.0.3
   Compiling tower-service v0.3.2
   Compiling smawk v0.3.2
   Compiling psl-types v2.0.11
   Compiling anstyle-query v1.0.2
   Compiling unicode-width v0.1.11
   Compiling matches v0.1.10
   Compiling colorchoice v1.0.0
   Compiling textwrap v0.15.2
   Compiling anstream v0.6.11
   Compiling idna v0.2.3
   Compiling publicsuffix v2.2.3
   Compiling mp4decrypt v0.4.2
   Compiling chrono v0.4.33
   Compiling hyper v0.14.28
   Compiling crossterm v0.25.0
   Compiling serde_with_macros v3.6.0
   Compiling sha1 v0.10.6
   Compiling derive_builder_macro v0.12.0
   Compiling cipher v0.4.4
   Compiling vsd-mp4 v0.1.0
   Compiling rand v0.8.5
   Compiling tokio-native-tls v0.3.1
   Compiling http v1.0.0
   Compiling terminal_size v0.3.0
   Compiling cookie v0.17.0
   Compiling clap_lex v0.6.0
   Compiling utf-8 v0.7.6
   Compiling unicode-segmentation v1.10.1
   Compiling data-encoding v2.5.0
   Compiling which v5.0.0
   Compiling tungstenite v0.21.0
   Compiling requestty-ui v0.5.0
   Compiling clap_builder v4.4.18
   Compiling hyper-tls v0.5.0
   Compiling system-configuration v0.5.1
   Compiling cookie_store v0.16.2
   Compiling derive_builder v0.12.0
   Compiling serde_with v3.6.0
   Compiling terminal_size v0.2.6
   Compiling iso8601 v0.6.1
   Compiling tokio-socks v0.5.1
   Compiling quick-xml v0.28.2
   Compiling base64-serde v0.7.0
   Compiling serde_urlencoded v0.7.1
   Compiling quick-xml v0.29.0
   Compiling clap_derive v4.4.7
   Compiling kdam_derive v0.1.0
   Compiling xattr v1.3.1
   Compiling nix v0.27.1
   Compiling encoding_rs v0.8.33
   Compiling shell-words v1.1.0
   Compiling vsd v0.3.0
   Compiling ipnet v2.9.0
   Compiling hex v0.4.3
   Compiling mime v0.3.17
   Compiling clap v4.4.18
   Compiling ctrlc v3.4.2
   Compiling reqwest v0.11.23
   Compiling dash-mpd v0.12.0
   Compiling requestty v0.5.0
   Compiling kdam v0.4.1
   Compiling aes v0.8.3
   Compiling cbc v0.1.2
   Compiling m3u8-rs v5.0.5
   Compiling threadpool v1.8.1
   Compiling glob v0.3.1
   Compiling rustls-webpki v0.101.7
   Compiling sct v0.7.1
   Compiling ureq v2.9.1
   Compiling headless_chrome v1.0.9
error: failed to run custom build command for `headless_chrome v1.0.9`

Caused by:
  process didn't exit successfully: `/var/folders/3h/9h8lqbnx0p372m8qh7fpd1dh0000gn/T/cargo-install4ByRq8/release/build/headless_chrome-64ada925bce50381/build-script-build` (exit status: 101)
  --- stderr
  thread 'main' panicked at /Users/werner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/auto_generate_cdp-0.4.4/src/lib.rs:116:61:
  called `Result::unwrap()` on an `Err` value: Os { code: 2, kind: NotFound, message: "No such file or directory" }
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
error: failed to compile `vsd v0.3.0`, intermediate artifacts can be found at `/var/folders/3h/9h8lqbnx0p372m8qh7fpd1dh0000gn/T/cargo-install4ByRq8`.
To reuse those artifacts with a future compilation, set the environment variable `CARGO_TARGET_DIR` to that path.

I am not familiar with Rust, but looking at the referenced file, I see:

        if env::var_os("DO_NOT_FORMAT").is_none() {
            Command::new("rustfmt").arg(&out_file).output().unwrap();
        }

So when I do:

brew install rustfmt

I can actually continue.
Should this be somehow added to the dependencies or README?

Info:

macOS 13.6.3 (22G436)

➜ brew info rust
==> rust: stable 1.75.0 (bottled), HEAD
Safe, concurrent, practical language
https://www.rust-lang.org/
/opt/homebrew/Cellar/rust/1.74.0 (39,060 files, 951.5MB) *
  Poured from bottle using the formulae.brew.sh API on 2023-12-07 at 09:51:40
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/r/rust.rb
License: Apache-2.0 or MIT
==> Dependencies
Required: libgit2 ✔, libssh2 ✔, llvm ✘, openssl@3 ✔, pkg-config ✔
==> Requirements
Required: macOS >= 10.12 (or Linux) ✔
==> Options
--HEAD
	Install HEAD version
==> Caveats
zsh completions have been installed to:
  /opt/homebrew/share/zsh/site-functions
==> Analytics
install: 35,589 (30 days), 92,337 (90 days), 352,529 (365 days)
install-on-request: 27,213 (30 days), 71,700 (90 days), 266,939 (365 days)
build-error: 141 (30 days)

Bunny CDN's M3U8 playlist links not detected

I am encountering an issue with Bunny CDN where M3U8 links are not being detected.

This page includes a protected Bunny CDN video
And this has an unprotected one

These two pages will help you understand how the videos are delivered

Navigate failed: net::ERR_ABORTED

Hi,

I am trying to use the pre-compiled release on my Mac (with macOS 13.1), to download an HLS stream.

Chrome opens briefly and then immediately I get error: Navigate failed: net::ERR_ABORTED

> ./vsd collect https://test-streams.mux.dev/x36xhzz/url_6/193039199_mp4_h264_aac_hq_7.m3u8
Chrome will launch with a window for 3 minutes.
error: Navigate failed: net::ERR_ABORTED

I also tried with --headless, but with no success

> ➜  ./vsd collect https://test-streams.mux.dev/x36xhzz/url_6/193039199_mp4_h264_aac_hq_7.m3u8 --headless
Chrome will launch without a window for 3 minutes.
error: The event waited for never came

only get to this and then nothing

i login i get to the video nothing happends only puts out this: (the link i put in) is the link i used to get to a video its a place holder here.

vsd capture (the link i put in) --save
INFO sometimes video starts playing but links are not detected
Browser launching in headful (window) mode
Browser setting cookies
Browser registering response listener
Browser navigating to (the link i put in)
INFO waiting for CTRL+C signal

Can collect function passing the headers data?

The collect function only passings the intercepted.request.url data. Is it possible to include the headers data from intercepted.request.headers? Several sites may use cookies to distinguish suspicious requests.

vsd/src/chrome.rs

Line 149 in d234217

sender.lock().unwrap().send(url).unwrap();

References

https://docs.rs/headless_chrome/latest/headless_chrome/protocol/network/struct.Request.html#structfield.headers

Segmentation faults occur very often

Hi, I'm encountering many segmentation fault crashes very very often (it happens 9/10 runs).

Using this command:

vsd save "url" -o file.mp4 --key [redacted key]

This is the usual output:

Run 1:
✔ Select one variant stream: · 720p (3.55 MB/s)
  Encrypted CENC streams found in playlist
Downloading video stream to (video) default (1).m4s
        Key [REDACTED]
 Downloaded video stream successfully
Downloading audio stream to (audio) default.m4s
        Key [REDACTED]
⠦ 2.67 / 2.78 GB ━━━━━━━━━━━━━━╸━━━━━━━━━━━━━━ 50.00% • 4591/9182 • 03:08 > 03:08 • 24.35 SEG/s • 14.54 MB/s
Segmentation fault (core dumped)

Run 2:
✔ Select one variant stream: · 720p (3.55 MB/s)
  Encrypted CENC streams found in playlist
Downloading video stream to (video) default (1).m4s
        Key [REDACTED]
 Downloaded video stream successfully
Downloading audio stream to (audio) default.m4s
        Key [REDACTED]
⠦ 2.67 / 2.78 GB ━━━━━━━━━━━━━━╸━━━━━━━━━━━━━━ 50.00% • 4591/9182 • 03:08 > 03:08 • 24.35 SEG/s • 14.54 MB/s
Segmentation fault (core dumped)

Run 3:
✔ Select one variant stream: · 720p (3.55 MB/s)
  Encrypted CENC streams found in playlist
Downloading video stream to (video) default (2).m4s
        Key [REDACTED]
⠏ 2.43 / 2.76 GB ━━━━━━━━━━━━━╸━━━━━━━━━━━━━━━ 45.76% • 4202/9182 • 02:19 > 02:45 • 30.03 SEG/s • 17.90 MB/s
Segmentation fault (core dumped)

Run 4:
✔ Select one variant stream: · 720p (3.55 MB/s)
  Encrypted CENC streams found in playlist
Downloading video stream to (video) default (3).m4s
        Key [REDACTED]
 Downloaded video stream successfully
Downloading audio stream to (audio) default (1).m4s
        Key [REDACTED]
⠦ 1.33 / 1.40 GB ━━━━━━━━━━━━━━━━━╸━━━━━━━━━━━━ 59.61% • 5473/9182 • 02:03 > 01:23 • 44.23 SEG/s • 2.57 MB/s
Segmentation fault (core dumped)

Detect ffmpeg

I place the url
It lets me choose the quality
Starts downloading it

FUCKING FINALLY. No retarded software that downloads a single .ts and calls it a day. Or only downloads the audio, or the worst possible quality.
THANK YOU FOR CREATING SUCH USEFUL APPLICATION!!!

Just a single suggestion: Maybe the command should check for ffmpeg before starting the downoad if it knows it will needed?
Just to avoid cases like this:

vsd save https://myfile.m3u8 -o video.mp4
• Select streams to download · 1080p 8.81 MiB/s (codecs: avc1.640028,mp4a.40.2, frame_rate: 25) | language: spa | language: spa
 Processing subtitles stream language: spa
Downloading stream to myfile_m3u8.vtt
 Downloaded stream successfully
79.16 KiB ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • 9/9 • 00:01 > 00:00 • 7.61 SEG/s
 Processing video stream 1080p 8.81 MiB/s (codecs: avc1.640028,mp4a.40.2, frame_rate: 25)
Downloading stream to myfile_m3u8.ts
 Downloaded stream successfully
 Processing audio stream language: spa
Downloading stream to myfile_m3u8.aac
 Downloaded stream successfully
2.20 / 2.20 GiB ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100.00% • 520/520 • 02:41 > 00:00 • 3.21 SEG/s • 7.92 MiB/s
  Executing ffmpeg -i myfile_m3u8.ts -i myfile_m3u8.aac -i myfile_m3u8.vtt -c copy -c:s mov_text -map 0 -map 1 -map 2 -metadata:s:a:0 language=spa -metadata:s:s:0 language=spa video.mp4
error: program not found

It downloaded the whole file just for it to end up failing.

Add support for downloading segments without merging them

Add --no-merge flag according to issue #17.

Regarding M3U8 Playlist Generation

Description:
I aimed to "Clone" or download the segments and playlist files as-is without merging them. However, VSD currently downloads and merges the video file into an .mp4 format before creating the HLS segments and playlist file. This approach has led to an issue.

Issue:
I have encountered an issue with the current behaviour of VSD (Video Stream Downloader) when generating M3U8 playlists using the -o option with a .m3u8 file extension. The generated playlist does not list all the individual segment files as expected.

Reproduction Steps:

Use VSD with the -o option to generate an M3U8 playlist with a .m3u8 extension, as follows:
```
vsd save <URL> -o playlist.m3u8 -q highest
```
Check the generated playlist file (playlist.m3u8) to observe the listed segments.

Current Behaviour:
The generated playlist might not list all individual segments correctly, and the media sequence numbers may not align with the actual filenames of the segments. Possibly only last three segments may get listed in playlist

Expected Behaviour:
When generating an M3U8 playlist using the -o option with a .m3u8 extension, it is expected that the playlist should accurately list all the individual segment files and align the media sequence numbers with the actual filenames of the segments.

Environment:

VSD Version: 0.2.5
Operating System: Windows 10

Solution:
I have successfully addressed this issue by using the -hls_time , -hls_list_size & -start_number option with the ffmpeg command when generating the M3U8 playlist, command:

ffmpeg -i video.mp4 -c:v copy -c:a copy -hls_time <segment_duration> -hls_list_size <list_size> -start_number <start_sequence_number> playlist.m3u8

Just for reference this was the command I used:

ffmpeg -i video.mp4 -c:v copy -c:a copy -hls_time 3 -hls_list_size 2000 -start_number 0 video.m3u8

Specifying these parameters along with sequence number is crucial for proper playlist generation. The "video.m3u8" should be at the end after all the options.

fix

Can you fix it? Without adding --no-decrypt, the downloaded files are two mp4 files instead of one audio and one video, causing the decryption to fail when adding the "--key" command.

Temporary file name too long (os error 63)

Seems like the temporary file path is too long even with specified output file name. I'm using apple-darwin binary.

❯ ./vsd -V
vsd 0.2.5

❯ ./vsd save 'https://hlss.club/cdn/hls/0a855362992ad95a6b173a0569da3a24/master.m3u8' -o asd.mp4
Only one variant stream found.
Selected variant stream of quality 720p (3.34 MB/s).
Downloading video stream to vsd_video_Vi9LaGU1dllCcEQ0VEpTb1AzYTRESVBJRXowYm1KMkcyTVhvNHpUdHB5V05GQS95NmNkemkvNmREd3dGcUdMajdFRHdGR0NRYmk1SFM5WFdiS2xsWGZxeWVZRjF6c1ZXREh3ZEFGREVYY2F4dE5kWU9PS3Q0UFNJU2U2N0FWVUl5VU9lc2lzQmJXQmxHb0FDOFMrWStDTHhpUVpQNFFXaXVQNjJLaHZTNmRwOWZaRldNNldaMVgyanQ2ZGhyL1RvSjArcWh4V3A0bmRRV1ZFSTgwVUFwTmF4NmRablFlNE5HQWFGN2FjVXg3V1FSa0tzandmYlV2L01HcCtYM0Z4OEF4bnlBSHQzSkZDdWJLVWNXaTRqdTVHRzV0U3NpZUxrUy80RVQ3ZVB1ZDZ0eEpUVGwxOFRVMW9XS0xnSFBTaVI%3D.mp4
⠋ ? ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.00% • 0/1280 • 00:00 > inf • ? SEG/s • ?error: File name too long (os error 63)

Open SSL issues on macos ventura (pre-build)

I used the install instructions for MacOS and I'm getting the following result:

$ vsd collect "https://example.com/media.m3u8"
dyld[30085]: Library not loaded: /opt/local/libexec/openssl3/lib/libssl.3.dylib
  Referenced from: <74757B88-CD7E-324D-8E72-4B7C313BBCA6> /usr/local/bin/vsd
  Reason: tried: '/opt/local/libexec/openssl3/lib/libssl.3.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/local/libexec/openssl3/lib/libssl.3.dylib' (no such file), '/opt/local/libexec/openssl3/lib/libssl.3.dylib' (no such file), '/usr/local/lib/libssl.3.dylib' (no such file), '/usr/lib/libssl.3.dylib' (no such file, not in dyld cache)
[1]    30085 abort      vsd collect

Never seen this before, so I'm unsure if you need/want any other info than this. When building from source I get the following:

$ RUST_BACKTRACE=1 cargo install vsd                                                                                                                                              
    Updating crates.io index
  Installing vsd v0.2.5
   Compiling libc v0.2.139
   Compiling autocfg v1.1.0
   Compiling proc-macro2 v1.0.50
   Compiling cfg-if v1.0.0
   Compiling quote v1.0.23
   Compiling unicode-ident v1.0.6
   Compiling syn v1.0.107
   Compiling memchr v2.5.0
   Compiling version_check v0.9.4
   Compiling log v0.4.17
   Compiling cc v1.0.78
   Compiling bitflags v1.3.2
   Compiling serde_derive v1.0.152
[.... ]
   Compiling derive_builder v0.12.0
   Compiling auto_generate_cdp v0.4.0
   Compiling requestty v0.5.0
   Compiling headless_chrome v1.0.1
error: failed to run custom build command for `headless_chrome v1.0.1`

Caused by:
  process didn't exit successfully: `/var/folders/0y/42sfx52d50d3g62rxgf9rsxm0000gn/T/cargo-install9bwt9I/release/build/headless_chrome-a638f6c2ef4c7bc0/build-script-build` (exit status: 101)
  --- stderr
  thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 2, kind: NotFound, message: "No such file or directory" }', /Users/xvilo/.cargo/registry/src/github.com-1ecc6299db9ec823/auto_generate_cdp-0.4.0/src/lib.rs:116:57
  stack backtrace:
     0: _rust_begin_unwind
     1: core::panicking::panic_fmt
     2: core::result::unwrap_failed
     3: core::result::Result<T,E>::unwrap
     4: auto_generate_cdp::init
     5: build_script_build::main
     6: core::ops::function::FnOnce::call_once
  note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
warning: build failed, waiting for other jobs to finish...
error: failed to compile `vsd v0.2.5`, intermediate artifacts can be found at `/var/folders/0y/42sfx52d50d3g62rxgf9rsxm0000gn/T/cargo-install9bwt9I`

Download video from site that requires login

I try to add session cookie through headers, but it does not work:

vsd save https://<some site>.m3u8 -o <output>.mp4 --header cookie <a value I copy from request header after login into the site>
error: Couldn't parse https://<some site>.m3u8 playlist.

I would like to know at least if this is because of Unauthorized response from server, but I don't see any --verbose option.

Also note that doing normal http content scraping I use with ruby open-uri and nokogiri works on that site when I pass the same header as in the command above to my scripts. Also I don't want to use collect or capture and manually login every time, because there are hundreds of videos and I would rather download them using a script. Any ideas?

clitic / vsd Goto Github PK

vsd's People

Contributors

Stargazers

Watchers

Forkers

vsd's Issues

description

command

error

tech stack

vsd

google-chrome

ffmpeg

cargo

os

another

also, gui version of the google-chrome usually works within wsl 2

description

Recommend Projects

Recommend Topics

Recommend Org