Giter VIP home page Giter VIP logo

yaydl's Introduction

yaydl

Crates.io

yet another youtube (and more) down loader

% yaydl "https://www.youtube.com/watch?v=jNQXAC9IVRw"

How? What? Why?

% yaydl --help

Features

  • Can download videos.
  • Can optionally keep only the audio part of them.
  • Could convert the resulting file to something else (requires the ffmpeg binary).
  • Comes as a single binary (once compiled) - take it everywhere on your thumbdrive, no Python cruft required.

Currently supported sites

  • porndoe.com · spankbang.com · vidoza.net · vimeo.com · vivo.sx · voe.sx · watchmdh.to · xhamster.com · youtube.com

There is an easy way to add more supported sites, see below for details.

Non-features

The list of features is deliberately kept short:

  • No output quality choice. yaydl assumes that you have a large hard drive and your internet connection is good enough, or else you would stream, not download.
  • No complex filters. This is a downloading tool.
  • No image file support. Videos only.

How to install

From the source code

Install Rust (e.g. with rustup), then:

using Fossil:

% fossil clone https://code.rosaelefanten.org/yaydl
% cd yaydl
% cargo build --release

using Git:

% git clone https://github.com/dertuxmalwieder/yaydl
% cd yaydl
% cargo build --release

From Cargo

% cargo install yaydl

From your package manager

pkgsrc (with pkg_add):

% pkg_add yaydl

pkgsrc (with pkgin):

% pkgin install yaydl

Other package managers:

  • Nobody has provided any other packages for yaydl yet. You can help!

How to use the web driver (very beta, at your own risk!)

For some video sites, yaydl needs to be able to parse a JavaScript on them. For this, it needs to be able to spawn a headless web browser. It requires Google Chrome, Microsoft Edge or Mozilla Firefox to be installed and running on your system.

  1. Install and run ChromeDriver (if you use Chrome), the typically named Microsoft Edge WebDriver (if you use Edge) or geckodriver (if you use Firefox) for your platform.
  2. Tell yaydl that you have a web driver running: yaydl --webdriver <port> <video URL>. (The drivers usually run on port 4444 or 9515, please consult their documentation if you are not sure.) Hint: If you need this feature regularly, you can also use the environment variable YAYDL_WEBDRIVER_PORT to set the port number for all further requests.
  3. In theory, it should be possible to use more sites with yaydl now. :-)

How to contribute code

  1. Read and agree to the Code of Conduct Merit.
  2. Implicitly agree to the LICENSE. Nobody reads those. I don't either.
  3. Find out if anyone has filed a GitHub Issue or even sent a Pull Request yet. Act accordingly.
  4. Send me a patch, either via e-mail (yaydl at tuxproject dot de), on the IRC or as a GitHub Pull Request. Note that GitHub only provides a mirror, so you'd double my work if you choose the latter. :-)

If you do that well (and regularly) enough, I'll probably grant you commit access to the upstream Fossil repository.

Add support for new sites

  1. Implement definitions::SiteDefinition as handlers/<YourSite>.rs.
  2. Push the new handler to the inventory: inventory::submit! { &YourSiteHandler as &dyn SiteDefinition }
  3. Add the new module to handlers.rs.
  4. Optionally, add new requirements to Cargo.toml.
  5. Send me a patch, preferably with an example. (I cannot know all sites.)

Minimal example that does nothing

// handlers/noop.rs

use anyhow::Result;

use crate::definitions::SiteDefinition;
use crate::VIDEO;

struct NoopExampleHandler;
impl SiteDefinition for NoopExampleHandler {
    // Parameters sent to the handler by yaydl:
    // - url:            The video page's URL.
    // - webdriver_port: The port that runs the WebDriver client.
    //                   Defaults to 0 if there is no WebDriver configured.
    // - onlyaudio:      true if only the audio part of the video should be
    //                   kept, else false.
    fn can_handle_url<'a>(&'a self, url: &'a str, webdriver_port: u16) -> bool {
        // Return true here if <url> can be covered by this handler.
        // Note that yaydl will skip all other handlers then.
        true
    }

    fn does_video_exist<'a>(&'a self, video: &'a mut VIDEO, url: &'a str, webdriver_port: u16) -> Result<bool> {
        // Return true here, if the video exists.
        Ok(false)
    }

    fn is_playlist<'a>(&'a self, url: &'a str, webdriver_port: u16) -> Result<bool> {
    	// Return true here, if the download link is a playlist.
    	Ok(false)
    }

    fn find_video_title<'a>(&'a self, video: &'a mut VIDEO, url: &'a str, webdriver_port: u16) -> Result<String> {
        // Return the video title from <url> here.
        Ok("".to_string())
    }

    fn find_video_direct_url<'a>(&'a self, video: &'a mut VIDEO, url: &'a str, webdriver_port: u16, onlyaudio: bool) -> Result<String> {
        // Return the direct download URL of the video (or its audio version) here.
        // Exception: If is_playlist() is true, return the playlist URL here instead.
        Ok("".to_string())
    }

    fn find_video_file_extension<'a>(&'a self, video: &'a mut VIDEO, url: &'a str, webdriver_port: u16, onlyaudio: bool) -> Result<String> {
        // Return the designated file extension of the video (or audio) file here.
        Ok("mp4".to_string())
    }

    fn display_name<'a>(&'a self) -> String {
        // For cosmetics, this is the display name of this handler.
        "NoopExample"
    }

    fn web_driver_required<'a>(&'a self) -> bool {
        // Return true here, if the implementation requires a web driver to be running.
        false
    }
}

// Push the site definition to the list of known handlers:
inventory::submit! {
    &NoopExampleHandler as &dyn SiteDefinition
}

Fix some bugs or add new features

  1. Do so.
  2. Send me a patch.

Donations

Writing this software and keeping it available is eating some of the time which most people would spend with their friends. Naturally, I absolutely accept financial compensation.

Thank you.

Contact

yaydl's People

Contributors

egdv avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

yaydl's Issues

Youtube regex parse error

I have a simple fix for this error. Provide access to upstream and I can test direct upstream update as you have suggested?

     Running `target\debug\yaydl.exe https://youtu.be/jNQXAC9IVRw`
Fetching from YouTube.
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Syntax(
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
regex parse error:
    (?:v=|\.be\/)(.*?)(&.*)*$
              ^^
error: unrecognized escape sequence
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
)', src\handlers\youtube.rs:174:65

or fix it by deleting a char

let id_regex = Regex::new(r"(?:v=|\.be\/)(.*?)(&.*)*$").unwrap();
                                      ^--- fix: delete this backslash 1 char before e 

Keep file on error

There is this msg which is same as youtube-dl when a download from music.youtube.com is finished :
'list' is not recognized as an internal or external command, operable program or batch file.

youtube-dl keeps the file whereas yay-dl doesn't (even though the download is successful).

Please keep it/ add a flag to allow keeping it :)

Unable to download any video from YouTube

No matter what YouTube link is used, the following error occurs.
The video could not be found. Invalid link?
I suspect this may have something to do with the API key as the same version (0.12.2) was working approximately a week or so ago.

yaydl can't download from YouTube playlists yet.

Here's a link I tried downloading:

https://www.youtube.com/watch?v=F8sZRBdmqc0&list=WL&index=10&t=1040s

yaydl can't parse this. It's a bit of a complicated link because it's part of a playlist and there's a timestamp attached. However, it can parse this link

https://www.youtube.com/watch?v=F8sZRBdmqc0

I think with a little bit of regex, the extractor could parse these links better. Probably in the future, it would be good to parse it and recognize it's part of a list and give the option to download the whole playlist, but an easy solution for now is to throw everything past the watch ID out and send it to the downloader. The timestamp can probably be thrown out in almost all cases.

yaydl does not support playlists yet, resulting in weird errors.

hi,

i came across your project when i was looking for a command line tool for video download from voe.sx. i tried to download a voe.sx and youtube video, but unfortunately it doesn't seem to work:

./yaydl https://www.youtube.com/watch?v=dd-dZJYwz4E
Fetching from YouTube.
Error: https://r4---sn-4g5ednss.googlevideo.com/videoplayback?expire=1620331434&ei=SveTYP3ILdWBpATu8qmYBA&ip=5.146.60.98&id=o-AG6g819X4du50aR2CnKMZVNRXOXHFy72i8kg0_DNq4ED&itag=22&source=youtube&requiressl=yes&mh=e6&mm=31%2C26&mn=sn-4g5ednss%2Csn-5hne6nsz&ms=au%2Conr&mv=m&mvi=4&pl=22&initcwndbps=1472500&vprv=1&mime=video%2Fmp4&ns=1vURrODGNxa3BwdwimkfATMF&cnr=14&ratebypass=yes&dur=2316.260&lmt=1620244524736849&mt=1620309655&fvip=4&fexp=24001373%2C24007246&c=WEB&txp=6316222&n=L-CEZ-2WwCP4G5BvM&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cvprv%2Cmime%2Cns%2Ccnr%2Cratebypass%2Cdur%2Clmt&sig=AOq0QJ8wRQIhAMyUVWpPMOlChPHKyjgbsy498jNyksonQD1IAu0F2J-7AiAUs1pwUyg7OgBtrrwloAkAgLxwTpDX2pmKRRF8wbTAXw%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRAIgdNrBNwDgmnVEjAakEClTSIzMso59awCNczx0YUC3b5ECIAD2FVwkQOoQOd7-MIbEBjnRHVwee-K9Gje74krBhk2q: 
Dns Failed: failed to lookup address information: Name or service not known
./yaydl "https://voe.sx/5mqmzgck2jx3#tabs-482b6qc1r"
Fetching from Voe.
The video could not be found. Invalid link?

0.12.2 build failure

Updating NetBSD package to 0.12.2 fails with:

   Compiling yaydl v0.12.2 (/usr/pkgsrc/wip/yaydl/work/yaydl-ee8bc6c2c1)
error[E0423]: expected value, found crate `url`
  --> src/handlers/youtube.rs:32:54
   |
32 |         if let Some(env_proxy) = env_proxy::for_url(&url).host_port() {
   |                                                      ^^^ not a value

error[E0433]: failed to resolve: use of undeclared crate or module `agent`
  --> src/handlers/youtube.rs:39:19
   |
39 |         let req = agent::post(&video_url).send_json(ureq::json!({
   |                   ^^^^^ use of undeclared crate or module `agent`
   |
help: you may have meant to call an instance method
   |
39 |         let req = agent.post(&video_url).send_json(ureq::json!({
   |                        ~

Some errors have detailed explanations: E0423, E0433.
For more information about an error, try `rustc --explain E0423`.
error: could not compile `yaydl` due to 2 previous errors
*** Error code 101

Stop.

NetBSD-current
Rust-1.68.2

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.