CVE-2021-33503 - High Severity Vulnerability

Vulnerable Library - urllib3-1.25.11-py2.py3-none-any.whl

HTTP library with thread-safe connection pooling, file post, and more.

Library home page: https://files.pythonhosted.org/packages/56/aa/4ef5aa67a9a62505db124a5cb5262332d1d4153462eb8fd89c9fa41e5d92/urllib3-1.25.11-py2.py3-none-any.whl

Path to dependency file: m3u8-dl/requirements.txt

Path to vulnerable library: m3u8-dl/requirements.txt,m3u8-dl

Dependency Hierarchy:

• requests-2.24.0-py2.py3-none-any.whl (Root Library)
  • ❌ urllib3-1.25.11-py2.py3-none-any.whl (Vulnerable Library)

Found in HEAD commit: 1b52e9c2aec378789d5de78f26a0b6a5ad192c19

Found in base branch: master

Vulnerability Details

An issue was discovered in urllib3 before 1.26.5. When provided with a URL containing many @ characters in the authority component, the authority regular expression exhibits catastrophic backtracking, causing a denial of service if a URL were passed as a parameter or redirected to via an HTTP redirect.

Publish Date: 2021-06-29

URL: CVE-2021-33503

CVSS 3 Score Details (7.5)

Base Score Metrics:

• Exploitability Metrics:
  • Attack Vector: Network
  • Attack Complexity: Low
  • Privileges Required: None
  • User Interaction: None
  • Scope: Unchanged
• Impact Metrics:
  • Confidentiality Impact: None
  • Integrity Impact: None
  • Availability Impact: High

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: GHSA-q2q7-5pp4-w6pg

Release Date: 2021-05-22

Fix Resolution: urllib3 - 1.26.5

Step up your Open Source Security Game with WhiteSource here

Add instructions on how to globally install m3u8-dl such that it can be started from command-prompt/powershell itself. An exe can be compiled using pyinstaller main.py --name m3u8-dl --onefile -p <venv-name>/lib/python3.*/site-packages/

Is your feature request related to a problem? Please describe.
It is very tedious to search for link and headers so we will try to have a generic header which will work for most of the cases

Describe the solution you'd like
So i will be desiging a generic header which will be deault header if the user is not providing the headers themselves and these headers will be generic for most the cases so it will improve overall user experience for the user.

Describe alternatives you've considered
I have considered using requests with original header of request and no headers at all , providing original request header are good but can be tough to find everytime and passing no headers will cause issue in most of the cases as the server is expecting request headers which in turn when not received may fail the download process.

Additional context

Describe the bug
Hello,

I tried to use your application to download an m3u8 playlist, but it seems that the code is failing in the download step. Apparently the method sched_getaffinity doesn't exist in module os on macOS.

press ctrl+c or ctrl+z if parsed headers of type http2 False are incorrect.
 
Starting Download process MainProcess. 
Traceback (most recent call last):  
  File "/Users/dayvsonsales/m3u8-dl/core/download_process.py", line 27, in download_process. 
    download_manager = DownloadProcess(links, total_links, session, http2,  
  File "/Users/dayvsonsales/m3u8-dl/core/download_process.py", line 55, in __init__. 
    self.__process_num = 4 if platform.system() == "Windows" else   len(os.sched_getaffinity(os.getpid())). 
AttributeError: module 'os' has no attribute 'sched_getaffinity'

To Reproduce
Steps to reproduce the behavior:

1. python3 main.py https://<mywebsite>.com/scripts/m3us/playlist.m3u
2. See error

Expected behavior
I expected that my playlist should be downloaded.

Desktop (please complete the following information):

• OS: macOS
• Version 10.13.6
• python version: Python 3.8.5

Adding support for encryption keys

The encrypted TS segments may require decryption. Adding AES-128 and other decryption methods will broaden its applications in downloading videos from video sharing platforms. Each key has three properties: method, uri, and iv. Sometimes the referrer link and the cache is also required to have access to the content.

Dependabot couldn't authenticate with https://pypi.python.org/simple/.

You can provide authentication details in your Dependabot dashboard by clicking into the account menu (in the top right) and selecting 'Config variables'.

View the update logs.

CVE-2020-26137 - High Severity Vulnerability

Vulnerable Library - urllib3-1.25.8-py2.py3-none-any.whl

HTTP library with thread-safe connection pooling, file post, and more.

Library home page: https://files.pythonhosted.org/packages/e8/74/6e4f91745020f967d09332bb2b8b9b10090957334692eb88ea4afe91b77f/urllib3-1.25.8-py2.py3-none-any.whl

Path to dependency file: m3u8-dl/requirements.txt

Path to vulnerable library: m3u8-dl/requirements.txt

Dependency Hierarchy:

• requests-2.24.0-py2.py3-none-any.whl (Root Library)
  • ❌ urllib3-1.25.8-py2.py3-none-any.whl (Vulnerable Library)

Found in HEAD commit: 92b1ee3e3d92237914f49d7cfcd9296c69ea4984

Found in base branch: master

Vulnerability Details

urllib3 before 1.25.9 allows CRLF injection if the attacker controls the HTTP request method, as demonstrated by inserting CR and LF control characters in the first argument of putrequest(). NOTE: this is similar to CVE-2020-26116.

Publish Date: 2020-09-30

URL: CVE-2020-26137

CVSS 3 Score Details (7.4)

Base Score Metrics:

• Exploitability Metrics:
  • Attack Vector: Network
  • Attack Complexity: High
  • Privileges Required: None
  • User Interaction: None
  • Scope: Unchanged
• Impact Metrics:
  • Confidentiality Impact: None
  • Integrity Impact: High
  • Availability Impact: High

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-26137

Release Date: 2020-09-30

Fix Resolution: 1.25.9

Step up your Open Source Security Game with WhiteSource here

Is your feature request related to a problem? Please describe.
Adding a cli -t option will user to customize and scale the download speed as per their needs

Describe the solution you'd like
We can add a cli option -t that on invokation will look like m3u8-dl <url-of-playlist> -t 8 this would mean starting the download processes with 8 threads per process, so if there are 4 download processes there will be a total of 32 threads.

Is your feature request related to a problem? Please describe.
Addition of support to display options to handle if a user wants to download all the listed playlists or select specific lists to download.

Describe the solution you'd like
expected functionality:-

Case 1: if user dosen't want to download all the listed playlists.

2 playlist were found while parsing the provided playlist

would you like to download all the listed playlist (y/n): n

select the playlists that you would want to download:

do you want to download list1.m3u8 (y/n): n
do you want to download list2.m3u8 (y/n): y

downloading selected lists

Case 2: if user wants to download all the listed playlists.

2 playlist were found while parsing the provided playlist

would you like to download all the listed playlist (y/n): y

downloading selected lists

Additional context
All m3u8 related changes should be added in the m3u8lib directory.

Is your feature request related to a problem? Please describe.
Addition of a -c cli option to keep the cached download and redirect it to a new location

Describe the solution you'd like
On specifying the -c option the command should look like this m3u8-dl <url-of-playlist> -c <absolute/relative path of new location directory> the specified directory must be a valid path.

Is your feature request related to a problem? Please describe.
Addition of a -f option will allow users to read the m3u8 playlist from a file if a file is available in the local filesystem, this will allow users to pass in the file path if the playlist has already been downloaded.

Describe the solution you'd like
We can add a cli option -f that on invokation will look like m3u8-dl -f <relative/absolute path-of-playlist> this would allow users to read the playlist from a file, all other functionality will work AS-IS.

Additional context
The python library m3u8 maybe used to accomplish this task https://pypi.org/project/m3u8/

Is your feature request related to a problem? Please describe.
Addition of support to convert to other popular video formats will provide flexiblity to the user.

Describe the solution you'd like
expected behaviour if user wants the video output to be of type mkv:-

The command should look like this after addition of --convert support m3u8-dl <url-of-playlist> -c mkv on running it should output a .mkv file.

if no file type is specified default to mp4

Additional Context
You can refer to https://ffmpeg.org/general.html#Video-Codecs to see the formats and codecs supported by ffmpeg

Describe the bug
While i am running the main.py file by taking fmovies link the download process,video handling process and producer process are getting started but suddenly the process is terminated due to stream reset error "hyper.http20.exceptions.StreamResetError: (Stream forcefully closed)".

To Reproduce
Steps to reproduce the behavior:

1. Go to online video streaming website "https://ww5.fmovie.sc/"
2. Play online streaming video then right click on the page and click on inspect.
3. Go to Network option and got to XHR, u can see several names.
   4.Click on .m3u8 extension link and on the right scroll down to copy the request headers to header.txt file in the project and save it
4. Scroll up and copy the url and run > python main.py url -c
5. See error

Expected behavior
what actually happens if there is no bug is the parser will parse all the links and download the chunks of the video in m3u8 format and later it is converted into mp4 format and into a single video.

Screenshots

Desktop (please complete the following information):

• OS: Windows
• Browser : Chrome
• Version Chrome: 83

Is your feature request related to a problem? Please describe.
Addition of a progress bar will help users get a sense of how much data has been downloaded and how much data is left to be downloaded

Describe the solution you'd like
Using a library like progress from pypi https://pypi.org/project/progress/, one can identify the areas where the downloaded chunks are tracked and record it using the library.

Is your feature request related to a problem? Please describe.
Addition of a -m option will allow users to specify a custom number of parallel processes to run to scale as per need.

Describe the solution you'd like
We can add a cli option -m that on invokation will look like m3u8-dl <url-of-playlist> -m 8 this would mean that there will be 8 download processes, so if each process has 4 threads there will be a total of 32 threads.

Description
Due to the hyper library not being maintained for the past few years, some http/2.0 requests throw errors on issues that are yet to be fixed in hyper library.

Solution
Integrate an actively maintained library like httpx to support http/2.0 requests, and deprecate hyper library usage.

Describe alternatives you've considered
Alternatives for making http/2.0 requests so far is the requests3 library, but it hasn't been released so far.

Add global installation instruction for macos, such that m3u8-dl can be started from the terminal itself. An exe can be compiled using pyinstaller main.py --name m3u8-dl --onefile -p <venv-name>/lib/python3.*/site-packages/

Describe the bug
m3u8-dl does not terminate when an internal process crashes due to an error. This can happen if any of the processes terminate abnormally.

Expected behavior
m3u8-dl should terminate correctly without getting stuck in an infinite loop.