huxuan / iptvtools Goto Github PK

View Code? Open in Web Editor NEW

173.0 17.0 50.0 52 KB

A set of scripts that help to better the IPTV experience.

Home Page: https://iptvtools.readthedocs.io/

License: MIT License

Makefile 5.32% Python 94.68%

iptv iptv-m3u iptv-playlist iptv-channels python python3 python-script m3u

iptvtools's Introduction

IPTVTools

Scripts currently provided:

iptv-filter
- Merge from different resources.
- Check the tcp/udp connectivity.
- Filter by custom criteria, e.g. resolution.
- Match with templates and EPG.
- Format the url with UDPxy if provided.
- Unify channels' titles.

Features planned on the road:

Scan certain ip and port range to find new channels.
Establish a lightweight database for routine maintenance.

Besides, all scripts should be lightweight and able to keep running regularly after proper configuration.

Last but not least, any ideas, comments and suggestions are welcome!

Prerequisites

To filter by stream information, e.g., resolution/height, ffmpeg (or ffprobe more precisely) is needed, please install according to the documentation.

Installation

pip install -U iptvtools

Usage

Please refer to the documentation while some useful information in wiki.

iptvtools's People

Contributors

Stargazers

Watchers

iptvtools's Issues

Check whether ffprobe/ffmpeg is installed when specifying `--min-height` option and show necessary help message.

Can't run script on raspberry pi

When I try to run the script I get:
pi@raspberrypi2:~/.local/bin $ ./iptv-filter Traceback (most recent call last): File "./iptv-filter", line 7, in <module> from iptvtools.iptv_filter import main File "/home/pi/.local/lib/python3.5/site-packages/iptvtools/iptv_filter.py", line 14, in <module> from .constants import helps File "/home/pi/.local/lib/python3.5/site-packages/iptvtools/constants/helps.py", line 12 f'Configuration file to unify title and id information, defaults to ' ^ SyntaxError: invalid syntax

Iptv

Καλησπέρα μοωρειτε να μου πείτε πως να δω κανάλια συνδρομιτιξσ

ValueError: invalid literal for int() with base 10

Running on Windows 10, using Python 3.8.2. Trying to use on iptv-org.github.io m3us, but to no avail.

C:\Users\Coloradohusky\Downloads\IPTV>iptv-filter -i https://iptv-org.github.io/iptv/languages/cat.m3u
https://videos.rtva.ad/live/rtva/playlist.m3u8, Inaccessible (No connectivity)!
http://ventdelnord.tv:8080/girona/directe.m3u8, Inaccessible (No connectivity)!
http://ventdelnord.tv:8080/escala/directe.m3u8, Inaccessible (No connectivity)!
http://5.255.90.184:2002/play/a01z, Inaccessible (No connectivity)!
https://cdnlive.shooowit.net/25televisiolive/smil:channel1.smil/master.m3u8, OK!
https://cdnapisec.kaltura.com/p/2346171/sp/234617100/playManifest/entryId/1_n6442jz0/format/applehttp/.m3u8?referrer=aHR0cHM6Ly9iZXRldmUuY2F0, OK!
https://bcovlive-a.akamaihd.net/1e7e91116b104391a4f22e13a694d94f/eu-central-1/6057955885001/playlist_dvr.m3u8, Inaccessible (No connectivity)!
100%|####################################################################################| 7/7 [00:31<00:00,  4.49s/it]
Traceback (most recent call last):
  File "C:\Users\Coloradohusky\AppData\Local\Programs\Python\Python38\lib\runpy.py", line 193, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\Coloradohusky\AppData\Local\Programs\Python\Python38\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Users\Coloradohusky\AppData\Local\Programs\Python\Python38\Scripts\iptv-filter.exe\__main__.py", line 7, in <module>
  File "C:\Users\Coloradohusky\AppData\Local\Programs\Python\Python38\lib\site-packages\iptvtools\iptv_filter.py", line 75, in main
    playlist.export()
  File "C:\Users\Coloradohusky\AppData\Local\Programs\Python\Python38\lib\site-packages\iptvtools\models.py", line 42, in export
    for url in sorted(self.data, key=self.__custom_sort):
  File "C:\Users\Coloradohusky\AppData\Local\Programs\Python\Python38\lib\site-packages\iptvtools\models.py", line 175, in __custom_sort
    res.append(int(entry['params'].get(key) or sys.maxsize))
ValueError: invalid literal for int() with base 10: '25TV.es'

The line in question (see 'tvg-id'):

#EXTINF:-1 tvg-id="25TV.es" tvg-logo="https://i.imgur.com/sADbx7S.png" group-title="Undefined",25 TV (480p) [Not 24/7]
https://cdnlive.shooowit.net/25televisiolive/smil:channel1.smil/master.m3u8

Add option to append resolution to channel name.

skip connectivity check

primary for debugging purposes (check if the exclude/unify work as we want) can we have a flag that skip the connectivity check assuming all channels valid and just doing the m3u parsing (Unify channels' titles, filtering groups and channels etc.)

AttributeError: module 'socket' has no attribute 'SO_REUSEPORT'. Did you mean: 'SO_REUSEADDR'?

Trying out your example given in the documentation gives me this error:

C:\Users\Coloradohusky\Downloads\IPTV>iptv-filter -i https://gist.githubusercontent.com/sdhzdmzzl/93cf74947770066743fff7c7f4fc5820/raw/11107d2dcfe2f5785e7ada94bb44c0cd349191c5/bj-unicom-iptv.m3u -t http://epg.51zmt.top:8000/test.m3u
  0%|                                                                                          | 0/165 [00:01<?, ?it/s]
Traceback (most recent call last):
  File "C:\Program Files\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Program Files\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Users\Coloradohusky\AppData\Roaming\Python\Python310\Scripts\iptv-filter.exe\__main__.py", line 7, in <module>
  File "C:\Users\Coloradohusky\AppData\Roaming\Python\Python310\site-packages\iptvtools\iptv_filter.py", line 74, in main
    playlist.filter()
  File "C:\Users\Coloradohusky\AppData\Roaming\Python\Python310\site-packages\iptvtools\models.py", line 160, in filter
    elif not utils.check_connectivity(url, self.args.timeout):
  File "C:\Users\Coloradohusky\AppData\Roaming\Python\Python310\site-packages\iptvtools\utils.py", line 98, in check_connectivity
    return check_udp_connectivity(parsed_url.netloc, timeout)
  File "C:\Users\Coloradohusky\AppData\Roaming\Python\Python310\site-packages\iptvtools\utils.py", line 108, in check_udp_connectivity
    sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
AttributeError: module 'socket' has no attribute 'SO_REUSEPORT'. Did you mean: 'SO_REUSEADDR'?

Windows 10, Python 3.10.6.

custom sorting

would be nice to have a way to set a custom order for some channels... i.e.
this is the starting list (assume this is the channel name, i stripped all other m3u values...)

PIPPO 2
ZAZZA 3
PIPPO 1
LALALA 3
ZAZZA 1
LALALA
LILLOZAZZA 1

Setting the flags like -R -s [['^ZAZZA','^LALALA$'], 'tvg-id', 'title', 'height']

Will generate this list:

ZAZZA 1 [720p]
ZAZZA 3 [1080p]
LALALA [1080p]
PIPPO 1 [1080p]
PIPPO 2 [1080p]
LALALA 3 [720p]
LILLOZAZZA 1 [720p]

TypeError: expected string or bytes-like object

运行示例出现如下错误，windows python 3.10

Traceback (most recent call last):
File "C:\Python310\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Python310\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "C:\Python310\Scripts\iptv-filter.exe_main.py", line 7, in
File "C:\Python310\lib\site-packages\iptvtools\iptv_filter.py", line 75, in main
playlist.export()
File "C:\Python310\lib\site-packages\iptvtools\models.py", line 42, in export
for url in sorted(self.data, key=self.__custom_sort):
File "C:\Python310\lib\site-packages\iptvtools\models.py", line 175, in __custom_sort
res.append(int(re.sub("\D", "", entry['params'].get(key)) or sys.maxsize))
File "C:\Python310\lib\re.py", line 209, in sub
return _compile(pattern, flags).sub(repl, string, count)
TypeError: expected string or bytes-like object

NL character cause AttributeError: 'NoneType' object has no attribute 'groupdict'

Hello, i need to manage an m3u list that have a \n in channel title.. this breaks the regex/perline parsing and make the tool unusable

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.9/bin/iptv-filter", line 8, in <module>
    sys.exit(main())
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/iptvtools/iptv_filter.py", line 73, in main
    playlist.parse()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/iptvtools/models.py", line 68, in parse
    self._parse(self.args.inputs)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/iptvtools/models.py", line 95, in _parse
    current_item = parsers.parse_tag_inf(line)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/iptvtools/parsers.py", line 35, in parse_tag_inf
    res = match.groupdict()
AttributeError: 'NoneType' object has no attribute 'groupdict'

Here a test case, put it in an m3u file and try to use the tool on it

#EXTM3U
#EXTINF:-1 tvg-id="" tvg-name="broken channel
name" tvg-logo="" group-title="US",broken channel
name
http://blabla.com/11111
#EXTINF:-1 tvg-id="" tvg-name="
" tvg-logo="" group-title="US",

http://blabla.com/2222
#EXTINF:-1 tvg-id="" tvg-name="good channel name" tvg-logo="https://logo.com/logo.png" group-title="USA",good channel name
http://blabla.com/3333

can we fix this somehow?
this is a similar issue as #16

filter by group

sometime i manage big list where i just want to get few group, is possible to add a flag to add a whitelist of wanted groups and ignore all other?

basically embed what this project do: https://github.com/dupontbenoit/iptv-m3u

Discussion

ubuntu@raspberrypi:~/iptv/iptvtools$ git status
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean
ubuntu@raspberrypi:~/iptv/iptvtools$ git config --list
core.repositoryformatversion=0
core.filemode=true
core.bare=false
core.logallrefupdates=true
remote.origin.url=https://github.com/huxuan/iptvtools.git
remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
branch.master.remote=origin
branch.master.merge=refs/heads/master
ubuntu@raspberrypi:~/iptv/iptvtools$ iptv-filter -v
0.1.8.dev1+g6f56d69
ubuntu@raspberrypi:~/iptv/iptvtools$ iptv-filter -i https://gist.githubusercontent.com/sdhzdmzzl/93cf74947770066743fff7c7f4fc5820/raw/0be4160f4024320f23daad65bce79e606da47995/bj-unicom-iptv.m3u -t http://epg.51zmt.top:8000/test.m3u --min-height 1080 -u http://192.168.50.1:8888
Retrieving playlists from url: https://gist.githubusercontent.com/sdhzdmzzl/93cf74947770066743fff7c7f4fc5820/raw/0be4160f4024320f23daad65bce79e606da47995/bj-unicom-iptv.m3u
Traceback (most recent call last):
  File "/usr/lib/python3.7/urllib/request.py", line 1317, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/usr/lib/python3.7/http/client.py", line 1252, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.7/http/client.py", line 1298, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.7/http/client.py", line 1247, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.7/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/usr/lib/python3.7/http/client.py", line 966, in send
    self.connect()
  File "/usr/lib/python3.7/http/client.py", line 1414, in connect
    super().connect()
  File "/usr/lib/python3.7/http/client.py", line 938, in connect
    (self.host,self.port), self.timeout, self.source_address)
  File "/usr/lib/python3.7/socket.py", line 727, in create_connection
    raise err
  File "/usr/lib/python3.7/socket.py", line 716, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/.local/bin/iptv-filter", line 10, in <module>
    sys.exit(main())
  File "/home/ubuntu/.local/lib/python3.7/site-packages/iptvtools/iptv_filter.py", line 52, in main
    playlist.parse(args)
  File "/home/ubuntu/.local/lib/python3.7/site-packages/iptvtools/models.py", line 58, in parse
    self._parse(args.input, udpxy=args.udpxy)
  File "/home/ubuntu/.local/lib/python3.7/site-packages/iptvtools/models.py", line 64, in _parse
    lines = parsers.parse_content_to_lines(source)
  File "/home/ubuntu/.local/lib/python3.7/site-packages/iptvtools/parsers.py", line 20, in parse_content_to_lines
    return _parse_from_url(content)
  File "/home/ubuntu/.local/lib/python3.7/site-packages/iptvtools/parsers.py", line 48, in _parse_from_url
    with urlopen(url) as response:
  File "/usr/lib/python3.7/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.7/urllib/request.py", line 525, in open
    response = self._open(req, data)
  File "/usr/lib/python3.7/urllib/request.py", line 543, in _open
    '_open', req)
  File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 1360, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/usr/lib/python3.7/urllib/request.py", line 1319, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 111] Connection refused>
ubuntu@raspberrypi:~/iptv/iptvtools$

Issue when creating man pages

I have created an AUR package for Arch Linux distro and discovered an issue when creating man pages using Sphinx tool.
It is solved by using sed command withing the PKGBUILD script for now, but it would be nice if You could fix this in the upstream package for version 0.2.10 (or a future release).

Issue found in:
docs/parameters.rst:101: WARNING: Title underline too short.

SKIP_CONNECTIVITY_CHECK
-----

Should be format like this:

SKIP_CONNECTIVITY_CHECK
-----------------------

group-include filter enhencement

Is it possible to insert a group-include filter with a multiple whitelist (regular expressions)?

Either by;
(1)
Comma separation reg.expression i.e. --group-include "News","Movies","Sports NBA"

(2)
A whitelist file which can contain all group-id.s to include, i.e.
News
Movies
Sports NBA

LS character will cause AttributeError: 'NoneType' object has no attribute 'groupdict'

On a specific list, i have a channel name with a strange LS character that will break the script:

Traceback (most recent call last):
  File "/usr/local/bin/iptv-filter", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.6/dist-packages/iptvtools/iptv_filter.py", line 71, in main
    playlist.parse()
  File "/usr/local/lib/python3.6/dist-packages/iptvtools/models.py", line 68, in parse
    self._parse(self.args.inputs)
  File "/usr/local/lib/python3.6/dist-packages/iptvtools/models.py", line 95, in _parse
    current_item = parsers.parse_tag_inf(line)
  File "/usr/local/lib/python3.6/dist-packages/iptvtools/parsers.py", line 27, in parse_tag_inf
    res = match.groupdict()
AttributeError: 'NoneType' object has no attribute 'groupdict'

Copy and paste this block to an m3u and use this command iptv-filter -i test.m3u --group-include "TESTING" -o TVList.m3u to to try:

#EXTM3U
#EXTINF:-1 tvg-id="" tvg-name="normalname" tvg-logo="" group-title="GROUP",normalname
http://blalblalbllalv
#EXTINF:-1 tvg-id="" tvg-name="STRANGECHARACTER>>>  " tvg-logo="" group-title="GROUP",STRANGECHARACTER>>>  
http://testblabla/559864

using sublimetext you will clearly see the strange char:

The script sometimes fails

Hi, I just installed iptv-filter today. I ran into a little problem when I tried to use it

I am not requesting assistance with this. I am only reporting this for your information

I use Ubuntu Linux 18.04, with python3 3.6.9 installed

I ran iptv-filter against some test data that I have, and it processed each file successfully. When it works, it seems to work nicely. I have been trying to write something similar. Your script is much faster and more feature-rich than anything that I have written

After testing your script, I downloaded the latest m3u file from https://satupdates.com/canada-iptv-m3u-playlist

I have gzipped the file and uploaded it. The link is on the next line

canada-270522.m3u.gz

When I try to run iptv-filter with this file, I see this:

iptv-filter -i canada-270522.m3u -o m27e.m3u
Traceback (most recent call last):
File "/home/werrent/.local/bin/iptv-filter", line 8, in
sys.exit(main())
File "/home/werrent/.local/lib/python3.6/site-packages/iptvtools/iptv_filter.py", line 73, in main
playlist.parse()
File "/home/werrent/.local/lib/python3.6/site-packages/iptvtools/models.py", line 68, in parse
self._parse(self.args.inputs)
File "/home/werrent/.local/lib/python3.6/site-packages/iptvtools/models.py", line 137, in _parse
if current_id not in self.id_url:
UnboundLocalError: local variable 'current_id' referenced before assignment