coach1988 / iptv-solution Goto Github PK

A simple IPTV streaming proxy and manager

Dockerfile 0.53% Python 96.70% Shell 2.22% HTML 0.55%

iptv-solution's Introduction

IPTV-Solution

What is it?

A Django-based web-UI to manage IPTV streaming playlists (M3U) + EPG's (XMLTV), upstream and downstream. It is basically my first Django project and it is one of only few of my projects that have a real use and that others get to see, so please bear with me if the code is not very pythonic;-)

It is split up into a management service (Django based, port 8088) and a streaming proxy (Flask based, port 8089).

PS: Yes, the name is not very creative.

Features:

Impressions

Here are some pictures:

How does it work?

The setup currently works like this:

You define upstream user agents; These strings will be sent to upstream servers when downloading files
You define upstream playlists; These can origin from local files or a http(s) URL's. They use the user agents defined in the first step and can be further filtered before their channels and icons will be imported.
- The filters apply to the "group-title" attribute within M3U files and filter out all channels that belong to groups with these names (one group per line)
- Tip: Trigger a manual download and import via the actions above the list to populate your channels, icons and groups
You define upstream proxy servers: These have an internal and external URL and port. If you are hosting the management server and the streaming proxy on the same host, you will leave the internal URL's pointing to the localhost and the default port, unless you've changed these settings. The external URL and port should match the URL's of the proxy servers and their ports.
- The information you're using for the internal network connection will be used only by the management server to connect to the proxy in case you want to drop a running streaming session
- The information for the external network connection will be inserted as a proxy for URL-entries in M3U's and XML's (for EPG's)
You define downstream playlists; These will be assigned an upstream proxy created in the third step and will be dynamically assembled from the groups you enter here
- Tip: There is a "Get group list" button on the top right of the group admin page that returns all known groups as a text to copy & paste
- You can eventually also filter out further channels based on their names
(Optional): Define upstream EPG's: You can proxy local or remote XMLTV files

Each defined downstream playlist and each EPG is available via individual URL's.

It is meant to allow you to import the set of channel groups you are interested in from your upstream, leaving out all the groups that you don't want anyways. You then define a playlist for each one of your clients by defining a set of channel groups they will receive. If these groups still contain individual channels you don't want them to receive, you can filter them out. You can also disable channel groups, channels and playlists individually to exclude them from playlists/downloads.

URL's and login

(Adjust the hostname, port and URI scheme according to your settings)

Web admin UI (default username: admin, default password: password)

http://localhost:8088/admin

URL for EPG (replace "<name>" with your EPG's name)

http://localhost:8088/manager/get/epg/<name>

URL for downstream playlist (replace "<name>" with your playlist's name)

http://localhost:8088/manager/get/playlist/<name>

Known issues

Code quality: I am no coder, especially not for Django. Expect a lot of dirty code. But what can i say? It works for me;)
Encodings: I am bad at encodings, I am bad at http-headers and I am bad at writing files; I am sure that you will run into issues with more exotic characters and encodings
Security: I am aware of a lot of conceptual security issues; This solution was designed to be deployed inhouse for my wife (and no, she would not know how to spoof anything or get around the firewall)
Webserver: Right now, everything is based on the Django's and Flask's webservers; I would like to give interested parties at least the option to deploy this solution with their own webservers, but I don't know enough about WSGI and ASGI for that yet
Dropping of sessions currently only works from the overview, but not from a session's details page
- It can also take up to ~45 seconds before the stream stalls
Many more (especially as I have only one concurrent upstream connection to test with)
Mean exceptions when dropping a stream;)

Why Django's AND Flask's webserver?

I wanted to use Django as a management UI once i learned about it and I had this plan for a rewrite of a proxy for a long time. I unfortunately had to find out that Django is not really streaming friendly, so I had to fall back to Flask for that. But hey, at laest we've gained support for distributed setups:)

ToDo's

Overwrite handling: Allow for a convenient way to replace things such as channel name, channel groups, etc.
Service files: Ship it with some systemd units
It is far from complete, so probably many additional things

How to run

docker-compose:

Clone the git repository:

git clone https://github.com/coach1988/IPTV-Solution.git

Edit the docker-compose.yml with the editor of your choice
Build the docker images
```
docker-compose build
```
Run the stack
```
docker-compose up -d
```

The docker container for the manager uses two volumes, one for the database, one for static files (icons, epg and playlists).

manual execution:

Clone the git repository:

git clone https://github.com/coach1988/IPTV-Solution.git

Edit the proxy.env and manager.env with the editor of your choice

Create a venv for, activate it and install the requirements (for both programs)

python3 -m venv venv
source venv/bin/activate
python3 -m pip install -r requirements.txt

Run the manager process
```
manager/run_local.sh
```
Run the proxy process
```
proxy/run_local.sh
```

Variables

Manager

Name	Default	Usage
ALLOWED_HOSTS	['*']	Django's list of allowed hosts
TIME_ZONE	UTC	Timezone to use for Django (see https://en.wikipedia.org/wiki/List_of_tz_database_time_zones)
DJANGO_SUPERUSER_USERNAME	admin	Default admin login
DJANGO_SUPERUSER_EMAIL	[email protected]	Default admin email address
DJANGO_SUPERUSER_PASSWORD	password	Default admin password
SECRET_KEY	Fill me in!	Some secret key for Django, e.g. from https://djecrety.ir/
DEBUG	True	Enable Django debugging; Should be set to "True" unless you want to deal with the static file management
LOGLEVEL	DEBUG	Set log level, see this list
ALLOWED_URL_SCHEMES	['http', 'https', 'mmsh', 'mmst', 'mmsu', 'mms', 'rtmp', 'rtsp']	Disable any channel on first import that does not use one of these URL schemes
BLOCKED_PATH_TYPES	['.m3u', '.m3u8', '.mpd']	Disable any channel on first import that has a path ending with one of these suffixes
BLOCKED_URL_REGEXS	['output=playlist.m3u[8]?', 'www.youtube.com/', ]	Disable any channel on first import that has a URL matching one of these RegEx's
SOCKET_ADDRESS	0.0.0.0	The IP to bind the socket to
MANAGEMENT_URL	http://localhost	Used during M3U and EPG generation for icon URL prefixes
INTERNAL_MANAGEMENT_PORT	8088	Used for the socket setup
EXTERNAL_MANAGEMENT_PORT	8088	Used during M3U and EPG generation for icon URL prefixes
INTERNAL_TIMEOUT	1	Timeout in seconds for internal control connections
USER_AGENT_STRING	Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36)	The User Agent string to use for communication to upstream playlist, epg and icon urls
PLAYLIST_TIMEOUT	120	Download timeout for playlist files
EPG_TIMEOUT	120	Download timeout for EPG files
ICON_TIMEOUT	15	Download timeout for icon files

Proxy

Name	Default	Usage
DEBUG	True	Flask debugging
SOCKET_ADDRESS	0.0.0.0	Streaming proxy server's IP to bind the socket to
REPORTING_URL	http://localhost	Reporting (Management) server URL
REPORTING_PORT	8088	Reporting service port
REPORTING_TIMEOUT	5	Used for reporting connections
PROXY_NAME	IPTV-Proxy	Name to use when registering with the management server
INTERNAL_PROXY_URL	http://localhost	Internal URL to use when registering with the management server (to receive connection control/drop requests from the server)
INTERNAL_PROXY_PORT	8089	Internal port to use when registering with the management server (to receive connection control/drop requests from the server)
EXTERNAL_PROXY_URL	http://localhost	External URL to use when registering with the management server (for URL's inside of playlists and EPG's)
EXTERNAL_PROXY_PORT	8089	External port to use when registering with the management server (for URL's inside of playlists and EPG's)
USER_AGENT_STRING	Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36)	Defines the User Agent string to use for communication to upstream playlist, epg and icon urls
STREAM_TIMEOUT	15	Seconds before connections to upstream sources time out and a server error is reported to the client

iptv-solution's People

Contributors

Stargazers

Watchers

Forkers

wergy ytok3a wearysky manoj4000 jumanji-swr pazzovalerio

iptv-solution's Issues

Cannot download m3u file

I have followed your instructions to the letter and I think I understand what's happening but I can't download the .m3u file... each time I try, it just updates the playlist:

End of an update the log gives:

2023-12-12 10:05:34 [INFO]: DOWNSTREAM: Generating playlist stoogle
2023-12-12 10:06:14 [INFO]: DOWNSTREAM stoogle: Filter mode: A
2023-12-12 10:06:14 [INFO]: DOWNSTREAM: Saved as /app/staticfiles/playlists/downstream/c3Rvb2dsZQ==.m3u
2023-12-12 10:06:14 [INFO]: views.py: /manager/get/playlist/stoogle ---> Transferred downstream playlist stoogle

Then when I go to http://localhost:8088/manager/get/playlist/stoogle it just refreshes the playlist again:

2023-12-12 10:07:52 [INFO]: DOWNSTREAM: Updating/importing upstream playlist StoogleTV
2023-12-12 10:07:52 [INFO]: UPSTREAM StoogleTV: Upstream playlist update requested
2023-12-12 10:07:52 [INFO]: UPSTREAM StoogleTV: File already exists, checking if download is required...
2023-12-12 10:07:52 [INFO]: UPSTREAM StoogleTV: File is 14.44 hours old, download skipped...
2023-12-12 10:07:52 [INFO]: UPSTREAM StoogleTV: Filtering channels
2023-12-12 10:07:53 [INFO]: UPSTREAM StoogleTV: Filtering out group

I also have a question about the docker-compose file. Should I change the reporting URL and Internal Proxy URL to IPTV-Manager and IPTV-Proxy (and do I need to include "")? Should I be uncommenting anything else?

version: "3"

services:
  manager:
    build:
      context: ./manager/
    container_name: "IPTV-Manager"
    image: iptv-manager
    network_mode: "container:gluetun"
    volumes:
      - ./volumes/static/:/app/staticfiles/
      - ./volumes/database/:/app/database/
    environment:
    # - ALLOWED_HOSTS: "['*']"    
    # - TIME_ZONE=UTC
    # - DJANGO_SUPERUSER_USERNAME=admin
    # - DJANGO_SUPERUSER_EMAIL=[[email protected]](mailto:[email protected])
    # - DJANGO_SUPERUSER_PASSWORD=password
      - SECRET_KEY=secret
    # - DEBUG=True
    # - DEBUGLEVEL=DEBUG
    # - SOCKET_ADDRESS="0.0.0.0"
      - MANAGEMENT_URL=https://mysite/
    # - INTERNAL_MANAGEMENT_PORT=8088
    # - EXTERNAL_MANAGEMENT_PORT=8088
    # - INTERNAL_TIMEOUT=1
    # - ALLOWED_URL_SCHEMES="['http', 'https', 'mmsh', 'mmst', 'mmsu', 'mms', 'rtmp', 'rtsp']"
    # - BLOCKED_PATH_TYPES="['.m3u', '.m3u8', '.mpd']"
    # - BLOCKED_URL_REGEXS="['output=playlist.m3u[8]?', '[www.youtube.com/'](http://www.youtube.com/')]"
    # - USER_AGENT_STRING="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36)"
    # - PLAYLIST_TIMEOUT=120
    # - EPG_TIMEOUT=120
    # - ICON_TIMEOUT=15
    restart: unless-stopped
      
  proxy:
    build:
      context: ./proxy/
    container_name: "IPTV-Proxy"
    image: iptv-proxy
    ports:
      - 8089:8089/tcp
    environment:
    # - DEBUG=True
    # - SOCKET_ADDRESS=0.0.0.0
      - REPORTING_URL=http://manager/ # Match your management container's name if running in the same stack
    # - REPORTING_PORT=8088
    # - REPORTING_TIMEOUT=5
    # - PROXY_NAME=IPTV-Proxy
      - INTERNAL_PROXY_URL=http://proxy/ # Match your proxy container's name if running in the same stack
    # - INTERNAL_PROXY_PORT=8089
    # - EXTERNAL_PROXY_URL=http://localhost/
    # - EXTERNAL_PROXY_PORT=8089
    # - USER_AGENT_STRING=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36)
    # - STREAM_TIMEOUT=15
    links:
      - manager
    restart: unless-stopped

Again, sorry to take your time! I feel like I'm so close, I don't want to give up!

Many thanks
Stuart

Unraid docker template

Can you please add you app to unraid community apps https://unraid.net/community/apps/p2?r=0 I think many people would use it

Configuring IPTV-Solution

Initially, when configuring IPTV-Solution, I followed the instructions and got stuck when I received an error

while downloading the play list and EPG from the upstream provider. I found that if I populate the name of the user agent, I named them both 1, the download would succeed. Now, I am moving on to configuring IPTV-Solution to proxy to Emby/Jellyfin/Plex but i am a bit confused. With Xteve you map channels for the EPG. How is this done with IPTV-Solution?

Also, when attempting to search the IPTV Groups I get this error.

I guess as things are right now I would just like to add all enabled groupsl to my downstream playlist. Whats the best way to do that?
Thanks for all your hard work. This is shaping up to be one of the best iptv tuners around!

EDIT: I continued pushing forward and it appears that your instructions indicate that the Downstream Playlist is to be populated by manually copying the groups using the get group list button into the groups field of the Downstream. If that is the case then whats the point of enabling and disabling IPTV-Groups? As I watch the logs during the chanel import it seems that all group channels are being imported. Is it safe to assume that this is because I didn't disable those grorups?

UnboundLocalError when there's no icon attribute

During the process of running "Download, filter and import upstream playlist(s)", within Upstream - Playlists, the error Error: local variable 'fk_logo' referenced before assignment is thrown

Environment:

Request Method: POST
Request URL: http://docker02:8088/admin/manager/iptvupstreamplaylist/

Django Version: 4.2.1
Python Version: 3.9.16
Installed Applications:
['manager.apps.ManagerConfig',
 'django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware']



Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.9/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/contrib/admin/options.py", line 688, in wrapper
    return self.admin_site.admin_view(view)(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/utils/decorators.py", line 134, in _wrapper_view
    response = view_func(request, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/views/decorators/cache.py", line 62, in _wrapper_view_func
    response = view_func(request, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/contrib/admin/sites.py", line 242, in inner
    return view(request, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/utils/decorators.py", line 46, in _wrapper
    return bound_method(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/utils/decorators.py", line 134, in _wrapper_view
    response = view_func(request, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/contrib/admin/options.py", line 1959, in changelist_view
    response = self.response_action(
  File "/usr/local/lib/python3.9/site-packages/django/contrib/admin/options.py", line 1593, in response_action
    response = func(self, request, queryset)
  File "/app/manager/admin.py", line 166, in update_upstream_playlists
    upstream_playlist_helper.import_channels()
  File "/app/lib/upstream_playlist_helper.py", line 131, in import_channels
    iptvChannel.objects.aupdate_or_create(name=name, defaults={'url':url, 'tvg_id':tvg_id, 'tvg_name':tvg_name,'tvg_logo':fk_logo, 'group_title':fk_group}))

Exception Type: UnboundLocalError at /admin/manager/iptvupstreamplaylist/
Exception Value: local variable 'fk_logo' referenced before assignment

Within upstream_playlist_helper.py, attempted the following at line 128:

if logo != '':
    logger.info(f'UPSTREAM {self.playlist.name}: Importing logo "{logo}"')
    fk_logo, created = iptvIcon.objects.update_or_create(url=logo)
else:
    fk_logo = "

and also

if logo != '':
    logger.info(f'UPSTREAM {self.playlist.name}: Importing logo "{logo}"')
    fk_logo, created = iptvIcon.objects.update_or_create(url=logo)
else:
    fk_logo = None

However the error unfortunately persists.