lardbit / nefarious Goto Github PK

This is going to sound a bit confusing at first, but bear with me:

In addition to the current workflow for downloading files, implement the following -

On each indexer, create a new setting which specifies if an indexer should be used for downloading AND seeding, or just seeding.

Once that's done, the normal download workflow continues - (item picked, downloaded, etc.)

Once the file is downloaded /completed, search each indexer that is marked as "seed only" for the exact file that has already been downloaded. If it exists, download it at add it to transmission.

Transmission should then see the file already exists, add the second torrent, but should automatically verify local data and begin seeding.

This allows a user to not just download a file, but to auto seed it on other trackers where they might be worried about ratio.

I'm an experienced Linux user but a complete noob when it comes to Docker, so apologies if I'm missing something obvious…

In my .env file, I have set the HOST_DOWNLOAD_UID and HOST_DOWNLOAD_GID values to correspond to the user that actually invokes docker-compose. I assume this is supposed to prevent the strange and alarming outcome of files created by Nefarious in the "Host Download Path" being owned by root on the host system, yes?

Those files are, however, all created with owner and group root in my case. I'm using the transmission-vpn setup (which, in general, seems to be working fine including port forwarding). From a quick look through the config files in ~/nefarious, I see that those two environment variables are referenced in docker-compose.yml, section transmission:

  environment:
      - PUID=${HOST_DOWNLOAD_UID:-1000}
      - PGID=${HOST_DOWNLOAD_GID:-1000}

In docker-compose.transmission-vpn.yml, however, they seem to be missing. (Or, if the yml files cascade the way they look like they do, maybe those two settings should be moved to docker-compose.base.yml and inherited by the with/without vpn variations rather than duplicated, but I'm pretty much guessing in the dark at this point.)

Also, I hope this isn't a rude question, but... is Nefarious itself running as root in the container, or is it just that root is needed for setting up the, um, mounts? volumes? I ask because decades of Linux experience makes me very uncomfortable with anything running as root that doesn't absolutely need root privileges, and container-escape vulnerabilities do turn up occasionally. I'm not too proud to admit that I pretty much freaked out when I saw that this containerized app was creating files and directories as root on the host system. ☺

Original discussion: #96

It would be really cool if it was possible to select "HDR" as a quality preset. Right now it is possible to select ultra-hd which selects 4k content but it does not select 4k HDR content.

Because of this I have to use the manual search every time I'd like to get HDR content.

Thanks in advance!

In the UI settings appears an option to execute webhook after a download is completed. This feature is amazing to synchronize the libraries of an external media server. The point here is that media servers like jellyfin expose an API with authentication and the authentication model uses a specific header.
If nefarious brings the possibility of add custom headers in the webhook executions will support to make direct calls to the Jellyfin API and trigger the synchronization when the movement after completion is done.

Thanks for you great job :)

Is it possible to filter out movies and tv shows if they don't have any download sources?

As far as I can tell, Nefarious looks in its own Host Download Path when trying to import my existing media library, which seems like a… kind of weird assumption?

My media library lives on a NAS that the torrent box doesn't have write access to (and I intend to keep it that way). I haven't tested this, but for now I intend to try symlinking the network mount into the Nefarious download path. I have no reason to think that won't work, but I'd call it a workaround.

Personally, I would expect there to be a config option in the .env file for path to media library, which I assume would then be mounted read-only in the container. I don't know if it's possible to add paths from inside the already-running container, but having having an "Add library path" option right near the "Import TV shows" and "Import movies" buttons would be handy if it's feasible.

Ideally, it would be nice to be able to specify an arbitrary number of import paths. Personally, my media library is optimized for Kodi, so it's split into separate paths for each family member's TV shows, and then again for each family member's movies, etc. I don't know whether it would confuse Nefarious to add the parent directory, which also includes music, random video content, music videos, and so forth. I expect I'll find out soon, though. ☺

I added trailer links to movies recently but didn't like how I added the actual links to the page. It doesn't look great.

Do you have any better ideas? Buttons or something?

Something to note, though, there are an unknown amount of trailers returned from tmdb.org so, if space is an issue, we should just limit the maximum to like 5 or something.

The movie component:
https://github.com/lardbit/nefarious/blob/master/src/frontend/src/app/media/media-movie.component.ts
https://github.com/lardbit/nefarious/blob/master/src/frontend/src/app/media/media-movie.component.html

Remember to run npm --prefix src/frontend run watch while you're developing so it'll constantly rebuild the app. run build just does it once.

New feature request.
new user here. I’ve been enjoying using nefarious for a few weeks now. Works perfectly but is there anyway to auto scrape subtitles and include them in the download path after nefarious processes the download?

basically when the movie is done downloading and is renamed to the download location. Grab subtitles from the internet and place them next to the movie file so it can be picked up by plex. I’ve never used bazarr but it claims to do this with radar and sonarr

I tried to download a single episode of a TV show and Transmission downloaded it directly inside /tv folder. This causes some problems with my media center (jellyfin) because it expects to have some folder hierarchy like Series > Season > Episode.

Do you think it is possible to create such folders? The problem is some torrents contain the file directly, others have a folder at the root. I don't have too much experience with nefarious/transmission so I don't really know what is possible.

I've noticed that every time I click the search tab in my shiny new Nefarious instance, the Movies vs TV selection is always set to Movies. It's a little thing, but personally I'm looking for TV shows far more often than movies, and it would be nice to have an option in the settings to tell it to default to TV instead, so I don't have to remember to click it every time.

Trying to run nefarious for the first time on a Pi. Getting these errors:

`root@MaxPi:~/nefarious# docker-compose up -d
nefarious_transmission_1 is up-to-date
nefarious_redis_1 is up-to-date
Creating nefarious_watchtower_1 ... error
Creating nefarious_jackett_1 ...
Creating nefarious_celery_1 ...

Creating nefarious_jackett_1 ... error
ver

Creating nefarious_celery_1 ... error

ERROR: for nefarious_celery_1 Cannot create container for service celery: unknown log opt 'max-size' for journald log driver

ERROR: for watchtower Cannot create container for service watchtower: unknown log opt 'max-size' for journald log driver

ERROR: for jackett Cannot create container for service jackett: unknown log opt 'max-size' for journald log driver

ERROR: for celery Cannot create container for service celery: unknown log opt 'max-size' for journald log driver
ERROR: Encountered errors while bringing up the project.`

Not really sure what it means. I'm using dietpi as the OS, if that matters.

My apology if it is already there and I just missed it.

Manual search is great feature but right now for series is bin unintuitive since you can only search whole seasons manually.

Another thing would be allow search from the status section (You see torrent was not found for episode / series, you will try your own luck right there)

Now if I am searching for episode where torrent was not found, I would like to know why without long search in logs. So last feature suggestion for this would be catch search data returned from Jackett (could be just for 24h or something). When applying search criteria on data again user can get grasp of why torrent wasn't found and also select one of the rejected torrents without new search.

The last part is bit of a stretch that would be nice for future but definitely low priority now.

When starting the docker nefarious tries to start on port 80 instead of 8000, which can not be changed (on a Synology NAS)

//edit
Okay so apparently it tries to bind on port 80 and this is not allowed by the Synology Docker because it's already in use on the NAS itself, is there any way to change this port?

Hi!
I tried to setup nefarious with https for the last few hours, and I couldn't get the websocket work, with https. Then I looked at the code and saw that you use ${windowLocation.protocol === 'https' ? 'wss' : 'ws'}: to determine wss or ws. However window.location.protocol returns "https:" with a colon, not simply "https", so it will always try to connect as ws, not wss.

See #64 for context.

Is there a way to just tweak the maximum size of the torrents it finds? Its finding 20-30 GB complete seasons and i want to push it down a bit to about 10-15 GB.

Hi,
Having a problem with the arm setup, I think. There is an error when I do docker-compose logs -f nefarious and it prevents me from seeing the login page on nefarious.

Here are the logs

Can someone help?

Thank you :)

I have not tested it yet but according to the features this is not yet integrated.

Since on trackers usually Scene Releases are offered and Sonarr or Radarr is not clear and this is not unpacked, so this can not be assigned, a feature here would be very good copy a finished release in another folder and unpack there so the nefarious then find it.

So far so good! Shouldn't have put off trying this out for so long lol

Anyway seems like there is no (easy? clear?) way to customize where the completed movies and tv shows go. Current setup via Xpenology:

$ nano .env
HOST_DOWNLOAD_PATH=/volume2/Downloads/

For movies: /volume2/Downloads/completed/movies
For tv shows: /volume2/Downloads/completed/tvshows

Actual/preferred media path:
/volume2/Media/Movies
/volume2/Media/TVShows

I've set it up this way so I can just rclone the /Media folder to my gdrive ever few hours or so.

I'm guessing currently I'd have to do a cron job of moving the subfolders in completed/movies and tvshows into /Media/Movies and /Media/TVShows. So it would be awesome if the .env file can set a custom folder for both media types.

Plex integration (as in the auth or user accounts feature) would be great, but really not necessary.

10 stars!!

I am running the last version of nefarious image with a docker-compose like this:

  nefarious:
    image: lardbit/nefarious:armv7
    container_name: nefarious
    labels:
      - com.centurylinklabs.watchtower.enable=true
    restart: unless-stopped
    ports:
    - "8000:80"
    environment:
      DATABASE_URL: "sqlite:////nefarious-db/db.sqlite3"
      REDIS_HOST: "redis"
      NEFARIOUS_USER: admin
      NEFARIOUS_PASS: admin
      HOST_DOWNLOAD_PATH: ${HOST_DOWNLOAD_PATH}
    volumes:
      # persistent named volume for sqlite database
      - ${VOLUMES}/nefarious/nefarious-db:/nefarious-db
    logging:
      options:
        max-size: 10m
    depends_on:
    - redis
    - jackett
    - celery

  # background task queue
  celery:
    image: lardbit/nefarious:armv7
    container_name: celery
    labels:
      - com.centurylinklabs.watchtower.enable=true
    restart: unless-stopped
    entrypoint: /env/bin/celery -A nefarious worker --concurrency ${NUM_CELERY_WORKERS:-0} --beat --loglevel=INFO
    environment:
      # https://github.com/kennethreitz/dj-database-url
      DATABASE_URL: "sqlite:////nefarious-db/db.sqlite3"
      REDIS_HOST: "redis"
      HOST_DOWNLOAD_PATH: ${HOST_DOWNLOAD_PATH}
      INTERNAL_DOWNLOAD_PATH: /downloads
    volumes:
      - ${HOST_DOWNLOAD_PATH}:/downloads
      # persistent named volume for sqlite database
      - ${VOLUMES}/nefarious/nefarious-db:/nefarious-db
    logging:
      options:
        max-size: 10m
    depends_on:
      - redis

  # in-memory database for task queue
  redis:
    image: arm32v7/redis
    container_name: redis
    restart: unless-stopped
    ports:
    - "6379:6379"

After some task executions, my downloads folder looks like this:

In this image you can see that the folder ..nefarious-unprocessed-downloads is being replicated inside other folders and the new torrents are added in the last nested .nefarious-unprocessed-downloads. The problem here is that with this situation, the final path of the series and the movies change all the time and I can't detect the downloaded files in plex.

Maybe my docker-compose is wrong? I tried some different configurations but all the time the folder is changing. I tried fixing the path in the transmission json. It looks like this:

{
    "alt-speed-down": 2000,
    "alt-speed-enabled": true,
    "alt-speed-time-begin": 450,
    "alt-speed-time-day": 127,
    "alt-speed-time-enabled": true,
    "alt-speed-time-end": 30,
    "alt-speed-up": 10000,
    "bind-address-ipv4": "0.0.0.0",
    "bind-address-ipv6": "::",
    "blocklist-enabled": false,
    "blocklist-url": "http://www.example.com/blocklist",
    "cache-size-mb": 16,
    "dht-enabled": true,
    "download-dir": "/downloads/complete",
    "download-queue-enabled": true,
    "download-queue-size": 15,
    "encryption": 0,
    "idle-seeding-limit": 300,
    "idle-seeding-limit-enabled": true,
    "incomplete-dir": "/downloads/incomplete",
    "incomplete-dir-enabled": true,
    "lpd-enabled": true,
    "message-level": 2,
    "peer-congestion-algorithm": "",
    "peer-id-ttl-hours": 6,
    "peer-limit-global": 200,
    "peer-limit-per-torrent": 50,
    "peer-port": 51413,
    "peer-port-random-high": 65535,
    "peer-port-random-low": 49152,
    "peer-port-random-on-start": false,
    "peer-socket-tos": "default",
    "pex-enabled": true,
    "port-forwarding-enabled": true,
    "preallocation": 1,
    "prefetch-enabled": true,
    "queue-stalled-enabled": true,
    "queue-stalled-minutes": 30,
    "ratio-limit": 100,
    "ratio-limit-enabled": true,
    "rename-partial-files": true,
    "rpc-authentication-required": true,
    "rpc-bind-address": "0.0.0.0",
    "rpc-enabled": true,
    "rpc-host-whitelist": "",
    "rpc-host-whitelist-enabled": true,
    "rpc-password": "{15a5eee232c570df1e95a901c5e320b479a9fd645YTFlxFH",
    "rpc-port": 9091,
    "rpc-url": "/transmission/",
    "rpc-username": "username",
    "rpc-whitelist": "*.*.*.*",
    "rpc-whitelist-enabled": true,
    "scrape-paused-torrents-enabled": true,
    "script-torrent-done-enabled": false,
    "script-torrent-done-filename": "",
    "seed-queue-enabled": true,
    "seed-queue-size": 25,
    "seedRatioLimit": 50,
    "seedRatioLimited": false,
    "speed-limit-down": 300000,
    "speed-limit-down-enabled": true,
    "speed-limit-up": 300000,
    "speed-limit-up-enabled": true,
    "start-added-torrents": true,
    "trash-original-torrent-files": true,
    "umask": 18,
    "upload-slots-per-torrent": 14,
    "utp-enabled": true,
    "watch-dir": "/watch",
    "watch-dir-enabled": true
}

Could you help me with this issue? This software is the best that I tried to automatize the torrent discover and download but I can't use now and I feeld that is because I am configuring wrong it :(

Issue appears to be related to transmission user id / group id but I'm unsure how to solve.

• At first I thought it was because the "tv/" and "movies/" paths in Nefarious caused a double slash: "tv//xyz"
• Then thought it was an issue with the destination being an nfs share so I changed it to a local path
• Noticed it keeps setting my downloads directory to be owned by 911:911 which is the uid:gid used as a placeholder when something goes wrong
• Was thinking perhaps the Nefarious .env user needed to exist so I created admin:admin and set that in the transmission user and password boxes
• I've even tried setting the docker user to my uid:gid in the docker compose file ... and it just wouldn't start afterwards so I removed this

I'm a software dev but generally unfamiliar with docker and transmission.

Context: running in a debian 10 minimal vm inside proxmox using the instructions from this git repo to install.

Hello !

Long story short :

• I have bound the different ports to 127.0.0.1:XXX instead of :XXXX
• I have added an nginx to serve as reverse proxy and add https to the whole stack.
• I have added a base path override to jackett on /jackett

My nginx configuration is similar to this :

location /jackett/ {
    proxy_pass         http://127.0.0.1:9117;
    proxy_http_version 1.1;
    proxy_set_header   Upgrade $http_upgrade;
    proxy_set_header   Connection keep-alive;
    proxy_cache_bypass $http_upgrade;
    proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header   X-Forwarded-Proto $scheme;
    proxy_set_header   X-Forwarded-Host $http_host;
}

It works fine for jackett itself but raised two issues :

First of all : I haven't found a way to tell nefarious than the jackett base path has changed, this results in these kinds of errors :

nefarious_1     | ERROR  2019-11-09 18:21:07,846  utils 18  140488504710976  HTTPConnectionPool(host='127.0.0.1', port=9117): Max retries exceeded with url: /api/v2.0/indexers/all/results?apikey=t8j6lnz1zlcmfj90u8yx40d513ug10go (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc60119f7b8>: Failed to establish a new connection: [Errno 111] Connection refused',))
nefarious_1     | WARNING  2019-11-09 18:21:07,849  log 18  140488504710976  Bad Request: /api/settings/1/verify-jackett-indexers/

(Kinda obvious since the path has changed.)

Second : There is no option to changed the base path of nefarious itself, which complicates things if you planned to put it on your.example.com/nefarious

I wil try to take a look at both these issues !

I am loving this project and all its usefulness, and I wish to recommend a feature regarding administrator accounts. If a user wants to stop watching after they complete a movie or show, one would need to log into an admin account and stop watching from there. Unfortunately, I am unable to grant admin privileges to user accounts.
I believe this would be incredibly useful because multiple users can become administrators, giving the ability to stop watching shows. In addition, if multiple moderators collaborate on this tool without wanting to share the same credentials, multiple administrator accounts would be beneficial.
I wish to propose two possible options towards making this feature possible:

1. A checkbox next to each user to select whether they have privileged access in the nefarious settings page.
2. The less secure option would be to allow users to stop watching their own shows but still prevent access to the settings page. It would not create the feature of having multiple administrators, but it would give extra privileges to multiple user accounts.

The VPS provider I use doesn't have ipv6 support. So when I try to install Nefarious; Jackett and Transmission fail to start up.

Is there a way to disable ipv6 entirely?

Where is the media stored?

This is more of an FYI then a full Issue report, but I figured I'd let people know in case it is a full issue.

If one of the indexers in Jackett takes too long (over 60 seconds) to respond, it will result in all querys to Jackett timing out. This occurs even if there are other Indexers available and/or the misbehaving indexer is unneeded for the query.

This is difficult to notice, as Nefarious will only report a timeout error contacting the Jacket server entirely. Validating the Indexers under settings doesn't show the error, as the page will lock up until the timeout occurs with no error messages, and the settings section will look fine since the other parts respond correctly.

It might already be a thing, I haven't checked yet. It would be useful for keeping the files created owned by the correct user. This feature is available in this image: https://hub.docker.com/r/linuxserver/transmission/

First of all this is an amazing project!

I'm getting everything setup but When saving my settings I get an error saying " could not connect to jackett" not sure what's causing this. I had jackett set up with sonar prior and was working fine

I tried getting sonarr/radarr to work all day and it would never start downloading or something else would go wrong.

Nefarious is the only program that just works. No more fiddling with complicated settings hoping and praying it works.

Thank you!

I hope you always work on this amazing project!

Please delete this issue . I can't do it on my own

When searching for movies only the first ~20 movies are shown. When your wished movie is not on that page, you can't add that movie. So I suggest to add paging like on themoviedb.
By the way: a filter like "release year" would be very nice to have.

Both of these features are supported by their API: https://developers.themoviedb.org/3/search/search-movies

I would like to replace Radarr and Sonarr with Nefarious but I'm using them with Usenet (the only reason why I use both of them).

Are there any plans to add Usenet support in the future?

Hi,
I see that the image was updated whit night. After the update, I found this error in celery:

[2021-01-14 08:41:52,554: INFO/MainProcess] Connected to redis://redis:6379/0
[2021-01-14 08:41:52,561: INFO/MainProcess] mingle: searching for neighbors
[2021-01-14 08:41:53,576: INFO/MainProcess] mingle: all alone
[2021-01-14 08:41:53,588: INFO/MainProcess] celery@25676028042f ready.
[2021-01-14 08:41:54,961: INFO/Beat] beat: Starting...
[2021-01-14 08:41:54,969: ERROR/Beat] Removing corrupted schedule file 'celerybeat-schedule': error(13, 'Permission denied')
Traceback (most recent call last):
  File "/env/lib/python3.8/site-packages/kombu/utils/objects.py", line 42, in __get__
    return obj.__dict__[self.__name__]
KeyError: 'scheduler'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/env/lib/python3.8/site-packages/celery/beat.py", line 519, in setup_schedule
    self._store = self._open_schedule()
  File "/env/lib/python3.8/site-packages/celery/beat.py", line 509, in _open_schedule
    return self.persistence.open(self.schedule_filename, writeback=True)
  File "/usr/lib/python3.8/shelve.py", line 243, in open
    return DbfilenameShelf(filename, flag, protocol, writeback)
  File "/usr/lib/python3.8/shelve.py", line 227, in __init__
    Shelf.__init__(self, dbm.open(filename, flag), protocol, writeback)
  File "/usr/lib/python3.8/dbm/__init__.py", line 95, in open
    return mod.open(file, flag, mode)
_gdbm.error: [Errno 13] Permission denied: 'celerybeat-schedule'
[2021-01-14 08:41:54,971: ERROR/Beat] Process Beat
Traceback (most recent call last):
  File "/env/lib/python3.8/site-packages/kombu/utils/objects.py", line 42, in __get__
    return obj.__dict__[self.__name__]
KeyError: 'scheduler'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/env/lib/python3.8/site-packages/celery/beat.py", line 519, in setup_schedule
    self._store = self._open_schedule()
  File "/env/lib/python3.8/site-packages/celery/beat.py", line 509, in _open_schedule
    return self.persistence.open(self.schedule_filename, writeback=True)
  File "/usr/lib/python3.8/shelve.py", line 243, in open
    return DbfilenameShelf(filename, flag, protocol, writeback)
  File "/usr/lib/python3.8/shelve.py", line 227, in __init__
    Shelf.__init__(self, dbm.open(filename, flag), protocol, writeback)
  File "/usr/lib/python3.8/dbm/__init__.py", line 95, in open
    return mod.open(file, flag, mode)
_gdbm.error: [Errno 13] Permission denied: 'celerybeat-schedule'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/env/lib/python3.8/site-packages/billiard/process.py", line 327, in _bootstrap
    self.run()
  File "/env/lib/python3.8/site-packages/celery/beat.py", line 707, in run
    self.service.start(embedded_process=True)
  File "/env/lib/python3.8/site-packages/celery/beat.py", line 622, in start
    humanize_seconds(self.scheduler.max_interval))
  File "/env/lib/python3.8/site-packages/kombu/utils/objects.py", line 44, in __get__
    value = obj.__dict__[self.__name__] = self.__get(obj)
  File "/env/lib/python3.8/site-packages/celery/beat.py", line 666, in scheduler
    return self.get_scheduler()
  File "/env/lib/python3.8/site-packages/celery/beat.py", line 657, in get_scheduler
    return symbol_by_name(self.scheduler_cls, aliases=aliases)(
  File "/env/lib/python3.8/site-packages/celery/beat.py", line 501, in __init__
    Scheduler.__init__(self, *args, **kwargs)
  File "/env/lib/python3.8/site-packages/celery/beat.py", line 257, in __init__
    self.setup_schedule()
  File "/env/lib/python3.8/site-packages/celery/beat.py", line 527, in setup_schedule
    self._store = self._destroy_open_corrupted_schedule(exc)
  File "/env/lib/python3.8/site-packages/celery/beat.py", line 515, in _destroy_open_corrupted_schedule
    return self._open_schedule()
  File "/env/lib/python3.8/site-packages/celery/beat.py", line 509, in _open_schedule
    return self.persistence.open(self.schedule_filename, writeback=True)
  File "/usr/lib/python3.8/shelve.py", line 243, in open
    return DbfilenameShelf(filename, flag, protocol, writeback)
  File "/usr/lib/python3.8/shelve.py", line 227, in __init__
    Shelf.__init__(self, dbm.open(filename, flag), protocol, writeback)
  File "/usr/lib/python3.8/dbm/__init__.py", line 95, in open
    return mod.open(file, flag, mode)
_gdbm.error: [Errno 13] Permission denied: 'celerybeat-schedule'
[2021-01-14 08:41:54,972: WARNING/Beat] Process Beat:
[2021-01-14 08:41:54,972: WARNING/Beat] Traceback (most recent call last):
[2021-01-14 08:41:54,972: WARNING/Beat] File "/env/lib/python3.8/site-packages/kombu/utils/objects.py", line 42, in __get__
    return obj.__dict__[self.__name__]
[2021-01-14 08:41:54,972: WARNING/Beat] KeyError: 'scheduler'
[2021-01-14 08:41:54,972: WARNING/Beat] During handling of the above exception, another exception occurred:
[2021-01-14 08:41:54,972: WARNING/Beat] Traceback (most recent call last):
[2021-01-14 08:41:54,972: WARNING/Beat] File "/env/lib/python3.8/site-packages/celery/beat.py", line 519, in setup_schedule
    self._store = self._open_schedule()
[2021-01-14 08:41:54,972: WARNING/Beat] File "/env/lib/python3.8/site-packages/celery/beat.py", line 509, in _open_schedule
    return self.persistence.open(self.schedule_filename, writeback=True)
[2021-01-14 08:41:54,972: WARNING/Beat] File "/usr/lib/python3.8/shelve.py", line 243, in open
    return DbfilenameShelf(filename, flag, protocol, writeback)
[2021-01-14 08:41:54,972: WARNING/Beat] File "/usr/lib/python3.8/shelve.py", line 227, in __init__
    Shelf.__init__(self, dbm.open(filename, flag), protocol, writeback)
[2021-01-14 08:41:54,972: WARNING/Beat] File "/usr/lib/python3.8/dbm/__init__.py", line 95, in open
    return mod.open(file, flag, mode)
[2021-01-14 08:41:54,972: WARNING/Beat] _gdbm.error: [Errno 13] Permission denied: 'celerybeat-schedule'
[2021-01-14 08:41:54,972: WARNING/Beat] During handling of the above exception, another exception occurred:
[2021-01-14 08:41:54,973: WARNING/Beat] Traceback (most recent call last):
[2021-01-14 08:41:54,973: WARNING/Beat] File "/env/lib/python3.8/site-packages/billiard/process.py", line 327, in _bootstrap
    self.run()
[2021-01-14 08:41:54,973: WARNING/Beat] File "/env/lib/python3.8/site-packages/celery/beat.py", line 707, in run
    self.service.start(embedded_process=True)
[2021-01-14 08:41:54,973: WARNING/Beat] File "/env/lib/python3.8/site-packages/celery/beat.py", line 622, in start
    humanize_seconds(self.scheduler.max_interval))
[2021-01-14 08:41:54,973: WARNING/Beat] File "/env/lib/python3.8/site-packages/kombu/utils/objects.py", line 44, in __get__
    value = obj.__dict__[self.__name__] = self.__get(obj)
[2021-01-14 08:41:54,973: WARNING/Beat] File "/env/lib/python3.8/site-packages/celery/beat.py", line 666, in scheduler
    return self.get_scheduler()
[2021-01-14 08:41:54,973: WARNING/Beat] File "/env/lib/python3.8/site-packages/celery/beat.py", line 657, in get_scheduler
    return symbol_by_name(self.scheduler_cls, aliases=aliases)(
[2021-01-14 08:41:54,973: WARNING/Beat] File "/env/lib/python3.8/site-packages/celery/beat.py", line 501, in __init__
    Scheduler.__init__(self, *args, **kwargs)
[2021-01-14 08:41:54,973: WARNING/Beat] File "/env/lib/python3.8/site-packages/celery/beat.py", line 257, in __init__
    self.setup_schedule()
[2021-01-14 08:41:54,973: WARNING/Beat] File "/env/lib/python3.8/site-packages/celery/beat.py", line 527, in setup_schedule
    self._store = self._destroy_open_corrupted_schedule(exc)
[2021-01-14 08:41:54,973: WARNING/Beat] File "/env/lib/python3.8/site-packages/celery/beat.py", line 515, in _destroy_open_corrupted_schedule
    return self._open_schedule()
[2021-01-14 08:41:54,973: WARNING/Beat] File "/env/lib/python3.8/site-packages/celery/beat.py", line 509, in _open_schedule
    return self.persistence.open(self.schedule_filename, writeback=True)
[2021-01-14 08:41:54,973: WARNING/Beat] File "/usr/lib/python3.8/shelve.py", line 243, in open
    return DbfilenameShelf(filename, flag, protocol, writeback)
[2021-01-14 08:41:54,973: WARNING/Beat] File "/usr/lib/python3.8/shelve.py", line 227, in __init__
    Shelf.__init__(self, dbm.open(filename, flag), protocol, writeback)
[2021-01-14 08:41:54,973: WARNING/Beat] File "/usr/lib/python3.8/dbm/__init__.py", line 95, in open
    return mod.open(file, flag, mode)
[2021-01-14 08:41:54,973: WARNING/Beat] _gdbm.error: [Errno 13] Permission denied: 'celerybeat-schedule'

I tried to create from 0 all also, and I have the same error. I can't fount where is the file that breaks the container in order to continue debugging or trying. Sorry for this poor information

After you update manual search function, it doesn't work anymore in a language other than English. If I search one movie with the Spanish Title, works ok, movie found, but when I use the manual search, Nefarious use the English name to search in Jackett and it return only English results. My Jackett providers are local (Spanish) and found no results or different movies.

Can you use the local Title of IMDB or TMDB, or maybe the input search for make it work again?

Thanks!!

It would be great to see "Test" buttons to help troubleshoot issues in communication between Jackett, transmission, etc.

Most libraries / setups are different, being able to specify where the files are being stored beyond the automatic "tv / movie" folder would be great

it would be nice to search for movies and have it display the year the movie was released to help manage movies of duplicate names

When downloading TV Show lists, it would be convenient to have a button to select downloading all seasons of the show rather than having to select each individual season

It is nice to be able to monitor requests that have not been filled yet. This can simply be a listing of episodes / movies that have been set to monitor, but no download source has been found

Having a calendar of shows that are upcoming / current and the current download progress would be great. This functionality is in sonarr and quite useful

It would be great to be able to view and dump the logs from the Nefarious console rather than have to run docker-compose logs -f .....

Since using a vpn is very common with a torrent application, it may be helpful to either add an example of using a vpn in the docker-compose or have a separate compose file to use with vpn.

below is the compose file modified to use the docker container with openvpn and transmission from haugene/transmission-openvpn

It still allows access to jacket and transmission web ui according to docs. The only change is that in the nefarious ui, the jacket host has to be changed to transmission so that all jacket searches use VPN

Working compose example:

version: '2.4'

services:
  # main application
  nefarious:
    labels:
    - com.centurylinklabs.watchtower.enable=true
    image: lardbit/nefarious
    restart: always
    ports:
    - "8000:80"
    environment:
      # https://github.com/kennethreitz/dj-database-url
      DATABASE_URL: "sqlite:////nefarious-db/db.sqlite3"
      REDIS_HOST: "redis"
      NEFARIOUS_USER: ${NEFARIOUS_USER:-admin}
      NEFARIOUS_PASS: ${NEFARIOUS_PASS:-admin}
      HOST_DOWNLOAD_PATH: ${HOST_DOWNLOAD_PATH:-/tmp/}
    volumes:
      # persistent named volume for sqlite database
      - nefarious-db:/nefarious-db
    logging:
      options:
        max-size: 10m
    depends_on:
    - redis
    - jackett
    - celery

  # background task queue
  celery:
    labels:
      - com.centurylinklabs.watchtower.enable=true
    image: lardbit/nefarious
    restart: always
    entrypoint: /env/bin/celery -A nefarious worker --concurrency ${NUM_CELERY_WORKERS:-0} --beat --loglevel=INFO
    environment:
      # https://github.com/kennethreitz/dj-database-url
      DATABASE_URL: "sqlite:////nefarious-db/db.sqlite3"
      REDIS_HOST: "redis"
      HOST_DOWNLOAD_PATH: ${HOST_DOWNLOAD_PATH:-/tmp}
      INTERNAL_DOWNLOAD_PATH: /downloads/completed
    volumes:
      - ${HOST_DOWNLOAD_PATH:-/tmp}:/downloads
      # persistent named volume for sqlite database
      - nefarious-db:/nefarious-db
    logging:
      options:
        max-size: 10m
    depends_on:
      - redis

  # in-memory database for task queue
  redis:
    image: redis
    restart: always
    mem_limit: 200m
    # TODO - use "expose" vs "ports" to limit external access when not in development
    ports:
    - "6379:6379"

  # torrent indexer service
  jackett:
    labels:
      - com.centurylinklabs.watchtower.enable=true
    image: linuxserver/jackett
    mem_limit: 200m
    restart: always
    network_mode: service:transmission
    depends_on: 
      - transmission
    logging:
      options:
        max-size: 10m
    volumes:
      - jackett-config:/config

  transmission:
    image: haugene/transmission-openvpn:latest
    container_name: transmission
    labels:
      - com.centurylinklabs.watchtower.enable=true
    cap_add:
      - NET_ADMIN
    devices:
      - "/dev/net/tun"
    restart: unless-stopped
    ports:
      - 9091:9091/tcp
      - 9091:9091/udp
      - 8888:8888 # web-proxy
      - 9117:9117 # jackket
    dns:
      - 1.1.1.1
      - 1.0.0.1
    volumes:
      - ${HOST_DOWNLOAD_PATH:-/tmp}:/data
      - ./transmission-settings.json:/config/settings.json:ro
      - '/etc/localtime:/etc/localtime:ro'
    environment:
      - OPENVPN_PROVIDER=VPN_PROVIDER
      - OPENVPN_USERNAME=***********
      - OPENVPN_PASSWORD=***********
      - OPENVPN_OPTS=--inactive 3600 --ping 10 --ping-exit 60
      - LOCAL_NETWORK=192.168.0.0/24,172.17.0.0/16
      - WEBPROXY_ENABLED=true
      - WEBPROXY_PORT=8888
      - GLOBAL_APPLY_PERMISSIONS=true
      - CREATE_TUN_DEVICE=true
    logging:
      driver: json-file
      options:
        max-size: 10m

  # auto update service
  # https://github.com/v2tec/watchtower
  watchtower:
    image: v2tec/watchtower
    restart: always
    command: --label-enable --cleanup
    logging:
      options:
        max-size: 10m
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

volumes:
  nefarious-db:
    driver: local
  jackett-config:
    driver: local

It's not doing it anymore but I ran into a problem where transmission threw an error because the /downloads/ folder in the container was only 6.2 GB. So anything over that in size would fail to download.

Thank you for this cool project! <3

Maybe somekind of notification saying that a download started, maybe by email or pushbullet ou gotify whould be nice?

Thanks!

is this system made only for 64bit architecture? because I had this problem when trying to install nefarious:

docker-compose up -d
Pulling jackett (linuxserver/jackett:)...
latest: Pulling from linuxserver/jackett
ERROR: no matching manifest for linux/386 in the manifest list entries

Loving Nefarious so far; thanks for this nifty project!

The Quality Profile setting is very handy, and I would suggest taking that feature one step farther. Some shows I don't care too much about the quality (eg talky dramas that don't rely much on visuals, or shows that I may get around to watching some day) so for those shows I would like to set a lower quality profile and save some disk space. But for the latest flashy sci-fi that I'm really actively watching, I want to get the highest quality my TV will support, of course.

For my purposes, it would be perfect to have a "Quality Profile for this show" option on the top-level series description page, right near the "Stop watching show" and "Watch all seasons" buttons. Presumably it would present the same listbox that's used in the settings page, with the current overall default pre-selected. (Or maybe just a ticky-box for "Specify Quality Profile for this show", unticked by default, and have the listbox only appear when you tick the box. I'm no expert in UI or web-app design. ☺)

I have an issue related with the accents in Spanish. I am trying to download a TV show named 'Érase una vez'.
When the celery container starts, it prints:

INFO  2020-12-24 17:58:10,288  processors 12  4159525392  Processing request to watch Érase una vez - S01E07

When jackett is queried, it returns :

But when celery process the result, it says this:

INFO  2020-12-24 17:57:59,498  processors 14  4159525392  Not matched: Erase Una Vez S01E07 SPANISH SDTV XviD
[2020-12-24 17:57:59,498: INFO/ForkPoolWorker-5] Not matched: Erase Una Vez S01E07 SPANISH SDTV XviD

My nefarious' config is this:

I also tried using DEBUG loglevel in celery in order to get more info related with this but I haven't got any reason in logs.
With this data, I suppose that the problem is related with the accents.

Sorry because I don't have knowledge in python to fix it or get more specify cause.
The tracker that I am using to resolve this TV show is DivxTotal (if you need to try)

EDIT: After download some episodes looking for them manually, I discover that the name after the torrent rename is Once Upon a Time. Is this the name that uses to match the jackett response? I don't understand it because that jackett request is for the episode in Spanish, but the internal name is in English

Thanks for support

I currently have Sonarr and Raddar install and love them. I want to set up similar on my roommates machine, but he needs something that's more like your offerings, everything in one. The issue is I do not want to use docker at all. Is there a way to install and get nefarious up and running without docker? Thanks

Hi,

Do you have plan to switch to Podman ?

In rehl8 and Fedora 31 podman is the new default system for running container :

https://www.redhat.com/en/blog/working-linux-containers-rhel-8-podman-image-builder-and-web-console

Thanks

The images from tmdb.org are not server over https, just http and because of this browser show mixed content errors, if nefarious itself is server over https.

Add support to docker image to make config persists after rebuild image.
With docker parameters or mounting a config file

After it downloads a season of episodes, I would rather the season folder contain just the video files and subtitles rather than a collection of folders for each episode.

I was trying nefarious today while running npm install I see following

found 1310 vulnerabilities (937 low, 13 moderate, 358 high, 2 critical)
run npm audit fix to fix them, or npm audit for details

Any plans to upgrade these ?

There are a lot of movies (in spanish) whose use colon in the movie name. For example Spider-Man: Lejos de casa. Thing this is that a lot of uploaders in trackers don't write the name with the colon. Following the previous sample:

The point here is I think that a simple replace of the colon in the string before the comparation would handle good a big part of them. I am not sure but I suppose it :)

You are doing a very good job with this project! Thanks for all :)

Hi, I just discovered this project and I think will be very goood!!!!
First I want to thank you the owner!

I've some question about this, in order to better understand the whole idea:

1. There's any possibility to have (in future) the regex capability like radarr? Is quiete convenient create custom filter to include/exclude releases
2. There's any idea to have more torrent downloading client supported? Like qBittorrent or others
3. When you say "Supports multiple languages", is only for movie or also for TV show? The bad of sonarr for example is that don't grab any title from tvdb like radarr do with TMDB. For that reason I proposed one simple idea (is just a starting point), on jackett side, you can find it here Jackett/Jackett#2743 then I asked to sonarr guys but on sonarr side seems that the foreign language are not considered so much.

Thanks and have a nice day.

Jo