I have to say, great project, but after crawling for about 5 minutes i'm getting this error:

Error: https://www.limetorrents.cc/browse-torrents/Anime/date/1/ just ain't workin.
[error] GenServer #PID<0.455.0> terminating
** (FunctionClauseError) no function clause matching in Floki.Finder.traverse/4
    lib/floki/finder.ex:61: Floki.Finder.traverse(nil, [], %Floki.Selector{attributes: [], classes: ["tt-name"], combinator: %Floki.Combinator{match_type: :descendant, selector: %Floki.Selector{attributes: [], classes: [], combinator: nil, id: nil, namespace: nil, type: "a"}}, id: nil, namespace: nil, type: nil}, [])
    lib/floki/finder.ex:73: Floki.Finder.traverse/4
    lib/floki/finder.ex:47: Floki.Finder.find_selectors/2
    (magnetissimo) lib/crawler/limetorrents.ex:74: Magnetissimo.Crawler.LimeTorrents.torrent_links/1
    (magnetissimo) lib/crawler/limetorrents.ex:37: Magnetissimo.Crawler.LimeTorrents.process/2
    (magnetissimo) lib/crawler/limetorrents.ex:26: Magnetissimo.Crawler.LimeTorrents.handle_info/2
    (stdlib) gen_server.erl:601: :gen_server.try_dispatch/4
    (stdlib) gen_server.erl:667: :gen_server.handle_msg/5
    (stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3
Last message: :work

Looks like it fails when it cannot reach a certain site.

App starts fine, the UI loads etc etc.

While its processing the feeds I see the following error.

[error] Process #PID<0.5222.0> raised an exception
** (UndefinedFunctionError) undefined function String.trim/1
    (elixir) String.trim("Youtubers Life v0.7.13 Cracked-3DM Verified")
    (magnetissimo) lib/parsers/isohunt.ex:53: Magnetissimo.Parsers.Isohunt.scrape_torrent_information/1
    (magnetissimo) lib/download_worker.ex:88: Magnetissimo.DownloadWorker.perform/3
    (exq) lib/exq/worker/server.ex:119: anonymous fn/3 in Exq.Worker.Server.dispatch_work/2

Installed all of the dependencies, and tried to run the command "mix ecto.create && mix ecto.migrate"
I'm assuming I need to change something in the config file?

Generated magnetissimo app
** (Mix) The database for Magnetissimo.Repo couldn't be created: an exception was raised:
** (KeyError) key :password not found in: types: true, username: "root", backoff_type: :stop, pool: DBConnection.Connection, database: "template1stname: "localhost", pool_size: 10 lib/keyword.ex:333: Keyword.fetch!/2
(postgrex) lib/postgrex/protocol.ex:444: Postgrex.Protocol.auth_md5/4
(postgrex) lib/postgrex/protocol.ex:361: Postgrex.Protocol.handshake/2
(db_connection) lib/db_connection/connection.ex:134: DBConnection.Connection.connect/2
(connection) lib/connection.ex:623: Connection.enter_connect/5
(stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3

09:33:32.067 [error] GenServer #PID<0.3633.0> terminating
** (KeyError) key :password not found in: types: true, username: "root", backoff_type: :stop, pool: DBConnection.Connection, database: "template1", ome: "localhost", pool_size: 10 lib/keyword.ex:333: Keyword.fetch!/2
(postgrex) lib/postgrex/protocol.ex:444: Postgrex.Protocol.auth_md5/4
(postgrex) lib/postgrex/protocol.ex:361: Postgrex.Protocol.handshake/2
(db_connection) lib/db_connection/connection.ex:134: DBConnection.Connection.connect/2
(connection) lib/connection.ex:623: Connection.enter_connect/5
(stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3
Last message: nil
State: Postgrex.Protocol

c24182d#commitcomment-18803508

@akira mentioned we can use module names instead of plain strings.

Switch them out for the module names:

Bad: Exq.enqueue(Exq, "demonoid", "Magnetissimo.DownloadWorker", [url, "demonoid", "root_url"])
Good: Exq.enqueue(Exq, "demonoid", Magnetissimo.DownloadWorker, [url, "demonoid", "root_url"])

Switch it out where applicable.

Title says it all.

Hi, after helping with EZTV I was thinking of helping on another new integration, and being honest I don't even remember what I did (lol). So I'm thinking to repeat the process and document it in a Example/Wiki/Blogpost fashion. Maybe it helps to get more contributors to push PR with new sites.

Anything of the like already exists? Should I pursue this myself?

Thanks!

Like so

I've seen this multiple times while looking through the listings. Haven't seen it for anything but LimeTorrents.

Just curious, does this just continuously scrape all the sites pausing 100ms between each page, repeating again at the top?

I know the comment on https://github.com/sergiotapia/magnetissimo/blob/master/lib/crawler/thepiratebay.ex#L17 says 5 seconds, but it seems to be 1 * 1 * 100 == 100ms.

Seems excessive to do this over and over without a longer break between a complete crawl.

The site is redirecting to demonoid.cc which renders a 404 not found.

Does anyone have a backup of the demonoid indexed torrents? I remember indexing them but I deleted that database while developing the EZTV parser

Hi,

I'm getting the following error using the last version of Magnetissimo,

Crawling: https://isohunt.tohttp://www.bitlord.com/share/?re=IsoHunt.to&ba=0E3B6B&co=fff&sh=HEYZO-1213-美癡女-淫亂熟女誘惑-甲斐美晴-無碼中文字 幕&ur=https://isohunt.to//torrent_details/17047072/HEYZO-1213-%E7%BE%8E%E7%99%A1%E5%A5%B3-%E6%B7%AB%E4%BA%82%E7%86%9F%E5%A5%B3%E8%AA%98%E6%83%91-%E7%94%B2%E6%96%90%E7%BE%8E%E6%99%B4-%E7%84%A1%E7%A2%BC%E4%B8%AD%E6%96%87%E5%AD%97%E5%B9%95 Error: https://isohunt.tohttp://www.bitlord.com/share/?re=IsoHunt.to&ba=0E3B6B&co=fff&sh=HEYZO-1213-美癡女-淫亂熟女誘惑-甲斐美晴-無碼中文字幕& ur=https://isohunt.to//torrent_details/17047072/HEYZO-1213-%E7%BE%8E%E7%99%A1%E5%A5%B3-%E6%B7%AB%E4%BA%82%E7%86%9F%E5%A5%B3%E8%AA%98%E6%83%91- %E7%94%B2%E6%96%90%E7%BE%8E%E6%99%B4-%E7%84%A1%E7%A2%BC%E4%B8%AD%E6%96%87%E5%AD%97%E5%B9%95 just ain't workin. 19:58:25.599 [error] Process #PID<0.17204.10> raised an exception ** (FunctionClauseError) no function clause matching in Floki.Finder.traverse/4 lib/floki/finder.ex:49: Floki.Finder.traverse(nil, [], %Floki.Selector{attributes: [], classes: ["torrent-header"], combinator: nil, id: n il, type: "h1"}, []) lib/floki/finder.ex:61: Floki.Finder.traverse/4 lib/floki/finder.ex:35: Floki.Finder.find_selectors/2 (magnetissimo) lib/parsers/isohunt.ex:46: Magnetissimo.Parsers.Isohunt.scrape_torrent_information/1 (magnetissimo) lib/download_worker.ex:138: Magnetissimo.DownloadWorker.perform/3 (exq) lib/exq/worker/server.ex:119: anonymous fn/3 in Exq.Worker.Server.dispatch_work/2

& should be shown as & in the names. This used to work, something must have gone wrong with one of the parsers.

Support for either of these sister sites would be extraordinary:

https://torrentz2.eu
https://torrentproject.se

or

http://torrentzwealmisr.onion
http://x4torrentjjjjuxy.onion

I need to figure out how much GenServer's crawling is impacting the user facing side of Magnetissimo.

If the crawler is deteriorating the experience it may be wise to split out into two different projects.

1. The Magnetissimo web UI, users use to search and get magnet links.
2. The Magnetissimo Crawler, backend project that crawls websites and dumps things into Postresql.

There are some errors in the Debian instructions:

sudo apt-get install portgresql redis-server nodejs esl-erlang exlixir git screen
should be (note postgres/elixir):
sudo apt-get install postgresql redis-server nodejs esl-erlang elixir git screen

After creating the database/user

\l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+----------+----------+---------+---------+-----------------------
torrent | torrent | UTF8 | C.UTF-8 | C.UTF-8 |

\dg
List of roles
Role name | Attributes | Member of
-----------+------------------------------------------------+-----------
torrent | Create DB | {}

./config/dev.exs was edited with the proper info:

# Configure your database
config :magnetissimo, Magnetissimo.Repo,
adapter: Ecto.Adapters.Postgres,
database: "torrent",
username: "torrent",
password: "mypass",
hostname: "localhost",
pool_size: 10

Executing: mix ecto.create

** (Mix) The database for Magnetissimo.Repo couldn't be created: FATAL 28000 (invalid_authorization_specification): role "torrent" does not exist

I have no idea how to fix this error, the role/user/db exists
Postgres is active and running on port 5433

I'm thinking on having some kind of ways to categorize or tag torrents, either by machine learning or by user_input

I'd be neat once one has a subset of torrents to movies, to be able to match them by title to an API like TMDb.
like k*t did with movie_posters, ratings, imdb links and such

I faced many times: "[info] Application magnetissimo exited: shutdown"

Reason: ** (WithClauseError) no with clause matching

The debug Log：

[error] GenServer #PID<0.1460.0> terminating
** (WithClauseError) no with clause matching: {:ok, %HTTPoison.Response{body: "\n\n\n<head lang="en">\n<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>\n<meta name="author" content="World Wide Torrents"/>\n<meta name="generator" content="World Wide Torrents"/>\n<meta name="description" content="WorldWide Torrents Community, A place to download TV series, movies, music, games torrents, make friends and to have fun, A Place to call home"/>\n<meta name="keywords" content="torrents, worldwide torrents, WWRG, WWT, torrents download, browse torrents, search torrents, best torrent site, fast torrent download, safe torrents, Movie torrents, TV torrents, Ebook torrents, Music Torrents, Games Torrent, kat alternative, torrentz alternative,"/>\n<link rel="stylesheet" type="text/css" href="themes/Final/css/w3.css"/>\n<link rel="stylesheet" type="text/css" href="themes/Final/css/w3-theme.css">\n<link rel="stylesheet" type="text/css" href="themes/Final/css/glyphicons.css"/>\n<link rel="shortcut icon" href="/themes/NB-TT-2c/images/favicon.ico"/>\n<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.3.0/css/font-awesome.min.css\">\n<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Allerta+Stencil\">\n<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Tangerine\">\n<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Josefin+Sans\">\n\n<link href="https://fonts.googleapis.com/css?family=Lora\" rel="stylesheet">\n<script type="text/javascript" src="/backend/java_klappe.js"></script>\n<style>.w3-allerta{font-family:"Allerta Stencil",Sans-serif;}.w3-tangerine{font-family:'Tangerine',serif;font-size:48px;}.w3-jose{font-family:'Lora',serif;font-style:normal;font-weight:800;src:local('Josefin Sans'),local('JosefinSans'),url(https://fonts.gstatic.com/s/josefinsans/v9/xgzbb53t8j-Mo-vYa23n5j0LW-43aMEzIO6XUTLjad8.woff2) format('woff2');unicode-range:U+0100-024F,U+1E00-1EFF,U+20A0-20AB,U+20AD-20CF,U+2C60-2C7F,U+A720-A7FF;}.fix-header{list-style-type:none;margin:0;padding:0;background-color:#333;position:fixed;top:0;width:100%;}</style>\n<title>WorldWide Torrents : Browse Torrents</title>\n\n\n \n<div class="w3-top w3-card-8">\n<ul class="w3-navbar w3-theme-d2 w3-left-align w3-large">\n<li class="w3-hide-medium w3-hide-large w3-opennav w3-right">\n<a class="w3-padding-large w3-hover-white w3-large w3-theme-d2" href="javascript:void(0);" onclick="openNav()"><i class="fa fa-bars">\n\n

Just an idea,

Most of the websites include the category of the torrent, wouldn't it be interesting to store the category in the database for filtering in the search, etc ?

Great project, Thanks!

Occasionally uploaders on torrentdownloads or thepiratebay will submit torrents with 9000+ or even 20,000+ seeders. These torrents are labelled to appear legitimate but contain malware.

I'm curious to know how you might handle scrubbing the database of fake/spam torrents? Through postgres? Command line? Or modifying magnetissimo to ignore fake torrents altogether?

Small Feature request. In the UI please make the "Size" column human readable. (KB/MB/GB)

• CentOS 7.3 (all updates installed)
• Elixir 0.12.5.2-el7

/opt # yum install elixir postgresql96-server
/opt # git clone https://github.com/sergiotapia/magnetissimo.git --depth=1
/opt # cd magnetissimo/
/opt/magnetissimo [0] # mix deps.get

** (UndefinedFunctionError) undefined function: Mix.compilers/0
    (mix) Mix.compilers()
    mix.exs:9: Magnetissimo.Mixfile.project/0
    (mix) lib/mix/project.ex:348: Mix.Project.get_project_config/1
    (mix) lib/mix/project.ex:52: Mix.Project.push/2
    (stdlib) lists.erl:1248: :lists.foldl/3

Sorry I'm not sure what other debugging information might be helpful here, if someone can point me in the right direction I can go from there.

https://zooqle.com/
https://eztv.ag/
http://gamestorrent.co/
http://www.btstorrent.cc/
https://concen.org/torrents/
https://metasearch.torrentproject.se/

Hello,

Thanks for your wonderful product. It appears that i've hit a snag,
After running the application form 2 or 3 days it enters a loop just showing this message:

warning: Float.to_string/2 is deprecated, use :erlang.float_to_binary/2 instead
(elixir) lib/float.ex:225: Float.to_string/2
(exq) lib/exq/redis/job_queue.ex:131: Exq.Redis.JobQueue.scheduler_dequeue/2
(exq) lib/exq/scheduler/server.ex:68: Exq.Scheduler.Server.dequeue/1
(exq) lib/exq/scheduler/server.ex:56: Exq.Scheduler.Server.handle_info/2
(stdlib) gen_server.erl:601: :gen_server.try_dispatch/4

If i restart the binary (mix phoenix.server) it restarts indexing correctly.

I've tested this on a VM and a Container... the errors are the same.

Is there a problem or am i doing something wrong?

May or may not be related to #49, #34.

[error] GenServer #PID<0.1864.0> terminating
** (WithClauseError) no with clause matching: {:ok, %HTTPoison.Response{body: "<html>\n<head><title>500 Internal Server Error</title></head>\n<body bgcolor=\"white\">\n<center><h1>500 Internal Server Error</h1></center>\n<hr><center>nginx/1.4.6 (Ubuntu)</center>\n</body>\n</html>\n", headers: [{"Date", "Mon, 30 Jan 2017 ..:..:.. GMT"}, {"Content-Type", "text/html"}, {"Transfer-Encoding", "chunked"}, {"Connection", "keep-alive"}, {"Set-Cookie", "__cfduid={…}; expires=Tue, 30-Jan-18 11:26:59 GMT; path=/; domain=.eztv.ag; HttpOnly"}, {"X-Varnish", "{…}"}, {"Age", "1"}, {"Via", "1.1 varnish-v4"}, {"Server", "cloudflare-nginx"}, {"CF-RAY", "{…}"}], status_code: 500}}
    (magnetissimo) lib/crawler/helper.ex:15: Magnetissimo.Crawler.Helper.download/1
    (magnetissimo) lib/crawler/eztv.ex:37: Magnetissimo.Crawler.EZTV.process/2
    (magnetissimo) lib/crawler/eztv.ex:26: Magnetissimo.Crawler.EZTV.handle_info/2
    (stdlib) gen_server.erl:601: :gen_server.try_dispatch/4
    (stdlib) gen_server.erl:667: :gen_server.handle_msg/5
    (stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3
Last message: :work
State: {[page_link: "https://eztv.ag/page_300", page_link: "https://eztv.ag/page_299", page_link: "https://eztv.ag/page_298", page_link: "https://eztv.ag/page_297", page_link: "https://eztv.ag/page_296", page_link: "https://eztv.ag/page_295", page_link: "https://eztv.ag/page_294", page_link: "https://eztv.ag/page_293", page_link: "https://eztv.ag/page_292", page_link: "https://eztv.ag/page_291", page_link: "https://eztv.ag/page_290", page_link: "https://eztv.ag/page_289", page_link: "https://eztv.ag/page_288", page_link: "https://eztv.ag/page_287", page_link: "https://eztv.ag/page_286", page_link: "https://eztv.ag/page_285", page_link: "https://eztv.ag/page_284", page_link: "https://eztv.ag/page_283", page_link: "https://eztv.ag/page_282", page_link: "https://eztv.ag/page_281", page_link: "https://eztv.ag/page_280", page_link: "https://eztv.ag/page_279", page_link: "https://eztv.ag/page_278", page_link: "https://eztv.ag/page_277", page_link: "https://eztv.ag/page_276", page_link: "https://eztv.ag/page_275", page_link: "https://eztv.ag/page_274", page_link: "https://eztv.ag/page_273", page_link: "https://eztv.ag/page_272", page_link: "https://eztv.ag/page_271", page_link: "https://eztv.ag/page_270", page_link: "https://eztv.ag/page_269", page_link: "https://eztv.ag/page_268", page_link: "https://eztv.ag/page_267", page_link: "https://eztv.ag/page_266", page_link: "https://eztv.ag/page_265", page_link: "https://eztv.ag/page_264", page_link: "https://eztv.ag/page_263", page_link: "https://eztv.ag/page_262", page_link: "https://eztv.ag/page_261", page_link: "https://eztv.ag/page_260", page_link: "https://eztv.ag/page_259", page_link: "https://eztv.ag/page_258", page_link: "https://eztv.ag/page_257", page_link: "https://eztv.ag/page_256", page_link: "https://eztv.ag/page_255", page_link: "https://eztv.ag/page_254", page_link: "https://eztv.ag/page_253", page_link: "https://eztv.ag/page_252", ...], [page_link: "https://eztv.ag/page_1", page_link: "https://eztv.ag/page_2", page_link: "https://eztv.ag/page_3", page_link: "https://eztv.ag/page_4", page_link: "https://eztv.ag/page_5", page_link: "https://eztv.ag/page_6", page_link: "https://eztv.ag/page_7", page_link: "https://eztv.ag/page_8", page_link: "https://eztv.ag/page_9", page_link: "https://eztv.ag/page_10", page_link: "https://eztv.ag/page_11", page_link: "https://eztv.ag/page_12", page_link: "https://eztv.ag/page_13", page_link: "https://eztv.ag/page_14", page_link: "https://eztv.ag/page_15", page_link: "https://eztv.ag/page_16", page_link: "https://eztv.ag/page_17", page_link: "https://eztv.ag/page_18", page_link: "https://eztv.ag/page_19", page_link: "https://eztv.ag/page_20", page_link: "https://eztv.ag/page_21", page_link: "https://eztv.ag/page_22", page_link: "https://eztv.ag/page_23", page_link: "https://eztv.ag/page_24", page_link: "https://eztv.ag/page_25", page_link: "https://eztv.ag/page_26", page_link: "https://eztv.ag/page_27", page_link: "https://eztv.ag/page_28", page_link: "https://eztv.ag/page_29", page_link: "https://eztv.ag/page_30", page_link: "https://eztv.ag/page_31", page_link: "https://eztv.ag/page_32", page_link: "https://eztv.ag/page_33", page_link: "https://eztv.ag/page_34", page_link: "https://eztv.ag/page_35", page_link: "https://eztv.ag/page_36", page_link: "https://eztv.ag/page_37", page_link: "https://eztv.ag/page_38", page_link: "https://eztv.ag/page_39", page_link: "https://eztv.ag/page_40", page_link: "https://eztv.ag/page_41", page_link: "https://eztv.ag/page_42", page_link: "https://eztv.ag/page_43", page_link: "https://eztv.ag/page_44", page_link: "https://eztv.ag/page_45", page_link: "https://eztv.ag/page_46", page_link: "https://eztv.ag/page_47", page_link: "https://eztv.ag/page_48", page_link: "https://eztv.ag/page_49", ...]}

Would you consider adding sortability to the fields? It might be nice to sort searches by number of seeds I'm thinking so that search results with 0 seeds don't clutter up the top of the results listing.

It would be awesome to the support the Torznab api, as it would allow many automation tools to work with this project.

Here is documentation from Sonarr, an automation tool.
https://github.com/Sonarr/Sonarr/wiki/Implementing-a-Torznab-indexer

[warn] 404 Page not Found on https://eztv.ag/page_1
[error] GenServer #PID<0.457.0> terminating
** (FunctionClauseError) no function clause matching in Floki.Finder.traverse/4
lib/floki/finder.ex:61: Floki.Finder.traverse(nil, [], %Floki.Selector{attributes: [], classes: ["epinfo"], combinator: nil, id: nil, namespace: nil, type: "a"}, [])
lib/floki/finder.ex:73: Floki.Finder.traverse/4
lib/floki/finder.ex:47: Floki.Finder.find_selectors/2
(magnetissimo) lib/crawler/eztv.ex:62: Magnetissimo.Crawler.EZTV.torrent_links/1
(magnetissimo) lib/crawler/eztv.ex:37: Magnetissimo.Crawler.EZTV.process/2
(magnetissimo) lib/crawler/eztv.ex:26: Magnetissimo.Crawler.EZTV.handle_info/2
(stdlib) gen_server.erl:601: :gen_server.try_dispatch/4
(stdlib) gen_server.erl:667: :gen_server.handle_msg/5
(stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3
Last message: :work
State: {[page_link: "https://eztv.ag/page_300", page_link: "https://eztv.ag/page_299", page_link: "https://eztv.ag/page_298", page_link: "https://eztv.ag/page_297", page_link: "https://eztv.ag/page_296",
page_link: "https://eztv.ag/page_295", page_link: "https://eztv.ag/page_294", page_link: "https://eztv.ag/page_293", page_link: "https://eztv.ag/page_292", page_link: "https://eztv.ag/page_291", page_link
: "https://eztv.ag/page_290", page_link: "https://eztv.ag/page_289", page_link: "https://eztv.ag/page_288", page_link: "https://eztv.ag/page_287", page_link: "https://eztv.ag/page_286", page_link: "https:
//eztv.ag/page_285", page_link: "https://eztv.ag/page_284", page_link: "https://eztv.ag/page_283", page_link: "https://eztv.ag/page_282", page_link: "https://eztv.ag/page_281", page_link: "https://eztv.ag
/page_280", page_link: "https://eztv.ag/page_279", page_link: "https://eztv.ag/page_278", page_link: "https://eztv.ag/page_277", page_link: "https://eztv.ag/page_276", page_link: "https://eztv.ag/page_275
", page_link: "https://eztv.ag/page_274", page_link: "https://eztv.ag/page_273", page_link: "https://eztv.ag/page_272", page_link: "https://eztv.ag/page_271", page_link: "https://eztv.ag/page_270", page_l
ink: "https://eztv.ag/page_269", page_link: "https://eztv.ag/page_268", page_link: "https://eztv.ag/page_267", page_link: "https://eztv.ag/page_266", page_link: "https://eztv.ag/page_265", page_link: "htt
ps://eztv.ag/page_264", page_link: "https://eztv.ag/page_263", page_link: "https://eztv.ag/page_262", page_link: "https://eztv.ag/page_261", page_link: "https://eztv.ag/page_260", page_link: "https://eztv
.ag/page_259", page_link: "https://eztv.ag/page_258", page_link: "https://eztv.ag/page_257", page_link: "https://eztv.ag/page_256", page_link: "https://eztv.ag/page_255", page_link: "https://eztv.ag/page_
254", page_link: "https://eztv.ag/page_253", page_link: "https://eztv.ag/page_252", ...], [page_link: "https://eztv.ag/page_1", page_link: "https://eztv.ag/page_2", page_link: "https://eztv.ag/page_3", pa
ge_link: "https://eztv.ag/page_4", page_link: "https://eztv.ag/page_5", page_link: "https://eztv.ag/page_6", page_link: "https://eztv.ag/page_7", page_link: "https://eztv.ag/page_8", page_link: "https://e
ztv.ag/page_9", page_link: "https://eztv.ag/page_10", page_link: "https://eztv.ag/page_11", page_link: "https://eztv.ag/page_12", page_link: "https://eztv.ag/page_13", page_link: "https://eztv.ag/page_14"
, page_link: "https://eztv.ag/page_15", page_link: "https://eztv.ag/page_16", page_link: "https://eztv.ag/page_17", page_link: "https://eztv.ag/page_18", page_link: "https://eztv.ag/page_19", page_link: "
https://eztv.ag/page_20", page_link: "https://eztv.ag/page_21", page_link: "https://eztv.ag/page_22", page_link: "https://eztv.ag/page_23", page_link: "https://eztv.ag/page_24", page_link: "https://eztv.a
g/page_25", page_link: "https://eztv.ag/page_26", page_link: "https://eztv.ag/page_27", page_link: "https://eztv.ag/page_28", page_link: "https://eztv.ag/page_29", page_link: "https://eztv.ag/page_30", pa
ge_link: "https://eztv.ag/page_31", page_link: "https://eztv.ag/page_32", page_link: "https://eztv.ag/page_33", page_link: "https://eztv.ag/page_34", page_link: "https://eztv.ag/page_35", page_link: "http
s://eztv.ag/page_36", page_link: "https://eztv.ag/page_37", page_link: "https://eztv.ag/page_38", page_link: "https://eztv.ag/page_39", page_link: "https://eztv.ag/page_40", page_link: "https://eztv.ag/pa
ge_41", page_link: "https://eztv.ag/page_42", page_link: "https://eztv.ag/page_43", page_link: "https://eztv.ag/page_44", page_link: "https://eztv.ag/page_45", page_link: "https://eztv.ag/page_46", page_l
ink: "https://eztv.ag/page_47", page_link: "https://eztv.ag/page_48", page_link: "https://eztv.ag/page_49", ...]}
[info] Application magnetissimo exited: shutdown

When searching for a tv episode using the typical nomenclature (example: "Magicians s02e01") doesn't catch all the results it should. I believe this is because the search is looking for the exact string entered, not performing a proper search of multiple strings. The example "Magicians s02e01" won't hit on "Magicians US s02e01" because of the "US".

Example of searching just "Magicians" so you can see all the available s02e01 torrents.

Results of searching "Magicians s02e01"

Make sure the text the user searched for is still visible on page load.

I really like this app, but I'd like to have the link to the actual torrent page somewhere. I miss information like torrent contents (files) and description about framerate, resolutions etc. Unless we can scrape these, a simple link to the individual torrent page could be useful.

It's possible to add t411.li and torrent9.biz ?

What's the best way to automatically redeploy the phoenix.server when it stops? If I wanted to leave magnetissimo unattended for a prolonged period of time.

A good majority of the users who will use Magnetissimo are not developers.

We need to write better documentation on how it works, what it does and how to run it locally.

In the future we'll offer a compiled release version of the app. Possible options include exrm but I'm entirely new to Elixir releases, I'm all ears for suggestions.

Errors in the Usage: Debian 7 wiki's command examples:

Typos:

sudo apt-get install portgresql redis-server nodejs esl-erlang exlixir git screen

shall be

sudo apt-get install postgresql redis-server esl-erlang elixir git screen

Deps error:
NodeJS doesn't seam to be required, at least there is no package.json file.

I recently deployed magnetissimo to Heroku and wrote up the steps:

Run it on Heroku

To run it on Heroku, you need to:

Clone the git repository from Github:

$ git clone [email protected]:sergiotapia/magnetissimo.git
$ cd magentissimo

Create the Heroku application using the Elixir build pack.

$ heroku create --buildpack "https://github.com/HashNuke/heroku-buildpack-elixir.git"
Creating mysterious-meadow-6277... done, stack is cedar-14
Buildpack set. Next release on mysterious-meadow-6277 will use https://github.com/HashNuke/heroku-buildpack-elixir.git.
https://mysterious-meadow-6277.herokuapp.com/ | https://git.heroku.com/mysterious-meadow-6277.git
Git remote heroku added

Create the Postgres database using the Heroku free plan:

$ heroku addons:create heroku-postgresql:hobby-dev

Generate a secret key for the project:

$ mix deps.get
$ mix phoenix.gen.secret
xvafzY4y01jYuzLm3ecJqo008dVnU3CN4f+MamNd1Zue4pXvfvUjbiXT8akaIF53

Add the following environment variables, including the previously generated key:

$ heroku config:set POOL_SIZE=18
$ heroku config:set SECRET_KEY_BASE=xvafzY4y01jYuzLm3ecJqo008dVnU3CN4f+MamNd1Zue4pXvfvUjbiXT8akaIF53

Edit config/prod.exs:

Add secret_key_base: System.get_env("SECRET_KEY_BASE") to the Magnetissimo.Endpoint block.

Add a block for configuring the production database:

config :magnetissimo, Magnetissimo.Repo,
  adapter: Ecto.Adapters.Postgres,
  url: System.get_env("DATABASE_URL"),
  pool_size: String.to_integer(System.get_env("POOL_SIZE") || "10"),
  ssl: true

Comment out import_config "prod.secret.exs".

Add a file called Procfile in the directory root with the following contents:

web: MIX_ENV=prod mix phoenix.server

Commit your changes:

$ git add .
$ git commit -am "deploy to heroku"

Deploy the app:

$ git push heroku master

Configure the database:

$ heroku run "POOL_SIZE=2 mix ecto.migrate"

I'm not sure what the best way is to merge these changes into the wiki, so I'm posting them here for now.

I think it would be safe to have the standard redis port and localhost set as default.

Adding a parser for the rarbg.to torrent site.

I will start working on this, as well as documenting (see issue #16)

I LOVE goal 1 of no javascript, I went to install then I saw npm install in the instructions, looked into it and indeed it uses nodejs ... JS being javascript.

wat?

Is it possible to deploy this application in a docker container?

I installed Magnetissimo today by clicking Deploy to Heroku. Did not tweak any settings. Just a few hours later I'm getting Heroku's Application Error page. The log suggests that the program got a 403 Forbidden from mononova and it crashed because of that.

2017-02-14T16:43:03.665578+00:00 app[web.1]: Downloading page: https://monova.org/latest?page=1
2017-02-14T16:43:03.795084+00:00 app[web.1]: 16:43:03.794 [info] Torrent saved to database: Tuba Skinny - 2014 - Owl Call Blues [FLAC]
2017-02-14T16:43:03.806473+00:00 app[web.1]: 16:43:03.806 [info] Torrent saved to database: Traffic - 1974 - When The Eagle Flies (2003 Reissue) [[email protected]]
2017-02-14T16:43:03.827054+00:00 app[web.1]: 16:43:03.826 [info] Torrent saved to database: Traffic - 1974 - When The Eagle Flies (2003 Reissue) [FLAC]
2017-02-14T16:43:03.837873+00:00 app[web.1]: 16:43:03.837 [info] Torrent saved to database: Taken.Heart.2017.HDRip.XViD-ETRG
2017-02-14T16:43:03.851448+00:00 app[web.1]: 16:43:03.850 [info] Torrent saved to database: 30.For.30.S03E19.This.Was.The.XFL.HDTV.x264-YesTV - [SRIGGA]
2017-02-14T16:43:03.902476+00:00 app[web.1]: 16:43:03.902 [error] GenServer #PID<0.398.0> terminating
2017-02-14T16:43:03.902483+00:00 app[web.1]: ** (WithClauseError) no with clause matching: {:ok, %HTTPoison.Response{body: "<html>\n<head><title>403 Forbidden</title></head>\n<body bgcolor=\"white\">\n<center><h1>403 Forbidden</h1></center>\n<hr><center>nginx</center>\n</body>\n</html>\n", headers: [{"Date", "Tue, 14 Feb 2017 16:43:03 GMT"}, {"Content-Type", "text/html; charset=utf-8"}, {"Transfer-Encoding", "chunked"}, {"Connection", "keep-alive"}, {"Set-Cookie", "__cfduid=d366f3c916a5580007c2e1485b41c40b31487090583; expires=Wed, 14-Feb-18 16:43:03 GMT; path=/; domain=.monova.org; HttpOnly"}, {"Server", "cloudflare-nginx"}, {"CF-RAY", "3311fa14de822975-DUB"}], status_code: 403}}
2017-02-14T16:43:03.902488+00:00 app[web.1]:     (magnetissimo) lib/crawler/helper.ex:15: Magnetissimo.Crawler.Helper.download/1
2017-02-14T16:43:03.902489+00:00 app[web.1]:     (magnetissimo) lib/crawler/monova.ex:36: Magnetissimo.Crawler.Monova.process/2
2017-02-14T16:43:03.902490+00:00 app[web.1]:     (magnetissimo) lib/crawler/monova.ex:25: Magnetissimo.Crawler.Monova.handle_info/2
2017-02-14T16:43:03.902491+00:00 app[web.1]:     (stdlib) gen_server.erl:615: :gen_server.try_dispatch/4
2017-02-14T16:43:03.902491+00:00 app[web.1]:     (stdlib) gen_server.erl:681: :gen_server.handle_msg/5
2017-02-14T16:43:03.902492+00:00 app[web.1]:     (stdlib) proc_lib.erl:240: :proc_lib.init_p_do_apply/3
2017-02-14T16:43:03.971042+00:00 heroku[router]: at=error code=H13 desc="Connection closed without response" method=GET path="/summary" host=gdrsmagnetissimo.herokuapp.com request_id=f2312591-9acf-4e80-ad0a-1be2bfeb1bdc fwd="89.69.141.112" dyno=web.1 connect=0ms service=998ms status=503 bytes=0
2017-02-14T16:43:04.336193+00:00 app[web.1]: 16:43:04.332 [info] Application magnetissimo exited: shutdown
2017-02-14T16:43:05.883369+00:00 app[web.1]: {"Kernel pid terminated",application_controller,"{application_terminated,magnetissimo,shutdown}"}

After running magnetissimo for some time the application crashes, it is caused by the eztv crawler.
I have screenshot of it when it happend:
https://i.imgur.com/JMh01i5.png

Small cosmetic issue with the size column. Resulted from the fix of issue #37

Sometimes when the size of the file exceeds 6 or so characters, the "MB" tag gets put onto a new line. I believe this is due to a hard coded rowsize for the column. Not sure how you want to fix it. Increase column size so it stays on one line, remove numbers after decimal place, or make all columns resize-able in the ui. Though that last one would probably take a lot more work than the others.

I wasn't able to get magnetissimo up and running locally, until I added a few steps. You have to have postgres up and running. I wasn't able to make an update to the wiki. I cloned it, and made the update locally. Let me know if this works for you.

Run it locally

To run Magnetissimo locally, you need to have Elixir and Postgresql installed.

Install Elixer

brew update

brew install elixir

Install Postgresql

brew install postgres

To start Magnetissimo locally:

• Install dependencies with mix deps.get
• Edit dev.exs and set your database username and password (see example below).
• Start postgres pg_ctl -D /usr/local/var/postgres -l /usr/local/var/postgres/server.log start
• Create and migrate your database with mix ecto.create && mix ecto.migrate
• Start Phoenix endpoint with mix phoenix.server
• Now you can visit localhost:4000 from your browser.

# Configure your database inside config/dev.exs
config :magnetissimo, Magnetissimo.Repo,
  adapter: Ecto.Adapters.Postgres,
  database: "magnetissimo_dev",
  username: "CHANGEME",
  password: "CHANGEME",
  hostname: "localhost",
  port: 5432,
  pool_size: 50

The only thing I get when I go to localhost:4000 is the landing page.

Centos7 followed the guide word for word.

I know, I know - streaming is bad for healthy torrents. But that was back in 2007, these days the swarm is gigantic.

Should I add in a button people can click and stream the content in their browser?

I imagine this would only work for torrents that have 1 file, and a playable file at that.

I could probably get in touch with a streaming service and ask them if they can help out, for example: https://instant.io/

It uses WebRTC so the traffic is peer to peer. Hm... food for thought.

I want to offer to write the guide, since I want to experiment with this and am not fond of heroku for scaling, I mean it's easy but it ain't cheap.

What would be the preferable OS for the VPS? Ubuntu? CentOS? Which Versions?

Should I figure that out based on the requirements of the project and its dependencies?

Would be cool to scrap libgen, it has a latest endpoints we can paginate on and also gives magnet links to torrent downloads of files, might be worth a look on

Thank you for creating magnetissimo!

Several questions:

1. How can I omit several searches without breaking functionality? I only want to crawl thepiratebay and extratorrents.

2. Could I modify thepiratebay or demonoid crawlers to use their tor .onion domains? If I were using a transparent proxy, would the crawl tell the difference?

3. Is it possible to stop the magnetissimo from saving magnets with 0/0 seeds/leechers? I would prefer to only store torrents that have x amount of seeds. If so, how to?

4. How can I organize the main page so that it shows the most seeded torrents at the top?

5. Could these torrent sites easily block your crawler or define this type of activity as malicious/abusive?

does the crawler update or override the seeders & leechers count in the database when it crawls the same torrent day after day?

Hello,

I've upgraded my magnetissimo today, and i got the following error:

[debug] QUERY ERROR db=28.6ms queue=0.2ms
INSERT INTO "torrents" ("filesize","leechers","magnet","name","seeders","source","inserted_at","updated_at") VALUES ($1,$2,$3,$4,$5,$6,$7,$8) RETURNING "id" ["1073741824", 0, "magnet:?xt=urn:btih:0XXXXXXXXXXXXXXD1E900&dn=blablabla.torret=udp%3A%2F%2Fopen.demonii.com%3A1337%2Fannounce&tr=udp%3A%2F%2Ftracker.openbittorrent.com%3A80%2Fannounce&tr=udp%3A%2F%2F9.rarbg.to%3A2710%2Fannounce&tr=udp%3A%2F%2Ftracker.trackerfix.com%3A80%2Fannounce&tr=udp%3A%2F%2Ftracker.coppersurfer.tk%3A6969%2Fannounce", "blablabla.torrent", 0, "TorrentDownloads", {{2016, 9, 30}, {14, 52, 47, 0}}, {{2016, 9, 30}, {14, 52, 47, 0}}]

[error] Process #PID<0.724.0> raised an exception
** (CaseClauseError) no case clause matching: {:error, #Ecto.Changeset<action: :insert, changes: %{filesize: "1073741824", leechers: 0, magnet: "magnet:?xt=blablablabla", name: "blabla blabla7", seeders: 0, source: "TorrentDownloads"}, errors: [magnet: {"has already been taken", []}], data: #Magnetissimo.Torrent<>, valid?: false>}
(magnetissimo) web/models/torrent.ex:51: Magnetissimo.Torrent.save_torrent/1
(exq) lib/exq/worker/server.ex:119: anonymous fn/3 in Exq.Worker.Server.dispatch_work/2

The DB is up and running correctly, i see connection.
Other thing that i found is that the images and css's are not loading correctly:

Hi Sergio!

I followed the Debian 7 guide and installed magnetissimo on my Ubuntu 16.04 machine. Runs great, but the Web UI is not showing up correctly, since css/js & images cannot be found by the router. Node.js is installed on my machine & npm install completed successfully... something else I need to do?