btdig / dhtcrawler2 Goto Github PK

Hi,

I pulled the utorrent search plugin from the web archive[1] and modified it for btdig.com. I am unable to test it.

I'm not sure it will even work with HTTPS (the original one used http):

<?xml version="1.0"?>
<OpenSearchDescription>
<ShortName>BTDig.com</ShortName>
<Description>Search torrents BTDigg</Description>
<Url template="https://btdig.com/search?q="/>
</OpenSearchDescription>

[1] http://web.archive.org/web/20160129043731/http://btdigg.org/plugins/btdigg.btsearch

Do you know if there is support for emule / ed2k links? There's a lot of rare files that are on there, and I seem to recall something similar to btdigg.com which had a decent amount of observed files.

This site had a collection of ed2k links as well:
https://web.archive.org/web/20131029071638/http://emule.synthasite.com/

There is also Bitzi.com, which they said they would release the database of files, but they never did. At least it was well indexed and the links are still working. This is the last version of the index:
http://web.archive.org/web/20131222112000/http://bitzi.com/browse/

utorrent or bittorrent can not find file. Stucked in loading screen

continue #3 (comment)

All the admins of https://github.com/btdig are shadowbanned on GitHub.

Even if they respond, we won't see it.

This should be moved to a more piracy-friendly hosting, something in Russia or China?

This should be moved to a more piracy-friendly hosting, something in Russia or China?

russia and china sound like bad candidates, but the whole world is heading towards more censorship... for example the digital services act to censor "illegal" content in europe.

i would prefer a distributed git solution, hosted on clearnet (github.com, gitlab.com, try.gitea.io, codeberg.org, sr.ht, ...) and darknet (darktea, humanrightstech, ... see also my darknet-git-hosting-services) and radicle (radicle is no longer based on IPFS) or git-ssb.

see also

• https://github.com/2gatherproject/decentralized-social-apps-guide#githubgitlab-alternatives
• https://github.com/gdamdam/awesome-decentralized-web#collaboration
• https://groups.google.com/a/monadic.xyz/g/radicle/c/ctR0tgua7Wo

with multiple git repos, we have the problem of syncing between multiple git repos. issues and PRs could be exported to the git repo, so they are shared across all mirrors. for github there is gh2md, or generally git-bug. but this syncing-of-user-feedback would require some moderation, to avoid abuse by malicious users...

as a communication channel, nostr (which sucks because nostr is based on JSON which is slow, and because nostr does not have a chain of events) or ssb (secure scuttlebutt) would be interesting, because they have a "nomadic identity" feature. or zeronet (based on bittorrent)...?

by the way, i have archived the github issues in my github-issues branch with archive-github-issues.sh

The torrent "dragon.-ball.-z.-saga.-saiyajins.-completa.-1080p.-sd.-dublado.-edicao-csj" is dead, I removed the files from "Internet Archive" because I made a better version. If you could delet that torrent from you site the new and better torrent file would be the only option, thanks.

I was in btdig.com looking for an adult performer and after a few pages, the algoritm suggested some links with the description of "child porn". Brothers, that is disgusting and a fellony, no matter the nationality or country that you live in. It seems tha the site contains a lot of links to archives like that.
I urge and beg you guys to not tolarate that kind of content in the site. So, please, I ask the admins to search for it and delete or block that kind of content from the site.

in fact 99% of them are so dead!

Dear Sirs,

This communication serves as a statement that

1. We are the duly authorized representative of the rights holders for:

NINTENDO for Super Mario Bros. Wonder (NSW)
NINTENDO for Detective Pikachu Returns (NSW)
NINTENDO for Pikmin 4 (NSW)
NINTENDO for Everybody 1-2-Switch! (NSW)
NINTENDO for The Legend Of Zelda: Tears Of The Kingdom (NSW)
NINTENDO for Kirby’s Return to Dream Land Deluxe (NSW)
NINTENDO for Fire Emblem Engage (NSW)
NINTENDO for Pokemon Scarlet (NSW)
NINTENDO for Pokemon Violet (NSW)
NINTENDO for Splatoon 3 (NSW)
NINTENDO for Xenoblade Chronicles 3 (NSW)
NINTENDO for Fire Emblem Warriors: Three Hopes (NSW)
NINTENDO for Mario Strikers: Battle League (NSW)
NINTENDO for Nintendo Switch Sports (NSW)
NINTENDO for Kirby and the Forgotten Land (NSW)
NINTENDO for Yoshi's Crafted World (NSW)
NINTENDO for Xenoblade Chronicles: Definitive Edition (NSW)
NINTENDO for The Legend of Zelda: Skyward Sword HD (NSW)
NINTENDO for The Legend of Zelda: Link's Awakening (NSW)
NINTENDO for Splatoon 2 (NSW)
NINTENDO for Pokemon: Let´s Go, Pikachu! (NSW)
NINTENDO for Pokemon: Let´s Go, Eevee! (NSW)
NINTENDO for Pokemon Mystery Dungeon: Rescue Team DX (NSW)
NINTENDO for Pikmin 3 Deluxe (NSW)
NINTENDO for Paper Mario: The Origami King (NSW)
NINTENDO for Miitopia (NSW)
NINTENDO for Metroid Dread (NSW)
NINTENDO for Mario Tennis Aces (NSW)
NINTENDO for Mario & Sonic at the Olympic Games Tokyo 2020 (NSW)
NINTENDO for Hyrule Warriors: Age of Calamity (NSW)
NINTENDO for Dr. Kawashima’s Brain Training (NSW)
NINTENDO for Pokemon Legends: Arceus (NSW)
NINTENDO for Big Brain Academy: Brain vs Brain (NSW)
NINTENDO for Pokemon Shining Pearl (NSW)
NINTENDO for Pokemon Brilliant Diamond (NSW)
NINTENDO for Mario Party Superstars (NSW)
NINTENDO for Metroid Dread (NSW)
NINTENDO for Super Mario 3D World + Bowser’s Fury (NSW)
NINTENDO for Animal Crossing: New Horizons (NSW)
NINTENDO for Pokemon Shield (NSW)
NINTENDO for Pokemon Sword (NSW)
NINTENDO for Ring Fit Adventure (NSW)
NINTENDO for Luigi’s Mansion 3 (NSW)
NINTENDO for Mario Kart Tour (mobile)
NINTENDO for Super Mario Party (NSW)
NINTENDO for Super Mario Odyssey (NSW)
NINTENDO for Super Smash Bros. Ultimate (NSW)
NINTENDO for New Super Mario Bros. U Deluxe (NSW)
NINTENDO for Mario Kart 8 Deluxe (NSW)
NINTENDO for The Legend Of Zelda: Breath Of The Wild (NSW)

2. These rights are being violated by material available upon your site at the following URL(s):

https://btdig.com/3fb55e3f18c989122573e8a4ee22851124f44ebf/warioware-move-it
https://btdig.com/b2ea2f78c15d84c7721ed3aa0f2f2b1e8fd7feb4/warioware-move-it
https://btdig.com/850c06ba901a4626e082e933ca3469c6aed4102b/warioware-move-it

3. I have a good faith belief that the use of this material in such a fashion is not authorized by the copyright holder, the copyright holder's agent, or the law;
4. Under penalty of perjury, I state that the information contained in this notification is accurate, and that I am authorized to act on the behalf of the rights holder for the material in question;
5. I may be contacted by the following methods:

Dieter Denzer

operations / customer service
mbargo services

Sonopress GmbH
Carl-Bertelsmann-Str. 161 F
33332 Gütersloh
Germany

Phone: +49 (0) 52 41 - 80-72996
Fax : +49 (0) 52 41 - 80-66928
E-mail: [email protected]

They seem to have their stuff gone, and they never download anything.

Hello! I feel like I have to report this weird problem. A popup appears when I search for this keyword "Natalia Lafourcade". It redirects the browser to ads for unwanted apps, games... This is really annoying me because the popup only appears when I search for that specific keyword. I do not get this annoying popup for any other keywords that I used. It seems really really weird.

Utilizing https://developer.mozilla.org/en-US/docs/Web/CSS/@media/prefers-color-scheme

Since yesterday the latest search results for anything are 1-2 days old. When you click recent findings, you can see new magnets in search results are being uploaded every minute but nothing new shows when u directly search for it via age or relevance. Haven't experienced this glitch before.

No function! Upload the neighed

continue kevinlynx#12

submit hash manually feature for user who newly created torrent should be helpful

alternative solutions:

• solidtorrents.to has an add torrent page, which allows adding new torrents by the torrent hash. currently, only v1 torrents are supported, and v2 torrents are ignored

• search exact sequence of words, for example "Some.Movie.2020-YIFY.mkv" or string(Some.Movie.2020-YIFY.mkv)
• search glob pattern, for example hello wor*
• semantic search
  • search in title, for example title(some words to search)
  • search in file names, for example file(some-file.txt)
  • search in file paths, for example path(**/relative/path/to/file.txt) or path(absolute/path/to/file.txt)
  • search by file hash (v2 torrents), for example btrh(xxxxxxxx) (btrh = bittorrent root hash)
• negate filters, for example title(good words) not(title(bad words))
• logic operators
  • and is default
  • or, for example (title(sometitle) size(over 1 GB)) or title(othertitle)
  • xor
• filter
  • filter by age, for example age(under 1 year) or age(between year 2000 and year 2010)
  • filter by size, for example size(over 1 GB)
  • filter by number of files, for example files(over 10)
• sort
  • sort by age, for example sort(age) and sort(-age)
  • sort by size, for example sort(size) and sort(-size)
  • sort by number of files, for example sort(files) and sort(-files)

current situation:

• dhtcrawler2 has optional support for sphinxsearch, which was open-source until version 2, and now is closed source, see sphinxsearch.com. sphinxsearch is written in C++
• btdig.com search is based on lucene, which is written in java (slower than C/C++/Rust). clucene is a rewrite of lucene in C++, so its faster than lucene
  • index size: btdig.com currently has indexed about 60 million torrents. as a comparison, solidtorrents.to has indexed about 40 million torrents

possible solutions:

• write a custom search engine from scratch, based on postgreSQL full text search, a search query parser, and a SQL query generator.
  • the query parser could be written with the lemon parser generator, and the web interface could have a query parser in javascript (based on tree-sitter and/or lezer-parser), to provide live incremental parsing (and validation) of the user input
• use some search engine library
  • clucene - written in C++
  • xapian - written in C++
  • meilisearch - written in Rust - stores index on disk
  • typesense - written in C++ - stores index in RAM - compare: typesense / meilisearch
  • blevesearch - written in Go, default search engine of gitea
  • txtai - written in Python

my first impression is that search engine libraries like sphinx/lucene/xapian are overkill, because they focus on document search, with features like fuzzy search, query suggestions, spell checking. im not sure how important these features are for btdig

alternative solutions:

• solidtorrents.to has support for some complex queries, see Introducing SolidTorrents § Advanced Search
  • wrap the keywords with quote like "Keyword1 keyword2" for exact match.
  • and you can also do -keyword to remove the keyword from search result.
  • you can also put | in between keywords to specify or. like keyword1 | Keyword2 It will return results which matches either of the keywords.
  • You can also use * parameter to match anything. like some web series s1ep*
  • etc etc

query languages

• SQL: why not... provide read-only access to the database, guarded by some rate-limiting (query time, result size). but SQL may be too verbose
• lucene
  • title:"The Right Way" AND text:go
  • similar to: google search, google issues, github, ...
  • java is slow
• confluence: CQL = Confluence Query Language
  • (type=page and Space=DEV) OR (creator=bobsmith and type=blogpost)
  • similar to: SQL
• https://news.ycombinator.com/item?id=17890760 - some discussion on query languages
• https://github.com/topics/query-language
• graphQL: too complex?
  • https://github.com/obi1kenobi/trustfall

How is it installed? It doesn't give any instructions and when I run the exe. it gives an error and opens steam.

Hi, i m interested in this magnet:?xt=urn:btih:7ce7e4fa0b43d5930e21612ccdee50a4eb770f5f&dn=Porodicno+blago+ep+1-62+by+djukahr&tr=udp://tracker.openbittorrent.com:80&tr=udp://tracker.opentrackr.org:1337/announce your torrent but seems like is totally dead,how to get it?Or second one one this same topic -> https://btdig.com/7ce7e4fa0b43d5930e21612ccdee50a4eb770f5f/porodicno-blago
Vuk Dabizljevic
Thank you.

Hi, i m interested in this magnet:?xt=urn:btih:31e8d907b2c8d23987aa3b89cbe3f7dd12f7dc6c&dn=LMS+001-500&tr=udp://tracker.openbittorrent.com:80&tr=udp://tracker.opentrackr.org:1337/announce your torrent but seems like is totally dead,how to get it?Or second one one this same topic -> https://en.btdig.com/f188ea6235d3bf7b0878b59c2bbe120deed0372b/lunov
Goran Serbia
Thank you.

1.) Identify the original copyrighted works you believe has been infringed.

https://trafficsecrets.com/thebook-5
https://www.yourfirstfunnelchallenge.com/?_gl=1*19jz1op*_ga*MTc0NDQ5OTQ2OS4xNzA3NzcxMTU4*_ga_WQ0X0WDQ6W*MTcwNzc3MTE1Ny4xLjEuMTcwNzc3MTE2My41NC4wLjA.
https://dotcomsecrets.com/getdcsfree-1
https://www.nobsletter.com/special-offer-1

2.) Identify the material that you allege is infringing the copyrighted work listed in item #1, above.

https://en.btdig.com/811c29a3aef462ddfea8830e37e885eacd8777e0/russell-brunson-=-traffic-secrets
https://en.btdig.com/0f33ee9f06df63a812d9d0b806ac8f95111aa2fe/russell-brunson-=-traffic-secrets
https://en.btdig.com/8a437acd079dcdabb598a06c02f3468947c008a7/russell-brunson-=-traffic-secrets
https://en.btdig.com/ca5510de3fa4df269e1693ce8f3f9c0bdfe89dde/russell-brunson-=-traffic-secrets
https://en.btdig.com/87d286bd3d2033b0e8811ad5d09e9ca93aa72786/russell-brunson-=-traffic-secrets
https://en.btdig.com/c9c2710de9269cc4989c06cf8d4dac53ae32c09c/russell-brunson-=-traffic-secrets
https://en.btdig.com/78ce912666fb91beab52021a2ccfa757441a0045/russell-brunson-=-traffic-secrets
https://en.btdig.com/09b47ff4e113f9ca1ae619e9c2fb5c725aa1962b/russell-brunson-=-traffic-secrets
https://en.btdig.com/e2098b41aa7d849292feae3c915db740e6cb6e49/russell-brunson-=-traffic-secrets
https://en.btdig.com/af4502fbf0bb226cd22a589e7dc226fe94e6a1ae/russell-brunson-=-traffic-secrets
https://en.btdig.com/780a4c9aa570b8b56a08fb2c302c95e5055d28f9/russell-brunson-=-traffic-secrets
https://en.btdig.com/86cd6f65efbeeefc33d84fc0020f9b62a69830c7/russell-brunson-=-traffic-secrets
https://en.btdig.com/f2886ca891d6ea349a9c7c07d56b45edee095e98/russell-brunson-=-traffic-secrets
https://en.btdig.com/71c4b51ae74e0f308543ca3e55796cea96776eda/russell-brunson-=-traffic-secrets
https://en.btdig.com/c8206b529df136cce4b18f0f020e2233a1b6fd3d/russell-brunson-=-traffic-secrets
https://en.btdig.com/4d2fc7bc50f54557e1bdfd998dc08d3c76790d27/russell-brunson-=-traffic-secrets
https://en.btdig.com/7a426c3d2a5ca18f0ec11d6c114597c8895f7824/russell-brunson-=-traffic-secrets
https://en.btdig.com/637404695730c7644072bc9d335bbc936584d3ae/russell-brunson-=-traffic-secrets
https://en.btdig.com/8f6566f7b32990a78be9a936136e6c0279d04848/russell-brunson-=-traffic-secrets
https://en.btdig.com/d738739a02eb86c2ecc20f81690f5df1f895f833/russell-brunson-=-traffic-secrets
https://en.btdig.com/cbf91ac6e71cf61ff81e248cd1df17c93e12381b/russell-brunson-=-traffic-secrets
https://en.btdig.com/ce917eac8ea1a3d5e15fcb4abf73b56968cc78e0/russell-brunson-=-traffic-secrets
https://en.btdig.com/0c55286bb009f469ac79320885662c90f568d7cf/russell-brunson-=-traffic-secrets
https://en.btdig.com/35e56f41a1ead3d23b28cd64bcd26c5943bb7295/russell-brunson-=-traffic-secrets
https://en.btdig.com/18f9541781cb79a55cbb90b9e260578971b633f7/russell-brunson-=-traffic-secrets
https://en.btdig.com/1a305600fd23df68b15d661509f6de8d953da0b9/russell-brunson-=-traffic-secrets
https://en.btdig.com/98c2631d7dd07732f7087bbd5a93428bd076a78b/russell-brunson-=-traffic-secrets
https://en.btdig.com/9da78f95e3d83ded9a4720c3b90feff97ad4a6bd/russell-brunson-=-traffic-secrets
https://en.btdig.com/be60fa49e11a574a0f22d0bae91e10a4de8da104/russell-brunson-=-traffic-secrets

3.) Explain what the affected user would need to do in order to remedy the infringement.

They must remove all of the linked content conation Russell Brunson and his original works.

4.) Provide your contact information. Include your email address, name, telephone number and physical address.

Johnathan Katz [email protected] 3443 W Bavaria St., Eagle, ID 83616 208-323-9451

5.) A"I have a good faith belief that use of the copyrighted materials described above on the infringing web pages is not authorized by the copyright owner, or its agent, or the law. I have taken fair use into consideration."

6.) "I swear, under penalty of perjury, that the information in this notification is accurate and that I am the copyright owner, or am authorized to act on behalf of the owner, of an exclusive right that is allegedly infringed."

7.) Include your full legal name to act as an electronic signature.
Johnathan Katz

any chance of crawling v2 torrents and Vuze DHT? literally no DHT crawler search website indexes v2 torrents and Vuze DHT

magnetico

• https://github.com/boramalper/magnetico
• archived
• 3K github stars
• last commit 2022
• written in: Go

magnetico: Autonomous (self-hosted) BitTorrent DHT search engine suite.

magnetico is the first autonomous (self-hosted) BitTorrent DHT search engine suite that is designed for end-users. The suite consists of two packages:

• magneticod: Autonomous BitTorrent DHT crawler and metadata fetcher.
• magneticow: Lightweight web interface for magnetico.

Both programs, combined together, allows anyone with a decent Internet connection to access the vast amount of torrents waiting to be discovered within the BitTorrent DHT space, without relying on any central entity.

magnetico liberates BitTorrent from the yoke of centralised trackers & web-sites and makes it truly decentralised. Finally!

magnetico forks

• https://github.com/ireun/magnetico - 15 github stars, last commit 2022
• https://github.com/ez-me/magnetico - 10 github stars, last commit 2023
• https://github.com/tgragnato/magnetico - 10 github stars, last commit 2023

magnetico issues

https://github.com/boramalper/magnetico/issues

use postgreSQL

• boramalper/magnetico#264
• boramalper/magnetico#280
  • switch to postgres database increases performance by about 4 times on my server

DHT database dumps

• boramalper/magnetico#218
  • https://tnt.maiti.info/dhtd/
  • https://antipoul.fr/dhtd/
  • https://www.kescher.at/magneticod-db - last release: 2021
  • https://framagit.org/Glandos/magnetico_merge - merge sqlite databases

DHT database sharing

• boramalper/magnetico#187

distributed crawling

• boramalper/magnetico#155
• boramalper/magnetico#106
• boramalper/magnetico#23

shiyanhui/dht

• https://github.com/shiyanhui/dht
• written in: Go
• 3K github stars
• last commit 2020

bitmagnet

• https://github.com/bitmagnet-io/bitmagnet
• written in: Go
• 1K github stars
• last commit 2023

dhtbay

• https://github.com/FlyersWeb/dhtbay
• written in: JavaScript
• 100 github stars
• last commit 2018

dhtsearch

• https://github.com/felix/dhtsearch
• written in: Go
• 50 github stars
• last commit 2020

hamster

• https://github.com/picotorrent/hamster
• written in: Go
• 5 github stars
• last commit 2022

Site won't even load and stuck on a loading spinner page. This extended downtime is not common imo.

When i try to download torrent, there's empty file list

Either something got messed up in my browser settings and it was always there, but I never noticed it or something happened and there's now a sketchy redirect whenever I do a search. I only noticed it this afternoon.

Was it always there and I messed up my browser settings that's allowing it through?

would u consider making a stremio addon for btdig.com torrent search? would be nice. With cached services. Thanks in advanced.