Reputation: 456
I am trying to create a torrent search engine. yet I found a way to collect torrent infohashes from DHT
network. My question is how can I extract .torrent
files from them?
I have a MongoDB
collection contains 1M-1.5M(Growing Very Quickly)
torrent info hash.
Upvotes: 3
Views: 4114
Reputation: 256591
Given just an infohash, e.g.:
463e408429535139a0bbb5dd676db10d5963bf05
you can use:
BEP: 9 - Extension for Peers to Send Metadata Files
The purpose of this extension is to allow clients to join a swarm and complete a download without the need of downloading a .torrent file first. This extension instead allows clients to download the metadata from peers. It makes it possible to support magnet links, a link on a web page only containing enough information to join the swarm (the info hash).
You use DHT to find the distributed trackers.
Then you use the distributed trackers to find the peers who have the torrent.
Then you can download the torrent metadata from a peer.
You send the bencoded metadata request to the peer:
{
"msg_type": 0, ; 0==>request
"piece": 0
}
Upvotes: 2
Reputation: 456
I solved the problem myself. I used webtorrent npm package and also created an algorithm to loop through the whole database and added the magnet link in the download task. once the client gets metadata, I just saved it to the torrent file and canceled the download.
Well, The code is not yet fully production ready. I'll Post The Code Snippet here afterward. Thank You!
UPDATE: I am using this class to download Torrent file from magnet
const Discovery = require('torrent-discovery');
const Protocol = require('bittorrent-protocol');
const ut_metadata = require('ut_metadata');
const addrToIPPort = require('addr-to-ip-port');
const net = require('net');
class TorrentDownloader {
constructor(port, trackers, timeout) {
this.SELF_HASH = '4290a5ff50130a90f1de64b1d9cc7822799affd5';
this.port = port | 6881;
this.trackers = trackers;
this.timeout = timeout | 80000;
}
downloadTorrent(infoHash) {
let self = this;
return new Promise((resolve, reject) => {
let dis = new Discovery({infoHash: infoHash, peerId: this.SELF_HASH, port: this.port, dht: true, announce: this.trackers})
.on('peer', function (peer) {
const peerAddress = {address: addrToIPPort(peer)[0], port: addrToIPPort(peer)[1]};
// console.log(`download metadata from peer ${peerAddress.address}:${peerAddress.port}`);
self.getMetadata(peerAddress, infoHash, resolve);
});
setTimeout(() => {
dis.destroy();
reject(new Error("Torrent Timeout"))
}, this.timeout)
})
}
getMetadata(peerAddress, infoHash, resolve) {
const socket = new net.Socket();
socket.setTimeout(this.timeout);
socket.connect(peerAddress.port, peerAddress.address, () => {
const wire = new Protocol();
socket.pipe(wire).pipe(socket);
wire.use(ut_metadata());
wire.handshake(infoHash, this.SELF_HASH, {dht: true});
wire.on('handshake', function (infoHash, peerId) {
wire.ut_metadata.fetch();
});
wire.ut_metadata.on('metadata', function (rawMetadata) {
resolve(rawMetadata);
wire.destroy();
socket.destroy()
})
});
socket.on('error', err => {
socket.destroy();
});
}
}
module.exports = TorrentDownloader;
Upvotes: 1