Author: Arvid Norberg, arvid@libtorrent.org
Version: 1.2.3

home

DHT

[report issue]

dht_state

Declared in "libtorrent/kademlia/dht_state.hpp"

This structure helps to store and load the state of the dht_tracker. At this moment the library is only a dual stack implementation of the DHT. See BEP 32

struct dht_state
{
   void clear ();

   node_ids_t nids;
   std::vector<udp::endpoint> nodes;
   std::vector<udp::endpoint> nodes6;
};
[report issue]
nodes
the bootstrap nodes saved from the buckets node
[report issue]
nodes6
the bootstrap nodes saved from the IPv6 buckets node
[report issue]

dht_storage_counters

Declared in "libtorrent/kademlia/dht_storage.hpp"

This structure hold the relevant counters for the storage

struct dht_storage_counters
{
   void reset ();

   std::int32_t torrents = 0;
   std::int32_t peers = 0;
   std::int32_t immutable_data = 0;
   std::int32_t mutable_data = 0;
};
[report issue]

reset()

void reset ();

This member function set the counters to zero.

[report issue]

dht_storage_interface

Declared in "libtorrent/kademlia/dht_storage.hpp"

The DHT storage interface is a pure virtual class that can be implemented to customize how the data for the DHT is stored.

The default storage implementation uses three maps in RAM to save the peers, mutable and immutable items and it's designed to provide a fast and fully compliant behavior of the BEPs.

libtorrent comes with one built-in storage implementation: dht_default_storage (private non-accessible class). Its constructor function is called dht_default_storage_constructor(). You should know that if this storage becomes full of DHT items, the current implementation could degrade in performance.

struct dht_storage_interface
{
   virtual void update_node_ids (std::vector<node_id> const& ids) = 0;
   virtual bool get_peers (sha1_hash const& info_hash
      , bool noseed, bool scrape, address const& requester
      , entry& peers) const = 0;
   virtual void announce_peer (sha1_hash const& info_hash
      , tcp::endpoint const& endp
      , string_view name, bool seed) = 0;
   virtual bool get_immutable_item (sha1_hash const& target
      , entry& item) const = 0;
   virtual void put_immutable_item (sha1_hash const& target
      , span<char const> buf
      , address const& addr) = 0;
   virtual bool get_mutable_item_seq (sha1_hash const& target
      , sequence_number& seq) const = 0;
   virtual bool get_mutable_item (sha1_hash const& target
      , sequence_number seq, bool force_fill
      , entry& item) const = 0;
   virtual void put_mutable_item (sha1_hash const& target
      , span<char const> buf
      , signature const& sig
      , sequence_number seq
      , public_key const& pk
      , span<char const> salt
      , address const& addr) = 0;
   virtual int get_infohashes_sample (entry& item) = 0;
   virtual void tick () = 0;
   virtual dht_storage_counters counters () const = 0;
};
[report issue]

update_node_ids()

virtual void update_node_ids (std::vector<node_id> const& ids) = 0;

This member function notifies the list of all node's ids of each DHT running inside libtorrent. It's advisable that the concrete implementation keeps a copy of this list for an eventual prioritization when deleting an element to make room for a new one.

[report issue]

get_peers()

virtual bool get_peers (sha1_hash const& info_hash
      , bool noseed, bool scrape, address const& requester
      , entry& peers) const = 0;

This function retrieve the peers tracked by the DHT corresponding to the given info_hash. You can specify if you want only seeds and/or you are scraping the data.

For future implementers: If the torrent tracked contains a name, such a name must be stored as a string in peers["n"]

If the scrape parameter is true, you should fill these keys:

peers["BFpe"]
with the standard bit representation of a 256 bloom filter containing the downloaders
peers["BFsd"]
with the standard bit representation of a 256 bloom filter containing the seeders

If the scrape parameter is false, you should fill the key peers["values"] with a list containing a subset of peers tracked by the given info_hash. Such a list should consider the value of dht_settings::max_peers_reply. If noseed is true only peers marked as no seed should be included.

returns true if the maximum number of peers are stored for this info_hash.

[report issue]

announce_peer()

virtual void announce_peer (sha1_hash const& info_hash
      , tcp::endpoint const& endp
      , string_view name, bool seed) = 0;

This function is named announce_peer for consistency with the upper layers, but has nothing to do with networking. Its only responsibility is store the peer in such a way that it's returned in the entry with the lookup_peers.

The name parameter is the name of the torrent if provided in the announce_peer DHT message. The length of this value should have a maximum length in the final storage. The default implementation truncate the value for a maximum of 50 characters.

[report issue]

get_immutable_item()

virtual bool get_immutable_item (sha1_hash const& target
      , entry& item) const = 0;

This function retrieves the immutable item given its target hash.

For future implementers: The value should be returned as an entry in the key item["v"].

returns true if the item is found and the data is returned inside the (entry) out parameter item.

[report issue]

put_immutable_item()

virtual void put_immutable_item (sha1_hash const& target
      , span<char const> buf
      , address const& addr) = 0;

Store the item's data. This layer is only for storage. The authentication of the item is performed by the upper layer.

For implementers: This data can be stored only if the target is not already present. The implementation should consider the value of dht_settings::max_dht_items.

[report issue]

get_mutable_item_seq()

virtual bool get_mutable_item_seq (sha1_hash const& target
      , sequence_number& seq) const = 0;

This function retrieves the sequence number of a mutable item.

returns true if the item is found and the data is returned inside the out parameter seq.

[report issue]

get_mutable_item()

virtual bool get_mutable_item (sha1_hash const& target
      , sequence_number seq, bool force_fill
      , entry& item) const = 0;

This function retrieves the mutable stored in the DHT.

For implementers: The item sequence should be stored in the key item["seq"]. if force_fill is true or (0 <= seq and seq < item["seq"]) the following keys should be filled item["v"] - with the value no encoded. item["sig"] - with a string representation of the signature. item["k"] - with a string representation of the public key.

returns true if the item is found and the data is returned inside the (entry) out parameter item.

[report issue]

put_mutable_item()

virtual void put_mutable_item (sha1_hash const& target
      , span<char const> buf
      , signature const& sig
      , sequence_number seq
      , public_key const& pk
      , span<char const> salt
      , address const& addr) = 0;

Store the item's data. This layer is only for storage. The authentication of the item is performed by the upper layer.

For implementers: The sequence number should be checked if the item is already present. The implementation should consider the value of dht_settings::max_dht_items.

[report issue]

get_infohashes_sample()

virtual int get_infohashes_sample (entry& item) = 0;

This function retrieves a sample info-hashes

For implementers: The info-hashes should be stored in ["samples"] (N x 20 bytes). the following keys should be filled item["interval"] - the subset refresh interval in seconds. item["num"] - number of info-hashes in storage.

Internally, this function is allowed to lazily evaluate, cache and modify the actual sample to put in item

returns the number of info-hashes in the sample.

[report issue]

tick()

virtual void tick () = 0;

This function is called periodically (non-constant frequency).

For implementers: Use this functions for expire peers or items or any other storage cleanup.

[report issue]

counters()

virtual dht_storage_counters counters () const = 0;

return stats counters for the store

[report issue]

dht_settings

Declared in "libtorrent/kademlia/dht_settings.hpp"

structure used to hold configuration options for the DHT

The dht_settings struct used to contain a service_port member to control which port the DHT would listen on and send messages from. This field is deprecated and ignored. libtorrent always tries to open the UDP socket on the same port as the TCP socket.

struct dht_settings
{
   int max_peers_reply = 100;
   int search_branching = 5;
   int max_fail_count = 20;
   int max_torrents = 2000;
   int max_dht_items = 700;
   int max_peers = 500;
   int max_torrent_search_reply = 20;
   bool restrict_routing_ips = true;
   bool restrict_search_ips = true;
   bool extended_routing_table = true;
   bool aggressive_lookups = true;
   bool privacy_lookups = false;
   bool enforce_node_id = false;
   bool ignore_dark_internet = true;
   int block_timeout = 5 * 60;
   int block_ratelimit = 5;
   bool read_only = false;
   int item_lifetime = 0;
   int upload_rate_limit = 8000;
   int sample_infohashes_interval = 21600;
   int max_infohashes_sample_count = 20;
};
[report issue]
max_peers_reply
the maximum number of peers to send in a reply to get_peers
[report issue]
search_branching
the number of concurrent search request the node will send when announcing and refreshing the routing table. This parameter is called alpha in the kademlia paper
[report issue]
max_fail_count
the maximum number of failed tries to contact a node before it is removed from the routing table. If there are known working nodes that are ready to replace a failing node, it will be replaced immediately, this limit is only used to clear out nodes that don't have any node that can replace them.
[report issue]
max_torrents
the total number of torrents to track from the DHT. This is simply an upper limit to make sure malicious DHT nodes cannot make us allocate an unbounded amount of memory.
[report issue]
max_dht_items
max number of items the DHT will store
[report issue]
max_peers
the max number of peers to store per torrent (for the DHT)
[report issue]
max_torrent_search_reply
the max number of torrents to return in a torrent search query to the DHT
[report issue]
restrict_routing_ips

determines if the routing table entries should restrict entries to one per IP. This defaults to true, which helps mitigate some attacks on the DHT. It prevents adding multiple nodes with IPs with a very close CIDR distance.

when set, nodes whose IP address that's in the same /24 (or /64 for IPv6) range in the same routing table bucket. This is an attempt to mitigate node ID spoofing attacks also restrict any IP to only have a single entry in the whole routing table

[report issue]
restrict_search_ips
determines if DHT searches should prevent adding nodes with IPs with very close CIDR distance. This also defaults to true and helps mitigate certain attacks on the DHT.
[report issue]
extended_routing_table
makes the first buckets in the DHT routing table fit 128, 64, 32 and 16 nodes respectively, as opposed to the standard size of 8. All other buckets have size 8 still.
[report issue]
aggressive_lookups
slightly changes the lookup behavior in terms of how many outstanding requests we keep. Instead of having branch factor be a hard limit, we always keep branch factor outstanding requests to the closest nodes. i.e. every time we get results back with closer nodes, we query them right away. It lowers the lookup times at the cost of more outstanding queries.
[report issue]
privacy_lookups
when set, perform lookups in a way that is slightly more expensive, but which minimizes the amount of information leaked about you.
[report issue]
enforce_node_id
when set, node's whose IDs that are not correctly generated based on its external IP are ignored. When a query arrives from such node, an error message is returned with a message saying "invalid node ID".
[report issue]
ignore_dark_internet
ignore DHT messages from parts of the internet we wouldn't expect to see any traffic from
[report issue]
block_timeout
the number of seconds a DHT node is banned if it exceeds the rate limit. The rate limit is averaged over 10 seconds to allow for bursts above the limit.
[report issue]
block_ratelimit
the max number of packets per second a DHT node is allowed to send without getting banned.
[report issue]
read_only
when set, the other nodes won't keep this node in their routing tables, it's meant for low-power and/or ephemeral devices that cannot support the DHT, it is also useful for mobile devices which are sensitive to network traffic and battery life. this node no longer responds to 'query' messages, and will place a 'ro' key (value = 1) in the top-level message dictionary of outgoing query messages.
[report issue]
item_lifetime
the number of seconds a immutable/mutable item will be expired. default is 0, means never expires.
[report issue]
upload_rate_limit
the number of bytes per second (on average) the DHT is allowed to send. If the incoming requests causes to many bytes to be sent in responses, incoming requests will be dropped until the quota has been replenished.
[report issue]
sample_infohashes_interval
the info-hashes sample recomputation interval (in seconds). The node will precompute a subset of the tracked info-hashes and return that instead of calculating it upon each request. The permissible range is between 0 and 21600 seconds (inclusive).
[report issue]
max_infohashes_sample_count
the maximum number of elements in the sampled subset of info-hashes. If this number is too big, expect the DHT storage implementations to clamp it in order to allow UDP packets go through
[report issue]

dht_default_storage_constructor()

Declared in "libtorrent/kademlia/dht_storage.hpp"

std::unique_ptr<dht_storage_interface> dht_default_storage_constructor (
   dht_settings const& settings);

constructor for the default DHT storage. The DHT storage is responsible for maintaining peers and mutable and immutable items announced and stored/put to the DHT node.

[report issue]

sign_mutable_item()

Declared in "libtorrent/kademlia/item.hpp"

signature sign_mutable_item (
   span<char const> v
   , span<char const> salt
   , sequence_number seq
   , public_key const& pk
   , secret_key const& sk);

given a byte range v and an optional byte range salt, a sequence number, public key pk (must be 32 bytes) and a secret key sk (must be 64 bytes), this function produces a signature which is written into a 64 byte buffer pointed to by sig. The caller is responsible for allocating the destination buffer that's passed in as the sig argument. Typically it would be allocated on the stack.

[report issue]

ed25519_create_seed()

Declared in "libtorrent/kademlia/ed25519.hpp"

std::array<char, 32> ed25519_create_seed ();

See documentation of internal random_bytes

[report issue]

ed25519_create_keypair()

Declared in "libtorrent/kademlia/ed25519.hpp"

std::tuple<public_key, secret_key> ed25519_create_keypair (
   std::array<char, 32> const& seed);

Creates a new key pair from the given seed.

It's important to clarify that the seed completely determines the key pair. Then it's enough to save the seed and the public key as the key-pair in a buffer of 64 bytes. The standard is (32 bytes seed, 32 bytes public key).

This function does work with a given seed, giving you a pair of (64 bytes private key, 32 bytes public key). It's a trade-off between space and CPU, saving in one format or another.

The smaller format is not weaker by any means, in fact, it is only the seed (32 bytes) that determines the point in the curve.

[report issue]

ed25519_sign()

Declared in "libtorrent/kademlia/ed25519.hpp"

signature ed25519_sign (span<char const> msg
   , public_key const& pk, secret_key const& sk);

Creates a signature of the given message with the given key pair.

[report issue]

ed25519_verify()

Declared in "libtorrent/kademlia/ed25519.hpp"

bool ed25519_verify (signature const& sig
   , span<char const> msg, public_key const& pk);

Verifies the signature on the given message using pk

[report issue]

ed25519_add_scalar()

Declared in "libtorrent/kademlia/ed25519.hpp"

public_key ed25519_add_scalar (public_key const& pk
   , std::array<char, 32> const& scalar);
secret_key ed25519_add_scalar (secret_key const& sk
   , std::array<char, 32> const& scalar);

Adds a scalar to the given key pair where scalar is a 32 byte buffer (possibly generated with ed25519_create_seed), generating a new key pair.

You can calculate the public key sum without knowing the private key and vice versa by passing in null for the key you don't know. This is useful when a third party (an authoritative server for example) needs to enforce randomness on a key pair while only knowing the public key of the other side.

Warning: the last bit of the scalar is ignored - if comparing scalars make sure to clear it with scalar[31] &= 127.

see http://crypto.stackexchange.com/a/6215/4697 see test_ed25519 for a practical example

[report issue]

ed25519_key_exchange()

Declared in "libtorrent/kademlia/ed25519.hpp"

std::array<char, 32> ed25519_key_exchange (
   public_key const& pk, secret_key const& sk);

Performs a key exchange on the given public key and private key, producing a shared secret. It is recommended to hash the shared secret before using it.

This is useful when two parties want to share a secret but both only knows their respective public keys. see test_ed25519 for a practical example

[report issue]

announce_flags_t

Declared in "libtorrent/kademlia/announce_flags.hpp"

seed
announce to DHT as a seed
implied_port
announce to DHT with the implied-port flag set. This tells the network to use your source UDP port as your listen port, rather than the one specified in the message. This may improve the chances of traversing NATs when using uTP.
ssl_torrent
Specify the port number for the SSL listen socket in the DHT announce.