Identification
BitTorrent is a peer-to-peer file sharing protocol designed by bram cohen. Visit His Pages At
Http://www.bittorrent.com. Bittorrent is Designed to Facilitate File Transfers Among Multiple Peers Across Unreliable Networks.
Purpose
The purpose of this specification is to docutorrent protocol specification in detail. BRAM's Protocol Specification Page
http://www.bittorrent.com/protocol.html outlines the protocol in somewhat general terms, and lacks behaviorial detail in some areas. The hope is that this document will become a formal specification, written in clear, unambiguous terms, which can BE Used as a basis for discussion and ustementation in the ful ful.
This document is intended to be maintained and used by the BitTorrent development community. Everyone is invited to contribute to this document, with the understanding that the content here is intended to represent the current protocol, which is already deployed in a number of client implementations.
This is not the place to suggest feature requests. For That, please go to the mailing list.
Scope
This document applies to the first version (ie version 1.0) of the BitTorrent protocol specification. Currently, this applies to the torrent file structure, peer wire protocol, and the Tracker HTTP / HTTPS protocol specifications. As newer revisions of each protocol are defined, The Should Be Specified On Their Own Separate Pages, Not here.
Related Documents
http://www.bittorrent.com/protocol.html - The official protocol specification.BitTorrentWishList - A wish list for developers and end users alike.BitTorrentTrackerExtensions - Describes the various extensions of the Tracker protocol that are in use.
Conventions
In this document, a number of conventions are used in an attempt to present information in a concise and unambiguous fashion · peer v / s client:.. In this document, a peer is any BitTorrent client participating in a download The client is also a PEER, HOWEVER IS RUNNING ON The Local Machine. Reader of this Specification May Choose To Think of Themsels as The Client Which Connects To Numerous Peers.
· Piece v / s block: In this document, a piece refers to a portion of the downloaded data that is described in the metainfo file, which can be verified by a SHA1 hash A block is a portion of data that a client may request. From a peer. two or more blocks make up a whole piece, Which may the be verified.
· Defacto Standard: Large Blocks of Text In Italics INDICATES A PractICE SO Common in Various Client Implementations of Bittorrent That IT ITRENTOTARD.
In order help others to find recent changes that have been made to this document, please fill out the change log (last section). This should contain a brief (ie one-line) entry for each major change that you've made to the Document.
Bencoding
Bencoding is a Way to Specify and Organize Data ITA TERSE FORMAT. IT Supports The Following Types: Byte Strings, INTEGERS, LISTS, AND DICTIONARIES.
BYTE STRINGS
Byte strings are encoded as follows:
INTEGERS
Integers are encoded as follows: i
.
Lists May Contain Any Bencoded Type, Including Integers, Strings, Dictionaries, And Other Lists.
Example: L4: SPAM4: Eggse Repesents The list of two strings: ["spam", "eggs"]
Dictionaries
.
Note that the keys must be bencoded strings. The values may be any bencoded type, including integers, strings, lists, and other dictionaries. Keys must be strings and appear in sorted order (sorted as raw strings, not alphanumerics).
Example: D3: COW3: MOOED4: SPAM4: Eggse represents the dictionary {"cow" => "mo", "spam" => "eggs"} this first example seems to be wrong? (SHOULD BE D3: COW3: MOO4: Spam4: eggse) Example: D4: Spaml1: a1: Bee represents the dictionary {"spam" => ["a", "b"]}
Metainfo File Structure
All Data I MetaInfo File Is Bencoded. The Specification for Bencoding IS Defined Above.
The content of a metainfo file (the file ending in ".torrent") is a bencoded dictionary, containing the keys listed below All character string values are UTF-8 encoded Keys not marked 'optional' are required fields..:
INFO: A Dictionary That Describes The File (s) of the Torrent. There is the case of a 'Single-File' Torrent With No Directory Structure, AND One for the case of a 'multi-file 'Torrent, Which Can Contain Subdirectory Trees.for The Case of The Single-File Mode, The Info Dictionary Contains The Following Structure
OLENGTH: Length of the file in bytes (Integer)
OMD5SUM: (OPTIONAL) A 32-Character Hexadecimal String Corresponding to The MD5 Sum of The File. This is not used by bittorrent at all, but it is inclined by some of Some Program for Greater Compament.
(String)
Opiece Length: Number of Bytes in Each Piece (Integer)
Opieces: String Consisting of the Concatenation Of All 20-Byte Sha1 Hash Values, One Per Piece (byte String)
For the case of the multi-file mode, The Info Dictionary Contains The Following Structure, SFLOWING STRUCTURE
Ofiles: a list of Dictionaries, One for Each File. Each Dictionary in this List Contains The Following Keys:
§ Length: Length of the file in bytes (Integer)
§ MD5Sum: (OPTIONAL) A 32-Character Hexadecimal String Corresponding to the MD5 Sum of The File. This is not used by bittorrent at all, but it it is inclined by Some Program for Greater Compament.
§ path: a list containing one or more string elements that together represent the path and filename Each element in the list corresponds to either a directory name or (in the case of the final element) the filename For example, a the file ".. Dir1 / Dir2 / File.ext "Would Consist of Three String Elements:" DIR1 "," Dir2 ", and" file.ext ". this is encode as a bencoded list of strings such as L4: Dir14: Dir28: file.exte
oname: the name of the top-most directory in the structure - the directory which contains all of the files listed in the above files list (character string) opiece length: number of bytes in each piece (integer).
Opieces: String Consisting of the Concatenation Of All 20-Byte Sha1 Hash Values, One Per Piece (byte String)
· Announce: The Announce Url of The Tracker (String)
· Announce-list: (optional) this is an extention to the official specification, Which is also backwards compatible. This key is buy tplement list of backup trackrs. The full specification can be found
http://home.elp.rr.com/tur/multitracker-spec.txt
Creation date: (optional) The Creage Time of The Torrent, In Standard Unix Epoch Format (Integer Seconds Since 1-JAN-1970 00:00:00 UTC)
· Comment: (optional) Free-form textual comments of the author (string)
· Created by: (optional) Name and version of the program use to create the .torrent (string)
Notes
· The piece length specifies the nominal piece size, and is usually a power of 2. The piece size is typically chosen based on the total amount of file data in the torrent, constrained by the fact that piece sizes too large cause inefficiency, and too small a piece size will result in a large .torrent metadata file The conventional wisdom used to be to pick the smallest piece size that results in a .torrent file no greater than approx 50 -.. 75 kB (presumably to ease the load on the server hosting the torrent files). However, now that hosting storage and bandwidth are not tightly constrained, it is best to keep the piece size to 512KB or less, at least for torrents under 8-10GB or so, even if that results in a larger torrent file, in order to have a more efficient swarm for sharing files. The most common sizes are 256 kB, 512 kB, and 1 MB. Every piece is of equal length except for the final piece, which is irregular. The number of Pieces Is Thus determined by 'CEIL (Total Length / Piece size) '. For the purposes of piece boundaries in the multi-file case, consider the file data as one long continuous stream, composed of the concatenation of each file in the order listed in the files list. The number of pieces and their boundaries are then determined in the same manner as the case of a single file. Pieces may overlap file boundaries. · Each piece has a corresponding SHA1 hash of the data contained within that piece. These hashes are concatenated to form the pieces value in the Above Info Dictionary. Note That this is not a list but rate a single string. The length of the string must be a multiple of 20 bytes.
Tracker HTTP / HTTPS Protocol
The tracker is an HTTP / HTTPS service which responds to HTTP GET requests. The requests include metrics from clients that help the tracker keep overall statistics about the torrent. The response includes a peer list that helps the client participate in the torrent. The base URL consists of the "announce URL" as defined in the metadata (.torrent) file. The parameters are then added to this URL, using standard CGI methods (ie a '?' after the announce URL, followed by 'param = value' sequences Separated by '&') Note That All Binary Data IN THE URL (Particular INFO_HASH AND Peer_ID) Must Be Properly Escaped. This Means Any Byte Not in The Set 0-9, AZ, AZ, And $ -_. ! * ' (), Must Be encoded sales the "% nn" format, where nn is the hexadecimal value of the byte. (see rfc1738 for details).
The Parameters Used in The Client-> Tracker Get Request Area Follows:
· Info_hash:. 20-byte SHA1 hash of the value of the info key from the Metainfo file Note that the value will be a bencoded dictionary, given the definition of the info key above Note:. This string is always urlencoded, as opposed to Peer_ID, Which Needs to be unencoed.
· Peer_id: 20-byte string used as a unique ID for the client, generated by the client at startup This is allowed to be any value, and may be binary data There are currently no guidelines for generating this peer ID However,... one may rightly presume that it must at least be unique for your local machine, thus should probably incorporate things like process ID and perhaps a timestamp recorded at startup. See peer_id below for common client encodings of this field.
· Port: The port number that the client is listening on Ports reserved for BitTorrent are typically 6881-6889 Clients may choose to give up if it can not establish a port within this range · uploaded:... The total amount uploaded (since the client Sent the 'Started' Event To The Ten Base Ten Ascii. While Not Explicitly Stated In The Official Specification, The Concensus Is That Should Be The Total Number of Bytes Uploaded.
· Downloaded: The total amount downloaded (since the client sent the 'started' event to the tracker) in base ten ASCII While not explicitly stated in the official specification, the consensus is that this should be the total number of bytes downloaded..
· Left: The Number of Bytes this Client Still Has To Download, Encoded In Base Ten ASCII.
· Compact: Indicates the client accepts a compact response The peers list is replaced by a peers string with 6 bytes per peer The first four bytes are the host (in network byte order), the last two bytes are the port (again in.. Network Byte Order.
· Event: if specified, Must Be One of Started, Completed, Stopped, (or Empty Which Is The Same As Not Being Specified). IF NOT SPECIFIED, The Request Is One Performed At Regular Interval.
Ostarted: The First Request to the Tracker Must Include The Event Key with The Started Value.
Ostopped: Must Be Sent To The Tracker If The Client Is Shutting Down GraceFully.
ocompleted:. Must be sent to the tracker when the download completes However, must not be sent if the download was already 100% complete when the client started Presumably, this is to allow the tracker to increment the "completed downloads" metric based soley. On this event.
· Ip: Optional The true IP address of the client machine, in dotted quad format or rfc3513 defined hexed IPv6 address Notes:.. In general this parameter is not necessary as the address of the client can be determined from the IP address from which the HTTP request came. The parameter is only needed in the case where the IP address that the request came in on is not the IP address of the client. This happens if the client is communicating to the tracker through a proxy (or a transparent web proxy / cache.) It also is necessary when both the client and the tracker are on the same local side of a NAT gateway. The reason for this is that otherwise the tracker would give out the internal (RFC1918) address of the client, which is not routeable. Therefore the client must explicitly state its (external, routeable) IP address to be given out to external peers. Various trackers treat this parameter differently. Some only honor it only if the IP address that the request came in on is in RFC1918 spa CE. Others Honor IT UNCONDITIONALLY. IG: 2001: DB8: 1: 2 :: 100) It inde indicates only That Client CAN Communicate Via IPv6. · Numwant: optional. Number of Peers That The Clom The Tracted To Be Zero. if omitted, Typically Defaults To 50 Peers.
· Key: optional. An Additional Identification That Not Shared with any! It is intended to allow aclient to pro, their identity,.
· TrackerID: Optional. If A Previous Announce Contained A Tracker ID, IT Should Be Set Here.
The Tracker Responds with "Text / Plain" Document Consisting of a Bencoded Dictionary with The Following Keys:
· Failure reason: If present, then no other keys may be present The value is a human-readable error message as to why the request failed (string) · warning message:.. (New) Similar to failure reason, but the response still Gets processed normally. The Warning Message IS Shown Just Like An Error.
· Interval: Interval in Seconds That The Client Should Wait Between Sending Regular Requests To The Tracker (Mandatory)
· Minimum announce interval. If present..............
· Tracker ID: a String That The Client Should Send Back on Its Next Announcements. If Absent and a Previous Anounce Sent A Tracker ID, Do Not Discard The Old Value; Keep Using IT.
· Complete: Number of Peers with the entire file, I.E. SEEDERS (Integer)
· Incomplete: Number of Non-Seeder Peers, Aka "Leechers" (Integer)
· Peers: The value is a list of dictionaries, Each with the folload keys:
Opeer ID: Peer's self-selected ID, as described Above for the Tracker Request (String)
OIP: Peer'S IP Address (Either IPv6 or IPv4) or DNS Name (String)
Oport: Peer's Port Number (Integer)
As mentioned above, the list of peers is length 50 by default. If there are fewer peers in the torrent, then the list will be smaller. Otherwise, the tracker randomly selects peers to include in the response. The tracker may choose to implement a More Intelligent Mechanism for Peer Selection When Responding to a Request. for Instance, Reporting Seeds To Other Seeders Could Be Avoided.
Clients may send a request to the tracker more often than the specified interval, if an event occurs (ie stopped or completed) or if the client needs to learn about more peers. However, it is considered bad practice to "hammer" on a tracker TO GET MULTIPLE PEERS. IF A Client Wants A Large Peer List in The Response, The It Should Specify The Numwant Parameter.Implementer's NOTE: EVEN 30 Peers Is Plenty, The Official Client Version
3 in
fact only actively forms new connections if it has less than 30 peers and will refuse connections if it has 55. This value is important to performance. When a new piece has completed download, HAVE messages (see below) will need to be sent to most active peers. As a result the cost of broadcast traffic grows in direct proportion to the number of peers. Above 25, new peers are highly unlikely to increase download speed. UI designers are strongly advised to make this obscure and hard to change as it is Very Rare to Be Useful to do so.
Tracker 'Scrape' Convention
By convention most trackers support another form of request, which queries the state of a given torrent (or all torrents) that the tracker is managing. This is referred to as the "scrape page" because it automates the otherwise tedious process of "screen scraping "The Tracker's Stats Page.
The scrape URL is also a HTTP GET method, similar to the one described above However the base URL is different To derive the scrape URL use the following steps:... Begin with the announce URL Find the last '/' in it If. the text immediately following that '/' is not 'announce' it will be taken as a sign that that tracker does not support the scrape convention. If it does, substitute 'scrape' for 'announce' to find the scrape page. Examples: (Announce Url -> Scrape URL)
http://example.com/announce -> http://example.com/scrape
Http://example.com/x/announce -> http://example.com/x/scrape
http://example.com/announce.php -> http://example.com/scrape.php
http://example.com/a -> (Scrape Not Supported)
http://example.com/announce?x=244 -> http://example.com/scrape?x=244
http://example.com/announce?x=2/4 -> (Scrape Not Supported)
http://example.com/x4announce -> (Scrape Not Supported)
Note Especially That Entity UNE. This Standard is Documented by Bram in The Bittorrent Development List Archive:
http://groups.yahoo.com/group/bittorrent/message/3275
The scrape URL may be supplimented by the optional parameter info_hash, a 20-byte value as described above. This restricts the tracker's report to that particular torrent. Otherwise stats for all torrents that the tracker is managing are returned. Software authors are strongly encouraged to Use the info_hash parameter When At All Possible, To Reduce The Load and Bandwidth of The Tracker.
The response of this HTTP GET method is a "text / plain" document consisting of a bencoded dictionary, containing the following keys · files: a dictionary containing one key / value pair for each torrent for which there are stats If info_hash was supplied and. Was Valid, this Dictionary Will Contain A Single Key / Value. Each Key Consists of A 20-Byte Binary Info_hash Value. The Value of That Key Is Yet Another Nested Dictionary Containing The Following:
o Complete: Number of Peers with the entire file, I.E. SEEDERS (Integer)
o Downloaded: Total Number of Times The Tracker Has Registered A Completion ("Event = COMPLETE", I.E. aclient finished downloading the torrent
o INCOMPLETE: NUMBER OF NON-SEEDER Peers, Aka "Leechers" (Integer)
o Name: (optional) The Torrent's Internal Name, as specified by the "name" file in the info section of the .torrent file
Note That this response Has Three Levels of Dictionary Nesting. Here's An Example:
D5: filesd20: .................... D8: companyTei5e10: Downloadedi50E10: Incompletei10eeee
WHERE .................. is The 20 byte info_hash and there. 5 Seeders, 10 Leechers, And 50 Complete Downloads.
Peer Wire Protocol (TCP)
Overview
.
Note here that the original specification also used the term "piece" when describing the peer protocol, but as a different term than "piece" in the metainfo file. For that reason, the term "block" will be used in this specification to describe The Data That Is Exchanged Between Peers over The Wire.
A client must maintain state information for each connection that it has with a remote peer: · choked: Whether or not the remote peer has choked this client When a peer chokes the client, it is a notification that no requests will be answered until the. CLIENT IS UNCHOKED. The Client Should Not Attempt To Send Requests for Blocks, And It Should Consider All PENDING (Unanswered) Requests to Be Discarded by The Remote Peer.
INTERESTED: WHETER OR NOT The Remote Peer Is INTERESTEDIN Sometying This Client Has That That The Remote Peer Will Begin Requesting Blocks When the Client Unchokes Them.
Note that this also implies that the client will also need to keep track of whether or not is interested in the remote peer, and if it has the remote peer choked or unchoked So, the real list looks something like this it.:
· Am_choking: this client is choking the peer
· AM_INTERESTED: this client is interested in the peer
Peer_choking: Peer Is Choking this Client
Peer_INTERESTED: Peer IS INTERESTED IN THIS CLIENT
Client Connections Start Out as "Choked" and "not interested". In Other Words:
· Am_choking = 1
· AM_INTERESTED = 0
PEER_CHOKING = 1
PEER_INTERESTED = 0
A block is downloaded by the client when the client is interested in a peer, and that peer is not choking the client. A block is uploaded by a client when the client is not choking a peer, and that peer is interested in the client.
It is important for the client to keep its peers informed as to whether or not it is interested in them. This state information should be kept up-to-date with each peer even when the client is choked. This will allow peers to know if The Client Will Begin Downloadning When IT ITOKED (and Vice-Versa) .DATA TYPES
Unless Spectherwise, All Integers in The Peer Wire Protocol Are Encoded As Four ByTe Big-endian Values. This include The Length Prefix ON All Messages That Come After The Handshake.
Message flow
............... ..
Handshake
.............
· Handshake:
Opstrlen: String Length of
Opstr: String Identifier of the Protocol
oreserved:.. eight (8) reserved bytes All current implementations use all zeroes Each bit in these bytes can be used to change the behavior of the protocol An email from Bram suggests that trailing bits should be used first, so that leading bits may. Be used to change the meaning of trailing bits.
.
.
In Version 1.0 of the bittorrent protocol, pstrlen = 19, and pstr = "bittorrent protocol".
The initiator of a connection is expected to transmit their handshake immediately. The recipient may wait for the initiator's handshake, if it is capable of serving multiple torrents simultaneously (torrents are uniquely identified by their info_hash). However, the recipient must respond as soon as it sees the info_hash part of the handshake. The tracker's NAT-checking feature does not send the peer_id field of the handshake.If a client receives a handshake with an info_hash that it is not currently serving, then the client must drop the connection.
If the initiator of the connection receives a handshake in which the peer_id does not match the expected peer_id, then the initiator is expected to drop the connection. Note that the initiator presumably received the peer information from the tracker, which includes the peer_id that was registered ..............
Peer_id
There area.
Azureus-style uses the following encoding: '-', Two Characters for Client ID, Four Ascii Digits for Version Number, '-', Followed by Random Numbers.
For example: '-az2060-de
KNown Clients That Uses this Encoding Style Are:
· 'AZ' -
Azureus
· 'Bb' -
BitBuddy
· 'Ct' -
Ctorrent
· 'Mt' -
Moonlighttorrent
· 'Lt' -
Libtorrent
· 'Bx' - BitTorrent X
· 'TS' -
TorrentStorm
· 'TN' - TorrentDotNet
· 'Ss' - swarmscope
· 'XT' -
Xantorrent
· 'Bs' -
Btslave
· 'Zt' -
Ziptorrent
· 'AR' -
Arctic
· 'Sb' - swiftbit · 'mp' -
Moopolice
· 'Lt' -
Libtorrent
· 'Bc' -
Bitcomet
· 'QT' - Qt 4 Torrent Example
· 'UT' -
μTORRENT?
· 'Sz' -
Shareaza
· 'RT' -
Retriever
· 'Lp' -
LPHANT
Shadow's Style Uses The Following Encoding: One Ascii Alphaumeric for Client Identification, Three Ascii Digits for Version Number, '----', Followed by Random Numbers.
For example: 'S587 ----'...
KNown Clients That Uses this Encoding Style Are:
· 'S' -
Shadow's Client
· 'U' -
Upnp Nat Bit Torrent
· 'T' -
BitTornado
· 'A' -
ABC
· 'O' -
Osprey permaseed
BRAM's Client Now Uses this style ... 'M
3-4-2
- '.
BitComet does something different still. Its peer_id consists of four ASCII characters 'exbc', followed by two bytes x and y, followed by random characters. The version number is x in decimal before the decimal point and y as two decimal digits after the decimal Point. The Encoding for? BitComet Peer IDS Changed to Azureus-style as of? BitComet Version 0.59. Therefore x Will Always Be Zero.
XBT Client has its own style too. Its peer_id consists of the three uppercase characters 'XBT' followed by three ASCII digits representing the version number. If the client is a debug build, the seventh byte is the lowercase character 'd', otherwise it IS A '-'. FOLLOWING THATS A '-' TEN Random Digits, Uppercase and LowerCase Letters. EXAMPLE: 'XBT054D-' At The Beginning Would Indicate A Debug Build of Version
0.5.4
.
Opera 8 previews use the following peer_id scheme: The first two characters are 'OP' and the next four digits equal the build number All following characters are random lowercase hexdecimal digits.Many clients are using all random numbers or 12 zeroes followed by random numbers. (Like Older Versions of
BRAM's Client).
Messages
All of the remaining messages in the protocol take the form of
Keep-alive:
The keep-alive message is a message with zero bytes, specified with the length prefix set to zero. There is no message ID and no payload. Peers may close a connection if they receive no messages for a certain period of time, so a keep -Alive Message Can Be Sent to Maintain The Connection. a Keep-Alive Message Is Generally Sent Once Every Two Minutes.
Choke:
The Choke Message Is Fixed-Length And Has No Payload.
UNCHOKE:
The Unchoke Message IS Fixed-Length And Has No Payload.
Interested:
The INTERESTED Message IS Fixed-Length And Has No Payload.
NOT INTERESTED:
The Not Interedment Message IS Fixed-Length And Has No Payload.
Have:
................ ..
Implementer's Note: That is the strict definition, in reality some games may be played In particular because peers are extremely unlikely to download pieces that they already have, a peer may choose not to advertise having a piece to a peer that already has that piece. . At a minimum "HAVE supression" will result in a 50% reduction in the number of HAVE messages, this translates to around a 25-35% reduction in protocol overhead.A malicious peer might also choose to advertise having pieces that it knows the Peer Will Never Download. Due to this attempting to model peers sale this information is a bad idea
Bitfield:
The Bitfield Message May Only Be Sent Immediely After The Handshaking Sequence Is Completed, And Before Any Other Messages Are Sent. It is Optional, And NOTBECES.
The bitfield message is variable length, where X is the length of the bitfield. The payload is a bitfield representing the pieces that have been successfully downloaded. The high bit in the first byte corresponds to piece index 0. Bits that are cleared indicated a missing Piece, And Set Bits Indicate a Valid and Available Piece. Spare Bits At The End Are Set To Zero.
A Bitfield of The Wrong Length Is Considered An Error. Clients SHOULD DROP The Connection If The Receive Bitfields That Aren't of the the bitfield Has any of the spare bits set.
Request:
The Request Message IS Fixed Length, and is used to request a block. The payload Contains the Following Information
· INDEX: Integer specifying the zero-based Piece Index
· Begin: Integer specifying the zero-based byte offset within the basce · lonsth: integer specifying the requested length.
According to the official specification associated with mainline version 3, "All current implementations use 2 ^ 15 (32KB), and close connections which request an amount greater than 2 ^ 17 (128KB)." As of version 4 of the mainline, this behavior was changed to use 2 ^ 14 (16KB) blocks, and that size was enforced against peers that tried to request more. Note that block requests are smaller than pieces (> = 2 ^ 18 bytes), so multiple requests will be needed to download A WHOLE PIECE.
Due to the new version of the mainline placing the limit at 16KB, attempting to use 32KB blocks can best be compared to playing Russian Roulette with about 4 bullets, you'll likely run into trouble at this time. Smaller requests will result in higher overhead Mostly in Time and Space Due to the overhead of tracking a greater number of requests. as a resultimlementers shouth likely used 16kb as all clients will allow.
The choice of request block size limit enforcement is not nearly so clear cut. With mainline version 4 enforcing 16KB requests, most clients will use that size, there is one crucial group of clients that will not. Most older clients use 32KB requests and disallowing those will distinctly shrink the set of possible peers. At the same time 16KB is the semi-official (only semi because the official protocol document has not been updated) limit now, so enforcing that is not wrong. At the same time, allowing larger requests enlarges the set of possible peers, and except on very low bandwidth connections (<256kbps) multiple blocks will be downloaded in one choke-timeperiod, thus merely enforcing the old limit causes minimal performance degradation. Due to this factor, it is recommended that Only the Older 128kb limit be enforced.piece:
THE PIECE Message Is Variable Length, Where x is The length of the block. The paying information
· INDEX: Integer specifying the zero-based Piece Index
· Begin: integer specifying the zero-based byte offset within the basce
· Block: Block of Data, Which is a subsset of the point specified by index.
Cancel:
The cancel message is fixed length, and is used to cancel block requests. The payload is identical to that of the "request" message. It is typically used during "End Game" (see the Algorithms section below).
Port:
The port message is sent by newer versions of the Mainline that implements a DHT tracker. The listen port is the port this peer's DHT node is listening on. This peer should be inserted in the local routing table (if DHT tracker is supported) .Algorithms
Queuing
In general peers are advised to keep a few unfullfilled requests on each connection. This is done because otherwise a full round trip is required from the download of one block to begining the download of a new block (round trip between PIECE message and next REQUEST message ). On links with high BDP (bandwidth-delay-product, high latency or high bandwidth), this can result in a substantial performance loss. At the time of this writing the official client defaults to 5 outstanding requests.
Implementer's note: This the most crucial performance item A static queue of 5 requests is reasonable for 32KB blocks on a 5mbps link with 50ms latency Links with greater bandwidth are becoming very common so UI designers are urged to make this readily available for changing... NOTABLY CABLE MODEMS WERE KNOWN FOR Traffic Policing and increasing this might of aleviated some of the problem
Auto-tuning: A reasonable method to tune this variable is to continuously measure bandwidth from individual links Try increasing the queue depth, if the bandwidth from that peer increases, likely the queuing is insufficient By the same token if decreasing queue depth does not.. Decrease Bandwidth (AND DOES DECREASE LATENCY) THEN LIKELY Queue Depth IS TOO GREAT.
Super Seeding
(This was not part of the original specification)
The super-seed feature in S-5.5 and on is a new seeding algorithm designed to help a torrent initiator with limited bandwidth "pump up" a large torrent, reducing the amount of data it needs to upload in order to spawn new seeds in the torrent.When a seeding client enters "super-seed mode", it will not act as a standard seed, but masquerades as a normal client with no data As clients connect, it will then inform them that it received a piece -. a Inly That Piece.
When the client has finished downloading the piece, the seed will not inform it of any other pieces until it has seen the piece it had sent previously present on at least one other client. Until then, the client will not have access to any of the Other Pieces of The Seed, And Therefore Will Not Waste The Seed's Bandwidth.
This method has resulted in much higher seeding efficiencies, by both inducing peers into taking only the rarest data, reducing the amount of redundant data sent, and limiting the amount of data sent to peers which do not contribute to the swarm. Prior to this, a seed might have to upload 150% to 200% of the total size of a torrent before other clients became seeds. However, a large torrent seeded with a single client running in super-seed mode was able to do so after only uploading 105% Of The Data. This is 150-200% More Efficient Than Whening a standard seed.
Super-seed mode is NOT recommended for general use. While it does assist in the wider distribution of rare data, because it limits the selection of pieces a client can downlad, it also limits the ability of those clients to download data for pieces they have Already Partially Retrieved. there irefore, super-seed mode is only Recommended for Initial Seeding Servers.Why Not Rename It to eg "Initial SEEDING MODE" or "Releaser Mode" Then?
Piece Downloading Strategy
Clients May Choose to Download Pieces in Random ORDER.
A better strategy is to download pieces in rarest first order. The client can determine this by keeping the initial bitfield from each peer, and updating it with every have message. Then, the client can download the pieces that appear least frequently in these peer bitfields .
End Game
When a download is almost complete, there's a tendency for the last few blocks to trickle in slowly. To speed this up, the client sends requests for all of its missing blocks to all of its peers. To keep this from becoming horribly inefficient, the Client Also Sends a Cancel to Everyone else Every Time A Block Arrives.
................ ..
When to enter end game mode is an area of discussion. Some clients enter end game when all pieces have been requested. Others wait until the number of blocks left is lower than the number of blocks in transit, and no more than 20. There seems to be agreement that it's a good idea to keep the number of pending blocks low (1 or 2 blocks) to minimize the overhead, and if you randomize the blocks requested, there's a lower chance of downloading duplicates. More on the protocol overhead can be Found here: http://hal.inria.fr/inria-00000156/EN
Choking and optimistic unchoking
Choking is done for several reasons. TCP congestion control behaves very poorly when sending over many connections at once. Also, choking lets each peer use a tit-for-tat-ish algorithm to ensure that they get a consistent download rate.
The choking algorithm described below is the currently deployed one. It is very important that all new algorithms work well both in a network consisting entirely of themselves and in a network consisting mostly of this one.
There are several criteria a good choking algorithm should meet. It should cap the number of simultaneous uploads for good TCP performance. It should avoid choking and unchoking quickly, known as 'fibrillation'. It should reciprocate to peers who let it download. Finally, IT SHOULD TRY OUSED CONNECTIONS ONCE IN A While To Find Out Iw Might Be Better Than THE CURRENTLY Used Ones, known as Optimistic UNCHOKING.
THE CURRENTLY Deployed Choking Algorithm Avoids Fibrillation by Only Changing Choked Peers Once Every Ten Seconds.
Reciprocation and number of uploads capping is managed by unchoking the four peers which have the best upload rate and are interested. This maximizes the client's download rate. These four peers are referred to as downloaders, because they are interested in downloading from the client.Peers which have a better upload rate (as compared to the downloaders) but are not interested get unchoked. If they become interested, the downloader with the worst upload rate gets choked. If a client has a complete file, it uses its upload rate rather Than ITS Download Rate To Decide Which Peers to Unchoke.
For optimistic unchoking, at any one time there is a single peer which is unchoked regardless of it's upload rate (if interested, it counts as one of the four allowed downloaders). Which peer is optimistically unchoked rotates every 30 seconds. Newly connected peers are Three Times as likely to start as anywhere else in the rotation. this gives the a decent chance of getting a completion one to upload.
Anti-Snubbing (Extension Not in The OffiIffial Protocol)
Occasionally a BitTorrent peer will be choked by all peers which it was formerly downloading from. In such cases it will usually continue to get poor download rates until the optimistic unchoke finds better peers. To mitigate this problem, when over a minute goes by without getting a single piece from a particular peer, BitTorrent assumes it is "snubbed" by that peer and does not upload to it except as an optimistic unchoke. This frequently results in more than one concurrent optimistic unchoke, (an exception to the exactly one optimistic UNCHOKE RULE MENTIONED ABOVE), Which Causes Download Rates To Recover Much More Quickly when They Falter.change Log
PLEASE PRESERVE The LOST MONTH'S WORTH OF CHANGE LOST MONTH'S WORTH OF CHANGE LOST MONTH'S WORTH OF CHANGE LOST MONTH'S WORTH OF CHANGE LOST MONTH'S WORTH OF CHANGE LOGE.
Juanjo
2006-03-10
Added LPHANT peer ID
Elliottmitchell
2006-03-07
Added Mention of Another Parameter to the Baseline for Queuing.
· Another Link Adjustment.
Added Exposition on Request Block size.
· Sample Link Changes, The Domain "Example.com" is expected for example, as such That Should be used instead of spam.com.
Added Qualification to Recommended Number of Peers.
Elliottmitchell
2006-03-01
(minor)
Link Changes, The Official Bittor Pages Are No Longer on Bitconjuit.org, But BitTorent.com.
Removed Unneeded Line Breaks from Paragraph.
· TYPO FIXES in Changelog (Yeah, I Suppose Do Have A Bit of Vanity)
Elliottmitchell
2006-03-01
Restored The Section On Queueing, AS It Is A HIGHLY CRUCIAL Performance Item. Feel Free To Rewrite Me if you design, uau. The developer's list,
http://lists.ibiblio.org/mailman/listinfo/bittorrent is a better place for debates. · Trimmed changelog entries older than one year, the above specifies one month, but this is changing slowly so more history seems pertinent.
HAYLEGEND -
2006-01-08
· Added Retriever's Peer ID.
UAU -
2005-12-13
· Fixed Sizes in Request Message Description Again. They Had Been Changed to IncorRect Values by Elliottmitchell.
Removed "Queuing" Section That Had Been Added by Elliottmitchell. It had so many errors and inaccuracies That Did More Harm Than Good As It Was, And i Didn't Feel Like Rewriting IT.
Daniel-gl at gmx.net -
2005-07-29
• Completed? BitComet Peer id According to
http://wiki.bitcomet.com/help/inside_bitcomet
· ADDED μTorrent (Correct Me if I'm Wrong) and Shareaza Peer ID
Bitbrat Andreas © Hanssen Dot Name -
2005-10-01
· Fixed QT Peer ID Information
· Added Something About the endgame mode
Bitbrat Andreas © Hanssen Dot Name -
2005-09-16
Added QT Peer ID Information
? Arvidnorberg c99ang at cs Dot Umu Dot SE -
2005-09-16
· New Mainline Disconnects on Requests> 16 KIB
? Arvidnorberg c99ang at cs Dot Umu Dot SE -
2005-09-15
Added Port Message Used by DHT Enabled Clients
DWKNIGHT AT Depthstrike Dot COM -
2005-08-31
-? DReadwingknight
· Updated? BitComet Peer ID Encoding Information.
Daniel-gl at gmx.net -
2005-07-29
· Added Libtorrent and Opera Peer_ID
Elliottmitchell -
2005-07-20
· Added the "algorithm", "queueing"
Elliottmitchell -
2005-07-20
· Added Implementation Notes on Have and Peer Count
? Mayhem -
2005-07-08
· Added Description of XBT Peer IDS
OspreyMainTainer -
2005-07-01
· Added "o" peer id code for osprey permaseed.
Spookyet-2005-06-19
Removed "No_peer_ID". Compact Should Be Used.
? Dehaacked -
2005-05-03
· Added More Tracker / Client Keys from Various Extensions.
Spookyet -
2005-03-27
Added The "SB" Client Code for SwiftBit.
? Audiohub -
2005-03-11
· Piece Size
