Gnutella Protocol Development
Home :: Developer :: Press :: Research :: Servents
Introducing New Bye (0x02) Packet in Gnutella 0.4 Raphael ManfrediSeptember, 11th 2001 Updated October, 5th 2001 Updated February, 4th 2002 Updated March 15th, 2002 1. Introduction This proposal is about a new Bye packet that would be sent by servents upon disconnecting from a connected node, giving a final reason that the remote end can display. It is safe to retrofit the Bye packet into the 0.4 protocol. Indeed, the packet is the last thing that will be sent by a disconnecting servent, and it will be otherwise ignored as a bad packet by older servents, which is not dramatic given it is the last thing they will see on that connection. This packet is believed to be useful to the developers of Gnutella servents. Users will be able to report the error, and maybe understand what is going wrong, with them or with the remote node. This packet is also considered by the tenant of that proposal as carrying some "social value". Friends say good-bye to each other when they part, and the Gnutella Network is some kind of modern electronic friendship, where people gather and friendly share and exchange files, to the mutual benefit of all parties. 2. The Bye (0x02) Packet The payload descriptor is 0x02, and the payload content is: +------+-----------------------+ | Code | NUL-terminated String | +------+-----------------------+ where Code is a 2-byte error code, encoded in little endian, as usual on Gnutella. The remaining is a NUL-terminated String, giving a description of the error. The presence of the Code allows for automated processing of the message, and the regular SMTP classification of error code ranges should apply. Of particular interests are the 200..299, 400..499 and 500..599 ranges. Here is the general classification ("User" here refers to the remote node that we are disconnecting from): 2xx The User did nothing wrong, but the servent chose to close the connection: it is either exiting normally (200), or the local manager of the servent requested an explicit close of the connection (201). 4xx The User did something wrong, as far as the servent is concerned. It can send packets deemed too big (400), too many duplicate messages (401), relay improper queries (402), relay messages deemed excessively long-lived [hop+TTL > max] (403), send too many unknown messages (404), the node reached its inactivity timeout (405), it failed to reply to a ping with TTL=1 (406), or it is not sharing enough (407). 5xx The servent noticed an error, but it is an "internal" one. It can be an I/O error or other bad error (500), a protocol desynchronization (501), the send queue became full (502). The format of the String is the following ( refers to "\r" and to "\n"): Error message, as descriptive as possible or optionally, something more qualified, with HTTP-like headers giving out more information: Error message, as descriptive as possible Server: some server/version X-Gnutella-XXX: some specific gnutella header for instance telling the host about alternate nodes it could connect to The presence of a at the end of message indicates that HTTP-like headers are present. The absence of any indicates that the short error message form was used. Unless circumstances making that impossible (urgent disconnection due to a memory fault), the HTTP-like headers version should be used, with at least a Server: header, allowing better tracing and debugging. 3. Sending a Bye (0x02) Packet Servents should send a Bye packet to a node as the last thing on the network, and then close the connection. TCP/IP will make its best to relay the final data to the other end, even though the local connection appears closed. The data may not be able to make it though, but at least the servent tried. A Bye packet must be sent with TTL=1 (to avoid accidental propagation by an unaware servent), and hop=0 (of course!). [Note on 15/03/2002: see Appendix 2 and Appendix 3 for slight amendments] 4. Reception of a Bye (0x02) Packet A Bye packet should have hop=0, TTL=1 upon reception, or it should be immediately discarded as being incorrect. Upon reading a a Bye packet, a node should immediately close the connection and stop processing of any other received packet from that connection that was still pending processing. A Bye packet should not be sent back on the connection before closing, since the remote party already initiated the farewell. 5. Write Errors When a servent is unable to write to an another node (TCP/IP reports the connection was closed, or the send buffer is full because the node does not read quickly enough), it should perform additional checking: * If the connection is reported closed, the servent should begin reading from the connection, as TCP/IP buffers the pending input and will deliver all the bytes sent by the remote party before closing and received by the local host. The servent should skip all the packets and look for a Bye (0x02) packet. If it finds it, it should display/log the error. Otherwise, it will report a disconnection as usual. * If the sending buffer is full, the servent should remove all pending messages, leaving only the only message that could possibly be partially sent, and which needs to be flush in whole to avoid protocol desynchronization. Then, it should enqueue a Bye packet, with error code 502, and attempt the delivery of that information for some reasonable time. In any case, no further data should be enqueued for this node. The servent still needs to read the incoming data to avoid filling of its own transmit queue at the other end, but discard all read data immediately. This includes any partially received packet that we got before discovery of the "send queue full" condition. 6. Examples Here are examples of Bye packet payloads. I am showing the error code in ASCII, but keep in mind it is encoded in the leading two bytes of the payload, and should not be repeated in ASCII in the message String. This is simply done here for reading convenience. 200 Shutting Down Server: my server/version 502 Send Queue Full (40960 bytes) Server: my server/version 401 Too Many Duplicates (Got 12 packets, 1.4% of your traffic) Server: my server/version 7. Conclusion Because the Bye (0x02) packet is the last one sent, it is fully compatible with the 0.4 protocol. Servents could begin sending it this very minute, nothing would break in the Gnutella Network. Due to this very desirable property, it is hoped that the Bye (0x02) packet will be retrofitted into the protocol document for version 0.4. Finally, due to its simple format and its nice social nature, it is expected to be widespread quite quickly. Expect to get more informative disconnection messages anytime soon. --RAM, 11/09/2001 A. Appendix 1: Handshaking Headers This proposal was initially made for the 0.4 protocol, but with 0.6 comes the ability for servents to exchange about supported features. There is no real need to declare any support for the Bye packet, since by construction this packet will be the last one in the stream. However, doing so provides implicit information to the remote node: If a servent declaring support for the Bye packet does not send one before the connection is broken, one can be fairly certain that the connection was closed unexpectedly (e.g. the process was killed). This specification of the Bye packet therefore requires servents to emit the following header during handshaking: Bye-Packet: 0.1 This advertising for Bye also allows possible extension of those specifications. Further versions will be backward compatible, so the lowest version number should be retained when negotiating. Even if the remote end advertises version 0.2, emitting support indication for 0.1 will force the peer to limit itself to the 0.1 specifications. --RAM, 04/02/2002 B. Appendix 2: A Step Towards Controlled TIME_WAIT Penalty Although not originally designed for that purpose, the combination of 0.6 handshaking and BYE gives the ability to avoid getting the TIME_WAIT state on a socket, for the party initiating the BYE. The TIME_WAIT problem stems from the TCP/IP state machine, whereby the side initiating the connection close (i.e. sending the first FIN) will end-up in a TIME_WAIT state where it will stay for twice the Maximum Segment Life, which is typically 2 minutes. That means 4 minutes before the TCP/IP Transfer Control Block can be reclaimed, since TCP keeps it around to be able to send back an RST (reset) condition for any packet reaching the port. After the 4 minutes delay, TCP can rest assurred that the whole connection is cleared, and can therefore release the TCB, making the port available again. Look at: http://www-old.ics.uci.edu/pub/ietf/http/draft-ietf-http-connection-00.txt for more detailed explainations. Since the amount of kernel memory and the amount of ports are both limited quantities, it can be important to not get too many TIME_WAIT. The important thing to note is that it is the side initiating the close that will get the TIME_WAIT penalty. How is this related to BYE? Simple: * A servent supporting BYE will have to close() the connection when receiving the BYE packet. * Because of Bye-Packet advertising in the handshake, we know whether a servent support BYE. Therefore, the following (optional) modification of the original specifications is added: * In section 3., after sending the BYE packet, a servent not willing to pay the TIME_WAIT penalty can defer the close() operation for "some time". This time can be estimated to twice the typical round-trip for an "alive PING" message (i.e. a PING with TTL=1, hops=0). When "some time" has elapsed, a close() is forced nonetheless to free the resources. Naturally, this deferring of close() only makes sense if the remote node gave explicit indication for BYE support in its handshaking. A servent handshaking at 0.4 cannot advertise BYE support conveniently, but can nonetheless support BYE. Therefore, although it makes sense to send a BYE to a 0.4 servent, for its original tracing purposes, it is almost a wasted bet to defer the close(). The idea is that section 4. will be strictly followed: upon reception of BYE, the servent MUST close() the connection. The receiving servent MUST NOT play tricks and try to avoid closing the connection itself to not pay the TIME_WAIT penalty, and that is irrespective of whether it initiated the connection or not to the remote node that is now sending the BYE packet. C. Appendix 3: Increasing Chances of BYE Getting Delivered To prevent having the stack of the receiving BYE packet discard it because further data sent reached a closed socket and were therefore rejected with a RST, thereby causing the yet-unacknowledged BYE to be discarded, the side sending BYE should refrain from calling close(), but instead use shutdown(SHUT_WR) to close the writing side, yet keep the reading side opened. This is sending a FIN anyway, so it will not prevent TIME_WAIT, only increase the chances that the remote end will get the BYE payload. After proper timeout, the reading side will be closed. The purpose here is simply to increase the chances that the remote application will read the BYE and therefore handle its payload. An application combining the use of shutdown(SHUT_WR) with the deferred shutdown() scheme of Appendix 2 will truly maximize the chances for BYE to be properly delivered. Only true network congestion could prevent delivery before timouts are triggered. --RAM, 15/03/2002