One issue with the selection of an initial sequence number (ISN) for the TCP connection is spoofing. If the initial sequence number is guessable, an attacker can spoof the TCP handshake. The canonical attack has three parties: A the spoofed host, B the hapless server, X the attacker. The attacker first floods A to prevent it from receiving messages from B. This can be accomplished by any number of denial of service attacks. The attacker then connects to B to get sequence number data from B. Let's call this sequence number ISNb. If B is using a sequence number scheme that is easy to guess, the attacker can extrapolate the next sequence number(s) that B will use. Let's call this number ISNb'. The attacker will then connect to B pretending to be A. B will send a reply to A, that will never get there due to the denial of service on A, that contains B's sequence number, z. If the attacker has guessed correctly, z = ISNb'. The attacker can then spoof an acknowledgement of ISNb', and B will be tricked into thinking it has a connection with A. The flow of events can be summarized as follows:
X->A: Flood A <--- Denial of service X->B: SYN, ISNx <--- Attacker makes legitimate connection to B B->X: SYN, ISNb, ACK(ISNx+1) <--- From ISNb, X can guess next ISN, ISNb' X->B: Closes connection <--- Legit connection no longer needed Ax->B: SYN, ISNx' <--- From attacker, spoofing A B->A: SYN, z, ACK(ISNx'+1) <--- Goes to real A, attacker does not see z, guesses z = ISNb' Ax->B: ACK(ISNb'+1) <--- X uses guessed ISNb' to generate ACKIf the attacker can also guess how B will increment its sequence number during the connection, then the attacker can continue sending data to B while posing as A. The increment is usually the size of the data that B sends. This can be guessed if the application protocol uses relatively fixed sized responses from B for certain input.
Note that if the attacker can also guess ISNb' without connecting to B (such as if B is using a clock-based sequence number), then the connection from X->B is not needed and the attacker can skip down to the Ax->B step.
Also note that the attacker does not even need to be able to see packets traveling from B to A to make this attack succeed. As long as B uses a guessable sequence number generator and sends predictably sized responses, the attacker can guess the appropriate sequence number to use in the ACK field without any additional input.
The following papers, whitepages and advisories give more details about the ISN vulnerability:
Another intriguing consequence of this research into ISN predictability is using the ISN to fingerprint the system. Even if the operating system has taken care to make its ISN unpredictable, if it has a unique ISN generator, it can still be identified. Take for example the Nmap fingerprinting methods. Michal Zalewski also covers this issue in Chapter 10 of his book Silence on the Wire.
Since the specification originally required unique IDs for all packets within the maximum packet lifetime, many systems used a very predictable method of generating ID values, such as increasing the value by 1 for each new source/destination/protocol tuple. One can use this predictability to gauge how many packets a machine has processed since you last connected to them. This can be useful as a side-channel attack to portmap a victim. This method is used by Nmap to do their IP ID scan (Idle scan).
Additionally, an attacker can use this predictability to interfer with normal fragment reassembly by injecting spoofed packets into the subnet. This can cause destruction of data (spurious injected fragments), intentional changes to data (specially crafted fragments to overwrite other legitimate fragments), or interception of data. See the following paper for more details: Fragmentation Considered Vulnerable by Yossi Gilad and Amir Herzburg.
SYN cookies are a way to create the server's ISN such that the server doesn't need to store information in the connection table to realize an incoming ACK is the last packet of the connection establishment handshake. There is also an extension header (TCP Cookie Transactions) which supports a similar exchange, but it requires both the client and server understand the extension header (e.g. it can be easily bypassed by a non-compliant botnet).
The SYN cookie consists of a timestamp (5 bits), an encoded maximum segment size (MSS) requested for the transaction (3 bits), and a cryptographic hash computed over the src IP, src port, dest IP, dest port tuple and the timestamp (24 bits). This is sent as the server's chosen ISN. (Note: there is some predictability here because of the timestamp and encoded MSS). When the client sends the ACK with the server's ISN+1, the server subtracts 1 and verifies the timestamp and cryptographic hash.