The NS2 Network Protocol
I've been doing a little research about Spark's/NS2's network protocol and want to share my results.
The info in this post is largely based on a mail from Max (big thanks!).
<!--sizeo:4--><span style="font-size:14pt;line-height:100%"><!--/sizeo--><b>Fragments</b><!--sizec--></span><!--/sizec-->
There are two levels to the protocol. The first level is a low level protocol that's just used for sending reliable and unreliable messages. In Spark's protocol, we call the UDP packets fragments, to differentiate them from the messages of the protocol. A fragment may contain part of one or more protocol messages.
All fragments begin with an 8-bit integer specifying the type:
UInt8 fragmentType;
enum FragmentType
{
FragmentType_Reliable = 0,
FragmentType_Unreliable = 1,
FragmentType_Acknowledgement = 2, // Special acknowledgement packet with no data.
};
Based on the fragment type, the rest of the fragment format will vary.
<ul><li><b>Reliable Fragment</b>
Consists of the fragment header, followed by L1 packet chunks.
</li><li><b>Unreliable Fragment</b>
Unreliable fragments are very similar, except the header omits the receive fragment sequence.
Protocol packet chunks follow, just like in the reliable case.
</li><li><b>Acknowledgement Fragment</b>
Acknowledgment fragments are special types of fragments that are sent to acknowledge receiving reliable fragments. Normally these acknowledgments are piggy backed on reliable packets, but if we aren't sending any of those, we send a separate acknowledgment. No chunks.</li></ul>
<!--sizeo:3--><span style="font-size:12pt;line-height:100%"><!--/sizeo--><i>Fragment Header</i><!--sizec--></span><!--/sizec-->
<!--c1--><div class='codetop'>CODE</div><div class='codemain'><!--ec1-->Length (in Bytes) Data type Description
1 UInt8 Fragment type (0x00 - 0x02)
1 UInt8 Receive fragment sequence (last reliable fragment received,
omitted for type Unreliable)
1 UInt8 Send fragment sequence (omitted for type Acknowledgement)<!--c2--></div><!--ec2-->
<!--sizeo:4--><span style="font-size:14pt;line-height:100%"><!--/sizeo--><b>L1 Packet Chunks</b><!--sizec--></span><!--/sizec-->
Next comes the L1 protocol packet "chunks". The chunks are one or more pieces of a L1 packet. The protocol limits the size of a UDP packet to 1400 bytes, so bigger packets will be split between multiple fragments. Small packets may be combined into a single UDP packet.
Repeat until the end of the UDP packet:
<!--sizeo:3--><span style="font-size:12pt;line-height:100%"><!--/sizeo--><i>L1 Packet Header</i><!--sizec--></span><!--/sizec-->
<!--c1--><div class='codetop'>CODE</div><div class='codemain'><!--ec1-->Length (in Bytes) Data type Description
1 UInt8 Packet flags *
3 UInt24 Packet length *
1 UInt8 Marker, always 0xFF **
2 UInt16 Data length
* only present if start of L1 packet
** only present if NOT start of L1 packet<!--c2--></div><!--ec2-->
<!--sizeo:4--><span style="font-size:14pt;line-height:100%"><!--/sizeo--><b>High level protocol</b><!--sizec--></span><!--/sizec-->
This is were the game related communication happens.
I'm trying to figure it out by investigating the network stream.
The info in this post is largely based on a mail from Max (big thanks!).
<!--sizeo:4--><span style="font-size:14pt;line-height:100%"><!--/sizeo--><b>Fragments</b><!--sizec--></span><!--/sizec-->
There are two levels to the protocol. The first level is a low level protocol that's just used for sending reliable and unreliable messages. In Spark's protocol, we call the UDP packets fragments, to differentiate them from the messages of the protocol. A fragment may contain part of one or more protocol messages.
All fragments begin with an 8-bit integer specifying the type:
UInt8 fragmentType;
enum FragmentType
{
FragmentType_Reliable = 0,
FragmentType_Unreliable = 1,
FragmentType_Acknowledgement = 2, // Special acknowledgement packet with no data.
};
Based on the fragment type, the rest of the fragment format will vary.
<ul><li><b>Reliable Fragment</b>
Consists of the fragment header, followed by L1 packet chunks.
</li><li><b>Unreliable Fragment</b>
Unreliable fragments are very similar, except the header omits the receive fragment sequence.
Protocol packet chunks follow, just like in the reliable case.
</li><li><b>Acknowledgement Fragment</b>
Acknowledgment fragments are special types of fragments that are sent to acknowledge receiving reliable fragments. Normally these acknowledgments are piggy backed on reliable packets, but if we aren't sending any of those, we send a separate acknowledgment. No chunks.</li></ul>
<!--sizeo:3--><span style="font-size:12pt;line-height:100%"><!--/sizeo--><i>Fragment Header</i><!--sizec--></span><!--/sizec-->
<!--c1--><div class='codetop'>CODE</div><div class='codemain'><!--ec1-->Length (in Bytes) Data type Description
1 UInt8 Fragment type (0x00 - 0x02)
1 UInt8 Receive fragment sequence (last reliable fragment received,
omitted for type Unreliable)
1 UInt8 Send fragment sequence (omitted for type Acknowledgement)<!--c2--></div><!--ec2-->
<!--sizeo:4--><span style="font-size:14pt;line-height:100%"><!--/sizeo--><b>L1 Packet Chunks</b><!--sizec--></span><!--/sizec-->
Next comes the L1 protocol packet "chunks". The chunks are one or more pieces of a L1 packet. The protocol limits the size of a UDP packet to 1400 bytes, so bigger packets will be split between multiple fragments. Small packets may be combined into a single UDP packet.
Repeat until the end of the UDP packet:
<!--sizeo:3--><span style="font-size:12pt;line-height:100%"><!--/sizeo--><i>L1 Packet Header</i><!--sizec--></span><!--/sizec-->
<!--c1--><div class='codetop'>CODE</div><div class='codemain'><!--ec1-->Length (in Bytes) Data type Description
1 UInt8 Packet flags *
3 UInt24 Packet length *
1 UInt8 Marker, always 0xFF **
2 UInt16 Data length
* only present if start of L1 packet
** only present if NOT start of L1 packet<!--c2--></div><!--ec2-->
<!--sizeo:4--><span style="font-size:14pt;line-height:100%"><!--/sizeo--><b>High level protocol</b><!--sizec--></span><!--/sizec-->
This is were the game related communication happens.
I'm trying to figure it out by investigating the network stream.
Comments
<b>Send sequence</b>
I noticed that the unreliable fragments use more than one byte for the send fragment sequence. The header is three bytes longer than expected. After some 100,000 packets the sequence counter uses three bytes, so I assume it uses four in total.
<b>Handshake (?)</b>
Client and server perform something like a handshake protocol after the connecting is established.
Client: "SPARKNET"
Server: ACK
Server: <Empty packet>
Client: ACK
Client: "SPARKNET"
Then the normal communication begins.