Compression: IP Header compression

[pflarr]

Expected Savings: 1.7%
Difficulty: Moderate

Outside of the IP addresses, most IPv4 header information is redundant or unused. The IP addresses themselves will be common across each flow, and stored separately in the flow index. So, given that flow index information, what headers do we actually need? (The counts/percentages are taken from a typical intranet to outside network sample of 10^5 packets)

• version - known via flow index
• IHL - The IP header length is 20 in 100% of IPv4 packets seen, though it can vary.
• DSCP - 0 in 60% of packets. Probably needs to be preserved
• ECN - 0 in 99% of packets.
• Total Length - Computable from Pcap packet header data
• Identification - Can generally be ignored on unfragmented packets, though is sometimes used for other purposes. In our sample, most packets had identification set to one of a few numbers, but the normal case is that it be 0.
• flags
  • Don't Fragment flag - set in 63% of packets.
  • More Fragment flag - not set in 100% of packets
• Fragment Offset - only used in fragmented packets. (Set to 0 in 100% of sample).
• TTL - arguably important. Varies widely.
• protocol - In session data
• checksum - can, and should, simply be regenerated.
• src/dst ip address - in flow header
• options - If the packet has options, then enough other fields (namely IHL) will be different that packet compression already won't make sense for the packet. The most commonly expected options will involve IPsec.

Proposed Compressed Header:

   0      |    1     |     2    |    3     |    4     |
 Flags    |  TTL     | DSCP/ECN | Identification      |

• In the above header, DSCP/ECN and identification are optional, and only present if not 0.
• Flags consists of:
• 0xE - to indicate that this is a compressed packet.
• A DSCP/ECN included flag
• An Identification included flag
• The Don't Fragment and More Fragment bits.

This headers that have any fields that don't conform to our defaults (<1%) or optional fields would remain uncompressed. Fully compressed headers would be reduced in size by 18 bytes per packet. For 1 GB of captured data, this would result in an average savings (for 10^6 packets per GB) of 1.7%.