Aprs-is server requirements

These are collected server requirements for APRS-IS type core server

Requirements

Performance

APRS messages observed at APRS-IS core systems have following statistics:

• Hourly message rate is 120-150 thousand
• every exact 5 minutes there is a high burst of CWOP data, system must be able to handle 10 000 clients logging in and sending their messages within 5 seconds
• about 25% of all are CWOP messages
• Message lengths vary greatly:
  • 98.0% of all messages are shorter than 130 bytes long
  • reminder can be up to 300 bytes long

Buffering

This will likely need two kinds of message buffer pools: one with up to 136 (17*8) char messages, other with up to 300 char messages. (This count includes full generated path prefix, and line ending CRLF)

• The buffer contains a message text string that is ready to transmit with write() or in particulr with sendmsg() with indirect iovecs.
• The buffer is ready both for peer servers, and for clients, the difference is fixed size peer-to-peer prefix part the buffer, which is not sent to clients.
• There will be total about 60-80 thousand of such buffers at any given time, oldest ones being dropped to free list after 30 minutes

Communication

Core-internal communication:

• Connections are authenticated with peer specific token pair (username + password)
• Connections use own separate port for listening, and are essentially single directional flows
• Connections use TCP or SCPT streams with maximum possible segment size to optimize network flow
• Protocol is line oriented text ending with CRLF character pair, can it can contain even NUL bytes (character code 0) !
• Communication protocol with core peers carries a short prefix tag flagging which core nodes have already seen the message
  • This can be single byte with value 0x40 ("@"), which will permit BIT POSITION ENCODING of up to 6 "I have seen" flags in it.
  • For (in-)compability reasons this prefix should probably be 4 characters: "@xy@" where the "x" is flag encoder, and "y" is reserved for extension in case 6 peers is not enough. The "@" characters are literals.
• Average flow is 2-3 Mbit/s, CWOP bursts considerably higher

Core-Client communication:

• Protocol is line oriented text ending with CRLF character pair, can it can contain even NUL bytes (character code 0) !
• Majority is write-only, not receiving anything (CWOP bursts, especially)
• Complex filters make system to need pre-process each arrived packet for: message type, lat/lon coordinates if any, source callsign

Duplicate filter:

• Messages flowing core-internal, and from external need to be subjected to Duplicate detection.
• When message has not been seen before (within duplicate detector time window) it is forwarded to peers and clients wanting to receive messages, otherwise it is silently discarded
• Core-core messages are forwarded to peers that have not been flagged as "have seen this" in the header byte.

Duplicate detection

In order that the APRS-IS shall not unnecessarily relay packets — or multiply them in core-core network, it can have an in-memory collection of recently received packets, and compare that to new candidate packet.

Duplicate detection picks following bits off the packet:

• innermost AX.25 source address with SSID value (and without the VIA-path)
• innermost AX.25 destination address without SSID value
• datafield content with possible trailing CR, LF and whitespace characters removed

If same packet is seen within 60 seconds, it need not to be relayed again.

This comparison can be done with collecting comparison parts of the packet, and then calculating a checksum on them. Then placing it into a hash table with the checksum as key.

Every packet is always relayed without altering the number of whitespace characters at its tail.

Example: Following packets produce same comparison/checksum:

"OH2XYZ-11>APZYXW:>packet"
"OH2XYZ-11>APZYXW-4,RELAY,WIDE:>packet  "
"OH1YYY>APRS,WIDE:}OH2XYZ-11>APZYXW-4,TCPIP,OH1YYY*:>packet  "