Aprs-is server requirements

Radioamatööriwikistä
Siirry navigaatioon Siirry hakuun

These are collected server requirements for APRS-IS type core server

Requirements

Services

Services are roughly following:

  • Message inject to APRS-IS network without interest for messages in there
  • Filtered message feeds to clients
    • Filters fixed by ports and/or optionally given by the connecting client
  • Filtered message feeds to clients with 30 minute history upon connect
  • Full message traffic

Standard port (and filter/feature) list at the end of page: http://core.aprs.net/

Some pseudo-web-services to show current list of clients, ports, etc. Standard port for this is usually 15401, but other can be used as well (like 80) See: http://core.aprs.net/ for examples.

The "feed objects within NN km from your current location" and explicit filter of "NN km of my current location" requires two things:

  • Connecting client sends location beacon to network
  • The location beacon source callsignSSID matches exactly the used identification when the client connected to the system
  • Match area can vary with each location beacon received into the system

Filters

Some ports have implicit filters, others allow explicit to be given.

About filter specification language, see:

http://www.aprs-is.net/javaprssrvr/javaprsfilter.htm

About user validation, see:

http://www.aprs-is.net/valid.htm

Filters need common parsing of the packet data:

  • For duplicate removal filter use
  • For client selective filters

Data collected at pre-parser:

  • Canonic packet (for dupe filter)
  • Source callsign
  • Destination callsign
  • Packet content type
  • Possible lat/lon data

Callsign specific filters (radius N of callsign X) need to have a database of all known callsigns and their coordinates (e.g. all beacons). Data TTL is 30-60 minutes.

Dupe-filter TTL is around 30 to 60 seconds.

Performance

APRS messages observed at APRS-IS core systems have following statistics:

  • Hourly message rate is 120-150 thousand
  • every exact 5 minutes there is a high burst of CWOP data, system must be able to handle 10 000 clients logging in and sending their messages within 5 seconds
  • about 25% of all messages are CWOP type
  • around 85-90 % of all messages have lat/lon coordinates
  • To reduce system calls per second per socket, a 16 kB buffer is usually sufficient to write outbound data into, then doing write() for all client sockets for the accumulated data around once a second. If more data is to be added to buffer but it does not fit in, it is appropriate to do write() right then, and then continue with buffer filling.
  • Per minute flows outbound have been observed to be below 8.0 kB/s even with full feed at IS-IS core; peak per minute amount was seen approaching 330 kB/min.

This will likely mean that for normal clients there is no need to alter UNIX kernel setsockopt(SO_SNDBUF) default value.

  • For normal clients the value is expected to be 16 kB
  • For clients wanting large amounts of data (like full feeds), there may be need to increase the per-socket buffering a bit. (To about 160 kB ?)
  • Possibly auto-adjust the limit live by doubling the initial default value when socket write completes only partially or fails by giving EAGAIN or EWOULDBLOCK (non-blocking sockets are to be used, after all)
    • If reaching some configuration defined upper limit, close the socket
    • A control option for adjusting this upper limit value is needed

Overall design rules

  • Minimize number of context switches
  • Minimize number of system calls
  • Minimize number of data copies
  • Make number of threads configurable and not depend on the number of users or amount of traffic, so that it can be matched to the number of available CPU cores
  • Have online management console tool (web ?) (instead of needing the program stdin/stdout available at all times)
  • Enable at least some reconfigurability online
  • Consider storing message history data on disk file for a quick restart making such thing less "expensive" to overall performance, than just restarting cold

Buffering

Message lengths vary greatly:

  • 98.0% of all messages are shorter than 130 bytes long
  • reminder can be up to 300 bytes long

This will likely need two kinds of message buffer pools: one with up to 136 (17*8) char messages, other with up to 300 char messages. (This count includes full generated path prefix, and line ending CRLF)

  • The buffer contains a message text string that is ready to transmit with write() or in particular with sendmsg() with indirect iovecs.
  • The buffer is ready both for peer servers, and for clients, the difference is possible fixed size peer-to-peer prefix part the buffer, which is not sent to clients.
    • It is debatable if the core servers need to support the history dumps service, or only tier-2, those that do, will need about 60-80 thousand buffers at any given time, oldest ones being dropped to free list after 30 minutes
    • Message relaying service to client connection needs to accompany latest available position coordinate for given message originator, expected time window is around 15 minutes. Traditionally this all happens at iGates, which thus need to take full feed of the world...

Communication

Core-internal communication:

  • Connections use own separate port for listening, and are essentially single directional flows
  • Connections do not authenticate per se, rather they use IP ACLs and separate ports
  • Connections use TCP or SCPT streams with maximum possible segment size to optimize network flow
  • Usage of UDP packets carrying each peer packet is considerably less efficient in network terms than TCP or SCTP, and thus not recommended
  • Protocol is line oriented text ending with CRLF character pair, can it can contain even NUL bytes (character code 0) !
  • Core-core communication does not alter message labeling, only the insertion point stays visible
  • Average flow is 2-3 Mbit/s, CWOP bursts considerably higher
  • Complex "client interest filters" need preprocess each arrived packet for: message type, lat/lon coordinates if any, source callsign
  • N-way connected mesh where each message is sent exactly once to each core peer, and messages received from core peer channel are not sent onward to other core peers, only to clients
  • Possibly: Communication protocol with core peers carries a short prefix tag flagging which core nodes have already seen the message - thus permitting system which is not fully N-way connected!
    • This can be single byte with value 0x40 ("@"), which will permit BIT POSITION ENCODING of up to 6 "I have seen" flags in it.
    • For (in-)compatibility reasons this prefix should probably be 4 characters: "@xy@" where the "x" is flag encoder, and "y" is reserved for extension in case 6 peers is not enough. The "@" characters are literals.
    • Alternative to this is simply to use duplicate filter with penalty of extra copies being sent out from every peer when such flagging could tell which peers have already seen the message

Core-Tier2 communication:

  • Protocol is line oriented text ending with CRLF character pair, can it can contain even NUL bytes (character code 0) !
  • Feed is full unfiltered to both directions

Core/Tier2-Client communication:

  • Protocol is line oriented text ending with CRLF character pair, can it can contain even NUL bytes (character code 0) !
  • Majority is write-only, not receiving anything (CWOP bursts, especially)
  • Complex "client interest filters" mandate preprocessing of each arrived packet for: message type, lat/lon coordinates if any, source callsign
  • Channel shall have "heartbeat" message sent to client every 20 seconds, a fixed text line with leading "#" character, and ending with CRLF pair. Possibly this is a buffer like any other message, just typed as "heartbeat". This way it can carry also timestamp of its creation.
  • Initial greeting shall be a text line of "# xyzserver @ ... CRLF"
  • Client must do identification ("user mmmm pass pppp ...")
  • Validation is permitted to fail (wrong passcode, for example: -1)
    • Upon failed validation the messages are enforced with qAX flagging, and TCPXX via path.
  • If the client socket has been waiting for write ( = it has been choked from kernel services side ) for more than 30 seconds, the socket is discarded, and everything queued to it is discarded as well. (Perhaps limit of 60 seconds ?)

Duplicate filter:

  • Messages flowing core-internal, and from external need to be subjected to Duplicate detection.
  • When a message has not been seen before (within duplicate detector time window) it is forwarded to peers and clients wanting to receive messages, otherwise it is silently discarded
  • Core-core messages are forwarded to peers that have not been flagged as "have seen this" in the header byte.

Message receiver pre-parser looks following details:

  • The APRS message type character of the innermost packet
  • Origination callsign of the innermost packet
  • Destination callsign of the innermost packet
  • Offset and length of the innermost packet in the buffer
  • Calculate duplicate filter hash
  • After duplicate filtering: for packets with location coordinates, it shall parse lat/long data as floats for filter use. For math reasons they shall be stored in radians format.

Duplicate detection

In order that the APRS-IS shall not unnecessarily relay packets — or multiply them in core-core network, it can have an in-memory collection of recently received packets, and compare that to new candidate packet.

Duplicate detection picks following bits off the packet:

  • innermost AX.25 source address with SSID value (and without the VIA-path)
  • innermost AX.25 destination address without SSID value
  • datafield content with possible trailing CR, LF and whitespace characters removed

If same packet is seen within 60 seconds, it need not to be relayed again.

This comparison can be done with collecting comparison parts of the packet, and then calculating a checksum on them. Then placing it into a hash table with the checksum as key.

Every packet is always relayed without altering the number of whitespace characters at its tail.

Example: Following packets produce same comparison/checksum:

"OH2XYZ-11>APZYXW:>packet"
"OH2XYZ-11>APZYXW-4,RELAY,WIDE:>packet  "
"OH1YYY>APRS,WIDE:}OH2XYZ-11>APZYXW-4,TCPIP,OH1YYY*:>packet  "