Protocol Reference

This document describes the RIPE Atlas probe-controller communication protocol as implemented by starla. It was reverse-engineered from the official C probe (busybox) and verified against live RIPE Atlas controllers.

1. Connection Lifecycle

Registration

On startup, the probe connects via SSH to a registration server (e.g., reg03.atlas.ripe.net:443) and executes the INIT command with probe identification on stdin:

P_TO_R_INIT
TOKEN_SPECS fluffy 1000 5120 generic/unknown/x86_64
REASON_FOR_REGISTRATION NEW

The server responds with a controller assignment:

OK
CONTROLLER ctr-dub-sw01.atlas.prod.ripe.net 443 ssh-rsa AAAA...
REREGISTER 3000
PROBE_ID 1015186

If the probe is not yet registered, the server responds with plain OK (no CONTROLLER line). The probe retries with backoff until approved at atlas.ripe.net/apply/swprobe.

Controller INIT

After receiving the controller assignment, the probe connects to the controller via SSH and sends another INIT (no stdin data this time):

OK
SESSION_ID fa0b28e5f26291ad4a41ceecffc73457cbb4180291f53abdcda5667357440a9f
REMOTE_PORT 2023

The SESSION_ID is used for telnet authentication and result upload footer. REMOTE_PORT is the port the controller listens on for the reverse tunnel.

KEEP Session

The probe opens a long-lived SSH connection and executes KEEP. This channel blocks indefinitely and serves as a health monitor — if the channel closes, the connection is lost and the probe reconnects.

On this same SSH connection:

Reverse tunnel (tcpip_forward): controller connects to REMOTE_PORT to send measurement commands via telnet
Direct-tcpip channels: probe opens channels to 127.0.0.1:8080 on the controller for result uploads

2. Telnet Command Protocol

Authentication

When the controller connects via the reverse tunnel, the probe sends:

IAC DO ECHO
IAC DO NAWS
IAC WILL ECHO
IAC WILL SGA

Atlas probe, see http://atlas.ripe.net/

Probe 1015186 (hostname) login:

The controller sends C_TO_P_TEST_V1 as the login name, then the SESSION_ID as the password. On success: OK\r\n\r\n. On failure: BAD_PASSWORD\r\n\r\n and disconnect.

Command Formats

CRONLINE (recurring measurements)

CRONLINE <interval> <offset> <end_time> <spread_type> <spread> <command> [args...]

Field	Type	Description
interval	u64	Seconds between executions (0 = one-shot)
offset	u64	Initial delay offset (ignored by starla)
end_time	i64	Unix timestamp to stop (0 = never)
spread_type	string	Usually `UNIFORM`
spread	u32	Random spread in seconds
command	string	Measurement tool name

Example:

CRONLINE 240 269 1777902395 UNIFORM 3 evping -4 -c 3 -A "1001" -O /home/atlas/data/new/7 193.0.14.129

ONEOFF (one-shot measurements)

ONEOFF <path> <command> [args...]

User-triggered via the RIPE Atlas API. The path is ignored (it's where the C probe writes results). Internally converted to CRONLINE with interval=0.

JSON commands

{"type":"ping","msm_id":1001,"target":"8.8.8.8","af":4,"packets":3}

Ignored commands

CRONTAB, httppost, condmv, rptaddrs, buddyinfo, conntrack, dfrm — internal Atlas utilities that starla doesn't need.

3. Measurement Commands

Ping (`evping`)

evping [-4|-6] [-c count] [-s size] [-A msm_id] [-O output] [-I interval] <target>

Flag	Description	Default
`-4/-6`	Address family	4
`-c`	Packet count	3
`-s`	Packet size (bytes)	64
`-A`	Measurement ID	required

Result (PreFormatted array):

RESULT { "id":"1001", "fw":5120, "mver": "2.6.4", "lts":10,
  "time":1700000000, "dst_name":"193.0.14.129", "af":4,
  "dst_addr":"193.0.14.129", "src_addr":"10.0.0.1",
  "proto":"ICMP", "ttl":56, "size":64,
  "result": [ { "rtt":10.500000 }, { "rtt":11.200000 }, { "rtt":10.800000 } ] }

Timeouts: { "x":"*" }. RTT has 6 decimal places. ttl is omitted if all packets timed out. Requires CAP_NET_RAW.

Traceroute (`evtraceroute`)

evtraceroute [-4|-6] [-I|-U|-T] [-f first_hop] [-m max_hops]
  [-p paris_id] [-S size] [-A msm_id] [-O output] <target>

Flag	Description	Default
`-I/-U/-T`	ICMP/UDP/TCP protocol	UDP
`-f`	First hop	1
`-m`	Max hops	32
`-p`	Paris traceroute ID	0
`-S`	Packet size	40

Result (FullLine — custom envelope with endtime and paris_id):

RESULT { "id":"5001", "fw":5120, "mver": "2.6.4", "lts":0,
  "time":1700000000, "endtime":1700000030,
  "dst_name":"192.112.36.4", "dst_addr":"192.112.36.4",
  "src_addr":"10.0.0.1", "proto":"UDP", "af": 4,
  "size":40, "paris_id":5,
  "result": [
    { "hop":1, "result": [ { "from":"10.0.0.1", "ttl":255, "size":76, "rtt":3.723 } ] },
    { "hop":2, "result": [ { "x":"*" }, { "x":"*" }, { "x":"*" } ] },
    { "hop":13, "result": [ { "from":"192.112.36.4", "ttl":54, "rtt":31.981, "size":88 } ] }
  ] }

RTT has 3 decimal places. 3 probes per hop. Requires CAP_NET_RAW.

DNS (`evtdig`)

evtdig [-4|-6] [-t qtype] [-c qclass] [-T] [-r] [-d] [-e bufsize]
  [-R] [-h] [-b] [-i] [--soa] [--resolv] [--type N] [--class N]
  [--query NAME] [-A msm_id] [-O output] [@server] [query]

Flag	Description
`-T`	Use TCP (default: UDP)
`-h`	Query `hostname.bind CH TXT`
`-b`	Query `version.bind CH TXT`
`-i`	Query `id.server CH TXT`
`--soa`	Query SOA for `.`
`-r/-d/+dnssec`	Enable DNSSEC (DO bit)
`-R`	Disable recursion desired
`-e`	EDNS buffer size
`--type N`	Numeric query type (1=A, 28=AAAA, 16=TXT, etc.)
`--class N`	Numeric class (1=IN, 3=CH)
`--query NAME`	Query name
`@server`	Target DNS server IP

Template variables in query names

Variable	Expansion	Purpose
`$r`	8-char random alphanumeric	Prevent DNS caching
`$p`	Probe ID (decimal)	Per-probe uniqueness
`$t`	Unix timestamp (decimal)	Per-query uniqueness

Result (PreFormatted object with abuf):

RESULT { "id":"10310", "fw":5120, "mver": "2.6.4", "lts":51,
  "time":1775314719, "af":4, "dst_addr":"170.247.170.2",
  "dst_port":"53", "src_addr":"204.168.188.81", "proto":"UDP",
  "result": { "rt":29.846,"size":50,
    "abuf":"qr2AAAABAAEAAAAACGhvc3RuYW1lBGJpbmQAABAAA8AMABAAAwAAAAAABwZiNC1mcmE=",
    "ID":43709, "ANCOUNT":1, "QDCOUNT":1, "NSCOUNT":0, "ARCOUNT":0,
    "answers":[ {"TYPE":"TXT", "NAME":"hostname.bind.", "RDATA":[ "b4-fra" ]} ] } }

On timeout: "result": { "error":{"timeout":5000} }

abuf is the base64-encoded raw DNS wire response. This is the most important field — the RIPE Atlas API parses it server-side for detailed analysis. Without it, results are accepted but incomplete.

HTTP (`evhttpget`)

evhttpget [-4|-6] [-A msm_id] [-O output] [-M max_body] <url>

Result (PreFormatted array):

RESULT { "id":"12023", "fw":5120, "mver": "2.6.4", "lts":19,
  "time":1775317966, "dst_name":"example.com", "af":4,
  "dst_addr":"93.184.216.34", "proto":"TCP",
  "result": [ { "method":"GET", "af": 6,
    "dst_addr":"2600:140f:3::17df:2fba",
    "src_addr":"10.0.0.1", "rt":106.612889,
    "res":200, "ver":"1.1", "hsize":296, "bsize":0 } ] }

TLS (`evsslgetcert`)

evsslgetcert [-4|-6] [-p port] [-h hostname] [-A msm_id] [-O output] <target>

Result (FullLine — custom envelope):

RESULT { "id":"14002", "fw":5120, "mver": "2.6.4", "lts":0,
  "time":1775325499, "dst_name":"atlas.ripe.net", "dst_port":"443",
  "ttr":0.390310, "method":"TLS", "ver":"1.3",
  "dst_addr":"193.0.11.37", "af": 4, "src_addr":"10.0.0.1",
  "ttc":0.191244, "rt":0.390310, "server_cipher":"0x1302",
  "cert":[ "-----BEGIN CERTIFICATE-----\nMIIG...\n-----END CERTIFICATE-----",
           "-----BEGIN CERTIFICATE-----\nMIIF...\n-----END CERTIFICATE-----" ] }

Times (ttr, ttc, rt) in seconds. ver is "1.2" or "1.3" (not "Tls12"). server_cipher is hex ("0xc030"). Certificates are full PEM with escaped newlines.

NTP (`evntp`)

evntp [-4|-6] [-c count] [-A msm_id] [-O output] <target>

Result (FullLine with 3 samples):

RESULT { "id":"99998", "fw":5120, "mver": "2.6.4", "lts":12,
  "time":1775323252, "dst_addr":"213.239.234.28",
  "src_addr":"204.168.188.81", "proto":"UDP", "af": 4,
  "li":"no", "version":4, "mode":"server", "stratum":2,
  "poll":8, "precision":5.96046e-08,
  "root-delay":0.00956726, "root-dispersion":0.0142212,
  "ref-id":"7cd8a40e", "ref-ts":3984311600.292619705,
  "result": [
    { "origin-ts":3984312052.977718830, "receive-ts":3984312052.989268303,
      "transmit-ts":3984312052.989336491, "final-ts":3984312053.002320766,
      "rtt":0.024534, "offset":0.011981 },
    ...
  ] }

Field	Format
`li`	`"no"`, `"61"`, `"59"`, `"unknown"`
`poll`	Raw exponent (NOT 2^n)
`precision`	Scientific notation (2^n seconds)
`ref-id`	ASCII for stratum ≤ 1, hex for stratum ≥ 2
Timestamps	NTP epoch (seconds since 1900-01-01) with 9 decimal places

4. Result Upload Protocol

Results are uploaded via HTTP POST through SSH direct-tcpip channels to 127.0.0.1:8080 on the controller side.

POST /?PROBE_ID=1015186&SESSION_ID=fa0b28e5... HTTP/1.1
Host: 127.0.0.1
Content-Type: application/x-www-form-urlencoded
User-Agent: httppost for atlas.ripe.net
Connection: close
Content-Length: 1234

P_TO_C_REPORT
RESULT { "id":"9018", "fw":5120, ... disk stats ... }
RESULT { "id": "7001", "fw":5120, ... uptime ... }
RESULT { "id": "9002", "fw":5120, ... interface stats ... }
RESULT 9901 ongoing 1775483034 starla
RESULT { "id":"1001", "fw":5120, ... measurement result ... }
RESULT { "id":"1002", "fw":5120, ... measurement result ... }

SESSION_ID fa0b28e5...

The controller responds 200 OK with body OK\n on success. 429 means rate limiting — honor the Retry-After header. The official httppost uploads once per 60 seconds.

The controller requires system status results (9018, 7001, 9002, 9901) alongside measurement results. Without them, the controller rate-limits uploads with persistent 429 responses that survive reconnections. The official probe includes these in every upload batch.

System status results

These are included in every upload batch, before measurement results:

ID	Description	Format
`9018`	Disk stats	`bsize`, `blocks`, `bfree`, `free` (from `statvfs("/")`)
`7001`	Probe uptime	`uptime` in seconds since start, `lts`
`9002`	Network interfaces	Per-interface `bytes_recv`, `pkt_recv`, `bytes_sent`, `pkt_sent` (from `/proc/net/dev`)
`9901`	Ongoing status	Plain text: `RESULT 9901 ongoing <timestamp> starla`

5. RESULT Line Format Quirks

The controller may do rigid string parsing. These quirks must be matched exactly.

mver spacing: "mver": "2.6.4" has a space after the colon. All other fields use "key":value (no space). This matches the C probe's fprintf format string.
Field order: id, fw, mver, lts, time, [bundle], [dst_name], af, [dst_addr], [dst_port], [src_addr], proto, [ttl], [size], [endtime], [paris_id], result
Array spacing: [ { "rtt":10.5 }, { "rtt":11.2 } ] (spaces inside brackets and braces)
prb_id: NOT in the RESULT line — controller gets it from the URL query parameter
fw: Must be 5120 (or a real deployed version). Arbitrary values may be rejected.
FullLine bypass: TLS, NTP, and traceroute use the FullLine data variant which provides the complete body between RESULT { and }, bypassing the standard envelope formatter.

6. Known Issues and Lessons Learned

Serialization: tagged vs untagged enums

The MeasurementData enum must use #[serde(tag, content)] (tagged), not #[serde(untagged)]. With untagged serialization, FullLine("body") deserializes back as Generic(String("body")), losing the variant discriminator. This caused TLS/NTP/traceroute results to silently produce invalid RESULT lines.

HTTP af:0

The HTTP measurement computed the address family correctly but hardcoded af: 0 in the result struct. The controller silently drops results with af:0 while returning OK.

DNS timeouts must be results, not errors

When a DNS query times out (e.g., ISP blocks outbound UDP 53), the official probe reports {"error":{"timeout":5000}} as a valid measurement result that gets uploaded. Treating timeouts as execution errors causes them to be lost.

Upload rate: 60 seconds, not 10

The official httppost runs once per 60 seconds. Uploading more frequently triggers 429 rate limiting from the controller. The penalty is per-probe (tied to SSH key/probe ID) and persists across reconnections.

System status results are mandatory

The controller expects system health data (disk stats, uptime, network interface counters, ongoing status lines) in every upload batch. Sending only measurement RESULT lines causes persistent 429 rate limiting that survives reconnections and lasts hours. The official probe always includes these via its simpleping and condmv utilities.

DNS template variables

Query names may contain $r, $p, and $t templates. Failing to expand $p causes hickory-dns to reject the label as malformed (measurement 30001 uses these).