04 · protocol

Protocol reference.

How probes and controllers actually talk: lifecycle, command formats, result lines, upload envelopes, and the rigid little quirks the controller demands.

This document describes the RIPE Atlas probe-controller communication protocol as implemented by starla. It was reverse-engineered from the official C probe (busybox) and verified against live RIPE Atlas controllers.

1Connection lifecycle

Registration

On startup, the probe SSHes to a registration server (e.g. reg03.atlas.ripe.net:443) and runs INIT with probe identification on stdin:

stdinP_TO_R_INIT
P_TO_R_INIT
TOKEN_SPECS fluffy 1000 5120 generic/unknown/x86_64
REASON_FOR_REGISTRATION NEW

The server responds with a controller assignment:

stdoutresponse
OK
CONTROLLER ctr-dub-sw01.atlas.prod.ripe.net 443 ssh-rsa AAAA...
REREGISTER 3000
PROBE_ID 1015186

If the probe is not yet registered, the server responds with plain OK (no CONTROLLER line). The probe retries with backoff until approved at atlas.ripe.net/apply/swprobe.

Controller INIT

After receiving the controller assignment, the probe connects to the controller via SSH and sends another INIT (no stdin data this time):

stdoutsession
OK
SESSION_ID fa0b28e5f26291ad4a41ceecffc73457cbb4180291f53abdcda5667357440a9f
REMOTE_PORT 2023

The SESSION_ID is used for telnet authentication and the result-upload footer. REMOTE_PORT is the port the controller listens on for the reverse tunnel.

KEEP session

The probe opens a long-lived SSH connection and runs KEEP. This channel blocks indefinitely and serves as a health monitor; if it closes, the connection is lost and the probe reconnects.

On this same SSH connection:

Reverse tunnel (tcpip_forward): controller connects to REMOTE_PORT to send measurement commands via telnet.
Direct-tcpip channels: probe opens channels to 127.0.0.1:8080 on the controller for result uploads.

2Telnet command protocol

Authentication

When the controller connects via the reverse tunnel, the probe sends:

telnetbanner + login
IAC DO ECHO
IAC DO NAWS
IAC WILL ECHO
IAC WILL SGA

Atlas probe, see http://atlas.ripe.net/

Probe 1015186 (hostname) login:

The controller sends C_TO_P_TEST_V1 as the login, then the SESSION_ID as the password. On success: OK\r\n\r\n. On failure: BAD_PASSWORD\r\n\r\n and disconnect.

CRONLINE — recurring

grammarcronline
CRONLINE <interval> <offset> <end_time> <spread_type> <spread> <command> [args...]

Field	Type	Description
interval	u64	Seconds between executions (0 = one-shot)
offset	u64	Initial delay offset (ignored by starla)
end_time	i64	Unix timestamp to stop (0 = never)
spread_type	string	Usually `UNIFORM`
spread	u32	Random spread in seconds
command	string	Measurement tool name

examplecronline ping
CRONLINE 240 269 1777902395 UNIFORM 3 evping -4 -c 3 -A "1001" -O /home/atlas/data/new/7 193.0.14.129

ONEOFF — one-shot

grammaroneoff
ONEOFF <path> <command> [args...]

User-triggered via the RIPE Atlas API. The path is ignored (it's where the C probe writes results). Internally converted to a CRONLINE with interval=0.

JSON

jsondirect command
{"type":"ping","msm_id":1001,"target":"8.8.8.8","af":4,"packets":3}

Ignored commands

CRONTAB, httppost, condmv, rptaddrs, buddyinfo, conntrack, dfrm: internal Atlas utilities starla doesn't need.

3Measurement commands

Ping — `evping`

grammar
evping [-4|-6] [-c count] [-s size] [-A msm_id] [-O output] [-I interval] <target>

Flag	Description	Default
`-4/-6`	Address family	4
`-c`	Packet count	3
`-s`	Packet size (bytes)	64
`-A`	Measurement ID	required

jsonresult · PreFormatted array
RESULT { "id":"1001", "fw":5120, "mver": "2.6.4", "lts":10,
  "time":1700000000, "dst_name":"193.0.14.129", "af":4,
  "dst_addr":"193.0.14.129", "src_addr":"10.0.0.1",
  "proto":"ICMP", "ttl":56, "size":64,
  "result": [ { "rtt":10.500000 }, { "rtt":11.200000 }, { "rtt":10.800000 } ] }

Timeouts: { "x":"*" }. RTT has 6 decimals. ttl is omitted if all packets timed out. Requires CAP_NET_RAW.

Traceroute — `evtraceroute`

grammar
evtraceroute [-4|-6] [-I|-U|-T] [-f first_hop] [-m max_hops]
  [-p paris_id] [-S size] [-A msm_id] [-O output] <target>

Flag	Description	Default
`-I/-U/-T`	ICMP / UDP / TCP	UDP
`-f`	First hop	1
`-m`	Max hops	32
`-p`	Paris traceroute ID	0
`-S`	Packet size	40

jsonresult · FullLine
RESULT { "id":"5001", "fw":5120, "mver": "2.6.4", "lts":0,
  "time":1700000000, "endtime":1700000030,
  "dst_name":"192.112.36.4", "dst_addr":"192.112.36.4",
  "src_addr":"10.0.0.1", "proto":"UDP", "af": 4,
  "size":40, "paris_id":5,
  "result": [
    { "hop":1, "result": [ { "from":"10.0.0.1", "ttl":255, "size":76, "rtt":3.723 } ] },
    { "hop":2, "result": [ { "x":"*" }, { "x":"*" }, { "x":"*" } ] },
    { "hop":13, "result": [ { "from":"192.112.36.4", "ttl":54, "rtt":31.981, "size":88 } ] }
  ] }

RTT 3 decimals. 3 probes per hop. Requires CAP_NET_RAW.

DNS — `evtdig`

grammar
evtdig [-4|-6] [-t qtype] [-c qclass] [-T] [-r] [-d] [-e bufsize]
  [-R] [-h] [-b] [-i] [--soa] [--resolv] [--type N] [--class N]
  [--query NAME] [-A msm_id] [-O output] [@server] [query]

Flag	Description
`-T`	Use TCP (default UDP)
`-h`	Query `hostname.bind CH TXT`
`-b`	Query `version.bind CH TXT`
`-i`	Query `id.server CH TXT`
`--soa`	Query SOA for `.`
`-r/-d/+dnssec`	Enable DNSSEC (DO bit)
`-R`	Disable recursion desired
`-e`	EDNS buffer size
`--type N`	Numeric type (1=A, 28=AAAA, 16=TXT…)
`--class N`	Numeric class (1=IN, 3=CH)
`--query NAME`	Query name
`@server`	Target DNS server IP

Template variables in query names

Variable	Expansion	Purpose
`$r`	8-char random alphanumeric	Prevent DNS caching
`$p`	Probe ID (decimal)	Per-probe uniqueness
`$t`	Unix timestamp (decimal)	Per-query uniqueness

jsonresult · PreFormatted object · abuf
RESULT { "id":"10310", "fw":5120, "mver": "2.6.4", "lts":51,
  "time":1775314719, "af":4, "dst_addr":"170.247.170.2",
  "dst_port":"53", "src_addr":"204.168.188.81", "proto":"UDP",
  "result": { "rt":29.846,"size":50,
    "abuf":"qr2AAAABAAEAAAAACGhvc3RuYW1lBGJpbmQAABAAA8AMABAAAwAAAAAABwZiNC1mcmE=",
    "ID":43709, "ANCOUNT":1, "QDCOUNT":1, "NSCOUNT":0, "ARCOUNT":0,
    "answers":[ {"TYPE":"TXT", "NAME":"hostname.bind.", "RDATA":[ "b4-fra" ]} ] } }

On timeout: "result": { "error":{"timeout":5000} }

⚠ abuf

abuf is the base64-encoded raw DNS wire response. The RIPE Atlas API parses it server-side. Without it, results are accepted but incomplete.

HTTP — `evhttpget`

grammar
evhttpget [-4|-6] [-A msm_id] [-O output] [-M max_body] <url>

jsonresult
RESULT { "id":"12023", "fw":5120, "mver": "2.6.4", "lts":19,
  "time":1775317966, "dst_name":"example.com", "af":4,
  "dst_addr":"93.184.216.34", "proto":"TCP",
  "result": [ { "method":"GET", "af": 6,
    "dst_addr":"2600:140f:3::17df:2fba",
    "src_addr":"10.0.0.1", "rt":106.612889,
    "res":200, "ver":"1.1", "hsize":296, "bsize":0 } ] }

TLS — `evsslgetcert`

grammar
evsslgetcert [-4|-6] [-p port] [-h hostname] [-A msm_id] [-O output] <target>

jsonresult · FullLine
RESULT { "id":"14002", "fw":5120, "mver": "2.6.4", "lts":0,
  "time":1775325499, "dst_name":"atlas.ripe.net", "dst_port":"443",
  "ttr":0.390310, "method":"TLS", "ver":"1.3",
  "dst_addr":"193.0.11.37", "af": 4, "src_addr":"10.0.0.1",
  "ttc":0.191244, "rt":0.390310, "server_cipher":"0x1302",
  "cert":[ "-----BEGIN CERTIFICATE-----\nMIIG...\n-----END CERTIFICATE-----",
           "-----BEGIN CERTIFICATE-----\nMIIF...\n-----END CERTIFICATE-----" ] }

Times (ttr, ttc, rt) in seconds. ver is "1.2" or "1.3" (not "Tls12"). server_cipher is hex ("0xc030"). Certificates are full PEM with escaped newlines.

NTP — `evntp`

grammar
evntp [-4|-6] [-c count] [-A msm_id] [-O output] <target>

jsonresult · 3 samples
RESULT { "id":"99998", "fw":5120, "mver": "2.6.4", "lts":12,
  "time":1775323252, "dst_addr":"213.239.234.28",
  "src_addr":"204.168.188.81", "proto":"UDP", "af": 4,
  "li":"no", "version":4, "mode":"server", "stratum":2,
  "poll":8, "precision":5.96046e-08,
  "root-delay":0.00956726, "root-dispersion":0.0142212,
  "ref-id":"7cd8a40e", "ref-ts":3984311600.292619705,
  "result": [
    { "origin-ts":3984312052.977718830, "receive-ts":3984312052.989268303,
      "transmit-ts":3984312052.989336491, "final-ts":3984312053.002320766,
      "rtt":0.024534, "offset":0.011981 },
    ...
  ] }

Field	Format
`li`	`"no"` · `"61"` · `"59"` · `"unknown"`
`poll`	Raw exponent (NOT 2^n)
`precision`	Scientific notation (2^n seconds)
`ref-id`	ASCII for stratum ≤ 1, hex for stratum ≥ 2
Timestamps	NTP epoch (seconds since 1900-01-01) · 9 decimal places

4Result upload protocol

Results are uploaded via HTTP POST through SSH direct-tcpip channels to 127.0.0.1:8080 on the controller side.

httpPOST · over SSH channel
POST /?PROBE_ID=1015186&SESSION_ID=fa0b28e5... HTTP/1.1
Host: 127.0.0.1
Content-Type: application/x-www-form-urlencoded
User-Agent: httppost for atlas.ripe.net
Connection: close
Content-Length: 1234

P_TO_C_REPORT
RESULT { "id":"9018", "fw":5120, ... disk stats ... }
RESULT { "id": "7001", "fw":5120, ... uptime ... }
RESULT { "id": "9002", "fw":5120, ... interface stats ... }
RESULT 9901 ongoing 1775483034 starla
RESULT { "id":"1001", "fw":5120, ... measurement result ... }
RESULT { "id":"1002", "fw":5120, ... measurement result ... }

SESSION_ID fa0b28e5...

The controller responds 200 OK with body OK\n on success. 429 means rate limiting; honor the Retry-After header. The official httppost uploads once per 60 seconds.

⚠ Mandatory system status

The controller requires system status results (9018, 7001, 9002, 9901) alongside measurement results. Without them, the controller rate-limits with persistent 429 responses that survive reconnections.

System status results

Included in every upload batch, before measurement results:

ID	Description	Format
`9018`	Disk stats	`bsize`, `blocks`, `bfree`, `free` from `statvfs("/")`
`7001`	Probe uptime	`uptime` seconds since start, `lts`
`9002`	Network interfaces	Per-iface `bytes_recv`, `pkt_recv`, `bytes_sent`, `pkt_sent` from `/proc/net/dev`
`9901`	Ongoing status	Plain: `RESULT 9901 ongoing <ts> starla`

5RESULT line quirks

⚠ Match exactly

The controller may do rigid string parsing. These quirks must be matched.

mver spacing — "mver": "2.6.4" has a space after the colon. All other fields use "key":value (no space). Matches the C probe's fprintf format string.
Field order — id, fw, mver, lts, time, [bundle], [dst_name], af, [dst_addr], [dst_port], [src_addr], proto, [ttl], [size], [endtime], [paris_id], result.
Array spacing — [ { "rtt":10.5 }, { "rtt":11.2 } ] with spaces inside brackets and braces.
prb_id — NOT in the RESULT line; controller reads it from the URL query.
fw — must be 5120 (or a real deployed version). Arbitrary values may be rejected.
FullLine bypass — TLS, NTP, traceroute use the FullLine variant; the complete body sits between RESULT { and }, bypassing the standard envelope formatter.

6Known issues & lessons

Tagged vs untagged enums

The MeasurementData enum must use #[serde(tag, content)] (tagged), not #[serde(untagged)]. With untagged, FullLine("body") deserializes back as Generic(String("body")), losing the variant discriminator. This caused TLS / NTP / traceroute results to silently produce invalid RESULT lines.

HTTP af:0

The HTTP measurement computed the address family correctly but hardcoded af: 0 in the result struct. The controller silently drops results with af:0 while returning OK.

DNS timeouts must be results, not errors

When a DNS query times out (e.g. ISP blocks outbound UDP 53), the official probe reports {"error":{"timeout":5000}} as a valid measurement result. Treating timeouts as execution errors loses them.

Upload rate: 60 seconds, not 10

The official httppost runs once per 60 seconds. Uploading more often triggers 429 rate limiting from the controller. The penalty is per-probe (tied to SSH key / probe ID) and persists across reconnections.

System status results are mandatory

The controller expects system health data (disk stats, uptime, network interface counters, ongoing status) in every upload batch. Sending only measurement RESULT lines causes persistent 429 rate limiting that survives reconnections and lasts hours.

DNS template variables

Query names may contain $r, $p, and $t templates. Failing to expand $p causes hickory-dns to reject the label as malformed (measurement 30001 uses these).

Protocol reference.

1Connection lifecycle

Registration

Controller INIT

KEEP session

2Telnet command protocol

Authentication

CRONLINE — recurring

ONEOFF — one-shot

JSON

Ignored commands

3Measurement commands

Ping — evping

Traceroute — evtraceroute

DNS — evtdig

Template variables in query names

HTTP — evhttpget

TLS — evsslgetcert

NTP — evntp

4Result upload protocol

System status results

5RESULT line quirks

6Known issues & lessons

Tagged vs untagged enums

HTTP af:0

DNS timeouts must be results, not errors

Upload rate: 60 seconds, not 10

System status results are mandatory

DNS template variables

Ping — `evping`

Traceroute — `evtraceroute`

DNS — `evtdig`

HTTP — `evhttpget`

TLS — `evsslgetcert`

NTP — `evntp`