Conversation
HERMES has a blog post and a paper but no home on the site, so there is nowhere to point someone who wants to query the data. This adds that section. It sits under /tests/ rather than /data/, in a new "Analysis Systems" group: HERMES is not a test M-Lab runs, but it is a system M-Lab operates, and the site's existing pattern is that a system's pages live with the system while /data/ links across to them. NDT works the same way. The framing throughout keeps HERMES distinct from its traceroute-enrichment stage. Enrichment produces the annotated paths HERMES uses as evidence; it is one stage of the pipeline, reachable only through the methodology page, and every enrichment page opens by saying so. Without that boundary the enrichment tends to become the description of the whole product, which understates the detection and localization work that is most of the published data. Pages added: /tests/hermes/ landing /tests/hermes/quickstart/ access, cost, first queries /tests/hermes/schema/ both tables, field by field /tests/hermes/examples/ worked queries /tests/hermes/methodology/ pipeline, stage by stage /tests/hermes/methodology/path-enrichment/ the enrichment component /learn/traceroute/ general-audience explainer The traceroute explainer is in /learn/ rather than under HERMES because it serves scamper1, reverse traceroute, and IPRS readers equally, and /learn/ already says it is being restructured to hold exactly this kind of resource. Field names, types and structure are read from create_events_enriched.sql and 04_mapping_union.sql in m-lab/hermes, and from the pipeline DDLs. Enrichment data sources are as confirmed by the HERMES team. Still a draft, and marked as such on every page. Open before publication: - access path is unresolved (mlab-collaboration grants vs. measurement-lab) - no example query has been run against the live table - row grain and coverage window need confirming - types need checking against INFORMATION_SCHEMA.COLUMNS - events_enriched does not expose the statistical test outputs Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0167k8uZh7uL3Ei33tgpBfAV
| * HERMES identifies when groups of Internet users experience a statistically significant drop in performance, and reports the parts of the network most closely associated with that drop. | ||
| * HERMES analyzes M-Lab NDT, traceroute, and reverse traceroute data; it does not collect measurements of its own. | ||
| * More information is available in the [HERMES Methodology]({{ site.baseurl }}/tests/hermes/methodology/) and the [SIGCOMM 2026 paper](https://doi.org/10.1145/3789240.3829129){:target="_blank"}. | ||
| * [HERMES Data in BigQuery]({{ site.baseurl }}/tests/hermes/schema/) - [HERMES Access and QuickStart]({{ site.baseurl }}/tests/hermes/quickstart/) |
There was a problem hiding this comment.
Instead of repeating links to resources, I suggest adding a link to the documentation e.g., to /tests/hermes/#start-here (so that we don’t have duplicate info in the website)
| * [Access and QuickStart]({{ site.baseurl }}/tests/hermes/quickstart/) | ||
| * [Table schema]({{ site.baseurl }}/tests/hermes/schema/) | ||
| * [Example queries and tutorials]({{ site.baseurl }}/tests/hermes/examples/) | ||
| * [Methodology]({{ site.baseurl }}/tests/hermes/methodology/) |
There was a problem hiding this comment.
To avoid having duplicate info in the website, I suggest removing these links and keep only the main link to Hermes
| | Table | Use it for | | ||
| | --- | --- | | ||
| | [`mlab-collaboration.hermes.events_enriched`](#events_enriched) | Almost everything. Nested, clearly named, stable, with path summaries precomputed. | | ||
| | [`mlab-collaboration.hermes_union.events_with_as_and_geoloc`](#events_with_as_and_geoloc) | The raw statistical test outputs, and pipeline work. 75 flat columns, historical names. | |
There was a problem hiding this comment.
What's "historical names"?
|
|
||
| **Load balancing.** Consecutive probes may take different paths through a load-balanced network, so a single traceroute is one sample of a set of possible routes. | ||
|
|
||
| **A hop's RTT is not a link's latency.** RTT to a hop is a round trip that includes the return path from that hop, which may differ entirely from the path back from the destination. Rising RTT at a hop does not localize a problem to that hop by itself — which is precisely why HERMES combines path evidence across many measurements rather than reading a single traceroute. |
There was a problem hiding this comment.
| **A hop's RTT is not a link's latency.** RTT to a hop is a round trip that includes the return path from that hop, which may differ entirely from the path back from the destination. Rising RTT at a hop does not localize a problem to that hop by itself — which is precisely why HERMES combines path evidence across many measurements rather than reading a single traceroute. | |
| **A hop's RTT is not a link's latency.** RTT to a hop is a round trip that includes the return path from that hop, which may differ entirely from the path back from the destination. Rising RTT at a hop does not localize a problem to that hop by itself. |
| > Traceroute enrichment is **one stage** of the HERMES pipeline. It produces the annotated paths that HERMES uses as evidence when localizing a performance degradation; it is not the HERMES data product. For the product, see the [HERMES overview]({{ site.baseurl }}/tests/hermes/); for the stage's place in the pipeline, see the [methodology]({{ site.baseurl }}/tests/hermes/methodology/). | ||
|
|
||
| A raw traceroute hop is an IP address and some round-trip times. On its own that cannot answer "which network was this, and where was it?" — the questions localization depends on. This page describes how HERMES turns each hop into an annotated hop, which sources it uses, how it records where each annotation came from, and how it flags annotations that are probably wrong. | ||
|
|
There was a problem hiding this comment.
I suggesting writing this differently / in a way that highlights the generality of the traceroute enrichment pipeline. Here we describe the annotation. Indeed that is part of HERMES (and we should provide pointers and context), but it may confuse people if we presented as a part of another pipeline.
For example, sth along the lines:
- Start with the generic statement
A raw traceroute hop is an IP address and some round-trip times. On its own that cannot answer "which network was this, and where was it?" — the questions localization depends on. This page describes how the traceroute enrichment pipeline turns each hop into an annotated hop, which sources it uses, how it records where each annotation came from, and how it flags annotations that are probably wrong. - And then provide links to HERMES: (i) write that the pipeline is part of HERMES and (ii)
It produces the annotated paths that HERMES uses as evidence when localizing a performance degradationand (iii)For the HERMES system, see the [HERMES overview]({{ site.baseurl }}/tests/hermes/); for the stage's place in the pipeline, see the [methodology]({{ site.baseurl }}/tests/hermes/methodology/)
|
|
||
| ## The published data | ||
|
|
||
| HERMES publishes one row per NDT measurement that belongs to a monitored group _[confirm]_, carrying everything HERMES computed around that measurement: the group's baseline, how far the measurement sits from it, both directions of the annotated network path, and the metadata that says how much weight the path evidence deserves. |
There was a problem hiding this comment.
- remove confirm
- clarify/define (or add pointer) what measurement group is
|
|
||
| New to traceroutes? [Introduction to traceroutes]({{ site.baseurl }}/learn/traceroute/) explains what a traceroute measures, how to read one, and the ways it can mislead. | ||
|
|
||
| M-Lab traceroute data is used by [HERMES]({{ site.baseurl }}/tests/hermes/) to localize Internet performance degradations. |
There was a problem hiding this comment.
Add also a line with (text and) pointer to the annotated traceroute pipeline
Co-authored-by: Pavlos Sermpezis <sermpezis.pavlos@gmail.com>
Co-authored-by: Pavlos Sermpezis <sermpezis.pavlos@gmail.com>
Co-authored-by: Pavlos Sermpezis <sermpezis.pavlos@gmail.com>
HERMES has a blog post and a published paper, but there is not currently a good place on the M-Lab site to point someone who wants to understand what HERMES produces or how to query the data. This PR adds that missing piece.
This is still a content draft and is opened for review, not merge. Every new page has a draft banner, and I have listed the remaining open questions below.
Where it lives
I put HERMES under
/tests/, in a new Analysis Systems section, rather than under/data/.HERMES is not itself a test that M-Lab runs, but it is a system M-Lab operates. The existing site structure already follows this pattern for NDT: the main pages for the system live with the system, while
/data/points people to them.The new group is described as systems that “analyze the measurements collected above, rather than collecting new measurements of their own.” I think that captures HERMES reasonably well without suggesting that it is another measurement experiment.
How I am framing HERMES
One thing I wanted to be careful about is not reducing HERMES to traceroute enrichment.
At a high level, HERMES detects statistically significant performance degradations across groups of NDT measurements and then tries to localize them using network paths. Annotating traceroute hops with AS, geolocation, and interconnection information is one part of that pipeline. It is also the most visually obvious part, so it is easy for it to become the description of the whole system.
I therefore kept enrichment one level down, under the methodology section. The top-level pages focus instead on the full pipeline: detection, baselines, statistical tests, localization, and the resulting events. The enrichment page starts by explaining where that component fits into the broader system.
Pages
I put the traceroute explainer under
/learn/rather than HERMES because it should also be useful to readers ofscamper1, reverse traceroute, and IPRS./learn/already describes itself as moving toward this kind of material. I also linked it from the relevant traceroute test pages.The schema page treats
hermes.events_enrichedas the main interface users should query, withhermes_union.events_with_as_and_geolocdocumented as the operational table underneath it. I also included a complete mapping from the legacy field names to the canonical ones.The field names, types, and structure currently come from
create_events_enriched.sql,04_mapping_union.sql, and the pipeline DDLs inm-lab/hermes.Things I would especially like checked
@sermpezis:
/tests/feel like the right home for it?Things that still need to be resolved before merge
mlab-collaboration, and users with access tomeasurement-labcan already query them. This means the existing M-Lab BigQuery access model and QuickStart can be reused.INFORMATION_SCHEMA.COLUMNS. They are currently inferred from the pipeline DDLs and SQL rather than read directly from the live tables.events_enricheddoes not currently expose the statistical-test outputs. The Mann-Whitney, Welch’s t, and Wasserstein results are present in the operational table but are not projected into the published view. I documented the current workaround, but this means the stable interface is missing some of the fields that most directly describe HERMES’s statistical results.Preview
The
sandbox-*branch is accessible. Here is a rendered URL that should be much easier to review than the Markdown diff: http://website.mlab-sandbox.measurementlab.net/tests/hermes/.One unrelated issue surfaced while editing
_pages/05-tests.md: the grid uses Bootstrap’s floatedcol-*classes, so tiles with different heights leave whitespace holes. I have intentionally kept that fix out of this PR. It is a one-line stylesheet change onfix/grid-tile-float-whitespace, and keeping it separate should make this PR easier to review as a content change.This change is