Skip to content

Add HERMES documentation section (draft for review) - #891

Open
Burdantes wants to merge 4 commits into
mainfrom
sandbox-loqman-hermes-section
Open

Burdantes wants to merge 4 commits into
mainfrom
sandbox-loqman-hermes-section

Conversation

@Burdantes

@Burdantes Burdantes commented Sep 14, 2026

Copy link
Copy Markdown
Collaborator

HERMES has a blog post and a published paper, but there is not currently a good place on the M-Lab site to point someone who wants to understand what HERMES produces or how to query the data. This PR adds that missing piece.

This is still a content draft and is opened for review, not merge. Every new page has a draft banner, and I have listed the remaining open questions below.

Where it lives

I put HERMES under /tests/, in a new Analysis Systems section, rather than under /data/.

HERMES is not itself a test that M-Lab runs, but it is a system M-Lab operates. The existing site structure already follows this pattern for NDT: the main pages for the system live with the system, while /data/ points people to them.

The new group is described as systems that “analyze the measurements collected above, rather than collecting new measurements of their own.” I think that captures HERMES reasonably well without suggesting that it is another measurement experiment.

How I am framing HERMES

One thing I wanted to be careful about is not reducing HERMES to traceroute enrichment.

At a high level, HERMES detects statistically significant performance degradations across groups of NDT measurements and then tries to localize them using network paths. Annotating traceroute hops with AS, geolocation, and interconnection information is one part of that pipeline. It is also the most visually obvious part, so it is easy for it to become the description of the whole system.

I therefore kept enrichment one level down, under the methodology section. The top-level pages focus instead on the full pipeline: detection, baselines, statistical tests, localization, and the resulting events. The enrichment page starts by explaining where that component fits into the broader system.

Pages

Page Purpose
/tests/hermes/ What HERMES is, what questions it answers, and what it publishes
/tests/hermes/quickstart/ Access, query cost, and first queries
/tests/hermes/schema/ Both tables, field by field
/tests/hermes/examples/ Worked query examples
/tests/hermes/methodology/ The HERMES pipeline, stage by stage
/tests/hermes/methodology/path-enrichment/ The path-enrichment component
/learn/traceroute/ A general-audience traceroute explainer

I put the traceroute explainer under /learn/ rather than HERMES because it should also be useful to readers of scamper1, reverse traceroute, and IPRS. /learn/ already describes itself as moving toward this kind of material. I also linked it from the relevant traceroute test pages.

The schema page treats hermes.events_enriched as the main interface users should query, with hermes_union.events_with_as_and_geoloc documented as the operational table underneath it. I also included a complete mapping from the legacy field names to the canonical ones.

The field names, types, and structure currently come from create_events_enriched.sql, 04_mapping_union.sql, and the pipeline DDLs in m-lab/hermes.

Things I would especially like checked

@sermpezis:

  1. Does the overall HERMES/enrichment framing look right? And does Analysis Systems under /tests/ feel like the right home for it?
  2. Things that still need to be resolved before merge

    • Access. The HERMES tables live in mlab-collaboration, and users with access to measurement-lab can already query them. This means the existing M-Lab BigQuery access model and QuickStart can be reused.
    • The example queries still need to be tested against the live table. Before publication, each query should be run and its scanned-bytes estimate recorded.
    • Row grain and coverage window need confirmation. I currently describe a row as one NDT measurement belonging to a monitored group, based on the schema and row counts.
    • Field types should be checked against INFORMATION_SCHEMA.COLUMNS. They are currently inferred from the pipeline DDLs and SQL rather than read directly from the live tables.
    • events_enriched does not currently expose the statistical-test outputs. The Mann-Whitney, Welch’s t, and Wasserstein results are present in the operational table but are not projected into the published view. I documented the current workaround, but this means the stable interface is missing some of the fields that most directly describe HERMES’s statistical results.
    • The grid icon could be better. The current one is cropped from the HERMES logo. It works at 80 px, but the surrounding tiles use simple line art, so a dedicated icon would fit better.

    Preview

    The sandbox-* branch is accessible. Here is a rendered URL that should be much easier to review than the Markdown diff: http://website.mlab-sandbox.measurementlab.net/tests/hermes/.

    One unrelated issue surfaced while editing _pages/05-tests.md: the grid uses Bootstrap’s floated col-* classes, so tiles with different heights leave whitespace holes. I have intentionally kept that fix out of this PR. It is a one-line stylesheet change on fix/grid-tile-float-whitespace, and keeping it separate should make this PR easier to review as a content change.


    This change is Reviewable

HERMES has a blog post and a paper but no home on the site, so there is
nowhere to point someone who wants to query the data. This adds that section.

It sits under /tests/ rather than /data/, in a new "Analysis Systems" group:
HERMES is not a test M-Lab runs, but it is a system M-Lab operates, and the
site's existing pattern is that a system's pages live with the system while
/data/ links across to them. NDT works the same way.

The framing throughout keeps HERMES distinct from its traceroute-enrichment
stage. Enrichment produces the annotated paths HERMES uses as evidence; it is
one stage of the pipeline, reachable only through the methodology page, and
every enrichment page opens by saying so. Without that boundary the enrichment
tends to become the description of the whole product, which understates the
detection and localization work that is most of the published data.

Pages added:

  /tests/hermes/                                landing
  /tests/hermes/quickstart/                     access, cost, first queries
  /tests/hermes/schema/                         both tables, field by field
  /tests/hermes/examples/                       worked queries
  /tests/hermes/methodology/                    pipeline, stage by stage
  /tests/hermes/methodology/path-enrichment/    the enrichment component
  /learn/traceroute/                            general-audience explainer

The traceroute explainer is in /learn/ rather than under HERMES because it
serves scamper1, reverse traceroute, and IPRS readers equally, and /learn/
already says it is being restructured to hold exactly this kind of resource.

Field names, types and structure are read from create_events_enriched.sql and
04_mapping_union.sql in m-lab/hermes, and from the pipeline DDLs. Enrichment
data sources are as confirmed by the HERMES team.

Still a draft, and marked as such on every page. Open before publication:
  - access path is unresolved (mlab-collaboration grants vs. measurement-lab)
  - no example query has been run against the live table
  - row grain and coverage window need confirming
  - types need checking against INFORMATION_SCHEMA.COLUMNS
  - events_enriched does not expose the statistical test outputs

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0167k8uZh7uL3Ei33tgpBfAV
@Burdantes
Burdantes requested a review from sermpezis September 14, 2026 03:16
Comment thread _pages/learn-traceroute.md Outdated
Comment thread _pages/04-data.md
* HERMES identifies when groups of Internet users experience a statistically significant drop in performance, and reports the parts of the network most closely associated with that drop.
* HERMES analyzes M-Lab NDT, traceroute, and reverse traceroute data; it does not collect measurements of its own.
* More information is available in the [HERMES Methodology]({{ site.baseurl }}/tests/hermes/methodology/) and the [SIGCOMM 2026 paper](https://doi.org/10.1145/3789240.3829129){:target="_blank"}.
* [HERMES Data in BigQuery]({{ site.baseurl }}/tests/hermes/schema/) - [HERMES Access and QuickStart]({{ site.baseurl }}/tests/hermes/quickstart/)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of repeating links to resources, I suggest adding a link to the documentation e.g., to /tests/hermes/#start-here (so that we don’t have duplicate info in the website)

Comment thread _pages/datadocs.md Outdated
Comment thread _pages/datadocs.md
Comment on lines +38 to +41
* [Access and QuickStart]({{ site.baseurl }}/tests/hermes/quickstart/)
* [Table schema]({{ site.baseurl }}/tests/hermes/schema/)
* [Example queries and tutorials]({{ site.baseurl }}/tests/hermes/examples/)
* [Methodology]({{ site.baseurl }}/tests/hermes/methodology/)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To avoid having duplicate info in the website, I suggest removing these links and keep only the main link to Hermes

| Table | Use it for |
| --- | --- |
| [`mlab-collaboration.hermes.events_enriched`](#events_enriched) | Almost everything. Nested, clearly named, stable, with path summaries precomputed. |
| [`mlab-collaboration.hermes_union.events_with_as_and_geoloc`](#events_with_as_and_geoloc) | The raw statistical test outputs, and pipeline work. 75 flat columns, historical names. |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's "historical names"?

Comment thread _pages/tests/hermes/path-enrichment.md Outdated

**Load balancing.** Consecutive probes may take different paths through a load-balanced network, so a single traceroute is one sample of a set of possible routes.

**A hop's RTT is not a link's latency.** RTT to a hop is a round trip that includes the return path from that hop, which may differ entirely from the path back from the destination. Rising RTT at a hop does not localize a problem to that hop by itself — which is precisely why HERMES combines path evidence across many measurements rather than reading a single traceroute.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
**A hop's RTT is not a link's latency.** RTT to a hop is a round trip that includes the return path from that hop, which may differ entirely from the path back from the destination. Rising RTT at a hop does not localize a problem to that hop by itself — which is precisely why HERMES combines path evidence across many measurements rather than reading a single traceroute.
**A hop's RTT is not a link's latency.** RTT to a hop is a round trip that includes the return path from that hop, which may differ entirely from the path back from the destination. Rising RTT at a hop does not localize a problem to that hop by itself.

Comment on lines +12 to +15
> Traceroute enrichment is **one stage** of the HERMES pipeline. It produces the annotated paths that HERMES uses as evidence when localizing a performance degradation; it is not the HERMES data product. For the product, see the [HERMES overview]({{ site.baseurl }}/tests/hermes/); for the stage's place in the pipeline, see the [methodology]({{ site.baseurl }}/tests/hermes/methodology/).

A raw traceroute hop is an IP address and some round-trip times. On its own that cannot answer "which network was this, and where was it?" — the questions localization depends on. This page describes how HERMES turns each hop into an annotated hop, which sources it uses, how it records where each annotation came from, and how it flags annotations that are probably wrong.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggesting writing this differently / in a way that highlights the generality of the traceroute enrichment pipeline. Here we describe the annotation. Indeed that is part of HERMES (and we should provide pointers and context), but it may confuse people if we presented as a part of another pipeline.

For example, sth along the lines:

  • Start with the generic statement
    A raw traceroute hop is an IP address and some round-trip times. On its own that cannot answer "which network was this, and where was it?" — the questions localization depends on. This page describes how the traceroute enrichment pipeline turns each hop into an annotated hop, which sources it uses, how it records where each annotation came from, and how it flags annotations that are probably wrong.
  • And then provide links to HERMES: (i) write that the pipeline is part of HERMES and (ii)
    It produces the annotated paths that HERMES uses as evidence when localizing a performance degradation and (iii) For the HERMES system, see the [HERMES overview]({{ site.baseurl }}/tests/hermes/); for the stage's place in the pipeline, see the [methodology]({{ site.baseurl }}/tests/hermes/methodology/)


## The published data

HERMES publishes one row per NDT measurement that belongs to a monitored group _[confirm]_, carrying everything HERMES computed around that measurement: the group's baseline, how far the measurement sits from it, both directions of the annotated network path, and the metadata that says how much weight the path evidence deserves.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • remove confirm
  • clarify/define (or add pointer) what measurement group is


New to traceroutes? [Introduction to traceroutes]({{ site.baseurl }}/learn/traceroute/) explains what a traceroute measures, how to read one, and the ways it can mislead.

M-Lab traceroute data is used by [HERMES]({{ site.baseurl }}/tests/hermes/) to localize Internet performance degradations.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add also a line with (text and) pointer to the annotated traceroute pipeline

Burdantes and others added 3 commits September 23, 2026 10:05
Co-authored-by: Pavlos Sermpezis <sermpezis.pavlos@gmail.com>
Co-authored-by: Pavlos Sermpezis <sermpezis.pavlos@gmail.com>
Co-authored-by: Pavlos Sermpezis <sermpezis.pavlos@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants