Methodology

How events enter the Observatory, what AI does and does not do, how sources are recorded, and the limits of what the Observatory covers.

Current status: automated discovery of curated events is in operation for two monitored sources on X — the curator’s bookmarks and White Box Autonomy’s own posts. Broader searching is not yet active. Every curated event is reviewed and published by a curator; nothing is published automatically. Source-derived records are added separately, from regulatory data (see “Record origin” below).

How curated events are found and published

Curated events pass through the following steps. The earlier steps are automated and decide only what deserves a curator’s attention; the decisions about what an event is, and whether it is published, are made by a person.

Monitored sources. Two monitored sources are currently in operation: posts the curator bookmarks on X, and White Box Autonomy’s own posts on X. Both are checked once a day, and each check looks only for material that is new since the previous one.
Automated screening. Each new post is screened, with AI assistance, for whether it plausibly describes a specific real-world automated-driving occurrence worth a closer look. Posts that do not are set aside.
Evidence and provenance. For a post worth a closer look, the system identifies the post that actually carries the evidence — for example, the original post of a video that someone else quoted or re-shared — and records where it came from: its publisher, publication date and when it was retrieved.
AI-assisted qualification. The evidence post must contain video. An AI-assisted check then assesses whether it is likely relevant. Posts that do not qualify go no further, and evidence that is already recorded is not added again.
Candidate. What remains becomes a candidate: a private item in the curator’s review queue. A candidate is not an Observatory record and is not public.
Curator review. A curator opens the original post, watches the video, checks whether the event is already recorded, and then decides. A candidate may be published as a new event, added as further evidence to an existing event, deferred, or rejected.
Publication. To publish, the curator writes the event’s title and summary in White Box Autonomy’s own neutral words and records its developer or operator, event type and outcome. Only then does the event appear in the Observatory, with the original post linked as its evidence.

What AI does — and does not do

AI assists the automated steps above: it helps screen new posts and assess whether a candidate is likely relevant. That is all it is used for. AI does not establish any event fact, does not write the Observatory’s event descriptions, does not make the curator’s judgement, and cannot publish anything. Its assessments decide only which items reach the curator’s queue; they are never evidence, and they are not shown as part of an event record.

Coverage and its limits

Automated discovery currently covers only the two monitored sources named above. Broader searching of X or the wider web is not yet active, so the Observatory does not reflect everything published about automated driving.
No claim of historical completeness is made, even for those sources. White Box Autonomy’s own posts are monitored only from when monitoring began, and earlier posts have not been retrieved.
Candidates are reviewed by a curator over time. Not every candidate has been reviewed yet, and a candidate that has not been reviewed is not a negative finding.
The Observatory is a collection of evidence, not a census or a representative sample (see “Selection bias” below).

Sources and evidence

Every event links to its original sources. Each source is listed with its type, its publisher where known, its publication date and the date it was retrieved. The sources are the evidence; the title and summary on an event page are White Box Autonomy’s own description of what they show, not evidence in themselves. We show only this metadata and a link, not copies of the source material. Please cite the original sources rather than our summary.

A source’s own claims are not treated as established facts. A summary that reports what a source says attributes it to that source.

What we record — and leave unrecorded

A curated event records the curator’s title and summary, the developer or operator, the event type and the outcome. Where the evidence shows something the summary does not already say, the curator may also list individual observed facts; these are optional. Other details — such as the date of the event, its location, the system version, or whether automation was engaged — are recorded only when the evidence supports them.

Anything not recorded is shown as not recorded. That is not a finding that the evidence lacks it, and it is never filled with an estimate. A post’s publication date, for example, is not assumed to be the date of the event.

Location precision

An event’s location is recorded only to the precision its sources actually support — a specific area, a city, a region, or only a country — and that precision is stated explicitly rather than assumed. On the interactive map, each point is a representative location for that recorded precision, not a claim about the exact incident location; several events may therefore share the same point when only a coarse location is known. Events with no recorded location do not appear on the map. The map is an exploratory display aid for spatial distribution, not a precise geospatial record.

Involvement is not causation

Recording that an automated vehicle was involved in an event is separate from claiming it caused the event. A cause is recorded only where a source supports it, and the record says when no cause has been recorded or established.

Verification status

Where a curator has reached one of the standings below, it is stated in words; any colour used with it elsewhere is a secondary cue only, and the status is understandable without it. A source-derived record may carry no verification status at all — see “Record origin” below for why that is a legitimate, not a missing, state.

Curator-reviewed: A curator has checked the record against its sources before publication. This is not an assertion that every detail is true.
Verified: A curator judges that the core facts are supported by strong sources.
Disputed: The sources conflict with each other on a material point.

Items that have not been through curator review — discovery candidates and community submissions — are not shown in the Observatory at all.

Record origin

Every event also carries an origin, separate from its verification status above: origin describes how the record was created, not a curator’s judgement about it. An event is either Curated — authored or individually reviewed by a curator before publication, including every event that comes through the discovery workflow above — or Source-derived. The two pathways are complementary, not competing: curated evidence and source-derived regulatory records each cover ground the other cannot. A source-derived record is created instead through a validated, deterministic transformation of identified regulatory source records — currently records derived from NHTSA’s Standing General Order incident reports, grouped using NHTSA’s own source-defined “Same Incident ID” where present. That pathway does not use AI. It does not necessarily involve incident-by-incident curator review: White Box Autonomy structures and classifies the source evidence but does not independently verify the underlying physical incident, and does not claim to have adjudicated NHTSA’s own incident grouping. A source-derived record may therefore carry no verification status above, and its record page says so rather than showing one that was never assigned.

Origin is not the same as a source’s type(shown next to each link in a record’s source list, e.g. “Regulatory record” or “X post”): source type describes one piece of supporting evidence, while origin describes how the WBA record itself came to exist.

Community submissions

Anyone can submit a public link to an event. Submissions go to a private curator queue and are never added to the Observatory automatically; an event is added only if a curator reviews the submission and enters it through the curator workflow.

Selection bias

A catalogue built from what people post, report and record is a convenience sample, weighted toward what gets noticed and shared. To reduce a failure-only skew, the Observatory deliberately includes the successful handling of difficult situations alongside failures, and the distribution of outcomes is shown rather than hidden.

WBA seeks to preserve eligible event-level observations from its defined evidence sources rather than construct a statistically representative sample of AV performance. The number of events associated with a developer, system, country, event type, outcome, or geographic location — including its distribution on the map — reflects the composition and reporting characteristics of the underlying evidence sources and should not be interpreted as a measure of relative safety or performance.

WBA provides a structured research index of publicly available evidence rather than an authoritative determination of what occurred. Users should consult the underlying source records when making incident-level interpretations or safety-related conclusions.

Open source

A public GitHub repository contains the source code of this website and of the regulatory-record pipeline, together with documentation of the Observatory’s methodology. It is updated periodically and does not yet include every internal tool, such as the discovery and curator-review workflow described above.

View White Box Autonomy on GitHub →

See About for the project’s purpose and scope.