# Units, tiers and decision records

Shared by all four skills. Contents: [attribution](#attribution) · [data-unit schema](#data-unit-schema) · [citation-identity fields](#citation-identity-fields) · [five-tier taxonomy](#five-tier-linkage-taxonomy) · [decision record](#decision-record-schema) · [validation rule](#validation-rule) · [composite ten steps](#oecdjrc-composite-ten-steps) · [concept IDs](#sdmx-concept-ids-used-by-the-predicates) · [coherence vs comparability](#coherence-vs-comparability)

## Attribution

Stated once here and inherited by all four skills rather than repeated in each.

The statistics are produced, owned and licensed by the publishing agencies — ABS, OECD, Eurostat, ILO, BIS, national statistical offices and others. Aard is the access infrastructure; it owns none of the data. Every data unit carries `agency`, the producing agency, and that field is the attribution anchor for all downstream output. Never describe the statistics as Aard's, and never attribute a series to the access layer.

Licence terms attach to the producing agency, not to the access path, so the obligations in `references/licensing.md` apply regardless of how the data was retrieved.

## Data-unit schema

A data unit is one dataflow plus one partially specified key returning a coherent series or slice.

| Field | Meaning |
|---|---|
| `dataflow_id` | SDMX dataflow identifier |
| `agency` | producing agency — the attribution anchor |
| `version` | artefact version; never leave unset, an unversioned reference silently re-points |
| `concept_supplied` | what this unit contributes to the operation |
| `role` | numerator / denominator / aggregate / component / part / whole / series |
| `freq`, `unit_measure`, `unit_mult`, `price_base`, `adjustment`, `reference_period`, `ref_area` | the compatibility attributes |
| `metadata_url` | the unit's own reference metadata (ESMS/SIMS page, methodology note), taken from what Aard returns for the dataflow — see below |
| `metadata_absent` | true only where the **producer publishes no** reference metadata; a finding about the source |
| `metadata_unavailable` | true where the producer may publish reference metadata but **none reached you**; a finding about the access path, not the source |
| `query_url` | the REST data query that returns these observations — the reproducible artifact |
| `extracted_at` | extraction timestamp, distinct from `reference_period` |
| `resource_title` | the provider-authored title of the resource, verbatim; null **only** where `resource_title_status=unavailable` |
| `resource_kind` | `resource` or `dataflow` — which kind of artefact supplied the formal title |
| `resource_title_status` | `verified` or `unavailable` |
| `selected_members` | every selected dimension, in DSD order — see below |
| `selection_labels` | the ordered provider-authored identity labels projected from `selected_members` |
| `selection_basis` | `whole_resource` or `subset` |
| `selection_status` | `verified`, `not_applicable` or `unavailable` |
| `response_format` | the transport representation (CSV, SDMX-CSV, JSON), kept separate from every name |
| `licence_label` | the verified per-product licence display label |
| `licence_url` | the verified provider/product terms URL |
| `notes` | ordered metadata-derived interpretation clauses |

These names are the **one canonical vocabulary**. Every skill that produces, carries or consumes a unit uses them unchanged; a renderer that has to guess at a synonym is a renderer that will eventually guess wrong.

`metadata_url` has two consumers and one definition. `reasoning-linkage-confidence` reads it as evidence for a tier predicate; `citing-official-statistics` cites it as an instance-level source. Do not duplicate the field per consumer.

**Populating it.** Take the reference-metadata link from what the Aard MCP server returns for the dataflow. Never infer one from a URL pattern and never construct one by hand — an invented link is a false citation, which is the worse failure for a skill whose purpose is verifiability. Resolving these by hand is not even one operation: providers expose reference metadata incompatibly, Eurostat as an `ESMS_HTML` annotation carrying a URL, ABS as an `EXT_RESOURCE` annotation with URLs pipe-delimited inside the annotation text, the OECD not at all through its SDMX API. An agent that guesses is right for one provider and fabricating for the others.

**When no link comes back, the two flags are not interchangeable — and this is the distinction that keeps the validation rule honest.** Set `metadata_absent=true` only when you have positive grounds that the producer publishes none, as with the OECD through its SDMX API. Otherwise set `metadata_unavailable=true`, which says the evidence was not obtained, not that it does not exist.

The difference matters because they carry opposite weight. `metadata_absent` is a finding about the source and can stand in for an instance citation, since there is nothing to cite. `metadata_unavailable` is a gap in your own evidence and **cannot** — a Tier-2-or-above verdict resting on it is unvalidated, and must be reported as such rather than presented as a completed application of these skills. Collapsing the two would let every verdict pass validation whenever the access path is simply not returning links, which is the failure this contract exists to prevent.

## Citation-identity fields

The fields above exist so the citation skill never has to reconstruct — or invent — a name at emission time. They are source-native inputs, carried forward, not text composed downstream.

**`selected_members` is an ordered list, one record per selected dimension, in DSD order:**

| Key | Meaning |
|---|---|
| `dimension_id` | the DSD dimension id, e.g. `SECTOR` |
| `dimension_label` | the provider's label for that dimension, e.g. `Sector` |
| `code` | the selected code, e.g. `3000999` |
| `member_label` | the provider's label for that code, e.g. `General Government Total` |
| `citation_role` | `identity`, `interpretation` or `neutral_scope` |
| `role_evidence` | what justified the role — see the grammar below |

**`role_evidence` is a prefixed string, not free prose.** The prefix is what makes
the claim checkable; a sentence describing the evidence is not the evidence:

| Value | Justifies | Meaning |
|---|---|---|
| `default_identity` | `identity` only | no metadata justified another role, so the member stayed identity |
| `concept:<ID>` | `interpretation` | the dimension binds to that cross-domain concept — `concept:FREQ`, `concept:UNIT_MEASURE`, `concept:ADJUSTMENT`, `concept:PRICE_BASE` |
| `codelist:<ID>` | `neutral_scope` | that codelist's own aggregate/default member — `codelist:CL_MATURITY` |
| `provider_aggregate:<ID>` | `neutral_scope` | provider metadata identifies the selected code as the aggregate — `provider_aggregate:CL_SEX.SEX_T` |

Any other value on `identity` is accepted and recorded. On the other two roles it
is rejected: a bare prefix naming nothing (`concept:`) is rejected too, and
`default_identity` never justifies `interpretation` or `neutral_scope`.

**Where the labels come from.** The provider's labelled response, its DSD and codelists, or other source-native metadata. Never the user's wording, never a summary you wrote. A label you composed is not the provider's label even when it is a fair description.

**Missing metadata is a typed state, not an empty string.** Two status fields carry it, and neither is allowed to be faked with `""` or `[]`:

- `resource_title_status=unavailable` — the unit keeps its codes, triplet and identifiers, and **fails Profile 4 publication validation**. It is still a valid unit for construction and arithmetic; it is not publishable as a named citation.
- `selection_status=not_applicable` — valid **only** when `selection_basis=whole_resource`. A `subset` requires verified, non-empty selected members and labels.
- `selection_status=unavailable` — fails publication validation on the same terms as an unavailable title.

Where the provider publishes a formal title for a series *inside* a larger resource, store that title as the subset's single verified `selection_labels` entry. It is a selection with one label, not a reason to bypass selection.

**`selection_labels` may be empty only when `selection_status=not_applicable`** and the basis permits that status.

**`notes` carries the `interpretation` projection only** — plus unit, multiplier, frequency, adjustment, price basis and other interpretation-critical attributes. Verified `neutral_scope` members stay in `selected_members` and in the bounded query, and are deliberately absent from the human note. That asymmetry is the whole point of the roles: provenance is complete, the reader's line is not padded with aggregate defaults.

## Five-tier linkage taxonomy

| Tier | Condition (metadata-only) | Mandated action |
|---|---|---|
| **1 SAFE** | same agency and dataflow id; keys differ only in the compared dimension(s); identical `UNIT_MEASURE` / `PRICE_BASE` / `ADJUSTMENT` / reference period | compute silently — except chain-linked additivity, which switches to previous-year-price or contribution keys |
| **2 PROBABLY SAFE** | same producer, different dataflows; shared `CL_AREA` / `CL_FREQ`; no methodological divergence flagged in ESMS/SIMS | compute and state the assumption |
| **3 DEFENSIBLE** | different producers; shared harmonised standard (SNA/ESA, BPM6, COICOP, ISIC, a global DSD) | compute with an explicit caveat covering vintage, benchmark year, derogations, revision timing |
| **4 ESCALATE** | different producers, divergent methodology, nominally the same concept — the clerical-review region | do not compute silently; use the four-slot escalation template |
| **5 REFUSE** | no shared dimensionality supporting the operation; no valid conversion bridge; category error | refuse, explain, cite the blocking rule, offer an alternative |

**Gradation.** The tier is a floor, not a point estimate. Within tiers 2–4, caveat strength scales with the count and severity of divergence axes: unit, price base, adjustment, vintage, population concept, reference period, revision status. A borderline 3/4 case is treated as 4.

Tiers 4 and 5 still emit numbers or series names to the user, so `citing-official-statistics` fires on them. An escalation that names two candidate series without citing them is incomplete.

## Decision record schema

Every combination verdict and every guarded computation emits:

```yaml
decision_record:
  operation: ratio | share | per_capita | growth | contribution |
             correlation | ranking | comparison | composite
  units:
    - dataflow_id, agency, version, role, metadata_url, metadata_absent,
      metadata_unavailable, query_url, extracted_at,
      resource_title, resource_kind, resource_title_status,
      selected_members, selection_labels, selection_basis, selection_status,
      response_format, licence_label, licence_url, notes
  tier: 1-5                      # if reasoning-linkage-confidence ran
  rules_fired: [a, c, ...]       # if guarding-derived-indicators ran
  predicates:                    # the metadata facts the decision rests on
    same_agency: true|false
    same_dataflow: true|false
    shared_codelists: [CL_AREA, ...]
    price_base: {unit_1: L, unit_2: V}
    adjustment: {...}
    unit_mult: {...}
    metadata_break_flagged: true|false|null
  citations:
    - scope: rule                # authority citation: why the decision is correct
      applies_to: tier-4 | guard-a
      authority: title
      locator: chapter/section/concept id
      url: DOI or URL            # from references/source-registry.md
    - scope: instance            # data citation: which observations, from whom, when
      applies_to: unit_1
      rendered_by: citing-official-statistics
```

The two citation scopes answer different questions and never substitute. Rule-level says why the operation was legitimate; instance-level says which numbers were used. A record carrying only rule-level citations names a methodology manual while leaving the observations unidentified.

Instance-level entries are **built by `citing-official-statistics`**, not composed by the skill emitting the record. Supply it the unit set with `metadata_url`, `query_url`, `version`, `extracted_at`, `resource_title`, `resource_kind`, `resource_title_status`, `selected_members`, `selection_basis`, `selection_labels`, `selection_status`, `response_format`, `licence_label`, `licence_url` and `notes` populated; it applies identifier precedence, the display projection and licence wording.

That handoff list is required, not indicative. A field that arrives missing is a field the citation skill must resolve from provider metadata or record as `unavailable` — it is never one it may compose.

## Validation rule

An auditing agent can check this mechanically. A decision is **valid** if and only if:

1. Every fired rule and every assigned tier carries at least one rule-level citation whose URL resolves and whose locator names the governing clause.
2. Every unit carries producing-agency attribution.
3. Every Tier-2-or-above verdict carries an instance-level citation, or `metadata_absent=true` is set explicitly. `metadata_unavailable=true` does **not** satisfy this leg — it records that the evidence was never obtained, so a verdict resting on it is unvalidated by definition.
4. **Every observation surfaced to the user resolves to an identifier and an extraction timestamp.** Legs 1–3 can all pass while the numbers themselves remain untraceable — this leg closes that gap, and applies to Tier-4 escalations and Tier-5 refusals as much as to computed results.
5. **Every observation published externally carries a provider-authored title or an explicit unavailable state, enough labelled selection metadata to distinguish the slice, and a confirmed licence.** `resource_title_status=unavailable` or `selection_status=unavailable` is a valid, honest state — it just is not publishable as a named citation, and a codes-only citation must be emitted instead. What fails this leg is a title or a selection label with no source-native provenance behind it.

A decision failing any leg is unvalidated and must not be presented as a completed application of these skills.

**How much of this is actually mechanical.** Legs 1–4 are: an auditor can resolve a URL, check a field is populated and compare two values. Leg 5 is only *partly* mechanical, and the difference matters more than a checklist suggests. An auditor can establish that the states are internally consistent — a `subset` carrying verified members and labels, a licence label paired with a terms URL, no `not_applicable` selection on a subset, roles carrying named evidence. What no auditor can establish from the record alone is that `resource_title` is the string the provider actually publishes, or that `licence_label` is the licence that product actually carries: a record populated with invented strings is structurally identical to one populated from provider metadata. That check requires going back to the provider. Leg 5 is therefore an obligation on the agent populating the record, enforced downstream only in shape — which is why the availability states exist, and why recording `unavailable` is always preferable to a plausible guess.

## OECD/JRC composite ten steps

1 theoretical framework; 2 variable selection; 3 multivariate and exploratory analysis; 4 imputation of missing data; 5 normalisation; 6 weighting and aggregation; 7 uncertainty and sensitivity analysis; 8 back to the data (deconstruction); 9 links to other indicators; 10 visualisation and presentation.

Step 7 is mandatory, not optional. Step 10 straddles the construction/emission boundary: the presentation requirements it implies are governed by `citing-official-statistics` and the relevant document profile.

## SDMX concept IDs used by the predicates

`UNIT_MEASURE`, `UNIT_MULT`, `PRICE_BASE` (a.k.a. `PRICES`), `ADJUSTMENT` (a.k.a. `SEASONAL_ADJUST`), `OBS_STATUS` (`CL_OBS_STATUS`), `CONF_STATUS` (`CL_CONF_STATUS`), `REF_AREA` (`CL_AREA`), `FREQ` (`CL_FREQ`), `TRANSFORMATION`.

**Match on codelist identity, not literal spelling.** Agencies vary the concept id while binding to equivalent cross-domain codelists; a predicate matching on the label will mis-fire on non-OECD providers.

`references/flags.md` is the single owner of publication-symbol semantics for all four skills. `OBS_STATUS` and `CONF_STATUS` are deliberately not tabulated there — the DSD names their code list and providers use their own, so they are resolved at runtime; `flags.md` documents how.

## Coherence vs comparability

From the SDMX Glossary and Metadata Common Vocabulary. **Comparability** refers to comparisons between statistics based on usually unrelated populations. **Coherence** refers to comparisons between statistics for the same or largely similar populations.

Comparability breaks surface through ESMS concepts 15.1 (comparability — geographical) and 15.2 (comparability — over time). Coherence surfaces through the coherence concepts (cross-domain, sub-annual vs annual, internal).