> For the complete documentation index, see [llms.txt](https://atomic-blend.gitbook.io/mongo2pg/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://atomic-blend.gitbook.io/mongo2pg/reference/mapping-reference.md). # The mapping reference Every key the mapping YAML accepts, its default, and what it does. Two complete, runnable mappings live in `examples/auth.yaml` and `examples/productivity.yaml` in the repository if you want to see every key used together. Background on the concepts referenced below lives in [Concepts](/mongo2pg/concepts/concepts.md). ## Top level ```yaml apiVersion: mongo2pg/v1 serviceName: productivity mongoClusterName: mongo-prod-0 source: {...} target: {...} defaults: {...} cutover: {...} collections: {...} ``` | Key | Default | Meaning | | ------------------ | ------------ | ------------------------------------------------------------------------------------------------------------------------ | | `apiVersion` | — | Always `mongo2pg/v1`. | | `serviceName` | — (required) | Labels metrics as `service_name` and seeds the default PK namespace (`uuidv5(ROOT_NS, serviceName + "/" + collection)`). | | `mongoClusterName` | — | Labels metrics as `mongo_cluster` only — nothing connects to it by this name. | ## `source` | Key | Default | Meaning | | ---------------- | -------------- | ------------------------------------------------------------------------------------- | | `uri` | — (required) | MongoDB connection URI. Supports `${VAR}` interpolation from the process environment. | | `database` | — (required) | MongoDB database name. | | `readPreference` | driver default | e.g. `secondaryPreferred`. | | `batchSize` | `1000` | Documents per snapshot scan page. | ## `target` | Key | Default | Meaning | | -------------------- | ------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `dsn` | — (required) | PostgreSQL connection string (pgx/v5 pool DSN). Supports `${VAR}` interpolation. | | `schema` | `public` | Schema the generated target tables live in. | | `migrationSchema` | `_migration` | Schema for mongo2pg's own bookkeeping tables (`collection_state`, `checkpoint`, `id_map`, `pending_fk`, `quarantine`, `verify_runs`). | | `maxConns` | `8` | pgxpool max connections. | | `statementCacheMode` | `prepare` | `prepare` \| `describe`. Use `describe` behind a transaction-pooling proxy such as PgBouncer — a server-side prepared statement cached against one backend connection cannot be reused safely once the proxy hands out a different one. | ## `defaults` Applied to every collection unless a collection overrides the same key. | Key | Default | Meaning | | ------------------ | ----------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `pk.strategy` | — | The default primary-key strategy for every collection — see [Primary keys and identity](/mongo2pg/concepts/concepts/primary-keys.md). | | `pk.type` | derived from `strategy` | The default target PostgreSQL type. | | `softDelete.field` | — | The default soft-delete timestamp field name. | | `softDelete.mode` | `column` | The default soft-delete mode — see [Soft deletes and normalization](/mongo2pg/concepts/concepts/soft-deletes-and-normalization.md). | | `onCoerceError` | `quarantine` | The only accepted value. `null` and `halt` are rejected at validation time rather than silently ignored — see [Quarantine](/mongo2pg/concepts/concepts/quarantine.md). | | `keepLegacyId` | `false` | Keep the original `_id` in a `legacy_mongo_id` column on every table. Only settable under `defaults:` — there is no per-collection override. | ## `cutover` | Key | Default | Meaning | | ------------------------ | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `maxLagSeconds` | `30` | The `cutover check` lag gate. Rejected at validation if ≤ 10 (the change stream's periodic no-op interval — the gate could never open); warns below 25. See [Operational realities](/mongo2pg/operating-a-migration/operations/operational-realities.md). | | `requireEmptyQuarantine` | `true` | Whether `cutover check` requires zero open quarantine rows. | | `requireVerify` | — | Which of `count`, `checksum`, `sample` must pass, on live data, for `cutover check` to succeed. An unrecognized name is refused at startup rather than silently skipped. See [Verification and checks](/mongo2pg/operating-a-migration/operations/verification-and-checks.md). | ## `collections.` ```yaml collections: tasks: table: tasks pk: {...} softDelete: {...} # overrides defaults.softDelete strictFields: false fields: {...} indexes: [...] ``` | Key | Default | Meaning | | -------------- | ------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------ | | `table` | the collection's own name | Target PostgreSQL table name. | | `pk` | `defaults.pk` | See [Primary keys](#pk). | | `softDelete` | `defaults.softDelete` (only if `defaults.softDelete.field` is set) | Per-collection override — see [Soft deletes and normalization](/mongo2pg/concepts/concepts/soft-deletes-and-normalization.md). | | `strictFields` | `false` | Promote an undeclared BSON field to quarantine instead of only incrementing `mongo2pg_unmapped_field_total`. | | `fields` | — (required) | One entry per mapped field — see [Field keys](#field-keys). | | `indexes` | — | See [Indexes](#indexes). | ### `pk` ```yaml pk: from: _id strategy: uuid_v7 type: uuid namespace: # optional — see Primary keys uuidEncoding: java # only meaningful for BSON binary subtype 3 timestampFrom: [id, {field: created_at}] onMissingTimestamp: quarantine onError: quarantine ``` | Key | Default | Meaning | | -------------------- | ------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `from` | `_id` | The MongoDB field the PK is derived from. | | `strategy` | `defaults.pk.strategy` | One of `passthrough_text`, `passthrough_objectid`, `passthrough_uuid`, `uuid_v5`, `uuid_v7`, `uuid_v4`, `bigint`. See [Primary keys and identity](/mongo2pg/concepts/concepts/primary-keys.md). | | `type` | derived from `strategy` (e.g. `uuid`, `bigint`, `text`, `char(24)`) | Target PostgreSQL type. | | `namespace` | derived deterministically from `serviceName` + collection name | See [Namespace](/mongo2pg/concepts/concepts/primary-keys.md#namespace). Changing it after a first run is refused at startup. | | `uuidEncoding` | — | `java` \| `csharp` \| `python` \| `rfc4122`. Required only if the collection's `_id` holds BSON binary subtype 3; startup fails loudly otherwise. | | `timestampFrom` | `[id]` (the `_id`'s own timestamp, if it has one) | Ordered list `uuid_v7` consults for its timestamp: the bare scalar `id`, or `{field: }`. The first that yields a value wins. | | `onMissingTimestamp` | — | Accepted in YAML (`quarantine` \| `halt`) but not currently read by the mapper — a missing v7 timestamp always quarantines regardless of this value, matching the tool's fail-closed rule everywhere else. | | `onError` | `defaults.onCoerceError` | Must be `quarantine` if set. | | `coerce` | — | **Accepted in YAML but not consumed at runtime.** Canonicalizing an `_id` across BSON types (ObjectID vs. its 24-hex string form vs. a UUID) happens automatically and unconditionally — see [Recognising the `_id`](/mongo2pg/concepts/concepts/primary-keys.md) — so this key currently has no effect regardless of what is declared here. Do not rely on it. | ### Field keys Which `mode` a nested field wants is the one choice this reference cannot make for you — the tree below is the same decision [Type coercion § Nested data](/mongo2pg/concepts/concepts/coercion.md#nested-data) walks through in prose: ```mermaid flowchart TD F["a field in the document"] --> Q1{"scalar value?"} Q1 -- yes --> Q2{"is it a reference
to another collection?"} Q2 -- yes --> REF["ref: {collection: ...}
own column, FK relinked"] Q2 -- no --> COL["plain column
column + type, optional coerce"] Q1 -- "no — nested" --> Q3{"array or object?"} Q3 -- "array of scalars" --> ARR["mode: array
column + array type
ref relinks every element"] Q3 -- "array of documents,
queried or joined" --> CT["mode: child_table
its own table + fk + ord"] Q3 -- "fixed sub-object,
fields queried individually" --> FL["mode: flatten
prefix + columns on the parent table"] Q3 -- "unbounded shape,
or read as a unit" --> JS["mode: jsonb (default)
BSON extras keep Extended JSON form"] ``` | Key | Applies to | Meaning | | ------------ | ------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------- | | `column` | scalar | Target column name. | | `type` | scalar | Target PostgreSQL type. | | `coerce` | scalar | Ordered list of accepted BSON inputs — see [Type coercion](/mongo2pg/concepts/concepts/coercion.md). | | `ref` | scalar | This value is a foreign key — see [`ref` fields](#ref-fields). | | `mode` | nested | `jsonb` (default) \| `child_table` \| `array` \| `flatten` — see [Type coercion § Nested data](/mongo2pg/concepts/concepts/coercion.md#nested-data). | | `transform` | scalar | An ordered list from the six-entry allowlist — see [Type coercion](/mongo2pg/concepts/concepts/coercion.md). | | `opaque` | scalar | `true` marks ciphertext: copied byte-identically, coercion/transform/trim/case-fold all rejected at validation time. | | `notNull` | any | Column constraint; a `NULL` value here is a `FieldError` (quarantines the row). | | `default` | any | Value substituted when the source field is absent. | | `nullIf` | any | Value(s) that should be written as SQL `NULL` instead of their literal form. | | `onError` | scalar | Per-field override of `onCoerceError`. Must be `quarantine` if set. | | `skip: true` | any | Explicitly not migrated. Documents intent and silences the unmapped-field warning for that field. | | `index` | scalar/array | e.g. `gin` for an array column that needs a GIN index. | ### `ref` fields ```yaml folder_id: {column: folder_id, ref: {collection: folders, onMissing: 'null'}} user: {column: user_id, ref: {collection: users, external: {service: auth}}, notNull: true} ``` | Key | Default | Meaning | | ------------------- | ---------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `collection` | — (required) | The target collection this value's `_id` refers to. | | `onMissing` | writes `NULL` | `quarantine` parks the row when the source field is absent. Any other value (or leaving it unset) writes `NULL`. This is narrower than resolving an unreconcilable *present* value — see below. | | `external.service` | — | Marks this as a [cross-service reference](/mongo2pg/concepts/concepts/cross-service-references.md) into another mongo2pg-managed service's database. | | `external.strategy` | `uuid_v7` (from an ObjectID) | The target's PK strategy, since a cross-service config has no way to read it from the target directly. A declared non-derivable strategy (`uuid_v4`, `bigint`, field-sourced `uuid_v7`) is refused at startup. | A **present** reference value that cannot be reconciled against the target collection — no canonical form matches, or `bsonval.CanonicalizeID` itself errors (e.g. a BSON binary subtype 3 `_id` with no declared `uuidEncoding` on the target) — always quarantines the row; `onMissing` does not govern that case, only a genuinely absent source field. A present reference to a not-yet-migrated parent resolves automatically according to the target's PK strategy — minted immediately for `uuid_v4`/`bigint`, deferred via `_migration.pending_fk` for a field-sourced `uuid_v7` — see [Foreign-key relinking](/mongo2pg/concepts/concepts/foreign-keys.md). ### `mode: jsonb` (default) ```yaml headers: {mode: jsonb, column: headers} ``` No further keys beyond `column`. A nested BSON date/ObjectID/binary/decimal keeps its relaxed MongoDB Extended JSON form (`{"$date": "…"}`) rather than being flattened. ### `mode: child_table` ```yaml attendees: mode: child_table table: event_attendees fk: event_id onParentDelete: cascade # cascade | soft columns: email: {column: email, type: text, opaque: true} ``` | Key | Default | Meaning | | ---------------- | ------------ | ----------------------------------------------------------------------------------------------------------------------------------------- | | `table` | — (required) | Child table name. | | `fk` | — (required) | Foreign-key column name on the child table, pointing at the parent. | | `onParentDelete` | — | `cascade` \| `soft`. | | `columns` | — (required) | One `Field` entry per child-table column. | | `link` | — | Shorthand for a pure join table that only takes a referenced id (see below) — an alternative to `columns` for the denormalized-copy case. | The child table is rewritten as a set on every parent change: `DELETE ... WHERE = $1` followed by the new inserts, within the parent's own transaction. ```yaml tags: mode: child_table table: task_tags fk: task_id link: {from: _id, column: tag_id, ref: {collection: tags}} ``` `link.from`, `link.column`, `link.ref` describe a pure join row — take only the referenced id and relink it, rather than copying the referenced document's own fields. ### `mode: array` ```yaml role_ids: {mode: array, column: role_ids, type: 'uuid[]', ref: {collection: user_roles}} ``` `column` and `type` (an array type, e.g. `'timestamptz[]'`) as usual. A `ref` here relinks **every element**. ### `mode: flatten` ```yaml start: mode: flatten prefix: start_ columns: time: {column: at, type: timestamptz} # → start_at tzid: {column: tzid, type: text} # → start_tzid ``` | Key | Default | Meaning | | --------- | ------------ | --------------------------------------------------------- | | `prefix` | — | Prepended to every child column name on the parent table. | | `columns` | — (required) | One `Field` entry per flattened sub-value. | ### `indexes` ```yaml indexes: - {columns: [user_id, updated_at]} - {columns: [email], unique: true} - {columns: [user_id], where: "deleted_at IS NULL"} ``` | Key | Default | Meaning | | --------- | ------------ | ------------------------ | | `columns` | — (required) | Column list, in order. | | `unique` | `false` | Unique index. | | `where` | — | Partial-index predicate. | Every index is created `CONCURRENTLY`, as its own migration — a non-concurrent build takes a write lock on a table that may be actively replicating, so index migrations are split out on purpose rather than for tidiness. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://atomic-blend.gitbook.io/mongo2pg/reference/mapping-reference.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.