get_destination_id

Looks up a source-to-destination ID mapping from the mapping store, so a pipeline can skip records that were already migrated.

When to use it

Use get_destination_id with skip_if and set_destination_id to build the manual dedup pattern. For new tasks, prefer sync_record — it combines the mapping lookup, hash-based change detection, create/update, and mapping persistence in one composite step.

Configuration

Example

name: migrate-accounts
output: none
connectors:
  source:
    type: csv
  destination:
    type: dynamics365
  mapping_db:
    type: sqlite
steps:
  - name: read_accounts
    type: call_method
    config:
      connector: source
      method: read
      args:
        resource: accounts
 
  - name: check_existing
    type: get_destination_id
    config: |
      {
        "connector": "mapping_db",
        "resource": "account",
        "source_id": input.account_id
      }
 
  - name: create_account
    type: call_method
    skip_if: "steps.check_existing.output.destination_id != null"
    config: |
      {
        "connector": "destination",
        "method": "create",
        "args": { "resource": "accounts", "data": input }
      }

Behavior notes

• The lookup returns destination_id and hash — both null when no mapping exists. Downstream steps read them as steps.<name>.output.destination_id and steps.<name>.output.hash.
• The input record is preserved. When the input is a plain object, the step emits the input merged with the lookup result (lookup fields take precedence), so downstream expressions still see the original source fields. A non-object input is replaced by the lookup result alone.
• The hash value supports change detection: compare it with a hash from compute_hash in a skip_if to update only changed records.

Related

• set_destination_id — the write half of the mapping pair
• sync_record — the preferred composite replacement
• Idempotent sync and the mapping DB — the dedup pattern end to end