Skip to content

datwiz/dbt-snapshot-metadata

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

21 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Adding operational metadata to dbt snapshots

See the article at enhancing-dbt-snapshots-with-operational-metadata for a full description.

The dbt_snapshot_ops_metadata directory in this repository contains the dbt project that implement an extension to the dbt snapshot materialization macros to add operational metadata to snapshot tables.

Using the dbt snapshot materialization macros, the following operational metadata fields are added to snapshot tables:

  • insert_id - the process id of the job that inserted the record
  • update_id - the process id of the job that updated the record
  • start_dttm - the start date/time of the record
  • end_dttm - the end date/time of the record
  • insert_dttm - the date/time the record was inserted
  • update_dttm - the date/time the record was updated

As shown here:

erDiagram
  ENHANCED_THING {
    date txn_dt
    int id
    string status
    string insert_id
    string update_id
    timestamp start_dttm
    timestamp end_dttm
    timestamp insert_dttm
    timestamp update_dttm
    string dbt_scd_id
    datetime dbt_updated_at
    datetime dbt_valid_from
    datetime dbt_valid_to
  }
Loading

enhanced snapshot table

About

Example to inject additional operational metadata into dbt snapshots

Topics

Resources

Stars

Watchers

Forks

Contributors