Skip to content

z3z1ma/dbt-osmosis

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1,045 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

dbt-osmosis

PyPI Downloads License: Apache 2.0 Streamlit App

dbt-osmosis is a Python CLI and package for dbt development workflows.

It centers on four primary surfaces:

  • schema YAML management (yaml organize, yaml document, yaml refactor)
  • column-level documentation inheritance across dbt lineage
  • ad-hoc SQL compile/run helpers
  • an optional Streamlit workbench for interactive dbt SQL development

The repository also ships additional command families for generation, natural-language helpers, schema diffing, SQL linting, and test suggestions.

The Docusaurus site is the canonical reference for the current CLI, configuration model, support matrix, and workflow guides:

dbt-osmosis

Supported runtime

dbt-osmosis currently targets:

  • Python 3.10-3.13
  • dbt Core 1.8+ package resolution
  • a dbt adapter version compatible with the dbt Core runtime in that environment

Audited blocking support covers dbt Core 1.8.x through 1.11.x in CI. The package metadata intentionally remains dbt-core>=1.8 without an upper bound so installers can resolve newer dbt releases. Future dbt Core minors are canary-only until explicitly audited; scheduled/manual canary CI uses unpinned latest dbt-core and dbt-duckdb to make upstream breakage visible without redefining audited support. Install a dbt adapter version that is compatible with the dbt Core runtime in your environment; adapter compatibility is owned by the adapter and dbt Core pairing, not by dbt-osmosis extras.

Optional extras:

  • dbt-osmosis[workbench] for the Streamlit workbench and related UI dependencies
  • dbt-osmosis[duckdb] for the DuckDB adapter used by the demo project and local fixture workflows
  • dbt-osmosis[openai] for LLM-assisted synthesis and natural-language generation features
  • dbt-osmosis[azure] for Azure AD authentication used with Azure OpenAI
  • dbt-osmosis[proxy] only installs dependencies for the experimental opt-in SQL proxy runtime; it does not expand the supported product surface, start a proxy server, configure authentication, TLS, or listen/bind settings, or make comment middleware durable. The proxy module entrypoint is a local-only experiment with mysql-mimic defaults, not a hardened user-facing server; do not expose it to untrusted networks. The proxy comment middleware is in-memory only, and ticket:c10proxy25 owns proxy support semantics.

Install

With uv:

uv tool install --with="dbt-<adapter>" dbt-osmosis

With pip:

pip install "dbt-osmosis" "dbt-<adapter>"

Replace <adapter> with your dbt adapter package, for example duckdb, snowflake, bigquery, postgres, or redshift.

Quick start

  1. Configure YAML routing in dbt_project.yml:
models:
  your_project_name:
    +dbt-osmosis: "_{model}.yml"
  1. Optionally set per-folder behavior with +dbt-osmosis-options and a repo-level YAML formatter in dbt-osmosis.yml:
formatter: "prettier --write"
  1. Preview changes safely:
dbt-osmosis yaml refactor --dry-run --check
  1. Apply the update once the diff looks right:
dbt-osmosis yaml refactor --auto-apply

CLI surface

Top-level commands currently exposed by dbt-osmosis --help:

  • yaml — manage schema YAML files and documentation inheritance
  • sql — compile or run ad-hoc SQL in dbt context
  • workbench — launch the Streamlit workbench
  • generate — generate sources, staging models, models, and SQL
  • nl — natural-language query/model helpers
  • test — suggest dbt tests
  • test-llm — validate LLM client configuration
  • diff — report schema drift between YAML and the database
  • lint — lint SQL strings, models, or a whole project

For command-by-command flags and examples, use the docs-site CLI reference rather than relying on this landing page.

Developer tooling

Local development in this repository is built around uv, task, and Ruff.

Common workflows:

task format
task lint
task test

Notes:

  • Ruff is the active formatter, linter, and import sorter.
  • task is not just verification; the default task formats, lints, runs tests, and then ensures the dev environment is synced.
  • Repository test fixtures are DuckDB-only today; contributor examples use demo_duckdb, and targeted core tests may need uv run dbt parse --project-dir demo_duckdb --profiles-dir demo_duckdb -t test to refresh demo_duckdb/target/manifest.json.
  • Docs-site commands use the Node toolchain under docs/:
npm --prefix docs run start
npm --prefix docs run build
npm --prefix docs run serve

Workbench

The optional workbench is a Streamlit app for interactive dbt SQL development.

Install the extra and launch it with:

pip install "dbt-osmosis[workbench]" "dbt-<adapter>"
dbt-osmosis workbench

The hosted demo is linked from the badge at the top of this README.

Pre-commit hook

You can run dbt-osmosis yaml refactor -C as a pre-commit hook:

repos:
  - repo: https://github.com/z3z1ma/dbt-osmosis
    rev: v1.3.0
    hooks:
      - id: dbt-osmosis
        files: ^models/
        args: [--target=prod]
        additional_dependencies: [dbt-<adapter>]

That hook keeps schema YAML changes visible in the commit that introduced them.

About

Provides automated YAML management and a streamlit workbench. Designed to optimize dev workflows.

Topics

Resources

License

Stars

Watchers

Forks

Contributors