Skip to content

diqu

Documentation PyPI version License: Apache 2.0 python-cli python codecov

Automate and streamline the alerting/ notification process for dbt test results using this versatile CLI companion tool. Receive detailed alerts & test metadata seamlessly on various platforms, promoting improved collaboration on dbt project issues 🐞🚀.

Who is this for

This tool is designed for individuals or teams seeking to automate the management of their dbt project issues (test warnings, errors... etc) outside the dbt environment.

Features

  • Automated alerts and notifications based on recorded dbt test results.
  • Built-in support for dq-tools and custom query input.
  • Auto-labels deprecated tests for quick & easy identification.
  • Sends succinct and informative messages to a dedicated Slack channel.
  • Creates and updates Jira tickets with the latest tests' metadata.

Supported Modules

  • Sources (DWH connections)
    • Snowflake
    • CSV file
  • Package (parsing dbt test results)
    • dq-tools
    • Custom query
  • Alert Modules (alert/ notification)
    • Jira
    • Slack

Concept ⭐

dbt alert rant 🟢 🟡 🔴 ⚫ - by diqu OG Contributors

Run errors are red,

Test warnings are yellow,

Where's my bug alert,

And joint bugfix workflow?

diqu Concept

We made a cool thing called diqu (pronounced 'deekoo'), a CLI tool to make bugfix lives a lil bit easier. Its goal is simple: streamlining collaboration and enhancing agility in our daily (if not hourly) bugfix with dbt.

Let's face it, the dbt result log is not built for alerting or team bugfix collaboration. Firstly, test warnings exist only in dbt's log, there's no way to get alerted on new warnings every day (or worse, every hour, depending on your ETL schedules) unless you open the log. Secondly, scrolling through a thousand-line log with the whole team to decide who gonna do what is, well, not a smart idea. diqu solves these by simply shipping all the test warnings/ errors, along with their metadata (e.g. latest failure, previous statuses ...) to other platforms (e.g. Slack, Jira) that support better alerting & collaborations.

diqu reads your test results table (provided by dbt packages that parse result log, such as dq-tools), transform it into simple yet insightful bug metadata, and send it to your output of choices. The output platforms are modularized, which enables contributors to improve & add more modules if needed.

Basic Usage

diqu alert --to slack --to jira

04:33:17  diqu: INFO - Run with diqu==1.0.0 🏃
04:33:19  diqu: INFO - Using dbt project at: /path/to/dbt/project
04:33:19  diqu: INFO - Using dbt profiles.yml at: ~/.dbt
04:33:19  diqu: INFO - Using snowflake connection
04:33:19  diqu: INFO - Looking for the query in: ./query.sql
04:33:23  diqu: INFO - Alerting to: SLACK
04:33:23  diqu: INFO - ✅ Done > Slack
04:33:23  diqu: INFO - Alerting to: JIRA
04:33:23  diqu: INFO - ✅ Done > JIRA

📖 For more details, please jump to the User Guide page or the Quick Start page.

How to Contribute

This Auto Alert (diqu) tool is an open-source software. Whether you are a seasoned open-source contributor or a first-time committer, we welcome and encourage you to contribute code, documentation, ideas, or problem statements to this project.

👉 See CONTRIBUTING guideline for more details

🌟 And then, kudos to our beloved Contributors:

About Infinite Lambda

Infinite Lambda is a cloud and data consultancy. We build strategies, help organisations implement them and pass on the expertise to look after the infrastructure.

We are an Elite Snowflake Partner, a Platinum dbt Partner and two-times Fivetran Innovation Partner of the Year for EMEA.

Naturally, we love exploring innovative solutions and sharing knowledge, so go ahead and:

🔧 Take a look around our Git

✏️ Browse our tech blog

We are also chatty, so:

👀 Follow us on LinkedIn

👋🏼 Or just get in touch

About IL