Let’s be honest: the alerting experience for data incidents in dbt isn’t great. If your data teams use dbt Cloud, you most likely already know and experience the scenario below on a regular basis.
.jpg)
The primary issue is that dbt’s native alerting fails to provide critically needed information, and quickly causes alert fatigue even at the lowest incident frequencies.
Native dbt alerts do not:
To get into this information, you have to jump into dbt to sort through console logs–which is cumbersome and often annoying.
Because you’re lacking this information when you need it in Slack, you might see a notification on your phone but not be able to triage it until later—or there’s a chance you forget to dig into the issue later when you have system access. You also might not even realize in a subsequent run that something new requiring investigation has broken.
We’ve described the best-case scenario for dbt Cloud. dbt Core users only get what they build for themselves, or whatever alerting basics might be available through their orchestrator.
Large, mature teams have heavily invested in their data incident management process, and our Workstream team has spent months talking to hundreds of them to learn exactly what is needed for everyone to course correct.
Typically these larger teams have “enterprise” needs, and will deploy a few different technologies as part of a broader data incident management program.
To be clear, we champion Monte Carlo and support their leading data observability solution with an integration to Workstream. But the reality is that clients leverage their proprietary anomaly detection capabilities to complement the basics of dbt testing. For example, the same client most likely applies the dbt not_null and unique_values tests on their orders table, while simultaneously finding value in Monte Carlo volume monitors for anomalous fluctuations in record volume.
What this means is that even those leveraging data observability solutions still have the same, basic unmet needs for more sophisticated dbt alerting. And moreover, they are likely frustrated with attempting to plug dbt failures and Monte Carlo incidents into a tool like PagerDuty – which was not built for data teams, and therefore does not meet their alerting needs.
“Data teams have been handcuffed by bad incident alerting for too long. I’ve personally experienced the pain of missing a major outage because I assumed I knew what I would see after clicking ‘Open run in dbt Cloud.’ When scaling a Data team in the past, we had to design processes to ensure someone on the team clicked that button and diligently researched the failures and understood the fallout. Teams just need to be able to increase failure visibility while also maintaining flexibility as their team and investments in data quality grow and mature.” - Ted Conbeer, Data Leader at Workstream.io
What is the sum of all of this? dbt provides most teams everything they already need to implement badass testing; it just falls consequently short for most teams when it comes to alerting and everything that comes after.
Today, we're thrilled to announce two major updates in Workstream.io.
1. Seamless Integration with Slack: enabling sophisticated alerting in shared channels for all dbt incidents.
2. Support for dbt Core: In addition to supporting dbt Cloud users, we're extending our services to cater to dbt Core users, ensuring that all dbt users will benefit from our advanced alerting capabilities.
Now, any dbt client can enjoy, at no cost, sophisticated alerting in Slack. Here is what you can expect:
Going forward, understanding your dbt incidents will be straightforward, transparent and fast (and free).
We believe that the best avenue for building trust is by providing value before we ask for anything in return, and so we welcome your data team to come and give our free Slack alerts a try. To get going in the next 5 minutes, just follow the onboarding instructions available here.
If you like what you see, it is and will always be free for you to use – that is our promise.
The potential impact for your team is massive, so we want you to get started as soon as possible. For the month of March only, we are offering $100 Amazon gift cards to anyone who sets up a free Workstream account, connects their dbt instance, and set-ups up the Slack connection to receive alerts. Just create a free account, and we will follow up with you right away.
Ready to revolutionize your dbt incident alerting? Activate Workstream.io Core now and take advantage of our advanced alerting capabilities. Connect dbt and Slack effortlessly to start receiving real-time alerts.
You can also join our Community Office Hours every Friday for live activation assistance and Q&A.
With Workstream.io Core, say goodbye to cumbersome incident triaging and hello to seamless, efficient alerting for your dbt workflows.
In this article, we discuss how data-driven teams can tame entropy and empower stakeholders with a single access plane for analytics.
Data asset entropy has been an issue for as long as data has been used in decision making. But in the modern organization, where analytics tools have exploded in number and type, entropy has become a huge stumbling block to effective data-driven operations.
The goal of your workflow is not just to uncover valuable wisdom from your data. It is crucial to communicate those actionable data observations to the organization which can use them to make decisions.
Receive regular updates about Workstream, and on our research into the past, present and future of how teams make decisions.
