Add/review Slack messaging for the new data refresh #4917
Labels
💻 aspect: code
Concerns the software code in the repository
🧰 goal: internal improvement
Improvement that benefits maintainers, not users
🟩 priority: low
Low priority and doesn't need to be rushed
🧱 stack: catalog
Related to the catalog and Airflow DAGs
Problem
The current data refresh, run on the ingestion server, reports somewhat noisily to Slack. The new data refresh will happen almost entirely in Airflow where it is much more observable than on the ingestion server, so it's not as important to have such extensive Slack messaging -- but given that it is such an important and long running DAG, we should still consider adding some messaging at critical steps.
Description
We should look at all the places the current data refresh sends Slack notifications to double check if there are additional messages we need, but I think we should at least have a Slack message when the copy data step finishes and when the distributed reindex finishes.
Additional context
#4833
The text was updated successfully, but these errors were encountered: