In #6, I moved the webhook cred into the Secrets Manager API, and granted the default runtime service account read access to it:

Here's the SO post which made me realize that was necessary and the place in the GCP docs it references.

For a more fine-grained permission structure down the line, we could also make a dedicated service account for each function.

This seemed to add unnecessary complexity at this early stage of the project, but may be worth keeping in mind as we grow.

cc @sharkinsspatial @rabernat (No action needed now AFAICT, just keeping you both in the loop.)

This repo is my initial attempt at a Dataflow status monitoring and reporting approach as outlined in https://github.com/pangeo-forge/registrar/issues/47. There does not seem to be a consistently documented approach for providing external notifications of Dataflow job success or failure (most approaches describe using log sinks to identify and report failed jobs but don't provide a recommended strategy for successful jobs).

While the approach of Cloud Monitoring Alerts that I've taken here appears to work, I've noticed that job/status metric occasionally seems to not report successful jobs. This is my first time working with Cloud Monitoring and MQL so I'm unsure if this is related to an error in my table alignment logic or if this is an issue with my understanding of the metric itself.

@alxmrs If you have a moment, can you provide some feedback on if my approach is usable or if there is a recommended approach for monitoring Dataflow jobs and reporting their status to external applications (via a webhook). cc @cisaacstern