Skip to main content

Send alerts to Slack, email or PagerDuty

Slack, email and PagerDuty get alerts, not results: one short message when something needs a person, with a link to the place in the Platform where it is dealt with. They are your own channels, added beside your destinations on Integrations, Results out. Results themselves go to your MCS, Amazon S3, Grafana Cloud or a webhook.

The four alerts​

AlertSent whenIt says
A decision is waiting for youA live run holds actions for your approval"rain-fade-swap: 2 actions waiting for approval", one line per pass, the earliest deadline, and Open in Constellation to decide them on Passes
A pass is at riskA run's swap search expects the weather to cut more than the workflow's at risk share of a pass's dry volume"rain-fade-swap: 2 passes at risk", one line per pass (satellite, antenna, time and any backup), and Open in Constellation to Passes
A workflow run failedA run of a workflow that sends its results ends failed. A cancelled run, a test run and a run on the demo fleet alone send nothing"A run of pass-yield failed", a line saying the run stopped before it delivered results, the run's error after "Reason:" (up to 500 characters), and Open in Constellation to the workflow in Automations
A bridge stopped checking inNot sent yet. You can tick it today

A decision is waiting for you needs Act, part of Operator + Act; on a plan without it the choice says so, starts unticked, and nothing waits to be decided.

Each workflow alerts on its own, and once: a pass at risk or an action waiting is alerted the first time a run finds it, not again on every run. A failing workflow alerts once for the streak; after one of its runs succeeds, its next failure alerts again. Nothing is sent when the problem goes away, and passes on the demo fleet never alert.

Add one​

  1. On Integrations, Results out, choose Add connection, then Slack, Email or PagerDuty under Destinations.
  2. Fill in what it asks for, below. The alerts start ticked: all four for Slack and email, leaving out A decision is waiting for you on a plan without Act; for PagerDuty, A workflow run failed, since each alert pages someone. Change them as you like, keeping at least one.
  3. Choose Connect. Constellation sends a check, below, and the connection turns Live, or Needs attention with the reason when Slack, the mail service or PagerDuty refuses it. It is then a row in Connections, named for where it sends.
ChannelIt asks forWhere to find it
SlackWebhook URL, and a Channel name for your list if you want oneSee Get a Slack webhook URL. It starts https://hooks.slack.com/services/
EmailAddresses, up to 10, separated by commasThe address you signed in with is filled in. A mailing list counts as one address. Alerts come from Constellation Space <no-reply@constellation.space>: add it to your allowlist so they skip spam. Every email has a Manage alerts link to Results out. Email goes out through your deployment's own sender, which must be set up first; without it the check says "Email alerts are not set up in this deployment"
PagerDutyIntegration key, and a Service name for your list if you want oneSee Get a PagerDuty integration key. It is 32 letters and digits

The form checks each field before it is sent: a Slack URL on hooks.slack.com/services/, real email addresses, a 32 character key, and at least one alert ticked.

The Slack URL and the PagerDuty key are kept like passwords and never shown again. A channel's detail shows one as Set; choose Replace to paste a new one and Save.

Get a Slack webhook URL​

  1. Go to api.slack.com/apps, choose Create New App, then From scratch, name it (for example Constellation) and pick your workspace.
  2. Under Features, open Incoming Webhooks and turn Activate Incoming Webhooks on.
  3. Choose Add New Webhook to Workspace, pick the channel, and choose Allow.
  4. Copy the Webhook URL shown for that channel.

A workspace that limits app installs needs an admin to approve the app first. Slack's own guide, with screenshots: Sending messages using incoming webhooks.

Get a PagerDuty integration key​

  1. In PagerDuty, open Services, then the service that should be paged (or create one with its escalation policy).
  2. Open its Integrations tab and choose Add an integration.
  3. Pick Events API V2 and choose Add.
  4. Open the new integration and copy its Integration Key. This is the routing key.

PagerDuty's own guide, with screenshots: Services and integrations.

What each one sends​

When you choose Connect (before the channel is added: a refused check adds nothing and says why), and when Save changes its settings, Constellation sends a check:

  • Slack posts one message saying Constellation is connected to the channel.
  • Email sends each address one message saying who added it, so someone who did not expect it can ask for it to be removed.
  • PagerDuty sends a change event, which shows on the service's timeline and pages no one.

Send a test sends a real alert of the first kind the channel takes, its title starting "Test:", and then says Sent a sample alert. On PagerDuty that opens a real incident and pages your on-call; resolve it in PagerDuty.

Slack posts one message per alert and email sends one per alert to every address, each with the alert's title, detail and an Open in Constellation link. A Slack message for a failed run reads like this:

A run of pass-yield failed
The run stopped before it delivered results. Open the workflow to run it again.
• Reason: input_rows: propagator exploded
[Open in Constellation] https://platform.constellation.space/automations?view=builder&workflow=pass-yield

Email carries the title as its subject, the same lines in its body, and a Manage alerts link at the foot.

PagerDuty opens one incident per alert through the Events API v2: the title is its summary, the detail is in custom_details.detail, with the passes in items and any deadline in deadline, and Open in Constellation is in its links. Its severity is warning, or critical for actions within an hour of their deadline. Constellation never resolves an incident: it stays open until someone resolves it in PagerDuty. A failing workflow pages once, and pages again on its next failure after a run succeeds.

Where your plan does not include a channel, Connect is refused with 403 and {"error": "plan_required", "plan": "standard", "feature": "output:pagerduty"} (or "plan": "pro", "feature": "act" when A decision is waiting for you is ticked without Act), and the form says which plan it needs, with Compare plans. An account has up to 5 alert channels, apart from its 3 destinations. A workflow cannot send its results to one, and none counts as results going out in the setup checklist. Open a row to change where it sends and which alerts it sends, then Save; Send a test; Pause to stop its alerts until Resume; and Delete connection, which stops its alerts at once.

In the API​

Alert channels use the destination routes, with the sign-in.

On screenCall
ConnectPOST /destinations with "activate": true: type set to slack, email or pagerduty, and alerts. The create sends the check (one message) and stores the channel active only if it passes. A refused check stores nothing: 422 {"error": "check_failed", "code": "refused" or "unreachable", "message": ...}. A deployment that cannot run checks answers 503 {"error": "check_unavailable"}
ConnectionsGET /destinations; an alert channel comes back without its webhook URL or key
A check without savingPOST /destinations/check with the same body as the create sends the check and stores nothing: {"ok": true, "sent": true}, or {"ok": false, "sent": false, "error": "refused" or "unreachable", "message": ...}. Connect does not call it
Save, Pause, ResumePATCH /destinations/{id} with only what changed, POST /destinations/{id}/pause, POST /destinations/{id}/resume
Send a test, Delete connectionPOST /destinations/{id}/test, DELETE /destinations/{id}
{"type": "slack", "webhook_url": "https://hooks.slack.com/services/T0123/B0456/...", "channel": "#ops-alerts", "alerts": ["decision_waiting", "pass_at_risk", "run_failed", "bridge_quiet"]}
{"type": "email", "addresses": ["ops@example.com"], "alerts": ["decision_waiting", "pass_at_risk", "run_failed", "bridge_quiet"]}
{"type": "pagerduty", "routing_key": "<the 32 character integration key>", "service": "Ground operations", "alerts": ["run_failed", "bridge_quiet"]}

The API checks each request and refuses one it cannot take with 422 validation_error, naming the field:

  • alerts names at least one of decision_waiting, pass_at_risk, run_failed and bridge_quiet, each once.
  • webhook_url is https://hooks.slack.com/services/ followed by its three parts, letters and digits only.
  • addresses holds 1 to 10 addresses, each one address of at most 254 characters (no display names or lists in one entry), with no address twice in any case. Spaces around each are trimmed.
  • routing_key is 32 letters and digits.
  • channel and service are optional, at most 80 characters; left out or blank, they come back as null. Email takes no name.
  • No other field is taken.

A new channel comes back pending, with last_delivery_at and last_error null, and turns active once its check is accepted or error with last_error saying why. A sixth alert channel is refused with 409: {"error": "destination_limit_reached", "message": "An account can have at most 5 alert channels. Delete one to add another.", "max": 5}. Send a test answers 202 with no body; Delete connection answers 204.