Monitoring and Alerting
Superblocks sends logs to your preferred observability platform so you can easily monitor your internal tools.
For example, in Datadog, visualize the execution of Workflows or Scheduled Jobs broken out by integration and drill into each step for detailed metadata. Configure alerts on error logs to proactively notify for failures and identify root cause of failing steps.
The following monitoring platforms are currently supported.
- Datadog
- New Relic
- Coralogix
- Elastic Cloud
- Amazon OpenSearch
- Kafka
- Confluent Cloud
Logs
Logs are generated for certain events inside your Superblocks account. The following is a non-exhaustive list of events that generate a log.
- An Application, Workflow, or Scheduled Job is created, edited, deployed, or cloned
- A deployed Application is viewed
- An API step (across Applications, Workflows, or Scheduled Jobs) is executed, completes successfully, or fails
- An On-Premise Agent is registered, disconnected, or removed/deactivated
Configuration
Navigate to the Observability page and follow the instructions to configure sending logs to your preferred monitoring platform.
- Datadog
- New Relic
- Coralogix
- Elastic Cloud
- Amazon OpenSearch
- Kafka
- Confluent Cloud
Select the Datadog tile and fill out the form with the following configuration:
- Site - Datadog Site
- API Key - Datadog API key
Select the New Relic tile and fill out the form with the following configuration:
- API Key (Ingest License Key) - New Relic API key with
Ingest - License
permissions. - Region - New Relic region
- API Key (Send-Your-Data API key) - Coralogix Send-Your-Data API key
- Domain (region) - Coralogix domain
Select the Elastic Cloud tile and fill out the form with the following configuration:
- Cloud ID - Unique ID for hosted Elasticsearch cluster, found in the Elasticsearch Service web console
- Cloud Auth - <USERNAME>:<PASSWORD>
Select the Amazon OpenSearch tile and fill out the form with the following configuration:
- Host - Domain endpoint
- Port - HTTPS port
- Username - HTTPS username
- Password - HTTPS password
Select the Kafka tile and fill out the form with the following configuration:
- Bootstrap Server - <HOST>:<PORT> for establishing initial connection to the cluster
- Topic - Topic to send logs
- SASL Username/Password - Authentication method detailed here
Select the Confluent Cloud tile and fill out the form with the following configuration:
- Bootstrap Server - <HOST>:<PORT> for establishing initial connection to the cluster
- Topic - Topic to send logs
- API Key - Cluster API Key
- API Secret - Cluster Secret
Logging Schema
agent-id The id of the Superblocks Cloud or On-Premise Agent executing the request
agent-version The version of the Superblocks Cloud or On-Premise Agent executing the request
application-id The id of the parent Application where the API ran (if applicable)
component The Superblocks agent component (e.g, orchestrator, worker.py, worker.go)
correlation-id The unique id for aggregating all logs of a single API execution
level The severity level
message The log message
organization-id The Superblocks organization id
profile The Superblocks profile used during execution
resource-action The resource action
resource-id The id of the API executed
resource-name The resource name
resource-type The resource type
service An identifier for Superblocks
source An identifier for Superblocks
timestamp The timestamp of the log
user-email The user who executed the API
user-type The type of user, either Superblocks or Embed
Example
{
"agent-id": "018cd5aa-6b42-7f68-9384-e691f254a425",
"agent-version": "v0.1282.0",
"component": "orchestrator",
"correlation-id": "018cda86-2ff2-7f70-b6c9-038b0c4b34a1",
"level": "info",
"message": "The api API1 has been started.",
"organization-id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
"profile": "production",
"resource-action": "started",
"resource-id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
"resource-name": "API1",
"resource-type": "api",
"service": "superblocks",
"source": "superblocks",
"timestamp": 1701377369695,
"user-email": "name@company.com",
"user-type": "superblocks"
}