# Omas Cloud — Metrics documentation # Getting started Learn how to send metrics, build dashboards, and use the API reference. Metrics helps you collect custom time-series data, query it at useful resolutions, and turn it into dashboards and alerts for operational work. Start with the API guide when you want to send data from an application, service, job, or script. Move to dashboards when you want to turn those metrics into a workspace view your team can monitor. ## Guides - [Send your first metric](/docs/send-metric) — publish a single value or a pre-aggregated `min`, `max`, `avg`, and `count` statistic set. - [Query metrics in the UI](/docs/query-metrics) — inspect metric data with aggregations, time ranges, resolution, and dimension filters. - [Understand metric rollups](/docs/metric-rollups-resolution) — learn how resolution, buckets, sample count, and aggregations affect charts and alarms. - [Create an API token](/docs/tokens) — generate a workspace token, store the secret, set expiration, and revoke old tokens. - [Create a dashboard](/docs/dashboards) — build a UI view for related metrics, add charts, arrange them, and share a read-only iframe. - [Create an alarm](/docs/alarms) — define metric conditions, tune evaluation windows, and notify channels on state changes. - [Create notification channels](/docs/notification-channels) — configure email and webhook destinations for alarm notifications. - [Create and use roles](/docs/roles) — understand built-in roles, create custom policies, invite members, and scope API tokens. - [Invite users to a workspace](/docs/invitations-users) — send owner-only invitations, choose roles, and understand the acceptance flow. - [Manage account settings](/docs/account-settings-lifecycle) — update account details, change passwords, schedule deletion, and restore accounts. - [Manage workspaces](/docs/workspaces) — switch workspaces, understand owner/member access, and invite teammates. ## Reference - [API reference](/api-reference) — inspect endpoints, request shapes, response examples, and documented errors. ## Suggested first path 1. Create an API token with write access. 2. Send a single metric value. 3. Query the metric with `count`, `avg`, or another aggregation. 4. Create a dashboard and add a chart for the metric. 5. Create a notification channel. 6. Create an alarm for the signal that needs attention. 7. Invite teammates with the role they need. --- # Send your first metric Publish raw values or pre-aggregated statistic sets to the Metrics API. Metrics are sent with `POST /v1/metrics/{metricName}`. Each request contains one or more entries for the same metric name. Use a single numeric `value` when you have one observation at one timestamp. Use a `statisticSet` when your application has already aggregated multiple observations into `min`, `max`, `avg`, and `sampleCount`. ## Before you send data You need a workspace-scoped API token with `Write` access to the metric resource. Send it as a bearer token: ```bash Authorization: Bearer $METRICS_TOKEN Content-Type: application/json ``` Timestamps are Unix epoch milliseconds. If `resolution` is omitted, the API stores the point at one-second resolution. See [Metric rollups and resolution](/docs/metric-rollups-resolution) before choosing a resolution for production metrics. ## Option 1: Single value Send a `value` when one entry represents one sample. ```bash curl -X POST "$METRICS_API_URL/v1/metrics/cpu_usage" \ -H "Authorization: Bearer $METRICS_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "entries": [ { "timestamp": 1779433200000, "value": 42.7, "resolution": 60, "dimensions": [ { "name": "host", "value": "api-1" }, { "name": "region", "value": "us-west" } ] } ] }' ``` The API stores this as a sample count of `1`. Querying the metric with `aggregation=count` returns `1` for that bucket. ## Option 2: Min, max, avg, count Send a `statisticSet` when one entry represents several samples that were already summarized by your code. ```bash curl -X POST "$METRICS_API_URL/v1/metrics/request_latency" \ -H "Authorization: Bearer $METRICS_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "entries": [ { "timestamp": 1779433200000, "resolution": 60, "statisticSet": { "min": 12.4, "max": 88.9, "avg": 31.6, "sampleCount": 25 }, "dimensions": [ { "name": "route", "value": "GET /v1/metrics" }, { "name": "status", "value": "200" } ] } ] }' ``` `sampleCount` is the count for the statistic set. Querying with `aggregation=count` returns the sum of `sampleCount` values in each returned bucket. ## Choosing between them Use `value` for direct measurements such as a gauge, a queue depth, or a single request duration. Use `statisticSet` when you flush buffered measurements from a client, agent, or job. It reduces payload size without losing the data needed for `min`, `max`, `avg`, `sum`, and `count` queries. Distribution aggregations are the exception. A statistic set carries only `min`, `max`, `avg`, and `sampleCount`, so the distribution within the set is not preserved. `percentile(95)` and `trimmed_mean(10;10)` queries are accurate for individual `value` entries, but approximate toward the average for statistic sets. Send individual `value` entries when you need accurate percentiles or trimmed means. --- # Query metrics in the UI Use the Metrics page to inspect time-series data, choose aggregations, change time ranges, and filter by dimensions. Use the Metrics page when you want to explore data interactively before turning it into a dashboard chart or alarm. The UI uses the same query model as the API, but gives you controls for time range, resolution, aggregation, percentile, and dimensions. Metric views are workspace-scoped. The current workspace controls which metrics appear and which data you can query. ## Open the Metrics page Open **Metrics** from the app sidebar. The list shows metrics available in the current workspace. Each row includes the metric name, discovered dimensions, a small trend preview, and quick actions. Use search to find a metric by name. Use the dimension filters to narrow the list to metrics that include specific dimension names or values. Select a metric name to open its detail page. ## Read the metric detail page The metric detail page shows a chart for the selected metric and a set of query controls. Use it to answer questions such as: - what happened over the last hour, day, or week; - whether a spike is visible at a specific time; - which aggregation best represents the signal; - whether one dimension value behaves differently from another; - which dashboards already use the metric. The page also summarizes the displayed data with the latest value, peak, minimum, average of the returned bucket values, and the number of returned datapoints across the visible series. That datapoint count is not the metric's represented sample count; use the `count` aggregation when you need sample count. ## Choose a time range Use the time range picker to choose the query window. Relative ranges are useful for live investigation, such as the last hour or last day. Custom ranges are useful when you know the exact incident window. The picker also controls resolution. Resolution is the bucket size used for the chart. Smaller resolutions show more detail over short windows. Larger resolutions make long windows easier to read. See [Metric rollups and resolution](/docs/metric-rollups-resolution) for how bucket size changes `count`, `avg`, percentile, trimmed mean, and other aggregations. ## Choose an aggregation Use the aggregation selector to choose how the metric is aggregated in each bucket. Available aggregations: - **avg**: average value. - **sum**: sum of values. - **min**: smallest value. - **max**: largest value. - **count**: number of samples represented by the bucket. - **pct**: percentile value. - **trim**: mean after removing a percentage from the lower and/or upper ends. Use **avg** for typical gauge-like trends, **max** for peak behavior, **count** for volume, **pct** for latency-style metrics where tail behavior matters, and **trim** when outliers would otherwise distort the mean. When you select **pct**, set the percentile value. `95` is a common starting point for latency; use a higher percentile when you need to inspect rarer slow requests. When you select **trim**, set the lower and upper percentages to discard; both default to `10`. In the API, these choices are sent as the `aggregation` query parameter, for example `avg`, `count`, `percentile(95)`, or `trimmed_mean(10;10)`. For trimmed means, leave either trim field empty to use a one-sided trim such as `trimmed_mean(10;)` or `trimmed_mean(;10)`. ## Filter by dimensions Dimensions let you split a metric into slices, such as route, status, host, region, customer, or queue. On the metric detail page, open the dimension filter editor and add a filter: 1. Choose a dimension name. 2. Choose whether to include or exclude a value. 3. Select a value from the loaded values. Each active filter appears as a separate series in the chart. This makes it possible to compare values, such as two hosts or two routes, without creating a dashboard first. Clear filters when you want to return to the aggregate metric. ## Use metric actions The metric detail menu has shortcuts for common follow-up work: - **Add to Dashboard** starts a chart workflow for the metric. - **Create Alert** starts an alarm workflow using the metric. - **Copy Metric Name** copies the exact name for API calls, docs, or configuration. Use these actions after you have found the aggregation and filters that represent the signal correctly. ## Review dashboard usage The metric detail page shows where the metric is already used in dashboards. Use this before changing instrumentation or deleting dimension metadata. It helps you see which dashboards may depend on the metric. ## Delete dimension metadata Dimension metadata can be removed from the metric detail page. This removes the dimension from the metric's discovered dimension list. Use this when an old dimension should no longer appear in filters. Removing dimension metadata does not make it a general-purpose cleanup tool for historical data; treat it as a way to keep the UI selector list accurate. ## Move from exploration to monitoring Use the Metrics page for investigation and one-off checks. When the query becomes something you want to revisit, create a dashboard chart. When the query represents a condition that needs action, create an alarm. See [Create a dashboard](/docs/dashboards), [Create an alarm](/docs/alarms), and [Metric rollups and resolution](/docs/metric-rollups-resolution) for those workflows. --- # Metric rollups and resolution Understand metric resolution, rollup buckets, aggregations, and how single values differ from statistic sets. Resolution controls how metric data is grouped over time. It affects metric detail charts, dashboard charts, alarms, and API queries. Use this guide when a chart looks too detailed, too smooth, or when `count`, `avg`, `max`, percentile, or trimmed mean values do not match what you expected. ## What resolution means Resolution is the bucket size in seconds. When you query a metric at `60` second resolution, each returned point represents one one-minute bucket. When you query at `300` second resolution, each returned point represents one five-minute bucket. Smaller resolutions show more detail. Larger resolutions summarize more data into fewer points, which makes long time ranges easier to read. ## Resolution when sending metrics Each metric entry can include a `resolution`. If `resolution` is omitted, the API stores the point at one-second resolution. If you send `resolution: 60`, the point represents a one-minute bucket. Choose a send resolution that matches how often the data was measured or flushed: - use `1` for high-frequency raw samples; - use `60` for values emitted once per minute; - use a larger value when a job already summarizes a longer interval. When querying later, choose the resolution that matches the question you are asking. It does not have to match the resolution used when the metric was sent. See [Send your first metric](/docs/send-metric) for request examples. ## Single values A single `value` represents one observation at one timestamp. The API stores that observation with a sample count of `1`. If the value is queried alone: - `min`, `max`, and `avg` return the value; - `count` returns `1`; - `sum` returns the value. When several single values fall into the same query bucket, the selected aggregation is calculated across those values. ## Statistic sets A `statisticSet` represents several observations that were already summarized before sending. It includes: - `min`: the smallest observed value; - `max`: the largest observed value; - `avg`: the average value; - `sampleCount`: how many observations the set represents. Use statistic sets when a client, agent, or batch job buffers observations and flushes them later. This keeps payloads smaller while preserving the data needed for rollups. `count` uses `sampleCount`. If a query bucket contains multiple statistic sets, `count` returns the sum of their `sampleCount` values. ## How rollups affect aggregations When the query resolution is larger than the stored resolution, multiple source buckets are rolled up into one returned bucket. The returned aggregation is calculated from the data represented by the source buckets: - `min` returns the smallest value in the returned bucket; - `max` returns the largest value in the returned bucket; - `avg` returns the weighted average using sample counts; - `count` returns the total sample count; - `sum` returns the summed values represented by the bucket; - `percentile(95)` returns the requested percentile for the bucket; - `trimmed_mean(10;10)` returns the average after trimming the lower and upper percentages from the bucket. `percentile(...)` and `trimmed_mean(...)` are accurate for data sent as individual `value` entries. For `statisticSet` sources only the per-set average is available, so those distribution aggregations approximate toward the mean. For example, if six one-minute buckets are queried at `300` second resolution, the chart returns one point for each five-minute group. The one-minute detail is summarized into that larger point. ## Choosing resolution in the UI The time range picker controls the query window and resolution for metric detail pages, dashboards, and alarm previews. Use short ranges with smaller resolutions when investigating a recent spike. Use longer ranges with larger resolutions when looking for trends. Practical starting points: - last hour: `1m` often gives enough detail; - last day: `5m` or `15m` is easier to scan; - last week: larger buckets are usually better for trends. If a chart looks noisy, increase the resolution. If a spike disappears, lower the resolution or narrow the time range. ## Resolution in alarms Alarm resolution controls the buckets used during evaluation. Match alarm resolution to the reporting frequency of the metric. A metric reported once per minute usually works well with `1m`. A metric reported every few seconds may need a shorter resolution. A batch metric that reports every five minutes should not use a one-minute alarm resolution. Use **Datapoints to Alarm** together with resolution to control sensitivity. For example, requiring several breaching buckets avoids alerting on a single short spike. See [Create an alarm](/docs/alarms) for the alarm setup flow. ## Common surprises `count` is sample count, not number of API requests. A single API request can send many entries, and one statistic set can represent many samples. `avg` across statistic sets is weighted by `sampleCount`. A statistic set with `sampleCount: 100` has more weight than one with `sampleCount: 1`. Large resolutions can hide short spikes because several smaller buckets are summarized into one point. The query resolution is a viewing choice. Change it freely to trade detail for readability. --- # Create an API token Generate workspace tokens from the UI, store the secret, set expiration, and revoke tokens you no longer use. API tokens let scripts, services, jobs, and integrations call the Metrics API without signing in as an interactive user. Tokens are scoped to the current workspace. Create them from the workspace where the token should read or write data. They use workspace roles, so they can access workspace resources such as metrics, dashboards, alarms, notification channels, and role-managed workspace APIs. Tokens are not account sessions. They cannot perform account-specific actions such as changing account settings, selecting workspaces, or inviting workspace members. ## Create a token in the UI Open **API tokens** from the main menu. Use the **Create access token** form: 1. Enter a descriptive name, such as `metrics-importer` or `dashboard-refresh`. 2. Select the role the token should use. 3. Choose an expiration period. 4. Select **Generate**. The app shows the token secret once. Store it immediately in your secret manager, deployment environment, or local development configuration. After you leave the page, the secret cannot be shown again. For a least-privilege token, create a custom role first, then select that role when generating the token. ## Use the token Send the token as a bearer token when calling the API: ```bash curl -X POST "$METRICS_API_URL/v1/metrics/request_count" \ -H "Authorization: Bearer $METRICS_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "entries": [ { "timestamp": 1779433200000, "value": 1 } ] }' ``` Use a different token for each service or job. Separate tokens make it easier to rotate one integration without interrupting others. For example, use one token for a metrics ingester and another for a dashboard export job. If one secret leaks, you can revoke only that integration. ## Review existing tokens The tokens page lists existing tokens with: - the token name; - assigned role; - token ID; - creation time; - last-used time; - expiration time. The secret value is not listed. If a token secret is lost, revoke it and create a replacement. ## Revoke a token Open **API tokens**, find the token, then select **Revoke**. The API instance that handles the revocation rejects the token immediately. In a multi-replica deployment, another API instance may continue accepting a cached token for up to about 30 seconds. Allow for that propagation window when responding to a compromised secret. ## Change a token role Open **API tokens**, find the token, choose a different role, then save the change. The token secret stays the same. The handling API instance applies the new role immediately; another API replica may use the previous cached role for up to about 30 seconds. Through the API, send `PUT /v1/auth/tokens/{tokenId}` with a `role` value. The caller must be able to update that token, read the selected role, and already hold every permission granted by the selected role. ## Role-scoped tokens Roles define what a token can do inside the workspace. Built-in roles are available, and you can create custom roles from the **Roles** page using the policy dropdowns. The UI requires a role for every new token. You can also create role-scoped tokens through the API by passing the `role` field to `POST /v1/auth/tokens`. ```bash curl -X POST "$METRICS_API_URL/v1/auth/tokens" \ -H "Authorization: Bearer $METRICS_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "name": "metrics-writer", "expiresInDays": 30, "role": "Admin" }' ``` Prefer a custom least-privilege role for production automation. Use broad roles only when the integration genuinely needs broad workspace access. Role policies only cover workspace-specific actions. Account-specific actions are intentionally unavailable to workspace roles and API tokens. For example, workspace invitations are owner-only account actions, not token permissions. --- # Create a dashboard Build a workspace view for monitoring related metrics, sharing charts, and tracking live system behavior. Dashboards are workspace pages that collect charts for the metrics you care about. Use them to monitor a service, compare related signals, review incidents, or create an embeddable read-only view for another tool. ## What dashboards are for A dashboard groups charts around one operational question, such as API latency, queue health, or customer-facing error rates. Each chart can show one or more metric variables. A variable selects a metric, aggregation, dimensions, and display options. This lets one chart compare several series, such as latency by route or request count by status code. Dashboards also keep a shared time range and optional auto-refresh setting, so every chart is evaluated against the same window. ## Find and organize dashboards Open **Dashboards** from the app sidebar. The dashboard list shows every dashboard in the current workspace, the number of charts on each dashboard, and when it was last updated. Use search when the workspace has many dashboards. You can filter the list to show all dashboards, dashboards with charts, or empty dashboards. You can also sort by last update, name, or chart count. ## Create a dashboard Open **Dashboards** from the app sidebar, then select **New Dashboard**. Enter a name that describes the view, such as `API health` or `Checkout latency`, then select **Create**. The app opens the new dashboard immediately. You can rename the dashboard later by editing the title at the top of the dashboard page. Unsaved changes are marked until you select **Save**. ## Add charts Select **Add Chart** from the dashboard toolbar. In the chart editor: - Choose a chart title. - Pick the chart type, such as **Line** for time-series trends or **Number** for a single current value. - Add one or more variables. - For each variable, choose a metric, aggregation, and any dimension filters. - Set a legend label when the default variable name is not descriptive enough. The preview updates from the selected variables and time range. Save the chart when it shows the data you expect. ## Use metric variables Metric variables are the main way charts read data. For each metric variable: - choose the metric name; - choose the aggregation, such as `avg`, `min`, `max`, `sum`, `count`, `percentile(95)`, or `trimmed_mean(10;10)`; - enter percentile or trimmed-mean bounds when the aggregation needs parameters; - add dimension filters when you only want one slice of the metric; - set a legend label when the metric name alone is not clear. Dimension filters can include or exclude one dimension value. Use them to split operational views by values such as environment, region, endpoint, status, customer, or queue. ## Use expression variables Expression variables calculate a new series from other variables in the same chart. Use them when the raw metric is not the final value you want to show. For example, you can compare two counters, combine related series, or derive a ratio from separate variables. Give each source variable a short name, then reference those names in the expression. Keep helper variables hidden when they are only used to calculate the final visible series. ## Choose the chart type Use **Line** charts for trends over time, comparisons between series, and incident review. Use **Number** charts for a compact current value, such as latest latency, current queue depth, or current error rate. A Number chart displays the latest returned bucket, and `sum` is the sum within that bucket. To show a total for the whole selected window, choose a resolution and window that produce a single returned bucket. Number charts can also show a unit such as `ms`, `%`, or `req/s`. ## Arrange the dashboard Charts can be resized and moved inside the dashboard grid. Use this to keep the most important signals large and visible near the top. Layout changes are saved automatically. Dashboard-level changes, such as renaming the dashboard or changing auto-refresh settings, are saved with **Save**. ## Use time range and refresh controls The time range picker controls the window used by every chart on the dashboard. Use shorter windows for live operations and longer windows for trend review. Auto-refresh keeps the dashboard current while you are watching a service. Disable it when you are investigating a fixed historical window. See [Metric rollups and resolution](/docs/metric-rollups-resolution) when choosing the bucket size for dashboard charts. ## Share or embed a dashboard Select **Generate iframe** to create an embeddable dashboard iframe. The generated iframe uses a restricted embed token, so it can display that dashboard without giving access to the full workspace UI. Use embedded dashboards for status pages, internal runbooks, or other tools that need a live read-only view. If the embed token is missing, expired, or cannot access the dashboard, the embedded view shows an error instead of chart data. Generate a new iframe when an old embed stops working. ## Delete a dashboard Use the dashboard list to delete dashboards that are no longer needed. Deleting a dashboard removes the dashboard and all of its chart configuration. It does not delete the underlying metrics. ## Permissions Dashboards belong to the current workspace. Viewing, creating, updating, embedding, and deleting dashboards require role permissions for dashboard actions in that workspace. See [Create and use roles](/docs/roles) for custom role policies, and [Query metrics in the UI](/docs/query-metrics) for inspecting the metric data used by dashboard charts. --- # Create an alarm Define alarm rules in the UI, preview thresholds, and notify channels when metrics cross a condition. Alarms watch workspace metrics and change state when an expression is true for enough datapoints in an evaluation window. Use them for operational signals that need attention, such as high latency, missing throughput, queue growth, or error spikes. An alarm belongs to the current workspace. It can use metrics from that workspace and notify workspace notification channels. ## What alarms are for Use alarms when a metric condition should produce an operational signal, not just a chart. Good alarm rules are specific and actionable. For example: - API latency is above a threshold for several datapoints. - Error rate is higher than expected while traffic is present. - Queue depth keeps growing. - A periodic job stops reporting samples. Dashboards help you inspect data. Alarms help you notice when the data needs action. ## Create an alarm Open **Alarms** from the app sidebar, then select **New Alarm**. Enter a name that describes the condition, such as `checkout-latency-high` or `worker-queue-depth`. Alarm names are used in URLs and API calls, so prefer stable names that are easy to recognize. ## Add metric variables An alarm expression uses one or more variables. Each variable selects a metric and aggregation, similar to a dashboard chart variable. For each variable: - choose a variable name, such as `var1`, `latency`, or `errors`; - select the metric to read; - select the aggregation, such as `avg`, `max`, `count`, `percentile(95)`, or `trimmed_mean(10;10)`; - set percentile or trimmed-mean bounds when the aggregation needs parameters; - add dimension filters when the alarm should only evaluate a subset of the metric. Use **Add Variable** when the expression needs multiple signals. For example, one variable can represent errors and another can represent requests. ## Write the expression The expression decides when the alarm should fire. It can compare variables to numbers and combine conditions. Examples: ```text latency > 500 errors > 10 and requests > 100 queueDepth > 1000 ``` Use the variable names you defined in the variables section. The UI validates the expression and shows a preview with threshold reference lines when it can extract them. ## Choose evaluation settings The **Evaluation Period** is the time window, in minutes, used to decide the alarm state. **Datapoints to Alarm** is how many datapoints inside that window must breach the expression before the alarm changes to an alarm state. Use more than one datapoint for noisy metrics so a single spike does not page the team. Use a shorter period and fewer datapoints for conditions that need fast reaction. The **Resolution** controls the metric rollup interval used by the alarm. Match it to the expected reporting frequency of the metric. A service that reports every minute usually works well with `1m`; high-frequency metrics may use shorter resolutions. See [Metric rollups and resolution](/docs/metric-rollups-resolution) for how alarm buckets affect aggregations and sensitivity. ## Treat missing data Use **Treat Missing Data As** to decide what happens when the alarm cannot evaluate enough datapoints in the current window. This can happen when a metric has not reported yet, when some buckets in the window are empty, or when one side of a variable-to-variable expression is missing data. For example, if **Datapoints to Alarm** is `5` but only `3` datapoints are present in the window, the alarm does not have enough data to make a normal `OK` or `IN_ALARM` decision. Choose one of these options: - **Show insufficient data** stores the alarm as `INSUFFICIENT_DATA`. - **Treat as breaching** stores the alarm as `IN_ALARM`. - **Treat as not breaching** stores the alarm as `OK`. In the API, this setting is named `treatMissingDataAs`. The allowed values are `MISSING`, `BREACHING`, and `NOT_BREACHING`. ## Notify channels Use **Notification Channels** to choose which channels should receive alarm state changes. The alarm form loads the available notification channels in the current workspace. Check each channel that should receive notifications. Each row shows the channel name, type, and target email or webhook URL. If no channels are available, create one from **Notification Channels** first, then return to the alarm. Create and verify notification channels before selecting them for an alarm. The form may list an unverified email channel, but the API rejects an alarm that references it. ## Review and save The preview uses the selected metric variables and the last hour of data. Use it to check that the selected aggregation, filters, and threshold make sense before saving. Select **Create Alarm** when the rule is ready. The app opens the alarm detail page after creation. ## Manage alarms Open **Alarms** to review configured alarms and their current status. Open an alarm to inspect its configuration. Select **Edit Alarm** to change the rule, or **Delete** to remove it. ## Permissions Creating and managing alarms requires a role that allows the relevant alarm actions in the current workspace. See [Create and use roles](/docs/roles) for custom role policies, and [Create an API token](/docs/tokens) if automation should create or manage alarms through the API. --- # Create notification channels Configure email and webhook channels in the UI so alarms can notify the right destination. Notification channels are workspace destinations used by alarms. Create them before adding notifications to an alarm rule. Each channel has a stable name. Alarms refer to channels by that name, so choose names that describe the destination, such as `ops-email`, `incident-webhook`, or `checkout-oncall`. ## Open notification channels Open **Notification Channels** from the Alarms section of the app. The page shows existing channels, their type, target, verification status, and delete action. ## Create an email channel Use an email channel when alarm state changes should send email. 1. Enter a channel name. 2. Set **Type** to **Email**. 3. Enter the destination email address. 4. Select **Create**. The app sends a verification code to the email address. The channel appears as **Unverified** until the code is confirmed. ## Verify an email channel Find the unverified email channel in the channel list. Enter the six-digit verification code from the email and select **Verify**. If the code expired or was lost, select **Resend** to send a new verification code. Only verified email channels can be attached to an alarm. An unverified channel can appear in the channel list and alarm form, but the API rejects an alarm that references it. ## Create a webhook channel Use a webhook channel when alarm state changes should call an HTTP endpoint. 1. Enter a channel name. 2. Set **Type** to **Webhook**. 3. Enter the webhook URL. 4. Select **Create**. Webhook channels do not require email-style verification. Make sure the destination URL is stable and controlled by the system that should receive alarm notifications. ## Use channels in alarms When creating an alarm, use **Notification Channels** to select the destinations for alarm state changes. The alarm form lists the channels available in the current workspace. Check each channel that should receive notifications. You no longer need to type channel names manually. The alarm sends notifications to those channels when its state changes. Verify email channels before selecting them from alarms. Webhook channels do not require verification. ## Delete a channel Use **Delete** on the channel row to remove a notification channel. A channel cannot be deleted while an alarm references it; edit those alarms to remove the channel first. ## Permissions Creating, verifying, resending, and deleting notification channels require workspace permissions for notification channel actions. See [Create and use roles](/docs/roles) for custom role policies, and [Create an alarm](/docs/alarms) for adding channels to alarm rules. --- # Create and use roles Understand default roles, custom role policies, invitations, and token permissions. Roles define what a workspace member or API token can do. Use them to keep workspace access scoped to the job: viewing dashboards, sending metrics, editing resources, managing tokens, or administering workspace resources. A role is a named set of policy statements. Each statement has an `effect`, `service`, `accessLevel`, `resourceType`, and `resourceId`. ## Default roles Every workspace can use these built-in system roles: - `Admin` has full workspace access. Its policy is `ALLOW * * * *`. - `ReadOnly` can read and list workspace resources. Its policy allows `Read` and `List` operations on all services and resources. System roles cannot be edited, renamed, or deleted. Custom role names also cannot start with `#`, because that prefix is reserved for internal roles. ## Create a custom role in the UI Open **Roles** from the main menu. Use the **Create role** form: 1. Enter a role name, such as `Metrics reader` or `Dashboard viewer`. 2. Add an optional description. 3. Add one or more policy statements. 4. Select **Create role**. Each policy statement has: - **Effect**: choose `ALLOW` or `DENY`. - **Service**: choose the API area, such as `metrics`, `dashboards`, `alarms`, or `auth`. - **Access level**: choose an access level like `Read`, `Write`, `List`, `Create`, or `Delete`. - **Resource type**: choose the type of resource, such as `metric`, `dashboard`, `alarm`, `notification-channel`, `role`, or `token`. - **Resource ID**: enter one concrete resource ID or use `*` as a wildcard. The service, access-level, and resource-type controls are backed by the backend authorization catalog. Use the search field in each dropdown to filter long lists. Use `*` as a wildcard when the statement should match multiple services, access levels, resource types, or resource IDs. Policy statements are evaluated in order. The last statement that matches a request determines whether it is allowed or denied. Put broad statements first and narrower exceptions later; a later matching `ALLOW` can override an earlier `DENY`, and a later matching `DENY` can override an earlier `ALLOW`. Role creation and updates are permission bounded. A caller can only create or update a role when every `ALLOW` statement in the submitted role is already allowed by the caller's effective permissions. This prevents a delegated role manager from creating a broader role than they have themselves. ## What policies can target The role editor only shows workspace-specific policies. Account-specific policies are intentionally excluded from workspace roles because they depend on the signed-in account, not a workspace role. Examples of account-specific actions that roles cannot grant: - account profile and password changes; - selecting or deleting account-owned workspaces; - accepting workspace invitations; - sending workspace invitations; - removing workspace members. Workspace invitations and member removal are owner-only account actions. They are not permissions that can be delegated through a workspace role or API token. ## Use roles for invitations Workspace invitations assign a role to the invited account. To invite someone: 1. Open **Workspaces**. 2. Select the workspace you want to manage. 3. Use the **Invite member** form on the current workspace card. 4. Enter the person's email address. 5. Choose a role from the **Role** dropdown. 6. Select **Send invite**. Only the workspace owner can send invitations. The role dropdown shows built-in roles and custom roles available to the workspace. Internal roles that start with `#` are hidden. After the recipient accepts the invitation, the workspace appears on their **Workspaces** page with the assigned role. The owner can later change an accepted member's role, or remove the member, from **Members** on the current workspace card. ## Use roles for tokens Every API token has a role. See [Create an API token](/docs/tokens) for the token creation flow and API example. Creating a token is also permission bounded. The caller must be allowed to create tokens, read the selected role, and hold every allowed permission in that role. For example, a role that can create tokens and read the `Admin` role still cannot mint an `Admin` token unless it already has the permissions granted by `Admin`. ## Practical workflow Start with `ReadOnly` for people who only need to inspect dashboards and metrics. Use `Admin` only for trusted accounts that need broad workspace control. Create custom roles when a person or token needs a narrower policy than the default roles provide. Typical examples are separate roles for dashboard viewers, metric writers, alert managers, and import jobs. --- # Invite users to a workspace Invite teammates from the Workspaces page, choose their role, and understand what happens when they accept. Invitations add another account to a workspace. Use them when a teammate needs access to the workspace's metrics, dashboards, alarms, notification channels, roles, or tokens. Each invitation assigns one role. After the recipient accepts, that role controls what they can do inside the workspace. ## Who can invite users Only the workspace owner can send invitations. Inviting users is an account-specific owner action. It is not a workspace role permission, and it cannot be delegated to an API token or a custom role. ## Choose the workspace first Open **Workspaces** from the main menu. If your account has access to multiple workspaces, make sure the workspace you want to share is selected as the current workspace. The invite form appears on the current workspace card when you are the owner. Each workspace card shows whether you are the **Owner** or a **Member**. Members can use the workspace according to their role, but they cannot invite other users. ## Send an invitation On the current workspace card: 1. Find **Invite member**. 2. Enter the person's email address. 3. Choose the role they should receive. 4. Select **Send invite**. The role dropdown shows the built-in roles and custom roles available in the workspace. Internal roles are hidden. Start with `ReadOnly` when the person only needs to view dashboards and metrics. Use `Admin` only for trusted teammates who need broad workspace control. Create a custom role first when the person needs narrower access. See [Create and use roles](/docs/roles) for default roles and custom role policies. ## Accept an invitation The recipient receives an invitation link by email. When they open the link, the app shows: - workspace name and ID; - assigned role; - invited email address; - expiration time. If the recipient is signed in, they can select **Accept invitation**. If they are not signed in, they can sign in or create an account from the invitation page. The invite token is carried through the sign-in or sign-up flow. After acceptance, the workspace appears on their **Workspaces** page. They can switch to it like any other workspace available to their account. ## Change a member role After an invitation is accepted, the person appears in **Members** on the current workspace card. To change their role, choose a new role from the member row and select **Save**. The API instance that handles the change uses the new role immediately. In a multi-replica deployment, another API instance may use the previous role for up to about 30 seconds while its authorization cache expires. Only the workspace owner can change accepted member roles. ## Remove a member To remove an accepted member, select **Remove** on their row in **Members** on the current workspace card. The member loses access to the workspace and no longer sees it on their **Workspaces** page. Only the workspace owner can remove members. Removing a member is an owner-only account action; it is not a workspace role permission and cannot be delegated to an API token or a custom role. You cannot remove the workspace owner. A removed member can no longer select the workspace. Existing workspace sessions are also denied after membership caches refresh; they do not retain access for the full session-token lifetime. The API instance that handles the removal invalidates its cache immediately, while another API replica may continue accepting the previous membership for up to about 30 seconds. ## Expired or invalid invitations If an invitation is expired or invalid, the invitation page shows an error and the recipient cannot accept it. Send a new invitation from the **Workspaces** page when the original link can no longer be used. ## What invited users can do Invited users are workspace members. Their role determines which workspace resources they can read, list, create, update, or delete. Roles apply to workspace resources such as metrics, dashboards, alarms, notification channels, roles, and tokens. Roles do not grant account-specific actions such as changing account settings, selecting another user's workspace, accepting invitations for someone else, or sending workspace invitations. For automation, use an API token instead of a user invitation. Tokens also require a role, but they are scoped to the current workspace and are meant for scripts, jobs, and services. See [Create an API token](/docs/tokens) for token setup. --- # Account settings and lifecycle Manage account details, password changes, account deletion, and account restore from the frontend. Account settings are personal to the signed-in account. They are separate from workspace resources such as metrics, dashboards, roles, tokens, and alarms. Use this page when you need to change your password, schedule account deletion, or restore an account that was scheduled for deletion. ## Open account settings Open **Settings** from the main menu, then open the account settings section. The page shows: - email address; - account ID; - account status; - creation time; - last login time. If the account is isolated because deletion was scheduled, the page also shows the isolation time and scheduled deletion time. The email address is shown for reference and cannot be changed. ## Change password Use **Change Password** from the account settings page. To change your password: 1. Enter the old password. 2. Enter the new password. 3. Confirm the new password. 4. Select **Send code**. 5. Enter the verification code sent to your account email. 6. Select **Verify password**. The frontend requires the new password and confirmation to match. The new password must be at least 12 characters long and include an uppercase letter, a lowercase letter, a number, and a symbol. Use the password reset flow on the sign-in page if you cannot sign in with the old password. ## Schedule account deletion Use **Delete Account** from the account settings page. Deletion is a two-step flow: 1. Select **Delete Account**. 2. Check your account email for the deletion confirmation code. 3. Enter the code. 4. Select **Verify delete**. After deletion is verified, the app signs you out. The account is isolated and scheduled for deletion. An isolated account cannot continue normal workspace work until it is restored. API requests can return an account-isolated error while the account is in this state. ## Restore an account Use the account restore page when an account was scheduled for deletion and should be recovered. If you are not signed in: 1. Open the restore flow. 2. Enter your email and password. 3. Select **Send code**. 4. Enter the verification code sent to your account email. 5. Select **Restore account**. If you are already in an account-restore session, the app can request the restore code without asking for credentials again. After restore completes, the app may ask you to sign in again before returning to the normal workspace flow. ## Account actions vs workspace actions Account settings are account-specific actions. Workspace roles and API tokens cannot grant these abilities. Examples of account-specific actions include: - changing password; - scheduling or restoring account deletion; - selecting a workspace; - accepting workspace invitations; - sending workspace invitations as a workspace owner. Workspace roles control access to workspace resources. See [Create and use roles](/docs/roles) for workspace permissions and [Invite users to a workspace](/docs/invitations-users) for invitations. ## Related pages - [Manage workspaces](/docs/workspaces) for workspace selection and ownership. - [Create an API token](/docs/tokens) for automation access. - [Invite users to a workspace](/docs/invitations-users) for teammate access. --- # Manage workspaces Understand workspace selection, membership, owner actions, and invitations. Workspaces separate metrics, dashboards, alarms, notification channels, roles, and API tokens. Use them when different products, environments, teams, or customers need separate data and access control. Most app pages operate inside the currently selected workspace. The selected workspace controls which metrics you see, which dashboards you edit, which roles are available, and where new API tokens are created. ## Open the Workspaces page Open **Workspaces** from the main menu. The page lists every workspace available to your account. Each workspace card shows: - workspace name and ID; - whether it is the current workspace; - your effective role; - whether you are the owner or a member; - owner account ID; - creation and update times. ## Select a workspace Select **Switch** on a workspace card to make it the current workspace. After switching, the app returns to the dashboard home page. Requests you make after that use the selected workspace, including metrics, dashboards, alarms, roles, and API tokens. If your account has access to more than one workspace, switch before creating resources so they are created in the intended workspace. ## Owners and members Every workspace has one owner account. The owner has administrative control over the workspace and can invite other accounts. Members access the workspace through an assigned role. The role determines what the member can do inside that workspace. See [Create and use roles](/docs/roles) for how roles work and which permissions can be assigned. ## Invite a member Only the workspace owner can send invitations. To invite someone: 1. Open **Workspaces**. 2. Make sure the workspace is the current workspace. 3. Use the **Invite member** form on the current workspace card. 4. Enter the person's email address. 5. Choose the role they should receive. 6. Select **Send invite**. The recipient receives an invitation email. After accepting it, the workspace appears on their **Workspaces** page with the assigned role. Invitations are account-specific owner actions. They are not permissions that can be delegated to API tokens or custom workspace roles. ## Tokens and workspace scope API tokens are created inside the current workspace and use a role from that workspace. Switch to the right workspace before creating a token. See [Create an API token](/docs/tokens) for token creation and storage. ## Practical setup Use separate workspaces when you need clear operational boundaries. Common patterns are: - one workspace per environment, such as production and staging; - one workspace per product or service group; - one workspace per customer or tenant; - one workspace per team when teams should not share dashboards or alerts. Keep owner access limited, and invite members with the narrowest role that fits their work.