Which Labels and Dimensions Should Be Standardized for LLM Inference Metrics?

Standardize one canonical label dictionary covering model, provider, endpoint, location, workspace, credential, request, routing, cache, and serving infrastructure. Use the same definitions across counters, histograms, traces, logs, dashboards, usage reports, and billing exports where practical. Keep stable deployment metadata at the resource level, attach only bounded analytical dimensions to metrics, and distinguish what was requested from what actually served each request.

The short answer: use one canonical label dictionary across telemetry and cost data

A shared dictionary makes telemetry easier to join and interpret. If a workspace has one identifier in metrics, another in traces, and only a display name in billing data, teams spend time reconciling records instead of analyzing cost and operations.

The dictionary should define more than field names. For every dimension, document:

• Its precise meaning and data type
• Whether it describes a resource, deployment, or individual request
• Whether it is required or optional
• Its allowed values or formatting rules
• How missing or unknown values are represented
• Its expected cardinality and retention treatment
• Whether it is drawn from an industry convention or is an enterprise-specific field

This is an organizational schema recommendation, not a universal metric catalog. Individual observability frameworks and providers may use different names, and their conventions can change. The important implementation objective is semantic consistency: a field should represent the same concept everywhere it appears.

Separate stable resource attributes from request-level dimensions

Resource attributes describe the environment producing telemetry and usually remain stable across many requests. Examples include the service, cluster, cloud region, availability zone, logical deployment, and environment. Configure these once at the collector, service, or deployment level rather than reconstructing them for every instrument.

Request-level dimensions describe what changed for an operation. Examples include the requested model, served model, operation type, routing result, cache status, workspace ID, and an opaque credential ID.

This separation has practical benefits:

• Cleaner instrumentation: application code does not need to repeat deployment metadata on every metric call.
• Consistent aggregation: a region or cluster retains one meaning across instruments.
• Safer cardinality management: request-specific values receive more scrutiny before becoming time-series labels.
• Better portability: deployment metadata can change without rewriting request instrumentation.

The distinction is conceptual rather than absolute. An observability backend may materialize resource attributes as searchable metric dimensions. Teams should still maintain the resource-versus-request classification in the schema registry because it identifies who owns each value and where it should be populated.

Apply consistent names across metrics, traces, logs, dashboards, and billing exports

Use canonical field definitions even when a destination requires a different physical name. For example, if an analytics warehouse requires workspace_id while a telemetry system uses namespaced attributes, maintain an explicit mapping rather than allowing each integration to redefine “workspace.”

Consistency is especially important for cost allocation. Token counts, request counts, latency distributions, provider charges, and internal infrastructure costs become more useful when they can be grouped by the same durable model, workspace, deployment, and credential identifiers.

Not every trace or log attribute belongs on a metric. Metrics create time series from combinations of label values, so adding an unbounded value can produce uncontrolled cardinality. A canonical dictionary should therefore classify fields by telemetry surface:

• Metrics: bounded dimensions needed for routine aggregation and alerting
• Traces: detailed request context used for investigation and causal analysis
• Logs: controlled event details, errors, and diagnostic context
• Cost data: durable attribution keys and normalized usage units
• Dashboards: display labels derived from canonical IDs rather than new identifiers

High-cardinality context can remain in traces or access-controlled logs. Exemplars or correlation identifiers can connect an aggregate metric observation to a representative trace without turning every request ID into a metric label.

Define requested and served values separately

Routing, fallback, aliases, and provider-managed endpoints can cause the executed request to differ from the original request. Never use a single ambiguous model or provider label when this distinction matters.

A practical schema separates:

• requested_model_id from served_model_id
• requested_provider from served_provider
• A client-facing endpoint or model alias from the actual serving deployment
• The intended logical region from the cloud region and availability zone that processed the request

For a direct request with no routing change, requested and served values may be equal. Recording them as separate concepts still preserves the schema when routing policies evolve.

Model identity should use a canonical ID wherever one exists, plus a separate revision or version. Human-readable display names are helpful in dashboards but should not be the sole durable identifier. “Latest,” marketing names, and mutable aliases can otherwise combine materially different model revisions into one time series.

Provider identity also needs a clear definition. A provider is not necessarily the same as an endpoint, gateway, deployment, or infrastructure operator. Keep these concepts separate so teams can compare requested demand, routing behavior, and actual service delivery without conflating them.

Recommended dimension schema for inference observability

The following schema is a practical starting point. The key names are recommendations and should be mapped to the conventions and tools used by your organization. Fields marked as enterprise-specific commonly require custom definitions rather than assumptions that they are universally standardized.

Compact schema table: key, meaning, format, scope, requirement, and cardinality risk

Recommended dimension	Meaning	Example format	Scope	Requirement	Cardinality risk
model.requested.id	Canonical model or alias requested by the caller	model-family-large	Request	Required	Low–medium
model.served.id	Canonical model that handled the operation	model-family-large	Request	Required	Low–medium
model.served.revision	Version, checkpoint, or revision actually served	2026-08-15	Request/deployment	Optional	Medium
gen_ai.operation.type	Bounded operation or workload category	chat, embedding, batch	Request	Required	Low
provider.requested.name	Provider selected or implied by the request	Controlled enum	Request	Optional	Low
provider.served.name	Provider that actually served the request	Controlled enum	Request	Required for multi-provider routing	Low
service.endpoint.id	Durable ID for the logical endpoint or service	ep_7f31	Resource/request	Recommended	Medium
deployment.id	Durable serving-deployment identifier	dep_prod_eu_02	Resource	Recommended	Medium
cloud.region	Infrastructure provider’s cloud region	Provider region code	Resource	Recommended	Low
cloud.availability_zone	Infrastructure availability zone	Provider zone code	Resource	Optional	Low–medium
deployment.region	Organization-defined logical operating region	eu-primary	Resource	Optional, custom	Low
cluster.id	Durable compute or serving-cluster identifier	cluster_gpu_03	Resource	Optional, custom	Medium
workspace.id	Stable internal workspace or tenant ID	ws_01J...	Request	Required for attribution, custom	Medium–high
project.id	Stable project or application boundary	proj_support_ai	Request	Optional, custom	Medium
deployment.environment	Bounded lifecycle environment	prod, stage, dev	Resource	Recommended, custom	Low
credential.id	Opaque key ID, approved alias, or controlled hash—not the secret	key_ref_8a21	Request	Optional, custom	Medium–high
credential.principal.type	Bounded owner category	service_account, team	Request	Optional, custom	Low
routing.outcome	Bounded routing result	primary, fallback, blocked	Request	Optional, custom	Low
routing.policy.id	Stable ID for the applied routing policy	route_chat_prod_v3	Request	Optional, custom	Medium
cache.status	Bounded cache result	hit, miss, bypass	Request	Optional, custom	Low
batch.mode	Whether serving was interactive or batched	online, batch	Request	Optional, custom	Low
serving.quantization	Bounded deployed quantization profile	Controlled enum	Resource/deployment	Optional, custom	Low
accelerator.pool.id	Stable accelerator pool, not an individual request ID	gpu_pool_a	Resource	Optional, custom	Medium

“Required” means required for a useful baseline schema, not required by every external standard. For a single-provider environment, for example, provider.requested.name may add little analytical value. In a routed environment, requested and served provider fields become much more important.

Use display-name fields only as optional metadata. A workspace may be renamed from “Research” to “AI Platform,” while its durable workspace.id should remain unchanged. Apply the same rule to projects, endpoints, clusters, policies, and model deployments.

API credentials require particularly careful treatment. Record only an opaque key ID, an approved alias, or a controlled internal hash designed for attribution. Never place a raw API key, bearer token, secret, or reversible credential value in telemetry. Owner or service-account attribution should be added only when privacy, access, retention, and cardinality controls permit it; personal email addresses should not become metric dimensions.

Naming rules for keys, enums, unknown values, and units

A label dictionary is easier to operate when it follows a few strict rules:

1. Use lowercase, stable keys. Select one delimiter and namespace pattern, then apply it consistently. Avoid abbreviations that could have multiple meanings.
2. Prefer durable IDs to display names. Keep readable names in metadata or dashboards, but join and aggregate on stable identifiers.
3. Document bounded enums. Define allowed values for operation type, environment, routing outcome, cache status, and similar categories. Do not let arbitrary strings become new categories.
4. Represent unknowns explicitly. Choose a documented value such as unknown rather than mixing empty strings, nulls, omissions, and ad hoc values such as n/a.
5. Name requested and served values unambiguously. Do not depend on dashboard context to explain whether model means client intent or actual execution.
6. Keep units out of ambiguous values. Use instrument names and metadata that identify seconds, bytes, tokens, requests, or currency units consistently. Do not combine different units in one metric series.
7. Normalize without destroying source identity. A normalized provider or model category can support comparison, while separate source fields preserve the original billing or endpoint identity where needed.

Region labels deserve their own definitions. cloud.region, cloud.availability_zone, deployment.region, and cluster.id answer different questions. Overloading all four into region makes it unclear whether a chart represents physical infrastructure placement, fault domains, an internal operating boundary, or a user-facing geography.

Keep unbounded and sensitive values out of metric labels

Do not use the following as time-series dimensions:

• Request, trace, or session IDs
• Individual user IDs or email addresses
• Prompt or response text
• Tool inputs and outputs
• Raw API keys, tokens, or secrets
• Full endpoint URLs or arbitrary URL paths
• Raw exception messages or stack traces
• Unnormalized error text
• Timestamps or dynamically generated object names

Some of these values can be useful during an investigation, but they belong in appropriately controlled traces or logs—not in aggregate metric labels. For metrics, convert raw details into bounded categories such as error.type=timeout or routing.outcome=fallback. Preserve the detailed event in a trace and link it with an exemplar or correlation ID when the telemetry stack supports that workflow.

How standardized dimensions support analysis

A well-governed dictionary enables multiple teams to use the same telemetry without redefining attribution for every report:

• Cost allocation: group usage and cost records by durable model, workspace, project, deployment, or environment IDs.
• Provider and model comparison: compare requested demand with the provider and model that actually served it.
• Regional operations: separate logical deployment regions from infrastructure regions, zones, and clusters.
• Workspace chargeback: attribute bounded usage records to stable tenant or workspace IDs rather than mutable names.
• Credential governance: identify activity associated with an opaque credential reference without exposing the credential itself.
• Routing analysis: examine primary, fallback, blocked, or policy-selected outcomes by workload category.
• Cache analysis: compare cache outcomes by model, operation, workspace, or routing policy while keeping keys and content out of labels.
• Serving analysis: segment workloads by deployment characteristics such as batching mode, quantization profile, or accelerator pool when those categories are operationally meaningful.

These dimensions improve the structure of analysis; they do not by themselves guarantee accurate billing, lower costs, better latency, or complete governance. Instrumentation quality, source reconciliation, access controls, and ownership remain essential.

Govern the dictionary as a versioned interface

Treat the schema as a contract between application teams, infrastructure teams, observability owners, and finance systems. Maintain a versioned registry containing definitions, owners, data types, allowed values, scope, sensitivity, and cardinality expectations.

Useful governance practices include:

• Validate required fields and allowed enums in instrumentation or collection pipelines.
• Assign an owner to each namespace and enterprise-specific dimension.
• Use migration aliases when renaming fields so historical and new data can be reconciled.
• Publish deprecation dates and stop-writing rules rather than silently changing meanings.
• Review cardinality periodically by metric, workspace, and deployment.
• Test whether new labels create useful analytical segments before adding them broadly.
• Maintain mappings between telemetry fields and cost or billing export columns.
• Restrict access to credential, tenant, and detailed request context according to its sensitivity.

A field should never be repurposed in place. If deployment.region originally means an internal operating region, do not later redefine it as a cloud provider region. Introduce or map a distinct field instead.

Relating the schema to OpenTelemetry GenAI and OpenInference

OpenTelemetry GenAI semantic conventions and OpenInference can provide useful starting concepts for generative-AI operations, model identity, provider context, spans, and related telemetry. Exact attribute names and their stability can evolve, so implementations should verify the current authoritative specifications before adopting specific field names.

Use standard fields where their definitions fit your data. Maintain an explicit mapping for enterprise concepts that are not covered or do not have the required semantics. Workspace, tenant, project, chargeback, opaque credential, internal routing policy, cache, and serving-infrastructure dimensions should be treated as custom unless current specifications clearly define an applicable field.

Avoid forcing an internal concept into a similarly named standard attribute. Semantic alignment is more important than superficial name matching. The registry should identify the convention, version, and mapping used so future migrations can be managed deliberately.

Applying the dictionary to private inference operations

For private inference, the dictionary becomes a useful design tool for deciding what must remain visible across routing, caching, batching, quantization, and GPU scheduling workflows. It helps teams ask concrete questions: Can requested and served models be distinguished? Can costs be attributed to durable workspace IDs? Are cache and routing outcomes represented with bounded values? Can deployment and infrastructure placement be analyzed without exposing sensitive request data?

Token Forge Cloud Private LLM Inference supports private deployment and serving-layer optimization for enterprise AI workloads. Token Forge Cloud provides caching, routing, batching, quantization, and GPU scheduling, making these visibility questions relevant when planning an inference control plane. Token Forge Cloud also supports private deployment paths where models, prompts, and telemetry remain in the customer’s controlled environment.

For teams validating demand before private deployment, Token Forge Cloud Managed Model APIs provide an API-first route to model access and usage data. In either deployment path, teams should define the telemetry and attribution fields they need before connecting dashboards, operational workflows, and cost reporting.

Next Step

Contact Token Forge Cloud to discuss API access, private deployment, and LLM inference cost control.