Get backend exceptions into Issues ¶
The Nais APM Issues tab groups your service's backend exceptions from Loki. How good that grouping is depends entirely on how the exception reaches Loki — and in particular on which of the two log-ingest paths your app uses.
Most nais apps ship logs the default way — stdout → fluentbit → Loki — and deliberately do not enable OpenTelemetry log export (they keep control of stdout, often to multi-ship some logs elsewhere). So this guide leads with getting the best possible Issues on that default path, then covers the OpenTelemetry option that unlocks the richest shape.
Why the log shape matters ¶
Nais APM reads three shapes of backend exception from Loki, best to worst:
| Shape | What Loki has | Result in Issues |
|---|---|---|
| A — semconv metadata | exception_type, exception_message, exception_stacktrace as structured metadata | First-class. Grouped by type + message, every occurrence counted, full stack trace, pod impact. |
| B — JSON body | A JSON log line with message/msg and an error-class level | Grouped by message only (no type), counted per occurrence. |
| C — plain text | A multi-line stack trace printed to stdout — fragments into one Loki entry per line | Weak. The trace is split across separate Loki entries, so the Issue shows only the lead line with no coherent stack — lossy. Fix it with structured JSON (shape B) or OTLP (shape A). |
Which path do your logs take? ¶
Your ceiling is set by how your logs get into Loki, not by your logging code alone:
| Path | What Loki gets | Best shape reachable |
|---|---|---|
| stdout → fluentbit → Loki (default) | The raw line as the log body. fluentbit attaches only Kubernetes fields (service_name, service_namespace, k8s_cluster_name as labels; pod/node/container/collector as structured metadata). It does not parse your JSON or promote app fields like exception_type. | Shape B — the Issues tab parses your JSON body with a json stage at query time and groups by message. |
| OpenTelemetry log export (opt-in) | Each log-record attribute — including exception.* — stored as structured metadata by Loki's OTLP endpoint. | Shape A — grouped by type + message, with the class and full stack. |
So on the default stdout path, shape B is the ceiling: an exception_type
field inside your JSON body is never promoted to a label, and shape A selects
on exception_type != "" as structured metadata — so it can only match logs that
arrived over OpenTelemetry.
Get the best Issues over stdout (shape B) ¶
This is the path most teams are on. Three habits get you the tightest, most useful backend Issues that Loki can give you without OpenTelemetry.
1. Log structured JSON ¶
Shape B is parsed with | json, so your stdout lines must be JSON with a
message (or msg) field and an error-class level. Use your framework's JSON
encoder:
- JVM (Spring Boot / Ktor):
logstash-logback-encoder. - Node.js (Express / Next.js): Pino or Winston in JSON mode.
2. Keep the error message a stable literal ¶
Shape B groups by the message. If you interpolate a case id, URL, or count into the message, every occurrence becomes its own group and one error fragments into dozens of near-duplicate Issues. Keep the message a fixed string and put the variable data in separate JSON fields:
JVM (Kotlin)
// BAD — the id in the message fragments grouping
log.error("Failed to fetch behandling $behandlingId")
// GOOD — stable message; id as a structured field (StructuredArguments.kv)
log.error("Failed to fetch behandling", kv("behandlingId", behandlingId), e)Node.js (Pino)
// BAD — id in the message fragments grouping
logger.error(`Failed to fetch behandling ${behandlingId}`);
// GOOD — stable message, id as a field, error as an object
logger.error({ behandlingId, err }, "Failed to fetch behandling");3. Log the exception as an object ¶
Pass the throwable / Error object to the logger — never a string you built from
it. On the stdout path this puts the exception's stack trace into the JSON
line, which the Issues tab's occurrence drawer reads from a top-level
stack_trace or stack field:
-
logstash-logback-encoderwrites the stack tostack_traceautomatically. - Winston with
format.errors({ stack: true })writes a top-levelstack. - Pino nests it under
err.stack; add a top-levelstack(or a custom serializer) if you want it in the drawer.
What shape B gives you — and what it doesn't
Shape B groups by message only. The exception class is not used for grouping and is not shown in the occurrence detail on this path — only the OpenTelemetry path (shape A) carries the class. So on stdout your two levers are a stable message (tight grouping) and the stack trace in the drawer. That's a solid backend Issue; it just isn't the full shape A.
Reach shape A with OpenTelemetry log export (one option) ¶
If you already use OpenTelemetry logs, or you're willing to dual-ship, exporting logs over OTLP is the only way to reach shape A. The collector sends each log record to Loki's OTLP endpoint, which stores the semantic-convention exception attributes as structured metadata:
-
exception.type→exception_type -
exception.message→exception_message -
exception.stacktrace→exception_stacktrace
The Issues tab then groups on exception_type + exception_message, with the
class and full stack trace in each Issue.
This takes over your logs
OTEL_LOGS_EXPORTER=otlp sends all your logs through the collector and is
not compatible with Team Logs. Your
container also keeps writing to stdout, so lines can land in Loki twice (see
Known limitations). This is why most teams stay
on the stdout path — enable OTLP only if you actively want it.
JVM (Spring Boot and Ktor) ¶
Enable the Java agent's log bridge and log with the throwable. In nais.yaml:
spec:
observability:
autoInstrumentation:
enabled: true
runtime: java
env:
- name: OTEL_LOGS_EXPORTER
value: otlpThe auto-instrumentation Java agent
bridges Logback/Log4j2 to OTel logs; when a log event carries a Throwable, the
appender attaches exception.type/message/stacktrace automatically. Spring Boot
and Ktor both use SLF4J + Logback by default, so logging the throwable is all you
need — no logback.xml change or extra dependency.
Log what escapes your handlers once, with the throwable. Spring — a
@RestControllerAdvice:
@RestControllerAdvice
class GlobalExceptionHandler {
private val log = LoggerFactory.getLogger(javaClass)
@ExceptionHandler(Exception::class)
fun handle(e: Exception): ResponseEntity<String> {
log.error("Unhandled exception", e)
return ResponseEntity.internalServerError().body("Internal error")
}
}Ktor — the StatusPages plugin:
install(StatusPages) {
exception<Throwable> { call, cause ->
call.application.log.error("Unhandled exception", cause)
call.respondText("Internal error", status = HttpStatusCode.InternalServerError)
}
}Confirm your SLF4J binding
The agent can only bridge Logback or Log4j2. If your app uses a different SLF4J binding there is no appender to bridge and you'll stay in shape C. Logback is the Spring Boot and Ktor default, so most apps already have it.
Node.js (Express / Next.js) ¶
runtime: nodejs bundles the Winston and Pino instrumentations, whose
log-sending appenders map a logged Error onto exception.* for you:
spec:
observability:
autoInstrumentation:
enabled: true
runtime: nodejs
env:
- name: OTEL_LOGS_EXPORTER
value: otlpLog the Error object so the appender can read its name, message, and stack:
// Pino: the Error under `err` → exception.type/message/stacktrace
logger.error({ behandlingId, err }, "Failed to fetch behandling");
// Winston: the Error passed alongside the message
// logger.error("Failed to fetch behandling", { behandlingId, err });The four-argument Express error-handling middleware (err, req, res, next),
registered after your routes, is the natural place to catch what escapes a route
(the same appender applies to Next.js route handlers and API routes):
app.use((err, req, res, next) => {
logger.error({ err }, "Unhandled error");
res.status(500).send("Internal Server Error");
});Check your OpenTelemetry versions
The automatic Error → exception.* mapping needs OpenTelemetry JS
api-logs/sdk-logs ≥ 0.212.0 (bundled by the platform nodejs agent). On
older self-managed pins the Error is copied as a generic attribute instead,
leaving you in shape B. You can always set it explicitly by passing the error
as the log record's exception field via @opentelemetry/api-logs.
Field mapping ¶
What the app produces on the OTLP path, and how it lands in the Issues tab:
| OTel log attribute (semconv) | Loki structured metadata | Used by Issues for |
|---|---|---|
exception.type | exception_type | Grouping key + issue title; also gates a line into shape A |
exception.message | exception_message | Grouping key + issue title |
exception.stacktrace | exception_stacktrace | Stack trace shown in the issue |
k8s.pod.name (resource attr) | k8s_pod_name | Pod impact count |
The Issues tab selects shape A with, in effect:
sum by (exception_type, exception_message) (
count_over_time({service_name="<app>", service_namespace="<team>"} | exception_type != "" [$__range])
)Go ¶
Go is a smaller slice of nais apps and largely the platform team's domain, so
briefly: Go uses runtime: sdk — no agent — so your app wires OpenTelemetry
itself. For the stdout path the same shape-B advice applies (structured JSON,
stable message, error as a field). For shape A you build a LoggerProvider with
the OTLP log exporter and the
otelslog
bridge, then set the exception.* attributes yourself — the bridge does not map
errors, and because a Go error carries no stack, exception.stacktrace is
manual and weaker than the JVM's:
slog.ErrorContext(ctx, "failed to fetch behandling",
slog.String("behandlingId", behandlingId),
slog.String("exception.type", fmt.Sprintf("%T", err)),
slog.String("exception.message", err.Error()),
)Log the exception, not a bare message ¶
Whichever path you're on, this is the single biggest reason backend Issues come out thin: hand the logging framework the throwable/error object, not a string you built from it, and keep the message stable.
Log the object, never a pre-formatted string ¶
JVM (Kotlin)
// BAD — class and stack lost inside a string
log.warn("Failed to fetch behandling $behandlingId: ${e.message}")
// GOOD — stable message, throwable as the last arg, id as a field
log.warn("Failed to fetch behandling", kv("behandlingId", behandlingId), e)Node.js (Pino)
// BAD
logger.warn(`Failed to fetch behandling ${behandlingId}: ${err.message}`);
// GOOD
logger.warn({ behandlingId, err }, "Failed to fetch behandling");Don't log self-healing retries at error or warn ¶
A message that logs on every retry and then recovers is still counted as an
Issue — often a top Issue — even though nothing was actually broken. Log
retries at info/debug, and only escalate to error when the final
attempt fails. For example, a client calling the Oppgave API:
async function createOppgave(payload, attempts = 3) {
for (let i = 1; i <= attempts; i++) {
try {
return await oppgaveClient.create(payload);
} catch (err) {
if (i === attempts) {
// only the terminal failure becomes an Issue
logger.error({ err, saksId: payload.saksId, attempts }, "Failed to create oppgave");
throw err;
}
// transient, self-healing — not an Issue
logger.debug({ saksId: payload.saksId, attempt: i }, "Oppgave call failed, retrying");
}
}
}Put context in fields, not in the message ¶
Add identifiers and status as structured fields (saksId, behandlingId,
journalpostId, oppgaveId, status) rather than concatenating them into the
message. They become queryable, and — because grouping is on the message —
keeping variable data out of the message keeps one error from fragmenting into
dozens of near-duplicate Issues.
Never log personally identifying data
Do not put fnr/personident, names, addresses, or other directly
identifying data into log messages or fields. Internal case identifiers like
saksId / behandlingId are acceptable only when they don't themselves
identify the user or leak personal data. When in doubt, leave it out.
Verify it works ¶
-
Trigger the exception in your app (dev is fine).
-
In Grafana Explore, pick your environment's Loki data source and check which shape you're in.
Shape B (stdout):
logql{service_name="<your-app>", service_namespace="<your-team>"} | json | detected_level=~"(?i)error"You should see your JSON body parsed, with a stable
message/msgand — if you logged the object — astack_trace/stackfield.Shape A (OpenTelemetry):
logql{service_name="<your-app>", service_namespace="<your-team>"} | exception_type != ""Expand a line —
exception_type,exception_message, andexception_stacktraceshould appear under structured metadata (not inside the log body). If they're there, you're in shape A. -
Open your service's Issues tab in Nais APM. Within a few minutes the exception should appear as a backend Issue.
If you're on stdout and one error fragments into near-duplicate Issues, your message is carrying variable data — revisit step 2.
Known limitations and cost ¶
- Shape A needs OpenTelemetry. There is no way to reach shape A on the stdout path: fluentbit attaches only Kubernetes fields as structured metadata and never promotes app JSON fields like
exception_type. Merging plaintext stack traces or promotingexception.*/error.*fields in the log pipeline was evaluated and will not be done — the platform decision (Nais APM plugin ADR-0002) is that coherent stack traces come from OTLP (shape A) or structured JSON stdout (shape B), not from log-pipeline merging. So on stdout the ceiling is shape B, and shape A is reached only by enabling OTLP log export. - Log duplication (OTLP). Enabling
OTEL_LOGS_EXPORTER=otlpdoes not stop your container writing to stdout, and fluentbit keeps shipping it — so each line can land in Loki twice, once from stdout and once over OTLP. Keep console logging lean if you turn it on. - A possible duplicate Issue (OTLP). If your stdout logging is JSON and you enable OTLP export, the same exception can surface as two Issues: the rich shape-A Issue (from the OTLP copy) and a weaker message-only shape-B Issue (from the stdout JSON copy).
Related ¶
- How issues are grouped — the fingerprint model behind the type + message grouping.
- Auto-instrumentation reference — the
OTEL_LOGS_EXPORTERopt-in and other agent settings. - Loki labels reference — what structured metadata is, and which fields are available.
- Triage an issue — acting on the issue once it's grouped.