Get started with Team LogsΒΆ
Team Logs is a private logging solution leveraging Google Cloud Logs. It provides your team with a secure, isolated log storage system, ensuring logs are not mixed with those of other teams.
How to Configure Team LogsΒΆ
To send logs to your team's private index, configure your application to use the team-logs
appender. This configuration is typically done in your logback.xml
or log4j.xml
.
<configuration>
<appender name="team-logs" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
<destination>team-logs.nais-system:5170</destination>
<encoder class="net.logstash.logback.encoder.LogstashEncoder">
<customFields>{"google_cloud_project":"${GOOGLE_CLOUD_PROJECT}","nais_namespace_name":"${NAIS_NAMESPACE}","nais_pod_name":"${HOSTNAME}","nais_container_name":"${NAIS_APP_NAME}"}</customFields>
<includeContext>false</includeContext>
</encoder>
<filter class="ch.qos.logback.core.filter.EvaluatorFilter">
<evaluator class="ch.qos.logback.classic.boolex.OnMarkerEvaluator">
<marker>TEAM_LOGS</marker>
</evaluator>
<OnMatch>ACCEPT</OnMatch>
<OnMismatch>DENY</OnMismatch>
</filter>
</appender>
<appender name="default-json" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="net.logstash.logback.encoder.LogstashEncoder" />
</appender>
<root level="INFO">
<appender-ref ref="default-json" />
<appender-ref ref="team-logs" />
</root>
<logger name="team-logs-logger" level="INFO" additivity="false">
<appender-ref ref="team-logs" />
</logger>
</configuration>
You also need to add the following dependency to your pom.xml
:
<Configuration>
<Appenders>
<Console name="default-json" target="SYSTEM_OUT">
<JsonLayout compact="true" />
</Console>
<Socket name="team-logs" host="team-logs.nais-system" port="5170" protocol="tcp">
<JsonLayout compact="true">
<KeyValuePair key="google_cloud_project" value="${env:GOOGLE_CLOUD_PROJECT}"/>
<KeyValuePair key="nais_namespace_name" value="${env:NAIS_NAMESPACE}"/>
<KeyValuePair key="nais_pod_name" value="${env:HOSTNAME}"/>
<KeyValuePair key="nais_container_name" value="${env:NAIS_APP_NAME}"/>
</JsonLayout>
<Filters>
<MarkerFilter marker="TEAM_LOGS" onMatch="ACCEPT" onMismatch="DENY" />
</Filters>
</Socket>
</Appenders>
<Loggers>
<Logger name="team-logs-logger" level="info" additivity="false">
<AppenderRef ref="team-logs"/>
</Logger>
<Root level="info">
<AppenderRef ref="default-json"/>
<AppenderRef ref="team-logs"/>
</Root>
</Loggers>
</Configuration>
Info
The referenced environment variables are set automatically in your application, so you don't need to configure them manually.The referenced environment variables are set automatically in your application, you do not need to set them manually.
This configuration ensures that all logs are sent exclusively to your team's private log index on Google Cloud.
Other Logging LibrariesΒΆ
You can use other logging libraries that support JSON format and TCP/HTTP output.
- TCP:
team-logs.nais-system:5170
- HTTP:
team-logs.nais-system:9880
The format for the logs should be JSON, and you must include the following fields:
Field Name | Description | Environment Variable |
---|---|---|
google_cloud_project | The Google Cloud project ID | GOOGLE_CLOUD_PROJECT |
nais_namespace_name | The namespace of the application | NAIS_NAMESPACE |
nais_pod_name | The name of the pod | HOSTNAME |
nais_container_name | The name of the container | NAIS_APP_NAME |
message | The log message | |
severity | The log level (e.g., INFO, ERROR) |
Migrating from Secure LogsΒΆ
If you are migrating from Secure Logs to Team Logs, you need to ensure that your application is configured to send logs to the team-logs
appender instead of the Secure Logs appender. This involves updating your logging configuration files (logback.xml
or log4j2.xml
) and ensuring that the necessary dependencies are included in your project.
You should also remove any references to Secure Logs in your nais.yaml
configuration - specifically the secureLogs
section.
Writing LogsΒΆ
To ensure that your logs are sent to the team-logs
appender, you can use SLF4J markers or loggers. Below are detailed examples of how to do this in both Logback and Log4j2.
If you are using Logback, you can use the team-logs
appender in your logging configuration. You can also use SLF4J markers to route specific log messages to the team-logs
appender.
Below is an example in Kotlin using an SLF4J marker to ensure your log messages are routed to the "team-logs" appender:
import org.slf4j.LoggerFactory
import org.slf4j.MarkerFactory
class MyClass {
private val logger = LoggerFactory.getLogger(MyClass::class.java)
// Create a marker that your Logback configuration recognizes for "team-logs"
private val teamLogsMarker = MarkerFactory.getMarker("TEAM_LOGS")
fun logMessages() {
// Log to the default JSON appender
logger.info("This log goes to the default JSON appender")
// Log to the team-logs appender using the TEAM_LOGS marker
logger.info(teamLogsMarker, "This log goes to the team-logs appender")
// Log an error to the team-logs appender
logger.error(teamLogsMarker, "This is an error message for team-logs")
}
}
Ensure that your logback.xml
configuration includes the team-logs
appender and the marker-based filter as shown in the configuration section above.
If you are using Log4j2, you can use the team-logs
appender in your logging configuration. You can also use markers to route specific log messages to the team-logs
appender.
Below is an example in Kotlin using a Log4j2 marker to ensure your log messages are routed to the "team-logs" appender:
import org.apache.logging.log4j.LogManager
import org.apache.logging.log4j.MarkerManager
class MyClass {
private val logger = LogManager.getLogger(MyClass::class.java)
private val teamLogsMarker = MarkerManager.getMarker("TEAM_LOGS")
fun logMessages() {
// Log to the default JSON appender
logger.info("This log goes to the default JSON appender")
// Log to the team-logs appender using the TEAM_LOGS marker
logger.info(teamLogsMarker, "This log goes to the team-logs appender")
// Log an error to the team-logs appender
logger.error(teamLogsMarker, "This is an error message for team-logs")
}
}
Ensure that your log4j2.xml
configuration includes the team-logs
appender and the marker-based filter as shown in the configuration section above.
If you are using a logging library that does not support SLF4J or Log4j2, you can still send logs to the team-logs
appender using a simple HTTP request. Below is an example using curl
:
curl -X POST \
-H "Content-Type: application/json" \
-d '{
"google_cloud_project": "my-project",
"nais_namespace_name": "my-namespace",
"nais_pod_name": "my-pod",
"nais_container_name": "my-container",
"message": "This is a log message for team-logs",
"severity": "INFO"
}' \
http://team-logs.nais-system:9880
Key Points for DevelopersΒΆ
- Markers: Use the
TEAM_LOGS
marker to route logs to theteam-logs
appender. Without this marker, logs will go to the default JSON appender. - Error Handling: Always log errors with appropriate severity levels (e.g.,
error
,warn
) to ensure they are easily identifiable. - Structured Logging: Use JSON format for logs to make them easier to query and analyze.
- Configuration: Ensure your logging configuration files (
logback.xml
orlog4j2.xml
) are correctly set up with theteam-logs
appender and marker-based filters.
Accessing Team LogsΒΆ
Team Logs can be accessed through the Google Cloud Console or other tools that support Google Cloud Logging. Note that Team Logs are kept separate from standard log destinations like Grafana Loki and thus arenβt available through its standard logging interface.
Querying Team LogsΒΆ
To search your logs, use the Google Cloud Logging Query Language. Here are some practical examples:
-
Filter by app name:
-
Filter by severity:
-
Combine multiple filters:
-
Filter by a specific time range:
For more detailed guidance, review the following resources:
Log LabelsΒΆ
Team Logs automatically attaches labels to every log entry for better context. Below are some key labels youβll see:
{
"resource": {
"labels": {
"cluster_name": "dev",
"container_name": "my-app",
"location": "gcp",
"namespace_name": "my-team",
"pod_name": "my-app-7789dbcdb4-12345",
"extra_label": "my-label"
},
"type": "k8s_container"
}
}
This structure ensures that you can easily locate and analyze your team's logs with clarity and precision.