Better Java Logging with Loki and Grafana

The logging options through Render log streams while very easy to setup, unfortunately omit log levels and show everything as an ‘alert’, and overall feels a bit lacking.

If you’re here and using a Java application, I figured out a much better alternative with Loki and Grafana Cloud. The first 50GB of logging metrics is free making it more affordable than any of the partners Render has integrated and the setup is relatively straight forward taking only a bit of tinkering to get working. Plus! It will preserve your applications log formatting and log levels!

By simply adding a Maven dependency and a logback.xml file with a new appender I was able to push my logs to Grafana. More detailed setup guide here: Loki4j Logback · Pure Java Logback appender for Grafana Loki

Example logback.xml file which passes the Grafana API key, username and app name as environment variables. It uses the Spring profile to determine if you want to use the Loki appender - sending logs to Grafana, or a local Spring profile, sending logs to the local console.

Also no changes to the Dockerfile were needed.

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<property name="LOGS" value="./logs"/>

<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
    <encoder>
        <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
    </encoder>
</appender>

<appender name="LOKI" class="com.github.loki4j.logback.Loki4jAppender">
    <http>
        <url>https://logs-prod3.grafana.net/loki/api/v1/push</url>
        <auth>
            <username>${GRAFANA_USERNAME}</username>
            <password>${GRAFANA_API_KEY}</password>
        </auth>
    </http>
    <format>
        <label>
            <pattern>app=${APP_NAME},host=${APP_NAME},level=%level</pattern>
        </label>
        <message>
            <pattern>l=%level h=${APP_NAME} c=%logger{20} t=%thread | %msg %ex</pattern>
        </message>
        <sortByTime>true</sortByTime>
    </format>
</appender>

<springProfile name="sandbox">
    <root level="DEBUG">
        <appender-ref ref="LOKI"/>
    </root>
</springProfile>

<springProfile name="prod">
    <root level="DEBUG">
        <appender-ref ref="LOKI"/>
    </root>
</springProfile>

<springProfile name="local-prod">
    <root level="DEBUG">
        <appender-ref ref="STDOUT" />
    </root>
</springProfile>

</configuration>