tutorial · step 5 / 7
How to cap the journal so it does not fill or wear out an SD card: Logging with journald
- hardware
- Raspberry Pi 5
- os
- Raspberry Pi OS Bookworm
- verified
One of the most underappreciated benefits of running your application as a systemd service is that you get a powerful, structured logging system essentially for free. Any text your application prints to stdout or stderr is automatically captured, timestamped, and stored in something called the journal. journal is systemd’s centralised log database. This article covers how to read, filter, and manage those logs, and how to configure the journal to be a responsible citizen on embedded devices with limited storage. Yes, journal works well with embedded devices. But, it works well only if you configure it properly!
How systemd Captures Your Logs
When your service is running, systemd intercepts everything written to stdout and stderr and passes it to systemd-journald. So, you don’t need to change anything in your application code. Just a plain print("sensor reading: 42.3") in Python becomes a structured journal entry with a precise timestamp. This entry comes with the service name, the process ID, and the message text all stored together.
The only service file configuration needed to make this explicit is:
StandardOutput=journal
StandardError=journal
SyslogIdentifier=my-appStandardOutput=journal and StandardError=journal are actually the defaults in modern systemd, so you don’t strictly need to specify them. But being explicit makes the service file self-documenting. Anyone reading it can immediately see where the logs go. SyslogIdentifier sets a short tag attached to every journal entry from this service. This makes it easier to filter for your specific application even if multiple services share similar names.
Reading Logs with journalctl
The journalctl command is our primary interface to the journal. Its most important option for service debugging is -u. It filters entries to a specific unit:
# Show all historical logs for your service
journalctl -u my-app.service
# Follow the log in real time — like 'tail -f' on a log file
journalctl -u my-app.service -f
# Show only logs from the current boot
journalctl -u my-app.service -b
# Show only logs from the previous boot (useful after a crash)
journalctl -u my-app.service -b -1
# Show logs from the last hour
journalctl -u my-app.service --since "1 hour ago"
# Show logs between two specific timestamps
journalctl -u my-app.service --since "2025-12-16 10:00" --until "2025-12-16 11:00"
# Show only error-level and above (useful to find crashes quickly)
journalctl -u my-app.service -p err
# Show the last 50 lines, most recent first
journalctl -u my-app.service -n 50 -rThe option I use frequently is -b -1. It is particularly valuable for embedded systems. If your device rebooted unexpectedly due to a power cut, a kernel panic, you can look at the logs from the previous boot to understand what was happening right before it went down. Magical isn’t it!
What are Log Priority Levels
systemd’s journal understands syslog-compatible priority levels. If your application logs at different severity levels, you can filter journal output to show only entries at or above a certain severity. The levels are written from most to least severe. These are emerg (0), alert (1), crit (2), err (3), warning (4), notice (5), info (6), and debug (7).
For most embedded projects, simplest and most readable approach is Python’s standard logging module:
import logging
import sys
logging.basicConfig(
level=logging.INFO,
format="%(levelname)s: %(message)s",
stream=sys.stdout # systemd captures stdout to the journal
)
logger = logging.getLogger(__name__)
logger.info("Service started successfully")
logger.warning("Sensor reading out of expected range")
logger.error("Failed to connect to broker — will retry")If you use this pattern, your log messages stay clean and consistent. Not only that, you can filter them in the journal by searching for ERROR or WARNING in the message text with journalctl -u my-app.service -g "ERROR".
How to Manage Journal Size on Embedded Devices
This is an important practical concern that most tutorials skip. The Raspberry Pi typically uses an SD card for storage, and SD cards have two relevant limitations. They are finite in capacity, and a limited number of write cycles before they start to fail. By default, systemd’s journal will grow to fill a significant fraction of available disk space. For a small SD card, this will eventually cause problems. If you leave the journal unconfigured on an embedded device, it becomes a recipe for running out of disk space or unnecessarily wearing out the SD card.
The journal daemon is configured in /etc/systemd/journald.conf. Open it with:
sudo nano /etc/systemd/journald.confAdd or uncomment these lines in the [Journal] section:
[Journal]
# Maximum total size of all journal files on disk
SystemMaxUse=50M
# Maximum size of a single journal file
SystemMaxFileSize=10M
# Maximum time to store journal entries (optional, for low-activity devices)
MaxRetentionSec=1monthAfter making changes, restart the journal daemon:
sudo systemctl restart systemd-journald50 megabytes is a reasonable upper bound for most edge devices. It is large enough to retain several days of logs at typical verbosity, and small enough to not strain a 16GB or 32GB SD card. Let’s say your application is particularly chatty, then you may want to reduce this further, or adjust your application’s log verbosity before reaching for the journal size limit. This is something you will learn to adjust by doing.
Check the current journal disk usage at any time with:
journalctl --disk-usageA Side Note For Curious You - A .conf is a configuration file, not a service unit file (.service). journald.conf is a plain text INI file located in /etc/systemd/journald.conf. Its only job is to store settings—like size limits, retention times, and storage behavior—that tell the logging daemon how to behave. The systemd-journald daemon does have its own service file behind the scenes called systemd-journald.service, but you rarely edit that. Instead, you edit journald.conf to tune how that service handles log data.
You might have already guessed by now. We can write a .conf for our myapp.service as well. Instead of modifying our myapp.service file directly, we can create a .conf file to override or add specific settings. This is standard practice in production environments because it keeps the core service file clean. More on this should be in a future article but not this one.
Viewing Logs Without Paging
By default, journalctl opens output in a pager (like less) which requires you to press q to exit. When running quick diagnostic commands over SSH, this can be inconvenient. Add --no-pager to get plain terminal output you can scroll or pipe to grep:
# Print all logs without opening a pager
journalctl -u my-app.service --no-pager
# Find all lines containing "error" (case-insensitive)
journalctl -u my-app.service --no-pager | grep -i error
# Count how many warning lines appeared in the last hour
journalctl -u my-app.service --since "1 hour ago" --no-pager | grep -c WARNINGThe combination of journalctl filtering and standard Unix tools like grep, awk, and wc gives you a surprisingly capable log analysis workflow without needing any additional tooling.
Appendix
Here is the real output from my Raspberry pi.
admin@sbrsl:~ $ journalctl --disk-usage
Archived and active journals take up 8.8M in the file system.sudo nano /etc/systemd/journald.conf# This file is part of systemd.
#
# systemd is free software; you can redistribute it and/or modify it under the
# terms of the GNU Lesser General Public License as published by the Free
# Software Foundation; either version 2.1 of the License, or (at your option)
# any later version.
#
# Entries in this file show the compile time defaults. Local configuration
# should be created by either modifying this file, or by creating "drop-ins" in
# the journald.conf.d/ subdirectory. The latter is generally recommended.
# Defaults can be restored by simply deleting this file and all drop-ins.
#
# Use 'systemd-analyze cat-config systemd/journald.conf' to display the full config.
#
# See journald.conf(5) for details.
[Journal]
Storage=volatile
#Compress=yes
#Seal=yes
#SplitMode=uid
#SyncIntervalSec=5m
#RateLimitIntervalSec=30s
#RateLimitBurst=10000
#SystemMaxUse=
#SystemKeepFree=
#SystemMaxFileSize=
#SystemMaxFiles=100
#RuntimeMaxUse=
#RuntimeKeepFree=
#RuntimeMaxFileSize=
#RuntimeMaxFiles=100
#MaxRetentionSec=
#MaxFileSec=1month
#ForwardToSyslog=yes
#ForwardToKMsg=no
#ForwardToConsole=no
#ForwardToWall=yes
#TTYPath=/dev/console
#MaxLevelStore=debug
#MaxLevelSyslog=debug
#MaxLevelKMsg=notice
#MaxLevelConsole=info
#MaxLevelWall=emerg
#LineMax=48K
#ReadKMsg=yes
#Audit=no
Comments