Step 14: Logs & Diagnostics

1:51

Understanding ControlBird's Logging System

ControlBird maintains logs at multiple levels, from low-level service operations to protocol communications to script execution. These logs are essential for troubleshooting, performance monitoring, and understanding system behavior.

Viewing Instance Logs in Control Plane

The Control Plane provides a quick way to view recent logs for any node. Navigate to your node's detail page and expand the Instance Logs section.

Click to enlarge

Service Logs

Each ControlBird service writes structured logs to its own file. These logs track service startup, configuration changes, errors, and operational events.

Log File Organization

Service logs are stored in the node's data directory, organized by service name:

Log Format

Every service log line carries a timestamp, a severity level, a message, and any details as key=value pairs. Services log at the INFO level and above:

time=2026-01-30T15:23:45.123Z level=INFO msg="kernel requested a full sync" peer=node-b
time=2026-01-30T15:23:46.456Z level=INFO msg="kernel served a full sync" peer=node-b sync=7 chunks=12 bytes=1843200
time=2026-01-30T15:24:12.789Z level=WARN msg="kernel checkpoint failed" error="no space left on device"

Log Rotation

The kernel log rotates automatically to prevent unbounded disk usage:

When the kernel log reaches the size limit, it is rotated and a new file is started. The oldest files beyond the retained count are automatically deleted.

Communication Logs

Protocol endpoints can log all transmitted and received data, which is a great help when debugging device communication issues.

Enabling Communication Logging

Each endpoint has logging controls in the Database Browser:

Click to enlarge

Communication Log Format

Communication logs show each transmitted (TX) and received (RX) message with timestamps. For binary protocols like Modbus, data is displayed in hexadecimal:

[2024-01-30 15:23:45.123] [TX] 01 03 00 00 00 0A C5 CD
[2024-01-30 15:23:45.156] [RX] 01 03 14 00 64 00 C8 01 2C ...
[2024-01-30 15:23:46.123] [TX] 01 06 00 10 00 32 C9 C3
[2024-01-30 15:23:46.145] [RX] 01 06 00 10 00 32 C9 C3

For text-based protocols like MQTT, messages are shown as readable text:

[2024-01-30 15:23:45.123] [TX] SUBSCRIBE sensors/temperature/#
[2024-01-30 15:23:45.234] [RX] SUBACK
[2024-01-30 15:23:46.456] [RX] PUBLISH sensors/temperature/living-room {"value": 72.5}

Bandwidth Monitoring

Each endpoint also tracks cumulative bandwidth usage in real-time fields:

These counters are useful for monitoring bandwidth consumption, detecting communication issues, and capacity planning.

Program Logs

Scripts running in the Script Engine capture all console.log() output to individual log files, plus track execution status in entity fields.

Viewing Script Output

In the Script Manager app, click on any program and select the Log tab to see its output:

Click to enlarge

Console Methods

Scripts support standard console methods with different severity levels:

// Standard logging
console.log("Processing started");
console.log("Temperature:", store.read('/Sensors/Temp1/Value'));

// Warnings (non-critical issues)
console.warn("Sensor offline, using cached value");

// Errors (critical problems)
console.error("Failed to write to device:", error.message);

Execution Status Fields

Every Program entity tracks execution metrics in real-time fields:

Diagnostic Workflow

When troubleshooting issues, work through these log sources systematically:

Common Issues & Solutions

Environment Variables

For self-hosted deployments, these environment variables control how the kernel log rotates: