Idea #21646
closed
Investigate capabilities of logging systems
Added by Peter Amstutz 7 months ago.
Updated about 1 month ago.
Description
Things we'd like to know:
- Structured logging / ability to search specific fields
- Size limits
- Performance (how many log entries per second can it handle)
- Remote capabilities (want to be able to send logs to a central host)
- How quickly do logs appear on the remote host
- Ease of integration with standard logging libraries we use for Go/Python
- CLI and web UI for doing queries
- Archiving / log rotation
- Integration with cloud provider (AWS) log management
- Ease of deployment
The main options we know of are syslog, journald and AWS logwatch
- Description updated (diff)
- Description updated (diff)
Just want to nominate a project worth looking into, I've used it a long long time ago and it was great: https://www.fluentd.org
Answering the question in the description for Loki:
- Structured logging / ability to search specific fields
yes
don't know
- Performance (how many log entries per second can it handle)
don't know
- Remote capabilities (want to be able to send logs to a central host)
Yes
- How quickly do logs appear on the remote host
don't know
- Ease of integration with standard logging libraries we use for Go/Python
Available for Go, haven't looked at Python yet.
- CLI and web UI for doing queries
yes. CLI client exist, WebUI is integrated with Grafana.
yes, has retention rules https://grafana.com/docs/loki/latest/operations/storage/retention/
- Integration with cloud provider (AWS) log management
Seems like there's work on integration with OTel.
Has published deb packages.
Need to address access control.
- Status changed from New to Resolved
- Target version deleted (
Future)
Also available in: Atom
PDF