Commit 3336854d authored by Jordan Sissel's avatar Jordan Sissel
Browse files

update readme to better reflect reality

parent 351a8366
......@@ -6,33 +6,35 @@ Collect logs locally in preparation for processing elsewhere!
Problem: logstash jar releases are too fat for constrained systems.
Goal: Something small, fast, and light-weight to ship local logs externally.
## Goal s
## Requirements
* minimal resource usage
* minimize resource usage where possible (cpu, memory, network)
* secure transmission of logs
* configurable event data
* encryption and compression
* easy to deploy with minimal moving parts.
Simple inputs only:
* follow files, respect rename/truncation conditions
* local sockets, maybe, if syslog(3) is worth supporting.
* stdin, useful for things like 'varnishlog | lumberjack ...'
Simple outputs only:
## Implementation details
Below is valid as of 2012/09/19
### Minimize resource usage
* custom wire event protocol (TBD)
* sets small resource limits (memory, open files) on start up based on the number of files being watched
* cpu: sleeps when there is nothing to do
* network/cpu: sleeps if there's a network failure
* network: uses zlib for compression
## Tentative idea:
### secure transmission
# Ship apache logs in real time to somehost:12345
./lumberjack --target somehost:12345 /var/log/apache/access.log ...
* uses openssl to transport logs. Currently supports verifying the server
certificate only (so you know who you are sending to).
# Ship apache logs with additional log fields:
./lumberjack --target foo:12345 --field host=$HOSTNAME --field role=apt-repo /mnt/apt/access.log
### configurable event data
* Serialization: msgpack (likely)
* Encryption: SSL
* Authentication (both directions): SSL certificates
* Compression: TLS v1 comes with compression, might be sufficient.
* the protocol lumberjack uses supports sending a string:string map
* the lumberjack tool lets you specify arbitrary extra data with `--field name=value`
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment