README.md 1.02 KB
Newer Older
Jordan Sissel's avatar
-  
Jordan Sissel committed
1
2
3
# lumberjack

Collect logs locally in preparation for processing elsewhere!
Jordan Sissel's avatar
Jordan Sissel committed
4
5
6
7
8

Problem: logstash jar releases are too fat for constrained systems.

Goal: Something small, fast, and light-weight to ship local logs externally.

Jordan Sissel's avatar
Jordan Sissel committed
9
## Requirements
Jordan Sissel's avatar
Jordan Sissel committed
10

Jordan Sissel's avatar
Jordan Sissel committed
11
* minimal resource usage
12
* configurable event data
Jordan Sissel's avatar
Jordan Sissel committed
13
* encryption and compression
Jordan Sissel's avatar
Jordan Sissel committed
14
15
16
17

Simple inputs only:

* follow files, respect rename/truncation conditions
Jordan Sissel's avatar
Jordan Sissel committed
18
19
* local sockets, maybe, if syslog(3) is worth supporting.
* stdin, useful for things like 'varnishlog | lumberjack ...'
Jordan Sissel's avatar
Jordan Sissel committed
20
21
22
23

Simple outputs only:

* custom wire event protocol (TBD)
24
25
26
27
28
29
30
31
32

## Tentative idea:

    # Ship apache logs in real time to somehost:12345
    ./lumberjack --target somehost:12345 /var/log/apache/access.log ...

    # Ship apache logs with additional log fields:
    ./lumberjack --target foo:12345 --field host=$HOSTNAME --field role=apt-repo /mnt/apt/access.log

Jordan Sissel's avatar
Jordan Sissel committed
33
34
35
36
* Serialization: msgpack (likely)
* Encryption: SSL
* Authentication (both directions): SSL certificates
* Compression: TLS v1 comes with compression, might be sufficient.