[Ukfreebsd] New Syslog server suggestions

[Vincent Hoffman]

On 04/05/2013 16:58, James O'Gorman wrote:
> On Sat, May 04, 2013 at 04:28:23PM +0100, O'Connor, Kevin wrote:
>> The bottle neck is definitely on the search side. The database logs
>> several million events a day and there is a requirement to maintain 90
>> days of records in the live system. rsyslog does a brilliant job of
>> putting the events into the mysql database but the search side is
>> painful especially when we get requests for events that could have
>> occurred at any time in the last 90 days. Add to that the fact that
>> the people who are doing the search have never been near a *nix box so
>> need a web interface and you get some idea of the problem. 
>>
>> I had hoped someone on the list had a large syslog box up and had
>> worked out how to make it fly.
> Have you looked at Logstash [ http://logstash.net/ ]? It stores data in
> Elasticsearch rather than a relational database so is much faster.
>
> I haven't set this up at $dayjob yet but it's definitely on the todo
> list. There's a nice frontend for it called Kibana.
This is on my todo list also, along with looking at

https://code.google.com/p/enterprise-log-search-and-archive/
which claims to be pretty fast for logging and searching.


Vince


>
> James
> _______________________________________________
> Ukfreebsd mailing list
> Ukfreebsd at uk.freebsd.org
> http://mailman.uk.freebsd.org/mailman/listinfo/ukfreebsd
> UK BSD Groups http://www.bsdgroups.org.uk
> Hosted by EXOnetric http://www.exonetric.net
>