[BBLISA] Re: Large scale log processing

[Mike Devlin]

you can setup filters, and logging templates in syslog-ng, so you can split
up the the log entries for each host into their own log files. As an
example, on our syslog-ng server, we have the logging destination setup as:
file ( "/path/$FULLHOST_FROM/$FULLHOST_FROM-$YEAR-$MONTH-$DAY.log"

so in the directory structure, all the entries that are received are split
up per host per day into separate logs.


Mike Devlin
Manager of Operations
boston.com



On Fri, May 15, 2009 at 12:29 PM, Mike Sprague <mfs at komerex.com> wrote:

> Very rough guess, about 100 million lines/day from both mail and web.
> Though they would be broken up into various 'classes'.  For example, I
> would expect about 10 million lines/day from our outgoing mail servers
> and I would want them to be considered separate from our incoming servers.
>
> Thanks for your input!
>
> mikeS
>
> --
> Michael F. Sprague
> mfs at komerex.com
>
> _______________________________________________
> bblisa mailing list
> bblisa at bblisa.org
> http://www.bblisa.org/mailman/listinfo/bblisa
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.bblisa.org/pipermail/bblisa/attachments/20090515/95bd5cc5/attachment.htm