Apache & Loging Web Farm

Apache & Loging Web Farms I have multiple load-balanced servers and I want to generate one web report for the whole farm, but each server generates its own log file Webalizer will not work the log files are 500Gb's each I get about 160 hits a sec Is there some other software out there what will work with very big logs?

Everything Linux 1798 This topic was started by ,


data/avatar/default/avatar22.webp

3 Posts
Location -
Joined 2004-01-01
Apache & Loging Web Farms
 
I have multiple load-balanced servers and I want to generate one web report for the whole farm, but each server generates its own log file
 
Webalizer will not work the log files are 500Gb's each I get about 160 hits a sec
 
Is there some other software out there what will work with very big logs?

Participate on our website and join the conversation

You have already an account on our website? Use the link below to login.
Login
Create a new user account. Registration is free and takes only a few seconds.
Register
This topic is archived. New comments cannot be posted and votes cannot be cast.

Responses to this topic


data/avatar/default/avatar39.webp

336 Posts
Location -
Joined 2004-07-09
first off, you should read this document written by an admin of a site getting
about 45 million hits a day.
 
next, read this document written by an admin who has set up a multi-system-one-logfile
configuration. Note the use of a pipe as the log destination.
 
The general Apache documentation section on logs will round out your knowledge.
 
Basically, you'll need to have Apache pipe all of the logging to the syslog daemon.
 
With the logging all going to syslog, you set each system in the farm to send logging
to the one log host you designate, whose own syslog receives the log data via a LAN.