WHAT'S NEW?
- 2002-09-18
- arclog.pl v2.1.0dev2 is released. This release corrects several
documentation errors. It also adds SourceForge in the documentations
as one of the sources of arclog.pl. You can think of it as a
"Source Forge Memorial Release". :p You don't have to
upgrade to this version in a rush. You can click here
to download.
- 2002-09-17
- arclog.pl is hosted at SourceForge now!
Congratulations! (Although I'm still trying hard to get it working
at this time... ^^; )
- 2002-08-31
- arclog.pl v2.1.0dev1 is released. This release fixes the problem
that it wouldn't run with perl 5.8.0. I marked it as "dev"
because I wish I can implement
Compress::Bzip2 and
ExtUtils::MakeMaker
in the 2.1.0 stable release. But it is is in fact very stable. You
can click here
to download.
DESCRIPTION
arclog.pl archives the log files monthly. It strips off the log
entries that belongs to the previous months, and save them to the
archived files named logfile.yyyymm. The archive files
will be compressed. The entries of the current month will be kept in
the source. It then saves disk spaces and prevents potential
attacks on log files.
Current, only the Apache access log and Syslog formats are
recognized.
CAUTION
- Archiving takes time. From version 2.0.0, it will copy
the source log file to a temporary working file and restart it first.
Then it can take its time working on that temporary working file.
This can maximum reduce the time occupying the source log file,
avoiding the need of foreign log file restarting program. However,
please note: (1) If you have a huge log file (several tenths of
MBs), it still sucks copying. You had better stop logging first,
archive the log file and restart logging, to avoid racing condition
in writing. If you archive the log file periodly, it shall not grow
too big. (2) If it stops in the middle of execution, it will leave
a temporary working file. The next time it runs, it will stop when
it sees that temporary working file at the first sight. You have to
process that file first in order to keep working. You can change it
to a different filename and archive it like an ordinary log file, to
solve this.
- You can optionally sort the records in the archived files by
time (and then by record order), to solve some problems on merged log
files. But BE CAUTIONED: Sorting eats huge memory and
CPU. If you work on large log files, it may kill the
whole system in one minute. I had made some simple steps to avoid
that, but I CAN'T GUARANTEE THAT IT WORKS. Whether it kills
depends on which CPU you have, how much memory you have
and how large your archived file is. Because of this, sorting is by
default turned off.
- Be careful on the Syslog log files: Syslog does not record the
year. arclog.pl uses Date::Parse to parse the date, which smartly
judges the year between this month and last next month. For ex., if
today is 2001-06-08, it will then judges the year between 2001-06-30
back to 2000-07-01 if the year is missing. I think this is smart
enough. However, if you do have a Syslog log file that has records
older than one year, don't use arclog.pl. It will destroy your log
file.
- From version 1.5.0 you can specify "-" to
read from STDIN. But notice here: (1) You MUST
specify the output prefix if you want to read from STDIN,
since what it needs is an output pathname prefix, not an output file.
(2) If you read from a file, the records of this month will be
returned to that file, to be archived next month; but if you read from
STDIN, there's no place to return the records, so the
records of this month will be archived, too. (3) The answers of the
ask mode is obtained from STDIN, too.
Since you have only one STDIN, you cannot specify the
ask mode while reading from STDIN. It will
fall back to the fail mode in that case.
SYSTEM REQUIREMENT
- Perl, version 5.005 or above. This program uses the File::Temp
module, which requires some new features introduced from perl 5.005.
Perl is distributed with most Unix now. If not, or if you have an
older version of perl, you can download and install/upgrade it from
http://www.perl.com/.
If you are using MS-Win32, you can download and install from
http://www.activestate.com/.
- Required perl modules:
File::Spec::Functions,
File::Temp,
Date::Parse,
Compress::Zlib.
You can always search, download and install the missing perl modules
from the the CPAN archieve:
http://search.cpan.org/.
DOWNLOAD
arclog.pl's official websites at...
You can always download the newest version of arclog.pl from...
imacat's PGP public key at...
INSTALL
-
Extract the downloaded tar ball
% tar -xzf arclog.pl-x.xx.xx.tar.gz
-
Install using the installation script (recommended)
Although arclog.pl needs no configuration, it is still recommended
that you run the installation script. It will check anything that is
required by arclog.pl and warn you in advance if something is missing.
You'll have to install
File::Spec::Functions
first in order to run the installation script. It is included in the
perl distributions from v5.6.0. If you don't have it, you can
download and install it from the CPAN archive:
http://search.cpan.org/.
Goto the extracted directory and run the Install.PL:
% ./Install.PL
It will check everything it needs and prompt you for the necessary
settings. Answer them according to your needs. Then it will install
the program according to your answers. You can also try
% ./Install.PL --help
for a list of available command line options.
-
Manual set up and install
Copy the arclog.pl to your preferred directory and
run it. That's all.
-
Run
Run it and see if anything goes wrong. You can try:
% ./arclog.pl --help
for a list of available command line options. Try:
% perldoc arclog.pl
for a complete documentation.
OPTIONS
./arclog.pl [options] logfile... [output]
./arclog.pl [-h|-v]
- logfile
- The log file to be archived. Specify "-"
to read from STDIN. Multiple log files are supported.
Gzipped files are supported, too.
- output
- The prefix of the output files. The output files will be named as
output.yyyymm, ie: output.200101,
output.200102. If not specified, the default is the same
as the log file. You must specify this if you want to read from
STDIN. You cannot specify "-"
(STDIN), since this is only a name prefix, not the
output file.
- -c, --compress
- Compress the archived files. This is the default. Log files
usually have a large number of simular lines. Compress them saves
you lots of disk spaces. (And this is why we want to
archive the log files.) Currently only the gzip compressing
method is supported.
- --nocompress
- Don't compress the archived files. (Why? :p )
- -s, --sort
- Sort the records by time (and then the record order). Sorting
eats huge memory and CPU, so it is disabled by default.
See the description above for a detailed illustration on sorting.
- --nosort
- Don't sort the records. This is the default.
- -o, --override[=mode]
- Whether we should overwrite the existing archived files.
Currently the following modes are supported:
- o, overwrite
- Overwrite existing target files. If --override was
specified without mode, it assumes that you want to
overwrite. You will lost these existing records. Use with care.
This is helpful if you are sure the master log file has the most
complete records.
- a, append
- Append the records to the existing target files. You may destroy
the log file completely by putting irrelevant entries altogether
accidently. Use with care. This is helpful if you append want to
merge 2 or more log files, for ex., 2 log files of different periods.
- i, ignore
- Ignore any existing target file, and discard all the records of
those months. You will lost these log records. Use with care. This
is helpful if you are supplying log records for the missing months,
or if you are merging the log records in a complex manner.
- f, fail
- Stop processing whenever a target file exists, to prevent
destroying existing files by accident. This should be mostly wanted
when run from some automatic mechanism, like crontab. So, this is
the default if no terminal is found at STDIN.
- ask
- Ask you what to do when a target file exists. This should be
most wanted if you are running arclog.pl interactively. So, this is
the default if a terminal is found at STDIN. The answers
are read from STDIN. Since you have only one
STDIN, you cannot specify this mode if you want read the
log file from STDIN. In that case, it will fall back to
the fail mode. Also, if arclog.pl cannot get its answer
from STDIN, for ex., on a closed STDIN like
crontab, it will fall back to fail mode.
- -k, --keep[=mode]
- What to keep in the source file. Current the following modes
are supported:
- a, all
- Keep the source file after records are archived. If
--keep was specified without mode, it
assumes that you want to keep all.
- r, restart
- Restart the source log file after records are archived.
- d, delete
- Delete the source log file after records are archived.
- t, this-month
- Archive and strip records of previous months off from the log
file. Keep the records of this month in the source log file, to be
archived next month. This is designed to be run from crontab
monthly, so this is the default.
- -d, --debug
- Show the detailed debugging messages.
- -q, --quiet
- Shihhhhhh. Only yell when errors.
- -h, --help
- Display the help message and exit.
- -v, --version
- Output version information and exit.
COPYRIGHT
Copyright © 2001-2002 imacat. All rights reserved.
This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General
Public License as published by the Free Software Foundation;
either version 2 of the License, or (at your option) any later
version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty
of MERCHANTABILITY or FITNESS FOR A PARTICULAR
PURPOSE. See the GNU General Public
License for more details.
You should have received a copy of the GNU
General Public License along with this program; if not, write
to the Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA
02139, USA.
RELEASE NOTES
Please read the RELEASE_NOTES for the new functions and bug fixes.