Pflogsumm and gzip'd logs

[Oliver Marshall]

Does anyone know if its possible to get Pflogsumm or any other OS simple
postfix stats script, to use gzip'd logs as its source? For the sake of
disk space, we need to use logrotate to zip the logs each night, but
this then mucks up Pflogsumms output as it just reads 0 for everything.

Can anyone help ?

Olly

[Ralf Hildebrandt]

Post by Oliver Marshall
Does anyone know if its possible to get Pflogsumm or any other OS simple
postfix stats script, to use gzip'd logs as its source? For the sake of
disk space, we need to use logrotate to zip the logs each night, but
this then mucks up Pflogsumms output as it just reads 0 for everything.
Can anyone help ?

zcat mail.log.gz | pflogsumm.pl

[Oliver Marshall]

Thanks Ralf. Just to make it clear for me (as im not a linux bod, gimme
a start menu any day:) ), I currently use:

/usr/bin/pflogsumm.pl /var/log/maillog <switches go here>

Can I just use

zcat mail.log.gz | pflogsumm.pl <switches go here>

In its place in crontab ?

-----Original Message-----
From: Ralf Hildebrandt [mailto:***@charite.de]
Sent: 29 July 2005 12:28
To: postfix-***@postfix.org
Subject: Re: Pflogsumm and gzip'd logs

Post by Oliver Marshall
Does anyone know if its possible to get Pflogsumm or any other OS
simple postfix stats script, to use gzip'd logs as its source? For the
sake of disk space, we need to use logrotate to zip the logs each
night, but this then mucks up Pflogsumms output as it just reads 0 for

everything.

Post by Oliver Marshall
Can anyone help ?

zcat mail.log.gz | pflogsumm.pl

[Ralf Hildebrandt]

Post by Oliver Marshall
Thanks Ralf. Just to make it clear for me (as im not a linux bod, gimme
/usr/bin/pflogsumm.pl /var/log/maillog <switches go here>
Can I just use
zcat mail.log.gz | pflogsumm.pl <switches go here>
In its place in crontab ?

Yes.

[Cami]

Post by Oliver Marshall
Does anyone know if its possible to get Pflogsumm or any other OS simple
postfix stats script, to use gzip'd logs as its source? For the sake of
disk space, we need to use logrotate to zip the logs each night, but
this then mucks up Pflogsumms output as it just reads 0 for everything.
Can anyone help ?

zcat /path/to/log.gz | pflogsumm

Cami

[Oliver Marshall]

Thanks Ralf. Just to make it clear for me (as im not a linux bod, gimme
a start menu any day:) ), I currently use:

/usr/bin/pflogsumm.pl /var/log/maillog <switches go here>

Can I just use

zcat mail.log.gz | pflogsumm.pl <switches go here>

In its place in crontab ?

-----Original Message-----
From: Ralf Hildebrandt [mailto:***@charite.de]
Sent: 29 July 2005 12:28
To: postfix-***@postfix.org
Subject: Re: Pflogsumm and gzip'd logs

Post by Oliver Marshall
Does anyone know if its possible to get Pflogsumm or any other OS
simple postfix stats script, to use gzip'd logs as its source? For the
sake of disk space, we need to use logrotate to zip the logs each
night, but this then mucks up Pflogsumms output as it just reads 0 for

everything.

Post by Oliver Marshall
Can anyone help ?

zcat mail.log.gz | pflogsumm.pl

[D.J. Fan]

Post by Oliver Marshall
Thanks Ralf. Just to make it clear for me (as im not a linux bod, gimme
/usr/bin/pflogsumm.pl /var/log/maillog <switches go here>
Can I just use
zcat mail.log.gz | pflogsumm.pl <switches go here>
In its place in crontab ?

Yes.

I have the problem where my logs rotate (and compress) a couple times a day,
so I made a script to concatenate the 3 latest log files (or however many
that will insure you get a couple days worth of data), called it pflogsumm,
placed it in /etc/cron.daily and made it executable. I'm sure this could be
optimized a little better.

#!/bin/sh
zcat /var/log/mail.log.1.gz >/var/log/maillog1
zcat /var/log/mail.log.2.gz >/var/log/maillog2
cat /var/log/mail.log /var/log/mail.log.0 /var/log/maillog1
/var/log/maillog2 >/var/log/maillog4
/usr/sbin/pflogsumm.pl -i -d yesterday --problems_first /var/log/maillog4
2>&1 |/usr/bin/mail -s "DAILY mail summary" root
rm /var/log/maillog1
rm /var/log/maillog2
rm /var/log/maillog4

This example will wrap, see it unwrapped here:
http://www200.pair.com/mecham/spam/pflogsumm

D.J. Fan

_________________________________________________________________
Don’t just search. Find. Check out the new MSN Search!
http://search.msn.click-url.com/go/onm00200636ave/direct/01/

[Ralf Hildebrandt]

Post by D.J. Fan
#!/bin/sh
zcat /var/log/mail.log.1.gz >/var/log/maillog1
zcat /var/log/mail.log.2.gz >/var/log/maillog2
cat /var/log/mail.log /var/log/mail.log.0 /var/log/maillog1 /var/log/maillog2 >/var/log/maillog4

zcat /var/log/mail.log /var/log/mail.log.0 /var/log/mail.log.1.gz /var/log/mail.log.2.gz |
/usr/sbin/pflogsumm.pl -i -d yesterday --problems_first 2>&1 |/usr/bin/mail -s "DAILY mail summary" root

[D.J. Fan]

Post by D.J. Fan

Post by Oliver Marshall
Thanks Ralf. Just to make it clear for me (as im not a linux bod, gimme
/usr/bin/pflogsumm.pl /var/log/maillog <switches go here>
Can I just use
zcat mail.log.gz | pflogsumm.pl <switches go here>
In its place in crontab ?

Yes.

I have the problem where my logs rotate (and compress) a couple times a
day, so I made a script to concatenate the 3 latest log files (or however
many that will insure you get a couple days worth of data), called it
pflogsumm, placed it in /etc/cron.daily and made it executable. I'm sure
this could be optimized a little better.
#!/bin/sh
zcat /var/log/mail.log.1.gz >/var/log/maillog1
zcat /var/log/mail.log.2.gz >/var/log/maillog2
cat /var/log/mail.log /var/log/mail.log.0 /var/log/maillog1
/var/log/maillog2 >/var/log/maillog4
/usr/sbin/pflogsumm.pl -i -d yesterday --problems_first /var/log/maillog4
2>&1 |/usr/bin/mail -s "DAILY mail summary" root
rm /var/log/maillog1
rm /var/log/maillog2
rm /var/log/maillog4
http://www200.pair.com/mecham/spam/pflogsumm

Of course, this example will create large files, and if you are really that
short on disk space, may not be appropriate for you.

D.J. Fan

_________________________________________________________________
Don’t just search. Find. Check out the new MSN Search!
http://search.msn.click-url.com/go/onm00200636ave/direct/01/

[D.J. Fan]

Post by D.J. Fan

Post by D.J. Fan
#!/bin/sh
zcat /var/log/mail.log.1.gz >/var/log/maillog1
zcat /var/log/mail.log.2.gz >/var/log/maillog2
cat /var/log/mail.log /var/log/mail.log.0 /var/log/maillog1

/var/log/maillog2 >/var/log/maillog4
zcat /var/log/mail.log /var/log/mail.log.0 /var/log/mail.log.1.gz
/var/log/mail.log.2.gz |
/usr/sbin/pflogsumm.pl -i -d yesterday --problems_first 2>&1 |/usr/bin/mail
-s "DAILY mail summary" root
--

No, the problem with that is this:
# zcat /var/log/mail.log
zcat: /var/log/mail.log: not in gzip format
That is the reason the script was needed.

_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today - it's FREE!
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/

[Ralf Hildebrandt]

Post by D.J. Fan
# zcat /var/log/mail.log
zcat: /var/log/mail.log: not in gzip format
That is the reason the script was needed.

my zcat has not problems with "normal" files...

[Rob Foehl]

Post by Ralf Hildebrandt

Post by D.J. Fan
# zcat /var/log/mail.log
zcat: /var/log/mail.log: not in gzip format
That is the reason the script was needed.

my zcat has not problems with "normal" files...

(cat /var/log/mail.log{,.0}; zcat /var/log/mail.log.[12].gz) |pflogsumm ...

-Rob

[Oliver Marshall]

Thanks Dan. Ordinarily that would be the perfect answer to my prob, but
it would just use too much disk space. I reckon that two days worth
would use up about 1.5gb of space,which is just too much at the moment.

-----Original Message-----
From: D.J. Fan [mailto:***@hotmail.com]
Sent: 29 July 2005 15:13
To: postfix-***@postfix.org
Subject: Re: Pflogsumm and gzip'd logs

Post by Oliver Marshall
Thanks Ralf. Just to make it clear for me (as im not a linux bod,
/usr/bin/pflogsumm.pl /var/log/maillog <switches go here>
Can I just use
zcat mail.log.gz | pflogsumm.pl <switches go here>
In its place in crontab ?

Yes.

I have the problem where my logs rotate (and compress) a couple times a
day, so I made a script to concatenate the 3 latest log files (or
however many that will insure you get a couple days worth of data),
called it pflogsumm, placed it in /etc/cron.daily and made it
executable. I'm sure this could be optimized a little better.

#!/bin/sh
zcat /var/log/mail.log.1.gz >/var/log/maillog1 zcat
/var/log/mail.log.2.gz >/var/log/maillog2 cat /var/log/mail.log
/var/log/mail.log.0 /var/log/maillog1
/var/log/maillog2 >/var/log/maillog4
/usr/sbin/pflogsumm.pl -i -d yesterday --problems_first
/var/log/maillog4
2>&1 |/usr/bin/mail -s "DAILY mail summary" root
rm /var/log/maillog1
rm /var/log/maillog2
rm /var/log/maillog4

This example will wrap, see it unwrapped here:
http://www200.pair.com/mecham/spam/pflogsumm

D.J. Fan

_________________________________________________________________
Don't just search. Find. Check out the new MSN Search!
http://search.msn.click-url.com/go/onm00200636ave/direct/01/

[Jaime Bozza]

Post by D.J. Fan
# zcat /var/log/mail.log
zcat: /var/log/mail.log: not in gzip format
That is the reason the script was needed.

Try adding the option -f or --force to the zcat command. That should
allow zcat to "act" like cat when faced with uncompressed files.


Jaime Bozza

[D.J. Fan]

Post by Jaime Bozza

Post by D.J. Fan
# zcat /var/log/mail.log
zcat: /var/log/mail.log: not in gzip format
That is the reason the script was needed.

Try adding the option -f or --force to the zcat command. That should
allow zcat to "act" like cat when faced with uncompressed files.
Jaime Bozza

Thanks Ralf and Jaime, this trims it down, and works great for me:

zcat -f /var/log/mail.log /var/log/mail.log.0 /var/log/mail.log.1.gz
/var/log/mail.log.2.gz |/usr/sbin/pflogsumm.pl -i -d yesterday
--problems_first 2>&1 |/usr/bin/mail -s "DAILY mail summary" root

I have no idea if large amounts of disk space are still temporarily used
during the unpack process.

D.J. Fan

_________________________________________________________________
Don’t just search. Find. Check out the new MSN Search!
http://search.msn.click-url.com/go/onm00200636ave/direct/01/

[Magnus Bäck]

On Friday, July 29, 2005 at 17:50 CEST,

Post by D.J. Fan
zcat -f /var/log/mail.log /var/log/mail.log.0 /var/log/mail.log.1.gz
/var/log/mail.log.2.gz |/usr/sbin/pflogsumm.pl -i -d yesterday
--problems_first 2>&1 |/usr/bin/mail -s "DAILY mail summary" root
I have no idea if large amounts of disk space are still temporarily
used during the unpack process.

zcat does not use any disk space at all when writing output to stdout.

[Matthias Andree]

Post by Oliver Marshall
Does anyone know if its possible to get Pflogsumm or any other OS simple
postfix stats script, to use gzip'd logs as its source? For the sake of
disk space, we need to use logrotate to zip the logs each night, but
this then mucks up Pflogsumms output as it just reads 0 for everything.
Can anyone help ?

Can't logrotate call a script instead of gzip that would first call
pflogsumm and then gzip and rotate logs?

Mine apparently can, it allows to specify a compress command, and that's
the hook where you haul your pflogsumm aboard.

[Terry Gilsenan]

Post by Matthias Andree

Post by Oliver Marshall
Does anyone know if its possible to get Pflogsumm or any other OS simple
postfix stats script, to use gzip'd logs as its source? For the sake of
disk space, we need to use logrotate to zip the logs each night, but
this then mucks up Pflogsumms output as it just reads 0 for everything.
Can anyone help ?

Can't logrotate call a script instead of gzip that would first call
pflogsumm and then gzip and rotate logs?
Mine apparently can, it allows to specify a compress command, and that's
the hook where you haul your pflogsumm aboard.

I just tried this...:

# gunzip -c /var/log/maillog.1.gz | /usr/sbin/pflogsumm.pl

....and it worked fine.

but as always YMMV

[D.J. Fan]

Post by Magnus Bäck

Post by D.J. Fan
zcat -f /var/log/mail.log /var/log/mail.log.0 /var/log/mail.log.1.gz
/var/log/mail.log.2.gz |/usr/sbin/pflogsumm.pl -i -d yesterday
--problems_first 2>&1 |/usr/bin/mail -s "DAILY mail summary" root
I have no idea if large amounts of disk space are still temporarily
used during the unpack process.

zcat does not use any disk space at all when writing output to stdout.
--
Magnus Bäck

That's great, thanks for checking that out Magnus.

It does not matter to pflogsumm, but I thought I would mention that to
maintain correct chronological order, I found the order of the files should
be reversed.

zcat -f mail.log.2.gz mail.log.1.gz mail.log.0 mail.log

_________________________________________________________________
Don’t just search. Find. Check out the new MSN Search!
http://search.msn.click-url.com/go/onm00200636ave/direct/01/