Commit 6045790f authored by Christoph Berg's avatar Christoph Berg

New upstream version 10.1

parent 5d4fcb20
root = true
[*]
charset = utf-8
indent_style = tab
indent_size = 8
# Unix-style newlines
end_of_line = lf
# Remove any whitespace characters preceding newline characters
trim_trailing_whitespace = true
# Newline ending every file
insert_final_newline = true
[*.yml]
indent_style = space
indent_size = 2
# Swap files
*.swp
blib/
Makefile
MYMETA.json
MYMETA.yml
pm_to_blib
--backup-and-modify-in-place
--backup-file-extension=beforeTidy
--block-brace-tightness=2
--brace-tightness=2
--closing-token-indentation=1
--continuation-indentation=4
--indent-columns=4
--maximum-line-length=134
--cuddled-else
--opening-sub-brace-on-new-line
--noopening-brace-on-new-line
--nooutdent-labels
--paren-tightness=2
--square-bracket-tightness=2
--vertical-tightness=0
--vertical-tightness-closing=0
--break-at-old-comma-breakpoints
--entab-leading-whitespace=4
--tabs
......@@ -4,6 +4,22 @@
1. Upgrade to the latest version of pgBadger and see if the problem remains
2. Look at the [closed issues](https://github.com/dalibo/pgbadger/issues?state=closed), we may have already answered a similar problem
2. Look at the [closed issues](https://github.com/darold/pgbadger/issues?state=closed), we may have already answered a similar problem
3. [Read the doc](http://darold.github.com/pgbadger/). It is short and useful.
## Coding style
pgBadger project provides a [.editorconfig](http://editorconfig.org/) file to
setup consistent spacing in files. Please follow it!
## Keep Documentation Updated
The first pgBadger documentation is `pgbadger --help`. `--help` fills the
SYNOPSIS section in `doc/pgBadger.pod`. The DESCRIPTION section *must* be
written directly in `doc/pgBadger.pod`. `README` is the text formatting of
`doc/pgBadger.pod`. Update `README` and `doc/pgBadger.pod` with `make README`
and commit changes when contributing.
3. [Read the doc](http://dalibo.github.com/pgbadger/). It is short and useful.
2018-09-12 - v10.1
This release of pgBadger is a maintenance release that fixes reports
in incremental mode and multiprocess with -j option. Log parsing from
standard input was also broken. If you are using v10.0 please upgrade
now.
- Add test on pgbouncer log parser.
- Some little performances improvment.
- Fix not a valid file descriptor at pgbadger line 12314.
- Fix unwanted newline in progressbar at startup.
- Remove circleci files from the project.
- Remove dependency of bats and jq for the test suite, they are
replaced with Test::Simple and JSON::XS.
- Add more tests especially for incremental mode and input from
stdin that was broken in release 10.0.
- Sync pgbadger, pod, and README, and fix some syntax errors.
Thanks to Christoph Berg for the patch.
- Add documentation on how to install Perl module JSON::XS from
apt and yum repositories.
- Fix URI for CSS in incremental mode. Thanks to Floris van Nee
for the report.
- Fix fatal error when looking for log from STDIN. Thanks to
Jacek Szpot for the report.
- Fixes SED use for OSX builds. Thanks to Steve Newson for the
patch.
- Fix illegal division by zero in incrental mode. Thanks to
aleszeleny for the report.
- Replace SQL::Beautify with v3.1 of pgFormatter::Beautify.
2018-09-09 - v10.0
This release of pgBadger is a major release that adds some new
features and fix all issues reported by users since last release.
* Add support of pgbouncer syslog log file format.
* Add support to all auto_explain format (text, xml, json and yaml).
* Add support to %q placeholder in log_line_prefix.
* Add jsonlog format of Michael Paquier extension, with -f jsonlog
pgbadger will be able to parse the log.
* Replace the SQL formatter/beautify with v3.0 of pgFormatter.
There is some new command line option:
- Add --prettify-json command line option to prettify JSON output.
- Add --log-timezone +/-XX command line option to set the number
of hours from GMT of the timezone that must be used to adjust
date/time read from log file before beeing parsed. Note that you
might still need to adjust the graph timezone using -Z when the
client has not the same timezone.
- Add --include-time option to add the ability to choose times that
you want to see, instead of excluding all the times you do not
want to see (--exclude-time).
The pgBadger project and copyrights has been transfered from Dalibo
to the author and official maintainer of the project. Please update
your links:
- Web site: http://pgbadger.darold.net/
- Source code: https://github.com/darold/pgbadger
I want to thanks the great guys at Dalibo for all their investments
into pgBadger during these years and especially Damien Clochard and
Jean-paul argudo for their help to promote pgBadger.
- Fix checkpoint distance and estimate not reported in incremental
mode. Thanks to aleszeleny for the report.
- Fix title of pgbouncer simultaneous session report. Thansks to
Jehan Guillaume De Rorthais for the report.
- Add support of pgbouncer syslog log file format. Thanks to djester
for the feature request.
- Fix error when a remote log is empty. Thanks to Parasit Hendersson
for the report.
- Fix test with binary format. Binary file must be generated as it
is dependent of the plateform. Thanks to Michal Nowak for the
report.
- Fix case where an empty explain plan is generated.
- Fix parsing of autodetected default format with a prefix in
command line.
- Remove dependency of git command in Makefile.PL.
- Update documentation about options changes and remove of the
[%l-1] part of the mandatory prefix.
- Fix parsing of vacuum / analyze system usage for PostgreSQL 10.
Thanks to Achilleas Mantzios for the patch.
- Fix Temporary File Activity table.
- Remove dependency to git during install.
- Add --log-timezone +/-XX command line option to set the number
of hours from GMT of the timezone that must be used to adjust
date/time read from log file before beeing parsed. Using this
option make more difficult log search with a date/time because the
time will not be the same in the log. Note that you might still
need to adjust the graph timezone using -Z when the client has not
the same timezone. Thanks to xdexter for the feature request and
Julien Tachoire for the patch.
- Add support to auto_explain json output format. Thanks to dmius
for the report.
- Fix auto_explain parser and queries that was counted twice.
Thanks to zam6ak for the report.
- Fix checkpoint regex to match PostgreSQL 10 log messages. Thanks
to Edmund Horner for the patch.
- Update description of -f | --format option by adding information
about jsonlog format.
- Fix query normalisation to not duplicate with bind queries.
Normalisation of values are now tranformed into a single ? and no
more 0 for numbers, two single quote for string. Thanks to vadv
for the report.
- Fix log level count. Thanks to Jean-Christophe Arnu for the report
- Make pgbadger more compliant with B::Lint bare sub name.
- Made perlcritic happy.
- Add --prettify-json command line option to prettify JSON output.
Default output is all in single line.
- Fix Events distribution report.
- Fix bug with --prefix when log_line_prefix contain multiple %%.
Thanks to svb007 for the report.
- Add --log-timezone +/-XX command line option to set the number
of hours from GMT of the timezone that must be used to adjust
date/time read from log file before beeing parsed. Using this
option make more difficult log search with a date/time because the
time will not be the same in the log. Note that you might still
need to adjust the graph timezone using -Z when the client has not
the same timezone. Thanks to xdexter for the feature request.
- Remove INDEXES from the keyword list and add BUFFERS to this list.
- Fix normalization of query using cursors.
- Remove Dockerfile and documentation about docker run. pgBadger
comes as a single Perl script without any dependence and it can
be used on any plateform. It is a non sens to use docker to run
pgbadger, if you don't want to install anything, just copy the
file pgbadger where you want and execute it.
- Fix broken grid when no temp files activity. Thanks to Pierre
Giraud for the patch
- Add doc warning about log_in_duration_statement vs log_duration +
log_statement. Thanks to Julien Tachoire for the patch.
- Apply timezone offset to bar charts. Thanks to Julien Tachoire
for the patch.
- Delete current temp file info if we meet an error for the same PID
Thanks to Julien Tachoire for the patch.
- Consistently use app= in examples, and support appname=
Some of the usage examples used appname= in the prefix, but the
code didn't recognize that token. Use app= in all examples, and
add appname= to the prefix parser. Thanks to Christoph Berg for
the patch
- Fix wrong long name for option -J that should be --Jobs intead
of --job_per_file. Thanks to Chad Trabant for the report and
Etienne Bersac for the patch.
- Ignore blib files. Thanks to Etienne Bersac for the patch.
- Add consistency tests. Thanks to damien clochard for the patch.
- doc update : stderr is not a default for -f. Thanks to Christophe
Courtois for the patch.
- Always update pod and README. Thanks to Etienne Bersac for
the patch.
- Add some regression tests. Thanks to Etienne Bersac for the patch.
- Add editorconfig configuration. Thanks to Etienne Bersac for the
patch.
- Drop vi temp files from gitignore. Thanks to Etienne Bersac for
the patch.
- Add --include-time option to add the ability to choose times that
you want to see, instead of excluding all the times you do not
want to see. This is handy when wanting to view only one or two
days from a week's worth of logs (simplifies down from multiple
--exlucde-time options to one --include-time). Thanks to Wesley
Bowman for the patch.
- Check pod syntax. Thanks to Etienne Bersac for the patch.
- Add HACKING to document tests. Thanks to Etienne Bersac for the
patch.
- Drop obsolete --bar-graph option. Thanks to Etienne Bersac for
the patch.
- Drop misleading .perltidyrc. This file date from 2012 and
pgbadger code is far from compliant. perltidy unified diff is
10k lines. Let's drop this. Thanks to Etienne Bersac for the
patch.
- Fix use of uninitialized value in SQL formatting. Thanks to John
Krugger for the report and Jean-paul Argudo for the report.
2017-07-27 - v9.2
This release of pgBadger is a maintenance release that adds some new
......
# Contributing on pgBadger
Thanks for your attention on pgBadger ! You need Perl Module JSON::XS
to run the full test suite. You can install it on a Debian like system
using:
sudo apt-get install libjson-xs-perl
or in RPM like system using:
sudo yum install perl-JSON-XS
pgBadger has a TAP compatible test suite executed by `prove`:
$ prove
t/01_lint.t ......... ok
t/02_basics.t ....... ok
t/03_consistency.t .. ok
All tests successful.
Files=3, Tests=13, 6 wallclock secs ( 0.01 usr 0.01 sys + 5.31 cusr 0.16 csys = 5.49 CPU)
Result: PASS
$
or if you prefer to run test manually:
$ perl Makefile.PL && make test
Checking if your kit is complete...
Looks good
Generating a Unix-style Makefile
Writing Makefile for pgBadger
Writing MYMETA.yml and MYMETA.json
cp pgbadger blib/script/pgbadger
"/usr/bin/perl" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/pgbadger
PERL_DL_NONLAZY=1 "/usr/bin/perl" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
t/01_lint.t ......... ok
t/02_basics.t ....... ok
t/03_consistency.t .. ok
All tests successful.
Files=3, Tests=13, 6 wallclock secs ( 0.03 usr 0.00 sys + 5.39 cusr 0.14 csys = 5.56 CPU)
Result: PASS
$ make clean && rm Makefile.old
Please contribute a regression test when you fix a bug or add a feature. Thanks!
Copyright (c) 2012-2017, Dalibo
Copyright (c) 2012-2018, Gilles Darold
Permission to use, copy, modify, and distribute this software and its
documentation for any purpose, without fee, and without a written agreement
is hereby granted, provided that the above copyright notice and this
paragraph and the following two paragraphs appear in all copies.
IN NO EVENT SHALL Dalibo BE LIABLE TO ANY PARTY FOR DIRECT, INDIRECT,
IN NO EVENT SHALL Darold BE LIABLE TO ANY PARTY FOR DIRECT, INDIRECT,
SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING LOST PROFITS,
ARISING OUT OF THE USE OF THIS SOFTWARE AND ITS DOCUMENTATION, EVEN IF
Dalibo HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Darold HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Dalibo SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, BUT NOT LIMITED
Darold SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, BUT NOT LIMITED
TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE SOFTWARE PROVIDED HEREUNDER IS ON AN "AS IS" BASIS, AND Dalibo
PURPOSE. THE SOFTWARE PROVIDED HEREUNDER IS ON AN "AS IS" BASIS, AND Darold
HAS NO OBLIGATIONS TO PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS,
OR MODIFICATIONS.
# http://module-build.sourceforge.net/META-spec.html
#XXXXXXX This is a prototype!!! It will change in the future!!! XXXXX#
name: pgBadger
version: 8.1
version: 10.1
version_from: pgbadger
installdirs: site
recommends:
Text::CSV_XS: 0
JSON::XS: 0
distribution_type: script
generated_by: ExtUtils::MakeMaker version 6.17
use ExtUtils::MakeMaker;
# See lib/ExtUtils/MakeMaker.pm for details of how to influence
# the contents of the Makefile that is written.
use strict;
my @ALLOWED_ARGS = ('INSTALLDIRS','DESTDIR');
# Parse command line arguments and store them as environment variables
while ($_ = shift) {
my ($k,$v) = split(/=/, $_, 2);
if (grep(/^$k$/, @ALLOWED_ARGS)) {
$ENV{$k} = $v;
}
}
$ENV{DESTDIR} =~ s/\/$//;
# Default install path
my $DESTDIR = $ENV{DESTDIR} || '';
my $INSTALLDIRS = $ENV{INSTALLDIRS} || 'site';
my %merge_compat = ();
if ($ExtUtils::MakeMaker::VERSION >= 6.46) {
%merge_compat = (
'META_MERGE' => {
resources => {
homepage => 'http://projects.dalibo.org/pgbadger',
repository => {
type => 'git',
git => 'git@github.com:dalibo/pgbadger.git',
web => 'https://github.com/dalibo/pgbadger',
},
},
}
);
}
WriteMakefile(
'DISTNAME' => 'pgbadger',
'NAME' => 'pgBadger',
'VERSION_FROM' => 'pgbadger',
'dist' => {
'COMPRESS'=>'gzip -9f', 'SUFFIX' => 'gz',
'ZIP'=>'/usr/bin/zip','ZIPFLAGS'=>'-rl'
},
'AUTHOR' => 'Gilles Darold (gilles@darold.net)',
'ABSTRACT' => 'pgBadger - PostgreSQL log analysis report',
'EXE_FILES' => [ qw(pgbadger) ],
'MAN1PODS' => { 'doc/pgBadger.pod' => 'blib/man1/pgbadger.1p' },
'DESTDIR' => $DESTDIR,
'INSTALLDIRS' => $INSTALLDIRS,
'clean' => {},
%merge_compat
);
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
use Test::Simple tests => 2;
my $ret = `perl -wc pgbadger 2>&1`;
ok( $? == 0, "PERL syntax check");
$ret = `podchecker doc/*.pod 2>&1`;
ok( $? == 0, "pod syntax check");
use Test::Simple tests => 8;
my $LOG = 't/fixtures/light.postgres.log.bz2';
my $BIN = 't/fixtures/light.postgres.bin';
my $JSON = 't/out.json';
my $ret = `perl pgbadger --help`;
ok( $? == 0, "Inline help");
$ret = `perl pgbadger -q -o - $LOG`;
ok( $? == 0 && length($ret) > 0, "Light log report to stdout");
`rm -f out.html`;
$ret = `perl pgbadger -q --outdir '.' $LOG`;
ok( $? == 0 && -e "out.html", "Light log report to HTML");
$ret = `perl pgbadger -q -o $BIN $LOG`;
ok( $? == 0 && -e "$BIN", "Light log to binary");
`rm -f $JSON`;
$ret = `perl pgbadger -q -o $JSON --format binary $BIN`;
$ret = `cat $JSON | perl -pe 's/.*"SELECT":(\\d+),.*/\$1/'`;
ok( $? == 0 && $ret > 0, "From binary to JSON");
`mkdir t/test_incr/`;
$ret = `perl pgbadger -q -O t/test_incr -I --extra-files $LOG`;
ok( $? == 0 && -e "t/test_incr/2017/09/06/index.html"
&& -e "t/test_incr/2017/week-37/index.html", "Incremental mode report");
$ret = `grep 'src="../../../.*/bootstrap.min.js"' t/test_incr/2017/09/06/index.html`;
ok( $? == 0 && substr($ret, 32, 14) eq 'src="../../../', "Ressources files in incremental mode");
`rm -f $JSON`;
$ret = `bunzip2 -c $LOG | perl pgbadger -q -o $JSON -`;
$ret = `cat $JSON | perl -pe 's/.*"SELECT":(\\d+),.*/\$1/'`;
ok( $? == 0 && $ret > 0, "Light log from STDIN");
# Remove files generated during the tests
`rm -f out.html`;
`rm -r $JSON`;
`rm -f $BIN`;
`rm -rf t/test_incr/`;
use Test::Simple tests => 6;
use JSON::XS;
my $json = new JSON::XS;
my $LOG = 't/fixtures/light.postgres.log.bz2 t/fixtures/pgbouncer.log.gz';
my $BIN = 'out.bin';
my $OUT = 'out.json';
my $ret = `perl pgbadger -q -o $BIN $LOG`;
ok( $? == 0, "Generate intermediate binary file from log");
$ret = `perl pgbadger -q -o $OUT --format binary $BIN`;
ok( $? == 0, "Generate json report from binary file");
`rm -f $BIN`;
my $json_ref = $json->decode(`cat $OUT`);
#
# Assert that analyzing json file provide the right results
#
ok( $json_ref->{database_info}{postgres}{count} == 629, "Consistent count");
ok( $json_ref->{overall_stat}{histogram}{query_total} == 629, "Consistent query_total");
ok( $json_ref->{overall_stat}{peak}{"2017-09-06 08:48:45"}{write} == 1, "Consistent peak write");
ok( $json_ref->{pgb_session_info}{chronos}{20180912}{16}{count} == 63943, "pgBouncer connections");
`rm -f $OUT`;
......@@ -87,5 +87,5 @@ and look at the example searching on explain_slowest.
Regards,
--
Team Dalibo
Gilles Darold
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment