Adjust pgindent script to use pg_bsd_indent 2.0.

Update version-checking code and list of switches.  Delete obsolete
quasi-support for using GNU indent.  Remove a lot of no-longer-needed
workarounds for bugs of the old version, and improve comments for
the hacks that remain.  Update run_build() subroutine to fetch the
pg_bsd_indent code from the newly established git repo for it.

In passing, fix pgindent to not overwrite files that require no changes;
this makes it a bit more friendly to run on a built tree.

Adjust relevant documentation.

Remove indent.bsd.patch; it's not relevant anymore (and was obsolete
long ago anyway).  Likewise remove pgcppindent, since we're no longer
in the business of shipping C++ code.

Piotr Stefaniak is responsible for most of the algorithmic changes
to the pgindent script; I did the rest.

Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org
Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
This commit is contained in:
Tom Lane 2017-06-21 14:26:21 -04:00
parent 9ef2dbefc7
commit 8ff6d4ec78
5 changed files with 52 additions and 448 deletions

View File

@ -10,7 +10,9 @@ http://adpgtech.blogspot.com/2015/05/running-pgindent-on-non-core-code-or.html
PREREQUISITES:
1) Install pg_bsd_indent in your PATH (see below for details).
1) Install pg_bsd_indent in your PATH. Fetch its source code with
git clone https://git.postgresql.org/git/pg_bsd_indent.git
then follow the directions in README.pg_bsd_indent therein.
2) Install entab (src/tools/entab/).
@ -57,6 +59,9 @@ VALIDATION:
Your configure switches should include at least --enable-tap-tests
or else much of the Perl code won't get exercised.
The ecpg regression tests may well fail due to pgindent's updates of
header files that get copied into ecpg output; if so, adjust the
expected-files to match.
3) If you have the patience, it's worth eyeballing the "git diff" output
for any egregiously ugly changes. See below for cleanup ideas.
@ -100,10 +105,10 @@ you're at it.
BSD indent
----------
We have standardized on NetBSD's indent, and renamed it pg_bsd_indent.
We have fixed a few bugs which requre the NetBSD source to be patched
with indent.bsd.patch patch. A fully patched version is available at
https://ftp.postgresql.org/pub/dev.
We have standardized on FreeBSD's indent, and renamed it pg_bsd_indent.
pg_bsd_indent does differ slightly from FreeBSD's version, mostly in
being more easily portable to non-BSD platforms. You can obtain it from
https://git.postgresql.org/git/pg_bsd_indent.git
GNU indent, version 2.2.6, has several problems, and is not recommended.
These bugs become pretty major when you are doing >500k lines of code.
@ -132,8 +137,10 @@ src/backend/utils/fmgrtab.c is excluded because it confuses pgindent
and it's a derived file anyway.
src/interfaces/ecpg/test/expected/ is excluded to avoid breaking the ecpg
regression tests. Several *.h files are included in regression output so
they must not be changed.
regression tests, since what ecpg generates is not necessarily formatted
as pgindent would do it. (Note that we do not exclude ecpg's header files
from the run; some of them get copied verbatim into ecpg's output, meaning
that the expected files may need to be updated to match.)
src/include/snowball/libstemmer/ and src/backend/snowball/libstemmer/
are excluded because those files are imported from an external project,

View File

@ -1,288 +0,0 @@
diff -c -r bsd_indent/Makefile pg_bsd_indent/Makefile
*** bsd_indent/Makefile Wed Oct 26 17:13:34 2011
--- pg_bsd_indent/Makefile Wed Oct 12 12:17:12 2011
***************
*** 2,10 ****
# Makefile
#
#
! TARGET = indent
XFLAGS = -Wall -D__RCSID="static char *rcsid=" -D__COPYRIGHT="static char *copyright="
! CFLAGS = -g
LIBS =
$(TARGET) : args.o indent.o io.o lexi.o parse.o pr_comment.o
--- 2,10 ----
# Makefile
#
#
! TARGET = pg_bsd_indent
XFLAGS = -Wall -D__RCSID="static char *rcsid=" -D__COPYRIGHT="static char *copyright="
! CFLAGS = -O
LIBS =
$(TARGET) : args.o indent.o io.o lexi.o parse.o pr_comment.o
***************
*** 31,37 ****
clean:
rm -f *.o $(TARGET) log core
! install:
! make clean
! make CFLAGS=-O
install -s -o bin -g bin $(TARGET) /usr/local/bin
--- 31,35 ----
clean:
rm -f *.o $(TARGET) log core
! install: $(TARGET)
install -s -o bin -g bin $(TARGET) /usr/local/bin
diff -c -r bsd_indent/README pg_bsd_indent/README
*** bsd_indent/README Wed Oct 26 17:13:34 2011
--- pg_bsd_indent/README Mon Nov 14 19:30:24 2005
***************
*** 1,3 ****
--- 1,13 ----
+
+ This patch is from NetBSD current, 2005-11-14. It contains all the
+ patches need for its use in PostgreSQL.
+
+ bjm
+
+ ---------------------------------------------------------------------------
+
+
+
This is the C indenter, it originally came from the University of Illinois
via some distribution tape for PDP-11 Unix. It has subsequently been
hacked upon by James Gosling @ CMU. It isn't very pretty, and really needs
diff -c -r bsd_indent/args.c pg_bsd_indent/args.c
*** bsd_indent/args.c Wed Oct 26 17:13:34 2011
--- pg_bsd_indent/args.c Wed Oct 26 17:16:56 2011
***************
*** 83,88 ****
--- 83,90 ----
#include <string.h>
#include "indent_globs.h"
+ #define INDENT_PG_VERSION "1.1"
+
/* profile types */
#define PRO_SPECIAL 1 /* special case */
#define PRO_BOOL 2 /* boolean */
***************
*** 99,106 ****
--- 101,113 ----
#define STDIN 3 /* use stdin */
#define KEY 4 /* type (keyword) */
+ #define KEY_FILE 5 /* only used for args */
+ #define VERSION 6 /* only used for args */
+
char *option_source = "?";
+ void add_typedefs_from_file(char *str);
+
/*
* N.B.: because of the way the table here is scanned, options whose names are
* substrings of other options must occur later; that is, with -lp vs -l, -lp
***************
*** 118,123 ****
--- 125,136 ----
"T", PRO_SPECIAL, 0, KEY, 0
},
{
+ "U", PRO_SPECIAL, 0, KEY_FILE, 0
+ },
+ {
+ "V", PRO_SPECIAL, 0, VERSION, 0
+ },
+ {
"bacc", PRO_BOOL, false, ON, &blanklines_around_conditional_compilation
},
{
***************
*** 425,430 ****
--- 438,456 ----
}
break;
+ case KEY_FILE:
+ if (*param_start == 0)
+ goto need_param;
+ add_typedefs_from_file(param_start);
+ break;
+
+ case VERSION:
+ {
+ printf("pg_bsd_indent %s\n", INDENT_PG_VERSION);
+ exit(0);
+ }
+ break;
+
default:
fprintf(stderr, "\
indent: set_option: internal error: p_special %d\n", p->p_special);
***************
*** 459,461 ****
--- 485,508 ----
exit(1);
}
}
+
+
+ void
+ add_typedefs_from_file(char *str)
+ {
+ FILE *file;
+ char line[BUFSIZ];
+
+ if ((file = fopen(param_start, "r")) == NULL)
+ {
+ fprintf(stderr, "indent: cannot open file %s\n", str);
+ exit(1);
+ }
+ while ((fgets(line, BUFSIZ, file)) != NULL)
+ {
+ /* Remove trailing whitespace */
+ *(line + strcspn(line, " \t\n\r")) = '\0';
+ addkey(strdup(line), 4);
+ }
+ fclose(file);
+ }
Only in pg_bsd_indent: args.o
Only in pg_bsd_indent: indent.bsd.patch
Only in pg_bsd_indent: indent.o
diff -c -r bsd_indent/indent_globs.h pg_bsd_indent/indent_globs.h
*** bsd_indent/indent_globs.h Wed Oct 26 17:13:34 2011
--- pg_bsd_indent/indent_globs.h Mon Nov 14 19:30:24 2005
***************
*** 239,245 ****
scomf, /* Same line comment font */
bodyf; /* major body font */
! #define STACK_SIZE 150
EXTERN struct parser_state {
int last_token;
--- 239,249 ----
scomf, /* Same line comment font */
bodyf; /* major body font */
! /*
! * This controls the maximum number of 'else if' clauses supported.
! * If it is exceeded, comments are placed in column 100.
! */
! #define STACK_SIZE 1000
EXTERN struct parser_state {
int last_token;
Only in pg_bsd_indent: io.o
diff -c -r bsd_indent/lexi.c pg_bsd_indent/lexi.c
*** bsd_indent/lexi.c Wed Oct 26 17:13:34 2011
--- pg_bsd_indent/lexi.c Mon Nov 14 19:30:24 2005
***************
*** 93,99 ****
int rwcode;
};
! struct templ specials[1000] =
{
{"switch", 1},
{"case", 2},
--- 93,99 ----
int rwcode;
};
! struct templ specials[16384] =
{
{"switch", 1},
{"case", 2},
***************
*** 622,629 ****
else
p++;
if (p >= specials + sizeof specials / sizeof specials[0])
! return; /* For now, table overflows are silently
! * ignored */
p->rwd = key;
p->rwcode = val;
p[1].rwd = 0;
--- 622,632 ----
else
p++;
if (p >= specials + sizeof specials / sizeof specials[0])
! {
! fprintf(stderr, "indent: typedef table overflow\n");
! exit(1);
! }
!
p->rwd = key;
p->rwcode = val;
p[1].rwd = 0;
Only in pg_bsd_indent: lexi.o
diff -c -r bsd_indent/parse.c pg_bsd_indent/parse.c
*** bsd_indent/parse.c Wed Oct 26 17:13:34 2011
--- pg_bsd_indent/parse.c Mon Nov 14 19:30:24 2005
***************
*** 231,236 ****
--- 231,241 ----
} /* end of switch */
+ if (ps.tos >= STACK_SIZE) {
+ fprintf(stderr, "indent: stack size overflow\n");
+ exit(1);
+ }
+
reduce(); /* see if any reduction can be done */
#ifdef debug
Only in pg_bsd_indent: parse.o
diff -c -r bsd_indent/pr_comment.c pg_bsd_indent/pr_comment.c
*** bsd_indent/pr_comment.c Wed Oct 26 17:13:34 2011
--- pg_bsd_indent/pr_comment.c Mon Nov 14 19:30:24 2005
***************
*** 148,154 ****
ps.box_com = true;
ps.com_col = 1;
} else {
! if (*buf_ptr == '-' || *buf_ptr == '*' || *buf_ptr == '\n') {
ps.box_com = true; /* a comment with a '-', '*'
* or newline immediately
* after the start comment is
--- 148,158 ----
ps.box_com = true;
ps.com_col = 1;
} else {
! /*
! * Don't process '\n' or every comment is treated as a
! * block comment, meaning there is no wrapping.
! */
! if (*buf_ptr == '-' || *buf_ptr == '*') {
ps.box_com = true; /* a comment with a '-', '*'
* or newline immediately
* after the start comment is
***************
*** 328,333 ****
--- 332,350 ----
goto end_of_comment;
}
} while (*buf_ptr == ' ' || *buf_ptr == '\t');
+
+ /*
+ * If there is a blank comment line, we need to prefix
+ * the line with the same three spaces that "/* " takes up.
+ * Without this code, blank stared lines in comments have
+ * three too-many characters on the line when wrapped.
+ */
+ if (s_com == e_com) {
+ *e_com++ = ' '; /* add blanks for continuation */
+ *e_com++ = ' ';
+ *e_com++ = ' ';
+ now_col += 3;
+ }
} else
if (++buf_ptr >= buf_end)
fill_buffer();
Only in pg_bsd_indent: pr_comment.o

View File

@ -1,29 +0,0 @@
#!/bin/sh
# src/tools/pgindent/pgcppindent
trap "rm -f /tmp/$$ /tmp/$$a" 0 1 2 3 15
entab </dev/null >/dev/null
if [ "$?" -ne 0 ]
then echo "Go to the src/tools/entab directory and do a 'make' and 'make install'." >&2
echo "This will put the 'entab' command in your path." >&2
echo "Then run $0 again."
exit 1
fi
astyle --version </dev/null >/dev/null 2>&1
if [ "$?" -eq 0 ]
then echo "You do not appear to have 'astyle' installed on your system." >&2
exit 1
fi
for FILE
do
astyle --style=ansi -b -p -S < "$FILE" >/tmp/$$ 2>/tmp/$$a
if [ "$?" -ne 0 -o -s /tmp/$$a ]
then echo "$FILE"
cat /tmp/$$a
fi
cat /tmp/$$ |
entab -t4 -qc |
cat >/tmp/$$a && cat /tmp/$$a >"$FILE"
done

View File

@ -12,15 +12,12 @@ use IO::Handle;
use Getopt::Long;
# Update for pg_bsd_indent version
my $INDENT_VERSION = "1.3";
my $devnull = File::Spec->devnull;
# Common indent settings
my $INDENT_VERSION = "2.0";
# Our standard indent settings
my $indent_opts =
"-bad -bap -bc -bl -d0 -cdb -nce -nfc1 -di12 -i4 -l79 -lp -nip -npro -bbb";
"-bad -bap -bbb -bc -bl -cli1 -cp33 -cdb -nce -d0 -di12 -nfc1 -i4 -l79 -lp -nip -npro -sac -tpg -ts4";
# indent-dependent settings
my $extra_opts = "";
my $devnull = File::Spec->devnull;
my ($typedefs_file, $typedef_str, $code_base, $excludes, $indent, $build);
@ -74,11 +71,11 @@ sub check_indent
if ($? >> 8 != 1)
{
print STDERR
"You do not appear to have 'indent' installed on your system.\n";
"You do not appear to have $indent installed on your system.\n";
exit 1;
}
if (`$indent -V` !~ m/ $INDENT_VERSION$/)
if (`$indent --version` !~ m/ $INDENT_VERSION$/)
{
print STDERR
"You do not appear to have $indent version $INDENT_VERSION installed on your system.\n";
@ -89,13 +86,8 @@ sub check_indent
if ($? == 0)
{
print STDERR
"You appear to have GNU indent rather than BSD indent.\n",
"See the pgindent/README file for a description of its problems.\n";
$extra_opts = "-cdb -bli0 -npcs -cli4 -sc";
}
else
{
$extra_opts = "-cli1";
"You appear to have GNU indent rather than BSD indent.\n";
exit 1;
}
}
@ -198,60 +190,16 @@ sub pre_indent
{
my $source = shift;
# remove trailing whitespace
$source =~ s/[ \t]+$//gm;
## Comments
# Convert // comments to /* */
$source =~ s!^([ \t]*)//(.*)$!$1/* $2 */!gm;
# 'else' followed by a single-line comment, followed by
# a brace on the next line confuses BSD indent, so we push
# the comment down to the next line, then later pull it
# back up again. Add space before _PGMV or indent will add
# it for us.
# AMD: A symptom of not getting this right is that you see errors like:
# FILE: ../../../src/backend/rewrite/rewriteHandler.c
# Error@2259:
# Stuff missing from end of file
$source =~
s!(\}|[ \t])else[ \t]*(/\*)(.*\*/)[ \t]*$!$1else\n $2 _PGMV$3!gm;
# Indent multi-line after-'else' comment so BSD indent will move it
# properly. We already moved down single-line comments above.
# Check for '*' to make sure we are not in a single-line comment that
# has other text on the line.
$source =~ s!(\}|[ \t])else[ \t]*(/\*[^*]*)[ \t]*$!$1else\n $2!gm;
# Mark some comments for special treatment later
# Adjust dash-protected block comments so indent won't change them
$source =~ s!/\* +---!/*---X_X!g;
## Other
# Work around bug where function that defines no local variables
# misindents switch() case lines and line after #else. Do not do
# for struct/enum.
my @srclines = split(/\n/, $source);
foreach my $lno (1 .. $#srclines)
{
my $l2 = $srclines[$lno];
# Line is only a single open brace in column 0
next unless $l2 =~ /^\{[ \t]*$/;
# previous line has a closing paren
next unless $srclines[ $lno - 1 ] =~ /\)/;
# previous line was struct, etc.
next
if $srclines[ $lno - 1 ] =~
m!=|^(struct|enum|[ \t]*typedef|extern[ \t]+"C")!;
$srclines[$lno] = "$l2\nint pgindent_func_no_var_fix;";
}
$source = join("\n", @srclines) . "\n"; # make sure there's a final \n
# Prevent indenting of code in 'extern "C"' blocks.
# we replace the braces with comments which we'll reverse later
my $extern_c_start = '/* Open extern "C" */';
@ -260,6 +208,9 @@ sub pre_indent
s!(^#ifdef[ \t]+__cplusplus.*\nextern[ \t]+"C"[ \t]*\n)\{[ \t]*$!$1$extern_c_start!gm;
$source =~ s!(^#ifdef[ \t]+__cplusplus.*\n)\}[ \t]*$!$1$extern_c_stop!gm;
# Protect backslashes in DATA() and wrapping in CATALOG()
$source =~ s!^((DATA|CATALOG)\(.*)$!/*$1*/!gm;
return $source;
}
@ -269,26 +220,20 @@ sub post_indent
my $source = shift;
my $source_filename = shift;
# put back braces for extern "C"
# Restore DATA/CATALOG lines
$source =~ s!^/\*((DATA|CATALOG)\(.*)\*/$!$1!gm;
# Put back braces for extern "C"
$source =~ s!^/\* Open extern "C" \*/$!{!gm;
$source =~ s!^/\* Close extern "C" \*/$!}!gm;
## Comments
# remove special comment marker
# Undo change of dash-protected block comments
$source =~ s!/\*---X_X!/* ---!g;
# Pull up single-line comment after 'else' that was pulled down above
$source =~ s!else\n[ \t]+/\* _PGMV!else\t/*!g;
# Indent single-line after-'else' comment by only one tab.
$source =~ s!(\}|[ \t])else[ \t]+(/\*.*\*/)[ \t]*$!$1else\t$2!gm;
# Add tab before comments with no whitespace before them (on a tab stop)
$source =~ s!(\S)(/\*.*\*/)$!$1\t$2!gm;
# Remove blank line between opening brace and block comment.
$source =~ s!(\t*\{\n)\n([ \t]+/\*)$!$1$2!gm;
# Fix run-together comments to have a tab between them
$source =~ s!\*/(/\*.*\*/)$!*/\t$1!gm;
# cpp conditionals
@ -297,13 +242,10 @@ sub post_indent
## Functions
# Work around misindenting of function with no variables defined.
$source =~ s!^[ \t]*int[ \t]+pgindent_func_no_var_fix;[ \t]*\n{1,2}!!gm;
# Use a single space before '*' in function return types
$source =~ s!^([A-Za-z_]\S*)[ \t]+\*$!$1 *!gm;
# Move prototype names to the same line as return type. Useful
# Move prototype names to the same line as return type. Useful
# for ctags. Indent should do this, but it does not. It formats
# prototypes just like real functions.
@ -319,21 +261,6 @@ sub post_indent
)
!$1 . (substr($1,-1,1) eq '*' ? '' : ' ') . $2!gmxe;
## Other
# Remove too much indenting after closing brace.
$source =~ s!^\}\t[ \t]+!}\t!gm;
# Workaround indent bug that places excessive space before 'static'.
$source =~ s!^static[ \t]+!static !gm;
# Remove leading whitespace from typedefs
$source =~ s!^[ \t]+typedef enum!typedef enum!gm
if $source_filename =~ 'libpq-(fe|events).h$';
# Remove trailing blank lines
$source =~ s!\n+\z!\n!;
return $source;
}
@ -344,7 +271,7 @@ sub run_indent
my $error_message = shift;
my $cmd =
"$indent $indent_opts $extra_opts -U" . $filtered_typedefs_fh->filename;
"$indent $indent_opts -U" . $filtered_typedefs_fh->filename;
my $tmp_fh = new File::Temp(TEMPLATE => "pgsrcXXXXX");
my $filename = $tmp_fh->filename;
@ -461,21 +388,15 @@ sub run_build
$ENV{PGTYPEDEFS} = abs_path('tmp_typedefs.list');
my $pg_bsd_indent_url =
"https://ftp.postgresql.org/pub/dev/pg_bsd_indent-"
. $INDENT_VERSION
. ".tar.gz";
my $indentrepo = "https://git.postgresql.org/git/pg_bsd_indent.git";
system("git clone $indentrepo >$devnull 2>&1");
die "could not fetch pg_bsd_indent sources from $indentrepo\n"
unless $? == 0;
$rv = getstore($pg_bsd_indent_url, "pg_bsd_indent.tgz");
die "cannot fetch BSD indent tarfile from $pg_bsd_indent_url\n"
unless is_success($rv);
# XXX add error checking here
system("tar -z -xf pg_bsd_indent.tgz");
chdir "pg_bsd_indent";
system("make > $devnull 2>&1");
chdir "pg_bsd_indent" || die;
system("make all check >$devnull");
die "could not build pg_bsd_indent from source\n"
unless $? == 0;
$ENV{PGINDENT} = abs_path('pg_bsd_indent');
@ -504,8 +425,10 @@ sub build_clean
chdir "$code_base";
system("rm -rf src/tools/pgindent/bsdindent");
system("git clean -q -f src/tools/entab src/tools/pgindent");
system("rm -rf src/tools/pgindent/pg_bsd_indent");
system("rm -f src/tools/pgindent/tmp_typedefs.list");
system("git clean -q -f src/tools/entab");
}
@ -529,7 +452,7 @@ $filtered_typedefs_fh = load_typedefs();
check_indent();
# make sure we process any non-option arguments.
# any non-option arguments are files to be processed
push(@files, @ARGV);
foreach my $source_filename (@files)
@ -544,15 +467,11 @@ foreach my $source_filename (@files)
next if $otherfile ne $source_filename && -f $otherfile;
my $source = read_source($source_filename);
my $orig_source = $source;
my $error_message = '';
$source = pre_indent($source);
# Protect backslashes in DATA() and wrapping in CATALOG()
$source = detab($source);
$source =~ s!^((DATA|CATALOG)\(.*)$!/*$1*/!gm;
$source = run_indent($source, \$error_message);
if ($source eq "")
{
@ -560,13 +479,9 @@ foreach my $source_filename (@files)
next;
}
# Restore DATA/CATALOG lines; must be done here so tab alignment is preserved
$source =~ s!^/\*((DATA|CATALOG)\(.*)\*/$!$1!gm;
$source = entab($source);
$source = post_indent($source, $source_filename);
write_source($source, $source_filename);
write_source($source, $source_filename) if $source ne $orig_source;
}
build_clean($code_base) if $build;

View File

@ -11,11 +11,10 @@ it without any parameters at the top of the source tree you want to process.
If you don't have all the requirements installed, pgindent will fetch and build
them for you, if you're in a PostgreSQL source tree:
pgindent --build
If your indent program is not installed in your path, you can specify it
by setting the environment variable INDENT, or PGINDENT, or by giving the
If your pg_bsd_indent program is not installed in your path, you can specify
it by setting the environment variable INDENT, or PGINDENT, or by giving the
command line option --indent:
pgindent --indent=/opt/extras/bsdindent