.gitignore

+*~

.gitlab-ci.yml

+image: alpine
+
+pages:
+  stage: deploy
+  script:
+  - echo 'Nothing to do...'
+  artifacts:
+    paths:
+    - public
+  only:
+  - master

CHANGES.md

 # Changes
 
+## 2018/11/12
+
+* In `.mly` files, a new syntax for rules has been introduced, which is
+  slightly more pleasant than the old syntax. (A rule is the definition of a
+  nonterminal symbol.) The old syntax remains available; the user chooses
+  between the two syntaxes on a per-rule basis. The new syntax is fully
+  documented in the manual;
+  [a brief summary of the differences](doc/new-rule-syntax.md)
+  with respect to the old syntax is also available.
+  **The new syntax is considered experimental**
+  and is subject to change in the near future.
+
+* In the Coq back-end, avoid pattern-matching at type `int31`,
+  which will disappear in future versions of Coq.
+  Instead, convert `int31` to `Z`, and perform pattern matching in `Z`.
+  (Reported by Vincent Laporte, implemented by Jacques-Henri Jourdan.)
+
+* Implement a more economical renaming scheme for OCaml variables
+  during the elimination of `%inline` symbols. This leads to slightly
+  more readable code (more reasonable variables names, fewer `let`
+  bindings).
+
+* Another attempt at removing all trailing spaces in auto-generated
+  `.messages` files. (I hope I got it right, this time.)
+
+## 2018/10/26
+
+* A new syntactic sugar facility, "token aliases", has been added.
+  The declaration of a terminal symbol may also declare an alias,
+  which takes the form of a name between double quotes, as in
+  `%token PLUS "+"`.
+  Thereafter, `"+"` may be used freely in place of `PLUS` throughout
+  the grammar. This makes it slightly easier to read grammars.
+  (Contributed by Perry E. Metzger.)
+
+## 2018/10/25
+
+* Until today, the semicolon character `;` was insignificant: it was
+  considered as whitespace by Menhir. It is now accepted only in a
+  few specific places, namely: after a declaration; after a rule;
+  after a producer. If Menhir suddenly complains about a semicolon,
+  just remove it. This change is being made in preparation for
+  further syntactic changes.
+
+* New flag `--no-dollars`, which disallows the use of `$i` in semantic actions.
+  The default behavior remains to allow the use of `$i`.
+
+* When generating OCaml code, include all record fields in record patterns,
+  even when bound to a wildcard pattern. Thus, avoid triggering OCaml's
+  warning 9.
+
+## 2018/10/06
+
+* Standard library: add `rev`, `flatten`, `append`.
+  Add a link from the manual to `standard.mly` in the repository.
+
+* Update the manual to explain how to use `dune` and `menhir` together.
+
+* Install `.cmxs` files for menhirLib and menhirSdk.
+
+* Remove all references to `Pervasives` in the generated OCaml code.
+  These references were infrequent anyway,
+  and `Pervasives` is about to be deprecated in OCaml 4.08, it seems.
+
+* In `--interpret` mode, print `Ready!` once ready to accept input.
+
+* At verbosity level `-lg 2`, for each nonterminal symbol `N`, display
+  a sentence (composed of terminal symbols) of minimal length generated
+  by `N`.
+
+* When writing a `.cmly` file, open it in binary mode. This should
+  eliminate the failure that was observed under Windows:
+  `output_value: not a binary channel`. (Reported by Bryan Phelps.
+  A fix was present in the `mingw` patches for Menhir.)
+
+* Change the logic used in the root `Makefile` to deal with Unix and Windows
+  in a uniform way. (Also inspired by the `mingw` patches for Menhir.)
+
+* Coq back-end: add a few newlines in the generated file for readability.
+  (Suggested by Bernhard Schommer.)
+
+* Remove the trailing space at the end of every sentence in
+  auto-generated `.messages` files. (Suggested by Xavier Leroy.)
+
+## 2018/09/05
+
+* When `--explain` is enabled, always create a fresh `.conflicts` file
+  (wiping out any pre-existing file), even if there are in fact no conflicts.
+  This should avoid confusion with outdated `.conflicts` files.
+
+* Fix several bugs in the treatment of `--strict`. No `.conflicts` file
+  was created when `--strict` and `--explain` were both enabled. Also,
+  some warnings were not properly turned into errors by `--strict`.
+
+## 2018/07/04
+
+* Update the `man` page, which was woefully out of date.
+
+## 2018/07/03
+
+* New location keywords.
+  `$loc` is sugar for the pair `($startpos, $endpos)`.
+  `$loc(x)` is sugar for the pair `($startpos(x), $endpos(x))`.
+  `$sloc` is sugar for the pair `($symbolstartpos, $endpos)`.
+  (Contributed by Nicolás Ojeda Bär.)
+
+## 2018/06/08
+
+* Add two new parameterized nonterminal symbols, `endrule(X)` and `midrule(X)`,
+  to the standard library. These symbols have been available since 2015/02/11
+  under the names `anonymous(X)` and `embedded(X)`, but were not yet documented.
+  `endrule(X)` and `midrule(X)` are now documented,
+  while `anonymous(X)` and `embedded(X)` remain present but are deprecated.
+
 ## 2018/05/30
 
 * In `--coq` mode, Menhir now produces references to `MenhirLib.Grammar`

INSTALLATION.md

@@ -40,8 +40,8 @@ when running `make all` and `make install`.
 Compile and install as follows:
 
 ```
-       make PREFIX=/usr/local USE_OCAMLFIND=true all
-  sudo make PREFIX=/usr/local install
+       make -f Makefile PREFIX=/usr/local USE_OCAMLFIND=true all
+  sudo make -f Makefile PREFIX=/usr/local install
 ```
 
 If necessary, adjust `PREFIX`, `USE_OCAMLFIND` and `TARGET` as described above.

Makefile

@@ -50,32 +50,24 @@ bindir          := $(PREFIX)/bin
 docdir		:= $(PREFIX)/share/doc/menhir
 libdir	        := $(PREFIX)/share/menhir
 mandir          := $(PREFIX)/share/man/man1
-MANS            := menhir.1
-DOCS            := manual.pdf manual.html demos
+MANS            := doc/menhir.1
+DOCS            := doc/manual.pdf doc/manual.html doc/manual*.png demos
 MLYLIB          := src/standard.mly
 
 # ----------------------------------------------------------------------------
 
-# A few settings differ on Windows versus Unix.
+# The following incantations should work on both Windows and Unix,
+# and allow us to abstract away the differences.
 
-# If the compiler is MSVC, then object file names end in .obj instead of .o.
-
-ifneq (,$(shell ocamlc -config | grep -E "ccomp_type: msvc"))
-  OBJ          := obj
-# LIBSUFFIX    := lib
-else
-  OBJ          := o
-# LIBSUFFIX    := a
-endif
-
-# If we are under Windows (regardless of whether we are using MSVC or mingw)
-# then the name of the executable file ends in .exe.
-
-ifeq ($(OS),Windows_NT)
-  MENHIREXE    := menhir.exe
-else
-  MENHIREXE    := menhir
-endif
+# The extension of object files.
+OBJ := $(shell ocamlc -config | sed -n '/^ext_obj:/p' | sed 's/ext_obj: //')
+# The extension of executable files.
+# Note: the field "ext_exe" seems to have appeared in OCaml 4.05.
+# With earlier versions of OCaml, this incantation defines $(EXE)
+# as the empty string, which could be a problem under Windows.
+EXE := $(shell ocamlc -config | sed -n '/^ext_exe:/p' | sed 's/ext_exe: //')
+# The OS type.
+OS_TYPE := $(shell ocamlc -config | sed -n '/^os_type:/p' | sed 's/os_type: //')
 
 # The path $(installation_libdir), which is recorded in src/installation.ml (see
 # below), must sometimes be translated using cygpath.
@@ -92,8 +84,8 @@ endif
 # performed if "os_type" is "Win32" or "Win64", and must not be performed if
 # "os_type" is "Cygwin" or "Unix".
 
-ifneq (,$(shell ocamlc -config | grep -E "os_type: (Win32|Win64)"))
-installation_libdir := $(shell cygpath -m $(libdir))
+ifeq ($(OS_TYPE),$(filter $(OS_TYPE),Win32 Win64))
+  installation_libdir := $(shell cygpath -m $(libdir) || echo $(libdir))
 else
   installation_libdir := $(libdir)
 endif
@@ -168,7 +160,7 @@ all:
 
 MENHIRLIB       := menhirLib.mli menhirLib.ml menhirLib.cmi menhirLib.cmo
 ifneq ($(TARGET),byte)
-MENHIRLIB       := $(MENHIRLIB) menhirLib.cmx menhirLib.$(OBJ)
+MENHIRLIB       := $(MENHIRLIB) menhirLib.cmx menhirLib.cmxs menhirLib$(OBJ)
 endif
 
 # -------------------------------------------------------------------------
@@ -177,7 +169,7 @@ endif
 
 MENHIRSDK       := menhirSdk.cmi menhirSdk.cmo
 ifneq ($(TARGET),byte)
-MENHIRSDK       := $(MENHIRSDK) menhirSdk.cmx menhirSdk.$(OBJ)
+MENHIRSDK       := $(MENHIRSDK) menhirSdk.cmx menhirSdk.cmxs menhirSdk$(OBJ)
 endif
 
 # ----------------------------------------------------------------------------
@@ -186,7 +178,7 @@ endif
 install:
 # Install the executable.
 	mkdir -p $(bindir)
-	install $(BUILDDIR)/menhir.$(TARGET) $(bindir)/$(MENHIREXE)
+	install $(BUILDDIR)/menhir.$(TARGET) $(bindir)/menhir$(EXE)
 # Install Menhir's standard library.
 	mkdir -p $(libdir)
 	install -m 644 $(MLYLIB) $(libdir)
@@ -203,20 +195,21 @@ install:
 	  install -m 644 $(patsubst %,$(BUILDDIR)/%,$(MENHIRLIB)) $(libdir) ; \
 	  install -m 644 $(patsubst %,$(SDKDIR)/%,$(MENHIRSDK)) $(libdir) ; \
 	fi
-# Install the documentation, if it has been built.
-	if [ -f manual.pdf ] ; then \
-	  mkdir -p $(docdir) $(mandir) && \
-	  cp -r $(DOCS) $(docdir) && \
+# Install the documentation. (If building from the repository, the documentation
+# may be absent.)
+	if [ -f doc/manual.pdf ] ; then \
+	  mkdir -p $(docdir) $(mandir) ; \
+	  cp -r $(DOCS) $(docdir) ; \
 	  cp -r $(MANS) $(mandir) ; \
 	fi
 
 uninstall:
-	@if `$(bindir)/$(MENHIREXE) --suggest-ocamlfind` ; then \
+	@if `$(bindir)/menhir$(EXE) --suggest-ocamlfind` ; then \
 	  echo 'Un-installing MenhirLib and MenhirSdk via ocamlfind.' ; \
 	  ocamlfind remove menhirLib ; \
 	  ocamlfind remove menhirSdk ; \
 	fi
-	rm -rf $(bindir)/$(MENHIREXE)
+	rm -rf $(bindir)/menhir$(EXE)
 	rm -rf $(libdir)
 	rm -rf $(docdir)
 	rm -rf $(mandir)/$(MANS)

README.md

@@ -13,6 +13,8 @@ The latest released version of Menhir can be easily installed via
 
 For manual installation, see [INSTALLATION.md](INSTALLATION.md).
 
+Some instructions for developers can be found in [HOWTO.md](HOWTO.md).
+
 ## Authors
 
 * [François Pottier](Francois.Pottier@inria.fr)

check-tarball.sh

+#!/bin/bash
+set -euo pipefail
+IFS=$'\n\t'
+
+# This script checks that a Menhir tarball can be compiled and installed.
+# The command line argument should be the tarball's name without .tar.gz.
+
+PACKAGE="$1"
+TARBALL=$PACKAGE.tar.gz
+
+# Some of the demos assume Menhir has been installed using ocamlfind, so that
+# is what we do. For this reason, we must first check that ocamlfind does not
+# already have some version of menhirLib and menhirSdk.
+
+# We could also create a fresh opam switch, but that would be time-consuming.
+
+if ocamlfind query menhirLib >/dev/null 2>/dev/null ; then
+  if opam list -i menhir 2>/dev/null | grep -v -e "^#" | grep menhir ; then
+    echo "Warning: menhir is already installed." ;
+    read -p "Can I remove it [Enter/^C]?" -n 1 -r ;
+    opam remove menhir ;
+  else
+    echo "Warning: menhirLib is already installed." ;
+    read -p "Can I remove it [Enter/^C]?" -n 1 -r ;
+    ocamlfind remove menhirLib ;
+    ocamlfind remove menhirSdk || true ;
+  fi ;
+fi
+
+# Create a temporary directory; extract, build, and install the package into
+# it; build the demos using the installed binary.
+
+TEMPDIR=`mktemp -d /tmp/menhir-test.XXXXXX`
+INSTALL=$TEMPDIR/install
+
+cp $TARBALL $TEMPDIR
+
+echo "   * Extracting. "
+(cd $TEMPDIR && tar xfz $TARBALL)
+
+echo "   * Compiling and installing."
+mkdir $INSTALL
+(cd $TEMPDIR/$PACKAGE &&
+  make PREFIX=$INSTALL USE_OCAMLFIND=true &&
+  make PREFIX=$INSTALL USE_OCAMLFIND=true install
+) > $TEMPDIR/install.log 2>&1 || (cat $TEMPDIR/install.log; exit 1)
+
+echo "   * Building the demos."
+(cd $TEMPDIR/$PACKAGE &&
+  make MENHIR=$INSTALL/bin/menhir -C demos
+) > $TEMPDIR/demos.log 2>&1 || (cat $TEMPDIR/demos.log; exit 1)
+
+echo "   * Uninstalling."
+(cd $TEMPDIR/$PACKAGE &&
+  make PREFIX=$INSTALL USE_OCAMLFIND=true uninstall
+) > $TEMPDIR/uninstall.log 2>&1 || (cat $TEMPDIR/uninstall.log; exit 1)
+
+rm -rf $TEMPDIR

debian/changelog

+menhir (20181113-1) unstable; urgency=medium
+
+  * new upstream version
+  * d/watch: point to inria gitlab
+  * d/menhir-doc.docs: also install png pictures of the manual
+  * d/rules: remove .gitignore files in examples
+  * Standards-Version 4.2.1 (no change)
+
+ -- Ralf Treinen <treinen@debian.org>  Wed, 14 Nov 2018 22:14:32 +0100
+
 menhir (20180530-2) unstable; urgency=medium
 
   * menhir-doc: Breaks/Replaces menhir (<< 20170712-1) since examples

debian/control

@@ -12,7 +12,7 @@ Build-Depends:
   ocamlbuild,
   ocaml-findlib,
   dh-ocaml (>= 0.9)
-Standards-Version: 4.1.5
+Standards-Version: 4.2.1
 Homepage: http://gallium.inria.fr/~fpottier/menhir/
 Vcs-Browser: https://salsa.debian.org/ocaml-team/menhir
 Vcs-Git: https://salsa.debian.org/ocaml-team/menhir.git

debian/menhir-doc.docs

 debian/tmp/usr/share/doc/menhir/manual.pdf
 debian/tmp/usr/share/doc/menhir/manual.html
+debian/tmp/usr/share/doc/menhir/manual*.png
\ No newline at end of file

debian/rules

@@ -23,6 +23,15 @@ override_dh_auto_install:
 	mkdir -p $(OCAMLFIND_DESTDIR)
 	$(MAKE) TARGET=$(BEST) PREFIX=$(DESTDIR)/usr install
 
+override_dh_installexamples:
+	dh_installexamples
+	if [ -d debian/menhir-doc/usr/share/doc/menhir/examples ]; \
+	then \
+	  find debian/menhir-doc/usr/share/doc/menhir/examples \
+	    -name .gitignore -delete ; \
+	fi
+
+
 override_dh_missing:
 	dh_missing --fail-missing
 

debian/watch

-version=3
+version=4
 
-opts=dversionmangle=s/\+dfsg// \
-        http://pauillac.inria.fr/~fpottier/menhir/menhir-(.*)\.tar\.gz
+#http://pauillac.inria.fr/~fpottier/menhir/menhir-(.*)\.tar\.gz
+opts=filenamemangle=s/.*\/archive\/(\d\S+)\/menhir.*\.tar\.gz/menhir-$1\.tar\.gz/g \
+  https://gitlab.inria.fr/fpottier/menhir/tags?sort=updated_desc .*/archive/(\d\S+)/.*\.tar\.gz.*
\ No newline at end of file

demos/Makefile

+# These demos assume that Menhir has been installed and is available
+# under the name "menhir". This can be overridden by setting MENHIR.
+
 # The following demos require menhirLib:
 #   calc-incremental
 #   calc-inspection
 
-# The following demos require menhirSdk:
+# The following demos require menhirSdk and OCaml >= 4.03:
 #   generate-printers
 
-# The demos whose name ends in -dune require dune (a.k.a. jbuilder)
-# and assume that Menhir is installed.
+# The demos whose name ends in -dune require dune 1.4.
+# If dune is absent, attempting to build these demos
+# succeeds without actually doing anything.
 
 DEMOS := \
   calc \
+  calc-alias \
   calc-two \
   calc-param \
   calc-incremental \
   calc-inspection \
   generate-printers \
-#  calc-dune \             # this demo causes [make check] to fail
-#  calc-incremental-dune \ # this demo causes [make check] to fail
-#  calc-inspection-dune \  # this demo requires new Dune, not yet released
+  calc-dune \
+  calc-new-syntax-dune \
+  calc-ast-dune \
+  calc-param-dune \
+  calc-two-dune \
+  calc-incremental-dune \
+  calc-inspection-dune \
+  generate-printers-dune \
 
-.PHONY: all clean
+.PHONY: all clean test
 
-all clean::
+all clean test::
 	@ for i in $(DEMOS) ; do \
-	  $(MAKE) -C $$i $@ ; \
+	  $(MAKE) --no-print-directory -C $$i $@ ; \
 	done
 
 clean::

demos/calc-alias/.gitignore

+calc.native
+_build

demos/calc-alias/.merlin

+B _build

demos/calc-alias/Makefile

+.PHONY: all clean test
+
+MENHIR          := menhir
+
+MENHIRFLAGS     := --infer
+
+OCAMLBUILD      := ocamlbuild -use-ocamlfind -use-menhir -menhir "$(MENHIR) $(MENHIRFLAGS)"
+
+MAIN            := calc
+
+all:
+	$(OCAMLBUILD) $(MAIN).native
+
+clean:
+	rm -f *~ .*~
+	$(OCAMLBUILD) -clean
+
+test: all
+	@echo "The following command should print 42:"
+	echo "(1 + 2 * 10) * 2" | ./$(MAIN).native

demos/calc-alias/README

+This tiny program reads arithmetic expressions from the standard input
+channel. Each expression is expected to be complete when the current line
+ends. Its value is then displayed on the standard output channel. This
+code is adapted from ocamlyacc's documentation.
+
+The difference between this demo and "calc" is that in this case, we make use
+of token aliases to make the grammar look slightly nicer.

demos/calc-alias/calc.ml

+let process (line : string) =
+  let linebuf = Lexing.from_string line in
+  try
+    (* Run the parser on this line of input. *)
+    Printf.printf "%d\n%!" (Parser.main Lexer.token linebuf)
+  with
+  | Lexer.Error msg ->
+      Printf.fprintf stderr "%s%!" msg
+  | Parser.Error ->
+      Printf.fprintf stderr "At offset %d: syntax error.\n%!" (Lexing.lexeme_start linebuf)
+
+let process (optional_line : string option) =
+  match optional_line with
+  | None ->
+      ()
+  | Some line ->
+      process line
+
+let rec repeat channel =
+  (* Attempt to read one line. *)
+  let optional_line, continue = Lexer.line channel in
+  process optional_line;
+  if continue then
+    repeat channel
+  
+let () =
+  repeat (Lexing.from_channel stdin)
+

demos/calc-alias/lexer.mll

+{
+  open Parser
+
+  exception Error of string
+
+}
+
+(* This rule looks for a single line, terminated with '\n' or eof.
+   It returns a pair of an optional string (the line that was found)
+   and a Boolean flag (false if eof was reached). *)
+
+rule line = parse
+| ([^'\n']* '\n') as line
+    (* Normal case: one line, no eof. *)
+    { Some line, true }
+| eof
+    (* Normal case: no data, eof. *)
+    { None, false }
+| ([^'\n']+ as line) eof
+    (* Special case: some data but missing '\n', then eof.
+       Consider this as the last line, and add the missing '\n'. *)
+    { Some (line ^ "\n"), false }
+
+(* This rule analyzes a single line and turns it into a stream of
+   tokens. *)
+
+and token = parse
+| [' ' '\t']
+    { token lexbuf }
+| '\n'
+    { EOL }
+| ['0'-'9']+ as i
+    { INT (int_of_string i) }
+| '+'
+    { PLUS }
+| '-'
+    { MINUS }
+| '*'
+    { TIMES }
+| '/'
+    { DIV }
+| '('
+    { LPAREN }
+| ')'
+    { RPAREN }
+| _
+    { raise (Error (Printf.sprintf "At offset %d: unexpected character.\n" (Lexing.lexeme_start lexbuf))) }
+

demos/calc-alias/parser.mly

+(* In the following, we exploit token aliases: e.g., "+" is an alias
+   for PLUS, "-" is an alias for MINUS, and so on. We exhibit a tiny
+   bit of bad taste and define "42" as an alias for the token INT.
+   Obviously there are integers other than 42; the token INT and its
+   alias "42" stand for an arbitrary integer literal. *)
+
+(* Token, and token aliases, are declared here: *)
+
+%token<int> INT  "42"
+%token PLUS       "+"
+%token MINUS      "-"
+%token TIMES      "*"
+%token DIV        "/"
+%token LPAREN     "("
+%token RPAREN     ")"
+%token EOL
+
+(* Token aliases can be used throughout the rest of the grammar. E.g.,
+   they can be used in precedence declarations: *)
+
+%left "+" "-"       /* lowest precedence */
+%left "*" "/"       /* medium precedence */
+%nonassoc UMINUS    /* highest precedence */
+
+%start <int> main
+
+%%
+
+main:
+| e = expr EOL
+    { e }
+
+(* Token aliases can also be used inside rules: *)
+
+expr:
+| i = "42"
+    { i }
+| "(" e = expr ")"
+    { e }
+| e1 = expr "+" e2 = expr
+    { e1 + e2 }
+| e1 = expr "-" e2 = expr
+    { e1 - e2 }
+| e1 = expr "*" e2 = expr
+    { e1 * e2 }
+| e1 = expr "/" e2 = expr
+    { e1 / e2 }
+| "-" e = expr %prec UMINUS
+    { - e }