.github/ISSUE_TEMPLATE.md

-Before submitting an issue to ANTLR, please check off these boxes:
- 
-- [ ] I am not submitting a question on how to use ANTLR; instead, go to [antlr4-discussion google group](https://groups.google.com/forum/#!forum/antlr-discussion) or ask at [stackoverflow](http://stackoverflow.com/questions/tagged/antlr4)
-- [ ] I have done a search of the existing issues to make sure I'm not sending in a duplicate
-
-[cut]
-Please include information about the expected behavior, actual behavior, and the smallest grammar or code that reproduces the behavior. If appropriate, please indicate the code generation targets such as Java, C#, ...  Pointers into offending code regions are also very welcome.
-[/cut]

.github/PULL_REQUEST_TEMPLATE.md

-[cut]
-Thank you for proposing a contribution to the ANTLR project. In order to accept changes from the outside world, all contributors must "sign" the  [contributors.txt](https://github.com/antlr/antlr4/blob/master/contributors.txt) contributors certificate of origin. It's an unfortunate reality of today's fuzzy and bizarre world of open-source ownership.
-
-Make sure you are already in the contributors.txt file or add a commit to this pull request with the appropriate change. Thanks!
-[/cut]

CONTRIBUTING.md

+# Contributing to ANTLR 4
+
+1. [Fork](https://help.github.com/articles/fork-a-repo) the [antlr/antlr4 repo](https://github.com/antlr/antlr4)
+2. Install and configure [EditorConfig](http://editorconfig.org/) so your text editor or IDE uses the ANTLR 4 coding style
+3. [Build ANTLR 4](doc/building-antlr.md)
+4. [Run the ANTLR project unit tests](doc/antlr-project-testing.md)
+5. Create a [pull request](https://help.github.com/articles/using-pull-requests/) including your change
+
+
+**Note:** You must sign the `contributors.txt` certificate of origin with your pull request if you've not done so before.

LICENSE.txt

 [The "BSD 3-clause license"]
-Copyright (c) 2012-2016 The ANTLR Project. All rights reserved.
+Copyright (c) 2012-2017 The ANTLR Project. All rights reserved.
 
 Redistribution and use in source and binary forms, with or without
 modification, are permitted provided that the following conditions
@@ -24,3 +24,29 @@ DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
 THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
 THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+=====
+
+MIT License for codepointat.js from https://git.io/codepointat
+MIT License for fromcodepoint.js from https://git.io/vDW1m
+
+Copyright Mathias Bynens <https://mathiasbynens.be/>
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

README.md

@@ -21,6 +21,7 @@ ANTLR project lead and supreme dictator for life
 * David Sisson (C++ initial target and test)
 * [Janyou](https://github.com/janyou) (Swift target)
 * [Ewan Mellor](https://github.com/ewanmellor), [Hanzhou Shi](https://github.com/hanjoes) (Swift target merging)
+* [Ben Hamilton](https://github.com/bhamiltoncx) (Full Unicode support in serialized ATN and all languages' runtimes for code points > U+FFFF)
 
 ## Useful information
 

antlr4-maven-plugin/pom.xml

 <!--
-  ~ Copyright (c) 2012-2016 The ANTLR Project. All rights reserved.
+  ~ Copyright (c) 2012-2017 The ANTLR Project. All rights reserved.
   ~ Use of this file is governed by the BSD 3-clause license that
   ~ can be found in the LICENSE.txt file in the project root.
   -->
@@ -8,7 +8,7 @@
   <parent>
     <groupId>org.antlr</groupId>
     <artifactId>antlr4-master</artifactId>
-    <version>4.6</version>
+    <version>4.7.1</version>
   </parent>
   <artifactId>antlr4-maven-plugin</artifactId>
   <packaging>maven-plugin</packaging>

antlr4-maven-plugin/resources/META-INF/m2e/lifecycle-mapping-metadata.xml

 <?xml version="1.0" encoding="utf-8"?>
 <!--
-  ~ Copyright (c) 2012-2016 The ANTLR Project. All rights reserved.
+  ~ Copyright (c) 2012-2017 The ANTLR Project. All rights reserved.
   ~ Use of this file is governed by the BSD 3-clause license that
   ~ can be found in the LICENSE.txt file in the project root.
   -->

antlr4-maven-plugin/src/main/java/org/antlr/mojo/antlr4/Antlr4ErrorLog.java

 /*
- * Copyright (c) 2012-2016 The ANTLR Project. All rights reserved.
+ * Copyright (c) 2012-2017 The ANTLR Project. All rights reserved.
  * Use of this file is governed by the BSD 3-clause license that
  * can be found in the LICENSE.txt file in the project root.
  */

antlr4-maven-plugin/src/main/java/org/antlr/mojo/antlr4/Antlr4Mojo.java

 /*
- * Copyright (c) 2012-2016 The ANTLR Project. All rights reserved.
+ * Copyright (c) 2012-2017 The ANTLR Project. All rights reserved.
  * Use of this file is governed by the BSD 3-clause license that
  * can be found in the LICENSE.txt file in the project root.
  */
@@ -150,6 +150,13 @@ public class Antlr4Mojo extends AbstractMojo {
     @Parameter(property = "project", required = true, readonly = true)
     protected MavenProject project;
 
+	/**
+	 * Specifies whether sources are added to the {@code compile} or
+	 * {@code test} scope.
+	 */
+	@Parameter(property = "antlr4.generateTestSources", defaultValue = "false")
+	private boolean generateTestSources;
+
     /**
      * The directory where the ANTLR grammar files ({@code *.g4}) are located.
      */
@@ -190,8 +197,13 @@ public class Antlr4Mojo extends AbstractMojo {
     }
 
     void addSourceRoot(File outputDir) {
+		if (generateTestSources) {
+			project.addTestCompileSourceRoot(outputDir.getPath());
+		}
+		else {
 			project.addCompileSourceRoot(outputDir.getPath());
 		}
+    }
 
     /**
      * An instance of the ANTLR tool build
@@ -383,7 +395,7 @@ public class Antlr4Mojo extends AbstractMojo {
             String tokensFileName = grammarFile.getName().split("\\.")[0] + ".tokens";
             File outputFile = new File(outputDirectory, tokensFileName);
             if ( (! outputFile.exists()) ||
-                 outputFile.lastModified() < grammarFile.lastModified() ||
+                 outputFile.lastModified() <= grammarFile.lastModified() ||
                  dependencies.isDependencyChanged(grammarFile)) {
                 grammarFilesToProcess.add(grammarFile);
             }
@@ -400,10 +412,7 @@ public class Antlr4Mojo extends AbstractMojo {
 		// Iterate each grammar file we were given and add it into the tool's list of
 		// grammars to process.
 		for (File grammarFile : grammarFiles) {
-			if (!buildContext.hasDelta(grammarFile)) {
-				continue;
-			}
-
+			buildContext.refresh(grammarFile);
 			buildContext.removeMessages(grammarFile);
 
 			getLog().debug("Grammar file '" + grammarFile.getPath() + "' detected.");

antlr4-maven-plugin/src/main/java/org/antlr/mojo/antlr4/GrammarDependencies.java

 /*
- * Copyright (c) 2012-2016 The ANTLR Project. All rights reserved.
+ * Copyright (c) 2012-2017 The ANTLR Project. All rights reserved.
  * Use of this file is governed by the BSD 3-clause license that
  * can be found in the LICENSE.txt file in the project root.
  */
@@ -216,8 +216,7 @@ class GrammarDependencies {
             return;
 
         for (GrammarAST importDecl : grammar.getAllChildrenWithType(ANTLRParser.IMPORT)) {
-            Tree id = importDecl.getFirstChildWithType(ANTLRParser.ID);
-
+            for (Tree id: importDecl.getAllChildrenWithType(ANTLRParser.ID)) {
                 // missing id is not valid, but we don't want to prevent the root cause from
                 // being reported by the ANTLR tool
                 if (id != null) {
@@ -226,6 +225,7 @@ class GrammarDependencies {
                     graph.addEdge(id.getText() + ".g4", grammarPath);
                 }
             }
+        }
 
         for (GrammarAST options : grammar.getAllChildrenWithType(ANTLRParser.OPTIONS)) {
             for (int i = 0, count = options.getChildCount(); i < count; i++) {

antlr4-maven-plugin/src/main/java/org/antlr/mojo/antlr4/MojoUtils.java

 /*
- * Copyright (c) 2012-2016 The ANTLR Project. All rights reserved.
+ * Copyright (c) 2012-2017 The ANTLR Project. All rights reserved.
  * Use of this file is governed by the BSD 3-clause license that
  * can be found in the LICENSE.txt file in the project root.
  */

antlr4-maven-plugin/src/site/site.xml

 <?xml version="1.0" encoding="UTF-8"?>
 
 <!--
-  ~ Copyright (c) 2012-2016 The ANTLR Project. All rights reserved.
+  ~ Copyright (c) 2012-2017 The ANTLR Project. All rights reserved.
   ~ Use of this file is governed by the BSD 3-clause license that
   ~ can be found in the LICENSE.txt file in the project root.
   -->

antlr4-maven-plugin/src/test/java/org/antlr/mojo/antlr4/Antlr4MojoTest.java

 /*
- * Copyright (c) 2012-2016 The ANTLR Project. All rights reserved.
+ * Copyright (c) 2012-2017 The ANTLR Project. All rights reserved.
  * Use of this file is governed by the BSD 3-clause license that
  * can be found in the LICENSE.txt file in the project root.
  */
@@ -202,6 +202,7 @@ public class Antlr4MojoTest {
         Path genHello = generatedSources.resolve("test/HelloParser.java");
 
         Path baseGrammar = antlrDir.resolve("imports/TestBaseLexer.g4");
+        Path baseGrammar2 = antlrDir.resolve("imports/TestBaseLexer2.g4");
         Path lexerGrammar = antlrDir.resolve("test/TestLexer.g4");
         Path parserGrammar = antlrDir.resolve("test/TestParser.g4");
 
@@ -222,21 +223,20 @@ public class Antlr4MojoTest {
         assertTrue(Files.exists(genHello));
         assertTrue(Files.exists(genTestParser));
         assertTrue(Files.exists(genTestLexer));
+        byte[] origTestLexerSum = checksum(genTestLexer);
+        byte[] origTestParserSum = checksum(genTestParser);
+        byte[] origHelloSum = checksum(genHello);
 
         ////////////////////////////////////////////////////////////////////////
         // 2nd - nothing has been modified, no grammars have to be processed
         ////////////////////////////////////////////////////////////////////////
 
         {
-            byte[] testLexerSum = checksum(genTestLexer);
-            byte[] testParserSum = checksum(genTestParser);
-            byte[] helloSum = checksum(genHello);
-
             maven.executeMojo(session, project, exec);
 
-            assertTrue(Arrays.equals(testLexerSum, checksum(genTestLexer)));
-            assertTrue(Arrays.equals(testParserSum, checksum(genTestParser)));
-            assertTrue(Arrays.equals(helloSum, checksum(genHello)));
+            assertTrue(Arrays.equals(origTestLexerSum, checksum(genTestLexer)));
+            assertTrue(Arrays.equals(origTestParserSum, checksum(genTestParser)));
+            assertTrue(Arrays.equals(origHelloSum, checksum(genHello)));
         }
 
         ////////////////////////////////////////////////////////////////////////
@@ -245,50 +245,71 @@ public class Antlr4MojoTest {
 
         // modify the grammar to make checksum comparison detect a change
         try(Change change = Change.of(baseGrammar, "DOT: '.' ;")) {
-            byte[] testLexerSum = checksum(genTestLexer);
-            byte[] testParserSum = checksum(genTestParser);
-            byte[] helloSum = checksum(genHello);
-
             maven.executeMojo(session, project, exec);
 
-            assertFalse(Arrays.equals(testLexerSum, checksum(genTestLexer)));
-            assertFalse(Arrays.equals(testParserSum, checksum(genTestParser)));
-            assertTrue(Arrays.equals(helloSum, checksum(genHello)));
+            assertFalse(Arrays.equals(origTestLexerSum, checksum(genTestLexer)));
+            assertFalse(Arrays.equals(origTestParserSum, checksum(genTestParser)));
+            assertTrue(Arrays.equals(origHelloSum, checksum(genHello)));
         }
+        // Restore file and confirm it was restored.
+        maven.executeMojo(session, project, exec);
+        assertTrue(Arrays.equals(origTestLexerSum, checksum(genTestLexer)));
+        assertTrue(Arrays.equals(origTestParserSum, checksum(genTestParser)));
+        assertTrue(Arrays.equals(origHelloSum, checksum(genHello)));
 
         ////////////////////////////////////////////////////////////////////////
-        // 4th - the lexer grammar changed, the parser grammar has to be processed as well
+        // 4th - the second imported grammar changed, every dependency has to be processed
         ////////////////////////////////////////////////////////////////////////
 
         // modify the grammar to make checksum comparison detect a change
-        try(Change change = Change.of(lexerGrammar)) {
-            byte[] testLexerSum = checksum(genTestLexer);
-            byte[] testParserSum = checksum(genTestParser);
-            byte[] helloSum = checksum(genHello);
+        try(Change change = Change.of(baseGrammar2, "BANG: '!' ;")) {
+            maven.executeMojo(session, project, exec);
 
+            assertFalse(Arrays.equals(origTestLexerSum, checksum(genTestLexer)));
+            assertFalse(Arrays.equals(origTestParserSum, checksum(genTestParser)));
+            assertTrue(Arrays.equals(origHelloSum, checksum(genHello)));
+        }
+        // Restore file and confirm it was restored.
         maven.executeMojo(session, project, exec);
+        assertTrue(Arrays.equals(origTestLexerSum, checksum(genTestLexer)));
+        assertTrue(Arrays.equals(origTestParserSum, checksum(genTestParser)));
+        assertTrue(Arrays.equals(origHelloSum, checksum(genHello)));
 
-            assertFalse(Arrays.equals(testLexerSum, checksum(genTestLexer)));
-            assertFalse(Arrays.equals(testParserSum, checksum(genTestParser)));
-            assertTrue(Arrays.equals(helloSum, checksum(genHello)));
+        ////////////////////////////////////////////////////////////////////////
+        // 5th - the lexer grammar changed, the parser grammar has to be processed as well
+        ////////////////////////////////////////////////////////////////////////
+
+        // modify the grammar to make checksum comparison detect a change
+        try(Change change = Change.of(lexerGrammar, "FOO: 'foo' ;")) {
+            maven.executeMojo(session, project, exec);
+
+            assertFalse(Arrays.equals(origTestLexerSum, checksum(genTestLexer)));
+            assertFalse(Arrays.equals(origTestParserSum, checksum(genTestParser)));
+            assertTrue(Arrays.equals(origHelloSum, checksum(genHello)));
         }
+        // Restore file and confirm it was restored.
+        maven.executeMojo(session, project, exec);
+        assertTrue(Arrays.equals(origTestLexerSum, checksum(genTestLexer)));
+        assertTrue(Arrays.equals(origTestParserSum, checksum(genTestParser)));
+        assertTrue(Arrays.equals(origHelloSum, checksum(genHello)));
 
         ////////////////////////////////////////////////////////////////////////
-        // 5th - the parser grammar changed, no other grammars have to be processed
+        // 6th - the parser grammar changed, no other grammars have to be processed
         ////////////////////////////////////////////////////////////////////////
 
         // modify the grammar to make checksum comparison detect a change
         try(Change change = Change.of(parserGrammar, " t : WS* ;")) {
-            byte[] testLexerSum = checksum(genTestLexer);
-            byte[] testParserSum = checksum(genTestParser);
-            byte[] helloSum = checksum(genHello);
-
             maven.executeMojo(session, project, exec);
 
-            assertTrue(Arrays.equals(testLexerSum, checksum(genTestLexer)));
-            assertFalse(Arrays.equals(testParserSum, checksum(genTestParser)));
-            assertTrue(Arrays.equals(helloSum, checksum(genHello)));
+            assertTrue(Arrays.equals(origTestLexerSum, checksum(genTestLexer)));
+            assertFalse(Arrays.equals(origTestParserSum, checksum(genTestParser)));
+            assertTrue(Arrays.equals(origHelloSum, checksum(genHello)));
         }
+        // Restore file and confirm it was restored.
+        maven.executeMojo(session, project, exec);
+        assertTrue(Arrays.equals(origTestLexerSum, checksum(genTestLexer)));
+        assertTrue(Arrays.equals(origTestParserSum, checksum(genTestParser)));
+        assertTrue(Arrays.equals(origHelloSum, checksum(genHello)));
     }
 
     @Test

antlr4-maven-plugin/src/test/projects/dependencyRemoved/pom.xml

 <!--
-  ~ Copyright (c) 2012-2016 The ANTLR Project. All rights reserved.
+  ~ Copyright (c) 2012-2017 The ANTLR Project. All rights reserved.
   ~ Use of this file is governed by the BSD 3-clause license that
   ~ can be found in the LICENSE.txt file in the project root.
   -->

antlr4-maven-plugin/src/test/projects/importTokens/pom.xml

 <!--
-  ~ Copyright (c) 2012-2016 The ANTLR Project. All rights reserved.
+  ~ Copyright (c) 2012-2017 The ANTLR Project. All rights reserved.
   ~ Use of this file is governed by the BSD 3-clause license that
   ~ can be found in the LICENSE.txt file in the project root.
   -->

antlr4-maven-plugin/src/test/projects/importsCustom/pom.xml

 <!--
-  ~ Copyright (c) 2012-2016 The ANTLR Project. All rights reserved.
+  ~ Copyright (c) 2012-2017 The ANTLR Project. All rights reserved.
   ~ Use of this file is governed by the BSD 3-clause license that
   ~ can be found in the LICENSE.txt file in the project root.
   -->

antlr4-maven-plugin/src/test/projects/importsStandard/pom.xml

 <!--
-  ~ Copyright (c) 2012-2016 The ANTLR Project. All rights reserved.
+  ~ Copyright (c) 2012-2017 The ANTLR Project. All rights reserved.
   ~ Use of this file is governed by the BSD 3-clause license that
   ~ can be found in the LICENSE.txt file in the project root.
   -->

antlr4-maven-plugin/src/test/projects/importsStandard/src/main/antlr4/imports/TestBaseLexer.g4

@@ -11,6 +11,3 @@ Whitespace  :  ' ' | '\n' | '\t' | '\r' ;
 
 fragment
 Hexdigit    :  [a-fA-F0-9] ;
\ No newline at end of file
-
-fragment
-Digit       :  [0-9] ;

antlr4-maven-plugin/src/test/projects/importsStandard/src/main/antlr4/imports/TestBaseLexer2.g4

+lexer grammar TestBaseLexer2;
+
+fragment
+Digit       :  [0-9] ;

antlr4-maven-plugin/src/test/projects/importsStandard/src/main/antlr4/test/TestLexer.g4

 lexer grammar TestLexer;
 
-import TestBaseLexer;
+import TestBaseLexer, TestBaseLexer2;
 
 WS    :  Whitespace+              -> skip;
 TEXT  :  ~[<&]+ ; // match any 16 bit char other than < and &