Skip to content
Commits on Source (3)
# ANTLR v4
[![Build Travis-CI Status](https://travis-ci.org/antlr/antlr4.png?branch=master)](https://travis-ci.org/antlr/antlr4) [![Build AppVeyor Status](https://ci.appveyor.com/api/projects/status/5acpbx1pg7bhgh8v/branch/master?svg=true)](https://ci.appveyor.com/project/parrt/antlr4) [![Java 7+](https://img.shields.io/badge/java-7+-4c7e9f.svg)](http://java.oracle.com) [![License](https://img.shields.io/badge/license-BSD-blue.svg)](https://raw.githubusercontent.com/antlr/antlr4/master/LICENSE.txt)
[![Build Travis-CI Status](https://travis-ci.org/antlr/antlr4.svg?branch=master)](https://travis-ci.org/antlr/antlr4) [![Build AppVeyor Status](https://ci.appveyor.com/api/projects/status/5acpbx1pg7bhgh8v/branch/master?svg=true)](https://ci.appveyor.com/project/parrt/antlr4) [![Java 7+](https://img.shields.io/badge/java-7+-4c7e9f.svg)](http://java.oracle.com) [![License](https://img.shields.io/badge/license-BSD-blue.svg)](https://raw.githubusercontent.com/antlr/antlr4/master/LICENSE.txt)
**ANTLR** (ANother Tool for Language Recognition) is a powerful parser generator for reading, processing, executing, or translating structured text or binary files. It's widely used to build languages, tools, and frameworks. From a grammar, ANTLR generates a parser that can build parse trees and also generates a listener interface (or visitor) that makes it easy to respond to the recognition of phrases of interest.
......
......@@ -8,7 +8,7 @@
<parent>
<groupId>org.antlr</groupId>
<artifactId>antlr4-master</artifactId>
<version>4.7.1</version>
<version>4.7.2</version>
</parent>
<artifactId>antlr4-maven-plugin</artifactId>
<packaging>maven-plugin</packaging>
......
......@@ -178,3 +178,35 @@ YYYY/MM/DD, github id, Full name, email
2017/12/01, DavidMoraisFerreira, David Morais Ferreira, david.moraisferreira@gmail.com
2017/12/01, SebastianLng, Sebastian Lang, sebastian.lang@outlook.com
2017/12/03, oranoran, Oran Epelbaum, oran / epelbaum me
2017/12/12, janlinde, Jan Lindemann, jan@janware.com
2017/12/13, enessoylu, Enes Soylu, enessoylutr@gmail.com
2017/12/20, kbsletten, Kyle Sletten, kbsletten@gmail.com
2017/12/27, jkmar, Jakub Marciniszyn, marciniszyn.jk@gmail.com
2018/03/08, dannoc, Daniel Clifford, danno@google.com
2018/03/10, uvguy, kangjoni76@gmail.com
2018/01/06, kasbah, Kaspar Emanuel, kaspar@monostable.co.uk
2018/01/15, xgcssch, Sönke Schau, xgcssch@users.noreply.github.com
2018/02/08, razfriman, Raz Friman, raz@razfriman.com
2018/02/11, io7m, Mark Raynsford, code@io7m.com
2018/04/24, solussd, Joe Smith, joe@uwcreations.com
2018/15/05, johnvanderholt, jan dillingh johnvanderholte@gmail.com
2018/06/14, scadgek, Sergey Chupov, scadgek@live.com
2018/06/16, EternalPhane, Zongyuan Zuo, eternalphane@gmail.com
2018/06/27, wu-sheng, Wu Sheng, wu.sheng@foxmail.com
2018/02/25, chaseoxide, Marcus Ong, taccs97[at]gmail[dot]com
2018/05/15, johnvanderholt, jan dillingh johnvanderholte@gmail.com
2018/06/16, EternalPhane, Zongyuan Zuo, eternalphane@gmail.com
2018/05/15, johnvanderholt, jan dillingh johnvanderholte@gmail.com
2018/05/17, sinopsysHK, Eric Bardes, sinofwd@gmail.com
2018/05/23, srvance, Stephen Vance, steve@vance.com
2018/06/14, alecont, Alessandro Contenti, alecontenti@hotmail.com
2018/06/16, EternalPhane, Zongyuan Zuo, eternalphane@gmail.com
2018/07/03, jgoppert, James Goppert, james.goppert@gmail.com
2018/07/27, Maksim Novikov, mnovikov.work@gmail.com
2018/07/31, Lucas Henrqiue, lucashenrique580@gmail.com
2018/08/03, ENDOH takanao, djmchl@gmail.com
2018/10/29, chrisaycock, Christopher Aycock, chris[at]chrisaycock[dot]com
2018/11/12, vinoski, Steve Vinoski, vinoski@ieee.org
2018/11/14, nxtstep, Adriaan (Arjan) Duz, codewithadriaan[et]gmail[dot]com
2018/11/15, amykyta3, Alex Mykyta, amykyta3@users.noreply.github.com
2018/11/29, hannemann-tamas, Ralf Hannemann-Tamas, ralf.ht@gmail.com
\ No newline at end of file
antlr4 (4.7.1-2) UNRELEASED; urgency=medium
antlr4 (4.7.2-1) UNRELEASED; urgency=medium
[ Andrius Merkys ]
* Adding /usr/share/java/treelayout.jar to CLASSPATH in grun (Closes: #902230).
-- Andrius Merkys <andrius.merkys@gmail.com> Wed, 02 Jan 2019 06:04:26 -0500
[ Andrej Shadura ]
* New upstream release.
-- Andrej Shadura <andrewsh@debian.org> Tue, 29 Jan 2019 17:36:49 +0100
antlr4 (4.7.1-1) unstable; urgency=medium
......
......@@ -42,9 +42,9 @@ In order to perform the tests on all target languages, you need to have the foll
* `mono` (e.g., `brew install mono`) on non-Windows boxes (on Windows it uses the Microsoft .net stack). Also must [`xbuild` the runtime](https://github.com/antlr/antlr4/blob/master/doc/releasing-antlr.md) before tests will run; see below
* `nodejs`
* Python 2.7
* Python 3.5
* Python 3.6
* Go
* Swift 3 (via XCode 8.x) tested currently only osx
* Swift 4 (via XCode 10.x) tested currently only osx
* clang (for C++ target)
*
To **install into local repository** `~/.m2/repository/org/antlr`, do this:
......
......@@ -28,16 +28,7 @@ Checking out files: 100% (1427/1427), done.
# Compile
```bash
$ cd /tmp
$ git clone git@github.com:antlr/antlr4.git
Cloning into 'antlr4'...
remote: Counting objects: 59858, done.
remote: Compressing objects: 100% (57/57), done.
remote: Total 59858 (delta 28), reused 9 (delta 9), pack-reused 59786
Receiving objects: 100% (59858/59858), 31.10 MiB | 819.00 KiB/s, done.
Resolving deltas: 100% (31898/31898), done.
Checking connectivity... done.
$ cd antlr4
$ cd /tmp/antlr4
$ export MAVEN_OPTS="-Xmx1G" # don't forget this on linux
$ mvn clean # must be separate, not part of install/compile
$ mvn -DskipTests install
......
......@@ -32,6 +32,7 @@ Now a fully functioning code might look like the following for start rule `Start
```
using Antlr4.Runtime;
using Antlr4.Runtime.Tree;
public void MyParseMethod() {
String input = "your text to parse here";
......@@ -39,7 +40,7 @@ public void MyParseMethod() {
ITokenSource lexer = new MyGrammarLexer(stream);
ITokenStream tokens = new CommonTokenStream(lexer);
MyGrammarParser parser = new MyGrammarParser(tokens);
parser.buildParseTrees = true;
parser.BuildParseTree = true;
IParseTree tree = parser.StartRule();
}
```
......
......@@ -16,10 +16,10 @@ If you are going to integrate ANTLR into your existing build system using mvn, a
1. Download
```
$ cd /usr/local/lib
$ curl -O http://www.antlr.org/download/antlr-4.7.1-complete.jar
$ curl -O https://www.antlr.org/download/antlr-4.7.1-complete.jar
```
Or just download in browser from website:
[http://www.antlr.org/download.html](http://www.antlr.org/download.html)
[https://www.antlr.org/download.html](https://www.antlr.org/download.html)
and put it somewhere rational like `/usr/local/lib`.
2. Add `antlr-4.7.1-complete.jar` to your `CLASSPATH`:
......@@ -31,7 +31,7 @@ It's also a good idea to put this in your `.bash_profile` or whatever your start
3. Create aliases for the ANTLR Tool, and `TestRig`.
```
$ alias antlr4='java -Xmx500M -cp "/usr/local/lib/antlr-4.7.1-complete.jar:$CLASSPATH" org.antlr.v4.Tool'
$ alias grun='java org.antlr.v4.gui.TestRig'
$ alias grun='java -Xmx500M -cp "/usr/local/lib/antlr-4.7.1-complete.jar:$CLASSPATH" org.antlr.v4.gui.TestRig'
```
### WINDOWS
......@@ -39,9 +39,9 @@ $ alias grun='java org.antlr.v4.gui.TestRig'
(*Thanks to Graham Wideman*)
0. Install Java (version 1.6 or higher)
1. Download antlr-4.5.3-complete.jar (or whatever version) from [http://www.antlr.org/download/](http://www.antlr.org/download/)
1. Download antlr-4.7.1-complete.jar (or whatever version) from [https://www.antlr.org/download/](https://www.antlr.org/download/)
Save to your directory for 3rd party Java libraries, say `C:\Javalib`
2. Add `antlr-4.5.3-complete.jar` to CLASSPATH, either:
2. Add `antlr-4.7.1-complete.jar` to CLASSPATH, either:
* Permanently: Using System Properties dialog > Environment variables > Create or append to `CLASSPATH` variable
* Temporarily, at command line:
```
......@@ -53,6 +53,10 @@ SET CLASSPATH=.;C:\Javalib\antlr-4.7.1-complete.jar;%CLASSPATH%
java org.antlr.v4.Tool %*
```
```
@ECHO OFF
SET TEST_CURRENT_DIR=%CLASSPATH:.;=%
if "%TEST_CURRENT_DIR%" == "%CLASSPATH%" ( SET CLASSPATH=.;%CLASSPATH% )
@ECHO ON
java org.antlr.v4.gui.TestRig %*
```
* Or, use doskey commands:
......
......@@ -16,7 +16,7 @@ Also, I have prepared a [video](https://youtu.be/eW4WFgRtFeY) that will help you
### Eclipse
Edgar Espina has created an [eclipse plugin for ANTLR v4](https://youtu.be/eW4WFgRtFeY). Features: Advanced Syntax Highlighting, Automatic Code Generation (on save), Manual Code Generation (through External Tools menu), Code Formatter (Ctrl+Shift+F), Syntax Diagrams, Advanced Rule Navigation between files (F3), Quick fixes.
Edgar Espina has created an [eclipse plugin for ANTLR v4](https://marketplace.eclipse.org/content/antlr-ide). Features: Advanced Syntax Highlighting, Automatic Code Generation (on save), Manual Code Generation (through External Tools menu), Code Formatter (Ctrl+Shift+F), Syntax Diagrams, Advanced Rule Navigation between files (F3), Quick fixes.
### NetBeans
......
......@@ -62,7 +62,7 @@ The steps to create your parsing code are the following:
You are now ready to bundle your parsing code as follows:
- following webpack specs, create a webpack.config file
- in the webpack.config file, exclude node.js only modules using: node: { module: "empty", net: "empty", fs: "empty" }
- from the cmd line, nag-vigate to the directory containing webpack.config and type: webpack
- from the cmd line, navigate to the directory containing webpack.config and type: webpack
This will produce a single js file containing all your parsing code. Easy to include in your web pages!
......@@ -95,11 +95,16 @@ Let's suppose that your grammar is named, as above, "MyGrammar". Let's suppose t
Now a fully functioning script might look like the following:
```javascript
var antlr4 = require('antlr4');
var MyGrammarLexer = require('./MyGrammarLexer').MyGrammarLexer;
var MyGrammarParser = require('./MyGrammarParser').MyGrammarParser;
var MyGrammarListener = require('./MyGrammarListener').MyGrammarListener;
var input = "your text to parse here"
var chars = new antlr4.InputStream(input);
var lexer = new MyGrammarLexer.MyGrammarLexer(chars);
var lexer = new MyGrammarLexer(chars);
var tokens = new antlr4.CommonTokenStream(lexer);
var parser = new MyGrammarParser.MyGrammarParser(tokens);
var parser = new MyGrammarParser(tokens);
parser.buildParseTrees = true;
var tree = parser.MyStartRule();
```
......@@ -130,12 +135,12 @@ Let's suppose your MyGrammar grammar comprises 2 rules: "key" and "value". The a
In order to provide custom behavior, you might want to create the following class:
```javascript
KeyPrinter = function() {
var KeyPrinter = function() {
MyGrammarListener.call(this); // inherit default listener
return this;
};
// inherit default listener
// continue inheriting default listener
KeyPrinter.prototype = Object.create(MyGrammarListener.prototype);
KeyPrinter.prototype.constructor = KeyPrinter;
......@@ -149,7 +154,7 @@ In order to execute this listener, you would simply add the following lines to t
```javascript
...
tree = parser.StartRule() - only repeated here for reference
tree = parser.StartRule() // only repeated here for reference
var printer = new KeyPrinter();
antlr4.tree.ParseTreeWalker.DEFAULT.walk(printer, tree);
```
......
......@@ -150,13 +150,19 @@ Uploaded: https://oss.sonatype.org/content/repositories/snapshots/org/antlr/antl
The maven deploy lifecycle phased deploys the artifacts and the poms for the ANTLR project to the [sonatype remote staging server](https://oss.sonatype.org/content/repositories/snapshots/).
```bash
mvn deploy -DskipTests
export JAVA_HOME=`/usr/libexec/java_home -v 1.7`; mvn deploy -DskipTests
```
With JDK 1.7 (not 6 or 8), do this:
```bash
mvn release:prepare -Darguments="-DskipTests"
export JAVA_HOME=`/usr/libexec/java_home -v 1.7`; mvn release:prepare -Darguments="-DskipTests"
```
Hm...per https://github.com/keybase/keybase-issues/issues/1712 we need this to make gpg work:
```bash
export GPG_TTY=$(tty)
```
Side note to set jdk 1.7 on os x:
......@@ -166,6 +172,13 @@ alias java='/Library/Java/JavaVirtualMachines/jdk1.7.0_21.jdk/Contents/Home/bin/
alias javac='/Library/Java/JavaVirtualMachines/jdk1.7.0_21.jdk/Contents/Home/bin/javac'
alias javadoc='/Library/Java/JavaVirtualMachines/jdk1.7.0_21.jdk/Contents/Home/bin/javadoc'
alias jar='/Library/Java/JavaVirtualMachines/jdk1.7.0_21.jdk/Contents/Home/bin/jar'
export JAVA_HOME=`/usr/libexec/java_home -v 1.7`
```
But I think just this on front of mvn works:
```
export JAVA_HOME=`/usr/libexec/java_home -v 1.7`; mvn ...
```
You should see 0x33 in generated .class files after 0xCAFEBABE; see [Java SE 7 = 51 (0x33 hex)](https://en.wikipedia.org/wiki/Java_class_file):
......@@ -260,7 +273,9 @@ popd
### CSharp
*Publishing to Nuget from Windows*
Now we have [appveyor create artifact](https://ci.appveyor.com/project/parrt/antlr4/build/artifacts). Go to [nuget](https://www.nuget.org/packages/manage/upload) to upload the `.nupkg`.
### Publishing to Nuget from Windows
**Install the pre-requisites**
......@@ -326,13 +341,12 @@ index-servers =
pypitest
[pypi]
repository: https://pypi.python.org/pypi
username: parrt
password: XXX
password: xxx
[pypitest]
repository: https://testpypi.python.org/pypi
username: parrt
password: xxx
```
Then run the usual python set up stuff:
......@@ -340,8 +354,7 @@ Then run the usual python set up stuff:
```bash
cd ~/antlr/code/antlr4/runtime/Python2
# assume you have ~/.pypirc set up
python setup.py register -r pypi
python setup.py sdist bdist_wininst upload -r pypi
python2 setup.py sdist upload
```
and do again for Python 3 target
......@@ -349,8 +362,7 @@ and do again for Python 3 target
```bash
cd ~/antlr/code/antlr4/runtime/Python3
# assume you have ~/.pypirc set up
python setup.py register -r pypi
python setup.py sdist bdist_wininst upload -r pypi
python3 setup.py sdist upload
```
There are links to the artifacts in [download.html](http://www.antlr.org/download.html) already.
......@@ -384,12 +396,12 @@ cd runtime/Cpp
cp antlr4-cpp-runtime-source.zip ~/antlr/sites/website-antlr4/download/antlr4-cpp-runtime-4.7-source.zip
```
On a Windows machine the build scripts checks if VS 2013 and/or VS 2015 are installed and builds binaries for each, if found. This script requires 7z to be installed (http://7-zip.org).
On a Windows machine the build scripts checks if VS 2013 and/or VS 2015 are installed and builds binaries for each, if found. This script requires 7z to be installed (http://7-zip.org then do `set PATH=%PATH%;C:\Program Files\7-Zip\` from DOS not powershell).
```bash
cd runtime/Cpp
deploy-windows.cmd
cp antlr4-cpp-runtime-vs2015.zip ~/antlr/sites/website-antlr4/download/antlr4-cpp-runtime-4.7-vs2015.zip
cp runtime\bin\vs-2015\x64\Release DLL\antlr4-cpp-runtime-vs2015.zip ~/antlr/sites/website-antlr4/download/antlr4-cpp-runtime-4.7-vs2015.zip
```
Move target to website (**_rename to a specific ANTLR version first if needed_**):
......
# ANTLR4 Language Target, Runtime for Swift
## Requirements
ANTLR 4.7.2 requires Swift 4.2. It works on Swift 4.2.1 also.
ANTLR 4.7.1 requires Swift 4.0, and does not work on Swift 4.2. (The status of
Swift 4.1 support is unknown.)
## Performance Note
To use ANTLR4 Swift target in production environment, make sure to turn on compiler optimizations by following [these instructions](https://github.com/apple/swift-package-manager/blob/master/Documentation/Usage.md#build-configurations) if you use SwiftPM to build your project. If you are using Xcode to build your project, it's unlikely you will not use `release` build for production build.
......
......@@ -13,7 +13,7 @@
</parent>
<groupId>org.antlr</groupId>
<artifactId>antlr4-master</artifactId>
<version>4.7.1</version>
<version>4.7.2</version>
<packaging>pom</packaging>
<name>ANTLR 4</name>
......@@ -138,6 +138,8 @@
<filesets>
<fileset>
<directory>runtime/Swift/.build</directory>
</fileset>
<fileset>
<directory>runtime/Swift/Tests/Antlr4Tests/gen</directory>
</fileset>
</filesets>
......
......@@ -9,7 +9,7 @@
<parent>
<groupId>org.antlr</groupId>
<artifactId>antlr4-master</artifactId>
<version>4.7.1</version>
<version>4.7.2</version>
<relativePath>../../pom.xml</relativePath>
</parent>
<artifactId>antlr4-runtime-test-annotations</artifactId>
......
......@@ -10,7 +10,7 @@
<parent>
<groupId>org.antlr</groupId>
<artifactId>antlr4-master</artifactId>
<version>4.7.1</version>
<version>4.7.2</version>
</parent>
<artifactId>antlr4-runtime-testsuite</artifactId>
<name>ANTLR 4 Runtime Tests (2nd generation)</name>
......@@ -26,7 +26,7 @@
<dependency>
<groupId>org.antlr</groupId>
<artifactId>ST4</artifactId>
<version>4.0.8</version>
<version>4.1</version>
<scope>test</scope>
</dependency>
<dependency>
......@@ -68,7 +68,7 @@
<dependency>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-server</artifactId>
<version>8.1.16.v20140903</version>
<version>[9.3.24.v20180605,)</version>
<scope>test</scope>
</dependency>
<dependency>
......@@ -92,6 +92,11 @@
</resource>
<resource>
<directory>../runtime</directory>
<excludes>
<exclude>**/.build/**</exclude>
<exclude>**/target/**</exclude>
<exclude>Swift/*.xcodeproj/**</exclude>
</excludes>
</resource>
</resources>
<plugins>
......
......@@ -9,7 +9,7 @@
<parent>
<groupId>org.antlr</groupId>
<artifactId>antlr4-master</artifactId>
<version>4.7.1</version>
<version>4.7.2</version>
<relativePath>../../pom.xml</relativePath>
</parent>
<artifactId>antlr4-runtime-test-annotation-processors</artifactId>
......@@ -18,11 +18,9 @@
<dependencies>
<dependency>
<groupId>com.sun</groupId>
<artifactId>tools</artifactId>
<version>1.4.2</version>
<scope>system</scope>
<systemPath>${java.home}/../lib/tools.jar</systemPath>
<groupId>com.github.olivergondza</groupId>
<artifactId>maven-jdk-tools-wrapper</artifactId>
<version>0.1</version>
</dependency>
<dependency>
<groupId>org.antlr</groupId>
......
......@@ -9,7 +9,7 @@
<parent>
<groupId>org.antlr</groupId>
<artifactId>antlr4-master</artifactId>
<version>4.7.1</version>
<version>4.7.2</version>
<relativePath>../../pom.xml</relativePath>
</parent>
<artifactId>antlr4-runtime</artifactId>
......@@ -96,6 +96,7 @@
<phase>process-classes</phase>
<configuration>
<instructions>
<Automatic-Module-Name>org.antlr.antlr4.runtime</Automatic-Module-Name>
<Bundle-SymbolicName>org.antlr.antlr4-runtime</Bundle-SymbolicName>
</instructions>
</configuration>
......
......@@ -203,8 +203,8 @@ public abstract class CodePointCharStream implements CharStream {
/** Return the UTF-16 encoded string for the given interval */
@Override
public String getText(Interval interval) {
int startIdx = Math.min(interval.a, size - 1);
int len = Math.min(interval.b - interval.a + 1, size);
int startIdx = Math.min(interval.a, size);
int len = Math.min(interval.b - interval.a + 1, size - startIdx);
// We know there are no surrogates in this
// array, since otherwise we would be given a
......@@ -258,8 +258,8 @@ public abstract class CodePointCharStream implements CharStream {
/** Return the UTF-16 encoded string for the given interval */
@Override
public String getText(Interval interval) {
int startIdx = Math.min(interval.a, size - 1);
int len = Math.min(interval.b - interval.a + 1, size);
int startIdx = Math.min(interval.a, size);
int len = Math.min(interval.b - interval.a + 1, size - startIdx);
// Note that we pass the int[] code points to the String constructor --
// this is supported, and the constructor will convert to UTF-16 internally.
......
......@@ -67,7 +67,7 @@ public class RuntimeMetaData {
* omitted.</li>
* </ul>
*/
public static final String VERSION = "4.7.1";
public static final String VERSION = "4.7.2";
/**
* Gets the currently executing version of the ANTLR 4 runtime library.
......
......@@ -316,14 +316,14 @@ public abstract class PredictionContext {
}
else {
if ( a == EMPTY && b == EMPTY ) return EMPTY; // $ + $ = $
if ( a == EMPTY ) { // $ + x = [$,x]
if ( a == EMPTY ) { // $ + x = [x,$]
int[] payloads = {b.returnState, EMPTY_RETURN_STATE};
PredictionContext[] parents = {b.parent, null};
PredictionContext joined =
new ArrayPredictionContext(parents, payloads);
return joined;
}
if ( b == EMPTY ) { // x + $ = [$,x] ($ is always first if present)
if ( b == EMPTY ) { // x + $ = [x,$] ($ is always last if present)
int[] payloads = {a.returnState, EMPTY_RETURN_STATE};
PredictionContext[] parents = {a.parent, null};
PredictionContext joined =
......