Skip to content
Commits on Source (3)
Expand all Hide whitespace changes
Inline Side-by-side
.gitignore
View file @ 31fe85c1
......@@ -16,5 +16,3 @@ TAGS
*~
.#*
.lib
.ensime_cache
.idea
.travis.yml
View file @ 31fe85c1
language: scala
sudo: false
scala:
- 2.10.4
- 2.11.2
jdk:
- oraclejdk8
script:
- sbt "so test"
- openjdk7
README.md
View file @ 31fe85c1
......@@ -12,18 +12,13 @@ we get a catchy slogan.
Jawn was designed to parse JSON into an AST as quickly as possible.
[![Build Status](https://api.travis-ci.org/non/jawn.svg)](https://travis-ci.org/non/jawn)
[![Gitter](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/non/jawn?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
[![Latest version](https://index.scala-lang.org/non/jawn/jawn-parser/latest.svg?color=orange)](https://index.scala-lang.org/non/jawn/jawn-parser)
### Overview
Jawn consists of four parts:
Jawn consists of three parts:
1. A fast, generic JSON parser (`jawn-parser`)
2. A small, somewhat anemic AST (`jawn-ast`)
1. A fast, generic JSON parser
2. A small, somewhat anemic AST
3. Support packages which parse to third-party ASTs
4. A few helpful utilities (`jawn-util`)
Currently Jawn is competitive with the fastest Java JSON libraries
(GSON and Jackson) and in the author's benchmarks it often wins. It
......@@ -31,24 +26,22 @@ seems to be faster than any other Scala parser that exists (as of July
2014).
Given the plethora of really nice JSON libraries for Scala, the
expectation is that you're probably here for `jawn-parser` or a
support package.
expectation is that you are here for (1) and (3) not (2).
### Quick Start
Jawn supports Scala 2.10, 2.11, and 2.12.
Here's a `build.sbt` snippet that shows you how to depend on Jawn in
your own SBT project:
Jawn supports Scala 2.10 and 2.11. Here's a `build.sbt` snippet that
shows you how to depend on Jawn for your project:
```scala
resolvers += Resolver.sonatypeRepo("releases")
// required for all uses
resolvers += "bintray/non" at "http://dl.bintray.com/non/maven"
// use this if you just want jawn's parser, and will implement your own facade
libraryDependencies += "org.spire-math" %% "jawn-parser" % "0.11.0"
libraryDependencies += "org.jsawn" %% "jawn-parser" % "0.5.5"
// use this if you want jawn's parser and also jawn's ast
libraryDependencies += "org.spire-math" %% "jawn-ast" % "0.11.0"
// use this if you want to use jawn's parser and ast
libraryDependencies += "org.jsawn" %% "jawn-ast" % "0.5.5"
```
If you want to use Jawn's parser with another project's AST, see the
......@@ -56,26 +49,16 @@ If you want to use Jawn's parser with another project's AST, see the
you would say:
```scala
libraryDependencies += "org.spire-math" %% "jawn-spray" % "0.11.0"
resolvers += "bintray/non" at "http://dl.bintray.com/non/maven"
libraryDependencies += "org.jsawn" %% "support-spray" % "0.5.5"
```
There are a few reasons you might want to do this:
* The library's built-in parser is significantly slower than Jawn's.
* Jawn supports more input types (`ByteBuffer`, `File`, etc.).
* You need asynchronous JSON parsing.
(NOTE: previous to version 0.8.3 the support libraries would have been
named `"spray-support"` instead of `"jawn-spray"`.)
### Dependencies
*jawn-parser* has no dependencies other than Scala.
*jawn-ast* depends on *jawn-parser* but nothing else.
The various support projects (e.g. *jawn-argonaut*) depend on
the library they are supporting.
* The library's built-in parser is significantly slower than Jawn
* Jawn supports more input types (ByteBuffer, File, etc.)
* You need asynchronous JSON parsing
### Parsing
......@@ -97,52 +80,40 @@ the parser with data as it is available. There are three modes:
* `SingleValue` waits to return a single `J` value once parsing is done.
* `UnwrapArray` if the top-level element is an array, return values as they become available.
* `ValueStream` parse one-or-more json values separated by whitespace.
* `ValueStream` parser one-or-more json values separated by whitespace
Here's an example:
```scala
import jawn.ast
import jawn.AsyncParser
import jawn.ParseException
val p = ast.JParser.async(mode = AsyncParser.UnwrapArray)
def chunks: Stream[String] = ???
def sink(j: ast.JValue): Unit = ???
def chunks: Stream[String] = ...
def sink(j: ast.JValue): Unit = ...
def loop(st: Stream[String]): Either[ParseException, Unit] =
st match {
case s #:: tail =>
p.absorb(s) match {
case Right(js) =>
js.foreach(sink)
loop(tail)
case Left(e) =>
Left(e)
}
case _ =>
p.finish().right.map(_.foreach(sink))
case Stream.End => p.finish().fold(e => e, js => js.foreach(sink))
case s #:: tail => p.absorb(s).fold(e => e, { js => js.foreach(sink); loop(tail) })
}
loop(chunks)
```
You can also call `jawn.Parser.async[J]` to use async parsing with an
arbitrary data type (provided you also have an implicit `Facade[J]`).
arbitrary data type.
### Supporting external ASTs with Jawn
Jawn currently supports six external ASTs directly:
Jawn currently supports five external ASTs directly:
| AST | 2.10 | 2.11 | 2.12 |
|-----------|--------|--------|-------|
| Argonaut | 6.2 | 6.2 | 6.2 |
| Json4s | 3.5.2 | 3.5.2 | 3.5.2 |
| Play-json | 2.4.11 | 2.5.15 | 2.6.0 |
| Rojoma | 2.4.3 | 2.4.3 | 2.4.3 |
| Rojoma-v3 | 3.7.2 | 3.7.2 | 3.7.2 |
| Spray | 1.3.3 | 1.3.3 | 1.3.3 |
* Argonaut (6.0.4)
* Json4s (3.2.10)
* Play (2.3.0)
* Rojoma (2.4.3)
* Spray (1.2.6)
Each of these subprojects provides a `Parser` object (an instance of
`SupportParser[J]`) that is parameterized on the given project's
......@@ -156,18 +127,18 @@ Parser.parseFromFile(File) → Try[J]
Parser.parseFromChannel(ReadableByteChannel) → Try[J]
Parser.parseFromByteBuffer(ByteBuffer) → Try[J]
```
These methods parallel those provided by `jawn.Parser`.
For the following snippets, `XYZ` is one of (`argonaut`, `json4s`,
`play`, `rojoma`, `rojoma-v3` or `spray`):
`play`, `rojoma`, or `spray`):
This is how you would include the subproject in build.sbt:
```scala
resolvers += Resolver.sonatypeRepo("releases")
resolvers += "bintray/non" at "http://dl.bintray.com/non/maven"
libraryDependencies += "org.spire-math" %% jawn-"XYZ" % "0.11.0"
libraryDependencies += "org.jsawn" %% "XYZ-support" % "0.5.5"
```
This is an example of how you might use the parser into your code:
......@@ -189,14 +160,15 @@ To include Jawn's parser in your project, add the following
snippet to your `build.sbt` file:
```scala
resolvers += Resolver.sonatypeRepo("releases")
resolvers += "bintray/non" at "http://dl.bintray.com/non/maven"
libraryDependencies += "org.spire-math" %% "jawn-parser" % "0.11.0"
libraryDependencies += "jawn" %% "jawn-parser" % "0.5.5"
```
To support your AST of choice, you'll want to define a `Facade[J]`
instance, where the `J` type parameter represents the base of your JSON
AST. For example, here's a facade that supports Spray:
To support your AST of choice, you'll want to define a
`jawn.Facade[J]` instance, where the `J` type parameter represents the
base of your JSON AST. For example, here's a facade that supports
Spray:
```scala
import spray.json._
......@@ -221,192 +193,41 @@ You can also look at the facades used by the support projects to help
you create your own. This could also be useful if you wanted to
use an older version of a supported library.
### Using the AST
#### Access
For accessing atomic values, `JValue` supports two sets of
methods: *get-style* methods and *as-style* methods.
The *get-style* methods return `Some(_)` when called on a compatible
JSON value (e.g. strings can return `Some[String]`, numbers can return
`Some[Double]`, etc.), and `None` otherwise:
```scala
getBoolean → Option[Boolean]
getString → Option[String]
getLong → Option[Long]
getDouble → Option[Double]
getBigInt → Option[BigInt]
getBigDecimal → Option[BigDecimal]
```
In constrast, the *as-style* methods will either return an unwrapped
value (instead of returning `Some(_)`) or throw an exception (instead
of returning `None`):
```scala
asBoolean → Boolean // or exception
asString → String // or exception
asLong → Long // or exception
asDouble → Double // or exception
asBigInt → BigInt // or exception
asBigDecimal → BigDecimal // or exception
```
To access elements of an array, call `get` with an `Int` position:
```scala
get(i: Int) → JValue // returns JNull if index is illegal
```
To access elements of an object, call `get` with a `String` key:
```scala
get(k: String) → JValue // returns JNull if key is not found
```
Both of these methods also return `JNull` if the value is not the
appropraite container. This allows the caller to chain lookups without
having to check that each level is correct:
```scala
val v: JValue = ???
// returns JNull if a problem is encountered in structure of 'v'.
val t: JValue = v.get("novels").get(0).get("title")
// if 'v' had the right structure and 't' is JString(s), then Some(s).
// otherwise, None.
val titleOrNone: Option[String] = t.getString
// equivalent to titleOrNone.getOrElse(throw ...)
val titleOrDie: String = t.asString
```
#### Updating
The atomic values (`JNum`, `JBoolean`, `JNum`, and `JString`) are
immutable.
Objects are fully-mutable and can have items added, removed, or
changed:
```scala
set(k: String, v: JValue) → Unit
remove(k: String) → Option[JValue]
```
If `set` is called on a non-object, an exception will be thrown.
If `remove` is called on a non-object, `None` will be returned.
### Dependencies
Arrays are semi-mutable. Their values can be changed, but their size
is fixed:
*jawn-parser* has no dependencies other than Scala itself.
```scala
set(i: Int, v: JValue) → Unit
```
*jawn-ast* depends on [Spire](http://github.com/non/spire) in order to
provide type class instances.
If `set` is called on a non-array, or called with an illegal index, an
exception will be thrown.
(A future version of Jawn may provide an array whose length can be
changed.)
The various support projects (e.g. *argonaut-support*) depend on the
library they are supporting.
### Profiling
Jawn uses [JMH](http://openjdk.java.net/projects/code-tools/jmh/)
along with the [sbt-jmh](https://github.com/ktoso/sbt-jmh) plugin.
#### Running Benchmarks
The benchmarks are located in the `benchmark` project. You can run the
benchmarks by typing `benchmark/run` from SBT. There are many
supported arguments, so here are a few examples:
Run all benchmarks, with 10 warmups, 10 iterations, using 3 threads:
`benchmark/run -wi 10 -i 10 -f1 -t3`
Run just the `CountriesBench` test (5 warmups, 5 iterations, 1 thread):
`benchmark/run -wi 5 -i 5 -f1 -t1 .*CountriesBench`
#### Benchmark Issues
Currently, the benchmarks are a bit fiddily. The most obvious symptom
is that if you compile the benchmarks, make changes, and compile
again, you may see errors like:
```
[error] (benchmark/jmh:generateJavaSources) java.lang.NoClassDefFoundError: jawn/benchmark/Bla25Bench
```
The fix here is to run `benchmark/clean` and try again.
You will also see intermittent problems like:
Jawn provides benchmarks to help compare various JSON parsers on a
wide range of input files. You can run the benchmarks from SBT with:
```
[error] (benchmark/jmh:compile) java.lang.reflect.MalformedParameterizedTypeException
> benchmark/run
```
The solution here is easier (though frustrating): just try it
again. If you continue to have problems, consider cleaning the project
and trying again.
(In the future I hope to make the benchmarking here a bit more
resilient. Suggestions and pull requests gladly welcome!)
#### Files
The benchmarks use files located in `benchmark/src/main/resources`. If
you want to test your own files (e.g. `mydata.json`), you would:
* Copy the file to `benchmark/src/main/resources/mydata.json`.
* Add the following code to `JmhBenchmarks.scala`:
```scala
class MyDataBench extends JmhBenchmarks("mydata.json")
```
Jawn has been tested with much larger files, e.g. 100M - 1G, but these
are obviously too large to ship with the project.
With large files, it's usually easier to comment out most of the
benchmarking methods and only test one (or a few) methods. Some of the
slower JSON parsers get *much* slower for large files.
#### Interpreting the results
Remember that the benchmarking results you see will vary based on:
* Hardware
* Java version
* JSON file size
* JSON file structure
* JSON data values
I have tried to use each library in the most idiomatic and fastest way
possible (to parse the JSON into a simple AST). Pull requests to
update library versions and improve usage are very welcome.
### Future Work
More support libraries could be added.
It's likely that some of Jawn's I/O could be optimized a bit more, and
also made more configurable. The heuristics around all-at-once loading
versus input chunking could definitely be improved.
Any JSON files you put in `benchmark/src/main/resources` will be
included in the ad-hoc benchmark. There is a Python script I've used
to generate random JSON data called `randjson.py` which is a bit
quirky but does seem to work. I test on this random JSON as well as
data from projects I've worked on.
In cases where the user doesn't need fast lookups into JSON objects,
an even lighter AST could be used to improve parsing and rendering
speeds.
(I also test with larger data sets (100-600M) but for obvious reasons
I don't distribute this JSON in the project.)
Strategies to cache/intern field names of objects could pay big
dividends in some cases (this might require AST changes).
Of course, your mileage may vary, and these results do vary somewhat
based on file size, file structure, etc.
If you have ideas for any of these (or other ideas) please feel free
to open an issue or pull request so we can talk about it.
I have tried to understand the libraries well enough to write the most
optimal code for loading a file (given a path) and parsing it to a
simple JSON AST. Pull requests to update versions and improve usage
are welcome.
### Disclaimers
......@@ -415,13 +236,12 @@ future, but for now that's the target case. You can always decode your
data to a string, and handle the character set decoding using Java's
standard tools.
Jawn's AST is intended to be very lightweight and simple. It supports
simple access, and limited mutable updates. It intentionally lacks the
power and sophistication of many other JSON libraries.
Jawn's AST is intended to be a proof of concept of a very lightweight
AST. It lacks most of the fancy operators and DSLs of other libraries.
### Copyright and License
All code is available to you under the MIT license, available at
http://opensource.org/licenses/mit-license.php.
Copyright Erik Osheim, 2012-2017.
Copyright Erik Osheim, 2012-2014.
ast/build.sbt 0 → 100644
View file @ 31fe85c1
name := "jawn-ast"
libraryDependencies ++= Seq(
"org.spire-math" %% "spire" % "0.7.4"
)
ast/src/main/scala/jawn/Context.scala 0 → 100644
View file @ 31fe85c1
package jawn.ast
import scala.collection.mutable
private[jawn] sealed trait Context {
def add(s: String): Unit
def add(v: JValue): Unit
def finish: JValue
def isObj: Boolean
}
private[jawn] final class SingleContext extends Context {
var value: JValue = null
def add(s: String): Unit = value = JString(s)
def add(v: JValue): Unit = value = v
def finish = value
def isObj = false
}
private[jawn] final class ArrContext extends Context {
private val vs = mutable.ArrayBuffer.empty[JValue]
def add(s: String): Unit = vs.append(JString(s))
def add(v: JValue): Unit = vs.append(v)
def finish = new JArray(vs.toArray)
def isObj = false
}
private[jawn] final class ObjContext extends Context {
private var key: String = null
private val vs = mutable.Map.empty[String, JValue]
def add(s: String): Unit = if (key == null) {
key = s
} else {
vs(key) = JString(s)
key = null
}
def add(v: JValue): Unit = { vs(key) = v; key = null }
def finish = JObject(vs)
def isObj = true
}
ast/src/main/scala/jawn/ast/JParser.scala → ast/src/main/scala/jawn/JParser.scala
View file @ 31fe85c1
......@@ -15,9 +15,6 @@ object JParser {
def parseFromString(s: String): Try[JValue] =
Try(new StringParser[JValue](s).parse)
def parseFromCharSequence(cs: CharSequence): Try[JValue] =
Try(new CharSequenceParser[JValue](cs).parse)
def parseFromPath(path: String): Try[JValue] =
parseFromFile(new File(path))
......
ast/src/main/scala/jawn/JValue.scala 0 → 100644
View file @ 31fe85c1
package jawn
package ast
import scala.collection.mutable
import scala.annotation.switch
import scala.util.Sorting
import spire.algebra.{BooleanAlgebra, Field, IsReal, Monoid, NRoot, Order}
import spire.std.double._
import spire.syntax.field._
import spire.syntax.isReal._
import spire.syntax.nroot._
import spire.syntax.order._
sealed trait JValue {
def isNull: Boolean = this == JNull
def orElse(v: JValue): JValue = if (this == JNull) v else this
def render(r: Renderer): String = r.render(this)
override def toString: String = CanonicalRenderer.render(this)
}
sealed trait JAtom extends JValue
sealed trait JContainer extends JValue
case object JNull extends JAtom
sealed trait JBool extends JAtom {
def toBoolean: Boolean = this == JTrue
}
case object JTrue extends JBool
case object JFalse extends JBool
case class JString(s: String) extends JAtom {
def +(that: JString): JString = JString(this.s + that.s)
}
sealed trait JNum extends JAtom {
def toDouble: Double = this match {
case LongNum(n) => n.toDouble
case DoubleNum(n) => n
case DeferNum(s) => s.toDouble
}
}
case class LongNum(n: Long) extends JNum {
override def equals(that: Any): Boolean =
that match {
case LongNum(n2) => n == n2
case DoubleNum(n2) => n == n2
case DeferNum(s) => n.toString == s
}
}
case class DoubleNum(n: Double) extends JNum {
override def equals(that: Any): Boolean =
that match {
case LongNum(n2) => n == n2
case DoubleNum(n2) => n == n2
case DeferNum(s) => n.toString == s
}
}
case class DeferNum(s: String) extends JNum {
override def equals(that: Any): Boolean =
that match {
case LongNum(n2) => s == n2.toString
case DoubleNum(n2) => s == n2.toString
case DeferNum(s2) => s == s2
}
}
case class JArray(vs: Array[JValue]) extends JContainer {
def get(i: Int): JValue =
if (0 <= i && i < vs.length) vs(i) else JNull
override def equals(that: Any): Boolean =
that match {
case JArray(vs2) =>
if (vs.length != vs2.length) return false
var i = 0
while (i < vs.length) {
if (vs(i) != vs2(i)) return false
i += 1
}
true
case _ =>
false
}
}
case class JObject(vs: mutable.Map[String, JValue]) extends JContainer {
def get(k: String): JValue = vs.getOrElse(k, JNull)
}
object JValue {
implicit val monoid = new Monoid[JValue] {
val id: JValue = JNull
def op(x: JValue, y: JValue): JValue = {
if (x == JNull || y == JNull) return JNull
x match {
case j1: JNum => y match {
case j2: JNum => DoubleNum(j1.toDouble + j2.toDouble)
case _ => JNull
}
case j1: JString => y match {
case j2: JString => j1 + j2
case _ => JNull
}
case j1: JBool => y match {
case j2: JBool => JBool(j1.toBoolean && j2.toBoolean)
case _ => JNull
}
case JArray(js1) => y match {
case JArray(js2) => JArray(js1 ++ js2)
case _ => JNull
}
case JObject(js1) => y match {
case JObject(js2) => JObject(js1 ++ js2)
case _ => JNull
}
case JNull => JNull
}
}
}
}
object JBool {
val True: JBool = JTrue
val False: JBool = JFalse
def apply(b: Boolean): JBool = if (b) JTrue else JFalse
implicit val booleanAlgebra = new BooleanAlgebra[JBool] {
def zero: JBool = JFalse
def one: JBool = JTrue
def and(x: JBool, y: JBool): JBool = JBool(x.toBoolean && y.toBoolean)
def complement(x: JBool): JBool = JBool(!x.toBoolean)
def or(x: JBool, y: JBool): JBool = JBool(x.toBoolean || y.toBoolean)
}
implicit val monoid = new Monoid[JBool] {
def id: JBool = JTrue
def op(x: JBool, y: JBool): JBool = JBool(x.toBoolean && y.toBoolean)
}
}
object JNum { self =>
def apply(n: Long): JNum = LongNum(n)
def apply(n: Double): JNum = DoubleNum(n)
def apply(s: String): JNum = DeferNum(s)
val zero: JNum = LongNum(0)
val one: JNum = LongNum(1)
implicit val algebra = new Field[JNum] with IsReal[JNum] with NRoot[JNum] with Order[JNum] {
def zero: JNum = self.zero
def one: JNum = self.one
def abs(x: JNum): JNum = DoubleNum(x.toDouble)
def compare(x: JNum, y: JNum): Int = x.toDouble compare y.toDouble
def signum(x: JNum): Int = x.toDouble.signum
def negate(x: JNum): JNum = DoubleNum(-x.toDouble)
override def reciprocal(x: JNum): JNum = DoubleNum(1.0 / x.toDouble)
def plus(x: JNum, y: JNum): JNum = DoubleNum(x.toDouble + y.toDouble)
override def minus(x: JNum, y: JNum): JNum = DoubleNum(x.toDouble - y.toDouble)
def times(x: JNum, y: JNum): JNum = DoubleNum(x.toDouble * y.toDouble)
def div(x: JNum, y: JNum): JNum = DoubleNum(x.toDouble / y.toDouble)
def mod(x: JNum, y: JNum): JNum = DoubleNum(x.toDouble % y.toDouble)
def quot(x: JNum, y: JNum): JNum = DoubleNum(x.toDouble /~ y.toDouble)
def gcd(x: JNum, y: JNum): JNum = DoubleNum(x.toDouble gcd y.toDouble)
def ceil(x: JNum): JNum = DoubleNum(x.toDouble.ceil)
def floor(x: JNum): JNum = DoubleNum(x.toDouble.floor)
def round(x: JNum): JNum = DoubleNum(x.toDouble.round)
def isWhole(x: JNum): Boolean = x.toDouble.isWhole
def toDouble(x: JNum): Double = x.toDouble
def fpow(x: JNum, y: JNum): JNum = DoubleNum(x.toDouble fpow y.toDouble)
def nroot(x: JNum, k: Int): JNum = DoubleNum(x.toDouble nroot k)
}
}
object JString {
val empty = JString("")
implicit val monoid = new Monoid[JString] {
def id: JString = empty
def op(x: JString, y: JString): JString = x + y
}
}
object JArray { self =>
val empty = JArray(new Array[JValue](0))
implicit val monoid = new Monoid[JArray] {
def id: JArray = self.empty
def op(x: JArray, y: JArray): JArray = JArray(x.vs ++ y.vs)
}
def fromSeq(js: Seq[JValue]): JArray = JArray(js.toArray)
}
object JObject { self =>
def empty = JObject(mutable.Map.empty)
implicit val monoid = new Monoid[JObject] {
def id: JObject = self.empty
def op(x: JObject, y: JObject): JObject = JObject(x.vs ++ y.vs)
}
def fromSeq(js: Seq[(String, JValue)]): JObject = JObject(mutable.Map(js: _*))
}
ast/src/main/scala/jawn/JawnFacade.scala 0 → 100644
View file @ 31fe85c1
package jawn
package ast
import scala.collection.mutable
object JawnFacade extends MutableFacade[JValue] {
def jnull() = JNull
def jfalse() = JFalse
def jtrue() = JTrue
def jnum(s: String) = DeferNum(s)
def jint(s: String) = DeferNum(s)
def jstring(s: String) = JString(s)
def jarray(vs: mutable.ArrayBuffer[JValue]) = new JArray(vs.toArray)
def jobject(vs: mutable.Map[String, JValue]) = JObject(vs)
}
ast/src/main/scala/jawn/ast/Renderer.scala → ast/src/main/scala/jawn/Renderer.scala
View file @ 31fe85c1
......@@ -20,7 +20,6 @@ sealed trait Renderer {
case LongNum(n) => sb.append(n.toString)
case DoubleNum(n) => sb.append(n.toString)
case DeferNum(s) => sb.append(s)
case DeferLong(s) => sb.append(s)
case JString(s) => renderString(sb, s)
case JArray(vs) => renderArray(sb, depth, vs)
case JObject(vs) => renderObject(sb, depth, canonicalizeObject(vs))
......
ast/src/main/scala/jawn/Run.scala 0 → 100644
View file @ 31fe85c1
package jawn.ast
import java.io.File
import scala.util.{Success, Failure}
object Run {
def path = "qux3.json"
def usage() = println("usage: jawn | smart")
def main(args: Array[String]) {
args.foreach {
path =>
println("jawn: parsing %s" format path)
val f = new File(path)
val t0 = System.currentTimeMillis()
val j = JParser.parseFromFile(f) match {
case Success(j) => j
case Failure(e) => throw e
}
val t = System.currentTimeMillis - t0
println("jawn: finished in %d ms" format t)
}
}
}
ast/src/main/scala/jawn/ast/JValue.scala deleted 100644 → 0
View file @ e770b9e0
package jawn
package ast
import java.lang.Double.{isNaN, isInfinite}
import scala.collection.mutable
import scala.util.hashing.MurmurHash3
class WrongValueException(e: String, g: String) extends Exception(s"expected $e, got $g")
class InvalidNumException(s: String) extends Exception(s"invalid number: $s")
sealed abstract class JValue {
def valueType: String
def getBoolean: Option[Boolean] = None
def getString: Option[String] = None
def getInt: Option[Int] = None
def getLong: Option[Long] = None
def getDouble: Option[Double] = None
def getBigInt: Option[BigInt] = None
def getBigDecimal: Option[BigDecimal] = None
def asBoolean: Boolean = throw new WrongValueException("boolean", valueType)
def asString: String = throw new WrongValueException("string", valueType)
def asInt: Int = throw new WrongValueException("number", valueType)
def asLong: Long = throw new WrongValueException("number", valueType)
def asDouble: Double = throw new WrongValueException("number", valueType)
def asBigInt: BigInt = throw new WrongValueException("number", valueType)
def asBigDecimal: BigDecimal = throw new WrongValueException("number", valueType)
def get(i: Int): JValue = JNull
def set(i: Int, v: JValue): Unit = throw new WrongValueException("array", valueType)
def get(s: String): JValue = JNull
def set(s: String, v: JValue): Unit = throw new WrongValueException("object", valueType)
def remove(s: String): Option[JValue] = None
final def atomic: Option[JAtom] =
this match {
case v: JAtom => Some(v)
case _ => None
}
final def isNull: Boolean =
this == JNull
final def nonNull: Boolean =
this != JNull
final def render(): String =
CanonicalRenderer.render(this)
final def render(r: Renderer): String =
r.render(this)
override def toString: String =
CanonicalRenderer.render(this)
}
object JValue {
implicit val facade: Facade[JValue] = JawnFacade
}
sealed abstract class JAtom extends JValue {
def fold[A](f1: String => A, f2: Double => A, f3: Boolean => A, f4: => A): A =
this match {
case JString(s) => f1(s)
case v: JNum => f2(v.asDouble)
case JTrue => f3(true)
case JFalse => f3(false)
case JNull => f4
}
}
case object JNull extends JAtom {
final def valueType: String = "null"
}
sealed abstract class JBool extends JAtom {
final def valueType: String = "boolean"
final override def getBoolean: Option[Boolean] = Some(this == JTrue)
final override def asBoolean: Boolean = this == JTrue
}
object JBool {
final val True: JBool = JTrue
final val False: JBool = JFalse
final def apply(b: Boolean): JBool = if (b) JTrue else JFalse
}
case object JTrue extends JBool
case object JFalse extends JBool
case class JString(s: String) extends JAtom {
final def valueType: String = "string"
final override def getString: Option[String] = Some(s)
final override def asString: String = s
}
object JString {
final val empty = JString("")
}
sealed abstract class JNum extends JAtom {
final def valueType: String = "number"
}
object JNum { self =>
/**
* Create a JNum from a Long.
*
* This is identical to calling the LongNum(_) constructor.
*/
final def apply(n: Long): JNum =
LongNum(n)
/**
* Create a JNum from a Double.
*
* This factory constructor performs some error-checking (ensures
* that the given value is a finite Double). If you have already
* done this error-checking, you can use the DoubleNum(_) or
* DeferNum(_) constructors directly.
*/
final def apply(n: Double): JNum =
if (isNaN(n) || isInfinite(n)) throw new InvalidNumException(n.toString)
else DoubleNum(n)
/**
* Create a JNum from a String.
*
* This factory constructor validates the string (essentially,
* parsing it as a JSON value). If you are already sure this string
* is a valid JSON number, you can use the DeferLong(_) or
* DeferNum(_) constructors directly.
*/
final def apply(s: String): JNum =
JParser.parseUnsafe(s) match {
case jnum: JNum => jnum
case _ => throw new InvalidNumException(s)
}
final def hybridEq(x: Long, y: Double): Boolean = {
val z = x.toDouble
y == z && z.toLong == x
}
final val zero: JNum = LongNum(0)
final val one: JNum = LongNum(1)
}
case class LongNum(n: Long) extends JNum {
final override def getInt: Option[Int] = Some(n.toInt)
final override def getLong: Option[Long] = Some(n)
final override def getDouble: Option[Double] = Some(n.toDouble)
final override def getBigInt: Option[BigInt] = Some(BigInt(n))
final override def getBigDecimal: Option[BigDecimal] = Some(BigDecimal(n))
final override def asInt: Int = n.toInt
final override def asLong: Long = n
final override def asDouble: Double = n.toDouble
final override def asBigInt: BigInt = BigInt(n)
final override def asBigDecimal: BigDecimal = BigDecimal(n)
final override def hashCode: Int = n.##
final override def equals(that: Any): Boolean =
that match {
case LongNum(n2) => n == n2
case DoubleNum(n2) => JNum.hybridEq(n, n2)
case jn: JNum => jn == this
case _ => false
}
}
case class DoubleNum(n: Double) extends JNum {
final override def getInt: Option[Int] = Some(n.toInt)
final override def getLong: Option[Long] = Some(n.toLong)
final override def getDouble: Option[Double] = Some(n)
final override def getBigInt: Option[BigInt] = Some(BigDecimal(n).toBigInt)
final override def getBigDecimal: Option[BigDecimal] = Some(BigDecimal(n))
final override def asInt: Int = n.toInt
final override def asLong: Long = n.toLong
final override def asDouble: Double = n
final override def asBigInt: BigInt = BigDecimal(n).toBigInt
final override def asBigDecimal: BigDecimal = BigDecimal(n)
final override def hashCode: Int = n.##
final override def equals(that: Any): Boolean =
that match {
case LongNum(n2) => JNum.hybridEq(n2, n)
case DoubleNum(n2) => n == n2
case jn: JNum => jn == this
case _ => false
}
}
case class DeferLong(s: String) extends JNum {
lazy val n: Long = util.parseLongUnsafe(s)
final override def getInt: Option[Int] = Some(n.toInt)
final override def getLong: Option[Long] = Some(n)
final override def getDouble: Option[Double] = Some(n.toDouble)
final override def getBigInt: Option[BigInt] = Some(BigInt(s))
final override def getBigDecimal: Option[BigDecimal] = Some(BigDecimal(s))
final override def asInt: Int = n.toInt
final override def asLong: Long = n
final override def asDouble: Double = n.toDouble
final override def asBigInt: BigInt = BigInt(s)
final override def asBigDecimal: BigDecimal = BigDecimal(s)
final override def hashCode: Int = n.##
final override def equals(that: Any): Boolean =
that match {
case LongNum(n2) => n == n2
case DoubleNum(n2) => JNum.hybridEq(n, n2)
case jn: DeferLong => n == jn.asLong
case jn: DeferNum => JNum.hybridEq(n, jn.asDouble)
case _ => false
}
}
case class DeferNum(s: String) extends JNum {
lazy val n: Double = java.lang.Double.parseDouble(s)
final override def getInt: Option[Int] = Some(n.toInt)
final override def getLong: Option[Long] = Some(util.parseLongUnsafe(s))
final override def getDouble: Option[Double] = Some(n)
final override def getBigInt: Option[BigInt] = Some(BigDecimal(s).toBigInt)
final override def getBigDecimal: Option[BigDecimal] = Some(BigDecimal(s))
final override def asInt: Int = n.toInt
final override def asLong: Long = util.parseLongUnsafe(s)
final override def asDouble: Double = n
final override def asBigInt: BigInt = BigDecimal(s).toBigInt
final override def asBigDecimal: BigDecimal = BigDecimal(s)
final override def hashCode: Int = n.##
final override def equals(that: Any): Boolean =
that match {
case LongNum(n2) => JNum.hybridEq(n2, n)
case DoubleNum(n2) => n == n2
case jn: DeferLong => JNum.hybridEq(jn.asLong, n)
case jn: DeferNum => n == jn.asDouble
case _ => false
}
}
case class JArray(vs: Array[JValue]) extends JValue {
final def valueType: String = "array"
final override def get(i: Int): JValue =
if (0 <= i && i < vs.length) vs(i) else JNull
final override def set(i: Int, v: JValue): Unit =
vs(i) = v
final override def hashCode: Int = MurmurHash3.arrayHash(vs)
final override def equals(that: Any): Boolean =
that match {
case JArray(vs2) =>
if (vs.length != vs2.length) return false
var i = 0
while (i < vs.length) {
if (vs(i) != vs2(i)) return false
i += 1
}
true
case _ =>
false
}
}
object JArray { self =>
final def empty: JArray =
JArray(new Array[JValue](0))
final def fromSeq(js: Seq[JValue]): JArray =
JArray(js.toArray)
}
case class JObject(vs: mutable.Map[String, JValue]) extends JValue {
final def valueType: String = "object"
final override def get(k: String): JValue =
vs.getOrElse(k, JNull)
final override def set(k: String, v: JValue): Unit =
vs.put(k, v)
final override def remove(k: String): Option[JValue] =
vs.remove(k)
}
object JObject { self =>
final def empty: JObject =
JObject(mutable.Map.empty)
final def fromSeq(js: Seq[(String, JValue)]): JObject =
JObject(mutable.Map(js: _*))
}
ast/src/main/scala/jawn/ast/JawnFacade.scala deleted 100644 → 0
View file @ e770b9e0
package jawn
package ast
import scala.collection.mutable
object JawnFacade extends Facade[JValue] {
final val jnull = JNull
final val jfalse = JFalse
final val jtrue = JTrue
final def jnum(s: CharSequence, decIndex: Int, expIndex: Int): JValue =
if (decIndex == -1 && expIndex == -1) {
DeferLong(s.toString)
} else {
DeferNum(s.toString)
}
final def jstring(s: CharSequence): JValue =
JString(s.toString)
final def singleContext(): FContext[JValue] =
new FContext[JValue] {
var value: JValue = _
def add(s: CharSequence) { value = JString(s.toString) }
def add(v: JValue) { value = v }
def finish: JValue = value
def isObj: Boolean = false
}
final def arrayContext(): FContext[JValue] =
new FContext[JValue] {
val vs = mutable.ArrayBuffer.empty[JValue]
def add(s: CharSequence) { vs.append(JString(s.toString)) }
def add(v: JValue) { vs.append(v) }
def finish: JValue = JArray(vs.toArray)
def isObj: Boolean = false
}
final def objectContext(): FContext[JValue] =
new FContext[JValue] {
var key: String = null
val vs = mutable.Map.empty[String, JValue]
def add(s: CharSequence): Unit =
if (key == null) { key = s.toString } else { vs(key.toString) = JString(s.toString); key = null }
def add(v: JValue): Unit =
{ vs(key) = v; key = null }
def finish = JObject(vs)
def isObj = true
}
}
ast/src/test/scala/jawn/ArbitraryUtil.scala deleted 100644 → 0
View file @ e770b9e0
package jawn
package ast
import org.scalacheck._
import Gen._
import Arbitrary.arbitrary
object ArbitraryUtil {
// JSON doesn't allow NaN, PositiveInfinity, or NegativeInfinity
def isFinite(n: Double): Boolean =
!java.lang.Double.isNaN(n) && !java.lang.Double.isInfinite(n)
val jnull = Gen.const(JNull)
val jboolean = Gen.oneOf(JTrue :: JFalse :: Nil)
val jlong = arbitrary[Long].map(LongNum(_))
val jdouble = arbitrary[Double].filter(isFinite).map(DoubleNum(_))
val jstring = arbitrary[String].map(JString(_))
// Totally unscientific atom frequencies.
val jatom: Gen[JAtom] =
Gen.frequency(
(1, jnull),
(8, jboolean),
(8, jlong),
(8, jdouble),
(16, jstring))
// Use lvl to limit the depth of our jvalues.
// Otherwise we will end up with SOE real fast.
val MaxLevel: Int = 3
def jarray(lvl: Int): Gen[JArray] =
Gen.containerOf[Array, JValue](jvalue(lvl + 1)).map(JArray(_))
def jitem(lvl: Int): Gen[(String, JValue)] =
for { s <- arbitrary[String]; j <- jvalue(lvl) } yield (s, j)
def jobject(lvl: Int): Gen[JObject] =
Gen.containerOf[Vector, (String, JValue)](jitem(lvl + 1)).map(JObject.fromSeq)
def jvalue(lvl: Int = 0): Gen[JValue] =
if (lvl >= MaxLevel) jatom
else Gen.frequency((16, jatom), (1, jarray(lvl)), (2, jobject(lvl)))
implicit lazy val arbitraryJValue: Arbitrary[JValue] =
Arbitrary(jvalue())
}
ast/src/test/scala/jawn/AstTest.scala deleted 100644 → 0
View file @ e770b9e0
package jawn
package ast
import org.scalatest._
import org.scalatest.prop._
import scala.collection.mutable
import scala.util.{Try, Success}
import ArbitraryUtil._
class AstTest extends PropSpec with Matchers with PropertyChecks {
property("calling .get never crashes") {
forAll { (v: JValue, s: String, i: Int) =>
Try(v.get(i).get(s)).isSuccess shouldBe true
Try(v.get(s).get(i)).isSuccess shouldBe true
Try(v.get(i).get(i)).isSuccess shouldBe true
Try(v.get(s).get(s)).isSuccess shouldBe true
}
}
property(".getX and .asX agree") {
forAll { (v: JValue) =>
v.getBoolean shouldBe Try(v.asBoolean).toOption
v.getString shouldBe Try(v.asString).toOption
v.getInt shouldBe Try(v.asInt).toOption
v.getLong shouldBe Try(v.asLong).toOption
v.getDouble shouldBe Try(v.asDouble).toOption
v.getBigInt shouldBe Try(v.asBigInt).toOption
v.getBigDecimal shouldBe Try(v.asBigDecimal).toOption
}
}
property(".getBoolean") {
forAll((b: Boolean) => JBool(b).getBoolean shouldBe Some(b))
}
property(".getString") {
forAll((s: String) => JString(s).getString shouldBe Some(s))
}
property(".getInt") {
forAll { (n: Int) =>
JNum(n).getInt shouldBe Some(n)
JParser.parseUnsafe(n.toString).getInt shouldBe Some(n)
}
}
property(".getLong") {
forAll { (n: Long) =>
JNum(n).getLong shouldBe Some(n)
JParser.parseUnsafe(n.toString).getLong shouldBe Some(n)
}
}
property(".getDouble") {
forAll { (n: Double) =>
JNum(n).getDouble shouldBe Some(n)
JParser.parseUnsafe(n.toString).getDouble shouldBe Some(n)
}
}
property(".getBigInt") {
forAll { (n: BigInt) =>
JNum(n.toString).getBigInt shouldBe Some(n)
JParser.parseUnsafe(n.toString).getBigInt shouldBe Some(n)
}
}
property(".getBigDecimal") {
forAll { (n: BigDecimal) =>
if (Try(BigDecimal(n.toString)) == Success(n)) {
JNum(n.toString).getBigDecimal shouldBe Some(n)
JParser.parseUnsafe(n.toString).getBigDecimal shouldBe Some(n)
}
}
}
}
ast/src/test/scala/jawn/ParseCheck.scala
View file @ 31fe85c1
package jawn
package ast
import org.scalatest.matchers.ShouldMatchers
import org.scalatest._
import org.scalatest.prop._
import prop._
import org.scalacheck.Arbitrary._
import org.scalacheck._
import Gen._
......@@ -11,11 +12,61 @@ import Arbitrary.arbitrary
import scala.collection.mutable
import scala.util.{Try, Success}
import jawn.parser.TestUtil
class ParseCheck extends PropSpec with Matchers with GeneratorDrivenPropertyChecks {
// in theory we could test larger number values than longs, but meh?
// we need to exclude nan, +inf, and -inf from our doubles
// we want to be sure we test every possible unicode character
val jnull = Gen.oneOf(JNull :: Nil)
val jfalse = Gen.oneOf(JFalse :: Nil)
val jtrue = Gen.oneOf(JTrue :: Nil)
val jlong = arbitrary[Long].map(LongNum(_))
val jdouble = Gen.choose(Double.MinValue, Double.MaxValue).map(DoubleNum(_))
val jstring = arbitrary[String].map(JString(_))
// totally unscientific atom frequencies
val jatom: Gen[JAtom] =
Gen.frequency((1, 'n), (5, 'f), (5, 't), (8, 'l), (8, 'd), (16, 's)).flatMap {
case 'n => jnull
case 'f => jfalse
case 't => jtrue
case 'l => jlong
case 'd => jdouble
case 's => jstring
}
// use lvl to limit the depth of our jvalues
// otherwise we will end up with SOE real fast
def jarray(lvl: Int): Gen[JArray] =
Gen.containerOf[Array, JValue](jvalue(lvl + 1)).map(JArray(_))
def jitem(lvl: Int): Gen[(String, JValue)] =
for { s <- arbitrary[String]; j <- jvalue(lvl) } yield (s, j)
def jobject(lvl: Int): Gen[JObject] =
Gen.containerOf[List, (String, JValue)](jitem(lvl + 1)).map(JObject.fromSeq)
def jvalue(lvl: Int): Gen[JValue] =
if (lvl < 3) {
Gen.frequency((16, 'ato), (1, 'arr), (2, 'obj)).flatMap {
case 'ato => jatom
case 'arr => jarray(lvl)
case 'obj => jobject(lvl)
}
} else {
jatom
}
import ArbitraryUtil._
implicit lazy val arbJAtom: Arbitrary[JAtom] =
Arbitrary(jatom)
class AstCheck extends PropSpec with Matchers with PropertyChecks {
// implicit lazy val arbJArray: Arbitrary[JArray] =
// Arbitrary(jarray(3))
implicit lazy val arbJValue: Arbitrary[JValue] =
Arbitrary(jvalue(0))
// so it's only one property, but it exercises:
//
......@@ -30,14 +81,6 @@ class AstCheck extends PropSpec with Matchers with PropertyChecks {
val value2 = JParser.parseFromString(json1).get
val json2 = CanonicalRenderer.render(value2)
json2 shouldBe json1
json2.## shouldBe json1.##
value1 shouldBe value2
value1.## shouldBe value2.##
TestUtil.withTemp(json1) { t =>
JParser.parseFromFile(t).get shouldBe value2
}
}
}
......@@ -49,18 +92,6 @@ class AstCheck extends PropSpec with Matchers with PropertyChecks {
val json2 = CanonicalRenderer.render(jstr2)
jstr2 shouldBe jstr1
json2 shouldBe json1
json2.## shouldBe json1.##
}
}
property("string/charSequence parsing") {
forAll { value: JValue =>
val s = CanonicalRenderer.render(value)
val j1 = JParser.parseFromString(s)
val cs = java.nio.CharBuffer.wrap(s.toCharArray)
val j2 = JParser.parseFromCharSequence(cs)
j1 shouldBe j2
j1.## shouldBe j2.##
}
}
......@@ -90,25 +121,16 @@ class AstCheck extends PropSpec with Matchers with PropertyChecks {
import AsyncParser.{UnwrapArray, ValueStream, SingleValue}
property("async multi") {
val data = "[1,2,3][4,5,6]"
val p = AsyncParser[JValue](ValueStream)
val res0 = p.absorb(data)
val res1 = p.finish
//println((res0, res1))
true
}
property("async parsing") {
forAll { (v: JValue) =>
val json = CanonicalRenderer.render(v)
val segments = splitIntoSegments(json)
val parsed = parseSegments(AsyncParser[JValue](SingleValue), segments)
parsed shouldBe List(v)
parseSegments(AsyncParser[JValue](SingleValue), segments) shouldBe List(v)
}
}
property("async unwrapping") {
//forAll { (vs: List[JAtom]) =>
forAll { (vs0: List[Int]) =>
val vs = vs0.map(LongNum(_))
val arr = JArray(vs.toArray)
......@@ -117,53 +139,4 @@ class AstCheck extends PropSpec with Matchers with PropertyChecks {
parseSegments(AsyncParser[JValue](UnwrapArray), segments) shouldBe vs
}
}
property("unicode string round-trip") {
forAll { (s: String) =>
JParser.parseFromString(JString(s).render(FastRenderer)) shouldBe Success(JString(s))
}
}
property("if x == y, then x.## == y.##") {
forAll { (x: JValue, y: JValue) =>
if (x == y) x.## shouldBe y.##
}
}
property("ignore trailing zeros") {
forAll { (n: Int) =>
val s = n.toString
val n1 = LongNum(n)
val n2 = DoubleNum(n)
def check(j: JValue) {
j shouldBe n1; n1 shouldBe j
j shouldBe n2; n2 shouldBe j
}
check(DeferNum(s))
check(DeferNum(s + ".0"))
check(DeferNum(s + ".00"))
check(DeferNum(s + ".000"))
check(DeferNum(s + "e0"))
check(DeferNum(s + ".0e0"))
}
}
property("large strings") {
val M = 1000000
val q = "\""
val s0 = ("x" * (40 * M))
val e0 = q + s0 + q
TestUtil.withTemp(e0) { t =>
JParser.parseFromFile(t).filter(_ == JString(s0)).isSuccess shouldBe true
}
val s1 = "\\" * (20 * M)
val e1 = q + s1 + s1 + q
TestUtil.withTemp(e1) { t =>
JParser.parseFromFile(t).filter(_ == JString(s1)).isSuccess shouldBe true
}
}
}
benchmark/build.sbt
View file @ 31fe85c1
......@@ -3,18 +3,24 @@ name := "jawn-benchmarks"
javaOptions in run += "-Xmx6G"
libraryDependencies ++= Seq(
"io.argonaut" %% "argonaut" % "6.2",
"org.json4s" %% "json4s-native" % "3.5.2",
"org.json4s" %% "json4s-jackson" % "3.5.2",
"com.typesafe.play" %% "play-json" % "2.5.15",
// support deps
"io.argonaut" %% "argonaut" % "6.0.4",
"org.json4s" %% "json4s-native" % "3.2.10",
"com.typesafe.play" %% "play-json" % "2.3.0",
"com.rojoma" %% "rojoma-json" % "2.4.3",
"com.rojoma" %% "rojoma-json-v3" % "3.7.2",
"io.spray" %% "spray-json" % "1.3.3",
"org.parboiled" %% "parboiled" % "2.1.4",
"com.fasterxml.jackson.core" % "jackson-annotations" % "2.8.4",
"com.fasterxml.jackson.core" % "jackson-core" % "2.8.4",
"com.fasterxml.jackson.core" % "jackson-databind" % "2.8.4",
"com.google.code.gson" % "gson" % "2.8.1"
"io.spray" %% "spray-json" % "1.2.6",
// other deps
"org.json4s" %% "json4s-jackson" % "3.2.10",
"org.parboiled" %% "parboiled" % "2.0.0",
"org.scalastuff" %% "json-parser" % "1.1.1",
"net.minidev" % "json-smart" % "1.1.1",
"com.fasterxml.jackson.core" % "jackson-annotations" % "2.0.6",
"com.fasterxml.jackson.core" % "jackson-core" % "2.0.6",
"com.fasterxml.jackson.core" % "jackson-databind" % "2.0.6",
"com.google.guava" % "guava" % "r09",
"com.google.code.java-allocation-instrumenter" % "java-allocation-instrumenter" % "2.0",
"com.google.code.caliper" % "caliper" % "1.0-SNAPSHOT" from "http://plastic-idolatry.com/jars/caliper-1.0-SNAPSHOT.jar",
"com.google.code.gson" % "gson" % "2.2.4"
)
// enable forking in run
......
benchmark/src/main/resources/countries.geo.json deleted 100644 → 0
View file @ e770b9e0
source diff could not be displayed: it is too large. Options to address this: view the blob.
benchmark/src/main/resources/ugh10k.json deleted 100644 → 0
View file @ e770b9e0
This diff is collapsed.
benchmark/src/main/scala/jawn/AdHocBenchmarks.scala 0 → 100644
View file @ 31fe85c1
package jawn
package benchmark
import scala.collection.mutable
object AdHocBenchmarks {
def warmups = 2
def runs = 5
def json4sNativeParse(path: String) = {
import org.json4s._
import org.json4s.native.JsonMethods._
val file = new java.io.File(path)
val bytes = new Array[Byte](file.length.toInt)
val fis = new java.io.FileInputStream(file)
fis.read(bytes)
val s = new String(bytes, "UTF-8")
parse(s)
}
def json4sJacksonParse(path: String) = {
import org.json4s._
import org.json4s.jackson.JsonMethods._
val file = new java.io.File(path)
val bytes = new Array[Byte](file.length.toInt)
val fis = new java.io.FileInputStream(file)
fis.read(bytes)
val s = new String(bytes, "UTF-8")
parse(s)
}
def json4sJawnParse(path: String) = {
val file = new java.io.File(path)
jawn.support.json4s.Parser.parseFromFile(file).get
}
def playParse(path: String) = {
val file = new java.io.File(path)
val bytes = new Array[Byte](file.length.toInt)
val fis = new java.io.FileInputStream(file)
fis.read(bytes)
val s = new String(bytes, "UTF-8")
play.api.libs.json.Json.parse(s)
}
def playJawnParse(path: String) = {
val file = new java.io.File(path)
jawn.support.play.Parser.parseFromFile(file).get
}
def sprayParse(path: String) = {
val file = new java.io.File(path)
val bytes = new Array[Byte](file.length.toInt)
val fis = new java.io.FileInputStream(file)
fis.read(bytes)
val s = new String(bytes, "UTF-8")
spray.json.JsonParser(s)
}
def rojomaParse(path: String) = {
val file = new java.io.File(path)
val r = new java.io.FileReader(file)
val br = new java.io.BufferedReader(r)
com.rojoma.json.io.JsonReader(br).read()
}
def rojomaFastParse(path: String) = {
val file = new java.io.File(path)
val r = new java.io.FileReader(file)
val events = new com.rojoma.json.io.FusedBlockJsonEventIterator(r, blockSize = 100000)
com.rojoma.json.io.JsonReader.fromEvents(events)
}
def rojomaV3Parse(path: String) = {
val file = new java.io.File(path)
val r = new java.io.FileReader(file)
com.rojoma.json.v3.io.JsonReader.fromReader(r, blockSize = 100000)
}
def argonautParse(path: String) = {
val file = new java.io.File(path)
val bytes = new Array[Byte](file.length.toInt)
val fis = new java.io.FileInputStream(file)
fis.read(bytes)
val s = new String(bytes, "UTF-8")
argonaut.Parse.parse(s)
}
def sprayScalastuffParse(path: String) = {
val file = new java.io.File(path)
val bytes = new Array[Byte](file.length.toInt)
val fis = new java.io.FileInputStream(file)
fis.read(bytes)
val s = new String(bytes, "UTF-8")
org.scalastuff.json.spray.SprayJsonParser.parse(s)
}
def smartJsonParse(path: String) = {
val file = new java.io.File(path)
val reader = new java.io.FileReader(file)
net.minidev.json.JSONValue.parse(reader)
}
def parboiledJsonParse(path: String) = {
val file = new java.io.File(path)
val bytes = new Array[Byte](file.length.toInt)
val fis = new java.io.FileInputStream(file)
fis.read(bytes)
val s = new String(bytes, "UTF-8")
new ParboiledParser(s).Json.run().get
}
def jacksonParse(path: String) = {
import com.fasterxml.jackson.databind.ObjectMapper
import com.fasterxml.jackson.databind.JsonNode
val file = new java.io.File(path)
new ObjectMapper().readValue(file, classOf[JsonNode])
}
// def liftJsonParse(path: String) = {
// val file = new java.io.File(path)
// val reader = new java.io.FileReader(file)
// net.liftweb.json.JsonParser.parse(reader)
// }
def jawnParse(path: String) = {
val file = new java.io.File(path)
jawn.ast.JParser.parseFromFile(file).get
}
def jawnStringParse(path: String) = {
val file = new java.io.File(path)
val bytes = new Array[Byte](file.length.toInt)
val fis = new java.io.FileInputStream(file)
fis.read(bytes)
val s = new String(bytes, "UTF-8")
jawn.ast.JParser.parseFromString(s).get
}
def argonautJawnParse(path: String) = {
val file = new java.io.File(path)
jawn.support.argonaut.Parser.parseFromFile(file).get
}
def sprayJawnParse(path: String) = {
val file = new java.io.File(path)
jawn.support.spray.Parser.parseFromFile(file).get
}
def rojomaJawnParse(path: String) = {
val file = new java.io.File(path)
jawn.support.rojoma.Parser.parseFromFile(file).get
}
def rojomaV3JawnParse(path: String) = {
val file = new java.io.File(path)
jawn.support.rojoma.v3.Parser.parseFromFile(file).get
}
def gsonParse(path: String) = {
val p = new com.google.gson.JsonParser()
val r = new java.io.BufferedReader(new java.io.FileReader(path))
p.parse(r)
}
def test[A](name: String, path: String)(f: String => A): Double = {
var h = 0
(0 until warmups).foreach { _ =>
val result = f(path)
h = h ^ result.##
System.gc()
}
var t = 0.0
(0 until runs).foreach { _ =>
val t0 = System.nanoTime()
val result = f(path)
t += (System.nanoTime() - t0).toDouble / 1000000
h = h ^ result.##
System.gc()
}
t / runs
}
def run[A](name: String, path: String)(f: String => A) {
try {
val t = test(name, path)(f)
println(" %-18s %10.2f ms" format (name, t))
} catch {
case e: Exception =>
println(" %-18s %10s" format (name, "FAIL"))
println(e)
}
}
def main(args: Array[String]) {
val d = new java.io.File("src/main/resources")
val xs = d.listFiles.filter(_.getName.endsWith(".json")).sorted
val fs = if (args.isEmpty) xs else xs.filter(f => args.contains(f.getName))
fs.foreach { f =>
val path = f.getPath
val bytes = f.length
val (size, units) = if (bytes >= 1048576)
(bytes / 1048576.0, "M")
else if (bytes >= 1024.0)
(bytes / 1024.0, "K")
else
(bytes / 1.0, "B")
println("%s (%.1f%s)" format (f.getName, size, units))
// run("lift-json", path)(liftJsonParse) // buggy, fails to parse, etc
// run("parboiled-json", path)(parboiledJsonParse)
// run("smart-json", path)(smartJsonParse)
// run("json4s-native", path)(json4sNativeParse)
// run("json4s-jackson", path)(json4sJacksonParse)
// run("json4s-jawn", path)(json4sJawnParse)
// run("play", path)(playParse)
// run("play-jawn", path)(playJawnParse)
run("rojoma", path)(rojomaParse)
run("rojoma-fast", path)(rojomaFastParse)
run("rojoma-jawn", path)(rojomaJawnParse)
run("rojoma-v3", path)(rojomaV3Parse)
run("rojoma-v3-jawn", path)(rojomaV3JawnParse)
// run("argonaut", path)(argonautParse)
// run("argonaut-jawn", path)(argonautJawnParse)
// run("spray", path)(sprayParse)
// run("spray-scalastuff", path)(sprayScalastuffParse)
// run("spray-jawn", path)(sprayJawnParse)
// run("jackson", path)(jacksonParse)
// run("gson", path)(gsonParse)
run("jawn", path)(jawnParse)
run("jawn-string", path)(jawnStringParse)
}
}
}
benchmark/src/main/scala/jawn/CaliperBenchmark.scala 0 → 100644
View file @ 31fe85c1
// package jawn
//
// import com.google.caliper.Param
//
// object ParserBenchmarks extends MyRunner(classOf[ParserBenchmarks])
//
// class ParserBenchmarks extends MyBenchmark {
// // any benchmark which takes more than 10s to complete will explode :(
// @Param(Array("qux1", "qux2", "qux3"))
// var name: String = null
// var path: String = null
//
// override protected def setUp() {
// path = "src/main/resources/%s.json" format name
// }
//
// def timeSmartJson(reps: Int) = run(reps) {
// val file = new java.io.File(path)
// val reader = new java.io.FileReader(file)
// net.minidev.json.JSONValue.parse(reader)
// }
//
// def timeJackson(reps: Int) = run(reps) {
// import com.fasterxml.jackson.databind.ObjectMapper
// import com.fasterxml.jackson.databind.JsonNode
// val file = new java.io.File(path)
// new ObjectMapper().readValue(file, classOf[JsonNode])
// }
//
// def timeJawn(reps:Int) = run(reps) {
// new PathParser(path).parse(0)
// }
// }