Skip to content
Snippets Groups Projects
Commit 59068125 authored by Emmanuel Bourg's avatar Emmanuel Bourg
Browse files

New upstream version 1.1.7

parent 37541a4e
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
Showing
with 185 additions and 134 deletions
.travis.yml
+ 3
− 6
View file @ 59068125
......@@ -2,9 +2,8 @@ language: java
sudo: false
jdk:
- oraclejdk8
- oraclejdk7
- openjdk7
- oraclejdk12
- openjdk8
install: true
......@@ -12,7 +11,5 @@ branches:
only:
- master
script: mvn clean test jacoco:report
script: mvn clean test
after_success:
- mvn coveralls:report
CHANGELOG
+ 1
− 1
View file @ 59068125
......@@ -254,7 +254,7 @@ version 0.4.4 (May 21, 2012)
version 0.4.3 (April 9, 2012)
- fast aggregation through logical AND of many bitmaps using a new method
- fixed a rarely occuring bug in the set method due to faulty bitmap size extension
- fixed a rarely occurring bug in the set method due to faulty bitmap size extension
version 0.4.2 (April 5, 2012)
......
LICENSE
+ 1
− 1
View file @ 59068125
......@@ -176,7 +176,7 @@ recommend that a file or class name and description of purpose be included on
the same "printed page" as the copyright notice for easier identification within
third-party archives.
Copyright [yyyy] [name of copyright owner]
(c) 2009-2016 Daniel Lemire (http://lemire.me/en/), Cliff Moon, David McIntosh (https://github.com/mctofu), Robert Becho (https://github.com/RBecho), Colby Ranger (https://github.com/crangeratgoogle), Veronika Zenz (https://github.com/veronikazenz), Owen Kaser (https://github.com/owenkaser), Gregory Ssi-Yan-Kai (https://github.com/gssiyankai), and Rory Graves (https://github.com/rorygraves)
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
......
README.md
+ 14
− 4
View file @ 59068125
......@@ -5,7 +5,7 @@ JavaEWAH
[![][license img]][license]
[![docs-badge][]][docs]
[![Coverage Status](https://coveralls.io/repos/lemire/javaewah/badge.svg?branch=master)](https://coveralls.io/r/lemire/javaewah?branch=master)
[![Code Quality: Cpp](https://img.shields.io/lgtm/grade/java/g/lemire/javaewah.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/lemire/javaewah/context:java)
(c) 2009-2016
Daniel Lemire (http://lemire.me/en/),
......@@ -48,7 +48,7 @@ wasteful serialization/deserialization routines.
The library also provides a drop-in replacement for
the standard BitSet class. Like the other bitmap classes
in JavaEWAH, this uncompressed BitSet class supports
memory-mapped files as well as many other conviences.
memory-mapped files as well as many other conveniences.
For better performance, use a 64-bit JVM over
64-bit CPUs when using the 64-bit scheme (javaewah.EWAHCompressedBitmap).
......@@ -63,7 +63,7 @@ offered the best performance.
Real-world usage
----------------
JavaEWAH is part of Apache Hive and its derivatives (e.g., Apache Spark) and Eclipse JGit. It has been used in production systems for many years. It is part of major Linux distributions.
JavaEWAH is part of Apache Hive and its derivatives (e.g., Apache Spark) and Eclipse JGit. It has been used in production systems for many years. It is part of major Linux distributions. It is part of [Twitter algebird](https://github.com/twitter/algebird).
EWAH is used to accelerate the distributed version control system Git (http://githubengineering.com/counting-objects/). You can find the C port of EWAH written by the Git team at https://github.com/git/git/tree/master/ewah
......@@ -96,11 +96,14 @@ When the bitset approach is applicable, it can be orders of
magnitude faster than other possible implementation of a set (e.g., as a hash set)
while using several times less memory.
However, a bitset, even a compressed one is not always applicable. For example, if the
you have 1000 random-looking integers, then a simple array might be the best representation.
We refer to this case as the "sparse" scenario.
When should you use compressed bitmaps?
----------------------------------------
An uncompress BitSet can use a lot of memory. For example, if you take a BitSet
An uncompressed BitSet can use a lot of memory. For example, if you take a BitSet
and set the bit at position 1,000,000 to true and you have just over 100kB. That's over 100kB
to store the position of one bit. This is wasteful even if you do not care about memory:
suppose that you need to compute the intersection between this BitSet and another one
......@@ -115,6 +118,10 @@ to you. In fact, if you do not need compression, then a BitSet offers remarkable
One of the downsides of a compressed bitmap like those provided by JavaEWAH is slower random access:
checking whether a bit is set to true in a compressed bitmap takes longer.
The sparse scenario is another use case where compressed bitmaps should not be used.
Keep in mind that random-looking data is usually not compressible. E.g., if you have a small set of
32-bit random integers, it is not mathematically possible to use far less than 32 bits per integer,
and attempts at compression can be counterproductive.
How does EWAH compares with the alternatives?
-------------------------------------------
......@@ -327,7 +334,10 @@ API Documentation
http://www.javadoc.io/doc/com.googlecode.javaewah/JavaEWAH/
Mailing list and discussion group
---------------------------------
https://groups.google.com/forum/#!forum/javaewah
Further reading
......
pom.xml
+ 7
− 43
View file @ 59068125
......@@ -2,11 +2,11 @@
<modelVersion>4.0.0</modelVersion>
<groupId>com.googlecode.javaewah</groupId>
<artifactId>JavaEWAH</artifactId>
<version>1.1.6</version>
<version>1.1.7</version>
<packaging>bundle</packaging>
<properties>
<maven.compiler.source>1.6</maven.compiler.source>
<maven.compiler.target>1.6</maven.compiler.target>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
<encoding>UTF-8</encoding>
</properties>
<licenses>
......@@ -67,48 +67,9 @@
<configuration>
<forkCount>3</forkCount>
<reuseForks>true</reuseForks>
<argLine>-Xmx1024m ${argLine}</argLine>
<argLine>-Xmx1024m</argLine>
</configuration>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>animal-sniffer-maven-plugin</artifactId>
<version>1.9</version>
<configuration>
<signature>
<groupId>org.codehaus.mojo.signature</groupId>
<artifactId>java16</artifactId>
<version>1.0</version>
</signature>
</configuration>
<executions>
<execution>
<id>ensure-java-1.6-class-library</id>
<phase>test</phase>
<goals>
<goal>check</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.2.201409121644</version>
<executions>
<execution>
<id>prepare-agent</id>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.eluder.coveralls</groupId>
<artifactId>coveralls-maven-plugin</artifactId>
<version>3.1.0</version>
</plugin>
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
......@@ -139,6 +100,9 @@
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
<version>2.8</version>
<configuration>
<source>8</source>
</configuration>
<executions>
<execution>
<id>attach-javadocs</id>
......
src/main/java/com/googlecode/javaewah/BitCounter.java
+ 1
− 1
View file @ 59068125
......@@ -57,7 +57,7 @@ public final class BitCounter implements BitmapStorage {
@Override
public void addStreamOfEmptyWords(boolean v, long number) {
if (v) {
this.oneBits += number * EWAHCompressedBitmap.WORD_IN_BITS;
this.oneBits += (int)(number * EWAHCompressedBitmap.WORD_IN_BITS);
}
}
......
src/main/java/com/googlecode/javaewah/BufferedIterator.java
+ 8
− 1
View file @ 59068125
......@@ -52,7 +52,13 @@ public class BufferedIterator implements IteratingRLW, Cloneable {
}
}
}
@Override
public void discardLiteralWords(long x) {
this.iteratingBrlw.discardLiteralWords(x);
if (this.iteratingBrlw.getNumberOfLiteralWords() == 0)
this.next();
}
@Override
public void discardRunningWords() {
this.iteratingBrlw.discardRunningWords();
......@@ -138,4 +144,5 @@ public class BufferedIterator implements IteratingRLW, Cloneable {
private IteratingBufferedRunningLengthWord iteratingBrlw;
private CloneableIterator<EWAHIterator> masterIterator;
}
src/main/java/com/googlecode/javaewah/BufferedRunningLengthWord.java
+ 2
− 2
View file @ 59068125
......@@ -46,8 +46,8 @@ public final class BufferedRunningLengthWord implements Cloneable {
}
x -= this.runningLength;
this.runningLength = 0;
this.literalWordOffset += x;
this.numberOfLiteralWords -= x;
this.literalWordOffset += (int) x;
this.numberOfLiteralWords -= (int) x;
}
/**
......
src/main/java/com/googlecode/javaewah/EWAHCompressedBitmap.java
+ 21
− 20
View file @ 59068125
......@@ -7,7 +7,6 @@ package com.googlecode.javaewah;
import com.googlecode.javaewah.symmetric.RunningBitmapMerge;
import com.googlecode.javaewah.symmetric.ThresholdFuncBitmap;
import java.io.*;
import java.nio.ByteBuffer;
import java.nio.IntBuffer;
......@@ -16,6 +15,7 @@ import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
/**
* <p>
* This implements the patent-free(1) EWAH scheme. Roughly speaking, it is a
......@@ -262,6 +262,7 @@ public final class EWAHCompressedBitmap implements Cloneable, Externalizable,
*
* @param newData the word
*/
@Override
public void addLiteralWord(final long newData) {
this.sizeInBits += WORD_IN_BITS;
insertLiteralWord(newData);
......@@ -327,7 +328,7 @@ public final class EWAHCompressedBitmap implements Cloneable, Externalizable,
public void addStreamOfEmptyWords(final boolean v, long number) {
if (number == 0)
return;
this.sizeInBits += number * WORD_IN_BITS;
this.sizeInBits += (int)(number * WORD_IN_BITS);
fastaddStreamOfEmptyWords(v, number);
}
......@@ -427,8 +428,8 @@ public final class EWAHCompressedBitmap implements Cloneable, Externalizable,
for (int k = 0; k < nbre_literal; ++k) {
container.addWord(rlwi.getLiteralWordAt(k) & rlwj.getLiteralWordAt(k));
}
rlwi.discardFirstWords(nbre_literal);
rlwj.discardFirstWords(nbre_literal);
rlwi.discardLiteralWords(nbre_literal);
rlwj.discardLiteralWords(nbre_literal);
}
}
......@@ -523,8 +524,8 @@ public final class EWAHCompressedBitmap implements Cloneable, Externalizable,
if (nbre_literal > 0) {
for (int k = 0; k < nbre_literal; ++k)
container.addWord(rlwi.getLiteralWordAt(k) & (~rlwj.getLiteralWordAt(k)));
rlwi.discardFirstWords(nbre_literal);
rlwj.discardFirstWords(nbre_literal);
rlwi.discardLiteralWords(nbre_literal);
rlwj.discardLiteralWords(nbre_literal);
}
}
final boolean i_remains = rlwi.size() > 0;
......@@ -565,7 +566,7 @@ public final class EWAHCompressedBitmap implements Cloneable, Externalizable,
while (i.hasNext()) {
RunningLengthWord localrlw = i.next();
if (localrlw.getRunningBit()) {
counter += WORD_IN_BITS * localrlw.getRunningLength();
counter += (int)(WORD_IN_BITS * localrlw.getRunningLength());
}
final int numberOfLiteralWords = localrlw.getNumberOfLiteralWords();
final int literalWords = i.literalWords();
......@@ -774,7 +775,7 @@ public final class EWAHCompressedBitmap implements Cloneable, Externalizable,
v.add(pos++);
}
} else {
pos += WORD_IN_BITS * localrlw.getRunningLength();
pos += (int)(WORD_IN_BITS * localrlw.getRunningLength());
}
final int nlw = localrlw.getNumberOfLiteralWords();
for (int j = 0; j < nlw; ++j) {
......@@ -807,8 +808,8 @@ public final class EWAHCompressedBitmap implements Cloneable, Externalizable,
i.next();
if (i.rlw.getRunningBit()) {
final long rl = i.rlw.getRunningLength();
karprabin += B * (rl & 0xFFFFFFFF);
karprabin += B * ((rl>>>32) & 0xFFFFFFFF);
karprabin += (int)(B * (rl & 0xFFFFFFFF));
karprabin += (int)(B * ((rl>>>32) & 0xFFFFFFFF));
}
final int nlw = i.rlw.getNumberOfLiteralWords();
final int lw = i.literalWords();
......@@ -1067,8 +1068,8 @@ public final class EWAHCompressedBitmap implements Cloneable, Externalizable,
container.addWord(rlwi.getLiteralWordAt(k)
| rlwj.getLiteralWordAt(k));
}
rlwi.discardFirstWords(nbre_literal);
rlwj.discardFirstWords(nbre_literal);
rlwi.discardLiteralWords(nbre_literal);
rlwj.discardLiteralWords(nbre_literal);
}
}
final boolean i_remains = rlwi.size() > 0;
......@@ -1141,7 +1142,7 @@ public final class EWAHCompressedBitmap implements Cloneable, Externalizable,
final IteratingRLW j = getIteratingRLW();
final int wordi = i / WORD_IN_BITS;
while (wordChecked <= wordi) {
wordChecked += j.getRunningLength();
wordChecked += (int) j.getRunningLength();
if (wordi < wordChecked) {
return j.getRunningBit();
}
......@@ -1171,7 +1172,7 @@ public final class EWAHCompressedBitmap implements Cloneable, Externalizable,
if((rl > 0) && rb) {
return nword * WORD_IN_BITS;
}
nword += rl;
nword += (int) rl;
long lw = RunningLengthWord.getNumberOfLiteralWords(this.buffer, pos);
if(lw > 0) {
long word = this.buffer.getWord(pos + 1);
......@@ -1299,14 +1300,14 @@ public final class EWAHCompressedBitmap implements Cloneable, Externalizable,
setInRunningLength(value, i, nbits, pos, rl, rb, lw);
return;
}
nbits += rbits;
nbits += (int) rbits;
long lbits = lw * WORD_IN_BITS;
if(i < nbits + lbits) {
setInLiteralWords(value, i, nbits, pos, rl, rb, lw);
return;
}
nbits += lbits;
pos += lw + 1;
nbits += (int) lbits;
pos += (int) (lw + 1);
}
}
......@@ -1738,8 +1739,8 @@ public final class EWAHCompressedBitmap implements Cloneable, Externalizable,
if (nbre_literal > 0) {
for (int k = 0; k < nbre_literal; ++k)
container.addWord(rlwi.getLiteralWordAt(k) ^ rlwj.getLiteralWordAt(k));
rlwi.discardFirstWords(nbre_literal);
rlwj.discardFirstWords(nbre_literal);
rlwi.discardLiteralWords(nbre_literal);
rlwj.discardLiteralWords(nbre_literal);
}
}
final boolean i_remains = rlwi.size() > 0;
......@@ -2050,7 +2051,7 @@ public final class EWAHCompressedBitmap implements Cloneable, Externalizable,
/**
* Generate a new bitmap a new bitmap shifted by "b" bits.
* Generates a new bitmap shifted by "b" bits.
* If b is positive, the position of all set bits is increased by
* b. The negative case is not supported.
*
......
src/main/java/com/googlecode/javaewah/IteratingBufferedRunningLengthWord.java
+ 38
− 21
View file @ 59068125
......@@ -56,7 +56,7 @@ public final class IteratingBufferedRunningLengthWord implements IteratingRLW,
long toDiscard = x > this.brlw.numberOfLiteralWords ? this.brlw.numberOfLiteralWords
: x;
this.literalWordStartPosition += toDiscard;
this.literalWordStartPosition += (int) toDiscard;
this.brlw.numberOfLiteralWords -= toDiscard;
x -= toDiscard;
if ((x > 0) || (this.brlw.size() == 0)) {
......@@ -69,7 +69,18 @@ public final class IteratingBufferedRunningLengthWord implements IteratingRLW,
}
}
}
@Override
public void discardLiteralWords(long x) {
this.literalWordStartPosition += x;
this.brlw.numberOfLiteralWords -= x;
if (this.brlw.numberOfLiteralWords == 0) {
if (!this.iterator.hasNext()) {
return;
}
this.brlw.reset(this.iterator.next());
this.literalWordStartPosition = this.iterator.literalWords();
}
}
@Override
public void discardRunningWords() {
this.brlw.runningLength = 0;
......@@ -104,26 +115,31 @@ public final class IteratingBufferedRunningLengthWord implements IteratingRLW,
* @return how many written
*/
public long discharge(BitmapStorage container, long max) {
long index = 0;
while ((index < max) && (size() > 0)) {
// first run
long pl = getRunningLength();
if (index + pl > max) {
pl = max - index;
}
container.addStreamOfEmptyWords(getRunningBit(), pl);
index += pl;
int pd = getNumberOfLiteralWords();
if (pd + index > max) {
pd = (int) (max - index);
}
writeLiteralWords(pd, container);
discardFirstWords(pl + pd);
index += pd;
}
return index;
long index = 0;
while (true) {
if (index + getRunningLength() > max) {
final int offset = (int) (max - index);
container.addStreamOfEmptyWords(getRunningBit(), offset);
this.brlw.runningLength -= offset;
return max;
}
container.addStreamOfEmptyWords(getRunningBit(), getRunningLength());
index += getRunningLength();
if (getNumberOfLiteralWords() + index > max) {
final int offset =(int) (max - index);
writeLiteralWords(offset, container);
this.brlw.runningLength = 0;
this.brlw.numberOfLiteralWords -= offset;
this.literalWordStartPosition += offset;
return max;
}
writeLiteralWords(getNumberOfLiteralWords(), container);
index += getNumberOfLiteralWords();
if(!next()) break;
}
return index;
}
/**
* Write out up to max words (negated), returns how many were written
*
......@@ -283,4 +299,5 @@ public final class IteratingBufferedRunningLengthWord implements IteratingRLW,
private final Buffer buffer;
private int literalWordStartPosition;
private EWAHIterator iterator;
}
src/main/java/com/googlecode/javaewah/IteratingRLW.java
+ 6
− 0
View file @ 59068125
......@@ -50,6 +50,12 @@ public interface IteratingRLW {
*/
void discardRunningWords();
/**
* Discard x literal words (assumes that there is no running word)
* @param x the number of words to discard
*/
void discardLiteralWords(long x);
/**
* @return a copy of the iterator
* @throws CloneNotSupportedException this should not be thrown in theory
......
src/main/java/com/googlecode/javaewah/IteratorAggregation.java
+ 5
− 0
View file @ 59068125
......@@ -70,6 +70,11 @@ public final class IteratorAggregation {
throws CloneNotSupportedException {
throw new CloneNotSupportedException();
}
@Override
public void discardLiteralWords(long y) {
x.discardLiteralWords(y);
}
};
}
......
src/main/java/com/googlecode/javaewah/IteratorUtil.java
+ 1
− 1
View file @ 59068125
......@@ -79,7 +79,7 @@ public final class IteratorUtil {
int answer = 0;
while (true) {
if (i.getRunningBit())
answer += i.getRunningLength() * EWAHCompressedBitmap.WORD_IN_BITS;
answer += (int) (i.getRunningLength() * EWAHCompressedBitmap.WORD_IN_BITS);
int lw = i.getNumberOfLiteralWords();
for (int k = 0; k < lw ; ++k)
answer += Long.bitCount(i.getLiteralWordAt(k));
......
src/main/java/com/googlecode/javaewah/LongBufferWrapper.java
+ 1
− 1
View file @ 59068125
......@@ -13,7 +13,7 @@ import java.nio.LongBuffer;
*
* @author Gregory Ssi-Yan-Kai
*/
final class LongBufferWrapper implements Buffer {
final class LongBufferWrapper implements Buffer, Cloneable {
public LongBufferWrapper(LongBuffer buffer) {
this.buffer = buffer;
......
src/main/java/com/googlecode/javaewah/datastructure/BitSet.java
+ 16
− 2
View file @ 59068125
package com.googlecode.javaewah.datastructure;
import com.googlecode.javaewah.IntIterator;
import java.io.DataInput;
import java.io.DataOutput;
import java.io.Externalizable;
......@@ -11,6 +10,7 @@ import java.io.ObjectOutput;
import java.util.Arrays;
import java.util.Iterator;
/**
* <p>This is an optimized version of Java's BitSet. In many cases, it can be used
* as a drop-in replacement.</p>
......@@ -400,7 +400,21 @@ public class BitSet implements Cloneable, Iterable<Integer>, Externalizable ,Wor
return sum;
}
/**
* Remove a word.
*
*
* @param i index of the word to be removed.
*/
public void removeWord(int i) {
long[] newdata = new long[data.length - 1];
if (i == 0) {
System.arraycopy(data, 1, newdata, 0, i - 1);
}
System.arraycopy(data, 0, newdata, 0, i - 1);
System.arraycopy(data, i, newdata, i - 1, data.length - i);
data = newdata;
}
/**
* Resize the bitset
......
src/main/java/com/googlecode/javaewah/datastructure/ImmutableBitSet.java
+ 2
− 0
View file @ 59068125
......@@ -109,6 +109,8 @@ public class ImmutableBitSet implements Cloneable, Iterable<Integer>,WordArray
}
/**
* get value of bit i
*
* @param i index
* @return value of the bit
*/
......
src/main/java/com/googlecode/javaewah32/BufferedIterator32.java
+ 8
− 0
View file @ 59068125
......@@ -54,6 +54,14 @@ public class BufferedIterator32 implements IteratingRLW32, Cloneable {
}
}
}
@Override
public void discardLiteralWords(int x) {
this.iteratingBrlw.discardLiteralWords(x);
if (this.iteratingBrlw.getNumberOfLiteralWords() == 0)
this.next();
}
@Override
public void discardRunningWords() {
......
src/main/java/com/googlecode/javaewah32/EWAHCompressedBitmap32.java
+ 11
− 11
View file @ 59068125
......@@ -10,7 +10,6 @@ import com.googlecode.javaewah.IntIterator;
import com.googlecode.javaewah.LogicalElement;
import com.googlecode.javaewah32.symmetric.RunningBitmapMerge32;
import com.googlecode.javaewah32.symmetric.ThresholdFuncBitmap32;
import java.io.*;
import java.nio.ByteBuffer;
import java.nio.IntBuffer;
......@@ -18,6 +17,7 @@ import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
/**
* <p>
* This implements the patent-free EWAH scheme. Roughly speaking, it is a 32-bit
......@@ -420,8 +420,8 @@ public final class EWAHCompressedBitmap32 implements Cloneable, Externalizable,
for (int k = 0; k < nbre_literal; ++k)
container.addWord(rlwi.getLiteralWordAt(k)
& rlwj.getLiteralWordAt(k));
rlwi.discardFirstWords(nbre_literal);
rlwj.discardFirstWords(nbre_literal);
rlwi.discardLiteralWords(nbre_literal);
rlwj.discardLiteralWords(nbre_literal);
}
}
......@@ -532,8 +532,8 @@ public final class EWAHCompressedBitmap32 implements Cloneable, Externalizable,
for (int k = 0; k < nbre_literal; ++k)
container.addWord(rlwi.getLiteralWordAt(k)
& (~rlwj.getLiteralWordAt(k)));
rlwi.discardFirstWords(nbre_literal);
rlwj.discardFirstWords(nbre_literal);
rlwi.discardLiteralWords(nbre_literal);
rlwj.discardLiteralWords(nbre_literal);
}
}
final boolean i_remains = rlwi.size() > 0;
......@@ -824,7 +824,7 @@ public final class EWAHCompressedBitmap32 implements Cloneable, Externalizable,
final int lw = i.literalWords();
for (int k = 0; k < nlw; ++k) {
long W = this.buffer.getWord(lw + k);
karprabin += B * W;
karprabin += (int) (B * W);
}
}
return karprabin;
......@@ -1072,8 +1072,8 @@ public final class EWAHCompressedBitmap32 implements Cloneable, Externalizable,
container.addWord(rlwi.getLiteralWordAt(k)
| rlwj.getLiteralWordAt(k));
}
rlwi.discardFirstWords(nbre_literal);
rlwj.discardFirstWords(nbre_literal);
rlwi.discardLiteralWords(nbre_literal);
rlwj.discardLiteralWords(nbre_literal);
}
}
if ((rlwj.size() > 0) && (rlwi.size() > 0)) throw new RuntimeException("fds");
......@@ -1763,8 +1763,8 @@ public final class EWAHCompressedBitmap32 implements Cloneable, Externalizable,
for (int k = 0; k < nbre_literal; ++k)
container.addWord(rlwi.getLiteralWordAt(k)
^ rlwj.getLiteralWordAt(k));
rlwi.discardFirstWords(nbre_literal);
rlwj.discardFirstWords(nbre_literal);
rlwi.discardLiteralWords(nbre_literal);
rlwj.discardLiteralWords(nbre_literal);
}
}
final boolean i_remains = rlwi.size() > 0;
......@@ -2056,7 +2056,7 @@ public final class EWAHCompressedBitmap32 implements Cloneable, Externalizable,
}
/**
* Generate a new bitmap a new bitmap shifted by "b" bits.
* Generates a new bitmap shifted by "b" bits.
* If b is positive, the position of all set bits is increased by
* b. The negative case is not supported.
*
......
src/main/java/com/googlecode/javaewah32/IntBufferWrapper.java
+ 1
− 1
View file @ 59068125
......@@ -15,7 +15,7 @@ import java.nio.IntBuffer;
*
* @author Gregory Ssi-Yan-Kai
*/
final class IntBufferWrapper implements Buffer32 {
final class IntBufferWrapper implements Buffer32, Cloneable {
public IntBufferWrapper(IntBuffer buffer) {
this.buffer = buffer;
......
src/main/java/com/googlecode/javaewah32/IteratingBufferedRunningLengthWord32.java
+ 38
− 18
View file @ 59068125
......@@ -39,7 +39,6 @@ public final class IteratingBufferedRunningLengthWord32 implements
final EWAHCompressedBitmap32 bitmap) {
this(EWAHIterator32.getEWAHIterator(bitmap));
}
/**
* Discard first words, iterating to the next running length word if
* needed.
......@@ -71,6 +70,20 @@ public final class IteratingBufferedRunningLengthWord32 implements
}
}
}
@Override
public void discardLiteralWords(int x) {
this.literalWordStartPosition += x;
this.brlw.NumberOfLiteralWords -= x;
if (this.brlw.NumberOfLiteralWords == 0) {
if (!this.iterator.hasNext()) {
return;
}
this.brlw.reset(this.iterator.next());
this.literalWordStartPosition = this.iterator.literalWords();
}
}
@Override
public void discardRunningWords() {
......@@ -87,26 +100,32 @@ public final class IteratingBufferedRunningLengthWord32 implements
* @return how many written
*/
public int discharge(BitmapStorage32 container, int max) {
int index = 0;
while ((index < max) && (size() > 0)) {
// first run
int pl = getRunningLength();
if (index + pl > max) {
pl = max - index;
}
container.addStreamOfEmptyWords(getRunningBit(), pl);
index += pl;
int pd = getNumberOfLiteralWords();
if (pd + index > max) {
pd = max - index;
}
writeLiteralWords(pd, container);
discardFirstWords(pl + pd);
index += pd;
int index = 0;
while (true) {
if (index + getRunningLength() > max) {
final int offset = max - index;
container.addStreamOfEmptyWords(getRunningBit(), offset);
this.brlw.RunningLength -= offset;
return max;
}
return index;
container.addStreamOfEmptyWords(getRunningBit(), getRunningLength());
index += getRunningLength();
if (getNumberOfLiteralWords() + index > max) {
final int offset = max - index;
writeLiteralWords(offset, container);
this.brlw.RunningLength = 0;
this.brlw.NumberOfLiteralWords -= offset;
this.literalWordStartPosition += offset;
return max;
}
writeLiteralWords(getNumberOfLiteralWords(), container);
index += getNumberOfLiteralWords();
if(!next()) break;
}
return index;
}
/**
* Write out up to max words (negated), returns how many were written
*
......@@ -298,5 +317,6 @@ public final class IteratingBufferedRunningLengthWord32 implements
private final Buffer32 buffer;
private int literalWordStartPosition;
private EWAHIterator32 iterator;
}
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment