Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
126 changes: 87 additions & 39 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,33 +1,75 @@
java-generator-functions
========================

An implementation of Python-like generator functions in Java. This repository contains a single class, `Generator` with a method `yield(...)` which can be used to mimic the behaviour of the `yield` keyword in Python.
An implementation of Python-like generator functions in Java. This repository contains a functional interface, `GeneratorFunc`, which accepts an object with a method, `yield(...)`, that can be used to mimic the behaviour of the `yield` keyword in Python.

Examples
--------
The following is a simple generator that yields `1` and then `2`:

Generator<Integer> simpleGenerator = new Generator<Integer>() {
public void run() throws InterruptedException {
yield(1);
// Some logic here...
yield(2);
}
};
for (Integer element : simpleGenerator)
System.out.println(element);
// Prints "1", then "2".
```java
GeneratorFunc<Integer> simpleGenerator = s -> {
s.yield(1);
// Some logic here...
s.yield(2);
};

for (Integer element : simpleGenerator)
System.out.println(element);
// Prints "1", then "2".
```

Infinite generators are also possible:

Generator<Integer> infiniteGenerator = new Generator<Integer>() {
public void run() throws InterruptedException {
while (true)
yield(1);
}
};
```java
GeneratorFunc<Integer> infiniteGenerator = s -> {
while (true)
s.yield(1);
};
```

You can even use a generator to create a (parallel) `Stream`:

```java
GeneratorFunc<Integer> infiniteGenerator = s -> {
int i = 0;
while (true) {
s.yield(i++);
}
};

infiniteGenerator.stream().limit(100).parallel() // and so on
```

Or, equivalently:

```java
// Note that the generic parameter is necessary, or else Java can't determine
// the generator's type.
Generator.<Integer>stream(s -> {
int i = 0;
while (true) {
s.yield(i++);
}
}).limit(100).parallel() // and so on
```

If you need to use an anonymous inner class, it is more concise to have it extend `Generator`, at the cost of losing statelessness:

```java
Generator<Integer> infiniteGenerator = new Generator<Integer>() {
public void run() throws InterruptedException {
while (true)
yield(1);
}
};
```

You can iterate over a generator multiple times, resulting in multiple calls to the lambda or `run` method. If the generator modifies some state, you can expect that state to be modified each time you iterate over the generator (or create a `Stream` from it).

The `Generator` class lies in package `io.herrmann.generator`. So you need to `import io.herrmann.generator.Generator;` in order for the above examples to work.
For more examples, see [GeneratorTest.java](src/test/java/io/herrmann/generator/GeneratorTest.java).

The `Generator` class and `GeneratorFunc` interface lie in the package `io.herrmann.generator`, so you need to `import io.herrmann.generator.*;` in order for the above examples to work.

Usage
-----
Expand All @@ -38,37 +80,43 @@ This package is hosted as a Maven repository with the following url:

To use it from Maven, add the following to your `pom.xml`:

<project>
...
<repositories>
...
<repository>
<id>java-generator-functions</id>
<url>http://dl.bintray.com/filipmalczak/maven</url>
</repository>
</repositories>
```xml
<project>
...
<repositories>
...
<dependencies>
<dependency>
<groupId>io.herrmann</groupId>
<artifactId>java-generator-functions</artifactId>
<version>1.0</version>
</dependency>
</dependencies>
</project>
<repository>
<id>java-generator-functions</id>
<url>http://dl.bintray.com/filipmalczak/maven</url>
</repository>
</repositories>
...
<dependencies>
<dependency>
<groupId>io.herrmann</groupId>
<artifactId>java-generator-functions</artifactId>
<version>1.0</version>
</dependency>
</dependencies>
</project>
```

For Gradle:

compile(group: 'io.herrmann', name: 'java-generator-functions', version: '1.0')
```gradle
compile(group: 'io.herrmann', name: 'java-generator-functions', version: '1.0')
```

Caveats and Performance
-----------------------
The `Generator` class internally works with a Thread to produce the items. It does ensure that no Threads stay around if the corresponding Generator is no longer used. However:
The `Generator` library internally works with a Thread to produce the items. It does ensure that no Threads stay around if the corresponding Generator is no longer used. However:

**If too many `Generator`s are created before the JVM gets a chance to garbage collect the old ones, you may encounter `OutOfMemoryError`s. This problem most strongly presents itself on OS X where the maximum number of Threads is significantly lower than on other OSs (around 2000).**

The performance is obviously not great but not too shabby either. On my machine with a dual core i5 CPU @ 2.67 GHz, 1000 items can be produced in < 0.03s.
The performance is obviously not great, but not too shabby either. On my machine with a dual core i5 CPU @ 2.67 GHz, 1000 items can be produced in < 0.03s.

This version requires Java 8, as it takes advantage of functional interfaces in its API and provides integration with the Streams API. If you need support for an older version of Java, use version 1.0 of this library.

Contributing
------------
Contributions and pull requests are welcome. Please ensure that `mvn test` still passes and add any unit tests as you see fit. Please also follow the same coding conventions, in particular the line limit of 80 characters and the use of tabs instead of spaces.
Contributions and pull requests are welcome. Please ensure that `mvn test` still passes and add any unit tests as you see fit. Please also follow the same coding conventions, in particular the line limit of 80 characters and the use of tabs instead of spaces.
17 changes: 16 additions & 1 deletion pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -44,4 +44,19 @@
<scope>test</scope>
</dependency>
</dependencies>
</project>

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.5.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
</plugins>
</build>

</project>
133 changes: 29 additions & 104 deletions src/main/java/io/herrmann/generator/Generator.java
Original file line number Diff line number Diff line change
@@ -1,124 +1,49 @@
package io.herrmann.generator;

import java.util.Iterator;
import java.util.NoSuchElementException;
import java.util.stream.Stream;
import java.util.stream.StreamSupport;

/**
* This class allows specifying Python generator-like sequences. For examples,
* see the JUnit test case.
*
* The implementation uses a separate Thread to produce the sequence items. This
* is certainly not as fast as eg. a for-loop, but not horribly slow either. On
* a machine with a dual core i5 CPU @ 2.67 GHz, 1000 items can be produced in
* &lt; 0.03s.
*
* By overriding finalize(), the class takes care not to leave any Threads
* running longer than necessary.
* Implementation of {@link GeneratorFunc} as an abstract class. This class
* mainly exists for backwards compatibility, but it can also cut down some of
* the boilerplate when using an anonymous inner class instead of a lambda.
* However, unlike a {@link GeneratorFunc}, this class is not stateless, and
* cannot be used concurrently.
*/
public abstract class Generator<T> implements Iterable<T> {
public abstract class Generator<T> implements GeneratorFunc<T> {

private class Condition {
private boolean isSet;
public synchronized void set() {
isSet = true;
notify();
}
public synchronized void await() throws InterruptedException {
try {
if (isSet)
return;
wait();
} finally {
isSet = false;
}
}
}

static ThreadGroup THREAD_GROUP;

Thread producer;
private boolean hasFinished;
private final Condition itemAvailableOrHasFinished = new Condition();
private final Condition itemRequested = new Condition();
private T nextItem;
private boolean nextItemAvailable;
private RuntimeException exceptionRaisedByProducer;
private GeneratorIterator<T> iter;

@Override
public Iterator<T> iterator() {
return new Iterator<T>() {
@Override
public boolean hasNext() {
return waitForNext();
}
@Override
public T next() {
if (!waitForNext())
throw new NoSuchElementException();
nextItemAvailable = false;
return nextItem;
}
@Override
public void remove() {
throw new UnsupportedOperationException();
}
private boolean waitForNext() {
if (nextItemAvailable)
return true;
if (hasFinished)
return false;
if (producer == null)
startProducer();
itemRequested.set();
try {
itemAvailableOrHasFinished.await();
} catch (InterruptedException e) {
hasFinished = true;
}
if (exceptionRaisedByProducer != null)
throw exceptionRaisedByProducer;
return !hasFinished;
}
};
public void run(GeneratorIterator<T> gen) throws InterruptedException {
run();
}

protected abstract void run() throws InterruptedException;

protected void yield(T element) throws InterruptedException {
nextItem = element;
nextItemAvailable = true;
itemAvailableOrHasFinished.set();
itemRequested.await();
iter.yield(element);
}

private void startProducer() {
assert producer == null;
if (THREAD_GROUP == null)
THREAD_GROUP = new ThreadGroup("generatorfunctions");
producer = new Thread(THREAD_GROUP, new Runnable() {
@Override
public void run() {
try {
itemRequested.await();
Generator.this.run();
} catch (InterruptedException e) {
// No need to do anything here; Remaining steps in run()
// will cleanly shut down the thread.
} catch (RuntimeException e) {
exceptionRaisedByProducer = e;
}
hasFinished = true;
itemAvailableOrHasFinished.set();
}
});
producer.setDaemon(true);
producer.start();
@Override
public Iterator<T> iterator() {
iter = new GeneratorIterator<>(this);
return iter;
}

@Override
protected void finalize() throws Throwable {
producer.interrupt();
producer.join();
super.finalize();
/**
* Creates a {@link Stream} from a {@link GeneratorFunc}. For cases where
* the generator isn't a lambda passed directly, the instance method {@link
* #stream()} is generally more concise.
*
* @param g The generator
* @return An ordered, sequential (non-parallel) stream of elements yielded
* by the generator
* @see #stream()
*/
public static <T> Stream<T> stream(GeneratorFunc<T> g) {
return StreamSupport.stream(g.spliterator(), false);
}

}
55 changes: 55 additions & 0 deletions src/main/java/io/herrmann/generator/GeneratorFunc.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
package io.herrmann.generator;

import java.util.Iterator;
import java.util.Spliterator;
import java.util.Spliterators;
import java.util.stream.Stream;

/**
* This functional interface allows specifying Python generator-like sequences.
* For examples, see the JUnit test case.
*
* The implementation uses a separate Thread to produce the sequence items. This
* is certainly not as fast as eg. a for-loop, but not horribly slow either. On
* a machine with a dual core i5 CPU @ 2.67 GHz, 1000 items can be produced in
* &lt; 0.03s.
*
* By overriding finalize(), the underlying iterator takes care not to leave any
* Threads running longer than necessary.
*
* @see Generator
*/
@FunctionalInterface
public interface GeneratorFunc<T> extends Iterable<T> {

@Override
public default Iterator<T> iterator() {
return new GeneratorIterator<>(this);
}

public void run(GeneratorIterator<T> gen) throws InterruptedException;

/**
* Returns an ordered {@link Spliterator} consisting of elements yielded by
* this {@link GeneratorFunc}.
*/
@Override
default Spliterator<T> spliterator() {
return Spliterators.spliteratorUnknownSize(iterator(),
Spliterator.ORDERED);
}

/**
* Creates a {@link Stream} from a {@link GeneratorFunc}. If you are trying
* to call this on a lambda, you should either use the static method {@link
* Generator#stream()} or assign it to a variable first.
*
* @param g The generator
* @return An ordered, sequential (non-parallel) stream of elements yielded
* by the generator
*/
public default Stream<T> stream() {
return Generator.stream(this);
}

}
Loading