170

I can add streams or extra elements, like this:

Stream stream = Stream.concat(stream1, Stream.concat(stream2, Stream.of(element));

And I can add new stuff as I go, like this:

Stream stream = Stream.concat(
                       Stream.concat(
                              stream1.filter(x -> x!=0), stream2)
                              .filter(x -> x!=1),
                                  Stream.of(element))
                                  .filter(x -> x!=2);

But this is ugly, because concat is static. If concat were an instance method, the above examples would be much easier to read:

 Stream stream = stream1.concat(stream2).concat(element);

And

 Stream stream = stream1
                 .filter(x -> x!=0)
                 .concat(stream2)
                 .filter(x -> x!=1)
                 .concat(element)
                 .filter(x -> x!=2);

My question is:

1) Is there any good reason why concat is static? Or is there some equivalent instance method I'm missing?

2) In any case, is there a better way of doing this?

skiwi
  • 59,273
  • 29
  • 118
  • 198
MarcG
  • 21,878
  • 14
  • 83
  • 95
  • 4
    It looks like [thing were not always like this](http://mail.openjdk.java.net/pipermail/lambda-dev/2012-November/006887.html), but I just can't find the reason why. – Edwin Dalorzo Mar 30 '14 at 06:05

8 Answers8

168

Unfortunately this answer is probably of little or no help whatsoever, but I did a forensics analysis of the Java Lambda Mailing list to see if I could find the cause of this design. This is what I found out.

In the beginning there was an instance method for Stream.concat(Stream)

In the mailing list I can clearly see the method was originally implemented as an instance method, as you can read in this thread by Paul Sandoz, about the concat operation.

In it they discuss the issues that could arise from those cases in which the stream could be infinite and what concatenation would mean in those cases, but I do not think that was the reason for the modification.

You see in this other thread that some early users of the JDK 8 questioned about the behavior of the concat instance method when used with null arguments.

This other thread reveals, though, that the design of the concat method was under discussion.

Refactored to Streams.concat(Stream,Stream)

But without any explanation, suddenly, the methods were changed to static methods, as you can see in this thread about combining streams. This is perhaps the only mail thread that sheds a bit of light about this change, but it was not clear enough for me to determine the reason for the refactoring. But we can see they did a commit in which they suggested to move the concat method out of Stream and into the helper class Streams.

Refactored to Stream.concat(Stream,Stream)

Later, it was moved again from Streams to Stream, but yet again, no explanation for that.

So, bottom line, the reason for the design is not entirely clear for me and I could not find a good explanation. I guess you could still ask the question in the mailing list.

Some Alternatives for Stream Concatenation

This other thread by Michael Hixson discusses/asks about other ways to combine/concat streams

  1. To combine two streams, I should do this:

    Stream.concat(s1, s2)
    

    not this:

    Stream.of(s1, s2).flatMap(x -> x)
    

    ... right?

  2. To combine more than two streams, I should do this:

    Stream.of(s1, s2, s3, ...).flatMap(x -> x)
    

    not this:

    Stream.of(s1, s2, s3, ...).reduce(Stream.empty(), Stream::concat)
    

    ... right?

kapex
  • 26,163
  • 5
  • 97
  • 111
Edwin Dalorzo
  • 70,022
  • 25
  • 131
  • 191
  • 6
    +1 Nice research. And I will be using this as my Stream.concat taking varargs: `public static Stream concat(Stream... streams) { return Stream.of(streams).reduce(Stream.empty(), Stream::concat);}` – MarcG Mar 30 '14 at 20:11
  • 1
    Today i wrote my own concat version, and just after that I fund this topic. Signature is slightly different but thanks to that its more generic;) for example you could merge Stream and Stream to Stream.`@SafeVarargs private static Stream concat(Stream extends T>... streams) { return Stream.of(streams).reduce(Stream.empty(),Stream::concat).map(Function.identity());}` – kant Sep 14 '14 at 17:10
  • @kant Why do you need a `Function.identity()` map? After all, it returns the same argument it receive. This should have no effect in the resulting stream. Am I missing something? – Edwin Dalorzo Sep 14 '14 at 17:47
  • 1
    Have you tried to type it in your IDE? Without .map(identity()) you will get compilation error. I want to return Stream but statement: `return Stream.of(streams).reduce(Stream.empty(),Stream::concat)` returns Stream extends T>.(Someting is subtype of Something extends T>, not the other way, so it couldn't be cast) Additional `.map(identity())` cast extends T> to . It happens thanks to mix of java 8 'target types' of method arguments and return types and signature of map() method. Actually it is Function.identity(). – kant Sep 14 '14 at 19:18
  • @kant I just did without any problems. I guess your problem lies somewhere in your generics, not in the stream concatenation idiom. – Edwin Dalorzo Sep 14 '14 at 19:27
  • Could you paste your code? Did your method return Stream or Stream extends T> ? – kant Sep 14 '14 at 19:31
  • 1
    @kant I don't see much point in doing `? extends T`, since you can use [capture conversion](http://docs.oracle.com/javase/specs/jls/se8/html/jls-5.html#jls-5.1.10). At any rate, here's [my gist code snippet](https://gist.github.com/edalorzo/148c6e4f8f0c49d2d5f2) Let's continue the discussion in the Gist. – Edwin Dalorzo Sep 14 '14 at 19:48
127

If you add static imports for Stream.concat and Stream.of, the first example could be written as follows:

Stream<Foo> stream = concat(stream1, concat(stream2, of(element)));

Importing static methods with generic names can result in code that becomes difficult to read and maintain (namespace pollution). So, it might be better to create your own static methods with more meaningful names. However, for demonstration I will stick with this name.

public static <T> Stream<T> concat(Stream<? extends T> lhs, Stream<? extends T> rhs) {
    return Stream.concat(lhs, rhs);
}
public static <T> Stream<T> concat(Stream<? extends T> lhs, T rhs) {
    return Stream.concat(lhs, Stream.of(rhs));
}

With these two static methods (optionally in combination with static imports), the two examples could be written as follows:

Stream<Foo> stream = concat(stream1, concat(stream2, element));

Stream<Foo> stream = concat(
                         concat(stream1.filter(x -> x!=0), stream2).filter(x -> x!=1),
                         element)
                     .filter(x -> x!=2);

The code is now significantly shorter. However, I agree that the readability hasn't improved. So I have another solution.


In a lot of situations, Collectors can be used to extend the functionality of streams. With the two Collectors at the bottom, the two examples could be written as follows:

Stream<Foo> stream = stream1.collect(concat(stream2)).collect(concat(element));

Stream<Foo> stream = stream1
                     .filter(x -> x!=0)
                     .collect(concat(stream2))
                     .filter(x -> x!=1)
                     .collect(concat(element))
                     .filter(x -> x!=2);

The only difference between your desired syntax and the syntax above is, that you have to replace concat(...) with collect(concat(...)). The two static methods can be implemented as follows (optionally used in combination with static imports):

private static <T,A,R,S> Collector<T,?,S> combine(Collector<T,A,R> collector, Function<? super R, ? extends S> function) {
    return Collector.of(
        collector.supplier(),
        collector.accumulator(),
        collector.combiner(),
        collector.finisher().andThen(function));
}
public static <T> Collector<T,?,Stream<T>> concat(Stream<? extends T> other) {
    return combine(Collectors.toList(),
        list -> Stream.concat(list.stream(), other));
}
public static <T> Collector<T,?,Stream<T>> concat(T element) {
    return concat(Stream.of(element));
}

Of course there is a drawback with this solution that should be mentioned. collect is a final operation that consumes all elements of the stream. On top of that, the collector concat creates an intermediate ArrayList each time it is used in the chain. Both operations can have a significant impact on the behaviour of your program. However, if readability is more important than performance, it might still be a very helpful approach.

nosid
  • 45,370
  • 13
  • 104
  • 134
  • 1
    I don't find the `concat` collector much readable. It seems strange to a have single-parameter static method named like this, and also to use `collect` for concatenation. – Didier L Jul 08 '15 at 11:45
  • @nosid, perhaps a slightly orthogonal question to this thread but why do you claim `It's a bad idea to import static methods with names`? I am genuinely interested - I find it makes the code more concise and readable and a lot of people I had asked thought the same. Care to provide some examples why that is generally bad? – quantum Dec 08 '15 at 17:32
  • 1
    @Quantum: What's the meaning of `compare(reverse(getType(42)), of(6 * 9).hashCode())`? Note that I didn't say that _static imports_ are a bad idea, but _static imports_ for generic names like `of` and `concat` are. – nosid Dec 09 '15 at 22:22
  • 1
    @nosid: Won't hovering over each statment in a modern IDE quickly reveal the meaning? At any rate, I think this could arguably be a personal preference statement at best, as I still see no technical reason why static imports for "generic" names is bad - unless you are using Notepad or VI(M) for programming in which case you have bigger problems. – quantum Dec 10 '15 at 15:39
  • I'm not gonna say that the Scala SDK is better, but... oops I said it. – eirirlar Sep 17 '18 at 12:33
12

My StreamEx library extends the functionality of Stream API. In particular it offers methods like append and prepend which solve this issue (internally they use concat). These methods can accept either another stream or collection or varargs array. Using my library your problem can be solved this way (note that x != 0 look strange for non-primitive stream):

Stream<Integer> stream = StreamEx.of(stream1)
             .filter(x -> !x.equals(0))
             .append(stream2)
             .filter(x -> !x.equals(1))
             .append(element)
             .filter(x -> !x.equals(2));

By the way there's also a shortcut for your filter operation:

Stream<Integer> stream = StreamEx.of(stream1).without(0)
                                 .append(stream2).without(1)
                                 .append(element).without(2);
Tagir Valeev
  • 87,515
  • 18
  • 194
  • 305
11

Just do:

Stream.of(stream1, stream2, Stream.of(element)).flatMap(identity());

where identity() is a static import of Function.identity().

Concatenating multiple streams into one stream is the same as flattening a stream.

However, unfortunately, for some reason there is no flatten() method on Stream, so you have to use flatMap() with the identity function.

herman
  • 9,302
  • 4
  • 38
  • 51
2

You can use Guava's Streams.concat(Stream<? extends T>... streams) method, which will be very short with static imports:

Stream stream = concat(stream1, stream2, of(element));
Neuron
  • 3,776
  • 3
  • 24
  • 44
Kunda
  • 391
  • 1
  • 5
  • `Stream.of` is not meant to be static-imported. It is read as "stream of" similar to `Optional.of` which is read as "optional of". You wouldn't even be able to import both `Stream.of` and `Optional.of` in the same file. – herman Feb 02 '21 at 13:57
  • @herman you're right - I modified my answer to focus on the brevity Guava brings, and leave the use (or misuse) of static imports to the discretion of the reader. – Kunda May 30 '21 at 09:19
1

If you don't mind using 3rd Party Libraries cyclops-react has an extended Stream type that will allow you to do just that via the append / prepend operators.

Individual values, arrays, iterables, Streams or reactive-streams Publishers can be appended and prepended as instance methods.

Stream stream = ReactiveSeq.of(1,2)
                           .filter(x -> x!=0)
                           .append(ReactiveSeq.of(3,4))
                           .filter(x -> x!=1)
                           .append(5)
                           .filter(x -> x!=2);

[Disclosure I am the lead developer of cyclops-react]

John McClean
  • 4,767
  • 1
  • 20
  • 30
0

How about writing your own concat method?

public static <T> Stream<T> concat(Stream<? extends T> a, 
                                   Stream<? extends T> b, 
                                   Stream<? extends T>... args)
{
    Stream<T> concatenated = Stream.concat(a, b);
    for (Stream<? extends T> stream : args)
    {
        concatenated = Stream.concat(concatenated, stream);
    }
    return concatenated;
}

This at least makes your first example a lot more readable.

As @Legna pointed out, this could very fast result in a StackOverflowError due to nested calls to Stream::concat.

So here is another version that should fix the problem and looks quite neat:

public static <T> Stream<T> concat(final Stream<? extends T>... args)
{
    return args == null ? Stream.empty()
                        : Stream.of(args).flatMap(Function.identity());
}
Felix S
  • 88
  • 9
  • 2
    Use caution when constructing streams from repeated concatenation. Accessing an element of a deeply concatenated stream can result in deep call chains, or even StackOverflowError. – Legna Mar 15 '19 at 18:47
0

At the end of the day I'm not interested in combining streams, but in obtaining the combined result of processing each element of all those streams.

While combining streams might prove to be cumbersome (thus this thread), combining their processing results is fairly easy.

The key to solve is to create your own collector and ensure that the supplier function for the new collector returns the same collection every time (not a new one), the code below illustrates this approach.

package scratchpad;

import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.stream.Collector;
import java.util.stream.Stream;

public class CombineStreams {
    public CombineStreams() {
        super();
    }

    public static void main(String[] args) {
        List<String> resultList = new ArrayList<>();
        Collector<String, List<String>, List<String>> collector = Collector.of(
                () -> resultList,
                (list, item) -> {
                    list.add(item);
                },
                (llist, rlist) -> {
                    llist.addAll(rlist);
                    return llist;
                }
        );
        String searchString = "Wil";

        System.out.println("After processing first stream\n"
                + createFirstStream().filter(name -> name.contains(searchString)).collect(collector));
        System.out.println();

        System.out.println("After processing second stream\n"
                + createSecondStream().filter(name -> name.contains(searchString)).collect(collector));
        System.out.println();

        System.out.println("After processing third stream\n"
                + createThirdStream().filter(name -> name.contains(searchString)).collect(collector));
        System.out.println();

    }

    private static Stream<String> createFirstStream() {
        return Arrays.asList(
                "William Shakespeare",
                "Emily Dickinson",
                "H. P. Lovecraft",
                "Arthur Conan Doyle",
                "Leo Tolstoy",
                "Edgar Allan Poe",
                "Robert Ervin Howard",
                "Rabindranath Tagore",
                "Rudyard Kipling",
                "Seneca",
                "John Donne",
                "Sarah Williams",
                "Oscar Wilde",
                "Catullus",
                "Alfred Tennyson",
                "William Blake",
                "Charles Dickens",
                "John Keats",
                "Theodor Herzl"
        ).stream();
    }

    private static Stream<String> createSecondStream() {
        return Arrays.asList(
                "Percy Bysshe Shelley",
                "Ernest Hemingway",
                "Barack Obama",
                "Anton Chekhov",
                "Henry Wadsworth Longfellow",
                "Arthur Schopenhauer",
                "Jacob De Haas",
                "George Gordon Byron",
                "Jack London",
                "Robert Frost",
                "Abraham Lincoln",
                "O. Henry",
                "Ovid",
                "Robert Louis Stevenson",
                "John Masefield",
                "James Joyce",
                "Clark Ashton Smith",
                "Aristotle",
                "William Wordsworth",
                "Jane Austen"
        ).stream();
    }

    private static Stream<String> createThirdStream() {
        return Arrays.asList(
                "Niccolò Machiavelli",
                "Lewis Carroll",
                "Robert Burns",
                "Edgar Rice Burroughs",
                "Plato",
                "John Milton",
                "Ralph Waldo Emerson",
                "Margaret Thatcher",
                "Sylvie d'Avigdor",
                "Marcus Tullius Cicero",
                "Banjo Paterson",
                "Woodrow Wilson",
                "Walt Whitman",
                "Theodore Roosevelt",
                "Agatha Christie",
                "Ambrose Bierce",
                "Nikola Tesla",
                "Franz Kafka"
        ).stream();
    }
}
Legna
  • 1,443
  • 13
  • 15