107

Say we have a Map: let m = new Map();, using m.values() returns a map iterator.

But I can't use forEach() or map() on that iterator and implementing a while loop on that iterator seems like an anti-pattern since ES6 offer functions like map().

So is there a way to use map() on an iterator?

shinzou
  • 4,689
  • 6
  • 48
  • 103
  • Not out of the box, but you can use third party libraries like `lodash` `map` function which supports Map as well. – hazardous May 10 '17 at 06:56
  • Map itself has a [forEach](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map/forEach) for iterating over its key-value pairs. – hazardous May 10 '17 at 06:57
  • Converting the iterator to an array and map on it like `Array.from(m.values()).map(...)` works, but I think it's not the best way to do this. – JiminP May 10 '17 at 06:58
  • which problem like you to solve with using an iterator while an array would fit better for using [`Array#map`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/map)? – Nina Scholz May 10 '17 at 07:27
  • 1
    @NinaScholz I'm using a general set like here: http://stackoverflow.com/a/29783624/4279201 – shinzou May 10 '17 at 09:27

8 Answers8

101

The simplest and least performant way to do this is:

Array.from(m).map(([key,value]) => /* whatever */)

Better yet

Array.from(m, ([key, value]) => /* whatever */))

Array.from takes any iterable or array-like thing and converts it into an array! As Daniel points out in the comments, we can add a mapping function to the conversion to remove an iteration and subsequently an intermediate array.

Using Array.from will move your performance from O(1) to O(n) as @hraban points out in the comments. Since m is a Map, and they can't be infinite, we don't have to worry about an infinite sequence. For most instances, this will suffice.

There are a couple of other ways to loop through a map.

Using forEach

m.forEach((value,key) => /* stuff */ )

Using for..of

var myMap = new Map();
myMap.set(0, 'zero');
myMap.set(1, 'one');
for (var [key, value] of myMap) {
  console.log(key + ' = ' + value);
}
// 0 = zero
// 1 = one
ktilcu
  • 2,478
  • 1
  • 14
  • 13
  • Can maps have an infinite length? – ktilcu Nov 12 '17 at 21:33
  • 2
    @ktilcu for an iterator: yes. a .map on an iterator can be thought of as a transform on the generator, which returns an iterator itself. popping one element calls the underlying iterator, transforms the element, and returns that. – hraban Nov 14 '17 at 14:46
  • I think the `Array.from` portion guarantees the finite set. I do understand how a generator could provide a lazy and infinite sequence but thats not part of this question. – ktilcu Nov 14 '17 at 14:48
  • 9
    The problem with this answer is it turns what could be an O(1) memory algorithm into an O(n) one, which is quite serious for larger datasets. Aside from, of course, requiring finite, non-streamable iterators. The title of the question is "Using map() on an iterator", I disagree that lazy and infinite sequences are not part of the question. That's precisely how people use iterators. The "map" was only an example ("Say.."). The good thing about this answer is its simplicity, which is very important. – hraban Nov 16 '17 at 15:53
  • 1
    @hraban Thanks for adding to this discussion. I can update the answer to include a few caveats just so future travelers have the info front and center. When it comes down to it we will often have to make the decision between simple and optimal performance. I will usually side with simpler (to debug, maintain, explain) over performance. – ktilcu Nov 16 '17 at 17:22
  • This is so typical for js, a lot of hand waiving to archive simple result – ruX May 28 '18 at 14:13
  • Generator/iterator support in JS is very lacking. In Python it's easy to deal with these things. I use Py to process massive data sets streamed to/from Postgres databases, and it's just as easy as using simple arrays. Different use cases, though. Usually in JS you aren't doing any long-running synchronous tasks like that. But it would be nice to be able to use those annoying iterator interfaces without converting to arrays. – sudo Sep 28 '18 at 03:29
  • 3
    @ktilcu You can instead call `Array.from(m, ([key,value]) => /* whatever */)` (notice the mapping function is inside the `from`) and then no intermediate array is created ([source](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/from#Description)). It still moves from O(1) to O(n), but at least iteration and mapping happen in just one full iteration. – Daniel Apr 15 '19 at 05:38
22

You could define another iterator function to loop over this:

function* generator() {
    for (let i = 0; i < 10; i++) {
        console.log(i);
        yield i;
    }
}

function* mapIterator(iterator, mapping) {
    for (let i of iterator) {
        yield mapping(i);
    }
}

let values = generator();
let mapped = mapIterator(values, (i) => {
    let result = i*2;
    console.log(`x2 = ${result}`);
    return result;
});

console.log('The values will be generated right now.');
console.log(Array.from(mapped).join(','));

Now you might ask: why not just use Array.from instead? Because this will run through the entire iterator, save it to a (temporary) array, iterate it again and then do the mapping. If the list is huge (or even potentially infinite) this will lead to unnecessary memory usage.

Of course, if the list of items is fairly small, using Array.from should be more than sufficient.

sheean
  • 229
  • 2
  • 5
  • How can a finite amount of memory hold an infinite data structure? – shinzou Nov 13 '17 at 18:26
  • 4
    it doesn't, that's the point. Using this you can create "data streams" by chaining an iterator source to a bunch of iterator transforms and finally a consumer sink. E.g. for streaming audio processing, working with huge files, aggregators on databases, etc. – hraban Nov 14 '17 at 14:42
  • 1
    I like this answer. Can anyone recommend a library that offers Array-like methods on iterables? – Joel Malone Aug 24 '18 at 04:20
  • 1
    `mapIterator()` does not guarantee that underlying iterator will be properly closed (`iterator.return()` called) unless the return value's next was called at least once. See: https://repeater.js.org/docs/safety – Jaka Jančar Apr 26 '20 at 20:01
  • Why are you manually using the iterator protocol instead of just a `for .. of .. loop`? – cowlicks Nov 16 '20 at 17:34
  • @cowlicks I have no idea! Maybe there were compatibility issues or I was confused believing it was a TypeScript-only construct. I've updated it now to use a for .. of .. loop instead as this is much clearer. – sheean Nov 20 '20 at 09:12
11

This simplest and most performant way is to use the second argument to Array.from to achieve this:

const map = new Map()
map.set('a', 1)
map.set('b', 2)

Array.from(map, ([key, value]) => `${key}:${value}`)
// ['a:1', 'b:2']

This approach works for any non-infinite iterable. And it avoids having to use a separate call to Array.from(map).map(...) which would iterate through the iterable twice and be worse for performance.

Ian Storm Taylor
  • 7,530
  • 10
  • 45
  • 69
3

Other answers here are... Weird. They seem to be re-implementing parts of the iteration protocol. You can just do this:

function* mapIter(iterable, callback) {
  for (let x of iterable) {
    yield callback(x);
  }
}

and if you want a concrete result just use the spread operator ....

[...iterMap([1, 2, 3], x => x**2)]
cowlicks
  • 698
  • 7
  • 17
2

You could use itiriri that implements array-like methods for iterables:

import { query } from 'itiriri';

let m = new Map();
// set map ...

query(m).filter([k, v] => k < 10).forEach([k, v] => console.log(v));
let arr = query(m.values()).map(v => v * 10).toArray();
  • Nice! This is how JS’s APIs should have been done. As always, Rust gets it right: https://doc.rust-lang.org/std/iter/trait.Iterator.html – flying sheep Jul 23 '20 at 18:31
  • "As always, Rust gets it right" sure... There's a standardization proposal for all sort of helper functions for the iterator interface https://github.com/tc39/proposal-iterator-helpers You can use it today with corejs by importing the `from` fn from "core-js-pure/features/iterator" which returns the "new" iterator. – chpio Dec 16 '20 at 12:54
2

You could retrieve an iterator over the iterable, then return another iterator that calls the mapping callback function on each iterated element.

const map = (iterable, callback) => {
  return {
    [Symbol.iterator]() {
      const iterator = iterable[Symbol.iterator]();
      return {
        next() {
          const r = iterator.next();
          if (r.done)
            return r;
          else {
            return {
              value: callback(r.value),
              done: false,
            };
          }
        }
      }
    }
  }
};

// Arrays are iterable
console.log(...map([0, 1, 2, 3, 4], (num) => 2 * num)); // 0 2 4 6 8
redneb
  • 16,685
  • 4
  • 31
  • 50
MartyO256
  • 39
  • 1
  • 5
2

Take a look at https://www.npmjs.com/package/fluent-iterable

Works with all of iterables (Map, generator function, array) and async iterables.

const map = new Map();
...
console.log(fluent(map).filter(..).map(..));
kataik
  • 391
  • 1
  • 4
  • 17
2

There is a proposal, that is bringing multiple helper functions to Iterator: https://github.com/tc39/proposal-iterator-helpers (rendered)

You can use it today by utilizing core-js:

import { from as iterFrom } from "core-js-pure/features/iterator";

// or if it's working for you:
// import iterFrom from "core-js-pure/features/iterator/from";

let m = new Map();

m.set("13", 37);
m.set("42", 42);

const arr = iterFrom(m.values())
  .map((val) => val * 2)
  .toArray();

// prints "[74, 84]"
console.log(arr);
chpio
  • 565
  • 2
  • 12