-5

How can I convert IEnumerable<int> to int[] without ToArray()? I need a method or function that will work faster than ToArray(). I have one big int massive state and two small massive left and right using method's Take and Skip. Code:

 int[] left = state.Take(pivot).ToArray();
 int[] right = state.Skip(pivot).ToArray();

pivot - split point.

Caleb Bell
  • 519
  • 6
  • 23
Question
  • 33
  • 3
  • 7
    You will only get it faster, if you give it more information. Do you have more information than a simple `IEnumerable`? – nvoigt Mar 10 '16 at 18:37
  • 2
    [How to create a Minimal, Complete, and Verifiable example](http://stackoverflow.com/help/mcve) so people can use it to reproduce the problem and give much better answers. – Nkosi Mar 10 '16 at 18:45
  • I was able to convert an enum of 10 million ints to an array in ~240 ms using ToArray(). Is that not fast enough? Could there be some other issue causing it to take so long? – wentimo Mar 10 '16 at 18:50
  • What makes you think there is a faster way than `ToArray`? – juharr Mar 10 '16 at 18:50
  • i have one big int array. By method's Take and Skip I divide it into two massive. – Question Mar 10 '16 at 18:50
  • Why the downvotes? The question is clear enough even without an example. It might be the case that *there is no faster way* but then that's an appropriate answer and in itself no reason for downvoting. There is already a perfectly good answer with 3 upvotes! – Anders Forsgren Mar 10 '16 at 18:58
  • @AndersForsgren Not sure to what degree this question is deserving of down votes, but just because an answer is good, doesn't mean the question is. – juharr Mar 10 '16 at 19:01
  • I agree - but just because a question isn't good doesn't mean it deserves downvotes. It might simply not deserve many upvotes. If a question is clear enough that it can have a long, upvoted answer within minutes it's definitely good enough to be a 0 score... – Anders Forsgren Mar 10 '16 at 19:03
  • 3
    @AndersForsgren `but just because a question isn't good doesn't mean it deserves downvotes.` Actually, that's *exactly* what it means. Questions need to be more than just "clear" to be appropriate, quality questions. Questions do need to be clear, but they don't *just* need to be clear. Among other things, they also need to be well researched, reasonably scoped, on topic, and useful. – Servy Mar 10 '16 at 19:03
  • 1
    "i have one big int array. By method's Take and Skip I divide it into two massive". Put that in the question, and it'll be much improved. – Jon Hanna Mar 10 '16 at 19:09
  • @Servy I think it's very clear, very useful, a bit under-researched, and very well scoped. I don't think it's under-researched enough to be classified as "bad" in any way. What upsets me is the (apparent) behavior of drive-by hammering the downvote button for any reasonably poor question posted by newcomers. There should be at least as many constructive comments, edits etc. as there are downvotes, especially for newcomers. – Anders Forsgren Mar 10 '16 at 19:14
  • @AndersForsgren You consider it useful to ask how to implement a method that already does exactly what you want, extremely effectively, with no other information than, "do it better"? I can't possibly see how that's a useful endeavor. There is no actual problem to be solved; the question has its own working solution. And again, that wasn't an exhaustive list, the point is simply that there are *lots* of factors that go into making a good question, and for a question to be good it needs to meet *all* of those requirements, not just *one*. – Servy Mar 10 '16 at 19:17
  • @Jon Hanna, I corrected question. Now batter? – Question Mar 10 '16 at 19:23
  • @Servy Sure the question could be better -- especially phrased better "Is it possible to". There are basically a short and clear answer ("You either cast from the int array it is underneath or else you are stuck with ToArray()"), but I think that Q and A together might be useful. The main point was however that the idiocy of ninja-downvoting newbie questions is a terrible (and quite recent!) addition to this community. It makes it much less welcoming than it used to be. Downvotes are OK but for newcomers they should always be coupled with helpful comments about asking. – Anders Forsgren Mar 10 '16 at 19:23
  • @AndersForsgren What gives you the impression that the question is being downvoted because the author is new? I don't even see evidence that the author is new, let alone that that has anything to do with why people downvoted it. I think the real problem is people always assuming anyone that ever downvotes a question is being malicious, rather than that they have a serious concern with the quality of the question, even if you disagree with that evaluation. – Servy Mar 10 '16 at 19:30
  • @Question definitely. See the last edit to my answer. This should indeed be considerably faster on large arrays than with most implementations of Linq's `Skip()`, `Take()` and `ToArray()`, though only slightly faster than the latest in corefx. – Jon Hanna Mar 10 '16 at 19:32
  • The real point of course is that Skip/Take/Linq isn't the most practical or fastest way to split a source array. Fastest is to not split it at all, and just keep track of 2 ranges. – Henk Holterman Mar 10 '16 at 19:39
  • @Jon Hanna thank you for your answer and your attention to my question! It was wery useful and helped me slove my problem:) – Question Mar 10 '16 at 19:50
  • @Servy "What gives you the impression that the question is being downvoted because the author is new?" I don't but (It's my opinion that) downvotes should be accompanied by, or preferably preceded by, suggestions for improvement when the user is new (or at least inexperienced - most easily recognized by low rep). – Anders Forsgren Mar 10 '16 at 20:07
  • @AndersForsgren And that opinion is very much opposed to the site's guidelines, so your expectation simply isn't warranted. People are under no obligation, or expectation, to comment when downvoting, and it is incorrect to assume that an anonymous downvote is somehow malicious or unwarranted. – Servy Mar 10 '16 at 20:08
  • I didn't suggest it's (or should be) obligation, or that it is unwarranted. But it's both part of making it a more welcoming community (it used to be better!) and more importantly, downvoting before askers have time to respond to suggestions just means that after a few quick edits we *may* have a much better question than the one that received massive downvotes. At that point there is no correlation between the (now decent) question quality and the very low score any more. The goal is to have good questions with good scores and bad questions with bad scores *in the end*. – Anders Forsgren Mar 10 '16 at 20:20

2 Answers2

7

There are four possible ways to get an int[] from an IEnumerable<int>. In order of speed:

  1. It already is an int[], cast it back.
  2. It is a collection type with its own Count and CopyTo, use that.
  3. It is a type for which you can find the size, allocate an array of that size, and fill it by iterating through the source.
  4. Allocate an array, fill it but if you reach the end then re-allocate a new array and copy over. Keep doing this until you are done, then trim the array if necessary.

Linq's ToArray() will not do the first, as ToArray() is meant to protect the souce from changes to the result of ToArray().

Of the rest, Linq tries to pick the fastest option. As such, you aren't going to get much of an improvement.

If casting to an array would be acceptable to you then you will be able to cover that case with:

int[] array = intEnum as int[] ?? intEnum.ToArray();

(If you know intEnum is always an array, then you can just do the cast directly. Conversely if you know intEnum is never an array then the above is just slower, as it always has the cost of attempting without ever having the benefit of achieving that approach).

Otherwise while you aren't going to be able to do significantly better, except in the case where you know the size of the enumerable, but it is not an ICollection<int> so linq can't find it itself.

Edit:

A comment added to the question:

i have one big int array. By method's Take and Skip I divide it into two massive.

If you're using .NET Core then this case is already optimised for as of a commit merged into the blessed repository last month, so if you use the latest build of corefx or wait for an update then you've already got this optimised.

Otherwise you can apply the same logic as that version of Linq does:

Let's call our source array sourceArray.

We have an IEnumerable<int> from something like var en = sourceArray.Skip(amountSkipped).Take(amountTaken);

If amountSkipped or amountTaken are less than zero, then zero should be used here. If the Skip() was left out then zero should be used for amountSkipped. If the Take() was left out then int.MaxValue should be used for amountTaken.

The size of the output array will be:

int count = Math.Min(sourceArray.Length - amountSkipped, amountTaken);

We can then create and allocate to an array:

int[] array = new int[count];
int index = 0;
foreach (int item in en)
  array[index] = item;

Now array will be filled without the need to allocate and trim, saving roughly log count allocations. This will indeed be faster.

Again though, if you're using the lastest corefx then this is already done for you, with a further optimisation of being able to avoid the enumerator allocation.

Still, if all that was done was to Skip and Take then you could just drop that entirely and operate directly:

int count = Math.Min(sourceArray.Length - amountSkipped, amountTaken);
int[] array = new int[count];
Array.Copy(sourceArray, amountSkipped, array, 0, count);

No need to Skip() and Take() at all.

Edit again:

The question now has:

int[] left = state.Take(pivot).ToArray();
int[] right = state.Skip(pivot).ToArray();

We can make this simply:

int leftCount = Math.Min(state.Length, Math.Max(pivot, 0));
int[] left = new int[leftCount];
Array.Copy(state, 0, left, 0, leftCount);

int rightCount = Math.Min(state.Length - leftCount);
int[] right = new int[rightCount];
Array.Copy(state, leftCount, right, 0, rightCount);

And this will indeed be a considerable improvement.

If we know pivot is between 0 and state.Length, then we can use the simpler:

int[] left = new int[pivot];
Array.Copy(state, 0, left, 0, pivot);

int[] right = new int[state.Length - pivot];
Array.Copy(state, pivot, right, 0, state.Length - pivot);
Jon Hanna
  • 102,999
  • 9
  • 134
  • 232
  • Aren't 2 and 3 basically the same? Or at least `ToArray` doesn't handle the 3 case specifically. It's either a `ICollection` and it does 2 otherwise it does 4. – juharr Mar 10 '16 at 18:56
  • @juharr in 3 I mean with iterating through, rather than with a `CopyTo`. `ToArray()` always handled 2 and 4, and handles 3 in the current .NET Core version, including cases like `Select()` on an IList (it can find the size by doing `Count` on the source list, but has to implement the `CopyTo` itself). – Jon Hanna Mar 10 '16 at 19:06
1

It greatly depends on the actual type behind the IEnumerable. If it is already an array you can cast it. If it is anything else there usually isn't a way to do it besides ToArray(). You might want to consider why it is slow. Maybe there is a LINQ query underneath that does a database call which could be optimized

mnwsmit
  • 1,078
  • 2
  • 14
  • 29