6

Wondering why my memory accesses were somewhat slower than I expected, I finally figured out that the Visual C++ implementation of deque indeed has an extra layer of indirection built-in, destroying my memory locality.

i.e. it seems to hold an array of T*, not an array of T.

Is there another implementation I can use with VC++ that doesn't have this "feature", or is there some way (although I consider it unlikely) to be able to avoid it in this implementation?

I'm basically looking for a vector that has also O(1) push/pop at the front.
I guess I could implement it myself, but dealing with allocators and such is a pain and it would take a while to get it right, so I'd rather use something previously written/tested if possible.

user541686
  • 189,354
  • 112
  • 476
  • 821
  • 3
    "i.e. it holds an array of T*, not an array of T." – Billy ONeal Nov 29 '11 at 03:33
  • @BillyONeal: Confused... why is that 'expected'? And isn't every element just a single element? `vector` doesn't do this... – user541686 Nov 29 '11 at 03:34
  • 2
    @Mehrdad: And consequently, vector doesn't have O(1) push/pop at the front. There's a price for that feature. – Benjamin Lindley Nov 29 '11 at 03:35
  • @BenjaminLindley: I'm confused... couldn't it just leave more room at the front, like it does in the back? I remember implementing this myself in C# (no extra layers or anything), and it worked perfectly fine... I don't see why that would require an extra indirection. – user541686 Nov 29 '11 at 03:37
  • 2
    @Mehrdad: And what happens when it runs out of room at the front and push there again? No more O(1) insertions, because it now requires a reallocation, which copies the elements. That's an O(n) operation. Which is *not allowed* for a `deque`. Therefore, the only way to implement a `deque` correctly is to have this extra indirection. Did your C# implementation have `O(1)` pushes all the time? Or did you have to reallocate? – Nicol Bolas Nov 29 '11 at 03:42
  • @BillyONeal: Er, why can't you just pop them from the back like in any container? Same way a `vector` pops from the back... just return the element and decrease the count. – user541686 Nov 29 '11 at 03:44
  • @NicolBolas: Well, by that bar, `vector` isn't O(1) insertion either. IIRC, both structures only require constant amortized time. – Billy ONeal Nov 29 '11 at 03:44
  • @BillyONeal: Oh, really? I didn't know it shifts anything when you call `pop_back`. What's the time complexity? Hint: see [here](http://www.cplusplus.com/reference/stl/vector/pop_back/) – user541686 Nov 29 '11 at 03:46
  • 1
    A deque also doesn't invalidate references on push_front and others which makes it lets say problematic to store in a single block of memory. However I would assume that each block (as in pointer) of the deque stores more then one T, so I doubt that this indirection is really slowing your memory access down (of course from what I read visual c++ is extremely bad at optimization, so who nows how bad the standardlib might be written). Did you try without the indirection and get noticable better results? – Grizzly Nov 29 '11 at 03:46
  • @NicolBolas: Of course I reallocate, but it's O(1) amortized. – user541686 Nov 29 '11 at 03:47
  • @Grizzly: Yeah, it was definitely noticeable. (I was using it pretty heavily.) – user541686 Nov 29 '11 at 03:48
  • 1
    @Mehrdad: Allow me to quote from the C++ standard on `std::deque::push_front`: "Inserting a single element either at the beginning or end of a deque always takes constant time". That's not *amortized* constant time, that's *always* constant time. So what you implemented was not a `deque` as defined by the C++ standard. `deque` is a special-case container, for when you really need to add things to the beginning/end in constant time, *always*. It should not be your default container. – Nicol Bolas Nov 29 '11 at 03:50
  • 1
    @NicolBolas: Well, depending on how picky you get, the memory allocation definitely isn't constant time either. :P But in any case, I never said I *wanted* a deque anyway, right? I said I wanted a vector that could push/pop from the front in (amortized) constant time. I just said I was *originally* using a `deque` since that seemed like the best choice, that's all. – user541686 Nov 29 '11 at 03:52
  • 2
    @Billy: `std::vector::pop_back` is linear time? No it's not. It's just destroying an element and decrementing the size. – Benjamin Lindley Nov 29 '11 at 03:54
  • 1
    @BillyONeal: `vector` doesn't even *have* a pop front. – user541686 Nov 29 '11 at 03:56
  • @Mehrdad: +1, and I am a complete idiot :/. – Billy ONeal Nov 29 '11 at 04:56
  • @NicolBolas "_No more O(1) insertions, because it now requires a reallocation, which copies the elements. That's an O(n) operation. Which is not allowed for a deque._" How is the real deque different in this regard? There is also a realloc, so it is also O(n). – curiousguy Nov 29 '11 at 05:39
  • 1
    "_I said I wanted a vector that could push/pop from the front in (amortized) constant time._" do you need a guarantee that references to elements are not invalided by push/pop? – curiousguy Nov 29 '11 at 05:40
  • "_Therefore_" Wrong. "_the only way to implement a deque correctly is to have this extra indirection._" True. – curiousguy Nov 29 '11 at 05:42
  • @curiousguy: Nothing stronger than the equivalent of what `vector` provides, and in fact, weaker might still work. i.e.: if there is a reallocation, they can be invalidated. If there isn't, then they shouldn't be, although I can live with it if they are anyhow (but I don't see why they would be). In other words, it's not a crucial deal maker or deal breaker. – user541686 Nov 29 '11 at 05:42
  • @Mehrdad With deque you are getting more than you asked for, this might explain why you are unhappy! I have always wondered why there is no standard contiguous (vector-like, with the invariant that `&d[i] == &d[j] + (i-j)`) queue with pop/push front/back with O(1) contribution to whole-program complexity (I don't know "amortized" means). – curiousguy Nov 29 '11 at 05:51
  • @curiousguy: According to [this answer](http://stackoverflow.com/questions/6147618/why-typical-array-list-implementations-arent-double-ended), it's just to avoid the extra math (just an addition?), nothing else. Keeping the iterators valid shouldn't be *nearly* as much of a problem as the requirements `deque` has tried to satisfy. – user541686 Nov 29 '11 at 05:54
  • @NicolBolas "_"Inserting a single element either at the beginning or end of a deque always takes constant time"._" What is "constant time"? – curiousguy Nov 29 '11 at 05:54
  • 1
    @curiousguy: "constant time" means that the time an operation takes does not vary with N, where N is the size in question. In the case of containers, N is the number of elements in the list. ["amortized"](http://en.wikipedia.org/wiki/Amortized_analysis) is more complicated. As to "How is the real deque different in this regard? There is also a realloc, so it is also O(n)." when I said "reallocation", I meant realloc and copy. You're allocating memory and then doing N copies. – Nicol Bolas Nov 29 '11 at 06:04

3 Answers3

11

For whatever reason, at least as of MSVC 2010, the std::deque implementation appears to make use of an unbelievably small block size (the max of 16 bytes or 1 single element if I'm not mistaken!).

This, in my experience, can result in very significant performance issues, because essentially each "block" in the data structure only ends up storing a single element, which leads to all kinds of additional overhead (time and memory).

I don't know why it's done this way. As far as I understand it setting up a deque with such a small block size is exactly how it's not supposed to be done.

Check out the gcc stdlib implementation. From memory they use a much larger block size.

EDIT: In an attempt to address the other issues:

  • std::deque should have an extra layer of indirection. It is often implemented as a "blocked" data structure - i.e. storing an array of "nodes" where each node is itself an array of data elements. It's not ever like a linked-list - the array of nodes is never "traversed" like a list, it's always directly indexed (even in the case of 1 element per block).

  • Of course you can roll your own data structure that keeps some extra space at the front. It wont have worst case O(1) push/pop front/back behaviour, and as such it wont satisfy the requirements of the std::deque container. But if you don't care about any of that...

Darren Engwirda
  • 6,660
  • 3
  • 19
  • 41
  • +1 the first sensible answer. :) Will check it out. (Hopefully it works well with VC++...) – user541686 Nov 29 '11 at 04:01
  • 2
    @Mehrdad: GCC's `deque` still has the extra level of indirection we're speaking of here. (Even though it's probably more efficient with a larger block size) – Billy ONeal Nov 29 '11 at 04:04
  • @BillyONeal: If it's large enough that I don't notice it, then I can ignore its existence. I certainly couldn't in my first case, though, as this was the chief cause. (But yes, you're right -- if I can avoid it altogether, I would still rather go that route.) – user541686 Nov 29 '11 at 04:06
  • "_it wont be a standard conforming container_" why not? – curiousguy Nov 29 '11 at 05:57
  • @curiousguy: yeah, I didn't say that very well - fixed. – Darren Engwirda Nov 29 '11 at 06:07
  • 1
    I have _never_ understood that small block size in MSVC++, and furthermore, don't understand why they didn't at least implement a non-standard extension of an additional constructor or template parameter or factory or something to enable the user to create deque instances with a different chunk size. – davidbak Jan 25 '17 at 00:34
  • 1
    @davidbak: I just realized their block size is very close to the cache line size. That's probably part of an explanation, though still incomplete. – Mooing Duck Aug 08 '20 at 15:28
2

The C++ standard does not allow std::deque to do reallocations on pushes to the front or back. These operations are always constant time. Not amortized, always.

The C++ standard does not have such a container. Boost doesn't have one to my knowledge (thought the Boost.Container library might; I haven't looked into it).

Nicol Bolas
  • 378,677
  • 53
  • 635
  • 829
  • It seems like you interpreted my question to mean that I was looking for a vector-like *deque* (and hence you correctly pointed out that it's not possible), whereas I'm looking for a *vector* that has amortized O(1) push/pop front capability, like I mentioned in the post. – user541686 Nov 29 '11 at 03:58
  • 2
    @Nicol: Most `std::deque` implementations actually don't strictly satisfy the `O(1)` worst case requirement for either `push_front` or `push_back`, due to the way the backing array of "blocks" is re-sized. In (current) MSVC for instance, this backing array is grown like a `std::vector`. Because the length of the backing array should be small compared to the element count of the container this non-`O(1)` behaviour should be hard to see in practice. I guess this means that most `std::deque` implementations are non-conforming? – Darren Engwirda Nov 29 '11 at 04:53
  • 1
    @DarrenEngwirda No, it means the area of complexity requirements of the standard is a mess. – curiousguy Nov 29 '11 at 05:56
  • @curiousguy: How can you complain about the complexity requirements of the standard when you state that you don't know what "constant time" and "amortized" mean? The standard is written for C++ compiler writers, who are expected to know what these CS terms mean. There's nothing "messy" about them. – Nicol Bolas Nov 29 '11 at 06:02
  • @NicolBolas are you able to define those terms? – curiousguy Nov 29 '11 at 06:07
  • @NicolBolas are you saying that the standards complexity specification for deque's are simply unattainable? Or at least that you know of no implementation out there that actually satisfies the specification fully? – dcmm88 Nov 17 '16 at 05:44
0

The indirection you complain about is actually mandated by the standard requirement that references/pointers are never invalidated by push/pop front/back (except those references/pointers to removed elements, obviously).

So you see that this requirement has nothing to do with any complexity requirement.

This indirection also allows for faster (but still O(size)) push front/back when there is no available room.

curiousguy
  • 7,344
  • 2
  • 37
  • 52