Speeding up transform calculations

Question

I am programming an OpenGL3 2D Engine. Currently, I am trying to solve a bottleneck. Please hence the following output of the AMD Profiler: http://h7.abload.de/img/profilerausa.png

The data was made using several thousand sprites.

However, at 50.000 sprites the testapp is already unusable at 5 fps.

This shows, that my bottleneck is the transform function I use. That is the corresponding function: http://code.google.com/p/nightlight2d/source/browse/NightLightDLL/NLBoundingBox.cpp#130

void NLBoundingBox::applyTransform(NLVertexData* vertices) 
{
    if ( needsTransform() )
    {
            // Apply Matrix
            for ( int i=0; i<6; i++ )
            {
                glm::vec4 transformed = m_rotation * m_translation * glm::vec4(vertices[i].x, vertices[i].y, 0, 1.0f);
                vertices[i].x = transformed.x;
                vertices[i].y = transformed.y;
            }
            m_translation = glm::mat4(1);
            m_rotation    = glm::mat4(1);
            m_needsTransform = false;
    }
}

I can't do that in the shader, because I am batching all sprites at once. That means, I have to use the CPU to calculate transforms.

My Question is: What is the best way to solve this bottleneck?

I don't use any threads atm, so when I use vsync, I get an extra performance hit too, because it waits for the screen to finish. That tells me I should use threading.

The other way to go would be to use OpenCL maybe? I want to avoid CUDA, because as far as I know it only runs on NVIDIA cards. Is that right?

post scriptum:

You can download a demo here, if you like:

http://www63.zippyshare.com/v/45025690/file.html

Please note, that this requires VC++2008 installed, because it is a debug version for running a profiler.

Answer 1

The first thing I would do is concatenate your rotation and transform matricies into one matrix before you enter the for-loop ... that way you aren't calculating two matrix multiplications and a vector on every for-loop; instead you would only be multiplying a single vector and matrix. Secondly, you may want to look into unrolling your loop and then compiling with a higher optimization level (on g++ I would use at least -O2, but I'm not familiar with MSVC, so you'll have to translate that optimization level yourself). That would avoid any overhead that branches in the code might incur, especially on cache-flushes. Lastly, if you haven't already looked into it, check into doing some SSE optimizations since you're dealing with vectors.

UPDATE: I'm going to add one last idea that would involve threading ... basically pipeline your vertices when you do your threading. So for instance, let's say you have a machine with eight available CPU threads (i.e., quad-core with hyper-threading). Setup six threads for the vertex pipeline processing, and use non-locking single-consumer/producer queues to pass messages between stages of the pipeline. Each stage will transform a single member of your six-member vertex-array. I'm guessing there are a bunch of these six-member vertex arrays, so setup in a stream that is passed through the pipeline, you can very efficiently process the stream, and avoid the use of mutexes and other locking semaphores, etc. For more info on a fast non-locking single-producer/consumer queue, see my answer here.

UPDATE 2: You only have a dual-core processor ... so dump the pipeline idea since it's going to run into bottlenecks as each thread contends for CPU resources.

Answer 2

I can't do that in the shader, because I am batching all sprites at once. That means, I have to use the CPU to calculate transforms.

That sounds suspiciously like a premature optimization you made, under the assumption that batching is the most important thing you can do, and you therefore structured your renderer around making the fewest number of draw calls. And now it's coming back to bite you.

What you need to do is not have fewer batches. You need to have the right number of batches. You know you've gone too far with batching when you forgo GPU vertex transforms in favor of CPU transforms.

As Datenwolf suggested, you need to get some instancing happening to get the transformation back on the GPU. But even then, you need to undo some of the over-batching you've got here. You haven't spoken much about what kind of scene you're rendering (tilemaps with sprites on top, a large particle system, etc), so it's hard to know what to suggest.

Also, GLM is a fine math library, but it is not designed for maximum performance. It generally isn't what I would use if I needed to transform 300,000 vertices on the CPU every frame.

Answer 3

The assignment inside the loop could be a problem, I'm not familiar with the library though. Moving it outside the for loop, and doing the field assignments manually might help. Moving the transformations outside the loop would help as well.

Edit:

This is more along the lines of what I was thinking.

// Apply Matrix
glm::vec4 transformed;
glm::mat4 translation = m_rotation * m_translation;
for ( int i=0; i<6; i++ )
{
    transformed.x = vertices[i].x;
    transformed.y = vertices[i].y;
    transformed.z = vertices[i].z;
    transformed.w = 1.f; // ?
    /* I can't find docs, but assume they have an in-place multiply
    transformed.mult(translation);
    // */
    vertices[i].x = transformed.x;
    vertices[i].y = transformed.y;
}

Maybe, just maybe, the assignment is keeping the compiler from inlining or unrolling something. I kind of guess that the multiply is hefty enough to bump this out of the instruction cache though. And really, if you start talking about the sizes of caches, you aren't going to be resilient across many platforms.

You could try to duplicate some stack and make more, smaller loops.

glm::vec4 transformed[6];
for (size_t i = 0; i < 6; i++) {
    transformed[i].x = vertices[i].x;
    transformed[i].y = vertices[i].y;
    transformed[i].z = vertices[i].z;
    transformed.w = 1.f; // ?
}
glm::mat4 translation = m_rotation * m_translation;
for (size_t i = 0; i < 6; i++) {
    /* I can't find docs, but assume they have an in-place multiply
    transformed.mult(translation);
    // */
}
for (size_t i = 0; i < 6; i++) {
    vertices[i].x = transformed[i].x;
    vertices[i].y = transformed[i].y;
}

As Jason mentioned, unrolling these loops manually could be interesting.

I really don't think that you'll see an order of magnitude improvement on any of these changes, though.

I suspect that calling this function less is more important than making this function faster. The fact that you have this needsTransform check inside of this function makes me think that this is probably relevant.

When you have high level concerns like this in your low level code, you end up just blindly calling this method over and over thinking that it is free. Whether or not that your assumptions about how often needsTransform is true could be wildly incorrect.

The reality is that you should be just be calling this method once. You should applyTransform, when you want to applyTransform. You shouldn't call applyTransform when you might want to applyTransform. Interfaces should be a contract, treat them as such.

Answer 4

If you insist on doing your calculations on the CPU, you should do the math yourself.

Right now, you're using 4x4 matrices in a 2D environment, where one 2x2 matrix for rotation and a simple vector for translation should suffice. That's 4 multiplications and 4 additions for rotation, as well as two additions for translation.

If you absolutely need two matrices (because you need to combine translation and rotation), it'll still be a lot less than what you have now. But you can also "manually" combine these two by moving the position of the vector, rotating, and then moving it back again, which maybe might be a little bit faster than the multiplications, although I'm not sure about that.

Compared to the operations that those 4x4 matrices do right now, that's a lot less.