C++ and Java's Looping on Strings performance comparisons

Question

I have made some performance test against C++ (implemented in Visual Studio Community Edition 2015) and Java (1.7) for loops.
The following are the source code:

Java:

long startTime = 0;
long endTime = 0;
long totalTime = 0;

startTime = System.currentTimeMillis();
for (long counter = 0; counter < numberOfIterations; counter++)
{
    System.out.println("01234");
}
endTime = System.currentTimeMillis();
totalTime = endTime - startTime;

C++ (Windows based, release version x64, optimized for speed):

ULONGLONG startTime = 0;
ULONGLONG endTime = 0;
ULONGLONG elapsedTime = 0;

startTime = GetTickCount64();
for (LONGLONG counter = 0; counter < numberOfIterations; counter++)
{
    cout << "01234" << endl;
}
endTime = GetTickCount64();
elapsedTime = endTime - startTime;

The results really surprised me when I spin/loop them for 100,000 times. Here they are:
Java:

1st attempt: 31,361 milliseconds
2nd attempt: 6,316 milliseconds
3rd attempt: 6,865 milliseconds

C++:

1st attempt: 40,000 milliseconds
2nd attempt: 37,703 milliseconds
3rd attempt: 20,734 milliseconds

Then I had another set of test case

Java:

long startTime = 0;
long endTime = 0;
long totalTime = 0;
startTime = System.currentTimeMillis();
for(long counter = 0; counter < numberOfIterations; counter++) {
    String tempString = new String("test");
}
endTime = System.currentTimeMillis();

C++ (Windows based, release version x64, optimized for speed):

ULONGLONG startTime = 0;
ULONGLONG endTime = 0;
ULONGLONG elapsedTime = 0;

startTime = GetTickCount64();
for (LONGLONG counter = 0; counter < numberOfIterations; counter++)
{
    string tempString = "test";
}
endTime = GetTickCount64();
elapsedTime = endTime - startTime;

Again, the results is really more surprising, when I spin/loop them for 10,000,000 times. Here they are: Java

1st attempt: 7 milliseconds
2nd attempt: 7 milliseconds
3rd attempt: 7 milliseconds

C++:

1st attempt: 125 milliseconds
2nd attempt: 125 milliseconds
3rd attempt: 125 milliseconds

But on empty loops C++.

Before doing this test I was actually thinking that C++ will always out perform Java in low level or OS/Platform specific implementation. But, in this case, does this mean that Java has more efficient way of handling Strings specially if it is already in volumes?

Thanks

Obligatory: [How do I write a correct micro-benchmark in Java?](https://stackoverflow.com/questions/504103/how-do-i-write-a-correct-micro-benchmark-in-java) — Andy Turner, Jul 05 '17 at 11:37
The problem here: your examples are probably **way too** naive to result in meaningful numbers. See https://stackoverflow.com/questions/504103/how-do-i-write-a-correct-micro-benchmark-in-java for example. What I mean is: the Java JIT does a lot of optimizations. It might **quickly** figure that there is no point in creating string objects that just get thrown away. And for the other testcase: you understand that when printing to the console (?) most of the time would be spent in system calls anyway? — GhostCat, Jul 05 '17 at 11:37
Could you please add two lines before the loop in the first c++ example: `ios_base::sync_with_stdio(false);` and `cin.tie(NULL);`, and try it again? — DAle, Jul 05 '17 at 11:42
Try it with random generated strings again, so that the JIT compiler can't optimize things mentioned by other comments. — Andre Kampling, Jul 05 '17 at 11:42
@PeteBecker: No `endl` is right because `System.out.println` also flushes the buffer. To be fair both languages should either flush or not. — Andre Kampling, Jul 05 '17 at 11:43
@GhostCat, yes I was been thinking about JIT optimizations, but just want more votes/supporter on that idea. Thanks a lot for your ideas. It really helps =)! — Artanis Zeratul, Jul 05 '17 at 11:49
@all, I really appreciate what you have inputted, I already have the hunch of JIT optimizations because there was one time I observed that the number of seconds dropped from 30 seconds to 50 milliseconds hahaha... but yeah I just need support group to make my observations solid. Thanks a lot guys and I will try all your suggestions and I will post it as an answer here below. More power to all of you guys! — Artanis Zeratul, Jul 05 '17 at 11:51
@manni66: He wrote that he build a: `release version x64, optimized for speed`. — Andre Kampling, Jul 05 '17 at 12:22
On the first example, I get about 7 ms using `std::cout << "1234\n"`, which makes me believe that `System.out.println()` does not flush the buffer. — Michaël Roy, Jul 05 '17 at 12:38
@MichaëlRoy: `System.out.println()` flushes: https://stackoverflow.com/a/7166357/8051589 — Andre Kampling, Jul 05 '17 at 13:28
Hello @AndreKampling, PeteBecker and GhostCat I made a random number generator that will output the results in the console, implemented in Java, C++ Windows (same project to the one I mentioned above) and C++ on Cygwin (Windows). For 100,000 loops C++ Cygwin was the fastest, Java was next and C++ was the last. But when I changed the console output for the 2 C++ projects (Windows and Cygwin) to printf, C++ Windows project outmatched Java but still C++ Cygwin was still the fastest. Does that mean that Windows implemenation of cout is not really good for performance? — Artanis Zeratul, Jul 06 '17 at 10:47
By the way, I saw this article: https://stackoverflow.com/questions/1736267/c-cout-printing-slowly I think cout is not really that good in MSVC (Windows Development). — Artanis Zeratul, Jul 06 '17 at 10:53
@Artanis: Nice observations. To be fair to Java you could also add flushing to `printf` using `fflush (stdout)` because the `System.out.println` also does it. I would say you should post it as answer and accept it yourself. — Andre Kampling, Jul 06 '17 at 11:05
@AndreKampling, I have made some answers below, as I have promised. But I didn't do output to the console this time. I made the answer as fair as possible. — Artanis Zeratul, Jul 31 '17 at 12:10

Artanis Zeratul · Accepted Answer · 2017-07-31T12:58:15.597

Finally, I was able to make time to post an answer here as I promised. But my apologies for that. Anyways, here are the stats I have gathered. Please bear with me, this is quite a lengthy answer. By the way, both of them were executed in Windows 10 Pro x64 Machine =)!

First code (both C++ and Java): In Java on Windows:

public void testForLoopCreateInt(long numberOfIterations) {
    ArrayList<Integer> listOfIntegers = new ArrayList<Integer>();
    long startTime = 0;
    long endTime = 0;
    long totalTime = 0;
    System.out.println("\n===========================" + "\ntestForLoopCreateInt() Looping for: " + numberOfIterations);
    startTime = System.currentTimeMillis();
    for(long counter = 0; counter < numberOfIterations; counter++) {
        int i = 0;
        listOfIntegers.add(i);
    }
    endTime = System.currentTimeMillis();
    totalTime = endTime - startTime;
    System.out.println("Total time: " + totalTime + " milliseconds");

    System.out.println("===========================testForLoopCreateInt()");

    for (int indexer = 0; indexer < 10; indexer++) {
        int y = listOfIntegers.get(indexer);
    }
}

In C++ over some Win32 API:

void Loops::testForLoopCreateInt(LONGLONG numberOfIterations)
{
    cout << "\n===========================" << "\ntestForLoopCreateInt() Looping for: " << numberOfIterations << endl;
    vector<int> vectorOfInts;

    high_resolution_clock::time_point startTime = high_resolution_clock::now();
    for (LONGLONG counter = 0; counter < numberOfIterations; counter++)
    {
        int i = 0;
        vectorOfInts.push_back(i);
    }
    high_resolution_clock::time_point endTime = high_resolution_clock::now();

    duration<double, std::milli> totalTime = endTime - startTime;
    cout << "Total time: " << totalTime.count() << " milliseconds" << endl;
    cout << "===========================testForLoopCreateInt()" << endl;

    for (int indexer = 0; indexer < 10; indexer++) {
        int y = vectorOfInts.at(indexer);
    }
}

When each of these had number of iterations set to some value the following are the results:

Java:

10 Iterations: 0.00 ms
100 Iterations: 0.00 ms
1000 Iterations: 0.00 ms
10000 Iterations: 1.00 ms
100000 Iterations: 4.00 ms
1000000 Iterations: 12.00 ms
10000000 Iterations: 106.00 ms
100000000 Iterations: 1,747.00 ms

C++:

10 Iterations: 0.001803 ms
100 Iterations: 0.00601 ms
1000 Iterations: 0.013521 ms
10000 Iterations: 0.067005 ms
100000 Iterations: 0.506291 ms
1000000 Iterations: 4.4806 ms
10000000 Iterations: 61.1632 ms
100000000 Iterations: 679.341 ms

Second code (both C++ and Java): In Java on Windows:

public void testForLoopCreateUniformStringAndStoreToArrayList(long numberOfIterations) {
    ArrayList<String> listOfIntegers = new ArrayList<String>();
    long startTime = 0;
    long endTime = 0;
    long totalTime = 0;
    System.out.println("\n===========================" + "\ntestForLoopCreateUniformStringAndStoreToArrayList() Looping for: " + numberOfIterations);
    startTime = System.currentTimeMillis();
    for(long counter = 0; counter < numberOfIterations; counter++) {
        String string = new String("01234");
        listOfIntegers.add(string);
    }
    endTime = System.currentTimeMillis();
    totalTime = endTime - startTime;
    System.out.println("Total time: " + totalTime + " milliseconds");

    System.out.println("===========================testForLoopCreateUniformStringAndStoreToArrayList()");

    for (int indexer = 0; indexer < 10; indexer++) {
        String y = listOfIntegers.get(indexer);
    }
}

In C++ over Some Win32 API:

void Loops::testForLoopCreateUniformStringAndStoreToVector(LONGLONG numberOfIterations)
{
    cout << "\n===========================" << "\ntestForLoopCreateUniformStringAndStoreToVector() Looping for: " << numberOfIterations << endl;
    vector<string> vectorOfStrings;

    high_resolution_clock::time_point startTime = high_resolution_clock::now();
    for (LONGLONG counter = 0; counter < numberOfIterations; counter++)
    {
        string str000("01234");
        vectorOfStrings.push_back(str000);
    }
    high_resolution_clock::time_point endTime = high_resolution_clock::now();

    duration<double, std::milli> totalTime = endTime - startTime;
    cout << "Total time: " << totalTime.count() << " milliseconds" << endl;
    cout << "===========================testForLoopCreateUniformStringAndStoreToVector()" << endl;

    for (int indexer = 0; indexer < 10; indexer++) {
        string y = vectorOfStrings.at(indexer);
    }
}

When each of these had number of iterations set to some value the following are the results:

Java:

10 Iterations: 0.00 ms
100 Iterations: 0.00 ms
1000 Iterations: 1.00 ms
10000 Iterations: 1.00 ms
100000 Iterations: 6.00 ms
1000000 Iterations: 24.00 ms
10000000 Iterations: 2,742.00 ms
100000000 Iterations: 33,371.00 ms

C++:

10 Iterations: 0.003605 ms
100 Iterations: 0.018329 ms
1000 Iterations: 0.064301 ms
10000 Iterations: 0.71722 ms
100000 Iterations: 13.9406 ms
1000000 Iterations: 88.5781 ms
10000000 Iterations: 931.526 ms
100000000 Iterations: 10,768.9 ms

So these are the results, I am not sure if this is biased or not but I am making this as fair as I can to both C++ (on windows) against Java (on windows). So you'd be the judge.

Thanks.

C++ and Java's Looping on Strings performance comparisons

1 Answers1