Questions tagged [benchmarking]

Benchmarking is the process of comparing two or more systems or processes under controlled circumstances in order to have a quantitative measure with which to compare or rank them. The benchmarking tag should be used for questions about how to perform benchmarking tasks or theory questions, not for lists of benchmarking results or requests for benchmarking data; those questions are off-topic for Stack Overflow.

Benchmarking is the process of comparing two or more systems or processes under controlled circumstances in order to have a quantitative measure with which to compare or rank them.

For hardware, benchmarking typically involves either performing a simple task many times or a complex task to ascertain the performance characteristics desired (often speed, but power draw, heat, memory usage, and other characteristics may be of interest as well). Common benchmark operations include FLOPS (Floating Point Operations Per Second), write or read time for a large file, write or read time for many small files, rendering large images, and downloading or uploading large files over a network.

For software, benchmarking typically involves running the different software of interest (different versions of a program, different programs that accomplish a similar task, etc.) on an identical system (either one system, or two identical systems) and performing tasks that take sufficient time to notice a difference. This is often performed on very small differences in code, such as to verify which approach is superior to solving a particular problem.

Benchmarking also includes industry standard benchmarks and common benchmarking suites, used to assist users in making purchasing decisions or otherwise comparing their current systems to other available systems. However, it should be used in this context for issues with building or understanding the code and behavior of these benchmarks, not for general recommendations on the benchmarks suites or on the tested products.

Include additional tags to your question to indicate what type of benchmarking the question is about, such as , or what type of programming language if software, etc.

3156 questions
131
votes
7 answers

How to use clock() in C++

How do I call clock() in C++? For example, I want to test how much time a linear search takes to find a given element in an array.
dato datuashvili
  • 16,915
  • 53
  • 184
  • 303
124
votes
18 answers

How to Calculate Execution Time of a Code Snippet in C++

I have to compute execution time of a C++ code snippet in seconds. It must be working either on Windows or Unix machines. I use code the following code to do this. (import before) clock_t startTime = clock(); // some code here // to compute its…
AhmetB - Google
  • 35,086
  • 32
  • 117
  • 191
118
votes
8 answers

How can I benchmark JavaScript code?

Is there a package that helps me benchmark JavaScript code? I'm not referring to Firebug and such tools. I need to compare 2 different JavaScript functions that I have implemented. I'm very familiar with Perl's Benchmark (Benchmark.pm) module and…
Ionic Walrus
  • 1,455
  • 4
  • 13
  • 10
114
votes
10 answers

Why is Go so slow (compared to Java)?

As we could see from The Computer Language Benchmarks Game in 2010: Go is on average 10x slower than C Go is 3x slower than Java !? How can this be, bearing in mind that Go compiler produces native code for execution? Immature compilers for Go?…
Oleg Razgulyaev
  • 5,187
  • 3
  • 26
  • 28
112
votes
4 answers

Benchmarking (python vs. c++ using BLAS) and (numpy)

I would like to write a program that makes extensive use of BLAS and LAPACK linear algebra functionalities. Since performance is an issue I did some benchmarking and would like know, if the approach I took is legitimate. I have, so to speak, three…
Woltan
  • 12,751
  • 12
  • 70
  • 97
109
votes
4 answers

LINQ Ring: Any() vs Contains() for Huge Collections

Given a huge collection of objects, is there a performance difference between the the following? Collection.Contains: myCollection.Contains(myElement) Enumerable.Any: myCollection.Any(currentElement => currentElement == myElement)
SDReyes
  • 9,166
  • 15
  • 50
  • 91
102
votes
2 answers

Difference between as.POSIXct/as.POSIXlt and strptime for converting character vectors to POSIXct/POSIXlt

I have followed a number of questions here that asks about how to convert character vectors to datetime classes. I often see 2 methods, the strptime and the as.POSIXct/as.POSIXlt methods. I looked at the 2 functions but am unclear what the…
RJ-
  • 2,749
  • 2
  • 24
  • 34
100
votes
4 answers

How efficient can Meteor be while sharing a huge collection among many clients?

Imagine the following case: 1,000 clients are connected to a Meteor page displaying the content of the "Somestuff" collection. "Somestuff" is a collection holding 1,000 items. Someone inserts a new item into the "Somestuff" collection What will…
Flavien Volken
  • 14,820
  • 9
  • 78
  • 105
97
votes
8 answers

Why is splitting a string slower in C++ than Python?

I'm trying to convert some code from Python to C++ in an effort to gain a little bit of speed and sharpen my rusty C++ skills. Yesterday I was shocked when a naive implementation of reading lines from stdin was much faster in Python than C++ (see…
JJC
  • 7,445
  • 6
  • 39
  • 51
97
votes
4 answers

Weird performance increase in simple benchmark

Yesterday I found an article by Christoph Nahr titled ".NET Struct Performance" which benchmarked several languages (C++, C#, Java, JavaScript) for a method which adds two point structs (double tuples). As it turned out, C++ version takes about…
Groo
  • 45,930
  • 15
  • 109
  • 179
97
votes
6 answers

Measure and Benchmark Time for Ruby Methods

How can i measure the time taken by a method and the individual statements in that method in Ruby. If you see the below method i want to measure the total time taken by the method and the time taken for database access and redis access. I do not…
Phani
  • 1,404
  • 2
  • 12
  • 17
91
votes
13 answers

Interpreting a benchmark in C, Clojure, Python, Ruby, Scala and others

Disclaimer I know that artificial benchmarks are evil. They can show results only for very specific narrow situation. I don't assume that one language is better than the other because of the some stupid bench. However I wonder why results is so…
defhlt
  • 1,707
  • 2
  • 17
  • 24
86
votes
4 answers

What is the best way to measure execution time of a function?

Obviously I can do and DateTime.Now.After - DateTime.Now.Before but there must be something more sophisticated. Any tips appreciated.
inspite
  • 27,551
  • 21
  • 73
  • 91
85
votes
10 answers

Is there any simple way to benchmark python script?

Usually I use shell command time. My purpose is to test if data is small, medium, large or very large set, how much time and memory usage will be. Any tools for linux or just python to do this?
noomz
  • 1,587
  • 3
  • 15
  • 20
84
votes
6 answers

Why is looping over range() in Python faster than using a while loop?

The other day I was doing some Python benchmarking and I came across something interesting. Below are two loops that do more or less the same thing. Loop 1 takes about twice as long as loop 2 to execute. Loop 1: int i = 0 while i < 100000000: i…
A. Dorton
  • 1,065
  • 1
  • 9
  • 7