I want to track the boot time and time for certain operations of my app on android. I am using the code markers approach to measure time for all the basic operations in my app. I have inserted some code markers at various places in code. I record the system time whenever a code marker is reached. And then I take the time difference between the start and end markers to measure boot time and other operations time.
I get some variance in the milliseconds time every time I measure the time for a scenario. For example if I measure boot time in 10 iterations I get different values in the range 1400 ms to 1600 ms everytime.
Due to this variance in readings, I am not always 100% sure whether my change is impacting the boot time and other operations time.
As far as I know, variance can be due to the following reasons :
- CPU frequency scaling by Linux kernel. So I have disabled the scaling and set the scaling_governor to performance.
- Resources usage due to other processes executing on the device. I have eliminated this also. I use the stock android OS, without any external apps. And before running the tests I first reboot my device. So everytime I run my test, same set of applications are running on my device.
- Different battery levels. I always run my tests when my phone is 100% charged and is plugged in.
- Also I take the measurements on a same device every time. I have a dedicated device for this task.
What can be the other reasons for variance in performance measurement? Is there any better way to measure performance?