-1

I want to track the boot time and time for certain operations of my app on android. I am using the code markers approach to measure time for all the basic operations in my app. I have inserted some code markers at various places in code. I record the system time whenever a code marker is reached. And then I take the time difference between the start and end markers to measure boot time and other operations time.

I get some variance in the milliseconds time every time I measure the time for a scenario. For example if I measure boot time in 10 iterations I get different values in the range 1400 ms to 1600 ms everytime.

Due to this variance in readings, I am not always 100% sure whether my change is impacting the boot time and other operations time.

As far as I know, variance can be due to the following reasons :

  1. CPU frequency scaling by Linux kernel. So I have disabled the scaling and set the scaling_governor to performance.
  2. Resources usage due to other processes executing on the device. I have eliminated this also. I use the stock android OS, without any external apps. And before running the tests I first reboot my device. So everytime I run my test, same set of applications are running on my device.
  3. Different battery levels. I always run my tests when my phone is 100% charged and is plugged in.
  4. Also I take the measurements on a same device every time. I have a dedicated device for this task.

What can be the other reasons for variance in performance measurement? Is there any better way to measure performance?

Arpit Aggarwal
  • 743
  • 7
  • 17

1 Answers1

0

App performance can vary for several reasons. For example,

  1. Keeping often-requested data in caches by CPU.

  2. File caching by Dalvik/ART runtime.

  3. Since Android runs multiple processes at the same time. Other processes may be executed in between of execution phases of your program.

  4. Garbage collection may halt the execution at any time to analyze the memory.

Those external factors will affect your time measurements, by stealing away CPU time, tying up I/O channels, etc. You need to average your tests across several runs to try to average out those external factors, and accuracy/precision will suffer as a result. A useful discussion for getting accurate time while measure execution is discussed in Best method to measure execution time in Android?.

Apart from those, the performance can also vary on the app side too. For example,

  1. App initializes a lot of objects in the launch process.

  2. App is doing some heavy operations in Activity lifecycle starting methods — onCreate(), onStart() or onResume()

  3. Layout is too rich and/or too complicated to measure and draw it for the first time.

To see how performant are the most common operations like object initialization or Activity lifecycle methods, you can use AndroidDevMetrics

enter image description here

Also check, Using Traceview from Android Studio

enter image description here

Prokash Sarkar
  • 10,714
  • 1
  • 30
  • 47
  • What you are listing can be the reasons why your app is slow. If an app is initializing a lot of objects in launch process, then it will do it every time you run it. How do you account variance in app boot time due to this? My app takes 1400 ms if I run it once, while it takes 1500 ms if I run it again. How explanation do you have for this? – Arpit Aggarwal May 07 '17 at 14:49
  • Are you using Instant Run? – Prokash Sarkar May 07 '17 at 14:53
  • no Instant run. My app already installed on my device. I am just running it multiple times and measuring the time it takes to boot. I am getting some variance in the results. – Arpit Aggarwal May 07 '17 at 14:55