Something odd is happening in demo code I'm creating to illustrate tradeoffs between the Java ArrayList and LinkedList. Problem: I'm observing an increase in free memory after creating an ArrayList of size 100000. What am I missing here?
Observed output:
Total memory: 514850816 bytes Free memory: 511852088 bytes Used memory: 2998728 bytes
ArrayList random access: 8 ms
ArrayList rotation: 740 ms
Total memory: 514850816 bytes Free memory: 512526416 bytes Used memory: 2324400 bytes
Approximate ArrayList memory usage with 100000 4-byte integers: -674328 bytes (!?!?!?)
LinkedList random access: 2345 ms
LinkedList rotation: 5 ms
Total memory: 514850816 bytes Free memory: 507136648 bytes Used memory: 7714168 bytes
Approximate LinkedList memory usage with 100000 4-byte integers: 5389768 bytes
100000
100000
81838
46976
Note that the difference in free memory is negative after increasing the use of memory with the ArrayList. See the output line above with "(!?!?!?)"
This is the code that generated the output:
public static void main(String[] args) throws InterruptedException {
// An ArrayList is like a dynamically resizable array
// - has a capacity that can be beyond size() (trimToSize() for space efficiency)
// - good for random access
// - poor for insertion (must shift all elements to different indices)
// A LinkedList is a doubly-linked list with links forward and backward between nodes
// - poor for random access
// - good for insertion
System.gc(); // force system garbage collection
long totalMemory = Runtime.getRuntime().totalMemory();
long freeMemory = Runtime.getRuntime().freeMemory();
long memoryBefore = totalMemory - freeMemory;
System.out.println("Total memory: " + Runtime.getRuntime().totalMemory() + " bytes \tFree memory: " + Runtime.getRuntime().freeMemory()
+ " bytes \tUsed memory: " + memoryBefore + " bytes");
final int SIZE = 100000;
// ArrayList demo:
ArrayList<Integer> aList = new ArrayList<>();
for (int i = 0; i < SIZE; i++)
aList.add(i);
// Random access
long startTime = System.currentTimeMillis();
for (int i = 0; i < SIZE / 2; i++) {
int value = aList.get((int) (SIZE * Math.random()));
value = value + 1;
}
long endTime = System.currentTimeMillis();
System.out.printf("ArrayList random access: %d ms\n", endTime - startTime);
// Right rotate half of elements, one at a time
startTime = System.currentTimeMillis();
for (int i = 0; i < SIZE / 2; i++) {
int value = aList.remove(aList.size() - 1);
aList.add(0, value);
}
endTime = System.currentTimeMillis();
System.out.printf("ArrayList rotation: %d ms\n", endTime - startTime);
aList.trimToSize(); // ArrayLists are Object[] underneath that are copied to and replaced by arrays doubled in size as needed.
// trimToSize() trims the array to its currently needed size
System.gc(); // force system garbage collection
totalMemory = Runtime.getRuntime().totalMemory();
freeMemory = Runtime.getRuntime().freeMemory();
long memoryAfterArrayList = totalMemory - freeMemory;
System.out.println("Total memory: " + totalMemory + " bytes \tFree memory: " + freeMemory
+ " bytes \tUsed memory: " + memoryAfterArrayList + " bytes");
System.out.println("Approximate ArrayList memory usage with " + SIZE + " 4-byte integers: " + (memoryAfterArrayList - memoryBefore) + " bytes (!?!?!?)");
// LinkedList demo:
LinkedList<Integer> lList = new LinkedList<>();
for (int i = 0; i < SIZE; i++)
lList.add(i);
// Random access
startTime = System.currentTimeMillis();
for (int i = 0; i < SIZE / 2; i++) {
int value = lList.get((int) (SIZE * Math.random()));
value = value + 1;
}
endTime = System.currentTimeMillis();
System.out.printf("LinkedList random access: %d ms\n", endTime - startTime);
// NOTE: This is why you should use an iterator/for-each loop
// for a LinkedList rather than a for loop with .get(index) operations.
// Right rotate half of elements, one at a time
startTime = System.currentTimeMillis();
for (int i = 0; i < SIZE / 2; i++) {
lList.addFirst(lList.removeLast());
}
endTime = System.currentTimeMillis();
System.out.printf("LinkedList rotation: %d ms\n", endTime - startTime);
totalMemory = Runtime.getRuntime().totalMemory();
freeMemory = Runtime.getRuntime().freeMemory();
long memoryAfterLinkedList = totalMemory - freeMemory;
System.out.println("Total memory: " + totalMemory + " bytes \tFree memory: " + freeMemory
+ " bytes \tUsed memory: " + memoryAfterLinkedList + " bytes");
System.out.println("Approximate LinkedList memory usage with " + SIZE + " 4-byte integers: " + (memoryAfterLinkedList - memoryAfterArrayList) + " bytes");
// So both ArrayList and LinkedLists are AbstractLists, but with very
// different implementations and very different performance for different tasks.
// Conclusion: You should know _how_ data structures are implemented (CS 216)
// to make proper application of them.
// To prevent any optimizing compilation that would free aList or lList before this point:
System.out.println(aList.size());
System.out.println(lList.size());
System.out.println(aList.get((int) (SIZE * Math.random())));
System.out.println(lList.get((int) (SIZE * Math.random())));
}
Also, if you have suggestions for a better way to show real-time memory usage of data structures in Java, I'm all ears. Thanks!
Update: Following Jorn Vornee's advice, I ran this with the -Xlog:gc* option from the shell (rather than within Eclipse as above) with Java 1.9 (Java(TM) SE Runtime Environment (build 9.0.1+11)), and got the following output:
[0.019s][info][gc] Using G1
[0.088s][info][gc] GC(0) Pause Full (System.gc()) 1M->0M(8M) 6.102ms
Total memory: 8388608 bytes Free memory: 7623384 bytes Used memory: 765224 bytes
[0.095s][info][gc] GC(1) Pause Young (G1 Evacuation Pause) 2M->2M(8M) 1.963ms
ArrayList random access: 14 ms
[0.861s][info][gc] GC(2) Pause Young (G1 Evacuation Pause) 4M->2M(10M) 2.938ms
ArrayList rotation: 740 ms
[0.882s][info][gc] GC(3) Pause Full (System.gc()) 3M->2M(10M) 9.494ms
Total memory: 10485760 bytes Free memory: 7602584 bytes Used memory:
2883176 bytes
Approximate ArrayList memory usage with 100000 4-byte integers: 2117952 bytes (!?!?!?)
[0.897s][info][gc] GC(4) Pause Young (G1 Evacuation Pause) 4M->4M(10M) 6.709ms
LinkedList random access: 2603 ms
[3.516s][info][gc] GC(5) Pause Initial Mark (G1 Evacuation Pause) 6M->6M(12M) 14.172ms
[3.516s][info][gc] GC(6) Concurrent Cycle
LinkedList rotation: 23 ms
Total memory: 12582912 bytes Free memory: 4411232 bytes Used memory:
8171680 bytes
Approximate LinkedList memory usage with 100000 4-byte integers: 5288504 bytes
100000
100000
91956
67844
[3.536s][info][gc] GC(6) Pause Remark 7M->7M(12M) 2.713ms
[3.538s][info][gc] GC(6) Pause Cleanup 7M->7M(12M) 0.062ms
[3.539s][info][gc] GC(6) Concurrent Cycle 22.275ms
The previous run was with jre1.8.0_151. Any idea why there's such a significant difference in memory behavior between these Java versions? Also, Jorn Vornee, the related post has quite the list of recommendations (Thanks!), but are there particular recommendations you see as most beneficial here?
