VIJMark benchmarks the performance of three Java toolkits for working with VMware Infrastructure and vSphere: VI (vSphere) Java API, the vSphere SDK for Java, and the vSphere SDK.

VIJMark is an independent project of Andrew Kutz's at Hyper9 and is in no way associated with VMware.

The Tests

VIJMark uses seven tests to determine which Java toolkit is the fastest.

testConnect Tests how quickly a toolkit can establish a connection to a VI server.
testFindVMUsingSI Tests how quickly a toolkit can find a VM using the SearchIndex.
testFindVMUsingNav Tests how quickly a toolkit can find a VM using the toolkit's navigator (NA for vSphere SDK).
testDumpVMsUsingPC Tests how quickly a toolkit can dump all of a VI server's virtual machines using the PropertyCollector.
testDumpVMsUsingNav Tests how quickly a toolkit can dump all of a VI server's virtual machines using the toolkit's navigator (NA for vSphere SDK).
testDumpAllUsingPC Tests how quickly a toolkit can dump all of a VI server's 3.5 managed entities using the PropertyCollector (Heap error for vSphere SDK).
testDumpAllUsingNav Tests how quickly a toolkit can dump all of a VI server's 3.5 managed entities using the toolkit's navigator (NA for vSphere SDK).

The Results

Initial results showed that VI (vSphere) Java API was so blazingly technically superior that the other two toolkits simply did not belong in the same conversation. Through trial and error I discovered how to make the VI SDK for Java use the PropertyCollector (it's not very straight-forward at all), so the vSphere SDK for Java can be said to be on the right track, but...

VI (vSphere) Java API is the clear winner.

VI (vSphere) Java API

VI (vSphere) Java API is the brain-child of VMware's Steve Jin, and there are a few reasons that VI (vSphere) Java API is the best Java toolkit for working with VI.

  • It routinely wins 7 out of the 7 tests (even though the included results show the vSphere SDK for Java winning testDumpAllUsingPC, VI (vSphere) Java wins that test more than not, although it is always very close).
  • Steve Jin's serialization engine is Lance Armstrong from 1999-2007 and Apache Axis and Axis2 are everyone else. The VI (vSphere) Java API serialization technique is by far the fastest out of the three.
  • VI (vSphere) Java API provides a nice abstraction layer to the VI SDK, but it does not try not hide the VI SDK. This is in contrast to the vSphere SDK for Java which hides the under-pinnings of the VI SDK, renames them, or just in general makes migrating from the VI SDK to the vSphere SDK for Java a total pain in the derriere.
  • The source code is free and open. Even if Steve were to ever abandon the project, you or I can pick it up and take over.

vSphere SDK for Java

The vSphere SDK for Java is the official next-generation product from the VI SDK team for working with VI and vSphere in Java. Based on Axis2 instead of the VI SDK's Axis 1.4/1.5 dependency, its speed is much improved, but it still isn't as intuitive or as fast as VI (vSphere) Java API.

  • The vSphere SDK for Java has a great concept of traversing the inventory, except it does not retrieve an object's properties when using this method (findEntities must be used instead).
  • Dumping the tree with the vSphere SDK for Java's navigator is incredibly slow because a round-trip seems to be occurring for every property retrieved. This presents severe scaling issues.

vSphere SDK

The current and official SDK from VMware for working with VI and vSphere, the vSphere SDK is incredibly slow because of its Axis 1.4/1.5 dependency and should be avoided at all costs.

  • Not only is the vSphere SDK slow, but it has a major performance issue in that it causes a Heap error when attempting to dump all of a VI's inventory (even when setting the JVM's max heap size to 2GB).

Running VIJMark

Running VIJMark is a simple 4 step process:

  • Check out the source code from SourceForge.
    svn co https://vijmark.svn.sourceforge.net/svnroot/vijmark/trunk vijmark
  • Set the appropriate environment variables.
    VIJMARK_SERVER The IP address or FQDN of the VI server to use.
    VIJMARK_USERNAME The user name to use.
    VIJMARK_PASSWORD The password to use.
    VIJMARK_VM_NAME The name of the VM to search for (should be name of VM specified in VIJMARK_VM_FQDN.
    VIJMARK_VM_FQDN The FQDN of the VM to search for (should be FQDN of VM specified in VIJMARK_VM_NAME.
    VIJMARK_PRINT Simply set this variable if you want the dumper tests to print out the dumps to stdout, otherwise the dumps go to the bit bucket.
  • Run the tests (it is import to run the SureFire report first and separately as it will cause the tests to be run and the site MOJO is configured to not run the tests. The order is important as the site MOJO of the parent project is configured to aggregate the test results of the modules).
  • Open the results at target/site/index.html.
    mvn clean surefire-report:report site:site