VoiceObjects 9 on Sun Solaris

Symptoms

Original Enquiry

VoiceObjects isn't available during redeploys of a service:
Between
INFO 2010-07-30 15:05:42:782 Replacing Service '...' in list of available Services!
and
INFO 2010-07-30 15:12:21:467 Redeployed Service '...' successfully at July 30, 2010 3:12:21 PM CEST (after 6m 38s 965ms).
none of the instances in the cluster responded in reasonable time (<30s).

Detailed Description

Operation of VoiceObjects slows down significantly. Redeploys take very long time and block the whole instance during this time. Instance startup also take a lot of time.
Stopping / starting services also stops service on the whole cluster.
This happens only for bigger services - as we have seen 15.000 objects and up. Smaller services are not affected.

Environment

VoiceObjects >= 9
OS: Sun Solaris

Cause

With VoiceObjects 9.1 a capability to dynamically swap caches out of and back into memory in order to maintain an upper limit on the size of concurrently deployed applications has been introduced. To do this, it was needed to measure the in-memory size of caches through a Java library. This library works fine on e.g. Linux and Windows, but appears to take extremely long when working on Solaris and with very big applications. For moderate-sized ones (couple thousand objects) it works OK, but for very big ones (e.g with 20.000 objects) it becomes unusable.

Resolution

The immediate adjustment is to eliminate the newly introduced library.
Voxeo will further investigate the underlying problem to find out whether it comes from the library, the Solaris JDK, or Solaris itself.
In order to apply the change please do the following:

1. In the start_VOServer.sh' scripts remove the reference -javaagent:"%PLATFORM_WEB-INF%"\extlib\sizeOf.jar
2. Copy the file sizeOf.jar from the WEB-INF/extlib folder to the WEB-INF/lib folder
3. Restart the server instances

After applying this change you should experience a faster server instance start-up as well as faster redeploys without getting blocked.