Installing VoiceObjects on BEA WebLogic 9.2 / 10

Introduction

This document contains a guide for the installation and deployment of VoiceObjects Server and/or Desktop with BEA WebLogic Server 9.2 and 10. As both versions are similar in usage, the document will refer only to WebLogic or WLS instead of "BEA WebLogic Server 9.2 or 10". If necessary differences are pointed out per version.

Installing VoiceObjects with BEA WebLogic is different from standard installation procedures, as WAR files produced by the VoiceObjects installer have to be deployed in existing BEA WebLogic environment.

    Please note, that on Sun Solaris installations JDK 1.5 is mandatory. Therefore it is needed to use WebLogic 9 or higher to run VoiceObjects.

This installation guide assumes that

• you have created VoiceObjects WAR files using the VoiceObjects installer.
• a WebLogic Environment is in place, meaning that a WebLogic Administration Server is running and you can access the WebLogic Administration Console. Please consult the BEA WebLogic documentation for instructions on how to get to this point.
• there is a BEA WebLogic administrator available to help you with questions concerning BEA WebLogic.

Preliminary Work

During the installation process you may already want to change WebLogic's security policy in order to enable VoiceObjects to access non standard ports above 1024. In order to achieve this, edit

[BEA Home]/weblogic92/server/lib/weblogic.policy

and add the following line to the grant section

// default permissions granted to all domains

(usually at the very end of the file)

permission java.net.SocketPermission "localhost:1024-", "listen";

Installation

Installing VoiceObjects on BEA WebLogic consequently consists of the following steps:

• Create a logical WebLogic server for each server and desktop instance
• Configure and start server
• Prepare WAR files for deployment
• Deploy VoiceObjects Server / Desktop

All steps are identical for both deployment of VoiceObjects Server or VoiceObjects Desktop.

Create a logical WebLogic server for each server and desktop instance

Each instance (Server or Desktop) requires its own logical server running exclusively one application (VoiceObjects Server or Desktop)

• Log on to WebLogic Admin Console.
• In the Domain Structure, select Environment/Servers
• Lock the configuration by clicking Lock & Edit in the Change Center.

• Click New, and enter
  • Server Name (e.g. VOServer)
  • Server Listen Address (hostname of computer where the instance is physically located)
  • Server Listen Port (must be the same as configured in VODesktop_Configuration.xml or VOServer_Configuration.xml)
  • Set to stand-alone server
• Click Next and confirm settings on next page.
• There, click Finish and finally activate changes (green button) in Change Center.
• Repeat this step and the following step for the VoiceObjects Desktop instance and all other instances to be created.

Configure and start server

• Click Environment/Servers in Domain Structure
• In the server list, click on the server you just created.
• In the appearing configuration panel, select Configuration -> Server Start.
• Lock the configuration by clicking Lock & Edit in the Change Center.

• Add JVM Memory settings in the Arguments line, e.g. -Xms1024m -Xmx1024m -XX:MaxPermSize=128m

  It is especially important to set the MaxPermSize argument as displayed!

• Click Save in the Configuration Panel and Activate Changes in Change Center
• Click Environment/Servers in Domain Structure
• In the server list, click on the server you just created.
• Select Control in the Server Configuration Panel.
• Click Start to start the server and confirm on following page.
• The state of the server in the control panel will change to STARTING, and after a few seconds of TASK IN PROGRESS, refresh the browser page and the state will have switched to RUNNING - TASK COMPLETED.

Prepare WAR files for deployment

VoiceObjects Server or Desktop cannot be run directly from a WAR file, it needs to be unpacked. As opposed to previous versions, WebLogic will not automatically unpack WAR files on deployment, even if the "stage" application option is selected. "stage" here only means that the WAR file will automatically be copied to all participating machines.

However WebLogic allows to select a directory instead of a WAR file as a source of deployment. In this case, the directory will be copied to the participating machines, which has essentially the same effect as automatically unpacking the WAR files.

So in preparation for deployment on WebLogic, WAR files need to be unpacked to a clean directory. VoiceObjects will be started directly from this directory later on. This avoids having issues with missing logfiles on a failed VoiceObjects start (WebLogic will automatically delete the staging directory of a deployment when the deployment fails and thus needed information will be missing). Additionally some other goodies of VoiceObjects like changing the log-level of some loggers without restart can be used when running VoiceObjects directly from this folder.

Deploy VoiceObjects Server (or Desktop)

• Select Deployment in the Domain Structure.
• Click Lock & Edit in the Change Center.

• Click "Install" to create new deployment.
• On the following screen, select the directory to which you unpacked the WAR file
• Click Next...
• Select "Install this deployment as an application" on the following page.

• Click Next
• On the following page select the logical WebLogic Server you created for this instance.

• Click Next
• On the following page
• Give this instance a name (preferably the same one you specified in VOServer_Configuration.xml or VODesktop_Configuration.xml)
• Select DD Only: Use only roles and policies that are defined in the deployment descriptors. as security option
• Select I will make the deployment accessible from the following location.
  Usually the correct location (the chosen unpacked WAR file) is already in the Location box. Otherwise please paste the correct path here.

• Click Next, confirm changes on following screen and click finish.
• Activate Changes in Change Center.
• Click on Deployments in Domain Structure,
• Click Lock & Edit in the Change Center.
• Tick the deployment you just created and select Start -> Servicing all requests.
• Activate Changes in Change Center.

• The Web Application will soon become "Active" and you are done!

Log Files

Log files will be written in the normal VoiceObjects log folder ([VoiceObjects WAR dir]/WEB-INF/logs).
Additional logging can usually be found in the BEA server logs ([BEA User Projects]/domains/[VoiceObjects Domain]/servers/[server]).

Common pitfalls

Startup of an instance fails

When the startup of an instance of VoiceObjects fails early in the startup-sequence, the exceptions.dmp contains the needed information to solve this problem. In case you had chosen to create a server with staging mode ("Copy this application onto every target for me"), this information is not available as WebLogic cleans out the staging directory after the startup failure. In this case it is needed to deploy VoiceObjects directly from the directory where the war was unpacked to. The startup of this deployment will fail once again, but know you will be able to read the messages from the exceptions.dmp - and in the additionally created VoiceObjects log files in the /WEB-INF/log subfolder of the exploded WAR file.

Startup of an instance fails II

It is possible to specify a Configuration URL at the Service Objects. This URL will be called on startup/redeploy of the service, the XML will be interpreted, initializing variables and collections as specified within. When this file is now located in the Resources folder of VoiceObjects the following happens: When you start the first instance of VoiceObjects Server in WebLogic, it will load all the services, including the specified configuration page. WebLogic however does not open the listener port until at least one application is started completely. Since VoiceObjects is the only app on the WebLogic server, WebLogic won't open the listener port until VoiceObjects has started completely BUT the configuration XML page is requested during startup. So the request for it during startup will fail. Doing a reset on the server or redeploying all services affected helps, because at that time the configuration XML is reloaded and then the listener port is of course open.

The solution will most probably be to host the Configuration XML somewhere else, either in a separate WebLogic environment or in a separate, additional Web Application on the WebLogic servers that will start (due to small size compared to VoiceObjects) almost immediately (and hence before VoiceObjects).

Desktop stops to respond - BEA Error 002617

If VoiceObjects Desktop stops to respond from time to time, please check in <Server Name>.log, if the following sequence of errors can be found:

####<...> <DynamicListenThread[Default]> <<WLS Kernel>> <1173869070128> <BEA-002616> <Failed to listen on channel "Default" on 10.3.100.99:11000, failure count: 1, failing for 0 seconds, java.net.SocketException: Invalid argument>

####<...> <DynamicListenThread[Default]> <<WLS Kernel>> <1173869071149> <BEA-002616> <Failed to listen on channel "Default" on 10.3.100.99:11000, failure count: 2, failing for 1 seconds, java.net.SocketException: Invalid argument>

####<...> <DynamicListenThread[Default]> <<WLS Kernel>> <1173869073168> <BEA-002616> <Failed to listen on channel "Default" on 10.3.100.99:11000, failure count: 3, failing for 3 seconds, java.net.SocketException: Invalid argument>

####<...> <DynamicListenThread[Default]> <<WLS Kernel>> <1173869076187> <BEA-002616> <Failed to listen on channel "Default" on 10.3.100.99:11000, failure count: 4, failing for 6 seconds, java.net.SocketException: Invalid argument>

[?]

####<...> <DynamicListenThread[Default]> <<WLS Kernel>> <1173869115299> <BEA-002617> <Attempting to close and reopen the server socket on on channel "Default" on 10.3.100.99:11000.>

####<...> <DynamicListenThread[Default]> <<WLS Kernel>> <1173869125315> <BEA-002614> <Reopening channel "Default" on 10.3.100.99:11000.>

####<...> <DynamicListenThread[Default]> <<WLS Kernel>> <1173869142401> <BEA-002615> <After having failed to listen, the server is now listening on channel "Default" on 10.3.100.99:11000.>

####<...> <DynamicListenThread[Default]> <<WLS Kernel>> <1173869142404> <BEA-002623> <Connection rejected, the server is in the RUNNING state. Starting service weblogic.application.internal.BackgroundDeploymentService$WaitForBackgroundCompletion, 110 of 112.Last service weblogic.deploy.service.internal.adminserver.HeartbeatService took 1ms.>

This points to a known issue from BEA WebLogic. It is handled with the patch CR283953. After installing this patch, the Desktop should be working as expected.

Excerpt of the e-mail from BEA customer support:
The issue is a known one and has been fixed in CR283953. DIAGNOSIS: Solaris 9 onwards and with jdk 1.5, any set operation on a half-closed socket throws SocketException. So a client like a Cisco content switch, that pings the server and resets the socket just after it gets connected, can cause a SocketException to be thrown on the server. This delays the server to accept new connections on the same port and also causes the server socket to be recycled if the max backoff time between failures reaches its max.