Setting up Distributed Processing
"Calypso DevOps Center Installation"
The Calypso Distributed Processing system allows faster computing of intensive operations by distributing them over any number of processing machines. The Distributed Processing system can be set up to handle any operation that you can decompose into a non-ordered set of jobs.
The Distributed Processing system is made up of the central Calypso Dispatcher and a user-defined number of computing engines, called Calculators.
A user starts a client application, and requests a large task. The client application divides the large task into smaller jobs and sends the jobs to the Dispatcher, which monitors the status of its Calculators and forwards the jobs as Calculators become available. Each Calculator receives one job at a time. If a Calculator fails while attempting to complete a job, the Dispatcher will resend the job to another Calculator. As each Calculator completes each job and sends the results to the Dispatcher, the Dispatcher forwards the results to the client application as a result object. The client application relays the results to the user.
1. Specifying a Dispatcher Config
From the Calypso Navigator, navigate to Configuration > System > Dispatcher to invoke the Dispatcher Config Window:
Creating a Dispatcher Configuration
Step 1 - To create a new configuration, click New. You will be prompted to enter a configuration name.
| » | Enter the configuration name and click OK. |
Step 2 - Select the configuration name from the Name field, and select the dispatcher type from the Type field (Calypso or Symphony).
The new dispatcher configuration is displayed in the Dispatcher Config window.
| » | Enter values for the parameters as required. The parameters are described below. |
| » | You can also click Add Calculator/Parameter to add a parameter or a calculator. |
Step 3 - Click Save to save the configuration.
The Dispatcher Config window will notify you of missing data or bad entries.
Calypso Parameters
Ⓘ [NOTE: Calypso Dispatcher parameters are case sensitive - They need to be added to domain “DispatcherParamsCalypso” if not available for selection]
|
Full Parameter Name |
Description |
|---|---|
|
Dispatcher host |
The host name of the machine that will run the dispatcher. |
|
Dispatcher port |
The port number that the dispatcher will listen on. |
|
Notify on job error |
Set to True to enable notify on Job Error. |
|
Email address to |
When email notification is used, send the email to this user. |
|
Email address from |
When email notification is used, send the email as this user. |
|
Read only |
Make this configuration Read Only. |
| Datagrid Enabled |
Set to true to allow only datagrid enabled calculators to connect to the dispatcher, or false to allow only non datagrid enabled calculators to connect. Ⓘ [NOTE: The Data Grid is currently only supported for Market Risk reports, XVA reports, and the risk reports listed in the Data Grid for Risk Reports guide] |
|
User name |
Present jobs using this user name. |
|
UniqueLogFile |
Set to true to identify which grid the jobs are executed on. |
Calypso Dispatcher Environment Properties
Users can control the number of trades per job using the DIST_SIZE environment property.
DIST_SIZE = 750
Symphony Parameters
For information on the Symphony dispatcher, please consult the Calypso Symphony Integration Guide.
2. Starting the Dispatcher
Follow the steps below to start the dispatcher.
Step 1 - Set the configuration on the Dispatcher.
From the Calypso DevOps Center, choose Applications > Manage.
Edit the Dispatcher application configuration.
Enter the fields as needed and click Save & Continue.
In the Application Arguments area, specify the name of the dispatcher configuration next to the "-config" argument. The default configuration is "default".
Click Save when you are done.
Step 2 - Click 
to deploy the application components if you have made any change.
Step 3 - You can start the Dispatcher from Applications > Manage.
3. Starting the Calculators
Follow the steps below to start the calculators.
Step 1 - Set the configuration on the Calculators.
From the Calypso DevOps Center, choose Applications > Manage.
Edit the Calculator application configuration.
Enter the fields as needed and click Save & Continue.
In the Application Arguments area, specify the name of the dispatcher configuration next to the "-config" argument. The default configuration is "default".
Click Save when you are done.
Step 2 - Click 
to deploy the application components if you have made any change.
Step 3 - You can start the Calculators from Applications > Manage.
The environment property KEEP_ALIVE=true allows maintaining the connections between the dispatcher and the calculators in case of inactivity.
4. Running a Distributed Process
Distributed processing can be applied to risk reports using the Risk Servers or On Demand Analysis. You can select a dispatcher config in all the windows where you can compute risk reports.
Since different product types may have different computation times, a method is provided for assigning different “weightings” by product type. This weighting as well as the number of trades per job specified in the parameter set, are used for splitting each risk report into “jobs” to be dispatched.
For example, if the weighting for Bermudan swaptions is 10 and there are 50 Bermudan swaptions in your portfolio, each swaption counts as 10 trades, for a total of 500 “trades”. If the number of trades per job is 10, it will take 50 jobs to complete the risk analysis. By default, the weighting for European style swaptions and cancellable swaps is 5, for Bermudan style swaptions and cancelable swaps is 10.
These weightings may be customized using the Calypso API.
Refer to the Calypso Developer’s Guide for details.
How are we doing?