py2dx User Guide

For py2dx 2.0 released August 11, 2003

This user guide describes the use of py2dx after it has been installed. See www.psc.edu/general/software/packages/mfix/tools/py2dxInstall.html to install the package. Please send questions and comments to PSC User Services.

Introduction

System Requirements

1. Introduction

OpenDX (www.opendx.org) is a scientific visualization package where the graphics pipeline is constructed by stringing together modules either supplied with OpenDX or added by the user. Any module may have a number of parameters that control the operation of the module.

OpenDX can be run interactively using its own user interface or from another application using the DXLink library of C routines. When using its own user interface, DX consists of 3 processes:

the main script named dx
the user interface (UI) process named dxui
the executive process named dxexec

py2dx is a Python extension that turns most of the routines in the DXLink library into what looks like native Python commands. Once installed, a Python program can start OpenDX, load or define the graphics pipeline, and change the parameters. The Python program can even arrange for OpenDX to display its graphics in a window created and managed by Python.

The DXLink routines can be used to communicate with dxui or dxexec. However, the interface has only been tested with Python programs that communicate with dxexec while dxui is turned off. Other combinations may work but have not been tested.

Most of py2dx is an upgraded version of an extension called Py-OpenDX created by Randall H. Hopper. His contributions to py2dx are gratefully acknowledged.

2. System Requirements

Testing of py2dx has been conducted with the following components:

OpenDX 1.3 (www.opendx.org)
SWIG 1.3.19 (www.swig.org)
gcc 3.2.2
Python 2.2.2 (www.python.org)
Redhat Linux Fedora (www.redhat.com)

You may not need to use SWIG at all. Please see the installation notes.

Time and computer resources do not allow extensive testing with other components. However, I will be happy to help get py2dx working on other systems and to report the results and lessons learned in these notes.

3. DXLink Routines

The following routines are described in the DX documentation "Programmer's Reference", Chapter 17, and are made available to Python by py2dx:

DXLStartDX
DXLCloseConnection
DXLSetSynchronization
DXLConnectToRunningServer
DXLSetMessageDebugging
DXLGetSocket
DXLIsMessagePending
DXLHandlePendingMessages
DXLSetBrokenConnectionCallback
DXLSetErrorHandler
DXLSetMessageHandler
DXLRemoveMessageHandler
DXLSetValueHandler
DXLRemoveValueHandler
DXLSetValue
DXLSetInteger
DXLSetScalar
DXLSetString

DXLLoadVisualProgram
DXLLoadMacroFile
exDXLLoadScript
exDXLBeginMacroDefinition
exDXLEndMacroDefinition
DXLQuery
DXLSync
DXLGetExecutionStatus
DXLExecuteOnce
DXLExecuteOnChange
exDXLExecuteOnceNamed
exDXLExecuteOnChangeNamed
DXLEndExecution
DXLEndExecuteOnChange
DXLSequencerCtl
DXLExitDX
DXLSend

In addition, a new routine, WaitForDXIdle, has been created. It is called to wait for DX to complete a task. However, while waiting, it also displays error and warning messages (and sends them to callback routines, if any). This solves the problem where the user does not see the errors produced by a task while the task is in progress.

By default, DXL errors are turned into the Python exception DXError with data being the error message. The Python try command can be used to catch this error.

4. Usage

Two schemes are provided for interacting with DXLink:

call the DXL interface directly
use the DXServer class (which calls the DXL interface)

4.1 DXL Interface

Calling the DXLink routines from Python is similar to calling them from C. You can use the DXLink documentation for the C routines with a few exceptions:

DXL callbacks are written in Python instead of C.
You don't need to check return values for errors. (errors are turned into Python DXError exceptions)
A few function signatures have changed.

The connection handle returned by DXLStartDX (an integer) must be used in all subsequent calls to the DXL routines.

Example:

import DX
conn = DX.DXLStartDX( DX.DXEXEC + " -execonly -hwrender opengl", None )
DX.DXLSetMessageDebugging( conn, 1 )
DX.exDXLLoadScript( conn, "/my/path/somenetwork.net" )
DX.DXLExecuteOnce ( conn )
  ...
DX.CloseConnection( conn )

4.2 Server Interface

This approach works a little nicer with Python. You create an instance of the DXServer class and then call its member functions.The class has all the features of the DXL routines plus:

Default error and broken connection behavior prevents application hangs when a DX error occurs.
A slightly simpler interface; the class keeps track of DXL state data.
Can create new objects that inherit the server.

Example:

import DXServer
dx = DXServer.DXServer()
dx.SetDXCommand( DX.DXEXEC )
dx.Open()
dx.LoadScript( "/my/path/somenetwork.net" )
dx.ExecuteOnce()
  ...
dx.Close()

4.3 Starting DX

If you want special options passed to DX or if you called your main dx script something other than "dx", you should set the DXEXEC environment variable. For example:

setenv DXEXEC "dx" (same as the default)
setenv DXEXEC "dx -memory 64"
setenv DXEXEC "opendx"

5. Callbacks

Most users can skip this section.

A "callback" is a Python routine that is called in response to a DX event. In general, you will not know precisely when the event will occur (remember, there are 3 processes involved). Callbacks may deliver errors, warnings, or values that you requested.

Several callbacks are established by default for important events such as errors. Additional callbacks are created when you turn on certain features such as DX debug messages. You may also establish your own callback for any event.

The py2dx interface includes some new C routines just to handle callbacks. These are not merely wrappers around routines from the DXLink library. When py2dx creates a callback, it creates a data structure that includes information on the Python callback. It passes a pointer to this data structure to DXLink and DX when the callback is registered.

A callback first shows up when the DXLink library (in C) calls a special routine (in C) in the py2dx interface. This special routine gets back the pointer to the py2dx callback data structure and uses it to call the desired Python routine.

The Python programmer can blissfully proceed to register Python routines, class member functions, and even lambda expressions as callbacks.

6. Usage of DXLInput

Summary: it is usually better to set a variable with a command like

DXLSend( "myvariable = 7;" )

than by using one of the DXLSet* commands. This rest of this section can be skipped by the majority of users. These notes apply to earlier versions of DX such as 4.1.3 but may not be accurate for later versions.

6.1 The Problem

DXLInputNamed and DXLInput are NOT functionally equivalent in all circumstances in a DX network.

A difference comes up when you're in ExecuteOnChange mode. If you DXLSetInteger a DXLInput symbol for a .net-created DXLInput module, the network does NOT automatically re-execute in response to the change. If you only use DXLSet*, you must force an execution in order for this change to propogate (DXLExecute*Once), or simply wait for the next execution to propagate it for it to take effect. Not good.

However, if you use a DXLInputNamed module instead, setting its symbol to a value via DXLSetInteger will automatically re-execute the network like it's supposed to.

6.2 The Workaround

If you use DXLSend to set a DXLInput to a value, then it will actually work. So for every DXLSet* call, we both call the appropriate DXLSet* call as well as invoke DXLSend to set the value of the script variable with the same name. This covers both cases (i.e. where the module is a DXLInput, and where it's a DXLInputNamed).

Note that every DXLInput actually materializes as a script variable of the same name in the DX network introduced by both DXLSending and DXLSeting the value. In fact, DX is going to set this variable anyway so you might as well. However, if the module is a 'DXLInputNamed', it doesn't materialize as a script variable. So conceivably there might be a conflict by setting a script variable of the same name using DXLSend. The question is whether DX allows both a DXLInput and a DXLInputNamed which both answer to the same name. Probably not since DXLSet can set both; either way, it's perverse to do this. So by default our DXLSet* wrapper calls do both DXLSet and DXLSend for Set* wrapper calls.

If you directly set the output script variable for the 'virtual' DXLInput module (e.g. main_DXLInput_57_out_1 below) via DXLSetInteger, it still won't execute. However, different than before, it won't get picked up when you execute the net later either.

6.3 Notes

DXLSetValue doesn't help you any over DXLSetInteger.

Interesting related note: When you do a DXLSet*, the DX message you get back across the wire explicitly says it's a SetDXLInputNamed event:

 INTERRUPT :  begin /SetDXLInputNamed:0
 INTERRUPT :  end   /SetDXLInputNamed:0

Another interesting note (possibly related): In a .net file, notice that a DXLInputNamed module gets an explicit module definition:

// node DXLInputNamed[3]: x = 4299, y = 424, \
    inputs = 2, label = DXLInputNamed
// input[1]: defaulting = 0, visible = 1, \
    type = 32, value = "InteractorMode"
// input[2]: defaulting = 0, visible = 1, type = 29, \
    value = 0
// page group: Main
main_DXLInputNamed_3_out_1 = 
    DXLInputNamed(
    main_DXLInputNamed_3_in_1,
    main_DXLInputNamed_3_in_2
) [instance: 3, cache: 1];
// 
// node Receiver[277]: x = 392, y = 98, inputs = 1, \
   label = PickIsActive
// page group: IRR_MB

whereas DXLInput does not. It only gets a script variable placeholder to transfer to the output name of a 'virtual' module:

// node DXLInput[57]: x = 4301, y = 394, inputs = 1, \
   label = InteractorMode
// input[1]: defaulting = 0, visible = 1, type = 29, value = 0
// page group: Main
main_DXLInput_57_out_1 = InteractorMode;

7. Testing

These programs were used to develop and test py2dx. Some have system-dependent features and so may not work on all systems.

testServer.py: Test the DXServer class.
00-basic.py: Fire up dxexec and shut it down.
01-run_file_net.py: A really simple display test. Loads a .net file, runs it once, and then quits.
02-run_dynamic_net.py: Similar to 01-run_file_net, but creates the network on-the-fly via DXLSend, rather than loading a canned network from a .net file.
03-tk_win.py: Moving closer to doing something useful. This test program sets up DX to display images in a Python-created Tkinter window.
04-value_hdlr.py: Very simple test of value handlers. When a DXLOutput value changes, DX calls value handlers.
05-connect_hdlr.py: Same thing, but with connection handlers.
06-msg_hdlr.py: Same thing, but with message handlers.
07-hwrender.py: PYTHON-DX COLOR CUBE BENCHMARK. Displays a spinning cube using various rendering mechanisms (opengl, gl, software).
08-userctl.py: Mouse control of a DX window. (Hardware-window interaction only works with a patch to DX).
09-userctl2.py: Mouse control of a DX window.
10-supervise_spin.py: Test of SuperviseWindow/State modules for interaction.
11-supervise_net.py: Similar, but with a network loaded from a .net file rather than one that's dynamically created.
12-userctl3.py: Another user interaction demo in a Tk window.
13-userctl_gui.py: DX-in-a-window test. User selection of software/hardware rendering, interaction method (both software and hardware), resizable GUI, etc. Uses the DXServer interface.
14-value_relay.py: Simple test of chained network executions.