Recently there has been considerable interest in software distributed shared memory (SDSM) systems. SDSM is aimed at achieving "the best of both worlds"--the clarity of the shared-memory programming model and the price/performance benefits of running parallel applications on message-passing hardware such as networks of workstations (NOW).
Several SDSM public-domain tools are available, such as Adsmith, CVM and Quarks. However, these are in one sense less suitable for instructional purposes, if the instructor wishes to teach about internal SDSM operation. In other words, if one wishes to explain to the students how this ``miracle'' is accomplished in which we provide a shared-memory programming paradigm on top of non-shared memory hardware, the packages listed above are too complex in their implementation to be enable easy student access to the basic concepts.
With this in mind, TupleDSM is aimed not necessarily at efficient operation but instead at facilitating simple and clear student access to the major implementation issues involved with SDSM systems. The idea is to provide the user with simple calls which request permission to access a shared-data item and which release that permission, largely as in the CRL package, but in a manner more accessible to students.
In TupleDSM the application programmer declares global variables in a C or C++ program and accesses them in the usual shared-memory fashion. The programmer thinks that the values being accessed are stored in those global variables, but in actuality they are stored in tuple space. This is completely transparent to the programmer; at no time is the programmer asked to deal with a tuple, and he/she never even hears the term tuple. The request/release-permission routines which are called by the programmer do internally access the tuples, but again transparently.
TupleDSM application programs are written in C, with calls to TupleDSM library functions. At present, the only data types allowed are scalars and arrays of scalars. (Although the library routines technically declare int types, you should be able to apply it to float types, at least if all the machines are of the same architecture.) Though the programmer works under the shared memory paradigm, pointers do not make sense in the given environment and no mechanism is offered in which to simulate them. (Programmers can instead do "FORTRAN-style" pointers consisting of array indices.)
To prepare a TupleDSM application source file, make sure to insert a line to include DSMInclude.h, and then compile as with this sample Makefile line for our example application Prime.c:
prime: Prime.c
gcc -g -I. -o prime Prime.c DSMCalls.o -L$(POSYBL_LIB) -lposybl
To run the TupleDSM application, first start up POSYBL, by running tmanager in the background on each machine on which either DSMFrontEnd or the application program will run. Then prepare a nodefile, e.g.
chimi 1 #0 ~matloff/Pub/POSYBL/bin/tmanager #2 ~matloff/Pub/POSYBL/examples garnacha 1 #0 ~matloff/Pub/POSYBL/bin/tmanager #2 ~matloff/Pub/POSYBL/examples
The format is first a line with a machine name and a 1; then the full path of tmanager; finally, the directory which contains the application program. (See the POSYBL documentation if you need further details.)
Then invoke the application by
DSMFrontEnd nodefile application_name number_of_workers application_args
Here application_name is the file name, NOT the full path.
For instance, if our nodefile is as above, to run our program prime, we would type
DSMFrontEnd nodefile prime 2 100 10 0This would invoke 2 instances of prime, on the machines chimi and garnacha, and the command line arguments 100, 10 and 0 would be passed on to prime.
The following global variables are available to the TupleDSM application programmer, and are declared in DSMInclude.h:
int DSMNodeNum, /* i.d. number of this node */
DSMTotNodes; /* total number of nodes */
#define DSM_MAX_ARGS 20
#define DSM_MAX_ARG_LENGTH 20
int DSMArgc; /* command-line argument count */
char DSMArgv[DSM_MAX_ARGS][DSM_MAX_ARG_LENGTH];
/* command-line arguments */
Note that the first command-line argument is stored in DSMArgv[1], not DSMArgv[0].
DSMEnvInit()
Must be called by all nodes at the outset of an application program.
DSMVarInit(S,N) char *S; int N;
Must be called, by node 0 only, for each shared variable (scalar or array). The call must be made after the one to DSMEnvInit(). The given variable is initialized to 0.S is a string containing the variable name. N is the number of elements of the array (taken to be 1 for a scalar).
DSMExit()
Required call for all nodes, the last-executed line in the application source code.
DSMReadRequest(S,P,I) char *S; int *P,I;
Request permission for read access to a variable (scalar or array element). Note: There is no corresponding release call.S is a string giving the variable name. P is a pointer to the array or scalar (in the case of an array, use the array name). I is the element index in the array case, 0 in the scalar case.
int DSMSharedReadRequest(S,P,I) char *S; int *P,I;
Request permission for shared read access to a variable; return value is (new) number of readers. The sharing only has significance in that DSMExclWriteRequest() will wait until there are no more readers.S is a string giving the variable name. P is a pointer to the array or scalar (in the case of an array, use the array name). I is the element index in the array case, 0 in the scalar case.
DSMSharedReadRelease(S,P,I) char *S; int *P,I;
The release form of DSMSharedReadRequest(); the calls must come in pairs (if used at all) in order for DSMExclWriteRequest() to work properly.
DSMExclWriteRequest(S,P,I) char *S; int *P,I;
Waits until there are 0 readers on the affected item, after further reads are blocked.S is a string giving the variable name. P is a pointer to the array or scalar (in the case of an array, use the array name). I is the element index in the array case, 0 in the scalar case.
DSMExclWriteRelease(S,P,I) char *S; int *P,I;
Release form of DSMExclWriteRequest(). Writes the new value of the variable into tuple space, and re-allows further reads.S is a string giving the variable name. P is a pointer to the array or scalar (in the case of an array, use the array name). I is the element index in the array case, 0 in the scalar case.
DSMBarrierCreate(BarrierID) char BarrierID[];
Sets up a barrier, identified by the string BarrierID; called only by node 0.
DSMBarrier(BarrierID) char BarrierID[];
Executes a barrier operation, for the barrier identified by the string BarrierID: Adds 1 to a barrier count, and blocks until that count reaches DSMTotNodes. This is a "re-usable" barrier; i.e. the call can be safely used inside a loop.
DSMMutexCreate(MutexID) char MutexID[];
Sets up a mutex, identified by the string MutexID; called only by node 0.
DSMMutexBegin(MutexID) char MutexID[];
Executes a mutex-begin operation, for the mutex identified by the string MutexID. Blocks until the node (if any) currently executing this critical section leaves it, after which no other node can enter this critical section until this node leaves it.
DSMMutexEnd(MutexID) char MutexID[];
Executes a mutex-end operation, freeing the critical section for the mutex identified by the string MutexID.
See my tutorial on parallel debugging.