Message Passing Facility

Release 1-10

Marty Kraimer
April 2003

License Agreement

MPF is available via the open source license described at the end of this document

Contents

Acknowledgments
Requires
Introduction
Hardware Configurations
Install and Build MPF
Multiple Processor VME system
Getting Started
Using MPF in Applications
mpf/bin
MPF Directory Structure
Epics Record to IP - Flow of Control
Message Passing Overview
Diagnostic Aids
Message Routers
Message
MPF Device support
Utility Classes
Industry Pack Support
Performance

Acknowledgments

The following people have contributed to MPF.
Jim Kowalkowski
Jim developed HIDEOS, the precursor of MPF. Many of the basic ideas for MPF come from Hideos. In addition the IP support for the ip330, GpibGsTi9914, OctalUart, and dac128 started with the Hideos support for these modules
Joe Sullivan
Joe developed the vxWorks board support packages that allow VME back plane communication.
Tom Coleman
Tom developed the original 162 Industry Pack Support and GpibGsTi9914 for MPF.
Mohan Ramanathan
Mohan was an early user of MPF and suffered patiently through the initial problems.
Andrew Johnson
Andrew made the changes so that his ipic support could be used with MPF.
Mark Rivers
Mark contributed many ideas that make the serial support general purpose, ,which is described eleswhere. He also developed support for digital and analog support which is also described elsewhere.

Requires

NOTE: MPF does not, itself, provide any support for specific io modules. All such support is maintained elsewhere and must be obtained and build after mpf itself is built.

Introduction

This document describes a set of software which allows EPICS to access Industry Pack devices attached to the same or to remote processors. The following components are described:

Message Passing Facility

MPF provides client server message passing. The messages are handled by message routers. Both local and remote message routers are provided.
Industry Pack Support
Industry Pack modules are supported via drvIpac, which supports dumb VME carriers as well as the mv162 and mv172. For details see drvIpac.
Epics Device Support for the MPF
A base class DevMpf is provided for writing epics device support that communicates with MPF.
All code is written in C++. It is written to minimize dependencies between various components. For example the IP support can be used without MPF, MPF without the IP support, etc. The current implementation uses the standard C library but not the standard C++ library.

Hardware Configurations

The following configurations are supported: The first option has been tested on an mv162 and an mv172 It should work on any IP processor configured to run epics base release 3.13.beta12 or later. The second option has been tested with an mv167 communicating with a mv162, and with an mv172 communicating with an mv162. The third option is effectively identical to option one, but is not restricted to the mv162 and mv172 CPU boards.

When MPF is used between processors, communication is via a tcp connection. Thus the two processors can reside anywhere as long as it is possible for them to communicate via tcp.

Install and Build MPF

Installation and Building

After obtaining a copy of the distribution, it must be installed and built for use at your site. These steps only need to be performed once for the site (unless versions of the module running under different releases of EPICS and/or the other required modules are needed).
  1. Create an installation directory for the module, usually this will end with
  2. .../support/mpf/
  3. Place the distribution file in this directory. Then issue the commands (Unix style)
  4. gunzip <file>.tar.gz
    tar xvf <file>.tar
  5. Beginning with release R1-5 this creates a <top> application.
  6. .../support/mpf/mpfX-Y
    where X-Y is the release. For example.
    .../support/mpf/mpf1-5
  7. Edit the config/RELEASE file and set the paths to EPICS base and ipac.
  8. Run gnumake in the top level directory and check for any compilation errors.

  9. Please email Marty Kraimer so he can keep track of which sites are using this software.
In order to run the tests included with MPF, the following files must be edited in mpf/iocBoot
iocepicsclient/st.cmd
iocepicsserver/st.cmd
iocmpfclient/st.cmd
iocmpfserver/startServers
ioctcpclient/testClientWaitBuffer100
ioctcpclient/testClientWaitNoBuffer
ioctcplocal/tcpTestWaitLocalBuffer100
ioctcplocal/tcpTestWaitLocalNoBuffer
ioctcpserver/testServerWaitBuffer100
ioctcpserver/testServerWaitNoBuffer
The inet address must be replaced by the address of your test mv162.

Now issue the commands:

cd X-Y
gnumake


Now proceed to the next two sections.

Multiple Processor VME system

This section describes how to set up a system that has two or more processors in a single VME crate: The following IP address (all in same subnet) are needed: The following requires that the board support package was configured with INCLUDE_PROXY_SERVER and INCLUDE_PROXY_CLIENT. Board support packages for the mv167 and mv162 can be obtained from Andrew Johnson. The boot parameters have the following form:

MV167

boot device          : ei
processor number     : 0
host name            : <host>
file name            : <full path>/vxWorks
inet on ethernet (e) : <IPmv167>:<subnet mask>
inet on back plane (b): <IPgateway>
host inet (h)        : <IPhost>
gateway inet (g)     :
user (u)             : <user>
ftp password (pw) (blank = use rsh): <password>
flags (f)            : 0x0
target name (tn)     : ioc167
startup script (s)   : <full path name>/st.cmd
other (o)            :
MV162
boot device          : sm=0x80000600
processor number     : 1
host name            : <host>
file name            : <full path>/vxWorks
inet on ethernet (e) :
inet on back plane (b): <IPmv162>
host inet (h)        : <IPhost>
gateway inet (g)     : <IPgateway>
user (u)             : <user>
ftp password (pw) (blank = use rsh): <password>
flags (f)            : 0x0
target name (tn)     : ioc162
startup script (s)   : <full path name>/st.cmd

Getting Started

The mpf/iocBoot directory has a number of sub directories for running mpf tests and/or examples. The best example is a test which connects an epics database to a remote processor. The test can be run in either local (client and server on the same processor) or remote (client and server on different processors)To start the test: The test consists of the following: Studying this example should give a good idea of how MPF works. In particular look at the following files because they give examples of things developers are likely to do.
mpf/epicsDevApp/epic/TestSrc/DevMpfInt32Test.c This is an example of device support.
mpf/mpfServerApp/testSrc/serverInt32.cc The echo server for Int32 messages.
mpf/mpfServerApp/epicsDevApp/localSrc The subdirectory localSrc shows how to build for a system on which the client and server reside on the same processor.
mpf/mpfServerApp/epicsDevApp/clientSrc
mpf/mpfServerApp/epicsDevApp/serverSrc
The subdirectories clientSrc and serverSrc show how to build on a system where the client and server reside on different processors.
mpf/epicsDevApp/Db/*.db The databases for the example.
mpf/iocBoot/iocepicslocal/st.cmd The startup file for the local test.
mpf/iocBoot/iocepicsclient/st.cmd
mpf/iocBoot/iocepicsserver/st.cmd
The startup files for the remote test.

Studying the above will show how to write servers, device support, build servers and client, and boot servers and clients.

Using MPF in Applications

It is a good idea to keep mpf separate from applications which use it. An application can access mpf components by making the following changes in the application <top>/config directory.

In RELEASE add the line:
MPF=<full path name>
If you are using the config rules like those makeBaseApp release R3.31.2 or later then
gnumake
If you are using older rules then you must edit CONFIG_APP to add:
ifdef MPF
USR_INCLUDES += -I$(MPF)/include
MPF_BIN = $(MPF)/bin/$(T_A)
USER_DBDFLAGS += -I $(MPF)/dbd
endif
In Makefile.Vx statements like
LIBOBJS += $(MPF_BIN)/mpfLib
LIBOBJS += $(MPF_BIN)/ipLib
can be added.

Note that ipLib also includes ipacLib. Thus an application using mpf does not have to also reference the ipac support.

MPF does not include support for specific ip modules. Such support must be obtained elsewhere.

mpf/bin

This is where the various MPF binary files are installed. It contains a subdirectory for each target, e.g. mv167. This section describes the components most likely to be included by applications.

Epics applications

Server applications

MPF Directory Structure

Components which are of interest in the mpf tree are:
<top>/
    config/
           RELEASE
    utilApp/
           utilSrc/
    mpfApp/
           kernelSrc/
           messageSrc/
           libSrc/
           testSrc/
    ipApp/
           src/
    mpfServerApp/
    epicsDevApp/
           Db/
           adl/
           epicsDevSrc/
                 devMpf.*
           epicsTestSrc/
                 DevMpfInt32Test.c
           clientSrc/
           localSrc/
           serverSrc/
    iocBoot/
           iocepicsclient/
           iocepicslocal/
           iocepicsserver/
           iocmpfclient/
           iocmpflocal
           iocmpfserver/
           iocmpflocal/
           ioctcpclient/
           ioctcplocal/
           ioctcpserver/
config This is almost exactly like <top>config described in the 3.13.1 version of makeBaseApp
RELEASE EPICS_BASE must be defined correctly.
utilSrc DLList, DataFreeList, FreeList, and WatchDog; Builds mpfUtilLib.
kernelSrc Contains the MPF code: Message, Routers, Tcp support; Builds mpfKernelLib
messageSrc Contains the code to build specific message types. If new messages type are added this is the place. If it proves necessary to support an endless number of message types a better way of building new message types must be developed. Builds mpfMessageLib.
libSrc Builds mpfLib which contains mpfUtilLib, mpfKernelLib, and mpfMessageLib.
testSrc Contains some low level test programs. Not of interest except for MPF core development.
ipApp/src The ip support. Builds ipLib.
mpfServerApp Contains serverInt32 and serverChar8Array which just echoes the messages sent to them. They are intended for testing and examples. clientInt32 and clientChar8Array are MPF clients. They are good examples of an MPF client and are also used for testing.
epicsDevApp/Db and adl Db contains the epics databases for the example in iocrstclient. adl contains the example medm screen.
epicsDevApp/epicsDevSrc This contains DevMpf, the base class for mpf epics device support.
epicsDevApp/epicsTestSrc MpfInt32Test.cc is a simple example of device support derived from DevMpf.
iocepicsclient
iocepicsserver
Epics (client) side and server side of example described previously.
iocepicslocal The example where client and server run on the same processor.
iocmpfclient
iocmpfserver
Tests for remote MPF communication
iocmpflocal Test for local MPF communication
ioctcpclient
ioctcpserver
Test for TCP communication.
ioctcplocal Local test for TCP communication.

Epics Record to IP - Flow of Control

The following shows the flow of control for the example that appears in iocepicsxxx. The device support DevMpfInt32Test, which is attached to ai records, accepts INP fields with the format:
INP("C1 S0 @Int32")
where When the record is processed the following sequence of events occur:

Message Passing Overview

The message passing system follows a client server model. A client creates a connection to a server. Once a connection is established either the client or server can send messages to the other.

Messages are sent through a router. Two routers are provided: localRouter and RMR (Remote Message Router). The localRouter passes messages between a client and server residing on the same processor. RMR sends messages between a client and server on different processors via a tcp connection.

Message Client

typedef void (*clientCallback)(Message *message,void *clientPvt);
class MessageClient
{
public:
    MessageClient(clientCallback,void *clientPvt);
 
    int bind(char *server, int location);
    int send(Message *message);
    void *getClientPvt();
private:
    ...
}
A client of the message passing system must:
  1. Create an instance of MessageClient for each server with which it wants to communicate.
  2. Issue a bind call to establish communication.
  3. Call send to send messages.
  4. Provide a callback which is called whenever a connection is made or broken and is also called whenever the server sends a message to the client.
NOTE: DevMpf, described below, is a message client. Thus, for epics, new MPF device support is written rather than new message clients. MPF device support is implemented by creating a new class derived from base class DevMpf.

Message Server

class MessageServer
{
public:
    MessageServer(const char* name);
    void waitForMessage();
    Message *receive();
    Message *allocReplyMessage(Message *clientMessage,messageType type);
    int reply(Message *);
    void setQueueSize(int size);
    const char *getName() const;
    void report() const;
private:
     ...
}
A message server must
  1. Create an instance of MessageServer.
  2. Call waitForMessage if it wants to block until it receives messages.
  3. Call receive to accept messages.
  4. Call reply to send messages to a client.
  5. Call allocReplyMessage if it wants to send additional additional messages to a client

Messages

MPF is written so that an extensible set of message types can be provided. The following message types have been implemented: NOTE: The Int32Array, Float64, Float64Array, and OutOfBand messages have not been tested.

Message instances are kept on a free list. Thus malloc is called only if the free list is empty. Free is never called. Each message type has it's own free list.

Example

Look at mpf/mpfServer/testSrc/clientInt32.cc for an example client and mpf/mpfServer/testSrc/serverInt32.cc for an example server.

Diagnostic Aids

The diagnostics described in the section are commands issued via the vxWorks shell.

WARNING: If any queue overflows occur it probably means that you did not chose a large enough queue size.

mrr

Message Router Report provides a list of all client and server routers, connection state, queue sizes and state, and some statistics.

For servers it provides a report like the following:

iocrstserver> mrr
clientRouterList
serverRouterList
 1 RMRServer stateConnected queueSize 100 inQueue 0 replyQueueFull 0
   sendPerSec 240 receivePerSec 240  tcpSendPerSec 10 tcpReceivePerSec 10
It provides a report for each server router. It shows the connection state. The meaning of the other fields are: The statistics are calculated periodically and are truncated thus a value <1 is reported as 0.

For clients it produces the following type of report

iocrstclient> mrr
clientRouterList
 1 RMRClient stateConnected queueSize 100 inQueue 0 sendQueueFull 0
   Server Int32 has 4 clients. bindState connected
   sendPerSec 40 receivePerSec 40  tcpSendPerSec 10 tcpReceivePerSec 10
serverRouterList
It produces a report for each client router. It shows if the router is connected. The queueSize, inQueue, and sendQueueFull provide statistics about the send queue. It also shows a list of each server attached to the router.

msr ("serverName")
msr

Message Server Report provides statistics about the specified server or, if no argument is specified, all servers. For example:
iocrstserver> msr "Int32"
Int32
          queueSize 10
            inQueue 0
      queueRequests 132171
 queueFullResponses 0
      replyRequests 132171
The meaning of the fields are:

Message Routers

Local Message Router

The local message router passes message between a client and server residing in the same processor. When the client passes a message to a server the message is just appended to the server's queue. When the server sends a reply to the client, the client processes the message immediately. Thus when a client sends to the server the client never waits for the server. The server, however, will wait for the client to process a reply message.

Remote Message Router

The remote message router consists of two parts: the client router and the server router. Message are sent as architecture independent data(NOT TESTED). The sender deletes its message and the receiver has to allocate a new message, put the received data into the message and then pass it to the destination.

Router Configuration

Each processor that is a member of a set of MPF clients/servers is assigned a unique location, which is just an integer value. For example if the set is an mv167/mv162 the mv167 could be assigned location 0 and the mv162 location 1. All that matters is that each cpu is assigned a unique location and all cpus agree on the assignments.

Each vxWorks system using the message passing system needs the following command in the startup file:

routerInit
If local communication is desired then the following command must appear:
localMessageRouterStart(location)
The parameter is the location of the local server. Note that the client specifies a location when it binds to a server. Each cpu can start a local router.

If tcp communication is desired then for each remote system the client must have the following command:

tcpMessageRouterClientStart(location,port,"address",bufSize,queueSize)
Each system supporting remote servers must have the following command for each client machine:
tcpMessageRouterServerStart(location,port,"address",bufSize,queueSize)
NOTE CAREFULLY:Each server router can only handle a single client router. A particular client/server pair is established by the client and server having EXACTLY the same (location,port,addresss) tuple. Also each server/client pair must use a unique port number.

Message

Overview

A message is something that is sent from a client to a server or from a server to a client. MPF only knows how to handle messages defined by the files messageType.h and messageType.cc. In addition there are firm rules about how messages are allocated and freed. The basic rules are:
<message class> *pmessage = new <message class>;
for example an Int32 message can be allocated as follows:
Int32Message *pmessage = new Int32Message;
Int32Message *pmessage =(Int32Message*)pMessageServer->
    allocReplyMessage(preceive,messageTypeInt32);
where preceive must be a message received from the client. The reason is that the receive message contains information describing where the client is located.
delete pmessage;
Thus the server must delete any message received from a client and a client must delete reply messages from the server.The definition of class Message is:
class Message
{
public:
    messageType  getType();
    int32 getClientType();
    void setClientType(int32 type);
    int32 getClientExtra();
    void setClientExtra(int32 extra);
    virtual ~Message();
    virtual int  toBuffer(char **buffer);
    virtual int  fromBuffer(const char **buffer);
    virtual int  fromBufferSwitch(const char **buffer);
    virtual void print() const;
    static Message *allocate(messageType type);
protected:
    Message(messageType type);
    ....
}
Client and server code normally only call new, delete, the methods described in the following table, and message specific methods. The other methods are called by MPF itself. A message is allocated via a call to new and released via a call to delete. When delete is called the message is placed on a free list. Only one of the message types derived from Message can be created, i.e. it is not possible to create an object of type Message.

The virtual methods must be implement by any class that derives from Message. The toBuffer and fromBuffer methods are called to put and take messages from a buffer that passes over a communication link.

method Recommended Meaning
getType Returns the message type
getClientType
setClientType
These methods are for use by a client to access field clientType. When the server allocates a reply message via a call to allocateReplyMessage, the clientType from the clients message is copied to the reply message. In general a server should not use these methods. They are intended for clients that send multiple messages to a server and need to match reply messages with messages sent. Classes derived from DevMpf MUST NOT use these because DevMpf itself uses them. Use clientExtra instead.
getClientExtra
setClientExtra
These access a field clientExtra which is similar to clientType. They can be used by classes derived from DevMpf.

Architecture Independent Message Passing

MPF supports message passing between different architectures, i.e. different byte orders and/or different padding requirements. It does, however only work on architectures with 32 bit addressing. It provides support for the following data types: In order to support architecture independent message passing MPF provides some help but also requires that each message type provide methods to transfer data between a message instance and the network buffer.

MPF provides a header file "mpfType.h" which defines typedefs for int16, uint16, int32, uint32, float32, and float64. MPF provides static methods to transfer the following to/from network buffers:

Adding New Message Types

In order to provide a new message type the following must be done: mpfApp/messageSrc contains files xxxMessage.h and xxxMessage.cc which are a skeleton header and source file for a new message type. In addition look at other existing message types for examples.

ConnectMessage

A connect message is passed by MPF itself whenever a connection is made or broken or if a servers message queue is full. It contains the field status which can have one of the following values:

StandardFieldsMessage

This class is not an actual message type but is a class used by other types. These fields provide additional services. The meaning of these fields is entirely up to the client/server combination between which the message is passed. The fields are:
Field Recommended Meaning
timeoutUnits timeoutUnitsSeconds or timeoutUnitsMilliseconds. Default is seconds
timeout The timeout the server can use it it has to wait for a device. The units are determined by timeoutUnits..
cmd A command. Servers can supply a header file defining an enum for the commands.
status A status value.
address For anything that is an int32 and can be interpreted as an address.
extra An extra int32 word that client/server can use for anything they want as long as they agree on the meaning.

Free Lists for Array Messages

All array messages use the utility class DataFreeList for managing storage for the actual arrays. DataFreeList keeps free lists of various sizes from 16 to 4096 bytes. If an array is larger than 4096 bytes the storage is allocated via a call to calloc and freed via a call to free. Thus Char8Arrays can contain 4096 characters, Int32Arrays 1024 elements, and Float64Arrays 512 elements before calloc/free is invoked.

Char8ArrayMessage

This message type provides the standard fields in addition to the following additional fields
Field Recommended Meaning
value The address of a char8 array. The characters are not interpreted or translated by the message class, i.e. they are just a stream of octet values. The standard C library routines such as ::strcpy and ::memcpy can be used on the value field.
numberRetrys A useful field for servers. It should be the number of retrys if a command fails.
eomLen A useful field for servers. It should be the length of an end of message string.
eomString A useful field for servers. It should be a 1 or 2 character end of message string.

Char8ArrayMessage provides the following methods for allocating and freeing the string value.

method usage
allocValue This is a static method which looks for the smallest freelist that can provide the needed space. If the requested size is larger than the size for the largest freelist, it calls new to allocate space.
setSize Sets the current size for the array. This size MUST be less than that allocated by allocValue.
getSize Gets the current array size.
getMaxSize This gets the size allocated by the call to allocValue
freeValue This puts storage obtained by allocValue back on the appropriate free list or calls delete if the storage was obtained via a call to new. This method is called automatically when a Char8Array message is deleted.

Int32Message

This message type provides the standard fields in addition to the following:
Field Recommended Meaning
value The Int32 value being passed.

Int32ArrayMessage

This message type provides the standard fields in addition to an int32 array. The value field is just a pointer to an int32 array. An Int32ArrayMessage provides the following methods for allocating and freeing the array.
method usage
allocValue This is a static method which looks for the smallest freelist that can provide the needed space. If the requested size is larger than the size for the largest freelist, it calls new to allocate space.
setLength Sets the current length for the array. This length MUST be less than that allocated by allocValue.
getLength Gets the current array length.
getMaxLength This gets the length allocated by the call to allocValue
freeValue This puts storage obtained by allocValue back on the appropriate free list or calls delete if the storage was obtained via a call to new. This method is called automatically when an Int32Array message is deleted.

Float64Message

This message type provides the standard fields in addition to the following:
Field Recommended Meaning
value The Float64 value being passed

Float64ArrayMessage

This message type provides the standard fields in addition to an float64 array. The value field is just a pointer to an float64 array. A Float64ArrayMessage provides the following methods for allocating and freeing the array.
method usage
allocValue This is a static method which looks for the smallest freelist that can provide the needed space. If the requested size is larger than the size for the largest freelist, it calls new to allocate space.
setLength Sets the current length for the array. This length MUST be less than that allocated by allocValue.
getLength Gets the current array length.
getMaxLength This gets the length allocated by the call to allocValue
freeValue This puts storage obtained by allocValue back on the appropriate free list or calls delete if the storage was obtained via a call to new. This method is called automatically when an Float64Array message is deleted.

SerialConfigMessage

field usage
baud Baud rate. 0 means don't change
stopBits Stop bits. Normally 1 or 2. 0 means don't change.
bitsPerChar Bits per character. Normally 5,6,7, or 8. 0 means don't change.
parity Single character. Normally 'E', 'O', or 'N'. 0 means don't change.
flowControl Single character. Normally 'H' or 'N'. 0 means don't change.

OutOfBandMessage

This is exactly like an Int32Message except for the type. It is used by a server to send messages to a client that are not responses to a request message.

MPF Device support

NOTE: Since mpf does not supply device specific support the only example of device support is DevMpfInt32Test.. To see more complete examples look at module specific support. The digital support is a simple example. The serial support provides a complex example.

DevMpf is an abstract base class for implementing EPICS device support.

// return codes for startIO and completeIO.

#define MPF_OK                  0
#define MPF_NoConvert           2

enum replyType {replyTypeNone, replyTypeCompleteIO, replyTypeReceiveReply};
class DevMpf
{
public:
    DevMpf(dbCommon*,link*,bool iointValid);

    // Following must be implemented by device support modules
    virtual long startIO(dbCommon*)=0;  // start async IO
    virtual long completeIO(dbCommon*,Message *)=0; // end async IO
    // the following can be implemented by device support modules
    virtual void receiveReply(dbCommon*,Message *);
    virtual void connectIO(dbCommon*,Message *);     // connection message
    virtual void outOfBandIO(dbCommon*,OutOfBandMessage *);// outOfBand
    virtual long convert(dbCommon*,int pass);   // do linear conversion
    // The following must be called after object is constructed
    void bind();
    // The following are usefull during record initilization 
    bool connectWait() {return(connectWait(5.0));}
    bool connectWait(double timeout);
    // send a message to MPF server with no reply expected
    int send(Message*);
    // send a message to MPF server and wait for reply via completeIO
    int sendReply(Message*);
    // send a message stating the type of reply
    int send(Message*,replyType);
    int sendWait(Message *pmessage,replyType type)
        {return(sendWait(pmessage,type,5.0));}
    int sendWait(Message*,replyType,double timeout);
    // This routine gets a pointer to the user portion of the parm field
    const char* getUserParm() const { return((const char*)userParm); }
    long getStatus() const { return(status);}
    // Following are DSET routines
    static long read_write(void*);      // generic DSET read/write routine
    static long ioint(int cmd,dbCommon*,IOSCANPVT* iopvt); // DSET i/o intr
    static long linconv(void*,int); // calls convert
    IOSCANPVT ioscanpvt;
    bool iointValid;
    bool isConnected() { return((connectState==connectYes) ? true : false);}
private:
    ...
}
Before describing the methods a few comments may be helpful.

If the support is for a simple device, i.e. a device that can be supported via the following:

then device support only needs to implement startIO and completeIO. It calls sendReply to send messages. For such simple devices, the base class DevMpf takes care of everything else, e.g. connection and interfacing to record support.

For devices that are not simple, e.g. devices that must take special action when connecting to the server, additional methods are available: In particular:

connectIO is also useful for devices that generate unsolicited output . When it connects to a server, device support can send a message to the server stating that it wants to receive the unsolicited output. The server then sends a message to the client every time it receives a complete set of output from the device.

ioscanpvt is for provided for convenience. If a derived class wishes to support io interrupt processing it can request that the base class perform the necessary initialization. The derived class must, however, call scanIoRequest.

method Usage meaning
DevMpf Constructor The constructor must be given the address of the record and link (INP or OUT). iointValid specifies if the base class should initialize ioscanpvt. Normally this is only invoked via the constructor of a class derived from class DevMpf. After an object of the derived class is completely constructed, the bind method MUST be called.
startIO Called by DevMpf When a request is made to process a record this is called only when the following conditions are all true: The server is connected, PACT is false, and no reply message from the server is available. This MUST be implemented by any class derived from DevMpf.

This method is always called as a result of a record being processed and the record is not active. If startIO issues a send(message,replyTypeCompleteIO) then PACT is set TRUE, i.e. the record will not complete processing until the reply message is received and completeIO is called.

completeIO Called by DevMpf DevMpf calls this when the reply to a send(message,replyTypeCompleteIO) is received. This MUST be implemented by any class that derives from DevMpf.

This method is always called as a result of a record being processed

receiveReply Called by DevMpf DevMpf calls this when the reply to a send(message,replyTypeReceiveReply) is received. This call does not involve record processing.

The default method just prints a message and deletes the message it received.

connectIO Called by DevMpf
Default supplied by DevMpf.
DevMpf calls this when it receives a ConnectMessage. DevMpf provides a default implementation.

If device support implements this method it should call DevMpf::connectIO (just before returning) to ensure correct behavior.

outOfBandIO Called by DevMpf
Default supplied by DevMpf.
DevMpf calls this when it receives an OutOfBand message. For example if server receives a send (no reply) message that can not be processed, the server can send an OutOfBand message to the client.

The default method just prints a message and deletes the message it received.

convert Called by DevMpf, Default supplied by DevMpf This is the DSET convert routine for ai, ao type records. The default version does nothing. Device support derived from base can provide it's own version.
bind Called immediately after any object derived from class DevMpf is created. This binds to the MPF server. Since binding may result in virtual class methods being called, binding MUST be dove after construction. Thus it is not possible to bind in the constructor for DevMpf.
connectWait Called by derived class Wait for connection to server. It can be called by the constructor of the derived class or by a private thread belonging to the derived class. Do NOT call this as part of record processing.
send(Message*) Equivalent to send(message,replyTypeNone)
sendReply Equivalent to send(message,replyTypeCompleteIO)
send(Message*,replyType) Called by derived class Send a message to the server. SetClientType is called to specify the replyType. If the message is successfully sent and replyType is replyTypeCompleteIO pact is set true.

NOTE: Only one send(message,replyTypeCompleteIO) should be outstanding, i.e. after calling send(message,replyTypeCompleteIO) wait for completeIO to be called before issuing another send.

sendWait Called by derived class Send a message and wait for the reply. It can be called by the constructor of the derived class or by a private thread belonging to the derived class. Do NOT call this as part of record processing.
getUserParm Called by derived class Get the portion of the parm of the INP or OUT link that follows the server name.
getStatus Called by derived class Get the status. A 0 value is success. Any other value is device dependent.
read_write Called by record support This is the read or write DSET routine. Handled automatically by DevMpf
ioint Called by record support This is DSET get_io_intr routine. Handled automatically by DevMpf
linconv Called by record support This is DSET linr_conv routine. It calls convert.

The INP or OUT field MUST have the format:

field(INP or OUT, "#C<location> S<signal> @<server>,<deviceSpecific>")
Format of INP or OUT field
parameter meaning
location Location of the server. Must be integer which determines which message router handles messages.
signal For optional use by device support derived from DevMpf
server Name of the server which receives messages.
deviceSpecific For optional use by device support derived from DevMpf. Either a blank or comma can separate the server name from the device specific information.

Device Support Entry Tables

DevMpf.h provides macros for creating DSETs. They are:
MAKE_LINCONV_DSET(<dset name>,<dev init>)
MAKE_DSET(<dset name>,<dev init>)
The first is used for record types that support linear conversions, e.g. ai and ao, and the second for other record types.

Utility Classes

Several utility classes are used by the MPF classes: At least the DLList class should be replaced by using the Standard Template Library, which is part of the C++ .standard library. For now the standard C++ library is not being used by MPF. Instead only the ANSI standard C library is used. These classes are used extensively by MPF and can also be used by user code.

FreeList provides the same functionality as the freeList facility provided with epics base, in fact it is just freeList redone in C++.

DLList is a double linked list class that does not require nodes to be embedded in objects placed in a list. This is different than the ellList facility provided by epics base.

WatchDog is a class that provides functionality similar to the vxWorks wdLib. The major exception is that the user supplied callback is called by a WatchDog supplied task rather than being called at interrupt level.

DataFreeList is a class for data free lists. It provides free lists of sizes ranging from 16 bytes to 4096 bytes. It is used by the array message types.

Reboot can be used to turn off interrupts when a soft reboot is being performed.

These classes are not described in this document. If you want to use them in new code look at examples in existing code.

IndustryPack Support

The Industry Pack software has been modified to utilize the ipac interface since it was first written, thus the methods provided may not implement the most efficient of interfaces but they are backwards-compatible with the older software.

IndustryPack Support consists of the following components:

IndustryPackCarrier

The base class for an IndustryPack carrier. It allows access to IndustryPack carrier configuration and control independent of the specific carrier hardware board.
IndustryPackSite
The base class for a specific site on a specific carrier, i.e. the place where an IP module resides.
IndustryPackIpac
Implements the IndustryPackCarrier and IndustryPackSite classes which actually pass requests on to the ipac software to implement.
IndustryPackModule
Provides IndustryPack Module configuration independent of the specific IP carrier. This class is used by specific module support such as OctalUART and GpibGsTi9914.

IndustryPackCarrier

This is a base class for board level support for IndustryPack carriers. The carrier must be initialized by using a board specific initialization routine prior to use by IndustryPackModule.
class IndustryPackCarrier
{
 public:
  IndustyPackCarrier();
  int registerName(const char *carrierName);
  static IndustryPackCarrier *find(const char *carrierName);
  IndustryPackSite *findSite(const char *siteName);
  IndustryPackSite *addSite(IndustryPackSite *pSite);
  const char *getCarrierName();
  virtual volatile void *allocMemMapIP(int size) = 0;
 private:
    ...
};
class IndustryPackSite
{
 public:
  IndustryPackSite();
  const char *getSiteName();
  virtual volatile void* allocMemMapIP(int size) = 0;
  virtual volatile IP_ID_PROM* getMemBaseID() = 0;
  virtual volatile void* getMemBaseIO() = 0;
  virtual volatile void* getMemBaseIP() = 0;
  virtual int intConfig(int num) = 0;
  virtual void intEnable(int num) = 0;
  virtual void intDisable(int num) = 0;
 protected:
  const char *siteName;
  IndustryPackCarrier *pIpCarrier;
 private:
};
Method Implementor Usage Description
registerName IndustryPackCarrier Derived class. Register the carrier name.
find IndustryPackCarrier Public use. Find and return IndustryPackCarrier.
findSite IndustryPackCarrier IndustryPackModule Find and return site.
addSite IndustryPackCarrier Called by derived class. Add a site.
getCarrierName IndustryPackCarrier Public use. Return carrier name.
allocMemMapIP Derived Class IndustryPackModule Allocate memory.
getSiteName IndustryPackSite Derived class. Get site name
allocMemMapIP Derived class. IndustryPackModule See IndustryPackModule
getMemBaseID Derived class. IndustryPackModule See IndustryPackModule
getMemBaseIO Derived class. IndustryPackModule See IndustryPackModule
getMemBaseIP Derived class. IndustryPackModule See IndustryPackModule
intConfig Derived class. IndustryPackModule See IndustryPackModule
intEnable Derived class. IndustryPackModule See IndustryPackModule
intDisable Derived class. IndustryPackModule See IndustryPackModule

IndustryPackIpac

Mpf uses the drvIpac support from Andrew Johnson. The following example shows how to configure support for the mv162 . The same commands can be used for the MVME172 CPU, but the settings for sites C and D should be omitted as these do not exist on this board. Note that the following command must be on a single line in the startup file.
ipacAddCarrier(&ipmv162,"A:l=3,3 m=0xe0000000,64;B:l=3,3 m=0xe0010000,64;
C:l=3,3 m=0xe0020000,64;D:l=3,3 m=0xe0030000,64")
initIpacCarrier("carrierName", 0)

IndustryPackModule

This is the class which IP module specific code uses, e.g. the Green Springs OctalUart support.
class IndustryPackModule
{
public:
  static IndustryPackModule * createIndustryPackModule(
      const char *moduleName,
      const char *carrierName,
      const char *siteName);
  static IndustryPackModule *find(const char *moduleName);
  static IndustryPackModule *find(
      const char *carrierName,
      const char *siteName);
  volatile void *allocMemMapIP(int size);
  int intConfig(int num);
  int intEnable(int num);
  int intDisable(int num);
  volatile IP_ID_PROM *getMemBaseID();
  volatile void *getMemBaseIO();
  volatile void *getMemBaseIP();
  int getMemSizeIP();
  unsigned char getManufacturer();
  unsigned char getModel();
  unsigned char getRevision();
private:
  
  ...
};
All public methods are implemented by IndustryPackModule and used by module specific code.
Method Description
createIndustryPackModule Creates an instance of IndustryPackModule. It prints an error message and returns 0 if carrierName or siteName does not exist or if moduleName already exists.
find Two find methods are provided. One finds modules by module name. The other finds modules by carrier and site name.
allocMemMapIP Allocate IP memory.
intConfig Interrupt Configure. The argument must be 0 or 1.
intEnable Enable interrupt. The argument must be 0 or 1.
intDisable Disable interrupt. The argument must be 0 or 1.
getMemBaseID Get the address of the module ID space.
getMemBaseIO Get the address of the module IO space.
getMemBaseIP Get the address of the module IP space.
getMemSizeIP Get the size of allocated IP memory.
getManufacturer Get the manufacter ID.
getModel Get the model ID.
getRevision Get the revision

Performance

The purpose of this section is to give some performance numbers that help understand some of the MPF design decisions and also to help users decide when it might be worthwhile to have a non-EPICS MPF processor talking to an EPICS ioc.

The local tests were run on an mv162. The remote tests were run on an mv167 client and mv162 server. I think they were both 25 MHz processors. msgPerSec and tcpPerSec are round trip, i.e. a send and receive. The message in all cases was just an Int32 value. The no buffer case means that a value was sent and the client waited for that response before sending the next message. The buffer case means that the specified number of values were sent before waiting for the responses.

Raw TCP performance

Test msgPerSec tcpPerSec client idle server idle
tcpLocalNoBuffer 237 237 15% -
tcpLocalBuffer100 6050 60 5% -
tcpRemoteNoBuffer 485 485 54% 39%
tcpRemoteBuffer100 10050 100 52% 40%

MPF Performance

Test msgPerSec tcpPerSec client idle server idle
mpfLocalNoBuffer 3705 - 0% -
mpfLocalBuffer100 7966 - 0% -
mpfRemoteNoBuffer 331 331 52% 40%
mpfRemoteBuffer100 1440 14 52% 43%
mpfRemoteBuffer1000 2100 16 31% 18%

octalSerialPort performance

This test. which requires the MPF serial support, consisted of sending 100 character messages to each of the 8 serial ports. Each port was configured at 38400 baud. The results are for the server.
Test msgPerSec interrupt idle total
local 159 76% 0% 89%
remote 156 65% 10% 91%

License Agreement

Copyright (c) 2002 University of Chicago. All rights reserved.

MPF - Message Passing Facility
MPF is distributed subject to the following license conditions:

 SOFTWARE LICENSE AGREEMENT
 Software: MPF

 1. The "Software", below, refers to MPF (in either source code, or
    binary form and accompanying documentation). Each licensee is
    addressed as "you" or "Licensee."

 2. The copyright holders shown above and their third-party licensors
    hereby grant Licensee a royalty-free nonexclusive license, subject to
    the limitations stated herein and U.S. Government license rights.

 3. You may modify and make a copy or copies of the Software for use
    within your organization, if you meet the following conditions:
      a. Copies in source code must include the copyright notice and this
         Software License Agreement.
      b. Copies in binary form must include the copyright notice and this
         Software License Agreement in the documentation and/or other
         materials provided with the copy.

 4. You may modify a copy or copies of the Software or any portion of it,
    thus forming a work based on the Software, and distribute copies of
    such work outside your organization, if you meet all of the following
    conditions:
      a. Copies in source code must include the copyright notice and this
         Software License Agreement;
      b. Copies in binary form must include the copyright notice and this
         Software License Agreement in the documentation and/or other
         materials provided with the copy;
      c. Modified copies and works based on the Software must carry
         prominent notices stating that you changed specified portions of
         the Software.

 5. Portions of the Software resulted from work developed under a U.S.
    Government contract and are subject to the following license: the
    Government is granted for itself and others acting on its behalf a
    paid-up, nonexclusive, irrevocable worldwide license in this computer
    software to reproduce, prepare derivative works, and perform publicly
    and display publicly.

 6. WARRANTY DISCLAIMER. THE SOFTWARE IS SUPPLIED "AS IS" WITHOUT WARRANTY
    OF ANY KIND. THE COPYRIGHT HOLDERS, THEIR THIRD PARTY LICENSORS, THE
    UNITED STATES, THE UNITED STATES DEPARTMENT OF ENERGY, AND THEIR
    EMPLOYEES: (1) DISCLAIM ANY WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
    BUT NOT LIMITED TO ANY IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS
    FOR A PARTICULAR PURPOSE, TITLE OR NON-INFRINGEMENT, (2) DO NOT ASSUME
    ANY LEGAL LIABILITY OR RESPONSIBILITY FOR THE ACCURACY, COMPLETENESS,
    OR USEFULNESS OF THE SOFTWARE, (3) DO NOT REPRESENT THAT USE OF THE
    SOFTWARE WOULD NOT INFRINGE PRIVATELY OWNED RIGHTS, (4) DO NOT WARRANT
    THAT THE SOFTWARE WILL FUNCTION UNINTERRUPTED, THAT IT IS ERROR-FREE
    OR THAT ANY ERRORS WILL BE CORRECTED.

 7. LIMITATION OF LIABILITY. IN NO EVENT WILL THE COPYRIGHT HOLDERS, THEIR
    THIRD PARTY LICENSORS, THE UNITED STATES, THE UNITED STATES DEPARTMENT
    OF ENERGY, OR THEIR EMPLOYEES: BE LIABLE FOR ANY INDIRECT, INCIDENTAL,
    CONSEQUENTIAL, SPECIAL OR PUNITIVE DAMAGES OF ANY KIND OR NATURE,
    INCLUDING BUT NOT LIMITED TO LOSS OF PROFITS OR LOSS OF DATA, FOR ANY
    REASON WHATSOEVER, WHETHER SUCH LIABILITY IS ASSERTED ON THE BASIS OF
    CONTRACT, TORT (INCLUDING NEGLIGENCE OR STRICT LIABILITY), OR
    OTHERWISE, EVEN IF ANY OF SAID PARTIES HAS BEEN WARNED OF THE
    POSSIBILITY OF SUCH LOSS OR DAMAGES.