Experimental Physics and
Industrial Control System

I few months ago I sent a message proposing a replacement for "bare"
Hideos that is built on top of vxWorks. Attached is an html document
describing the current status. I have also made this document available
via the APS WWW Epics documents.

I am hoping to convert some of the APS/ASD systems to the new facility
during a scheduled shutdown this coming July. 

Marty Kraimer

MPF: Message Passing Facility

MPF will replace HIDEOS

Background:

Strategy:

The most important goal is to provide a migration path for existing HIDEOS applications. An EPICS  Hideos application uses the following  hideos components:
 

• The idea of task groups will no longer be supported. When needed a special startup routine will have to be provided which just starts the individual tasks.
• It will no longer be possible to send messages from interrrupt level

• Each message class must provide methods to transfer data between a message instance and a network buffer.
• A message class is allowed to define arbitrary methods

What hardware configurations do we want to support?

• Epics and IP support on a single processor
• Epics on one processor. vxWorks and IP support on another processor. Communication will be via a tcp connection. It will be possible to boot  a cpu via the VME backplane and also pass tcp messages over the backplane.
• Epics and IP support for a "dumb" VME IP module.

Message Passing Overview

Messages are sent through a router. Initially two routers will be provided: localRouter and tcpRouter. The localRouter passes messages between a clients and servers residing on the same processor. The tcpRouter sends messages between clients and servers on different processors.

Message Client Class

typedef void (*clientCallback)(Message *message,void *clientPvt);
class MessageClient
{
public:
    MessageClient(clientCallback,void *clientPvt);
 
    int bind(char *server, int location);
    int send(Message *message);
private:
    ...
}

1. Create an instance of MessageClient for each server with which it want to communicate.
2. Issue a bind call to establish communication.
3. Call send to send messages.
4. Must provide a callback which is called whenever a connection is made or broken and is also called whenever the server sends a message to the client.

Message Server Class

class MessageServer
{
public:
    MessageServer(const char* name);
    void waitForMessage();
    Message *receive();
    Message *allocReplyMessage(Message *clientMessage,messageType type);
    void  queueMessage(Message *);
    int reply(Message *);
private:
     ...
}

1. Create an instance of MessageServer.
2. Call receive to accept messages.
3. Call reply to send messages to a client.
4. Call allocReplyMessage if it wants to send additonal message to a client

Message Class

class Message
{
public:
    messageType  getType();
    virtual ~Message();
    virtual void init() = 0;
    virtual int  toBuffer(char **buffer);
    virtual int  fromBuffer(const char **buffer);
    virtual int  fromBufferSwitch(const char **buffer);
    virtual void print() const;
    static Message *allocate(messageType type);
protected:
    Message(messageType type);
    ....
}

Most methods are virtual and must be implement by any class that derives from Message. The init method is called whenever a messages in allocated. The toBuffer and fromBuffer methods are called to put and take messages from a buffer that passes over a communication link.

 Example

Client

typedef struct privateData {
    connectStatus connectState;
    int32         value;
    SEM_ID        sem;
}privateData;
 
void myCallback(Message *message,void *clientPvt)
{
    privateData *pprivateData = (privateData *)clientPvt;
 
    switch(message->getType()) {
    case messageTypeConnect:
        {
            ConnectMessage *pConnectMessage = (ConnectMessage *)message;
            pprivateData->connectState = pConnectMessage->status;
            delete pConnectMessage;
            semGive(pprivateData->sem);
        }
        break;
    case messageTypeInt32:
        {
            Int32Message *pInt32Message = (Int32Message *)message;
            pprivateData->value = pInt32Message->value;
            delete pInt32Message;
            semGive(pprivateData->sem);
        }
        break;
    default:
        printf("got illegal response message\n");
        break;
    }
}
 
int clientExample(char *server,int location,int ntimes)
{
    MessageClient *pMessageClient;
    privateData *pprivateData = new privateData;
    int         status;
    Message     *message;
 
    pprivateData->connectState = connectNo;
    pprivateData->value = -1;
    pMessageClient = new MessageClient(myCallback,pprivateData);
    pprivateData->sem  = semBCreate(SEM_Q_PRIORITY,SEM_EMPTY);
    status = pMessageClient->bind(server,location);
    if(status) {
        printf("bind returned %d\n",status);
        return(status);
    }
    //wait for connect
    semTake(pprivateData->sem,WAIT_FOREVER);
    for(int i=0; i<ntimes; i++) {
        while(pprivateData->connectState!=connectYes) {
            printf("client connectState %d\n",pprivateData->connectState);
            taskDelay(60);
        }
        message = new Int32Message;
        status = pMessageClient->send(message);
        if(status) {
            printf("send returned %d\n",status);
            continue;
        }
        semTake(pprivateData->sem,WAIT_FOREVER);
        printf("new value is %ld message %p\n",pprivateData->value,message);
    }
    return(0);
}

Server

struct privateData {
    int32 counter;
};

int serverExample(char *name)
{
    privateData   *pprivateData = new privateData;
    Message       *message;
    Int32Message   *pInt32Message;
    MessageServer *pMessageServer;
 
    pMessageServer = new MessageServer(name);
    pprivateData->counter = 0;
    while(TRUE) {
        pMessageServer->waitForMessage();
        message = pMessageServer->receive();
        if(message->getType()!=messageTypeInt32) {
            printf("serverExample got illegal message type\n");
        } else {
                pInt32Message = (Int32Message *)message;
                pInt32Message->value = pprivateData->counter++;
                pMessageServer->reply(pInt32Message);
        }
    }
    return(0);
}

Configuration

routerInit

localMessageRouterStart(0)

If tcp communication is desired then for each remote system the client must have the following command:

tcpMessageRouterClientStart(location,port,"address")

• location is the location assigned to the server.
• port is the port the server will use to listen for a connection.
• address is the inet address of the  server.

tcpMessageRouterServerStart(location,port,"address")

• location is the location assigned to the server.
• port is the port the server will use to listen for a connection.
• address is the inet address of the  server. This argument can be null, in which case the server will accept connections via all internet interfaces attached to it's cpu.

Architecture Independent Message Passing

• signed and unsigned 16 and 32 bit integers
• IEEE float and double.
• C style ascii strings
• octet arrays. Octets are uninterpreted 8-bit values whose contents are guaranteed not to undergo any conversions during transmission.

MPF provides a header file "mpfType.h" which defines typedefs for int16, uint16, int32, uint32, float32, and float64. MPF provides static methods to transfer the following to/from network buffers:

• signed and unsigned 16 and 32 bit integers
• IEEE float and double
• C style ascii strings
• Octet arrays

Adding New Message Types

• messageType.h and messageType.cc must be modified.

These files contain instructions about what to add. This is a simple change.
• A header and source file for the new message must be created.

xxxMessage.h

// xxxMessage.h

#ifndef xxxMessageH
#define xxxMessageH

class Message;
class DLList;

class xxxMessage : public Message
{
public:
    xxxMessage();
    virtual ~xxxMessage() {}
    virtual void init();
    virtual int32  toBuffer(char **buffer) const;
    virtual int32  fromBuffer(const char **buffer);
    virtual int32  fromBufferSwitch(const char **buffer);
    virtual void print() const;
    static void *operator new(size_t size);
    void operator delete(void *p, size_t size);
private:
    static DLList *pFreeList;
    // Disallow copy constructor, assignment operator 
    xxxMessage(const xxxMessage&);
    xxxMessage& operator=(const xxxMessage&);
};

inline xxxMessage:: xxxMessage() : Message(messageTypexxx), value(0)
{ init(); }

#endif /*xxxMessageH*/

xxxMessage.cc

// xxxMessage.cc

extern "C" {
#include <stdlib.h>
#include <stddef.h>
#include <string.h>
#include <stdio.h>
}

#include "Message.h"
#include "xxxMessage.h"

DLList* xxxMessage::pFreeList = 0;

void xxxMessage::init() { value=0; }

int32 xxxMessage:: toBuffer(char **buffer) const
{
    int32 nbytes;

    nbytes = Message::toBuffer(buffer);
    //now transfer any values added by the message type
    // nbytes += MPF::toBuffer(buffer,???);
    return(nbytes);
}

int32 xxxMessage:: fromBuffer(const char **buffer)
{
    int32 nbytes=0;

    nbytes = Message::fromBuffer(buffer);
    //now transfer any values added by the message type
    //nbytes += MPF::fromBuffer(buffer,&???);
    return(nbytes);
}

int32 xxxMessage:: fromBufferSwitch(const char **buffer)
{
    int32 nbytes=0;

    nbytes = Message::fromBufferSwitch(buffer);
    //now transfer any values added by the message type
    //nbytes += MPF::fromBufferSwitch(buffer,&???);
    return(nbytes);
}

void xxxMessage:: print() const
{
    Message::print();
    //print any values added by the message type
    //printf("value %d\n",???);
}

void *xxxMessage:: operator new(size_t size)
{
    if(size==sizeof(xxxMessage)) return MPF:: allocate(&pFreeList,size);
    printf("xxxMessage new size !=sizeof(xxxMessage)\n");
    return(0);
}
 
void xxxMessage:: operator delete(void *p, size_t size)
{
    if(size==sizeof(xxxMessage)) {
        MPF::free(pFreeList,p,size);
        return;
    }
    printf("xxxMessage delete size !=sizeof(xxxMessage)\n");
}

Discussion

1. Network Transparent message passing. This will allow messages to be passed between different machine architectures.  The initial version, however, only works on 32 bit architectures.
2. Connection Management - The initial release will allow either server or client to be started first. It will also allow either a client or server to be rebooted without requiring the partner to be rebooted.
3. Messages can have complicated methods. (HIDEOS required that all message methods be in-line)

Status

Joe Sullivan is currently developing mv167 and mv162 board support packages that allow a 162 to be booted via the VME backplane. this will be for vxWorks 5.3

The EPICS Hideos device support and the Hideas IP tasks have not been modified to  support the new Message Passing facility.
 

Navigate by Date:

Prev: EPICS 3.12 and VxWorks 5.3 Chris Witzig

Next: vme Record Jeffrey Mcdonald

Index: 1994  1995  1996  1997  <1998>  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 

Navigate by Thread:

Prev: EPICS 3.12 and VxWorks 5.3 Chris Witzig

Next: vme Record Jeffrey Mcdonald

Index: 1994  1995  1996  1997  <1998>  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024