Experimental Physics and
Industrial Control System

Benjamin Franksen <[email protected]> · Fri, 24 Jun 2005 01:09:12 +0200

These are comments on 
http://www.aps.anl.gov/epics/wiki/index.php/V4_Design:_Record_Processing.

First two remarks on the epicsEvent stuff:

# It is not clear from the text whether an epicsEvents carries data or 
not. If not, record support has to get the data, e.g. from an 
asynchronous input link, by asking the link for the value after waiting 
for the event. I have the impression that this is what was meant in the 
document. For the rest of this message I assume this is the case, i.e. 
events are just events, i.e. no data, only identity.

# Link support sending an event to record support is just a special case 
of general callback-based asynchronous (link) processing: the record 
sets the callback to a function that inserts an event into the event 
list. Such a function can be predefined in teh core libraries. Thus, 
link support does not need to support events directly, only regular 
callbacks are needed. An alternative implementation could use a 
counting semaphore instead of a list of events; link support would 
never notice.

Now to my main point:

If I understand the document correctly, record and link processing will, 
just like in V3, expose different behavior for synchronous and 
asynchronous processing. A synchronous record will have completed 
processing after a call to process returns, whereas an asynchronous one 
might still be (concurrently) involved in its processing.

I dare ask if this is the right model for record and link processing. 
Esspecially if we are re-thinking almost everything else.

My objections:

The distinction between synch/asynch complicates processing semantics up 
to the point where database designers as well as programmers of client 
code can become really confused why their design doesn't work as 
expected. For instance, an epics database must often be designed with 
exact knowledge about which records will be processed synchronous and 
which asynchronous. At the moment, synch/asynch cannot be detected at 
all, other than by looking at the device support code or, if exists, 
documentation. The document proposes to have a runtime property for 
this, but that is no use for the database designer. Thus, one cannot 
replace a device support in an existing database without first checking 
that synch/asynch is the same before and after, or redesign if not, in 
order to preserve semantics of link and record processing. This makes 
db design fragile to such changes, which in turn partly defeats the 
idea to separate device support from record support.

Similar scenarios apply for client code that calls dbProcess or gets and 
puts data over links.

Let me present an alternative model:

Conceptually, processing is always asynchronous. Optional synchronous 
processing is viewed as a low-level optimization that is semantically 
transparent.

Thus, conceptually, calling dbProcess merely triggers processing of the 
target record. The caller does not block. Never mind that in some cases 
processing is in fact done synchronously, i.e. before the call returns. 
The caller doesn't know and mustn't care and the callee doesn't 
advertise it. If the caller is interested in completion, it /must/ 
supply a callback, which is then guaranteed to be called whenever 
processing completes.

However, instead of making the callback send an event to us 
and /waiting/ for this event (blocking), we pass a /continuation/ as 
callback. ('Continuation' means 'whatever should be done after some 
action'.)

The idea is that record and link processing code /never/ actually calls 
stuff that might block indefinitely. Instead it triggers concurrent 
execution and supplies a continuation callback.

It took myself a while to see that this model works quite nicely even in 
cases where you need data from multiple inputs in order to continue: 
all you have to do is count! For each input, increment a counter before 
processing the link, decrement it inside the callback, and return 
immediately unless the counter goes back to zero.

Calcout example:

* call the link support for each input link, each time incrementing a 
counter, then return.
* as callbacks, supply a routine (the same for all inputs) that does:
** get the data from the link
** decrement counter
** if not zero, then return, else:
** perform calculation
** set val field
** call link support for output link; return
** as callback supply a routine that does:
*** complete processing

Note: In this example, link support is still free to perform its actions 
synchronously if it doesn't need to wait for anything. That is, it can 
read the data from the source, and then immediately call the callback. 
OTOH, a truly asynchronous link support will have to trigger some 
processing elsewhere, supplying its own continuation, which in turn has 
to call the original continuation. Again: this is completely 
transparent to the calling record support code.

BTW, input links with empty (NULL) continuation callback don't make much 
sense and should be signalled with an exception. For output links, 
however, it does make sense and should be allowed.

Downsides:

- The continuation might be called from a different thread than the 
original request to process. Code that _really_has_to_ maintain thread 
context must resort to supplying an event signalling callback, then 
wait (block) for events and then continue from there, as indicated 
above at the beginning of this message. However, I bet for the vast 
majority of processing code this is /not/ a requirement and thread 
context for the continuation is immaterial.
- Might be _sightly_ less efficient in certain (synchronous) cases. I 
don't expect this to be a show-stopper. Granted, nowadays most device 
supports are asynchronous anyway.
- Not yet clear to me, how priorities are inherited. Maybe through mutex 
semaphores, maybe need to implement extra stuff.
- Generally, could mean more work to implement this scheme. For 
instance, _maybe_ it will be necessary to implement some extremely 
light weight cooperative scheduling on top of native threads. OTOH, 
maybe clever use of a thread pool plus priority queues (as sketched in 
V4_Design:_Record_Processing) the will suffice.

Ben

PS: Please be aware that I use the term 'processing' generically for 
record /and/ link processing, expressly including device support (at 
least its 'read' or 'write' methods) as a special case of link support. 
Hope I got this one right.

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System