Experimental Physics and
Industrial Control System

Andrew Johnson <[email protected]> · Fri, 25 Feb 2005 11:25:39 -0600

Responding to these comments in the order they were made:

Ralph Lange wrote:

Does this imply that IOC load is evenly distributed? I.e. that records

with the same scanPeriod period are processed in a "spread over time"

manner rather than "en bloc" as in V3?

I suspect that's how it would behave, although I'm not 100% sure how we

should be measuring the delay period - presumably measured from the

start time of the previous record processing (i.e. we'd add another

timestamp field in dbCommon to record the start of processing time,

although we could just start the processing delay time from the *end* of

the previous processing). If we do it that way then records of the same

period would get spread out over time by the delays between their timer

going off and actually getting processed.

I would prefer such a spread behaviour, as it really should result in a

more "elastic" IOC behavior. But: How would overload situations be

handled (i.e. processing all records with a certain scanPeriod takes

longer than that period). While the V3 method was inherently stable, it

might be that this new way requires additional care. (When spread over

different threads, overload situations may be hard to detect.)

I think we could detect and report this problem relatively easily - a

record can only appear in the processing queue once, and an overload

just means that at the end of processing when it's re-arming the

record's timer, the 'time when I'm next due to be processed' has already

passed.

The V3 mechanism of the PHAS field determining the order of processing

within a scan period does not make sense anymore with your approach,

right?

Right (at least that's what I was thinking, but see below).

Will we have to have some kind of replacement for that mechanism,

or can we urge the user to go to hard links and events (yes: named

events!) instead?

Hmm, named events... Making the scanEvent field into a string might not

be a good idea, and I'm not sure about the idea of making it a menu

either as there's no way to add a new event type at runtime (unless we

allow new event menu choices to be added at runtime; removing or

renumbering menu choices should be impossible, but adding new ones isn't

beyond the realm of possibility...)

Ralph Lange wrote:

Bob Dalesio wrote:

Marty pointed out then that the use of forward links and passive records
made the load leveling a very difficult problem. Since then, I would add

that there are times that you want to have records of a slower period

still all process at the same time.

But most probably not *all* records of that slower period, only those

who belong to a certain subsystem, right?

Hmm ... let's assume that every of these user named and created scanner

threads can be configured to process its records either "synchronised en

bloc" or "balanced". In that case the desired behaviour could be

achieved by having your set of records all processed by a certain

thread, which is configured to do it "en bloc", right?

I don't know how we would actually do the load balancing, and I'm not

sure it's necessary to really be pro-active about it. Load balancing

happens naturally if we queue a record for processing when its timer

runs out, but implement it such that other records already on the

'process now' queue for this thread will cause the record's actual

processing to be delayed, and have the timer for the next processing

tick doesn't start running until the record processing actually begins.

 Thus a periodic scan is only a promise to initiate record processing

"no more often than every x seconds".

I don't think this is significantly different to what we actually end up

doing in V3, we're just being a bit more up front about how it actually

works in an overload situation, and the natural load balancing that I

think would result just falls out of the algorithm. I may be wrong

though...

As to how you tell the database to 'process these records all at once',

I'd rather that this requirement be stated explicitly in the database

design, either through record linking or by declaring a soft event.

With the soft event approach you can even use multiple threads to run

the records processed as a result of the event, according to their

scanThread field setting.

Hmm, here's another suggestion that makes 'process these records all at

once' easier to achieve - how about we make the FLNK field into an array

of links? That way you can kick off large numbers of other records if

you want to without using a fanout record; it might mean an extra set of

braces in the field setting for every forward link, but maybe not.

Perhaps an alternative would be to specify the base rate for the periodic
scanner.
Use PHAS to set which of the sub-periods to use to process the record
So - if the base scan period is 60 Hz, and you define a 1 second scan
record, PHAS of 30 would be at the halfway point of the 60 Hz scanning.

I can it being useful to be able to say "process this record 0.5 seconds

after that other 1 second scanned record gets processed" but it's not

that common a requirement. To do that with my approach we would need

some kind of delay - maybe we write a delay record that is asynchronous

and kicks off its forward link some fixed time interval after it first

gets processed. You can do this at the moment with a sequence record,

so it's not really that new a concept.

To bring this back to the functional specifications - the question

seems to be:

Do we need to be able to support "load balancing"

I'm not sure I want to call my suggestion "load balancing", which to me

would imply a different approach that I don't want to have to implement

myself.

I think that it does not buy us much - but it is nice to give the

database engineer more control.

I have seen some cases where in case of a serial-type connection

underneath a large number of IO-connected records in a slow task (e.g.

the 10 second scan taking more than 1.5 seconds to finish) was filling

up low level queues so that fast processing (going through the same

queue) was heavily affected. With almost no way to change this behaviour.

Load-balancing would make the IOC a lot more "elastic" in this regard.

You could manually spread the serial I/O over the 10 second scan period

in V3 by using a sequence record to delay different parts of it. I'm

not sure whether my suggestion will do so naturally or not, that really

depends on your configuration.

- Andrew
--
Dear God, I didn't think orange went with purple until I saw
the sunset you made last night.  That was really cool. - Caro

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System