Experimental Physics and
Industrial Control System

[email protected] (Jeff Hill) · Tue, 15 Aug 2000 11:58:57 -0600

John,

> 
> Since upgrading to 3.13.2 I've been encountering the following error one or
> more times per day on two IOCs I have running a Channel Access Server
> application for our ethernet-based Modicon PLCs:
> 

For those of you who are concerned that this problem might occur
in your IOCs I should 
clarify that the server mentioned here is 
the new C++ based server and it is not at this point used by the
EPICS IOC system code, and would not run under vxWorks unless you 
write a specialized application which calls the C++ based server 
library.

Also note that while the R3.13 C++ based server library might be 
compiled for vxWorks it is _not_ yet thread safe.

It appears that a pure virtual function is being called in 
the member function osiTimerQueue::process () (see attached).
Unfortunately with this information we dont know the name
of the class that derived from the osiTimer abstract base class.

I suspect that a timer object is still being used after its destructor 
has run. Its possible that some other thread has deleted the timer object 
(note that this code is not thread safe in R3.13) and it is also possible
that some internal error has occurred in the server system code
or in the server tool application code.

When a derived class's destructor is called the virtual function
jump table pointer in the base class is often updated by the compiler
implementation to point to the base class's virtual function
jump table which has stubs in it for pure virtual functions
that print the message you are seeing.

It difficult to make any additional conclusions with this information.
If I was debugging this I would look carefully at:

1) situations where multiple threads are allowed to access the 
server library routines simultaneously.
2) any classes you have created that derive from the osiTimer
abstract base class.
3) attempt to determine which class deriving from osiTimer
is involved. If the problem is repeatable, then you could
add breakpoints or print statements to the destructors for 
the osiTimer derived classes. You also might find that
compiling the code for debugging and using the source level
debugger in Tornado might provide more information in the stack 
trace.

I would look at the differences between R3.13.1 and R3.13.2 in 
CVS but unfortunately I have been unable to use CVS at APS 
for the last several days probably due to IP address changes 
here.

Jeff

//
// osiTimerQueue::process()
//
void osiTimerQueue::process()
{
	tsDLIterBD<osiTimer> iter;
	tsDLIterBD<osiTimer> tmp;
	osiTime cur(osiTime::getCurrent());
	osiTimer *pTmr;

	// no recursion
	if (this->inProcess) {
		return;
	}
	this->inProcess = osiTrue;

	iter = this->pending.first();
	while ( iter!=eol ) {	
		if (iter->exp >= cur) {
			break;
		}
		tmp = iter;
		++tmp;
		this->pending.remove(*iter);
		iter->state = ositExpired;
		this->expired.add(*iter);
		iter = tmp;
	}

	//
	// I am careful to prevent problems if they access the
	// above list while in an "expire()" call back
	//
	while ( (pTmr = this->expired.get()) ) {

		pTmr->state = ositLimbo;

#ifdef DEBUG
		double diff = cur-pTmr->exp;
		printf ("expired %lx for \"%s\" with error %f\n", 
			(unsigned long)pTmr, pTmr->name(), diff);
#endif

		//
		// Tag current tmr so that we
		// can detect if it was deleted
		// during the expire call back
		//
		this->pExpireTmr = pTmr;
		pTmr->expire();
		if (this->pExpireTmr == pTmr) {
			if (pTmr->again()) {
				pTmr->arm();
			}
			else {
				pTmr->destroy();
			}
		}
		else {
			//
			// no recursive calls  to process allowed
			//
			assert(this->pExpireTmr == 0);
		}
	}
	this->inProcess = osiFalse;
}

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System