Experimental Physics and
Industrial Control System

"Caizhen Liu" <[email protected]> · Fri, 14 Oct 2016 09:18:35 -0700

Dear Kay,

I got the following error in the log file
/var/opt/codac/css/archive-engine-CODAC/console.log. But this error appears
even though I try to archive only 90 channels when the state is ok. 

2016-10-14 09:12:53.711 WARNING [Thread 46]
org.csstudio.archive.engine.model.WriteThread (run) - Error, will try to
reconnect
org.postgresql.util.PSQLException: ERROR: value out of range: underflow
	at
org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorI
mpl.java:2198)
	at
org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.ja
va:1927)
	at
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)
	at
org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.j
ava:561)
	at
org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2St
atement.java:419)
	at
org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2State
ment.java:365)
	at
org.csstudio.archive.writer.rdb.NumericMetaDataHelper.insert(NumericMetaData
Helper.java:84)
	at
org.csstudio.archive.writer.rdb.RDBArchiveWriter.writeMetaData(RDBArchiveWri
ter.java:203)
	at
org.csstudio.archive.writer.rdb.RDBArchiveWriter.addSample(RDBArchiveWriter.
java:176)
	at
org.csstudio.archive.engine.model.WriteThread.write(WriteThread.java:258)
	at
org.csstudio.archive.engine.model.WriteThread.run(WriteThread.java:182)
	at java.lang.Thread.run(Thread.java:745)

Is this related to our EPICS IOC?

Thank you.

Caizhen

-----Original Message-----
From: Caizhen Liu [mailto:[email protected]] 
Sent: Friday, October 14, 2016 9:06 AM
To: 'Kasemir, Kay'; '[email protected]'
Cc: 'Ben Penaflor'; 'Ikenna Anyanetu'
Subject: RE: archive engine

Dear Kay,

Thank you very much for answering my question!

When I tried to archive only 90 channels, sometimes the "Write State" shows
"Write Error". When the state is ok, the write count is about 900 samples
and the write duration is only 0.4 sec. 

But when I tried to archive more channels, the "Write State" always shows
"Write Error". 

The Memory is about 6.2%, so I guess it is not memory issue. 

Any ideas/suggestions would be deeply appreciated. 

Caizhen

-----Original Message-----
From: Kasemir, Kay [mailto:[email protected]] 
Sent: Friday, October 14, 2016 8:23 AM
To: Caizhen Liu; [email protected]
Subject: Re: archive engine

Hi:

>We have about 4000 PVs being archived at the moment.
> It seems that the archiving have some issue starting from PV
?OPS-EPICS:R:CCG1?.
> The queue is full and overruns happens. And there is no data record for PV
OPS-EPICS:R:CCG1 in the postgres  database.
> When I reduced the number of channels to be 90, everything seems ok.

More important than the number of PVs or the specific PV names is the number
of samples that these channels produce.
The main web page of the archive engine will show you a "write count" and a
"write duration".
From watching that for a while, you can compute the number of values that
the engine can write to your relational database.

Bottom line is that the RDB approach for archiving is very reliable, but not
fast.
You may simply reach some maximum supported number of samples/sec that you
can write to your database.

When you build CSS from sources, you can execute the RDBArchiveWriterTest
(https://github.com/ControlSystemStudio/cs-studio/blob/master/applications/a
rchive/archive-plugins/org.csstudio.archive.writer.test/test/org/csstudio/ar
chive/writer/rdb/RDBArchiveWriterTest.java).
Way back, I got these types of performance readings:
PostgreSQL: ~12000 samples/sec
MySQL: ~21000 samples/sec

Those are write speeds when the database is on the same computer that's also
running the test, and nothing else is using the RDB.
In the real world, when you connect to a remote database that's also doing
its day job, the numbers a lower.

The EPICS Archiver Appliance,
https://slacmshankar.github.io/epicsarchiver_docs/index.html, is faster
because it uses its own set of files instead of a relational database.

-Kay

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System