Experimental Physics and
Industrial Control System

"Kasemir, Kay" <[email protected]> · Thu, 20 Nov 2008 17:01:57 -0500

Hi:

That error
> ../DataFile.cpp (105): Cannot reference '/arch/data/...'
which happens when you look at a wide time range is actually
caused by running out of file descriptors.

I've tried to keep data files open while processing one request
just in case I need to access that file again,
but eventually you might have too many files open
and then you get that error.

In Storage/RawDataReader.cpp around line 215 you should find
the start of the RawDataReader::getHeader() method.

Add a 
   DataFile::clear_cache();
as shown below to close previously opened data files
before opening a new one.

-Kay

void RawDataReader::getHeader(const stdString &basename, FileOffset offset)
{
    if (!Filename::isValid(basename))
        throw GenericException(__FILE__, __LINE__, "'%s': Invalid basename",
                               channel_name.c_str());
    try
    {
        AutoPtr<DataHeader>  new_header;
        {   // Read new header
            DataFile *datafile;
            if (basename[0] == '/') // Index gave us the data file with the
full path
                datafile = DataFile::reference("", basename, false);
            else // Look relative to the index's directory
                datafile = DataFile::reference(index_result->getDirectory(),
                                               basename, false);
            // If we keep opening data files, we'll hit the max-open-files
limit.
            // Close files which are no longer referenced:
            DataFile::clear_cache();

            try
            {
                new_header = datafile->getHeader(offset);
            }

On 11/20/08 15:36 , "Devin Bougie" <[email protected]> wrote:

> Hi, All.  We've started to see "Error in data header" errors when
> trying to view more than 55 days of archived data for some channels.
> For example, if I try to plot from "-55d" to "now", everything works.
> If I try to plot from "-56d" to "now" I get an error.  Likewise,
> "-60d" to "-5d" works, but "-60d" to "-4d" does not.
> 
> Is there anything I should look at or try before using ArchiveDataTool
> to copy the existing archive to a new one?  Here are examples of some
> of the errors we see in archserver.log.
> 
> Many thanks,
> Devin
> 
> 11/20/2008 14:50:31 ---- ArchiveServer 2.9.2 Started ----
> 11/20/2008 14:50:31 Client: Apache XML-RPC 1.2-a3-dev @
> 128.84.45.100:33129
> 11/20/2008 14:50:31 archiver.get_values
> 11/20/2008 14:50:31 how=3, count=785
> 11/20/2008 14:50:31 get_channel_data
> 11/20/2008 14:50:31 Method: Plot-Binning, 6168.15 sec bins
> 11/20/2008 14:50:31 Start:  09/25/2008 14:50:31.824000000
> 11/20/2008 14:50:31 End  :  11/20/2008 14:50:31.824000000
> 11/20/2008 14:50:31 Open index, key 1 = '/arch/all.xml'
> 11/20/2008 14:50:31 Handling 'YP1FIA02_GFL_cplr1_win'
> 11/20/2008 14:50:36 Error:
> ../RawDataReader.cpp (277): Error in data header '/arch/data/l0',
> '2008/11_19/20081119-3' @ 0x00D615F7 for channel 'YP1
> FIA02_GFL_cplr1_win'.
> ../DataFile.cpp (105): Cannot reference '/arch/data/
> l0/2008/11_19/20081119-3'
> 11/20/2008 14:50:36 ArchiveServer ran 4.17315 seconds
> 
> ...
> 
> 11/20/2008 14:51:54 ---- ArchiveServer 2.9.2 Started ----
> 11/20/2008 14:51:54 Client: Apache XML-RPC 1.2-a3-dev @
> 128.84.45.100:33136
> 11/20/2008 14:51:54 archiver.get_values
> 11/20/2008 14:51:54 how=3, count=785
> 11/20/2008 14:51:54 get_channel_data
> 11/20/2008 14:51:54 Method: Plot-Binning, 6278.22 sec bins
> 11/20/2008 14:51:54 Start:  09/24/2008 14:51:54.715000000
> 11/20/2008 14:51:54 End  :  11/20/2008 14:51:54.715000000
> 11/20/2008 14:51:54 Open index, key 1 = '/arch/all.xml'
> 11/20/2008 14:51:54 Handling 'YP1FIA02_GFL_cplr1_win'
> 11/20/2008 14:51:58 Error:
> ../RawDataReader.cpp (277): Error in data header '/arch/data/l0',
> '2008/11_18/20081118-5' @ 0x01410958 for channel 'YP1
> FIA02_GFL_cplr1_win'.
> ../DataFile.cpp (105): Cannot reference '/arch/data/
> l0/2008/11_18/20081118-5'
> 11/20/2008 14:51:58 ArchiveServer ran 4.17804 seconds
> 
> ...
> 
> 11/20/2008 14:52:35 ---- ArchiveServer 2.9.2 Started ----
> 11/20/2008 14:52:35 Client: Apache XML-RPC 1.2-a3-dev @
> 128.84.45.100:33137
> 11/20/2008 14:52:35 archiver.get_values
> 11/20/2008 14:52:35 how=3, count=785
> 11/20/2008 14:52:35 get_channel_data
> 11/20/2008 14:52:35 Method: Plot-Binning, 6058.09 sec bins
> 11/20/2008 14:52:35 Start:  09/21/2008 14:52:35.093000000
> 11/20/2008 14:52:35 End  :  11/15/2008 14:52:35.093000000
> 11/20/2008 14:52:35 Open index, key 1 = '/arch/all.xml'
> 11/20/2008 14:52:35 Handling 'YP1FIA02_GFL_cplr1_win'
> 11/20/2008 14:52:39 Error:
> ../RawDataReader.cpp (277): Error in data header '/arch/data/l0',
> '2008/11_15/20081115-13' @ 0x01C6AADE for channel 'YP
> 1FIA02_GFL_cplr1_win'.
> ../DataFile.cpp (105): Cannot reference '/arch/data/
> l0/2008/11_15/20081115-13'
> 11/20/2008 14:52:39 ArchiveServer ran 4.14733 seconds

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System