Experimental Physics and Industrial Control System

Hello.
I would like to share my experience on storing, processing and visualizing
time series data.
I am a physics graduate student from UIC (University of IL at Chicago).

The application.

http://131.193.191.37/~mnm/temperatures/menu.php

Let me explain what that is.
Our research  group has a beam-time this week at ChemMatCARS Sector-15 at
APS.
We wanted to look at the channels A and B of our own lakeshore 340 and
thermal probe connected to Keithley 2000.  Both connected to our laptop
via GPIB ? USB cable.
Using python with pyVisa we read those 3 variables and   write them  to
the MySQL database of the desktop that is located  at  UIC- that is the
link above. The connection to the UIC desktop is via the guest wirelesses
and SSH tunnel.
This web application is a stripped down version of something I am
currently developing for our in-house experiment at UIC that is being
controlled with  EPICS.

In general there is a need for visualization of real time data in the
scientific community.  However setting up and maintaining the
infrastructure is not a trivial matter. With this kind of centralized
system this problem could be significantly reduced.
For example for an EPICS environment such as the beamlines at APS the task
of visualizing small number of PV s could be reduced down to running a
simple python script on the client machine given that pyEpics installed on
the client machine.   The database server has to be setup only once at one
location.
My rough estimate is that a single modern  desktop system can serve tens
of clients simultaneously each individually recording at a rate of 10Hz.
The main requirement is fast hard drive array.

I have been thinking for a while starting a project on github to make this
web based application more general and would like to see if there is
anybody else doing something similar and would like to collaborate on
that.

One main distinctive feature of my database back-end is that I can process
tens of millions of records in as little as 2 milisecods and on average
less than 50ms which gave me solid  ground for building all the dynamic
interactivity on the front end.

Miroslav

Replies:

RE: Processing And Visualization of time series data. Chen, Xihui

Re: Processing And Visualization of time series data. Matt Newville

Navigate by Date:

Prev: Re: Re: Is it possible to create a cross-compile environment using cygwin and vxWorks on Windows XP 송영기

Next: EPICS and MEDM on ubuntu nalaka samarasinghe

Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  <2012>  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 

Navigate by Thread:

Prev: Re: Re: Is it possible to create a cross-compile environment using cygwin and vxWorks on Windows XP 송영기

Next: RE: Processing And Visualization of time series data. Chen, Xihui

Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  <2012>  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024