General Data Descriptor Library User's Guide


General Overview

The purpose of the General Data Descriptor Library (GDD library) is to fully describe, hold, and manage scalar and array data. Using a GDD, you can describe a piece of data's type, dimensionality, and structure. In addition you can identify the data with an integer application type value which can be translated into a text string. A user can store data into and retreived data from a GDD in any desired data type. A GDD can contain a list of GDDs. This allows users to create and describe hierarchical data structures dynamically. GDDs organized in this fashion are known as containers.

As mentioned above, a GDD can describe an n-dimension array. The user can describe the bounds of the n-dimensional array. To facilitate the use of large, and perhaps shared data arrays, a GDD allows a user to store a reference to an array of data. In addition, a destructor function can be registed in the GDD to inform the owner of the data array when the GDD referencing the data goes away. The purpose of the destructor function is to delete the array of data, since the GDD does not know what to do with referenced data.

To manage GDDs in a multi-tasking system or a system that uses many layers of software, GDDs implement reference counting. With reference counting, only one instance of a GDD can be shared by many subsystems. The GDD creator function can pass it to many other functions without worrying about deleting it, only when the last function using the GDD requests that it be destroyed does it really go away.

As menitioned above, a GDD allows the user to describe the data in terms of the application. This is done by the user by assigning an arbitrary integer identifier to a GDD. The user places a meaning on the identifiers such as 1=high-alarm-limit, 2=low-alarm-limit. This identifier is termed application type. A second component of the GDD library known as the Application Type Table is used to manage the application type identifiers. Application type values are registered in the table along with a text string and optionally a prototype GDD. The prototype GDD can be a container GDD. The table allows users to retreive GDDs of a specific application type.

A GDD describes and manages a piece of data using the following information:

The GDD library is a C++ class library and therefore requires using the C++ compiler.


GDD Description

The gdd class is the main class in the GDD library. It controls almost all actions performed on the data. The gdd class is further broken down into three subclasses for performing operations on specific types of GDDs: gddContainer, gddAtomic, and gddScalar. The gddContainer class aids in the creation of container type GDDs. It adds functions such as "insert GDD", "remove GDD", and "give me GDD x from the container" to the basic gdd class. The gddAtomic class helps create and manage array data GDDs. The gddScalar class makes it easy to create scalar GDDs.

All GDDs must be created dynamically, a GDD cannot be created on the stack as a local variable. The gdd class forbids the user from deleting the gdd. In order for reference counting to work correctly the user must "unreference" the gdd instance instead of deleting it. The gdd class does take over the memory management routines for itself and all it's subclasses. This means that you are not using the malloc()/free() memory management when GDDs are created and destroyed. It is important to remember that since reference counting is used, a GDD passed into a function must be referenced before the function returns if the GDD is to be kept for longer then the running of that function. In other words, if you are creating a library function "add" that records process variable names in a linked list, and the process variable names are passed to you as GDDs, then you must reference the GDD since the linked list exists after the return of the "add" function. If you are creating a GDD, you must unreference it when you are finished with it, even it you have passed it into other library functions. Generalizing on this, it is the responsibility of the GDD creator or GDD referencer to unreference the GDD instance when they are finished with it.

For a GDD to be useful after it is created, it must be given information to describe the data to will hold. This data was summarized in the overview section. To further understand the meaning of all these items and use them correctly, each needs to be discussed.


Primitive Types

As mentioned in the overview, a piece of descriptive information that a GDD maintains is the primitive type code. This field describes the format of the data (storage type). The primitive type field of a GDD is an enumeration of all the general storage types supported by the compilers. A user can determine dynamically what the storage format of the data in a GDD is using the primitive type code information. The GDD library redefines the general storage class names for integers and floating point numbers and enumerates them in the "aitTypes.h" header file. The redefined names describe the format and the bit length so that they can be used across architectures. The initial part of the header file name "aitTypes.h" is ait which stands for "Architecture Independant Types". The standard data types supported by the GDD library are
	aitInt8           8 bit character
	aitUint8          8 bit unsigned character
	aitInt16          16 bit short
	aitUint16         16 bit unsigned short
	aitEnum16         16 enumerated value
	aitInt32          32 bit integer
	aitUint32         32 bit unsigned integer
	aitFloat32        32 bit floating point number
	aitFloat64        64 bit floating point number
	aitPointer        Standard pointer
	aitIndex          32 bit index value
	aitStatus         32 bit unsigned integer for status value
	aitFixedString    40 byte string of characters
	aitString         Variable length string data type
	aitTimeStamp      Two 32 bit integers describing time (seconds/nanoseconds)
These data types should be used whenever possible to prevent problems when compiling programs for different architectures. Most of the data types described above are enumerated for use as a primitive type code. The enumerated names are just the above type names with the word "Enum" inserted after "ait". It should be noted that aitTimeStamp is not a standard primitive type.
typedef enum {
	 aitEnumInvalid=0,
	 aitEnumInt8,
	 aitEnumUint8,
	 aitEnumInt16,
	 aitEnumUint16,
	 aitEnumEnum16,
	 aitEnumInt32,
	 aitEnumUint32,
	 aitEnumFloat32,
	 aitEnumFloat64,
	 aitEnumFixedString,
	 aitEnumString,
	 aitEnumContainer
} aitEnum;
The enumerated type code allows a user to dynamically convert from one type to another. The AIT portion of the GDD library contains a large primitive type conversion matrix. The conversion matrix is indexed by the source and destination enumeration type codes. The 12x12 matrix contains functions pointers for each type of conversion that can take place. Generally this matrix is never accessed directly by the user, a convert function:
	void aitConvert(aitEnum dest_type, void* dest_data,
						 aitEnum src_type, const void* src_data,
						 aitIndex element_count)
runs the correct function to perform the calculation. This function is used extensively in the gdd class when data is put into or retrieved from a GDD.

The primitive type code really only describes the storage format and length of an element within a GDD. This code does not imply or assign any meaning to the data. Assigning meaning to the data is the job of the application type code.


Application Types

The application type is a 32 bit integer value that is user assigned. Each value is arbitrary and is designed to give meaning to the chunk of data. Normally each value has a character string associated with it which can be looked up through a standard hash table. The GDD library predefines many values and structures for standard control system applications; this portion of the library will be dicussed in detail later in this document.

A typical way that application type codes are used is in defining structures. A type code of 54 may be assigned the meaning "Temperature Reading". The "Temperature Reading" structure may be a container with four GDDs in it: a value, a high alarm limit, a low alarm limit, and units. Each of these GDDs also have an application type code. A generic program can actually be written to completely discover the contents of the "Temperature Reading" structure and configure itself to display meaningful data.


Time Stamp and Status

Each GDD can store a time stamp and status value. The time stamp is a standard 32-bit seconds, 32-bit nanoseconds since a standard epoch. The status is a 32-bit value that is user assigned. The purpose of the time stamp is to reflect the time when the data held within the GDD was read and stored. The status is intented to reflect the validity of the data within the GDD. In the context of EPICS, the status field is really a 16-bit status and 16-bit severity code.

Storing time stamp and status information in each GDD was really a processing versus storage compromise. Most data in a control system that is constantly changing and needs to be transfered in GDDs requires a status and time stamp. Transfering this type of data in one GDD in fairly easy. If GDDs did not contain time stamps and status, then this actively changing data would need to be transfered as a structure of three GDDs: one for the value, one for the status, and one for the time stamp. In addition to the three GDDs, a fourth that describes the GDD container will need to be transfered.


Dimension and Bounds Information

A GDD that is a scalar is self contained. All information to describe the data within the GDD is contained within the GDD. If the GDD describes a structure (a container), or the GDD describes an array, then dimension and array bounds information are needed. A GDD can describe an array of any dimension. Each dimension is required to have a description of it's size in terms of bounds.

Bounds, and consequencely a single dimension, in GDD are described by two fields, a start and an element count. With this setup, and n-dimensional space can be described. For example, a typical three dimension array would be described as A(5,4,6), which is a 5x4x6 cube. In a GDD this cube would be described with dimension=3, bounds={(0,5)(0,4)(0,6)}. If we want to describe a subset of this array, we would normally do so by giving two endpoints of the sub-cube, such as (1,2,3)->(2,3,5). In GDD terms, this would be bounds={(1,2),(2,3),(3,5)}.

The dimension information is stored directly within the GDD. The bounds information is not. If bounds are required, then they are allocated as a single dimensional array and referenced by the GDD. Methods exist in the GDD class to automatically manipulate, allocate, and deallocate bounds structures. Typically GDD are assigned a dimension when created and do not morph into a different space.


Data Field

A GDD contains a data field large enough to hold any scalar value or a pointer. This field is the union of all the primitive data types. Currently it takes up 8 bytes, which is the size of a double. If a GDD refers to an array of data, then the pointer to the actual data is stored in the data field. If the dimension is greater then zero, then the data field references the actual data.


Destructor

Since GDDs can reference arrays of data, the user can optionally register a destructor to be called when the GDD is destroyed. The GDD can store a reference to a user destructor. The GDD library contains a destructor base class called gddDestructor. A "destroy" method exists in this class that is called when the GDD is being destroyed. Users must derive a class from gddDestructor and define a "run" method. The "run" method gets invoked with the address of the data array. The user, in the "run" method, casts the address to the appropriate data type and deletes the array. The default behavior of the gddDestuctor if the user does not override the "run" method will be to cast the data array to a character array and delete it.

The gddDestructor allows reference counting similar to the GDD class. Typically a data array will be associated with one instance of the gddDestructor class. If more then one GDD needs to reference the array, then each GDD is registered with the same gddDestructor. Each time the gddDestructor is registered, the reference count should be bumped up. The gddDestructor "run" method will only be invoked when the reference count drops to zero.


Strings and Fixed Strings

Strings are special cases in the GDD library. There is a class called aitString designed to work with and manage strings. Strings is the GDD library generally contain two components: a reference to a character array and a length. Storing a string in a GDD always results in a reference to a character array, even if the GDD is a scalar. Character strings are first put into an aitString and then inserted into the GDD. The GDD data field can hold an instance of the aitString class for use when the GDD is a scalar with one string in it. A scalar string GDD still references the actual string within the aitString class.

A fixed string class exists in the GDD library in order to support certain features of EPICS. Fixed strings are used internally and should generally not be used when creating and maniplulating strings with the GDD library. Fixed strings are too big to fit into a GDD and also always referenced, even if the GDD is a scalar.


GDD Summary

Several important issues related to where the actual data is stored must be remembered when using GDDs:
  1. If a gdd is a scalar, then the gdd holds the data it describes, the dimension is zero, and the bounds are empty.
  2. If a gdd is an array, then the gdd refers to the data it describes, refers to bounds that describe it's structure, has a dimension greater then zero, and optionally refers to a user's data destructor.
  3. If a gdd is a container, then the dimension is fixed at one and the bounds describe how many elements (GDDs) are in the container. The destructor in the container case knows how to free up all the GDDs in the container.


Creating and Using GDDs

	#include "gdd.h"
	.
	.
	// my destructor ------------------
	class myDest : public gddDestructor
	{
	public:
		myDest(void) : gddDestructor() { }
		void run(void*);
	}

	void myDest::run(void* v)
	{
		aitInt16* i16 = (aitInt16*)v;
		delete [] i16;
	}
	// --------------------------------
	.
	.
	.
	int app_type_code = 100;

	// create a scalar GDD of type Int32 and put the value 5 into it
	aitInt32 ival = 5;
	aitFloat64 sval;
	gdd* dds = new gddScalar(app_type_code,aitEnumInt32);
	dds->put(ival);
	dds->getConvert(sval);
	dds->dump();

	aitString str = "test string";
	gdd* dd_str = new gddScalar(++app_type_code,aitEnumString);
	dd_str->put(str);
	printf("string length = %n",str->length());
	dd_str->dump();

	// create an array GDD and of dimension 1 and bound of 20 elements
	// reference the array into the container
	aitUint32 tot_elements = 20;
	int dim = 1;
	aitFloat64 a[20];
	gdd* dda = new gddAtomic(++app_type_code,aitEnumFloat64,dim,&tot_elements);
	dda->putRef(a);
	dda->dump();

	aitInt16* i16 = new aitInt16[tot_elements];
	gdd* ddb = new gddAtomic(++app_type_code,aitEnumInt16,dim,&tot_elements);
	ddb->putRef(i16,new myDest);
	ddb->dump();

	// create a container GDD that holds to GDDs and put the previously
	// created GDDs into it.
	int tot_in_container = 2;
	gddContainer* ddc = new gddContainer(++app_type_code,tot_in_container);
	ddc->insert(dds);
	ddc->insert(dda);
	ddc->dump();

	// clean up the container GDD, this will clean up all member GDDs,
	// myDest run() should also be invoked
	delete ddc;
	.
	.
	.


Application Type Table Description

A facility within the GDD library is designed to store application type code to string mappings. The facility can also be used to store and retrieve prototype GDDs by application type code. The application type table is really a simple database containing records with the following information: The type table has methods for registering or inserting records into the database. Methods also exist to query the type name given a type code, and to query the type code given the type name. Type codes are values assigned to a type name when the type name is registered; the user is given the type code value when the registeration takes place. A type code sometimes has a prototype GDD associated with it, especially if it is a container GDD. Methods also exist to retrieve a new GDD given an application type code. The new GDD structure will be a copy of the prototype GDD for that type code. The application type table is commonly used create GDDs. A given application type code usually implies a certain structure to the data. The prototype mechanism allows this imposed structure to be adhered to when the GDD is created. This is particularly important when it comes to container type GDDs. The type table basically copies the prototype when the user requests a GDD with a particular type code. The application type table also packs GDDs in an efficient manage, which allows very high performance creation and deletion of container GDDs and atomic GDDs.

GDDs in general have several components that are references, they do not hold all the information they need. Creating and using array GDDs or container GDDs can be very time consuming. For an array GDD, the bounds and a destructor must be allocated in addition to the GDD itself and referenced into the GDD. For a container GDD, the GDD elements are really stored as a linked list. To access the third element of a container, the linked list must be traversed. With aitString GDDs the situation is worse yet. The GDD class allow for a given GDD to be packed or flattened into a single linear buffer. This mechanism includes packing GDD containers. In the array case, the actual GDD is the first thing in the buffer, followed by any bounds information, followed by the actual data array. The fields of the GDD work exactly as before, except that they reference bounds and data that is in the same buffer. In the case of containers, all the GDDs are stored as an array of GDDs at the front of the buffer, followed by the bounds information for each of the GDDs, followed by each of the GDD's data arrays if present. There are many advantages of this configuration. Since all the GDDs in a container are stored as an array instead of a linked list, the user can directly index a particular GDD within the container. The application type table performs this packing on a GDD that is registered as a prototype for a particular type code. Since the GDD for a given type code is now a fixed size, the type code table can manage the GDDs on a free list. Each type code database entry with a prototype GDD contains a free list of GDDs for that type code. Creating a GDD using the type code table involves retrieving a preallocated, packed GDD from a particular free list and giving it to the user. This operation is very fast and efficient since complex GDD structures are already constructed.

As stated above, container GDDs managed by the application type table can be directly indexed as an array by the user. Unfortunetly the application type code and indexes have no correlation - you cannot use the application type code to index the GDD container. The type table has mapping functions that convert between an application type code and an index into a container GDD. Of course each type code maintains it's own index map and the container GDD type code determines which mapping is to be used. A mechanism exists in the GDD library for generating "#define" statements that label container index values with a unique string. Preregistered containers use this mechanism to generate indexing labels. The index label is a concatenation of the type code names. If a container GDD type code name is "TemperatureReading" and the first element is "Value", then the index label generated will be "TemperatureReading_Value". At any time in a running application, the user can request that index labels for all registered prototypes be generated and dumped into a file. The library preregisters a number of application type names:

In addition, if EPICS is used then the following are registered: The file gddApps.h contains all the index label defines for the EPICS related GDD containers.


Using the Application Type Table


GDD Reference Manual


Home page for Jim Kowalkowski.

Argonne National Laboratory Copyright Information
jbk@aps.anl.gov (Jim Kowalkowski)
updated 9/13/96