Chapter 16. Serialization
The term serialization refers to the ability to save more than one related object to either persistent data storage (such as a file) or to send an object over a network stream, such as TCP/IP. When an object is saved to disk (or sent over a `wire') we record a reference to the saved object, so that the original object can be restored at a later date. This reference is referred to as a `serial number', hence the term `serialization'.
Swarm has two forms of support for serialization:
Lisp. Lisp serialization reads and generates human readable text-file in Lisp format. This form of serialization is well-suited to applications that require either a human generated text file to create object parameters (such as simulation parameter files), or require a human-readable output.
HDF5. HDF5 is high density binary data storage format created by NSCA. HDF5 serialization is well-suited to applications that involve reading and/or saving large data sets. It is a database-oriented format which a number of third-party tools (such as the R statistical package which is a freely-available clone of SPlus) can read.
16.1. Using the LispArchiver to manage simulation parameters
 | Earlier versions of Swarm used the protocols ObjectSaver and ObjectLoader to read/write object state to disk using an ad-hoc file format. These protocols only partially implemented the saving of certain types and the continued use of these protocols is now officially deprecated and may go away in future releases. |
16.1.1. Using the Standard lispAppArchiver
Every Swarm application comes with a singleton [1] instance variable for reading object data formatted in Lisp. This instance is called lispAppArchiver. Like the probeLibrary and arguments instances, it is global to your entire application. This instance expects to find a file called appName.scm [2]: , in either the current datapath for the application ( SWARMHOME/share/swarm/appName ) or in the local directory. Using this variable obviates the need for hand creation of LispArchiver instances. It permits one datafile (which can contain as many keys to objects as is required), and imposes a naming convention for that datafile.
Example 16-1. Using a standard lispAppName instance
The heatbugs application uses this global singleton class. The datafile heatbug.scm looks like this:
(list
(cons 'batchSwarm
(make-instance 'HeatbugBatchSwarm
#:loggingFrequency 1
#:experimentDuration 200))
(cons 'modelSwarm
(make-instance 'HeatbugModelSwarm
#:numBugs 200
#:minIdealTemp 10000
#:maxIdealTemp 20000
#:minOutputHeat 10000
#:maxOutputHeat 20000
#:randomMoveProbability 0.0))) |
The Lisp file consists of two `keys' or `serial' numbers batchSwarm and modelSwarm for the parameters for two different objects. These keys are completely at the discretion of the user to choose. (Note also that the file syntax allows Lisp-style comments: a `;' colon followed by any text).
This input file would correspond with the following interface files, for the HeatbugBatchSwarm class we have HeatbugBatchSwarm.h
@interface HeatbugBatchSwarm: Swarm
{
int loggingFrequency; // Frequency of fileI/O
int experimentDuration; // When to Stop the Sim
id displayActions; // schedule data structs
id displaySchedule;
id stopSchedule;
HeatbugModelSwarm *heatbugModelSwarm; // the Swarm we're observing
// The EZGraph will be used
id unhappyGraph; // in FileI/O mode rather
// than the usual Graphics
// mode...
}
// omitting methods |
For the HeatbugModelSwarm class, HeatbugModelSwarm.h:
@interface HeatbugModelSwarm: Swarm
{
int numBugs; // simulation parameters
double evaporationRate;
double diffuseConstant;
int worldXSize, worldYSize;
int minIdealTemp, maxIdealTemp;
int minOutputHeat, maxOutputHeat;
double randomMoveProbability;
BOOL randomizeHeatbugUpdateOrder;
id modelActions; // scheduling data structures
id modelSchedule;
id heatbugList; // list of all the heatbugs
id <Grid2d> world; // objects representing
HeatSpace *heat; // the world
}
// omitting methods |
Note that for each instance variable name of the form #:ivarname somevalue in the Lisp parameter file there exists a corresponding instance variable in the class header file. However, not all instance variables in the header file have corresponding entries in the Lisp parameter file. This is because the other instance variables are either unimportant as parameters (i.e. they can be regenerated by other parameters), or they are instance variables that pertain to the running model itself (such as modelSchedule, which is a Schedule instance).
To generate the objects with these corresponding parameters set in each object, you need the request the global lispAppArchiver archiver to `generate' an instance of the object using the appropriate `key'. So here's an excerpt from main.m:
if (swarmGUIMode == 1)
{
// Do GUI mode creation (omitted)
}
else
// No graphics - make a batchmode swarm (using the key
// `batchSwarm' from the default lispAppArchiver) and run it.
if ((theTopLevelSwarm = [lispAppArchiver getWithZone: globalZone
key: "batchSwarm"]) == nil)
raiseEvent(InvalidOperation,
"Can't find the parameters to create batchSwarm");
[theTopLevelSwarm buildObjects]; |
Note that you still pass the globalZone Zone instance to the getWithZone:key:, as you would if you were using the standard create: functions.
The key thing to realize here is that the getWithZone:key: call actually instantiates the object (i.e. automatically runs the createBegin/createEnd apparatus internally[3]). This has implications for the design of parameter files, since it means, for one thing, that all the appropriate instance variables necessary for a complete creation of an object must be present in the input Lisp file. It is possible to have a subset of ivars, but that subset should be sufficient to completely specify the object, i.e. no CREATE time messages can be sent to the object once it has been created. (Of course you can still send SETTING or USING messages to instance once it has been created).
The HeatbugModelSwarm is created in a similar way, from the buildObjects method in HeatbugBatchSwarm.m:
// Create the model inside us - no longer create `Zone's explicitly.
// The Zone is now created implicitly through the call to create the
// `Swarm' inside `self'.
// But since we don't have any graphics, we load the object from the
// global `lispAppArchiver' instance which is created automatically
// from the file called `heatbugs.scm'
// `modelSwarm' is the key in `heatbugs.scm' which contains the
// instance variables for the HeatbugModelSwarm class, such as
// numBugs etc.
if ((heatbugModelSwarm = [lispAppArchiver getWithZone: self
key: "modelSwarm"]) == nil)
raiseEvent(InvalidOperation,
"Can't find the parameters to create modelSwarm");
// Now, let the model swarm build its objects.
[heatbugModelSwarm buildObjects]; |
Note that after the creation of the heatbugModelSwarm instance, it responds in the normal way to valid methods, such as buildObjects.
16.1.2. Using Custom LispArchiver Instances
This section addreses those situations that require custom creation of multiple data files or alternate data filenames.
Example 16-2. Creating a Lisp parameter file with an alternate name
Here is a sample Lisp input parameter for the Mousetrap simulation, batch.scm.
(list
(cons 'batchSwarm
(make-instance 'MousetrapBatchSwarm ; parameters for the batchSwarm
#:loggingFrequency 1))
(cons 'modelSwarm
(make-instance 'MousetrapModelSwarm ; parameters for the modelSwarm
#:gridSize 40
#:triggerLikelihood 1.0
#:numberOutputTriggers 4
#:maxTriggerDistance 4
#:maxTriggerTime 16
#:trapDensity 1.0))) |
The Lisp file consists of `keys' or `serial' numbers batchSwarm and modelSwarm identical to heatbugs
This Lisp input file has variables listed the following interface files (not shown) MousetrapBatchSwarm.h and MousetrapModelSwarm.h, for the MousetrapBatchSwarm and MousetrapModelSwarm classes.
The only difference with the previous example, is that we explicitly create an instance of the LispArchiver with the named file, and then ask the archiver to `generate' an instance of the object using the appropriate `key' as per the previous example. So here's the relevant excerpt from main.m:
// create an instance of the LispArchiver to retrieve the file
// set the path to `batch.scm'
id archiver = [LispArchiver create: globalZone setPath: "batch.scm"];
// retrieve the object from the archiver, if it can't be found
// just raise an event; note that the call to the
// archiver will actually *instantiate* the object if the
// parameters are found in the Lisp file
if ((theTopLevelSwarm =
[archiver getWithZone: globalZone key: "batchSwarm"]) == nil)
raiseEvent(InvalidOperation,
"Can't find archiver file or appropriate key");
[archiver drop]; |
The MousetrapModelSwarm is created in a similar way, from the buildObjects method in MousetrapBatchSwarm.m:
// create the instance to read the file
archiver = [LispArchiver create: self setPath: "batch.scm"];
// * `modelSwarm' is the key for the instance of the MousetrapModelSwarm
// with parameter values for the model instance variables: gridSize
// triggerLikelihood, numberOutputTriggers, maxTriggerDistance,
// maxTriggerTime, trapDensity
// if we can't find the right key from the LispArchiver, raise an event
if ((mousetrapModelSwarm =
[archiver getWithZone: self key: "modelSwarm"]) == nil)
raiseEvent(InvalidOperation,
"Can't find archiver file or appropriate key");
// don't need the archiver instance anymore
[archiver drop]; |
Note that when you have called the archiver instance to instantiate all the objects of interest, you have no need of the archiver instance and you can safely drop it.
Note also that, although the only difference from the previous example, is the name of the file does not conform to the appName.scm convention, but in principle the two keys could have been in different files, in which case in would have not been possible to use the global lispAppArchiver instance.