Implementation(s) of the canSAS2012 data format

Describe why we have multiple implementations (small data community demands simple, readable text while large data community demands efficient binary storage). Describe that canSAS must let the community decide. (Largely based on the decisions of data suppliers and analysis software creators.) Describe here some general information about the implementation(s). Describe what is common amongst the implementations and what is not.

Contents:

• Text: XML
• Binary: HDF5

Algorithm for Software to Read Data files Written with this Structure

1. open the file and read the SASroot (or root-level) group.

2. open each SASentry group:

   1. verify the version attribute: must be “1.0” (a string)

   2. note the name attribute, if present

   3. note the title, if present

   4. note the SASsample group contents, if present

   5. open each SASdata group

      1. note the name attribute, if present
      2. read the I_axes attribute, it tells the names (in storage order) of the other data objects on which I depends.
      3. read the Q_indices attribute, it tells which of the I_axes (zero-based numbering) have Q dependence
      4. if present, read the Mask_indices attribute (similar to the Q_indices), to get the dependencies of the Mask.
      5. read I data object as described below (see Read a data object)
      6. same for Q, Qx, Qy, Qz data objects (if present)
      7. same for the names in the I_axes attribute (except for Q since it was already handled)

Read a data object

1. read the size attribute, if present
2. allocate memory for the object
3. read the data object and its units
4. if present, read the uncertainty attribute and read the named data object it describes

Algorithm to Identify \(Q\) values given a set of indices on the I data

Given an intensity data object (called I), the algorithm to identify the associated Q values with any given intensity datum is described here:

1. Identify the given set of indices with the names of data objects

   1. note the set of indices for the intensity datum
   2. note the names of the various intensity axes from the I_axes attribute

2. note the list of indices for Q as given by the Q_indices attribute

3. ... this gets tedious without a couple examples...

4. analyze how we do it below and finish writing this part

Caution

This write-up is unfinished at this point. Follow below for a few examples.

simple time-series example

Simplified example

To simplify our example, only the relevant metadata has been left in this example.

Consider the SAS data example including a time-series (same model as 2-D ):

SASroot
  SASentry
    SASdata
      @name="sasdata01"
      @I_axes="Time,Q"
      @Q_indices=0,1
      Qx : float[4,35]
      Qy : float[4,35]
      Qz : float[4,35]
      I : float[4,35]
      Time : float[4]

1. The I_axes attribute describes a Time data object, in addition to some Q data. The Time index is in the first position.
2. It also says that there is only one index to use (the second index on intensity) when looking up a \(Q\) value.
3. Since there is no Q data object, there must be three data objects Qx, Qy, Qz that provide the scattering vector.
4. The Q_indices attribute indicates that the lookup of Q depends on both the Time (0) and Q (1) indices and that Q is time-dependent.
5. One index (position 0) is used to lookup the Time value.
6. index i is for Time and index j is for Q.

Given the indices i,j, return all the data for this datum:

Qx[i,j], Qy[i,j], Qz[i,j], Time[i], I[i,j]

simple time-series example

Consider the example of the 2-D time-dependent masked image.

Given the indices i,j,k, return all values for this datum:

Qx[i,j,k], Qy[i,j,k], Qz[i,j,k], Time[i], Mask[j,k], I[i,j,k]

another example

See the model for 2-D \(I(T,t,P,Q(t))\) images.

Given the indices i,j,k,l,m, return all values for this datum:

Qx[j,l,m], Qy[j,l,m], Qz[j,l,m], Temperature[i], Time[j], Pressure[k], I[i,j,k,l,m]