Notes on Statefile Formats
Comparing proposed statefile syntax to alternatives
- Statefile Format (.sff):
window1.subwindow2 # End-of-line comments are always welcome!
. . xaxis.label.text "Frequency (MHz)"
. . . . position –0.5
. . . . font.size 12, family arial, color blue, bold True
- ONX Equivalent (.onx):
:window1{:subwindow2{
:xaxis{:label{:text["Frequency (MHz)"]
:position[“-0.5”]
:font{:size[“12”] :family[“arial”] :color[“blue”] :bold[“True”]}
}}xaxis
}}window1
Note: No comments or leading dots. Have to be careful about { }.
- YAML Equivalent (.yml):
window1:{subwindow2: # End-of-line comments are always welcome!
xaxis:{label:
text: "Frequency (MHz)"
position: -0.5
font:{ size: 12, family: arial, color: blue, bold: True }
}
}
Note: YAML syntax is tricky, and I’m not sure I’ve got this right. We’ll need to have some discussion with the YAMLers, if we decide to use this format.
- XML Equivalent (.xml):
... big mess of header specs ...
. . . . . <Property NAME="size">12</Property>
. . . . . <Property NAME="family">arial</Property>
. . . . . <Property NAME="color">blue</Property>
. . . . . <Property NAME="bold">True</Property>
Note: I’ve added the dots for clarity. In a real XML file, all we would see is a vast expanse of white space in which we can draw lines with a ruler.
Can the XML be simplified to this?
. . . . . <size 12/>
. . . . . <family arial/>
. . . . . <color blue/>
. . . . . <bold True/>
I know very little about XML, but I know that machine-generated files are often not as simple as they could be.
- Python Equivalent (.py):
class wavescan:
class window1:
class subwindow2:
class xaxis:
class label:
text = "Frequency (MHz)"
position = -0.5
class font:
size = 12
family = 'arial'
color = 'blue'
bold = True
- A Python Dictionary: (alternative to deeply nested clases)
{'analyses': \
{'transient': \
{'start' : '10u', \
'stop' : '20u', \
'method': 'gear2', \
'skipdc': 'yes', \
'readic': 'final50u' \
}}}
The problem with using a dictionary is that accessing parameters deep in the hierarchy requires some tedious typing of special characters:
> x['analyses']['transient']['method']
'gear2'
A deeply nested class variable is much easier to access:
> wavescan.window1.subwindow2.xaxis.label.font.family
'arial'
The difficulty of typing a reference to the dictionary item may be an issue only for programmers, as we can make the user interface (script language and statefile format) independent of the internal representation of the data.
My worry is that there may be some large overhead with hundreds of deeply nested classes. We need to try a test with a class hierarchy having 1000 parameters in 300 classes up to ten levels deep. What is the cost in memory and in time to read and write the statefile?
Is there a better alternative to nested classes for our internal data structures?
SUMMARY
The current situation with EDA tools is chaos, with every tool having its own statefile format, most of them not humanly readable.
XML is a widely accepted standard with fully developed tools for parsing the syntax, display and editing the parameters, etc., but an XML file is not easily readable by humans.
ONX is almost as readable as our proposed statefile. ( It lacks dots and comments.) It has a parser written in C, which we might be able to use. It looks like it has had no support or further development in the 10 months since it was introduced. http://www.seairth.com/web/onx/index.html
YAML is almost as readable as our proposed statefile, and has many more capabilities than we need right now, or think we will need in the future. It is a simpler standard than XML, but the tools are not yet developed [1]. However, it looks like there is sufficient interest that we can expect this to be a well-supported standard. http://yaml.freepan.org/index.cgi?HomePage
Our proposed statefile syntax is the ultimate in readability for the intended task of storing large, deeply nested collections of statefile parameters, but we will have to write our own routines to read and write these files to and from our internal Python structures. Also, if there ever is a need to expand the capabilities of this format to tasks other than what we now anticpate, we may wish we had used YAML or XML.
The readability advantages of our statefile syntax over other formats are:
- End-of-line comments.
- Leading dots help in reading deeply nested hierarchies.
- Line continuation with + signs in column 1.
Performance may be an issue with some of these formats. We need to test loading and saving a statefile with 1000 parameters in 300 nodes, 10 levels deep.
NOTES:
[1] Brian Ingerson is working on an implementation of YAML in Perl, which he says is near finished and can be easily ported to Python.
module progess difficulty
------
YAML.pm 95% easy
YAML::Parser 3% hard
YAML::Loader 80% easy
YAML::Dumper 90% easy
YAML::Emitter 5% hard
YAML::Node 95% easy
http://sourceforge.net/mailarchive/message.php?msg_id=6483671
I’m not sure what the difference between a loader and a parser is, but this looks like it is worth checking out. Also, we may be able to use a partial implementation, without all the features in the YAML spec.
CDP\Statefiles\format_comparisons.doc 3 DMQ 11/16/03