Moving Data Between Tiers, Part 1
Erik Moore
One of the most disconcerting aspects of n-tier development is moving data between tiers. In this series of articles, Erik Moore slaps you in the face with a harsh new reality, and then gently leads you by the hand into a new world.
A developer's first foray into n-tier development often goes like this:
1. Lay out the architecture on a piece of paper. 2. Build a few data access classes and business classes.
3. Compile the classes into a COM DLL.
4. Test the business object and data access object from the Command Window.
5. Scratch your head and wonder why VFP can't see the tables opened in the methods of the COM server.
6. Post a question on a technical forum asking why the results of a query executed inside the DLL can't be seen with a BROWSE.
7. Realize, as a result of the answer, that cursors don't traverse COM boundaries.
A rude awakening
This tiny technical detail is often a huge rude awakening for the novice n-tier application builder. Through all the previous paradigm shifts in Fox development, the programmer has always been able to program directly against his or her data. In other words, changing the value of a field was as easy as:
USE MyTable
REPLACE MyField WITH "SomeValue"
USE
And, it was that simple. Even if you were already neck-deep in object-oriented programming and had objects finding and opening your tables for you, even then you could at least see the data after it was opened.
Things got a step more complex with client/server programming. Classic client/server in VFP generally had the programmer using either remote views or SQL pass-through (SPT) to retrieve data. Both of these methods are provided natively in Visual FoxPro, and both result in a cursor that can be browsed and manipulated. With remote views, all you had to do was call TABLEUPDATE(), and the data was sent back to the back-end database. Even cursors created with SQL pass-through can be configured to save their changes automatically. So, even though you didn't USE a table to get your data, it still looked and felt the same.
All this changes with n-tier.
Inter-application communication—a new frontier
Generally, when we talk about n-tier in the Windows world, we mean that objects (more specifically, COM objects) are accessing, serving, accepting, validating, and saving our data.
A COM-aware client can discover, instantiate, and manipulate any object built to comply with the COM standard. This brings us a great deal of flexibility in the Windows programming world. If you've ever automated Word or Outlook, used an ActiveX control on a form, or used MAPI to send e-mail, then you were using a COM object. As far as VFP was concerned, the object could have been compiled in COBOL. (I have no idea whether modern versions of COBOL can build COM objects, but the point is that the caller has no idea what language the server was written in—it knows only that the object has a COM-compliant interface.)
COM brings us Windows-wide portability for our objects, but this portability comes at a cost. When designing the COM architecture (and its predecessors and successors), the designers had to agree on a set of standards for all COM objects. This meant, of course, necessary interfaces to support registration and discovery. But, more importantly, it meant agreeing on a set of COM data types.
Though there are a lot of COM data types, and even one called "variant" for data that doesn't have a predefined type, the COM designers weren't generous enough to give us a "cursor" data type. This is understandable, I suppose—not all (or even most) development tools deal with cursors as a concept. As a matter of fact, the cursor as we use it in VFP is more or less a uniquely Xbase concept. (Just in case you've been living an extremely sheltered life and weren't aware of it, FoxPro isn't the only development tool out there and—gasp!—isn't even the most commonly used.)
So, instead of trying to add every conceivable type of tool-specific data type to the specification (such as types for e-mail messages, URLs, formulas, RTF docs, and so on), the COM designers simply gave us enough data types to get by on, and made those types flexible enough to carry any type of information. Among the myriad of COM data types are a string type, a couple of integer types, two different currency types, a date type, and even a multidimensional array type.
For database operations that need to retrieve only a single value, the basic COM data types are easy to work with. An operation, for example, that needs only to retrieve the balance of a bank account doesn't need to return the entire customer record, and can therefore use one of the simple data types.
Moving cursors across the COM boundary
Believe it or not, we can get by quite nicely without a data type for cursors. A couple of the COM data types (string and object, to be specific) are quite flexible and, with a little work, can be used to convey just about any data you can imagine, including our beloved cursor.
An n-tier system will typically consist of multiple tiers, obviously. These tiers can be categorized as logical tiers or physical tiers, and most often, there's a difference between logical tiers and physical tiers. Some applications might be suitable for the various tiers to be physically combined into the same EXE, instantiating normal VFP classes instead of VFP COM objects. In those cases, some of the issues discussed here don't apply in the same way, but your deployment options are also much more limited. However, this article's discussion will assume that we're dealing with a fully distributed and flexible architecture, which means our data must cross one or more COM boundaries.
Knowing now that you can't just open or create a cursor in a middle-tier object and access it in the presentation tier, you're going to have to figure out how to fit a cursor into a data type that COM understands. Of the types COM offers us, two are best suited to contain the information needed to convey a recordset: strings and objects.
Strings are suitable because of their sheer flexibility. On the other hand, objects are great because they can contain their own properties and even contain child objects. Each of these types has advantages and disadvantages, as you'll soon see.
Strings
Strings can hold all kinds of stuff, including binary data, numeric values, and dates—you name it. A string is basically a series of single-byte characters (or, in the case of Unicode strings, two-byte characters) that are "strung" together. Let me wander away from our main topic for just a moment to illustrate a very important concept: You can store almost anything in the space of a string.
An example of string flexibility
The flexibility of strings is demonstrated in one instance by how VFP programmers deal with VFP's lack of native support for structures—custom-defined data types often used in Windows API programming. A structure consists of two or more "members" of possibly different data types, stored contiguously in memory. It can be passed around and used by functions that understand what type of structure it is.
For example, if you want to use the Windows API to retrieve the current time from the system, you'd use the GetSystemTime function, which takes one parameter (a string), passed by reference, and returns nothing. However, after the function is called, the string that was passed in will now contain the contents of a SYSTEMTIME structure defined in the winbase.h include file:
typedef struct _SYSTEMTIME {
WORD wYear;
WORD wMonth;
WORD wDayOfWeek;
WORD wDay;
WORD wHour;
WORD wMinute;
WORD wSecond;
WORD wMilliseconds;
} SYSTEMTIME;
This may look like Greek to you (unless you're Greek), but what it says is that the SYSTEMTIME structure consists of eight members (of data type WORD), each two bytes long, representing the year, month, day of the week, day, hour, minute, second, and millisecond. After calling a function that returns a SYSTEMTIME structure named "Systime," a language that supports structures natively might use its own functions to access each part of the structure individually, perhaps something like this:
NDay = Systime.Day
But, if a VFP programmer wants to use this information, he or she needs to know how to: 1) parse the structure into its various parts; and 2) convert the WORD data type into an integer.
The fact that this is even possible in VFP demonstrates that a string can easily contain a complex data type. This isn't an API programming article, so I won't go into details on how to parse and convert the various parts of a SYSTEMTIME structure. I included this example simply to demonstrate a basic point—if everyone agrees on the "structure" of a string and knows how to break it apart into its "members," strings can be used to convey just about any complex data type you can imagine.
How do you force a cursor into a string?
If you've decided to use a string to carry your tabular data, the question now becomes, "What's the best format for encoding a cursor (a single record, or a set of records) into a string?" The answer, of course, is that it depends.
Early papers on the subject proposed what now seem like barbaric schemes. We were told to transform each field into a string, concatenate them all together (possibly with a delimiter), come up with a proprietary table structure map, and tack them all together, end on end—sort of like a glorified comma-delimited file. Then, the client-side code would do the reverse: find the structure definition and parse the delimited data out into its various fields and records. If you're thinking that this seems like a lot of work, you're right. Both the encoding and the decoding algorithms can get pretty complex, and this kind of work would get old extremely fast. What's needed is a generic encoding and decoding algorithm that works for any cursor.
XML to the rescue
Luckily, there's a text technology that's perfectly suited to conveying complex data, both in the VFP world and outside of it. It's flexible and self-describing, comes with built-in typing mechanisms, and is even easy for humans to read. You guessed it—I'm talking about XML (eXtensible Markup Language).
Not long after most developers were getting comfortable with HTML (the language of the Web), the voices from above and the trade magazines started buzzing about XML. Some magazines claimed that Web pages would soon be using it instead of HTML. This hype created a lot of confusion, and had many developers wondering why XML was so much better. The answer? It isn't. It shouldn't be compared to HTML, because they're not used for the same thing.
HTML was designed for one thing, and that's marking up Web pages. It has a full set of predefined elements that browsers understand how to display. The HTML specification provides a few elements (<FORM>, <INPUT>) that allow pages to submit data to a Web server. Filling out a form on a Web page with several input fields can result in a string being sent to the server that might look like this:
Fname=erik&lname=moore&pmtamt=
100.00&delivdate=01/01/2000
As you can see, this "encoding" scheme is quite simple. Name-value pairs are denoted with an equals sign and are delimited by an ampersand. An HTML form can send a whole slew of data to the server. But these elements are limited in several ways. First, there's only one possible type—string. Second, there's no way to send a "set" of data (duplicate field names will result in one of the values being ignored), or to convey concepts like arrays.
The problem, as mentioned earlier, is that HTML was originally designed to communicate "markup." Data submission was an afterthought. Sure, it can display an orderly looking table of data in a TABLE element, but try parsing that element on the client side to retrieve meaningful stuff. It can be done, but the code ain't pretty.
This weakness in HTML was exactly the hole that XML was designed to fill. Imagine HTML as the pictures in the book, and XML as the words. HTML was designed for humans to consume, and XML was designed for computers to consume. As an example, if I write a Web page that's supposed to communicate an address, the HTML code might look like this:
<HTML>
<BODY>
<H1>Address</H1>
<P>
123 Elm St<BR>
Anytown, TX USA<BR>
78717<BR>
</BODY>
</HTML>
Display this in a browser, and you'll get a document that's easily readable by humans (see Figure 1).
Figure 1. A simple browser display of the address information, created by HTML code. While this could be parsed to separate out the various fields, that could be difficult, especially if the address format varied from one address to another.
The same document might be written in XML like this:
<?xml version="1.0"?>
<address>
<street>123 Elm St.</street>
<city>anytown</city>
<state>TX</state>
<zipcode>78717</zipcode>
</address>
This XML could then be turned into an HTML display with an XSL stylesheet, or it could be parsed and the values easily identified and read by a program for whatever use was needed. Identifying the fields from the HTML only would be much more difficult and, in some cases, impossible.