Extending API, structures and so on

Once upon a time there was an Operating System called Amiga. At some step of its development this system started to use a way to unbind the latches of the obsolescence of APIs or data structures that so easily make the code incompatible from a past release of the O.S. to the next one.

The idea is easy (I don't know if it is original): do not pass pointers to structures that may change. Instead, give to the function a pointer to a list of tag–data pair. The function, using some tool provided by a system library, search for the tag it understands and fill in the private structure, if any.

The access point to that function can be kept the same even though the function itself is enhanced. No need to define a new API call when interested structures grow.

Let us make an example, just to show the idea. Imagine a graphics function that is used to draw a line. In order to do so, we must pass in a graphics context (an object returned by a function), the coordinate of the first point and of the last point.

The function (C-style prototyping) could be: DrawLine(GC *gc, uint16_t sx, uint16_t sy, uint16_t dx, uint16_t dy). Or we can define a structure that represent a point: DrawLine(GC *gc, Point2D *from, Point2D *to). This is fine because this atomic function hardly will grow in some way (graphics context holds information about the color of the stroke, where to stroke and so on)

But let us suppose that now our API wants to become more complex. Here there's no real need, but just to understand the problem that we need more information to be provided, and this information for some reason is not related to the graphics context, so it can't be kept in this object (that we imagine as a black box). But what we want to add is someway related to the starting and ending points.

We have two main path:

add one or more argument to the API. This changes the API and breaks old programs; if we want to keep the compatibility, we must provide a different name, e.g. NewDrawLine.
change the structure Point2D to hold more information; old programs do not initialize new data inside that struct, and so their behaviour in the new environment is unknown. New program in old environment should work (it happens in our example, it is not sure it is always through —it is false e.g. when the program thinks the function modifies someway the content of the struct), even though the new information passed are ignored. Again we must give a NewDrawLine and the old DrawLine becomes a sort of wrapper; so old program can run and the interpretation of Point2D as the new kind of extended struct is avoided.

The new API prototyping could be NewDrawLine(GC *gc, Point2D *from, Point2D *to, MoreDrawInfo *info) and similar if we use the uint16_l sx... approach; or NewDrawLine(GC *gc, Point2D *from, Point2D *to) if we expand the structs.

A long-term O.S. project maybe planned in the old version a prototype like DrawLine(GC *gc, Point2D *from, Point2D *to, DrawExp *exp), and the docs of the old version say to keep the DrawExp pointer to NULL. The new API will pretend a pointer if we want to use the extension, or NULL if we want to forget it. This keeps the same API and doesn't break compatibility. But what about the next next next versions? The DrawExp struct can provide a pointer to another extension... and so on.

We can imagine several scenarios, e.g. the one that leads to object oriented programming (GC is an object, Point2D is an object... we can extend their properties without worrying too much; the new properties bound to the Point2D objects are ignored by old apps, and are set by new one through a method).

But a good way to keep the OO far and rely on classical API could be the tag solution. We pass to the function only one single pointer to a null-terminated list of tag–data pairs. Each tag and each data are 32bit wide integers (if we need pointer as datum, a 32bit address space should be enough for every application! will it be so in the future? 64bit machines are real, and 2 Gigabytes of RAM are possible... what would it happen when the 4Gigabyte threshold will be surpassed? that day we will be sure of the fact that our way of thinking about home computers and their needs to finish a home task is ill or, better, poisoned by the needs of market!)

The past and present API will be DrawLine(TagList *tl). Some tags can be mandatory (here, the GFX_GC tag, used to pass the graphics context pointer, is mandatory), and the optional tags that are not specified gives default values/behaviour.

A call to the DrawLine function (using a variable argument C facility) would look like this:

   DrawLine(GFX_GC, &gc,  // mandatory
            GFX_FromX, sx,
            GFX_FromY, sy,
            GFX_ToX, ex,
            GFX_ToY, ey,
            TAG_END);

(TAG_END is a special tag value that marks the end of the list) If we need to add some information, we define a new tag (with a default, as usual). The API does not change. Old environment will safely ignore unknown tags, and new environment will provide a default value for unspecified tags.

From one side the situation is the same when we change the Point2D struct. But here we don't need to change the API (the access point, if you prefer...). Our old application will run fine in the new environment. New application can run safely in old environment (in this example; we can imagine there are scenarios where the back compatibility cannot be kept; anyway the tag-approach makes it easy to expand API)

We can say that our function inherits something of the object oriented approach. In fact we can use the tag–data idea as infractusture to an implementation of a OO system.

Another thing that can be noted is that we can also change the way we specify the points; we can pretend new code uses, say, the GFX_Point2D generic (to all graphics functions) tag, that has as data a pointer to a two-interger long array (the X and Y coord) or structure, if you prefer. We can forget the From/To difference: first specified point is from, next one is to. Moreover... if more than one pair is specified, we can iterate so that we can draw several segments, improving the function. We could implement fill options (yes the name DrawLine becomes incorrect... but we can make aliases just to make the code clearer)

DrawLine(GFX_GC,         &gc,
         GFX_Point2D,    { x1, y1 },
         GFX_Point2D,    { x2, y2 },
         GFX_Point2D,    { x3, y3 },
         GFX_Fill,       TRUE,           // implies closing the path
         GFX_FillColor,  &colorspec,
         TAG_END);

No one assures that this approach is efficient (maybe we prefer a different function, like DrawPolygon, to do this job, so we keep the code of a simple drawing line function smaller and faster). It seems like overloading too much!

This tag idea is suitable to build an OO system, as I have said before. But the most important fact is that it can help keeping cleaner and more compatible (between releases) APIs. It could be considered as a partial solution to some problems; avoid creating arguments or struct elements reserved for future development, giving de facto a sort of abstraction layer that hides details about data structures...

Home page