Subject: Re: book Fri Aug 21 12:17:51 1998 this is a piece i just wrote for a friend in London, who works for an ISP over there. i've been trying to teach him about OOP, and software design in general, for a year or so. he's at the Matt/Selena level of coding, and wants to learn more. we've both been too busy to keep the lessons up for the last few months, but he sent me a note a few days ago, saying that he had some free time and was interested in picking up where we left off. i started writing a recap of the basic ideas which also dealt with some of the stuff i've picked up myself in the interim, and got the following. it does deal with OOP, so it would probably need to be reworked to fit the context of the book, but it's a damned fine statement of position, if i do say so myself.. the muse was feeling gracious tonight: the most important thing i've learned about programming recently has had to do with the way you organize the pieces that make up the program. i've talked about OOP and modularity before, and all that still applies. making code modular is the best way to keep it manageable, and OOP is basically just a set of tricks that make it easier to keep code modular. the big problem with OOP as it's generally presented is that you don't get the reasons behind the techniques. the discussion jumps straight to the relative merits of various ways to make things modular, but never mentions what you should be trying to wrap up. you're either supposed to know that part already, or are expected to pick it up as you go. having done it the second way myself, i now feel comfortable laying out a general road map of the steps that come between regular function-block style programming and OO software design. it starts with the idea of building programs in layers. there are certain types of things every program has to do, like handling i/o and keeping track of individual data items. the better you understand the structure of a generic piece of software, the more sense the whole OO thing starts to make. the key idea is 'separation of responsibility', and both encapsulation and abstraction are tactics which make that happen. each layer of a program takes responsibility for a certain general category of operations, and within those categories you can start defining specific types of objects. the bottom layer in a program is the hardware abstraction layer. it's responsible for all the operations which specifically involve talking with the computer. the term 'hardware' is pretty flexible, depending on the kind of code you're writing. if it's something inside the OS, like a device driver, we're literally talking about the hardware. if it's a program that runs from the command-line, it will be all the stuff that interacts with the OS, like opening files, making network connections, and all that. the common denominator of it all is that you're dealing with those parts of the machine not entirely under the program's control. the next layer is the primitive data abstraction layer. this is where you keep track of all the parts of the system you do control completely, which basically means the memory allocated to the program, and the simple data structures which populate that memory. this is one of the hardest layers for self-taught coders to discover, because they're used to writing small programs with only a handful of basic data structures. since the number of data structures is small, there's no reason to devote extra code to keeping track of them. to be pedantic about it, the whole layer decomposes to a trivial implementation, which amounts to "just put 'em anywhere". that strategy falls apart when you have to deal with more than a couple dozen pieces of data, but by that point, the self-taught coder is used to considering basic, low-level data organization unnecessary. it's the old "code like hell" mindset, and they really can't learn any more about software development until they've unlearned that mindset. the next layer is the domain object layer, which is where we start getting into the whole OO thing. domain objects are the data structures which represent the actual objects you want to manipulate in the program. the term "program domain" is defined as "all the stuff in the real world the program is supposed to be able to deal with", and is another key concept in the OO biz. when you describe the domain of a program, you do it in terms of real-world objects and operations, not data structures and algorithms. those will come later in the design process, but not here. the first step is just to list all the parts of the real world the program will have to be able to simulate. once you know what objects you're dealing with, you can start defining complex data structures which describe the features of those objects which are important to the program's simulation of the real world. finally, at the top, is the program model layer. the term 'model' is used to mean a simulation, because almost every program is some kind of simulation of the real world. the model layer defines the specific kind of simulation being done, and all the operations involved. it's the part that the standard "code like hell" programmer wants to start with. the problem is that he probably hasn't stopped to figure out what domain objects he's dealing with, how the primitive data structures that make them up will be organized, or how the data from the outside world is supposed to get into the machine in the first place. the thing which ties all the layers together, and which more or less defines the concept of elegance in OOP, is that the objects in each layer should be designed to support the operations in the layer above. the logic at the hardware layer should make it fast & easy to build primitive data structures. it should also make it easy to translate events in the outside world into primitive data structures, and back again. the data structures themselves should be easy to assemble into structures at the domain object layer, and should be easy to manipulate in ways that will be useful in their parent objects. the domain objects, in turn, need to be designed in a way that makes them easy to manipulate in terms of the general simulation going on at the model layer. the work of software design is really a matter of finding ways to make all those layers work and play nicely with each other. the good thing about using layered design is that the influence from layer to layer carries up from the bottom as well as coming down from the top. you end up defining your model layer in terms of the primitive data structures just as much as you choose primitives to fit the model. better still, the bottom two layers are so generic that you can study machine/OS operations specifically, and data structures and algorithms generally, without having to tie them into specific data objects at all. all the tricks you learn at those two layers will give you better tools for building domain objects. the more time you spend programming simulations of a given domain, the more you learn about the important objects and operations in that domain. assuming you've managed to get a good solid grip on all three of the bottom layers, then you can start playing with the conceptual issues involved in the model itself, and with abstract issues of modeling in general. there's enough work at the hardware layer to keep the average programmer busy for at least a couple years. you can get a doctorate in data structures and algorithms and still not know all of them, although the ones you'll have missed will be pretty darned esoteric. people spend entire careers mastering the details of a given domain, but again, those details get pretty subtle after the first few years. the modeling process itself is pure research, and there's no end to what you can do with it. for every line you close off, there are an infinite number of others yet to be explored. in broad terms, if you spend a couple years hammering on data structures and algorithms, and get good with at least the basic forms of system interaction in the most common OSes, you'll have enough of a background to do basic modeling in more or less any domain. another couple years of studying and experimenting with ideas in a specific domain, and you should be able to hold your own with people at the world-class level. if, at that point, you want to go back and learn more about the fundamental concepts of modeling as they apply to that domain, at least you'll know what you want to study. mike .