I've been programming for 10 years now.
Posted by 08/14/2007
Fine Arts Library - University of Texas at Austin
My first job out of college was for the Fine Art library at U.T. This made sense because I had a Bachelor's in Studio Art. The job didn't pay very well. In fact, it was about $1000 a month take home. That seemed great at the time - compared to minimum wage.
This was a low skill job. It didn't involve much computer interaction - and at the time I had no interest in computers. In fact it was a struggle for me to use page layout software. In some ways I could have stayed at this job forever. I would still be shelving books and I would not be writing this. It was a fun job. I met a lot of great people. But something told me I should try to do something that paid better. They say money doesn't buy happiness. I'm not sure I believe them.
Tennessee Managed Care Network
The next job I got was at a library at an HMO in Nashville, Tennessee. It was good that I got this job because a month earlier I had cut down to half-time at my Fine Arts Library job. I was tired of working there, and it didn't seem like I would ever get another job unless I did something drastic like that. Looking back on it - a year and a half isn't all that long to work somewhere. But at the time it seemed like a very long time. I wanted to do something else.
So for a while I was living on $500 a month take home. I needed another job, and a friend in Nashville who was a librarian was hiring at $9/hr. It was perfect timing. Plus I had roommates from hell at the time. Little did I know that one of them would be dead within a year. But that's a long story.
This was 1995 - just when Netscape and email were becoming widely known and widely used. I worked in a small corporate library, a personal library for the CEO. The librarian that hired me had created an electronic index of books. It was just an Access database. I sort of took it over and made it more like an application and less like a spreadsheet. The scope of the library expanded to include newspaper, magazine and journal articles. And soon other executives besides the CEO wanted to use it as a resource.
So I was looking at trying to make an Access database a distributable application. This led to me eventually working with Visual Basic. I wrote an application to search the database and made it installable over the network. I also started reading about html and webservers and hashing out the details for making an intranet, because that seemed like it was going to be the best way to make content readable to the entire organization. The Visual Basic application wasn't going to cut it. I could sense that.
During this time I realized programming was something I could do. It appealed to my sense of arranging and categorizing things. And it beat shelving books for a living.
I spent hours and hours reading Visual Basic books. For months. And I labored for hours with the code behind data entry screens and search screens. And struggled with 'setFocus' and made a really stupid, overly-clever, 'creative' GUI with a search interface called "Bowling for Books" comprised of a hand drawn picture of a guy bowling. The user was supposed to click on a lane for different types of searches - by title, by subject etc... Jeesh. That was ridiculous. But I had just finished my art degree. What do you expect?
Looking back, I think Visual Basic is a good way to learn programming. Because with Visual Basic you can lay out a set of widgets, hit F5 and see what it would look like as if it were a working application. The rest is just a matter of working out the details. That is really encouraging when your just learning something.
Back to U.T.
Tennessee Managed Care Network went out of business eventually. You could see it coming a mile away. It was a train-wreck. The company was a small HMO that received a 1,000 fold increase in clients because of the TennCare program. They were the classic victim of too much business too fast. They just weren't ready for it. And this resulted in an endless stream of lawsuits. I actually quit after working there just one year. Remember, it seemed like a year was a long time back then. I was young.
Plus I had another roomate from hell in Nashville. He forged my checks and stole things from me - including my wallet. The first time I confronted him about it I mistakenly said "you can't fool me". Which, in retrospect I can see he took as a personal challenge. I mean, I didn't actually mean "you are incapable of fooling me" - but that's how he took it. After that he stole more.
I have a long list of things he did I could tell you about, but here's an example: He pretends to talk to his parents on the phone saying something like "can you send me some money? I owe my roomate rent". He says this very loudly to make sure I hear him from the other room, and then very loudly says something like "Okay good! In a few days? Great. I'll be looking in the mail". Then he tells me "I'll get you the rent money in a few days. I swear. I just talked with my parents ..." A few days pass, and then he disappears for a week. I think, great - maybe he's gone for good. But then he shows up and acts as if nothing has happened. No mention of the rent. So I ask and he says "what do you mean? My parents didn't send me any money. Are you kidding? I'll get you the rent soon though. I have a new job ..." It went on like this for months. He never actually gave me any rent money.
What an a-hole.
I went back to the Austin and landed a job with library system again. So I was back to shelving books. My brief foray into programming over. But now I had a career direction. So everything was different. I would get a job as a programmer at some point in the future. I knew that much. Even if I had to get a degree. For the time being though I was only taking home maybe $700 a month.
It was at this time that I experienced one of the most humiliating moments in my life. My supervisor stepped me aside and asked me how often I bathed and/or washed my clothes. I was smoking then and walking to work in sweltering Austin heat, and I only had one pair of pants. So I guess I stunk or something. I made some lame excuse about not having a car and not being able to drive to the laundromat. Then he offered to give me a ride. Uggh. That was possibly my greatest ignominy.
I also interviewed for a lot of programming jobs at this time, trying to break into the field. This was also humiliating. I got to hear a lot of pompous people summarize what they did for a living. "let me tell you what we do around here ..." I particularly remember this one interview that lasted more than an hour, 55 minutes of which was listening to this windbag talk about how hard his job was, and right at the end he says to me "to be frank, I know someone that has 7 years experience like yours - and I just gave him the ENTRY-LEVEL job ...".
What an a-hole.
I tried getting jobs through 'head-hunters' as they are sometimes called - tech-job recruiters, temp agencies etc... Often there was a disconnect between what they said the salary was, and what the people actually wanted to pay me. So I would go on interviews where the recruiter said. "They may pay as high as $25/hr". But the 'negotiating salary' part of the interview would go something like this:
Interviewer: "What kind of salary were you thinking of?"
Me: "12?" said apologetically while raising my shoulders, instantly realizing, by the reaction, that my number is too high.
Love of My Life
I met the love of my life at this time though. So it wasn't all bad. Thank god for her. If I hadn't have met her I'd probably be walking down the street now drooling and clutching my pants to keep them from falling off. You think I'm joking but I'm serious.
Texas State Library
Finally I landed a job with the Texas State Library Archives division, breaking into the elite "over 20K a year" club. Easy street. It wasn't a programming job technically, but at least I used a computer all day and I had to work with a database. I turned 30 years old around this time too. So that would make it circa 1997. A lot of people would call me a loser at this point in my life. But who asked them.
My job was to keep track of all the Texas county government paperwork regarding compliance with records management policies and procedures. Sound exciting? Very. I would come in to work and try to force myself not to look at the clock for an hour. An hour would pass. Then I would look at the clock. And only 5 minutes had passed. It was that bad.
I did some work on a FileMaker database, which was a Macintosh application. I worked on automating reports, automating various searches and data entry screens. It was my previous database experience that had landed me the job in the first place. When I printed out all the code I had written I was astonished to see it was over 300 pages long. During this time I grew to hate the Macintosh. Because programming on it felt like working on a car with the hood sealed shut. Plus everyone was so elitist about how much better the Mac was compared to the PC. Some things never change.
I only lasted a year at this job - and then moved on. A year was about all I could take. This job was what I like to call an "ass-wipe clerical" job. I answered phones, put together packets - and on the side did a little database and html work. In fact I learned Html and cgi somewhere in there. I did not use it to a great extent. But I learned it. This came in handy later on.
Next stop in my illustrious career was the Texas Department of Human Services, the agency formerly known as the Texas Department of Human Resources formerly known as the Welfare department. This was a job making web pages. Somehow I fooled them into hiring me. The job required a little Active Server Pages (ASP) programming, so I considered it a 'programming' job. I had officially broken into the field.
I remember being amazed that during my first week I went 3 days without even talking to my boss. It was a little strange for me. It was a "welcome to another world" moment. A world where you just have a job and you do it and no one is checking up with you every day to make sure your actually working. It was great. The world finally treated me like an adult. And it was a 35% raise - so it was serious cash.
The department had a bunch of handbooks in Wordperfect that outline polices, procedures and rules. There was a 10 year plan to get them all online. By directive of the commissioner. These handbooks were printed and distributed to thousands of people across the state and revised every week or so.
The plan of action at the time was to just make another version of the handbook in html, and update both the Wordperfect version and the html version whenever the content was revised. So far they had done three handbooks with this approach. One problem was that each of the Html versions looked radically different, with different navigation and styles because no one had figured out the importance of a consistent presentation yet. Another problem was that every time there was a revision someone would forget to change one of the two source files. All and all it was a mess. And it was obvious this approach was not going to work.
Some people suggested just putting all the handbooks up in Pdf. In fact, any person that thinks of themselves as a typesetter will think this counts as putting something online - and will always prefer this method. Why? Because they want absolute control of how the publication looks. And they've already created a perfectly arranged and legible document. With the most appropriate font, the best indention scheme, and with important things like bullets that center exactly with the text. And they use fancy terms like kerning.
If I sound like I'm being sarcastic it's because I am. At the same time I think typesetting is a lost art. I worked in an old fashioned letterpress shop for several years. So I actually have some experience setting lead type and even running a heidelberg windmill press.
But that's another story for another day.
Luckily our search engine couldn't search across multiple Pdfs in 1999 - and you could not link between them, so it was decided 'online' meant html, not slapping a bunch of Pdf files on the server. I was glad. I hated Pdf, even back then.
It was clear to everyone that all the handbooks should at least look the same, So we put together a sample html handbook. And presented it to a review board of sorts - representatives from each of the areas for whom we would be working. It was a pretty basic no-frills approach. We didn't expect much of a reaction. Boy were we wrong.
It was one of those dreaded presentations. Suddenly we had a room full of belligerent, hostile people. And one angry guy in particular saying "This is crap. I make web pages at home and I can tell you this is crap. My handbooks have won awards and I can tell you this is crap". His handbook had indeed won an award. In 1975. He's been introducing that factoid into one of every 4 sentences he speaks ever since.
My bosses were alarmed by this reaction so they proposed organizing a 'handbook committee' to have meetings to make decisions. And since everyone was so opinionated, everyone was invited. Almost 100 people showed up at these meetings. There was lots of yelling. It was insane. Another "welcome to another world" moment. There is nothing quite like a contentious gathering of state workers harping over minutia.
What followed was 6 months of meetings to discuss what the handbooks should look like and how they should work. I did prototypes in ASP - a relatively new server side technology at the time. My job was to figure out if things were feasible - like bulleted lists lining up with the exact center of letters, or lists that indent exactly as the tab order in a word processor, or lists that automatically display with a particular bulleting scheme, or lists that follow unusual numbering scheme, or lists that involve unusual formatting etc... Notice a pattern? Yes, there was a sick obsession with the stylistic presentation of lists. Is it psychologically revealing? Does it say something about state government? I think so.
If you ever work for the state you may find that everything you do fits into one of these 5 categories. You are either ->
- making a list
- adding to a list
- removing from a list
- referring to a list
- making a list of lists
Also, I had to find out if we could do things like make tabs for sections of a handbook that 'light up' when you hover over them. There were a lot of crazy ideas floating around. Some people had the idea of using a lot of Flash and transforming these dry policy tomes into some sort of multimedia experience. Others wanted to make the handbooks double as distributable help files. And everybody and their dog made web pages at home and had an opinion. Luckily the committee method has a way of silencing marginal ideas. Which was probably good in this case.
After 6 months or so a final 'look and feel' was decided on. It looked like this:
There were no good standard practices at the time for web development. It was a big hack-fest. A nightmare of competing incompatible browsers, half-thought-out and half-implemented w3c standards, code littered with nonsensical fragments, wide open security holes ... Little did I know it would be largely the same 10 years later. But I'll save that for my "Web Programming Sucks" post.
So once the look was established, it was just a matter of addressing these 2 questions:
- How to get all these word processor files converted to Html
- How to make it so there is only one source file to edit with any given revision - so there is not the duplicated effort problem between print and online.
At this time my department starting looking into some schemes to try and automate the process. We hired a contractor who wrote a macro in Microsoft Word to generate Html from a Word source file. He charged a lot of money and had back problems - so my bosses had to meet him at his house - where he would lie prone on a couch for the entire meeting, letting out the occasional groan of pain. This was my first experience with an overpriced contractor that did not make what the customer wanted, but rather what the contractor wanted. I didn't realize how common this was.
It turned out the system he built was not flexible enough so he was fired and his work never used. This is another common phenomenon. Thousands of dollars spent for nothing - absolutely nothing. At this time I started reading about Xml and realized that we could generate all the formats we needed, with as much flexibility as necessary, from a common source file. It just had to be an Xml file instead of a Wordperfect or Word file.
So I presented a proposal to convert the handbooks to Xml. This was just when Xml was being advertised to the masses as an interoperability panacea so some people had heard of it. The vague idea was that it was like Html - only better. "Extra-special Html". But that none of the browsers supported it yet. I often had to explain that we would be using Xml on the back-end, not the front-end, so the final product was still Html that would work in any browser. Not to worry.
The proposal was accepted, but then came the hard part. How do I actually pull this off? How do I generate Word and Html files from the same source? I only knew it was possible in theory.
I looked into methods for transforming Xml both to Html and to some kind of printable format. Xsl could produce Html easily - so that was no problem. For print output the w3c had a standard called formatting objects, but it wasn't ready for prime time. 8 years later it still isn't ready. To this day TeX is a better solution for typesetting. It's sad because TeX sucks. There I've said it. All TeX lovers out there, I challenge you to a duel.
There were not a lot of software packages available at the time to work with Xml content. Xml was still fairly new circa 1999. I tried Wordperfect 9, because it had an Xml editor, and because everyone in my department loved Wordperfect. But it had a lot of problems. Then I looked at expensive products from ArborText, Adobe, Documentum. All of these could have worked, possibly, but I had the added constraint of a $0 software budget. Plus, when evaluating all that software, I could never produce a finished product that matched the current set of conventions.
For instance, none of them allowed a specification for side-marks to indicate revisions. None of them allowed multi-line headers and footers, none of them made it possible to put a specific revision number on a specific page. One could argue that none of these features were necessary, but that's another issue and another post called "Should you change the business to match the software, or change the software to match the business?"
This led me to Jade - an opensource package that can produce RTF from Xml via DSSSL. DSSSL stands for Document Style Semantics and Specification Language. It's a variant on the Scheme language - with specific functions for transforming SGML and Xml into another format. At the time there was only one book I could find about it, a specification help file, and some very simple examples on the web. And then there were Norm Walsh's DocBook stylesheets.
DocBook is a set of tags for text that concern concepts related to documentation. It was designed chiefly for software documentation. It is sort of like Html, but with a lot more tags - many specifically relating to publishing and documentation.
So someone named Norm Walsh - a famous guy in the Xml community - wrote a series of DSSSL stylesheets to transform DocBook SGML into various formats. These stylesheets were a tour-de-force in DSSSL, but they were nevertheless very difficult to extract working code from. It was a very elaborate set of stylesheets, with some hooks for customization. It was even designed in a modular fashion. But it was designed to work with DocBook. Which I had decided not to use.
Why not use DocBook?
I decided to create my own DTD (My own set of tags). Why? I thought DocBook might scare people away and kill the whole project. I was dealing with a group of people to whom Html tags were a new concept and who were very skeptical. After all, this was a project that had gone through several failed attempts already. It's true DocBook is modularized such that you can use merely a subset of it. But after fumbling around with that, I found it easier to start from scratch. That way I could use the specific terminology the group already used to describe content - and I could cater it to the particular stylistic constructs they used.
For instance, this was something I saw a lot of:
1 = something 2 = something else 3 = another thingor
1 - something 2 - something else indented 3 - another thingwhich was conceptually a list, but because of the indenting I had to output it as an Html table (because tables were what you used for spacing, arrangement and alignment back then). I conceived of that construct as being tagged like this:
<ITEMIZED.LIST seperator="="> <LISTITEM>something</LISTITEM> <LISTITEM>something else</LISTITEM> <LISTITEM>another thing</LISTITEM> </ITEMIZED.LIST>which was consistent with this:
<LIST> <LISTITEM>an item</LISTITEM> <LISTITEM>another item</LISTITEM> </LIST>and this:
<RULE.LIST> <LISTITEM>first item</LISTITEM> <LISTITEM>second item</LISTITEM> <LISTITEM>third item</LISTITEM> </RULE.LIST>which was another list that had to look like this:
(a) first item (b) second item (c) third itemSo everything that was conceptually a list could be tagged as a list. This seemed important to me. Because the focus was on the content - and the meaning of the content. Notice too, everything is a list.
Style has meaning
In retrospect, that was a rather idealistic view of mine, because a lot of content is revealed by the style by which it is presented. Very quickly I was making code like this:
<EMPHASIS style="bold">bold text</EMPHASIS> <EMPHASIS style="italic">italic text</EMPHASIS>which isn't actually any more meaningful then
<ITALIC>italic text</ITALIC>Despite giving it the name 'emphasis', because there are an infinite number of reasons why someone might want to italicize text. And trying to keep up with every one of those reasons as contextual information is not feasible.
The Address problem
Another problem I ran into is what I'll call the 'Address problem' - which is the inability to adequately tag even the simplest compound objects. I started with tags to represent an address. It doesn't sound like a bad idea. Then you could, in theory, run a Xsl transformation to pick out all the addresses. It's a pretty easy concept right?:
<ADDRESS> <NAME></NAME> <CITY></CITY> <STATE></STATE> <ZIP></ZIP> </ADDRESS>But I forgot about an address with 2 names:
Texas Department of Aging and Disability Services Consumer Rights and Servicesor an address with a mail code
Mail Code E-100or an address with a fax:
Fax: 512-438-5555or email
email@example.comAlso, some people may want to present an address like this:
Texas Department of Aging and Disability Services Consumer Rights and Services Mail Code E-100 701 W. 51st St. P.O. Box 111111 Austin, TX 78751 firstname.lastname@example.org Fax: 512-438-5555
while someone else wants this:
Texas Department of Aging and Disability Services Consumer Rights and Services 701 W. 51st St. Mail Code E-100 P.O. Box 111111 Austin, TX 78751 Email: email@example.com Fax - 512-438-5555
Which means you've introduced a blank line as a conceptual component of an address.
Yet someone else wants this:
Consumer Rights and Services Texas Department of Aging and Disability Services c/o Thomas Doubter - MC E-100 P.O. Box 111111 Austin, TX 78751
And it has quickly degenerated into nothingness. Now, it's very easy to say "well, just force everyone to make their addresses all the same". But people take the way they format things very seriously. They are convinced it conveys information. And who am I to say?
I actually ended up with all the following tags being available within <ADDRESS> - and making them available in any order. That's how bad it got.
ORGANIZATION SUB-ORGANIZATION PO-BOX MAIL-CODE STREET CITY STATE ZIP-CODE PHONE PHONE-2 FAX CONTACT GROUP EMAIL HOURS SITE ADD-NAME (Don't ask)
So it's useless as a data element now. It might as well be a bunch of paragraphs. This is a lesson programmers learn again and again. Trying to conceptualize human concepts into named objects is difficult. It's a big problem with Object-Oriented design in general. It turns out an address is just series of paragraphs used for contact purposes. It could be long or short and have any number of components.
Names are problems
It's the same thing with names. Try putting a bunch of names into a database.
You'll end up with a
field eventually. Then a
middle_name, then a
everything is wrong.
A name is just a string of characters that
identifies someone. It could come in any order, have 8 parts or
just 1 (like Cher). Then someone will ask for a report sorted by
last name. Sort by last name? Except for people
that don't have last names. Or people that have 2 last names. The world
starts to break down. This happens again and again. We have not
invented a programming idiom that adequately models human conceptual
thought. I guess it's hard or something.
to be continued...