Pilgrim’s Progress

At least the shared stuff is. I’ve spent at least four hours today trying to get my SQL 2005 database configured on their server. Since they don’t allow remote access through SQL Server Management Studio, I’ve been forced to use their not-so-powerful web-based administration tool. Compounding that is the fact that I used to have references between two different databases on my local server that I now need to merge into a single monolithic database, I’m not a happy camper.

As part of this process I’ve had to populate a bunch of tables with application-specific values. After wrestling with the crappy CSV import function that GoDaddy gave me I decided to write my own little tool to generate SQL INSERT statements for the data in my tables. Since WordPress won’t let me upload it, here’s the code:

/*
 * Created by SharpDevelop.
 * User: smeans
 * Date: 5/6/2008
 * Time: 11:58 AM
 *
 * To change this template use Tools | Options | Coding | Edit Standard Headers.
 */
using System;
using System.Text;
using System.Data;
using System.Data.SqlClient;
using System.IO;

namespace sql2insert
{
  class sql2insert
  {
    public static void Main(string[] args)
    {
      foreach (string dsn in args) {
        Console.WriteLine("writing files to " + System.Environment.CurrentDirectory);

        using (SqlConnection cn = new SqlConnection(args[0])) {
          cn.Open();

          DataTable dt = cn.GetSchema("Tables");

          foreach (DataRow row in dt.Rows) {
            for (int i = 0; i < dt.Columns.Count; i++) {
              if (((string)row[3]).Equals("BASE TABLE")) {
                dumpTable(cn, (string)row[2]);
              }
            }
          }
        }
      }
    }

   static void dumpTable(SqlConnection cn, string tableName) {
     SqlCommand cmd = new SqlCommand(String.Format("SELECT * FROM [{0}]", tableName), cn);

     SqlDataReader sdr = cmd.ExecuteReader();

     if (sdr.HasRows) {
       Console.WriteLine("writing data for table " + tableName);

       using (TextWriter tw = new StreamWriter(String.Format("{0}.sql", tableName))) {
         tw.WriteLine(String.Format("SET IDENTITY_INSERT [{0}] ON", tableName));
         tw.WriteLine("GO\r\n");

         StringBuilder sbCols = new StringBuilder();

         for (int i = 0; i < sdr.FieldCount; i++) {
          switch (sdr.GetFieldType(i).ToString()) {
             case "System.Byte[]": {
             } break;
             default: {
              if (sbCols.Length > 0) {
                sbCols.Append(", ");
              }

              sbCols.Append(String.Format("[{0}]", sdr.GetName(i)));
             } break;
          }
         }

         tw.WriteLine();

         while (sdr.Read()) {
           tw.Write(String.Format("INSERT INTO [{0}] ({1}) VALUES (", tableName, sbCols.ToString()));

           for (int i = 0; i < sdr.FieldCount; i++) {
            if (sdr.IsDBNull(i)) {
               if (i > 0) {
                tw.Write(',');
               }

               tw.Write("null");
            } else {
             switch (sdr.GetFieldType(i).ToString()) {
             case "System.Int32": {
               if (i > 0) {
                tw.Write(',');
               }

               tw.Write(sdr[i].ToString());
             } break;

             case "System.Decimal": {
               if (i > 0) {
                tw.Write(',');
               }

               tw.Write(sdr[i].ToString());
             } break;

             case "System.Byte[]": {
             } break;

             case "System.DateTime": {
               if (i > 0) {
                tw.Write(',');
               }

               tw.Write(String.Format("'{0}'", ((DateTime)sdr[i]).ToString("M/d/yyyy h:m:s tt")));
             } break;

             default: {
               if (i > 0) {
                tw.Write(',');
               }

              tw.Write(String.Format("'{0}'", sdr[i].ToString().Replace("\'", "\'\'")));
             } break;
             }
           }
           }

           tw.WriteLine(")\r\nGO");
         }

         tw.WriteLine(String.Format("SET IDENTITY_INSERT [{0}] OFF", tableName));
         tw.WriteLine("GO\r\n");

         tw.Close();
       }
     }

     sdr.Close();
   }
}
}

Ok, so in the process of migrating to my new laptop I’ve been forced to look at several of my old projects. I don’t remember where I found the time, but ten years ago I must have written a _lot_ of code. One of the more bizarre projects I did back then was a Y2K solution that involved cross-compiling COBOL programs to Java bytecode. I only implemented about 50% of the COBOL feature set back then, but it still turns out to be 600+ source files and who-knows-how-many thousands of LOCs.
So rather than let it sit on my hard drive gathering virtual dust, I’ve created a Sourceforge project for it. Check out the Universal COBOL Compiler project when you have a chance. There’s nothing out there but a one-paragraph synopsis and a bunch of code in the CVS repository, but at least it’s not hidden on my computer anymore!

So I’m coding along this morning, minding my own business, when I run up against a really annoying problem with the XML support in the .Net SQL support classes. What I was trying to do was fetch some values from a support table as a simple XML document with an element for each row. The SQL FOR XML AUTO clause was just the ticket, so I wrote the following code:

SqlCommand cmd = new SqlCommand();
cmd.Connection = cn;
cmd.CommandType = CommandType.Text;
cmd.CommandText = "SELECT * FROM "
  + "supportTable AS valueName FOR XML AUTO";

XmlReader res = cmd.ExecuteXmlReader();

XmlDocument doc = new XmlDocument();
doc.Load(res);

but when I tried to run the code I got the following error on the doc.Load() call:

This document already has a DocumentElement node.

It really had me irritated, because I’d already used the exact same code in another part of my application without a hitch. So after some research, it turns out that the problem (which is obvious in retrospect) is that if multiple rows are returned, the resulting XML is not a valid document, but a document fragment. Since there is no top-level element, the Load() method chokes when it encounters the second row. Not good.

I found several workarounds on the web, but none that I really cared for. So I resorted to some MS SQL funny business that I’ve used in the past that did the trick. The problem here is that there is no single top-level element to serve as a root element. One possible approach is to abandon the FOR XML AUTO in favor of FOR XML EXPLICIT, but the explicit support is so incredibly complex and unusable that I’d rather cut my own foot off with a rusty tin can lid. So, to save my foot, I resorted to some fun with joins.

The FOR XML AUTO clause will actually create nested elements in the case of joins between tables, so to create a single top-level element for my support table document I created a new table in my DB called dummy. It has a single column (dummy) and a single row with a single value (dummy). The single row part is important, because by changing the SQL statement above slightly, I end up with the nice, valid XML document I want:

SELECT * FROM dummy AS rootElementName,
  supportTable AS valueName FOR XML AUTO

Voila! Now my code works and I can move on to more interesting pursuits, like the development of a Comet service for the masses. Good stuff!

One of the big limitations of web-based programming is the pull nature of the HTTP protocol. Services like Google Talk, etc. get around this by using a concept that’s now called Comet, which allows the server to “push” content to the client, rather than forcing the client to continually ask for updates.

Since I tend to need this functionality more and more in my Ajax-style apps, I think there should be a web service to make this easier. I’ll keep you posted on my progress.

Apple is finally ready to tell us about the iPhone SDK. Put it on your calendars, March 6th at 10:00 AM PST. If the thing didn’t cost $400+ I would have JailBroken it already. Let’s keep our fingers crossed, and hope that I won’t have to donate a lung or anything to get my SDK license.

Ok, so I’ve been patiently (at least as far as y’all know) waiting for the February iPhone SDK release that was promised last October. Well, here it is February 23rd and no SDK. According to the Businessweek blog it may be 1-3 weeks late. I’ve pretty much exhausted what can/should be done using DHTML and Javascript, and I’m itching to get my hands into the warm, steaming guts of my phone. So to speak.

In my quest for unclear/confusing/just plain bad code, I couldn’t in good conscience ignore my own code. Of course, code I’ve written recently is still too fresh in my mind to be looked at objectively. I would probably still see things now the way I did a few months or a year ago.

But what about code I wrote five years ago? Or more than ten years ago? To test my theory, I’ve started digging up some development projects from the archives so that I can see how my style has changed and also see if maybe some of my code can be of use to someone. I’ve created a new Programming section of my site and I’ve posted up a pretty simple example of my Java programming circa 1997.

The example I’ve chosen to start with is a set of classes for doing DNS queries. It was originally written for a SMTP/POP3 server I wrote in Java. Maybe if I get ambitious I’ll start posting some of that up as well.

There are many things that I’d do differently if I were writing this code today, but I’ll pull out one obvious example to talk about:

283   private void SortRRs(DNSResourceRecord [] arr, boolean fDescending) {
284     if (arr == null || arr.length < 2) {
285       return;
286     }
287 
288     boolean fSwapped;
289     DNSResourceRecord rrSwap;
290 
291     do {
292       fSwapped = false;
293       
294       for (int i = 0; i < arr.length - 1; i++) {
295         boolean fSwap = false;
296 
297         if (arr[i+1].m_iType < arr[i].m_iType) {  
298           fSwap = true;
299         } else if (arr[i+1].m_iType == arr[i].m_iType) {
300           switch (arr[i].m_iType) {
301           case TYPE_MX: {
302             fSwap = arr[i+1].m_lData < arr[i].m_lData;
303           } break;
304           }
305         }
306 
307         if (fSwap = fSwap ^ fDescending) {
308           rrSwap = arr[i];
309           arr[i] = arr[i+1];
310           arr[i+1] = rrSwap;
311           fSwapped = true;
312         }
313       }
314     } while (fSwapped);
315   }
316 

What this subroutine is doing should be obvious to most old-school programmers. Yes, it’s the infamous bubble sort. It’s widely regarded as one of the worst types of sorts available. So why did I use it? Probably because it was easy to remember and I couldn’t be bothered to create the extra class I would have needed to use the built-in Java sort() method.

But, the bubble sort itself isn’t the biggest problem with this code. This code is a victim of excess cleverness, on line 307:

307         if (fSwap = fSwap ^ fDescending) {

When I wrote this code, I was obviously in love with my knowledge of Boolean logic and operator precedence. I wrote this code and it actually took me a minute to decipher what it really meant. Here are some mistakes I made in this case:

1. Inaccurately naming my variables. The fSwap flag actually should be called the fBigger flag. It doesn’t necessarily mean that a swap should happen, it just means that the current array element is “bigger” than the next one.
2. Using obscure operators. Yeah, yeah, everyone should know their XOR truth table, but it would have been much easier to read if I had written (fBigger != fDescending). Not as “cool”, but much easier to decode after 11 years.
3. Unnecessary assignment. I assign the result of the XOR back into the fSwap variable, and promptly forget it. Removing the assignment would have made the logic clearer and saved the initial confusion that some programmers have when they see an assignment in a conditional expression (thinking that the programmer meant == when he really meant =).

All that from one little line of code. So had I written it today, it would have looked something like this:

if (fBigger != fDescending) { .. do swap }

In the long run, overly-clever code really doesn’t save us much. If you don’t believe me, consider what the father of the quicksort algorithm said:

We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil.
–C. A. R. Hoare

For our next installment, I’ll probably be tackling one of my other older Java projects, possibly the mail server, or even my Java-based COBOL compiler (it’s a long story).

Good (or bad) code is almost impossible to classify through rigid rules or guidelines. Like Justice Potter Stewart said (in reference to what constituted hard core pornography) “I know it when I see it.”

There are some web sites, such as thedailywtf.com, dedicated to collecting examples of inefficient, confusing, or completely inexplicable code. While they are entertaining to look at, they are so blatant that they are not really a problem in the every-day life of a programmer.

It’s the little things that get you over the long haul. It’s the too-complex method here, the overly elaborate expression there. I went out browsing open-source projects for an example of the kind of thing that makes code more difficult to maintain over the long run and found this in the very first file I saw. I’ve simplified it and removed the product-specific identifiers, to protect the innocent.

do {
    bool res = func();
    if (res) {
       // more processing
    } else
       break;
} while (1);

While being very simple, the choices the programmer made cause it to be difficult to understand at a glance. Here is the equivalent code, using the correct looping structure:

while (func()) {
  //more processing
}

As written, the code is difficult to understand. Why is there an endless loop? Where does the loop terminate? Why? When rewritten using the correct type of loop, its operation is obvious.

Now imagine having to read through thousands of lines of code written like that. It’s difficult enough to understand other people’s logic without the additional stress of deciphering overly-elaborate constructs. Keep it simple.

I’ve been writing code for basically as long as I can remember. I started programming on a TRS-80 Model I in 1982, and I haven’t stopped since. Sometimes people ask me how I did that, how I “made” myself learn to program. I didn’t, it was just natural to me. I’ve never really treated programming as anything special, any more important to me than say reading, writing, or cooking. But it is important. It’s what I do to earn a living, it’s what I’ve spent most of my adult life doing. And for the foreseeable future it’s what I’ll continue to do.

Not all programmers are the same. When I was at Microsoft, I worked with some of the greatest programmers in the world. And the thing about that is that you already have to be a pretty good programmer yourself to even appreciate the difference between a so-so programmer and a super-star. It’s so technical, so abstract that even a so-so programmer often can’t tell the difference between good code and great code. There are certain qualitites that make code great, and even though I tend to appreciate them without conscious thought, I’ll try to list them.

• succinct — All great code is brief. I’m not saying it’s short, but it’s absolutely no longer than it needs to be. If it needs to be 4 lines long, it is. If it needs to be four pages long, it is (but that happens rarely).
• simple — Great code doesn’t require a lot of effort to understand. The logic behind it is explicit. It doesn’t use tricks and shortcuts to make the programmer look clever. You can go back to great code after five years and understand it just as well as you did the day it was written.
• repetitive — Very well written code tends to consist of repeated patterns, with small variations. This isn’t to say that it has redundancies or blocks of cut-and-paste search-and-replace code. But similar tasks are done using similar means, which increases the likelihood that the code will work as designed. If it works once, it works every time.
• fail-safe — Great code degrades gracefully. When abnormal conditions exist, it fails cleanly. This doesn’t mean that it is cluttered with hundreds of assertions and error tests, but it does mean that variables are always initialized to something safe, conditional statements take possible null values into account, etc.

That’s all I have on great code for now. Maybe I’ll go track down some examples and post them with commentary later.

My latest creation is ready for late-night TV. Hopefully it’ll be ready for prime time soon, but one step at a time. For your enjoyment, go play my Internet rock, paper, scissors game. It’s designed for the iPhone, but you can play it on IE or Safari. Right now you need to have an opponent to play, but maybe I’ll be adding an AI in the near future. Stay tuned!