Hey, what smells like blue?

Authorization And You

2010-02-12T00:27:00.000-08:00

I have been noticing a dangerous anti-pattern recently. It comes in the form of not enough authorization checking. When you have a function in your website that should only be available for particular users (administrators, moderators, whatever), you might be tempted to implement the code to restrict access once, in the view. Consider the following view to delete from a list of users (I snuck in the %: scriptlet tag which I've discussed before):


<ul>
  <% @users.each do |user| %>
    <li>
      <%: user.name %>
      <% if @me.can_delete_users? %>
        <a href="/users/delete?id=<%: user.id %>">delete</a>
      <% end %>
    </li>
  <% end %>
</ul>

This list might be accessible to everyone, but the deletion should be allowed only to privileged users. This could be a good argument to put all administrative features like this in their own actions/controllers, but there's really no need. Just be sure to check authorization again on the controller side:


class UsersController < ApplicationController
  before_filter :construct_me # Set up @me from session

  def delete
    raise "Nice try!" unless @me.can_delete_users?
    User.delete params["id"].to_i
    redirect_to "/users/list"
  end

  def list
    @users = User.find :all
  end
end

Did you see the extra check that the user is allowed to delete users from within the delete action? That is extra critical. It's easy to consider relying on security through obscurity. Who will try that URL to delete the user if they have no idea what the URL is and what parameters are needed? If the URL were less obvious, you may be even more strongly considering just hiding the URL. While the thought to reduce code clutter and duplicated work may be tempting, it leaves a gaping hole to anyone malicious who decides to start playing around with URLs, searching for administrative functions they shouldn't have access to. You may even get disgruntled former administrators/moderators/whatever who knew the URLs to decide to have some nasty fun with those functions they've been since blocked from.

Security is a hard thing to get right, but there's no excuse to succumb to laziness and knowingly leave dangerous holes in your system. The more powerful the function, the more on edge you need to be for possible ways to break your security.

Note that my snippets include an XSRF hole, but that's a problem for another post.

Design for Extension

2010-01-15T00:18:00.000-08:00

I'm in the beginning of writing a simple library, primarily written in Java, with the intention of open sourcing it when it is in a reasonably functional state. As I began work on this project, I decided to try out a new way of designing my classes.

Ruby is my favorite programming language, and one of my favorite aspects of the language is that anything goes. Any object is extensible, even its private methods. Not even the built in classes are safe from extension. This is what's called giving you enough rope to hang yourself. Sure, you could hang yourself with this much power... but you could also use that rope to do some pretty cool things.

Nothing irks me more than non-extensible code. Many times I have run across Java code in the wild that either made a method final that I felt the need to override, or didn't expose a critical bit of API that prevented me from extending it in a key way I needed. It is fine if it is code I can change... I can just make something more open and extend as needed. The problem lies in third party libraries that I don't want to maintain private patches for. This, combined with my love of Ruby's openness, has led me to the goal of making all my Java classes as extensible as possible. The final keyword will only be used in dire circumstances (aka likely never), or to fake a closure in an anonymous class. The private keyword will be reserved only for fields and methods of no consequence.

Before you prepare to write a nasty comment about how ignorant and stupid this design will be, let me jump the gun and tell you why I think it is a bad idea. Either the API becomes locked into potentially poor choices, or the API changes underneath the feet of users of my code. The former is a serious problem if I want to have the freedom to evolve my code, so I opt for the latter. I see exposing a bunch of protected methods as a way to let the users of my code extend my classes in ways I didn't anticipate, yet also imply that it is fair game to change as I see fit. My goal would be to keep public methods as stable as possible, but even those I would be willing to shift around in the name of a better design (be it for code clarity, performance, or whatever).

Granted, this exposes implementation details that aren't really necessary for the client code. However, if someone abuses the details enough to cause themselves pain... I feel the pain is necessary to teach them a lesson. After all, we grow by making mistakes.

The benefit is code that can be essentially patched without needing to modify the source directly, and the ability to use the code in whatever way the client author can imagine. I'm always in favor of more freedom. If it were really that bad of an idea, I don't think Ruby (and other dynamic languages) would be as successful and popular as they are.

As I begin to try out this design mentality, I have run across some important discoveries immediately. It is very important to not depend on fields, because if you do, you limit the ways your code can be extended. Consider:


class Person {
    private String firstName;
    private String lastName;

    public String getFirstName() {
        return firstName;
    }

    public String getLastName() {
        return lastName;
    }

    @Override
    public String toString() {
        return firstName + " " + lastName;
    }
}

The above code is fairly extensible, but what if I wanted to change how getFirstName() works? Maybe I want to prepend a prefix, for example. Regardless, I can override getFirstName() and getLastName(), but toString() will not follow suit to the newly defined behavior in the subclass. Instead, I should have written Person as follows:


class Person {
    private String firstName;
    private String lastName;

    public String getFirstName() {
        return firstName;
    }

    public String getLastName() {
        return lastName;
    }

    @Override
    public String toString() {
        return getFirstName() + " " + getLastName();
    }
}

I could have also resolved this by making the fields protected, or exposing setters. Exposing fields beyond private still doesn't typically sit well with me in Java land (perhaps that's foolishly old school, but I'm stuck in some of my ways). Exposing setters seems to just expose complexity that isn't needed. I may not want my Person to have changeable names. Besides, the ability to redefine a getter by appending, prepending, replacing, or mangling the result is quite different from needing to set the value ahead of time.

The other key aspect I have discovered is to be very cautious when designing your constructors. Constructors in Java pose one of the biggest ways to accidentally lock your code into specific details. For example:


public class Person {
    public Person(String username) {
        // Connect to a database and load the person
        ...
    }
}

Do you see the problem? Yeah... you have made it so that you cannot construct a Person without connecting to a database and loading a person. Even if you subclass this, you are still stuck with that behavior no matter what. This has a couple possible solutions. You could extract the database connection details like so:


public class Person {
    public Person(String username) {
        loadFromDatabase(username);
    }

    protected void loadFromDatabase(String username) {
        // Connect to a database and load the person
        ...
    }
}

This will allow a subclass to override loadFromDatabase(String) so that it does nothing. This feels a bit hacky to me, but it works perfectly. The other way around this is to expose an alternative constructor (such as one without parameters). This other constructor could have an empty body, and you could even make it protected if you only want to expose it to give more power to subclasses.

So, I will see how this design works for me, but I already like it. Also, don't take my meaning to say that all methods will be exposed as protected or public. For example, a method that has a very well defined behavior might grow rather large and require some breaking apart. There is no need to make all the pieces protected if they really only make sense as a means of implementing the bigger method. There might be no serious harm in exposing them, but I don't believe in absolute rules either.

There is no One True Way to design. The flaws in this plan might prove more serious than I anticipate, but it doesn't negate that there is some value in allowing free extension. Design is also a bit of a personal adventure, so if this doesn't float your boat, look for alternatives that give you the Happy Feeling.

Any thoughts? Am I destined to fail? Is this something you've tried and enjoy?

Static State

2010-01-13T02:07:00.000-08:00

Beware of static state. Need I say more?

Ok, I guess I shall, regardless of if I need to. If your automated tests all run from the same context (in my case, a JVM instance), static state will bite you... eventually. You may think you are dealing with the state just fine, but it will hurt you eventually.

I have seen many problems crop up with static state so far. Cached data from the database causes data from one test to bleed into another test. Between tests, you will likely have different data. If the data is not based on a fixed set, it is inevitable that some of the data will be slightly tweaked between tests. If that data is cached, the later test will end up with a cached copy of the earlier test's data. This may not crop up right away, and can happen at any time with cached data. A new test that runs before some old tests could cause the old tests to fail. If the order of tests is not predictable in your automated build, you could get a new ordering that sets up a perfect storm to cause some invalid cached data to stick around and haunt another test.

Similarly, you might have some mock versions of static data that makes testing easier. If you don't clean up the mock data, another test may be expecting the default behavior, and again you are failing because of an indirect cause.

Ultimately, static data can be useful and convenient at times, just be aware that your test environment gets weakened with every static cache or storage you use. You can't always simply try to clean up the cached data when you use it. Consider a class, A, that caches some form of data. Your current test depends on class B, which luckily doesn't use the cached data in A, but it does use class C. You don't deal with cleaning up A. Then, down the road, someone changes C to update or view the static state in A. All of a sudden the tests for B are dependent on static state you had no idea about. Or you are modifying that data unknowingly, and affecting all tests that run after yours that depend on the state of A being in a default state.

I've found the most effective approach is a setup or tear down method in your tests that explicitly clears all known cache. This will ensure no test can hide data that leaks into another test, causing an unexpected (and difficult to track) test failure.

Just be on the lookout for the telltale sign of static state causing failures. If a test class fails on the build machine (or in a group), but it doesn't when run individually... you might be seeing static state showing its ugly head.

Meaningful Comments

2010-01-11T00:34:00.000-08:00

Comments are an interesting beast. You can comment your code too much, which will clutter the code so much as to decrease readability. You can comment your code too little such that it is difficult to understand the more complicated areas. You can even not comment at all, and hope the code speaks for itself.

Probably due mostly to laziness, I don't comment my code heavily. When I do, I like my comments to have meaning. I try to write JavaDocs or RDocs for every method, even if it is just a simple comment describing the purpose of the function, with expected behavior. More often than not, I inspect unknown code by looking at the actual implementation. However, sometimes I prefer to jump to the online documentation, so I try to do my part in making sure it is fleshed out.

The problem I see with JavaDocs and RDocs is that it takes diligence to keep them up to date. Sometimes you can unintentionally change the behavior of some method by twiddling with a private or protected method, causing an undocumented change that breaks the usefulness of your JavaDocs and RDocs. Even so, there is benefit in being able to quickly see what a method does, and use that as your base assumptions. If tests prove otherwise, the code can hopefully speak for itself and resolve your conflicts.

The trickiest comments, though, are inline comments. When it comes time to document the details of your code, I've found it very difficult to do a good job of thoroughly commenting it. The problem lies in the fact that the comments are useless until it is time to revisit old code. Once you have lost the context of why you wrote some code, it is very difficult to get it back. We have all run into this problem... you find a bug in some month old code, and start to retrace what the code is doing. You are browsing around, and you see some oddities, but can't for the life of you figure out why the code was written in this way. If you wrote it, you might remember you made some critical decision based on some immediate goal, but the goal and decision are now completely lost.

Therein lies the problem. Code needs to be documented to show why you wrote it that way, not what that code does. This comment is useless:


// Add 1 to the count of items.
itemCount++;

If you describe why you did something, you might give your future self a break when it comes time to track down an error, so try a comment more like:


// The item count is off by 1 because the first item is
// implicitly dealt with already.
itemCount++;

Now, this still leaves the problem open of when to write those comments. The best advice I can give is to look for those moments that you had to think a while to figure out what to do, or the code just looks too complicated. The best solution is so obvious it documents itself, but that can sometimes be rather difficult to achieve.

What are your guidelines of when to document code? Do you still run into those "what was I thinking" moments?

Peer Demo

2010-01-08T01:06:00.000-08:00

A lot of software practices make sense when described. Source control, automated tests, automated build server, etc. Even the most obvious practices become crystal clear the moment they have a real impact on your development. That automated test that caught the bug as soon as you created it. The automated build that notified you of an old test that is now failing because of a new bug. A massive refactoring that turns out to be fruitless, easily reverted with some easy source control commands. An old diff helping you regain context of why you (or someone else) made a particular change.

Recently I had the revelation with simple peer review. Not even peer review of code, but the simple practice of demoing your finished code to a peer. Before going in to this recent demo, I fully understood the benefits of such practice, but it transformed a neat idea to a must do.

Now, like most developers, I am very obsessive about my code. I do everything I can to make sure it is polished and perfect when it is time to deploy. I try to write as many reasonable unit tests as I can. I constantly exercise my code, both with test driven development and with actually looking at the visible output to make sure it is doing what I expect. With all my obsessing over my code, I tend to finish with a fairly confident feeling that it is ready for production. Don't misunderstand my meaning to think that I feel my code is perfect, or even necessarily good, and most definitely not bug free, but I take pride in my craft and try to call it done when I really think it is done.

There is a big problem though... like every other developer, I am human. I don't anticipate a certain kind of error condition. I forget to check a particular path in the code. I over complicate the solution and miss a way to make the UI a lot simpler and easier to use. Ultimately, I am left with a couple bugs that I just couldn't quite run across.

For one, several, or all of the above reasons, I end up with a demo that makes blatantly clear a few key mistakes that I made. This happened with my most recent demo. Together, the reviewer and I filed quite a few bugs in our issue tracking software that I needed to resolve. Not all were directly caused by me and my code... some were issues that dated back quite a while... but quite a few of them were most definitely because I hadn't exercised the code quite as thoroughly as I had thought.

The key thing I take away from this reminds me of Jeff Atwood's post on not going dark. I'm not afraid to share my code as I develop it. In fact, I enjoy showing off a small snippet or clever bug fix I just wrote that I feel does something neat. I similarly like to see when other developers finish something they feel proud of. However, I still end up writing my feature in relative darkness. The demo is like shining a flashlight in the corners that I didn't get to. By having another person walk through your code, or even just the simple act of showing that it works, you will often end up seeing a different perspective that you never thought of while you were working. Sometimes you run across code that had been manually tested, but changed since the tests and is now broken. Whatever the case, you will inevitably run across something that could be improved upon that you didn't quite catch.

It doesn't even have to be a bug, necessarily. For example, in the demo, we decided that the action we were exposing to the user could benefit from being run multiple times, sometimes. So, we made an easy to access link that allows you to retry the action when you want to, which saved the user from the need to search a table for the same row to run the action again. This was a missing feature in the original spec, not exactly a bug. Nevertheless, the demo made this change an obvious need. Perhaps it is the halfway point between just deploying new code, and dogfooding your code. In the absence of being able to dogfood your code, a simple demo of all new features will expose a good chunk of what you would have been able to find through dogfooding.

So before you deploy that new feature... grab a friend and just demo it to them. Even if you don't find a single issue, at least you will be more confident that your code is ready.

Android Critique

2010-01-06T00:07:00.000-08:00

I like to think that I can evaluate and judge a product based on its merits, but there are some companies that elicit emotional responses from me. Apple is one. Contrary to common emotional responses, I get repulsed by Apple products, just because it is Apple. I used to be neutral with them... I liked the iPod, but was never very interested in anything else they did. With their handling of the iPhone (and the App Store specifically), I have come to mistrust them, and now I will actively avoid their products when possible.

Google also brings up emotional feelings for me, but quite the opposite. I have a soft spot for Google for a few reasons. They are strong supporters of open source, both by providing a lot of open source projects (Android, Chrome, Chrome OS, some neat Java libraries... to name a few), but also by providing infrastructure (google code). Most of their products are free, which is hard to dislike. Not only are they free, they tend to be pretty good. I have never had problems with their search. GMail has always been so simple and easy to use that I would never think of switching (especially after seeing people struggle with Yahoo Mail, which seems so cluttered and complicated by contrast). Android blew me away as my first smart phone (with the G1).

With the emotional responses, it's a bit hard to judge whether I like a product from its merits, or it is just my fanboyism that is blinding me to the issues. I want to take a moment to critique Android in this light. I still love the platform, but it would be foolish to claim there are no issues. I honestly believe that overall it is a step up from the iPhone, but I have never had extended time with the iPhone to really have an honestly unbiased opinion there.

First, the good things. Given the right hardware, it can be a snappy OS. The G1 can be sluggish, but my new Droid has proven a significant speed boost. Widgets represent a huge advantage over the bland icon-only iPhone interface. I like having the calendar straight there on my desktop, no touching to see what I have coming up. The ability for an application to always display pertinent information is pretty cool.

Multithreading and background processes can be a pain to implement, but it is a neat feature that Android has over the iPhone. I can have applications actively retrieving useful information, or interacting with each other, or switch between processes while long running tasks are executing. It makes my phone experience much more in line with my desktop computer experience.

The updates are constantly making the Android experience better. Before I switched to the Droid, I had navigation on my G1, something that was nowhere in sight when I first got it. Widgets didn't really exist, and I can now do things such as search with voice commands. The Market (equivalent to the Apple App Store) has slowly evolved with the platform. It only supported free apps initially, but now has both paid and free. Screenshots were added recently, along with the ability to view the top free and paid apps separately. I expect the Market experience only to improve with time.

Now to move on to some bad things. I really like the Droid hardware, but one thing I miss from the G1 are a dedicated number row on the physical keyboard. It is really annoying to have to press alt to type a number. I also miss a dedicated call/answer and hangup button. Those 2 missing features of the hardware are offset by the much better screen and the increased processing and storage power. However, I really miss them.

As for Android itself, the biggest issue is probably caused by the choice of Java as the development language. Applications tend to pause every now and then, including the standard apps like the browser. It could be too many background processes running, but it is very reminiscent of garbage collection pauses which are common in Java. I hope this will be improved with ever better hardware along with improvements in the Dalvik VM that Android uses.

My G1 didn't seem to lock up that often, but my Droid has already locked up a few times, and experienced other glitches that you might not be used to happening in a phone. The oddest recently happened where I could hear just fine during a call, but only when the speaker was on. Switch to non-speaker mode and I couldn't hear a word. Rebooting the phone fixed it. I expect software to fail like this now and then, but it is not good to have my phone crash when I could be receiving an important call, or otherwise experience weird glitches while speaking with someone. I expect updates to resolve these kinds of issues, but it is unfortunate to see any of such catastrophic failures. Probably the worst failure I have had was on my G1, when I had lost reception for about a day without realizing it. The phone just stopped receiving phone calls until I rebooted.

There are a lot of features that are very useful, but very hard to discover. For example, long press on the home button will bring up a dialog that will let you run recent applications. This is extremely useful when switching between a couple applications. Similarly, I only recently learned how to cut and paste with keyboard shortcuts instead of clunky menu controls. I had to find those by looking online. Perhaps it isn't fair to blame the platform for hard to discover features, but it might be nice if the platform gave easy to use, short tutorials the first time you do certain key tasks.

The Market still needs a lot of work. What is the one thing Google should get right in its mobile platform? Search! Searching in the Market doesn't work very well. I have run keywords looking for specific applications, only to get back no results, even though the keywords should have worked. For example, "snogg" will not find the "DoggCatcher" application, even though the developers are "SnoggDoggler." There are also very minimal ways to find applications. You can search, you can browse various categories, and you can look at the top paid, top free, or "just in" of a particular category. Unfortunately "top" has no clear meaning. Under the top free list of all games, PapiJump is below WordSearch Unlimited Free. Except, WordSearch has less downloads (50k - 250k vs. PapiJump's over 250k), and a lower rating (4 stars vs. 4.5 stars). It would be nice to sort the lists in various ways, such as by overall vote score, number of downloads, whatever the current "top" algorithm is, alphabetically, etc.

Overall I am very happy with the platform, but some of the issues I have run into are probably non-starters for some people. It has clearly gotten better since I first started with my G1, so I have no doubts that the platform will only improve with time, and eventually beat out Apple as the #1 smart phone.

Love Hate

2010-01-04T00:14:00.000-08:00

Is there a language feature you absolutely adore? For me it's closures. Blocks in Ruby make me warm and fuzzy, and anonymous functions in JavaScript make me giddy. Anonymous classes in Java make me... puke. Sorry, Java just doesn't have good closure support.

If you have a language feature you adore, chances are good that there is a language feature you loath. For me, it is the ternary operator. I don't care which language you are talking about, ternary operators cause me to run in the corner and hide, cowering, and waiting till they go away, on their merry business.

Perhaps it is a bit of an irrational loathing. I like dense code, but I want my dense code to be readable. I find nothing wrong with something like:


str = list.map { |x|
  x * x if x
}.compact.uniq.sort.join ","

But write something like:


str = value.nil? ? "null" : "'#{value}'"

And I will refactor your code. Whatever the case, a ternary operator makes me stop and think more carefully about what is going on whenever I see it. Code nirvana, to me, means code that is intuitive and self documenting. Method names like compact, uniq and sort all lend themselves to self documentation. The ternary operator, on the other hand, feels cluttered and doesn't lend itself to immediately understanding what the purpose of the code is.

I will go to great lengths to avoid a ternary operator. I prefer the more verbose if control structure. Ruby makes the deal even sweeter since an if is an expression (like everything else in the language). I will happily pay the cost of the added lines of code, though, if it means I can stay clear of any ternary code.

If you feel you must use a ternary, do me a favor and use just one per expression. Not just use one, do nothing else on that line of code, so the ternary avoids external clutter to make it even more non-obvious what is going on in the code.

Are there language features that make you cringe every time a fellow developer uses it, regardless of context?

Blame Yourself First

2009-12-30T00:27:00.000-08:00

Why is it that the stupidest, most obvious bugs are the ones you end up spending half an hour on? Did I say half an hour? Make that several hours... maybe even a day or more, sometimes. Consider the following JavaScript code (written with jQuery):


$(function() {
  $(".clickme").click(function() {
    var link = $(this);
    link.css("color", "gray");

    finish = function() {
      link.css("color", "blue");
    };

    $.ajax({
      url: "/my/remote/url",
      success: function(data) {
        // ... process the result ...
        finish();
      },
      error: function(data) {
        // ... process the error ...
        finish();
      }
    });

    return false;
  });
});

I wrote code very similar to this recently, albeit a bit more complicated, but with the same bug. Can you spot the bug? What if I renamed the finish function variable to iAmABonehead? Well, if that doesn't make it obvious, you might want to brush up on JavaScript, if you plan on using it much. Instead of creating a local reference to a closure named finish, I was creating a global reference to a closure named finish that gets overwritten every time the click handler fires. Thus, a quick couple clicks in a row, and you will end up with some permanently gray links.

Here's the funny part, though. I did what every good software engineer does... I saw the behavior in my browser of choice (Chrome at the moment), and thought... gee, that's an odd bug in jQuery with Chrome. My code surely doesn't cause that problem. How can I figure out a way to work around this odd bug?

Thankfully I caught myself, and gave myself a good reprimand:

Self! Don't be such an idiot! You are surely the cause of this bug... drill down a centimeter and you will find it!

So I took this quite excellent advice and decided to give the same bug a shot in Firefox. My reason being that surely our version of jQuery is thoroughly tested with Firefox... if the bug comes up again, it's either jQuery, or me. Except that it probably isn't jQuery.

A quick test indicated that ~~jQuery~~ I was the likely cause of the problem. A couple minutes of a closer inspection of the code and I found the accidental global and fixed my bug.

I guess this is the "Science" aspect of Computer Science. When dealing with bugs, you need to treat it like a (hopefully) repeatable experiment. Form a hypothesis on why you are seeing your experimental results, then conduct further tests and experiments to drill down until you have found and squelched the bug.

For your first hypothesis, don't think about which of your frameworks is likely causing the bug. Eliminate them as the possible issue. The bug is 99.99942% likely to be in your code. When I come across a bug in fresh code, it's weird how the frameworks and platforms I am using are the first my mind blames. It takes all those times being wrong with such accusations before I finally started to stop myself and reconsider which part of my code might exhibit the behavior.

The sooner you blame your own code, the sooner you will find the bug. It helps to take a breather and come back with fresh eyes, too. Usually the bug isn't just in your code, it's something so blatantly obvious that your eyes skip over it and can't see the semicolons for the braces.

Better Estimation

2009-12-23T00:11:00.000-08:00

Your time estimations suck. I don't care how accurate you are sure they are, they suck. The reason they suck is not because you are a bad developer, or you don't have experience estimating... they suck because you don't know the whole story yet.

It is impossible to predict everything that will come up during the course of your development iterations, and many things will come up that you had no way of predicting. Even if you break down your problem to the tiniest, bite sized pieces, you will have emergencies come up. Your colleague's dog will get sick causing him or her to be out half a day, making it impossible for you to complete the planned integration between your 2 services. A user won't be able to input their Woozit into your Whatzit page because of some overly zealous input validation you shipped last month, and it will take a day to track it down and fix it. You will meticulously estimate each minute detail of how to build the new Bazzle Integration Service, and you will get it all perfectly done, but forget to leave some extra time to do some dry runs at the end, only to realize their system doesn't quite match up with yours the way you expected, and 2 more days will be lost patching up the differences.

I guarantee you, no matter what you do, your estimates cannot be perfect. So stop thinking they can be, and figure out a way to adjust for the unadjustable.

I can't tell you how much I would love to see Joel's Evidence Based Scheduling implemented at my workplace. However, it's always something to tackle next iteration. Beyond the lack of commitment from the whole team, it seems like a lot of work to track the time you spend on everything, even though the results, I am sure, are spectacular.

If you can't commit to such effort, I suggest a more low-tech technique. For every work item that you are a little unsure of, add in a task to figure things out. Break your work items down to the most meaningless, mindless 20 minute tasks you can, and set them as half hour tasks... hour tasks... or even 2 hour tasks. You will have to test that 20 minute's worth of coding, after all. Understanding exactly what you are building will help prevent missing key aspects that might have been forgotten. More things will come up as you are in the details of your implementation, but I find it helps to try to think of as many contingencies as possible.

When you are done with your estimation, however you do it, double the final amount you have. This gives you significant buffer to help you reach your milestone, and it will help address a lot of the unknowns that will crop up between now and the milestone. Then, adjust this 2x multiplier each iteration based on how long it took the last iteration. If your initial estimates for the last iteration was 1 month of work, and you doubled it to 2 months, but you finished in 1.5 months, then multiply the next amount by 1.6, to be safe. Keep track of your multiplier at every milestone, and over time you should be able to develop a multiplier that works for your team to hit almost every milestone.

This is a much cruder form of Evidence Based Scheduling, but consequently, much easier to employ. It may not result in perfect results, but if you are missing every deadline, it should help better than continuing down the same path over and over and over, like a broken record.

Protect Your Environment, Part 2

2009-12-21T00:18:00.000-08:00

Last time I talked about separating your environment, and automating it such that almost nothing needs to be customized outside your source control. Today, I want to expand on that thought a bit and specifically talk about your build environment. If you aren't using a build environment like Cruise Control, you really should be. By automating your build, you can quickly detect when you've changed code that causes a regression test to fail, or if you've otherwise unintentionally broken things.

I'm not really trying to espouse the benefits of an automated build system, though, so I'm going to assume you have one set up. The next step is to protect your build environment.

With the build environment, it's especially important to keep everything in source control, and contained within the root directory. If you can't have 2 parallel checkouts of your source simultaneously running your build without failing in some way, you are doing it wrong.

This means a few things. Primarily, do not depend on external files or libraries. If your code looks for a file in /usr/share/myconfig.properties, then move that file to within your root checkout and change your code to point to that properties file with a relative path instead. Commit every jar or other runtime library into source control, and point your build to load the files locally.

What this gains you is flexibility to have parallel development. You can have a branch that represents what is currently deployed, and a branch that represents your current development. You can update that external jar you depend on in your current development branch and it won't affect your currently deployed branch. This means you can have 1 build server constantly running your tests on both branches, and neither will need to worry that they share the same file system.

If you have any kind of branching going on, you run a much higher risk in hitting build issues if you depend on something outside what is committed to source control. If you have a test that needs to look for a file in a special location on your file system, there will be a problem the moment that file diverges between your branches.

If you must depend on some custom environment variables, prepare those environment variables within the build. Just like diverging files, it's entirely possible a separate branch will need to tweak those variables. Once you need to, your other branches will be hosed.

So, once again, once you've prepared your environment to be bulletproof, protect it vigorously. Don't let your build depend on anything outside what is committed to your source control repository, and your automated build server will thank you.

Protect Your Environment

2009-12-18T00:08:00.000-08:00

Recently, I have been thinking of the environment a lot. Not global warming. Not the O-Zone disappearing. Not the ocean. Not the humpback whale or the spotted owl. I'm talking about your build environment. Your development environment. And, of course, your production environment.

These environments need protection just as much as our physical environment around us. Well, I suppose our physical environment should matter more, but as someone who lives on the computer, my computer environments matter a whole heck of a lot to me.

I'm here to tell you to set up an environment that automates just about everything you do, store it in your source control system, and then protect it vigorously. The moment someone strays and tries to let strands of wild growth choke your environment with custom tweaks, special path settings, and other such nonsense, trim those vines and get back to a clean garden. It's really not that hard.

I can't vouch for any fancy tools that manage this stuff automatically, because thus far I've stuck with living within my basic build environment. I've heard of tools that can manage your environment for you, but sometimes simple will work just as well. At my day job, we use Ant to run our builds, and little by little I've been molding our kudzu of a system into a manageable environment that requires little more than a subversion client, and a few additional packages. Ant can manage the classpath, so there's really no need to keep throwing jars into your shell's CLASSPATH variable.

My most recent addition was a simple custom Ant task to mimic Rail's environment setup. I created an environments directory and stored all the custom files that differ between our production environment, development environment, and any environment in between. It will hold custom properties files that are loaded at runtime, and Tomcat's context.xml so we don't have a local diff pointing to our development database. Within the environments directory is a directory for each environment, and the ant task will figure out which environment you are using, then copy over files from the appropriate environments directory to the correct location. This is much like Rail's environment.rb, with the different scripts in the environments directory.

A colleague recently set us up with Rails migrations, and I can't express how direly we needed this. It is important to version control all your database changes, but if you don't have a very orderly process to migrate between one release to the next, you are in for a world of hurt. Not knowing what has been run on your live database means you might miss something that could have drastic consequences with the code you are releasing along with it. Forget an important index and your queries will grind to a halt. Forget a key table, and your code could fail in an unexpected manner that might trigger other unintended failures, create corrupted data, or lose unrecoverable data.

By keeping your environment consistent, you can be more confident that deploying your new code will work exactly as you tested. By automating your environment, removing all of those special files you need to move to the right location or custom environment variables you need to tweak, you make it a lot easier to work within your team. You also can then bring in new colleagues and have a new development environment up and running for them faster.

The moment you tweak some kind of configuration that everyone in your team should do, automate it and put it in source control. If your tweak should only be in development or test, set up a simple environment switch in your build system, and grab the right file automatically. The production version and the development version should both be checked in.

So there... protect your environment. Who wants to be working in a jungle, anyways?

A Simple Ruby Pattern

2009-12-16T00:00:00.000-08:00

By being a multi-paradigm language, Ruby provides numerous possible styles to write your programs with. I am a big fan of metaprogramming in Ruby, and one style strikes my fancy especially. By defining methods in your Class instances, your subclasses start to act like a domain specific language. You end up setting up a structural framework for how your subclasses behave, then declare the behaviors specific to each specific subclass.

To illustrate this, consider a simple calculator application. You could tackle such an application any number of ways, but we can come up with an interesting solution using class methods.

For starters, we need our base CalculatorBase class, which will provide the plumbing for dealing with input, handling if the input is numeric versus operations, and overall flow. I am not interested in these details for this post, so I will leave them as a homework problem for you to play with. Don't worry, it's fun!

First, your base calculator needs to expose a way to define operations:

class CalculatorBase
  def self.operation(op, arity, &block)
    define_method(op.to_sym) { |*args|
      unless args.size == arity
        raise "Wrong number of arguments"
      end
      block.call *args
    }
  end
end

From this, you can declaratively define what operations your calculator may support:

class Calculator < CalculatorBase
  operation(:+, 2) { |x, y| x + y }
  operation(:-, 2) { |x, y| x - y }
  operation(:/, 2) { |x, y| x / y }
  operation(:*, 2) { |x, y| x * y }
  operation(:sin, 1) { |x| Math.sin x }
  operation(:cos, 1) { |x| Math.cos x }
  operation(:tan, 1) { |x| Math.tan x }
end

Now, you can invoke operations with ease:

calc = Calculator.new
calc.+ 2, 3
calc.sin 3.14159

However, I feel we can do better than this. There is a lot of repetition going on with the arity. To make our calculator implementation more domain specific, let's allow binary and unary operators to be defined easier:

class CalculatorBase
  def self.operation(op, arity, &block)
    define_method(op.to_sym) { |*args|
      unless args.size == arity
        raise "Wrong number of arguments"
      end
      block.call *args
    }
  end

  def self.binary(op, &block)
    operation op, 2, &block
  end

  def self.unary(op, &block)
    operation op, 1, &block
  end
end

With these new methods, we can make our operation declarations a bit easier:

class Calculator < CalculatorBase
  binary(:+) { |x, y| x + y }
  binary(:-) { |x, y| x - y }
  binary(:/) { |x, y| x / y }
  binary(:*) { |x, y| x * y }
  unary(:sin) { |x| Math.sin x }
  unary(:cos) { |x| Math.cos x }
  unary(:tan) { |x| Math.tan x }
end

This isn't a particularly difficult approach in Ruby, but I really like the results whenever I am able to employ it. It can remove a lot of repetitive method declaration, and make your main class very clearly declare the behaviors it employs.

Of course, with great power comes great responsibility. It is easy to obfuscate the behavior you are creating with this pattern. Used appropriately, you can design an API and then speak the language of your domain in a much easier to comprehend fashion.

Common Java Implementations

2009-12-13T23:36:00.000-08:00

Java makes a lot of tasks more difficult than they should be. For example, checking if 2 objects differ can be cumbersome when you take into consideration that values could be null. Consider the following example implementing equals for a Name object (yes, some of that duplication could be simplified, but ignore that for now):

public class Name {
    private String first;
    private String middle;
    private String last;

    ...

    @Override
    public boolean equals(Object other) {
        if (other == null || !(other instanceof Name)) {
            return false;
        }

        Name otherName = (Name) other;

        if ((first == null) != (otherName.first == null) {
            return false;
        }

        if ((middle == null) != (otherName.middle == null) {
            return false;
        }

        if ((last == null) != (otherName.last == null) {
            return false;
        }

        return first.equals(otherName.first) &&
               middle.equals(otherName.middle) &&
               last.equals(otherName.last);
    }
}

Now consider the task of reading all the lines of a file:

File file = ...;
BufferedReader br = new BufferedReader(new FileReader(file));
String line = br.readLine();
List<String> lines = new ArrayList<String>();

while (line != null) {
    lines.add(line);
    line = br.readLine();
}

Maybe you need to build a string that is a colon delimited list of the items in a list:

if (list.isEmpty()) {
    return "";
}

String result = "";
result += list.get(0);

for (int i = 1; i < list.size(); i++) {
    result += ":";
    result += list.get(i);
}

return result;

Well, whenever you start cracking your knuckles, preparing to dig in and write a bit of code that you feel should be easier, stop. Take a deep breath. Consider who might have solved this problem first. It has likely been done before, so you can bank on another's implementation, with the benefit of it being thoroughly tested and likely highly performant.

In particular, the Apache Commons projects has a suite of useful tools to bank on. The equals example can be simplified to the following (thanks to the lang commons project):

@Override
public boolean equals(Object other) {
    if (other == null || !(other instanceof Name)) {
        return false;
    }

    Name otherName = (Name) other;
    return new EqualsBuilder()
            .append(first, otherName.first)
            .append(middle, otherName.middle)
            .append(last, otherName.last)
     .isEquals();
}

While the reading of lines can be simplified to (thanks to the IO commons project):

File file = ...;
List<String> lines = (List<String>) FileUtils.readLines(file);

Besides the Apache Commons projects, you could try out the Google Collection Library to solve the colon delimited list:

return Joiner.on(":").join(list);

You may also want to browse Google's Guava Libraries project.

Whenever these libraries can be applied, you can be sure to cut the amount of code you are writing to a fraction of what it would have been, and likely a lot easier to understand and maintain.

Don't make V1 TOO Crappy

2009-12-10T21:33:00.000-08:00

Avery is a company that provides some... paper needs. I don't know all of what they provide, but I know they provide labels. I know this because my fiancée and I were printing out labels for our upcoming wedding. They provide an online tool that could be... very good. Instead, it is... so so.

They provide web tools to construct a PDF which can be printed on to your labels. This is an awesome idea because it is a very portable way to print to their paper products, and could provide highly customized behavior geared towards the type of paper you are printing to. Labels, in our case, could have some really cool software designed specifically around producing the exact labels you want on the specific Avery label product you purchased. If Avery wanted to be a leader, they could even support their leading competitor's products, so when you think of a good experience for your printing needs, you think Avery. This means you might be more likely go for the Avery products the next time you wanted to buy some labels, if they can leave that good of an impression on you. Nevermind the ad revenue they could rake in on top of that.

There is no reason this can't work on any browser with any operating system, right? Well, that is their first failure. The tool (which seemed to be primarily built with flash) wouldn't work on Chrome or Firefox in Ubuntu. I see no compelling reason why it had to be built with Flash, except that it probably was easier to develop their customized UI. Well, I don't believe Flash really gains you much there, but I can understand the thinking that it does. Regardless, some of the controls just failed to respond in my browsers. Namely, a checkbox to toggle the first row in a mail merge as header names vs a separate data row wouldn't respond. Also, the button to add a new span of text on the label didn't respond. Ultimately, I was forced to use IE within my Windows VM, as disgusting as that was. Firefox might have worked on Windows (and maybe even Chrome), but after 2 failed browser attempts, I wanted to go to the crappiest browser that too many unskilled developers still seem to target exclusively.

Ok, force me to use a shitty browser to get my work done... that's a huge mark against you, but if the rest of the user experience is just mind blowingly awesome, I can forgive you. But seriously... I have to have a blown mind by the time I'm done for me to forgive you. It's just too easy these days to write cross browser HTML and JavaScript to excuse the lack of portability. This is not an application that screams the need for Flash, so I think it's an indication of a lack of talent and creativity that they resorted to Flash.

Moving on, I ran across another bug that showed the lack of polish they put into this product. They have a nice mail merge feature that allowed me to upload an Excel spreadsheet and use the rows as addresses. This is a must have for printing labels, but also one of those features that makes you happy to be using a computer instead of doing this stuff before our digital age, like attempting homebrew calligraphy with a ballpoint pen. So, they had a nice feature that I already mentioned. You could mark the first row as the row specifying your field names, effectively starting your data on the second row. Neat! It even worked! Except... for our recent batch of labels, when selecting fields to place in the label, only the first field was showing, and truncated as "Mr." That's odd. We tried dropping the header row, and the problem still showed up. I guess I should add that the first column value of the first row was something like "Mr. & Mrs. John Doe." It dawned on me that maybe these mindless developers were so bad that a simple & character could screw up the mail merge. I moved the row down and put in a more innocuous row at the top. Sure enough, it worked smoothly! So, let me get this straight... a mail merge that could have completely arbitrary data in it breaks down because of a simple & character???!?!? I shudder at the thought of what security holes might be present in this application.

Ok, I have saved by far the most egregious issue I have with their application for last. Their login process. Now, I can't for the life of me understand why they need any of my information to let me generate a PDF. However, they had multiple required fields, including my name and email. Why do you, Avery, need this information to construct a PDF to help me utilize your real product? It baffles me. Well, I gave the information, and then had to go back and start again due to some of the issues above. I figured this process had implicitly registered me, since the form had looked like a registration form. When I went back and attempted to log in, I discovered that this was not the case at all. That irked me, so I decided to register. And, FYI, the registration form looked exactly like the form to just use their web tools directly... just with an added password field. I registered and then clicked the link to get back to printing labels, and guess what? I got the form again to get my name and email to start the label making process again! Are you KIDDING ME?!?!?!?!!!!!

In what could have been an amazing experience, Avery thoroughly dashed my hopes that more companies are starting to understand that cross platform web tools are the future. Instead of being a leader in the specialized paper printing business, they have made me shudder at the incompetent developers that they likely hired to do this job. There were nuggets of a really cool product, so there must be some smart people there... but they are likely drowning from a few really bad apples.

I think this kind of disproves Jeff Atwood's recent post that you should release a crappy version 1 and iterate. Or at least, it reinforces the thought that he included that you shouldn't release total crap for version 1. I guess Avery could redeem themselves in my view if they are iterating and actually listening to good feedback. If I come back in a couple months, and all of these issues are somehow tackled... I could forgive these transgressions. I seriously doubt it will be the case though.

I think Avery is stepping in the right direction with their online tools. They have a lot of potential there. Unfortunately, their execution makes me embarrassed as a fellow web developer.

A Few Worthy Bytes

2009-12-04T01:30:00.000-08:00

Bandwidth is a lot cheaper now than it was 10 years ago. Dialup is a dying breed. So please, please, please don't craft your HTML to save every last byte. It is a lot more worthwhile to make your code readable and immediately intuitive than to prevent an extra 10 bytes from going to the user. If you find that you are causing huge downloads of extra HTML, and it is becoming an issue based on real statistics... then start looking into addressing the issue.

For example, consider the following snippet:

<a href="/path/elsewhere"><%
  if @condition
    %>True text<%
  else
    %>False text<%
  end
%></a>

It may not look that bad alone, but when you try to save every byte, the result months from now will be a garbled mess that is difficult to comprehend. Instead, ignore those extra bytes and produce the following:

<a href="/path/elsewhere">
  <% if @condition %>
    True text
  <% else %>
    False text
  <% end %>
</a>

If outputting all that extra useless whitespace makes you feel too icky, Rails has an option for you. If you close your scriptlet tag with a dash, as "-%>", Rails will strip some of the whitespace.

<a href="/path/elsewhere">
  <% if @condition -%>
    True text
  <% else -%>
    False text
  <% end -%>
</a>

Just remember that someone has to maintain the code, and making code more difficult to comprehend will end up costing you more than the bandwidth you saved.

Free and Loving It

2009-12-02T01:56:00.000-08:00

I have a soft spot for Open Source Software. When I compare two software products, I will almost always lean towards the Open Source alternative, if it can accomplish my task with some amount of pleasure.

Just tonight, I was working with Open Office with my fiancée. We ran into several kinks along the way, ultimately costing us the night. We couldn't finish our task, though I mostly blame my procrastination.

In particular, we were trying to print out some labels, given a spreadsheet of addresses. I had no problem finding howtos and working through the problem. However, we ran into small problems here and there. For example, I couldn't create a database out of a spreadsheet right away because Ubuntu didn't package the Database portion of Open Office along with the rest. I then innocuously named our database SomeName_12.1.2009. When trying to print, this ended with an error that it couldn't connect to the database. Renaming the database to SomeName_12_1_2009 solved the problem. Something we could figure out easily enough, but not something I would expect an average user to think of (nor do I think they should have to think of it). She became frustrated, rightfully so, and ultimately exclaimed that she's going back to Windows (for her Office needs).

At this point, it dawned on me that I don't mind giving Open Source the benefit of the doubt. The hacker in me enjoys figuring out how to work with the software (as long as what I want to do is possible, and reasonably easy). The hacker in me also appreciates that the project was built by people that are likely much more passionate about writing software, and much more passionate about getting something useful to people (instead of making a bunch of money). This is not to say there aren't people in the proprietary world that love software and want to deliver awesome stuff to people, but I just can't identify with that crowd of enthusiastic developers as much.

This is also why I foam at the mouth when I read comments like Jeff Atwood's:

as predicted, Google's "let's copy how Microsoft does phones, but open source!" model is a fail: http://is.gd/589U8

I've read the article he links to, and I consider it complete bullshit. I have a G1, and I love it. I have played with the Droid, and I drool over it. I know several people that have one (or some other Android powered phone), none are unhappy about the pick. Jason Calacanis of This Week in Startups (among many other things) has commented on his show that he loves his new Droid. Browsing some of the comments on that negative post, I see several that point to rogue processes as the likely culprit of the device slowdown discussed in the article. I've found this to be true of my G1. Sometimes I will discover my battery drained much faster with little reason during the day, or it will become extremely sluggish. Both cases I've had more than enough reason to believe it was a rogue process from something I had installed. With greater power comes greater responsibility.

In the Linux side of things, I have become far too attached to Emacs and the powerful command line based applications that I would never willingly go back to a Microsoft prompt. The Free world gets me, and I get them. I tend to believe in live and let live sort of philosophies, which has no room for restrictive licenses and the likes of DRM. Software patents scare me because I want to be able to develop anything I want, without having to worry that someone else may have already thought of the idea and patented it. I also don't care if someone takes my ideas and tries to make them better. I may be a bit envious, but I firmly believe the meat of a product is in the execution, not the imagination. Ideas are a dime a dozen, but passion for your users and the desire to develop something of quality and value is truly rare.

I don't understand the Microsoft world, and I don't want to. Which world do you identify with, and why?

Goodbye Mercury News

2009-11-30T00:02:00.000-08:00

Newspapers can't go away quick enough. I blogged about the newspapers recently, but I was speaking merely as an indifferent party at the time. I don't read the paper, and so I don't really care if they survive or perish. Now I have a strong opinion. I want them to go away, and I want them to go away soon. Well, this isn't entirely fair, because I'm basing this strong hatred on the actions of a single particular paper... the San Jose Mercury News. Presumably not all papers act in such sleazy, annoying manners, but any that do... I hope they disappear tomorrow. Scratch that, I hope any business at all that acts this way perishes tomorrow.

Ok, enough teasing. Rewind a few months. My doorbell rang, and I went and looked out the peephole. There was a strange kid... and I foolishly opened the door. It was a kid going door to door selling subscriptions to the San Jose Mercury News in an effort to get help going to college. I had actually helped a kid doing the same thing a year or two before, and I didn't mind helping this kid too. The last time I had given cash, but this time I was out. Then, I made my second mistake... I paid with a check. The kid needed my phone number, supposedly so the Mercury News could verify I was indeed helping him out. I reluctantly gave him my number... my third mistake. Somehow, I knew I was making a mistake, and immediately wished I had just closed the door on him. I had half a mind to call him back and write a new check out to him, and let him cash it and keep it, or buy a paper for his school or something. I indicated I didn't actually want the paper, and that he could give it away... I think he ended up giving it to a neighbor nextdoor that wasn't home.

Fast forward back to present time. The whole thing ended up being a scam. I'm sure the kid got some help from the Mercury News towards college, but at the cost of the Mercury News getting my phone number. I got a call shortly after the subscription ended with a request to resubscribe. Damn are they aggressive when calling! It took a few minutes for me to dash her confidence and finally end the call. I thought that was that, but have since received at least two more calls, one coming just this last Saturday. A holiday weekend no less! I would think they would stop calling if I emphatically tell them I am not interested, and even explicitly say I have never read the newspaper, and never intend to. I guess I'm just a number in a big list of possible sources of revenue now. Next time I guess I just need to tell them to make sure I'm off their list. And the next time some poor kid is going door to door selling San Jose Mercury News? Sorry! I will send 'em packing. I don't mind helping a kid go to college, but not if it is just to get my number on a list to cold call every other month.

I think I walked away with another lesson on how to treat your customers. If you might be bugging them, stop and reconsider what you are doing. Your sources of revenue should be from people that love what you are doing for them, not people so annoyed by you that they buy your product just to shut you up. It's definitely a no no to set up a faux charity just to turn around and annoy the contributers.

Closures for Java 7: DOA

2009-11-24T23:36:00.000-08:00

To start off today's (probably) brief post, I want to quote Stephen Colebourne's blog:

JDK 7 closures will not have control-invocation statements as a goal, nor will it have non-local returns. He also indicated that access to non-final variables was unlikely. Beyond this, there wasn't much detail on semantics, nor do I believe that there has been much consideration of semantics yet.

Now, I can't vouch for his facts, but it seems accurate, so I am going forward with the assumption that I'm getting it from the horse's mouth.

That said, I want to state what closures in Java 7 will be if they are implemented as stated above. Can you guess? Yeah... sugar syntax for anonymous classes.

As a Java developer, I don't want to complain. Anonymous classes are a major pain in the ass. Any time I attempt some functional programming with them, I always look back and think... that would be so much more elegant with a foreach loop, or some such thing. Anonymous classes are a lot of syntax for very little meat. Getting rid of the painful parts of that syntax will definitely be a good thing.

As a Ruby developer, these so-called "closures" are a laughing stock. Can anyone really claim these are actually closures? Let's just stop kidding ourselves and call them elegant anonymous classes.

Without control-invocation statements or non-local returns, you can't turn a foreach into a method call with a closure. Without access to non-final variables, you have to either move the variable into the class, wrap it in an object, or do the horrible 1 element array trick. You know... construct a 1 length final array which you can then both get and set the element from within the anonymous inner class... it blows.

If my vote matters, it is for waiting till Java 8 and doing closures right, or giving us a little meat for Java 7 closures. Bare minimum, non-final variables have to be accessible.

Smart XML Processing with Regexes

2009-11-22T16:34:00.000-08:00

Recently, Jeff Atwood wrote about parsing HTML with regular expressions. I want to speak about it briefly, because I came across this issue last week. I gathered from his post that the lesson is to consider your options with an open mind, and only block a possible solution if you really understand the alternatives. Use facts and knowledge to choose your implementation details, not superstition and theoretical best practices. Best practices usually are created for a reason, but that's not to say there's never a reason to turn your head on them.

This post hit home with me because I had an XML file to parse that was over a gigabyte. From this XML file, I needed a very small handful of the data, and it was very regular XML. XML parsing is a solved problem, but most XML libraries I've used would easily choke on such a file.

Instead of even considering attempting to process this data with a normal XML processor, I wrote a simple Ruby script to extract the information. It looped over each line, looking for key parts of the data with lines like:

if line["<expectedTag>"]
  # deal with this tag
end

Then, I processed the key tags and data I was looking for with regular expressions, such as:

data = line[/<expectedTag>(.+)</expectedTag>/, 1]

The above was done within the if blocks. The key point being regexes would have been too slow alone, so I used the simple indexer method to quickly determine if the line contained something that mattered to me. Then I used the regex to pull the data that I actually wanted.

Can you write XML to break my processing? Of course! The question is... does it matter? And that answer was no. I only need to process this data once, maybe another time sometime in the distant future, but the XML is so regular that I know it will work for all the data. On top of this, if I missed some data, it wouldn't matter in the slightest for my purposes. So, in short, proper XML processing would have severely slowed me down (ignoring all lines that don't contain a keyword is much faster), and it would have produced no real benefit.

I ended up processing all the data in little over a minute or two, and I considered it a huge success. Over a gigabyte of XML to process seemed a rather daunting task initially!

Serve Your Users

2009-11-18T22:21:00.000-08:00

I'm a bit upset. Some friends and I were planning a trip to San Francisco soon, and a few of them have booked a night at the Sheraton Fisherman's Wharf (don't worry, I will tie this in to software in a bit, trust me). I needed to book a night for me and my fiancée, so I brought up their website. Uh oh! The hotel was booked solid that night. This was bad... what if we have a hard time finding a place? This wasn't what made me mad though... well, besides at myself for not booking earlier.

What if the website wasn't accurate? I dialed up the hotel, just to be sure. It went something like the following (though it's coming from memory, so expect a bit of embellishment):

Me: Hi! Do you have a room available for the night of X?

Them: I'm sorry, I don't see anything available. Is the night flexible?

Me: Well, my friends already booked the night with you, soooo...

Them: I can check the Starwood Hotels, Le Meridien. It is about a mile away. Shall I check availability for you?

Me: Uuuuh, well, my friends are already staying at your hotel. Is there anything nearby that might have a room?

Them: ... It's only a mile away. Shall I look that up for you?

Me: Sure.

... She proceeds to book a night at Le Meridien, informing me of an offer comparable to what my friends had, though I made sure I had a refundable option so I could think it over ...

Ok, so this may seem like pleasant help from the reservations department at the Sheraton, but it's not quite why I'm angry. You see, after I hung up, I first checked how far on the map the 2 hotels were. It ended up being 1.4 miles... not exactly easily walkable for a night on the town. This wasn't why I was steaming though.

I then did a quick search of the nearby hotels to the Sheraton. I zoomed in on Google Maps, and all the hotels nearby were listed right on the map. The Hyatt, a block away. Holiday Inn, a block away. Best Western, across the street. Radisson, across the street. This was when my anger bubbled up. I called up the Best Western and found out that not only was a room available, but I could get the same price my friends got (and which was offered me at Le Meridien). I quickly cancelled the night at Le Meridien, quite thankful I didn't rush into the no refund deal I was initially offered.

Let's be clear, I fully understand where the Sheraton employee was coming from. They may get some kind of commission for redirecting my business to their sister hotel. They want to ensure they are getting my money. What irks me, though, is that I made it clear I preferred to be near my friends, yet she proceeded to push an option to me when she very likely knew full well there were alternatives that would have suited me better. It may be that any of the hotels I saw would do the exact same thing in a heartbeat, but I feel it is a grave mistake.

First, the Sheraton had a great opportunity to turn me into a fan. Had they pointed me to one of the numerous walking distance competitors, I would have remembered that fondly, and told everyone about my experience. Not many companies clearly have your best interests at heart. Instead, I remember it angrily... and tell everyone about my experience.

This is how I feel this story relates to software... well, more about business, but same thing if you are a software company. The way you need to treat your customers is as if your goal is to see their goal achieved in the way that best makes them happy. If that means pointing them to a competitor who would solve their problem better... then happily point them to your competitor's open arms. Don't treat your customers (or potential customers) as if their money is the only thing you care about, like the Sheraton did in this case. Your users will find out you weren't being completely honest, and they will hate you for it. They will speak out and write on some puny but public blog and tell everyone about the experience. Ultimately, your users will find their way to the option that is aligned with solving their problem, not extracting their money.

Cautious Development

2009-11-11T21:08:00.000-08:00

One of the most appealing features of Test Driven Development for me is that it helps you write code that actually works when you are done. If you are testing at each step, you end up with code that works for all those features that you explicitly tested. This is not to say you will end up with bug free code, of course. Nobody but a pointy haired boss would expect that.

All too often, I see code that is supposedly done, but a cursory run through some simple examples shows earth shattering bugs. Bugs where the basic features being implemented don't even work. What possesses a developer to think something is done when it hasn't been played with for a while, showing some level of completion? I guess we all fall into the trap of simplistic changes that can't possibly cause a problem, only to have some bug crop up specifically because we aren't looking. Sometimes some manual (or even automated) tests would be so tedious to set up that it just doesn't seem worth the effort. But I've seen cases where it's clear the code was not very carefully crafted in any regard, with no reason that it couldn't have been done better.

The most absurd example came from someone I helped interview quite a while ago. I distinctly remember going in and running the beginnings of the code with him. I put in some input and saw some output... all was good. When I returned a while later, the IDE still showed the exact session we had run as much as an hour earlier. Not only did the code not even compile, it had no hope of working even if we could trick the compiler into giving us a green light. Is it a rare quality among developers to actually play with the code as you go along?

I'm not saying you need to exercise the code in a particular manner, just that you exercise it in any manner. Run the code at every step, playing with the new features you are working on, and sanity testing some older features. Write it test first and watch new tests succeed as older tests continue to succeed. Hell, even take a waterfall approach with a big bang batch of code, then furiously cycle through short runs to find and fix a bug. I don't care, as long as when you say you are done, it isn't trivial to find a bug in the code.

Unsurprisingly, I'm also a fan of writing your code in the most cautious manner possible. Some developers seem to like to go in to old code and just run wild in it, tearing things out and replacing them left and right with no regard or respect for something that might be doing the job well, or at least well enough. Sometimes a piece of code can prove to be so obscure and hard to maintain (or even understand) that it makes little sense but to throw it out and start from scratch. That should be more rare than common, though.

When it comes to refactoring, I like to go in and be very careful that the new code preserves equivalent functionality, especially if the code isn't covered by a good suite of tests. Don't go in and replace a whole class with a new class that does the same thing in a way you like better. Instead, move code around and massage it into the shape you need, slowly but surely preparing it for the new feature you need to add. Continually ask yourself if the shape of the code does exactly what was being done before (unless of course you discover a bug). Pretend that if you introduce a new bug as a result of your changes, the entire company and all the users will come and yell at you and deride you for making such a mistake. Worry for every millisecond that you might be making a breaking change.

Care for your code. Envision the code is your lover. Would you do a single thing that might hurt your lover's feelings? Would you want her to stub her toe because you moved the table to the wrong location? Then don't make sweeping changes without being extremely cautious, because you will only guarantee to stub your code's toes. If you treat your code right, she will only get more beautiful, while learning new and exotic tricks.

View Sanitizing and Micro-Optimizing

2009-11-09T23:18:00.000-08:00

Maurício Linhares posted an intriguing response to my recent post about auto-sanitizing Rails views. I was just gearing up to respond via a comment when I realized I could probably turn it into a full post, so here goes my response!

So, first of all, let me applaud Maurício for actually writing some code and sharing it, rather than keeping the discussion academic and merely flinging arguments around the interwebs. I can't say I always do it, but I have the most fun writing a blog post where I show some code to achieve a solution to a problem I am having. He even went the extra mile to create a plugin for his idea.

That said, I must respectfully disagree with his approach. The gist of how he tackles the problem is to sanitize data as it comes in rather than when you are displaying it. He even argues in his blog post that it is "more adherent to the MVC":

Now, as the data is cleanly stored in your database, you don’t have to waste CPU cycles cleaning up data in your view layer (and you can even say that you’re more adherent to the MVC, as cleaning up user input was never one of it’s jobs).

He makes a convincing argument that the view layer should not sanitize input. My big problem with this is that you have actually taken very specific view details and moved it into the controller and model layers, contrary to what he is claiming. Namely, you have introduced into the controller/model layers the idea that your data is going to be displayed via HTML. However, what if you want to expose a JSON API later on via the same controllers, but with new views? Now, you will need to unsanitize the data, and resanitize it for JavaScript output! You have inadvertently snuck view information into the database! Your data is pigeonholed as HTML data, and it now takes double effort to use the data in another matter (such as JSON data).

This last point deserves some extra attention. Consider if our transform on the data was a lossy transform. This case isn't, because you can easily unsanitize sanitized HTML, but forget that for a second. For example, let's say we wanted all data to be sanitized and censored, such that words like "ass" and "crap" got changed to "***". If we had a bug that caused "crass" to be changed to "cr***", we have just lost information that is irretrievable. If we saved the sanitizing and censoring for the view, where it belongs, we could always fix the censoring code and our "high fidelity" representation will allow us to now correctly show "crass." Let me quote a Stack Overflow podcast, where Joel explains this same position:

Spolsky: Here's my point. Uhh, in general, my design philosophy, which I have learned over many years, is to try and keep the highest fidelity and most original document in the database, and anything that can be generated from that, just regenerate it from that. Every time I've tried to build some kind of content management system or anything that has to generate HTML or anything like that. Or, for example, I try not to have any kind of encoding in the database because the database should be the most fidelitous, (fidelitous?) highest fidelity representation of the thingamajiggy, and if it needs to be encoded, so that it can be safely put in a web page then you run that encoding later, rather than earlier because if you run it before you put the thing in the database, now you've got data that is tied to HTML. Does that make sense? So for example, if you just have a field that's just their name, and you're storing it in the database, they can type HTML in the name field, right? They could put a < in there. So, the question is what do you store in the database, if they put a < as their name. It should probably just be a < character, and it's somebody else's job, whoever tries to render an HTML page, it's their job to make sure that that HTML page is safe, and so they take that string, and that's when you convert it to HTML. And the reason I say that is because, if you try to convert the name to HTML by changing the less than to < before you even put it in the database. If you ever need to generate any other format with that name, other than HTML - for example you get to dump it in HTML to an Excel file, or convert it to Access, or send it to a telephone using SMS, or anything else you might have to do with that, or send them an email, for example, where you're putting their name on the "to" line, and it's not HTML - in all those cases, you'd rather have the true name. You don't want to have to unconvert it from HTML.

Yes, it is tedious and error prone to use "h" everywhere, but that is the exact same problem I was trying to address in my post. However, I feel training myself to use <%: foo %> over <%= h foo %> is a better muscle memory than marking all input as sanitizing. Let's consider the consequences if you forget to apply the new scriptlet versus if you forget to apply the sanitizing of inputs. If you forget the new scriptlet, you have a new XSS hole that needs to be closed by simply changing "=" to ":" (or alternatively adding a call to "h"). If you forget to use the sanitizing of inputs, you have 2 major problems. You have an unknown amount of XSS exploits (everywhere you display that data, which could be in many places). You also have a bunch of data that is now invalid. You now need to either add sanitizing to all the view locations you output the information (which would be tedious and contrary to the whole point of Maurício's approach), or you need to update all existing records to be sanatized, just before enabling sanitizing of input.

There is another issue with this approach that a new scriptlet tag avoids. By making the sanitize decision in the view layer, you have the option of exactly what you will sanitize. Let's consider a site like a blog or Stack Overflow. In such applications, you want some amount of HTML displayed, though not necessarily on all fields of the model. You might want to whitelist sanitize the blog post, question text, or answer text, yet fully sanitize the labels or tags. Granted, you could update the plugin to allow such complexity, but it will be just that... complexity that will bleed through how you invoke the plugin. You will now need to not only specify which actions or controllers use this sanitizing, but also which parameters are excluded from being sanitized.

All of the above pales in comparison to the biggest sin of sanitizing the parameters, and it is one of my biggest pet peeves. It is one of the key points for why Maurício chose the path he did. Premature optimization.

The argument goes that rather than waste the CPU cycles every time you load the page (which is hopefully a lot), you should waste the cycles once as the input is being passed in and saved to the database. Premature optimization usually rears its ugly head in the form of much more insane choices, like insisting on how you should concat your strings. Thankfully, Jeff Atwood has already done metrics showing us that it doesn't matter.

Is sanitizing as quick as string concatenation? Probably not. I would be willing to bet, though, that it is fast enough for a small website. Why waste extra consideration on it until you have the awesome problem of having too many users?

Let's take a step back. Is pushing View logic into Model/Controller territory even worth the possible performance benefit? If I am going to throw proper MVC separation concerns out the window, it better be for a damn good reason. Allowing us to get orders of magnitude more pageviews might be worth it (if metrics proved that it was the best possible improvement, which is doubtful, but let's consider it). The whole concept is to cache something you are doing a lot by doing it once, before it even goes to the database. Let's extrapolate that concept. Instead of sanitizing first so we don't have to sanitize on every page view, why don't we cache the result of the action invocation itself? For every action/params pair that produces a view based strictly on data in the database, we could invoke the action once and just cache the rendered view for future use. Then, simply blow the cache away and re-render when you change the related database row(s). It is typically much more common to view than to update, so I expect this approach would give significantly better performance benefits than simply avoiding a bunch of sanitization calls.

With some careful thinking, we now have a much better solution to remove all the redundant sanitize invocations... and we've even removed redundant calls to the database, and any other costly algorithms we have done within our actions! All while preserving proper MVC separation of concerns. You can bet that I will explore this space when I have the fortunate problem of having too many users (and I wouldn't be surprised if there are available solutions that match my description).

Sorry for going off so much on your very well-meaning post, Maurício! I think you brought a very interesting possible solution, and it's always great to see code brought to the table. However, I do feel we should all seriously consider all approaches, and fully consider the consequences of the path we choose... not just to this particular problem, but any problem. It's best to drill down early and think about what issues our code may cause for us in the future. Don't consider this an excuse to dwell on issues to the point of failing to release useful functionality, though.

Easy Partials in Rails

2009-11-05T10:36:00.000-08:00

I created something at my job that has proven to be extremely useful that I think many people could benefit from. I like to call it Easy Partials, and the goal is to make using partials a bit easier (in case the name didn't make that glaringly obvious). The problem is that rendering a partial requires method calls when a little extra work will allow simpler and more readable partial invocation via convention.

You are probably lost, so let me give you some examples. As it stands today, to render a partial you would do:

<%= render :partial => "my_partial" %>

Which would take the partial "_my_partial.erb" from the same directory and render it.

It works, but what if you could take the whole "convention over configuration" idea and change it into:

<% _my_partial %>

A lot simpler, and pretty intuitive, right? Since partials by Rails conventions start with "_", it makes sense to just name a method as such to render the partial. Note that there is no "=" in the scriptlet, the _my_partial method will concat the partial, so you won't obtain a string to render.

We could create a helper method for every single partial we want to render like that, but that's rather cumbersome, isn't it? It's also not very DRY. You won't have exactly the same code, but you will find yourself with a lot of helpers that look rather similar. Instead, let's try overriding method_missing in our application_helper so we can avoid all those repeated helpers!

module ApplicationHelper
  alias_method :method_missing_without_easy_partials, :method_missing

  def method_missing_with_easy_partials(method_name, *args, &block)
    method_str = method_name.to_s

    if method_str =~ /^_.+$/
      partial_name = method_str[/^_(.+)$/, 1]
      concat_partial partial_name
    else
      method_missing_without_easy_partials method_name, *args, &block
    end
  end

  alias_method :method_missing, :method_missing_with_easy_partials

  # Concat the given partial.
  def concat_partial(partial_name)
    content = render :partial => partial_name
    concat content
    nil
  end
end

What we've done here is check on method_missing to see if the method name starts with "_", and, if so, treat it as a partial and concat it. If the method doesn't start with "_", we fall back to the original method_missing implementation.

This works, but what if the partial needs some local variables? Before you would do:

<%= render :partial => "my_partial", :locals => { :var => "123" } %>

Instead, let's do:

<% _my_partial :var => "123" %>

Again, a lot simpler, and quite intuitive. To achieve this, our code will look like this:

module ApplicationHelper
  alias_method :method_missing_without_easy_partials, :method_missing

  def method_missing_with_easy_partials(method_name, *args, &block)
    method_str = method_name.to_s

    if method_str =~ /^_.+$/
      partial_name = method_str[/^_(.+)$/, 1]
      concat_partial partial_name, *args
    else
      method_missing_without_easy_partials method_name, *args, &block
    end
  end

  alias_method :method_missing, :method_missing_with_easy_partials

  # Concat the given partial.
  def concat_partial(partial_name, options = {})
    content = render :partial => partial_name, :locals => options
    concat content
    nil
  end
end

Now we are passing the Hash passed in from the view on to concat_partial so we can specify the locals we want to render. We could check that there is no more than 1 argument passed into method_missing, but I prefer not to (feel free to use and improve anything you see here, in case that wasn't clear).

The next improvement we can make is to allow blocks to be passed in. There is no direct correlation for this next part, that I know of, except to build it yourself with helper methods. It was inspired by Ilya Grigorik.

Here is an example of what we will build:

<% _my_partial :var => "123" do %>
  <p>
    Some block content.
  </p>
<% end %>

This will allow us to effectively pass in a block to the partial, so we can abstract some of the content in the partial so that the caller can define it. And now for the code:

module ApplicationHelper
  alias_method :method_missing_without_easy_partials, :method_missing

  def method_missing_with_easy_partials(method_name, *args, &block)
    method_str = method_name.to_s

    if method_str =~ /^_.+$/
      partial_name = method_str[/^_(.+)$/, 1]
      concat_partial partial_name, *args, &block
    else
      method_missing_without_easy_partials method_name, *args, &block
    end
  end

  alias_method :method_missing, :method_missing_with_easy_partials

  # Concat the given partial.
  def concat_partial(partial_name, options = {}, &block)
    unless block.nil?
      options.merge! :body => capture(&block)
    end

    content = render :partial => partial_name, :locals => options
    concat content
    nil
  end
end

Within your partial, you will use a "body" variable to output the contents of the block passed in. If you try to use a variable named "body" along with partial blocks, the block will override the body variable, so keep that in mind.

For the final improvement, consider a partial that belongs to more than one controller. What do we do then? Well, how about we maintain a shared directory that we pull from if the partial cannot be found within the local directory. Thus:

module ApplicationHelper
  alias_method :method_missing_without_easy_partials, :method_missing

  def method_missing_with_easy_partials(method_name, *args, &block)
    method_str = method_name.to_s

    if method_str =~ /^_.+$/
      partial_name = method_str[/^_(.+)$/, 1]

      begin
        concat_partial partial_name, *args, &block
      rescue ActionView::MissingTemplate
        partial_name = "shared/#{partial_name}"
        concat_partial partial_name, *args, &block
      end
    else
      method_missing_without_easy_partials method_name, *args, &block
    end
  end

  alias_method :method_missing, :method_missing_with_easy_partials

  # Concat the given partial.
  def concat_partial(partial_name, options = {}, &block)
    unless block.nil?
      options.merge! :body => capture(&block)
    end

    content = render :partial => partial_name, :locals => options
    concat content
    nil
  end
end

So, if you use the _my_partial examples from above within the views for "person_controller", but there is no views/person/_my_partial.erb, it will fall back on views/shared/_my_partial.erb.

Using Easy Partials, you can avoid redundant helper methods, keep html in easy to access ERB templates, improve the readability of your views that use partials, and make your code more accessible for your non-programmer UI designers. Note that you can even invoke Easy Partials from within helper methods.

Special thanks to Ilya Grigorik's post on block helpers, which planted the seeds for Easy Partials.

Dev Days: Mobile

2009-11-04T21:15:00.000-08:00

Let's revisit Dev Days San Francisco. Last time, I talked about the Microsoft talk, and how I was able to incorporate auto sanitization into Rails. This time, I would like to discuss the three mobile talks that were presented. Just for full disclosure, I am a big fan of Android, so you will likely see that bias come through. I know I felt the bias while taking in the three talks.

iPhone

First up was Rory Blythe talking about iPhone development. Of the three presenters, he was the most at ease speaking in front of a crowd. I'm sure Rory is a nice guy, but he had a swagger that... felt very Apple-esque. He was the hipster Apple guy smugly looking at us and telling us why we should cow down to the almighty Steve Jobs, and why we should be happy to do so. He made the audience laugh the most, most often with flaws in the iPhone development platform that we all know is an issue. Cheekily telling us three or four times that you cannot develop for the iPhone on anything but an Apple machine sure doesn't sit well with an Ubuntu fan like me. I don't think he was trying to sell the platform, though, just give us a taste for what it was like.

Most of the talk was him walking through the steps to develop a simple application in Xcode. It was very visual. Click here and drag over there to hook up a click handler of some kind. Drag this widgit in line with these others. Ultimately, we got to peek at some actual Objective C. If you weren't at Dev Days and you have never seen Objective C, be thankful you didn't have to, because it ain't pretty. He joked about how many places you have to adjust the code just to set up a simple property. As a Rails web developer, it made me want to claw my eyes out. I would never want to develop in such a redundant, cumbersome language.

There was good news, though. He introduced us to MonoTouch, a development environment for producing iPhone applications written in C# with the .NET platform. I'll take Ruby any day, but between the choices of C# and Objective C, it is a no brainer (C# of course... from someone who generally despises Microsoft technology). Too bad I can't use Ruby to develop an iPhone app on an Ubuntu machine. I might consider iPhone development then, though not seriously. If you wonder what is wrong with me... well, I choose an open platform, because I feel in the long run it will win out to a closed platform. I am convinced all the bad press towards the App Store approval process fiasco, and Apple's draconian attitude towards the 3rd party development community will ultimately be the iPhone's undoing.

Maemo/QT

The next mobile talk came from Daniel Rocha of Nokia. He talked about Qt, specifically in the context of Maemo. He was very dry. Dryer than a desert when compared to Rory. Some people just aren't built for speaking... I'm sure I would freeze up and be just as bad, but thankfully I don't seek such torture for myself.

It was a running theme to do some actual development on stage to demo the platform for the audience. The core message of his talk was "Look! Cross platform! Same code for all the major platforms, including mobile!" Alright, C++ cross platform I guess is kinda cool, except it's not that impressive in today's context. Java has been highly cross platform from the beginning, and all the primary scripting languages are quite cross platform as well.

The most impressive aspect of his cross platform demo is that the UI is among this cross-platformness, which I suppose is the whole point. However, he admitted about still needing macros to segregate the platforms now and then. And I really don't believe that a mobile UI should be the same as a desktop UI. With the state of mobile devices, I think it's wiser to develop a UI with a small screen (and touch screen capabilities) in mind. Negative points also for running Linux in a VM inside Windows instead of the other way around.

The Qt IDE he was showing off was also quite unimpressive. It looked like Visual Studio circa 2000. The UI designer looked exactly like an old Visual Studio UI designer, with that grid of dots and all. I got the feeling that anyone used to a good IDE like Eclipse would feel shackled in this thing. Anyone used to an awesome editor like Emacs will... well, an Emacs user would never stoop to a lesser editor of any kind.

The funny thing about this talk is that it was followed by an Android presenter who pointed out how most major manufacturers, except Nokia, were coming out with devices with Android. All I can say is, Nokia is dropping the ball if they stick with this platform over hopping on Android. I saw no compelling reason to develop for Qt/Maemo.

Android

James Yum finished the Mobile hat trick with an Android talk. I wanted to be blown away by this talk, but unfortunately Rory was the only awesome speaker of the three. To be fair, James was up front that he was asked to do this talk last minute, replacing the original Google speaker, who probably would have crushed it.

He started off by showing the "Droid Does" Verizon commercial, which was an awesome commercial (despite not actually showing the device, and despite coming from a company I'm not a fan of). He then read some of the YouTube comments for the commercial, to humorous effect. I really hope T-Mobile comes out with a phone this compelling. According to James and the commercial, it has an 800x480 screen with a physical keyboard and a powerful chipset.

Great start, lousy finish. He followed this up with an uninspiring demo of how to deal with threads in Android, with the specific goal of making a snappier UI. He went from a simplistic but completely incorrect way to do threading to an object oriented way using the Android API. It made developing with Java look almost as bad as Objective C. I'm not exactly a fan of Java (though I work in it almost every day), but it really isn't this bad to work with Java, and specifically Android. I think he would have better shown off the platform by taking a real (though small) application idea, and implementing it on stage. This is what Rory did, and it had a much greater effect than slogging through the most complicated aspect of modern programming you could possibly think of.

Overall, James was far too green for the speech. He was (understandably) clearly nervous, and he was unable to answer most of the questions at the end of his talk. I was silently rooting for him, hoping he could show all the would-be iPhone developers what they are missing, but I was disappointed. Maybe Google will learn from this and keep a good speaker on hand as a backup, should the primary speaker have to drop out last minute.

Accidental Complexity

2009-11-03T04:02:00.000-08:00

I thought I had some interesting insights in my last post about Postgres and why I was going back to MySQL, at least for the time being. So much so, that I posted it on Reddit. Alas, it didn't go over well. It had a lot of downvotes, and most of the comments were fairly negative. I think I'm still right about what I did and why, but I think I can better articulate why I think so in two words:

Accidental Complexity.

It's definitely not a new concept, and I'm sure many people have talked about it before, but in reading the responses on Reddit, it became clear that this is precisely the reason why I switched to MySQL, and precisely why I probably won't switch back anytime soon.

Wikipedia defines accidental complexity as "complexity that arises in computer programs or their development process (computer programming) which is non-essential to the problem to be solved."

This perfectly describes the issues I was having. Think about if I could sudo apt-get install postgres, hook it up in Rails, and be on my way. I could focus nearly all of my time on essential complexity (ie developing features for my actual application), or at the very least, accidental complexity arising from other applications or from my coding choices.

The choice may bite me later on with other accidental complexity that Postgres tackles well, but right now I want to actually get the project to a point where I can start showing people, and maybe start getting some users. After all, getting users is the whole point. To that end, reducing the hassle being caused by the database is a Good Thing.

So, I really think this should be a lesson for all the applications I write, and all the applications you write as well. Make sure your application reduces the accidental complexity you are forcing on your users. If a significant portion of them are spending a lot of their time configuring your application instead of solving the problem they are trying to solve, you have failed them.

Don't take this to the extreme, though... you still have to see the big picture. Who knows, maybe the default user settings is good for the majority of users, and I just happen to fall in the unlucky minority, but comments like "pg_hba.conf can be a bitch, but updating it to allow transparent local access is just a couple of lines away" tells me that I'm not the only person who has had trouble configuring Postgres, even if the configuration changes are small.

Postgres... is it worth it?

2009-11-01T19:01:00.001-08:00

In working on my side projects, I needed to pick a database. I decided to pick Postgres, and it has been nothing but problems since the beginning. It sucks, because half the problems don't really seem to be the fault of Postgres, though they all seem to stem from the issue that IS their fault.

So, when I first started with Postgres, right off the bat it is a pain. Every single time I need to install Postgres (usually on a new development machine), it is a pain in the butt! They provide a lot of flexibility for database user security. However, the last time I installed MySQL (which is admittedly a long time ago), you set it up, and it asks you for a root user password, and that's it. The rest seemed a snap, at least in the rose colored glasses of the past. With Postgres, each and every install requires me to search to figure out what settings to change in which config file just to get the damn users to be able to connect! Give me some reasonable defaults! If I try to connect to my database locally with a specific user, it would be awesome if the system would give full permissions to any databases I create with that user. Then things like rails would work out of the box, no configuration necessary. I will lock down the system to my hearts content if I need to, but make it JUST WORK first.

That was probably the extent of what I can actually blame Postgres for. It's a big fault in my opinion, though. Software that just works out of the box with no configuration necessary is nirvana. That is why Rails typically feels like heaven to me.

The next issue was probably HostMonster's fault. They give Postgres a back-seat treatment. They run an old Postgres version (I think 8.1), with the excuse that they need to wait till cPanel (their site management web app) upgrades what they support. Ok, I can live with that, but it sucks. When I actually tried to create a database, though, Rails couldn't connect to it! Grrr. HostMonster was on it though, and actually fixed my configuration issue. That was cool of them, go HostMonster! I have been a fan of their support, but that's not what this blog post is about. The issue had been a Rails configuration, I think changing how it was connecting... probably because HostMonster's user connection settings were set up a certain way, contrary to my Rails configuration.

Things were smooth until I needed to create a new database for a new website I wanted to create. It just didn't work. I couldn't create the database, and there was nothing I could do about it... possibly something HostMonster had done, but I didn't want to wait for a fix, so I took the dive and just used a MySQL database.

But I stuck with Postgres on my development machines. Why change? I still wanted it more than MySQL.

Until now.

I just upgraded to Karmic Koala on my netbook. I think it went well, except the upgrade from Postgres 8.3 to 8.4 didn't go so well (which was part of the Karmic upgrade). Both versions are now installed, and when I try to load Rails, I can't connect to the database. Running rake db:migrate even fails. No nirvana, no more Postgres. I'm fed up with the issues. I'm going to the dark side of MySQL, and I'm not going to come back until it is dead simple to set up a Postgres database with Rails.

Postgres intrigued me so much because of the recent drama surrounding MySQL... namely that the Evil Empire (Oracle) now owns them. Perhaps it's a bit superficial, but it's drama I didn't want to get caught up in. I want to know my database system will be open and free throughout the life of my products. However, MySQL makes things so easy that I just can't resist switching back. I will reconsider if it becomes easier to set up a database on Postgres out of the box.

Lesson? Make defaults that work towards achieving your users goals without them needing to dig into any documentation. If it's not easy to start, your users won't start, so you won't have users (at least not as many as you could have).

A Nerd's Proposal

2009-10-31T22:24:00.000-07:00

I got out of the car, and then helped her out. I was a bit nervous as we walked towards the restaurant, arm in arm. She reached her arm around me, edging her hand close to my inner left breast pocket. Uh oh! I stiffened my arm and blocked a potential revelation on her part. Safe for now (so I thought, anyway).

We walked into the restaurant and took a booth. It was New Year's Day, and not really that full. I had wanted a more significant location, but my first choice place was closed that day... drat! After some idle chat and ordering our meal, I figured it was now or never... or at least now or some other day. I wanted to do it now, so I took out my phone.

"I found this really cool Will and Grace app!" Her Christmas present was the Will and Grace box set. She was a big fan, so I figured a Will and Grace app would be interesting to her.

"Check it out, it's pretty cool. Click through that splash screen." I reached down into my inner coat pocket and grabbed the box. I opened it and waited.

She clicked through, selected a season and episode. She hit the item to play the episode, and after a brief pause, "Can't Take My Eyes Off You" from the Jersey Boys musical started playing. That's our special song, so I figured it was a good pick. I pulled up the box and held it out as she looked at a graphic with a couple hearts with the words "Will you marry me?"

Each moment that passed after that was stretched out like an eternity. She finally looked up and simply said "yes." That's it! I'm now engaged. She's the most wonderful woman in the world, and now I get to share my life with her.

Now that you know how it went down, let me tell you what went wrong. I settled on the day a few weeks before, while I was at my parent's house for Christmas vacation. I realized my Android phone would be the perfect way for me to pop the question... I could write a customized little app just for her! Google images proved useful for the "Will you marry me" graphic. It was a snap to rip our song from my CD musical recording. The hardest part was gathering all the content for all the Will and Grace episodes. Episode names was all I needed, but I had to take them from IMDB and convert them into Java strings. It was time consuming, but once all the content was set up, the Java app itself was a snap.

I guess that was what went well. The blunders came closer to the day. First came when I was asking her for dinner on the first. "Let's have dinner New Year's Day to ring in the new year!" Whoops. Very bad choice of words, especially since we picked the ring together... she knew it was coming, just not when. Why the hell did I say THAT??? She was of course suspicious, but I assured her it was just for dinner... I totally hadn't meant to give her a hint so early, but what's done is done.

We decided on the perfect place for dinner. Our 6 month anniversary had been at a nice fondu place. Perfect! I called them up, hoping to make a reservation, but alas! They are closed on the first! I expressed my disappointment to her... perhaps too emphatically. We agreed on another place. Not ideal, but I was determined to do this on the first.

New Year's Eve was at her parent's place. It was a lot of fun, and we were all packed to head over to Sacramento, which was our destination. I spilled some drink or something on my pants and all I had to spare was my nice clothes prepared for dinner. She suggested I change, and when she saw I only had nice clothes, she looked at me odd "we are dressing up for dinner?" I gave an excuse that it was New Year's... while wishing I had planned this a little better. All my effort went into the program to propose, why hadn't I worked out these details a little better? I'm not exactly good at thinking ahead and planning things well. If I could do it all over again, you can bet I would figure it out a lot better, but in the end it's the marriage that excites me more than the proposal.

With all those hints, she knew it was coming. In fact, she confessed she was reaching around while we were walking in because she wanted to test my inner pocket for the box. Sneaky! She saw me fiddling under the table when I passed her the phone, and thought "this is it." Thankfully, my application fooled her enough to second guess her inclination, so I salvaged a bit of a surprise.

I'm almost embarrassed to post to the world how unromantic my proposal was, but I think all the blunders made it memorable in it's own right... and it's very much me. I focused on the part I liked most, the programming, and assumed the rest would just go smoothly... because... you know, I had wrote a PROGRAM to PROPOSE! ON MY PHONE! I'm now just excited to become a part of her wonderful family, and know that I can spend the rest of my life trying to make her as happy as she makes me.

Quick Lesson from DevDays

2009-10-21T01:55:00.000-07:00

Ok, so I went to Stack Overflow DevDays in San Francisco this last Monday, and it was a lot of fun. I went with a friend from work, and we both walked away happy, stickers in hand.

I took some notes, and I would like to share what I thought of all the speakers and their presentations, but we'll save that for another day... if I actually end up writing it (post a comment if you are interested, and hopefully that will push me to write it up).

What I want to talk about is a small bit I learned from one of the presenters. If you don't know me, I'm a big Linux fan, I heart Google, I have an Android G1, I love open source and Ruby and Rails, and Microsoft is practically the devil. Given that, you might find it interesting that the thing that struck me most... enough to actually write about it... came from Microsoft. That's right, it came from the near-devil. Well, it came from Scott Hanselman, who was a pretty good speaker.

Scott talked about Microsoft .NET MVC. It seems like a relatively cool product, if you can bear to be in the Microsoft world, and if you can bear to deal with such a clunky language as C#. Ok, I gotta give props that C# is a decent language... for a statically typed language... but the platform restrictions (excluding Mono), and it not being nearly as awesome as Ruby, will keep me from ever dealing with it again (I worked in C# professionally for a few years). Yes, .NET provides access to a fleet of languages, but C# is still their flagship. Anyway, from the discussion, MVC seems like it does a decent job at mimicking Rails, though it still didn't seem nearly as elegant.

That was a long lead-up for a simple point I extracted from the Big M... <%: value %>. That's it. I have wished the default of scriptlets was to sanitize html output for a long time, and I'm pretty sure Jeff Atwood had a post on it at one point. That tag, if I understood Scott right, differs from <%= value %> in that it will sanitize the output automatically.

On the train ride back to the south bay, I set it upon myself to implement this auto sanitization in Rails for my current side project. Initially I thought I could do something like:

def :(arg)
 h arg
end

But alas, that doesn't work. Ruby won't let you define ":" as a method... it throws a syntax error... drat!

So, I figured, maybe I could monkeypatch the ERB handling so that <%: value %> works as I want, so I looked around the Rails source for how ERB compiles the views. It didn't take long to find erb.rb, which deals with the compilation. I inspected it for a bit, and toyed with some ways to deal with the problem, and ultimately came up with:

class ERB::Compiler::Scanner
  alias_method :initialize_without_sanitize, :initialize

  def initialize_with_sanitize(src, trim_mode, percent)
    initialize_without_sanitize src, trim_mode, percent
    @src.gsub! /<%:(.*?)%>/, "<%= h(\\1) %>"
  end

  alias_method :initialize, :initialize_with_sanitize
end

If the above code is cryptic for you, it basically overrides the initialize method (which is the constructor method for all you non-Rubyists) and uses a regex to replace all of the new sanitizing scriptlets with what would actually work to sanitize it. So, <%: value %> gets transformed to <%= h( value ) %>. I can't vouch for the performance (because I haven't done any performance testing), but it works.

Put the above snippet wherever you put your monkeypatches that you want run once. I have mine in a monkeypatches.rb file set to load after Rails has initialized (so it runs just once, at startup time).

Go forth in sanitized goodness.

Why Newspapers Can Go Away

2009-09-10T18:40:00.000-07:00

Joel Spolsky and Jeff Atwood's most recent podcast struck a nerve with me. They go on for a period of time about Craig's List and the effect it has had on newspapers. Joel in particular tries to argue that Craig's List has essentially killed off the public good of investigative journalism, and by refusing to cash out on Craig's List in some way, has not returned the public with anything other than the marginal usefulness of free classified ads.

Normally I think very highly of Joel and his opinions, but in this instance, I think he has completely missed the boat. Jeff tries to poke holes in his argument, but he didn't articulate the argument that I most strongly feel....

Investigative journalism is now a distributed affair, and no longer needs the legwork of the newspapers that Joel alludes to. On a side note, I want to point out that the linking to eachother that Joel accuses bloggers of is quite prevalent in newspapers as well... aren't popular Associated Press articles picked up and run nationally all the time?

I just want to point out one recent case study to illustrate my point... the shooting of Oscar Grant on New Years Day in an Oakland BART station.

I admit, I haven't followed up on how the case against the officer panned out (or if it is even through yet), but the point remains that something like excessive police force cannot go unnoticed in todays world. We don't need newspapers to do the legwork anymore because the ubiquity of the web, combined with a computer in almost everyone's pocket in the form of a smart phone capable of snapping a photo, shooting a video, or recording a conversation and then posting it online immediately has put the power of investigative journalism into the hands of everyone, everywhere.

If something corrupt happens within earshot (or eyesight) of someone, there is a much greater chance nowadays that someone is listening and could easily publish the wrongdoings within minutes. This is not to say that we can do without real journalists going out and finding corruption and shining the light on it, but I firmly believe the Internet and technology is our salvation, not the newspapers.

Jeff, I think you are right that we are in a transition, and the spotlight on the bugs under the rock is being lit by everyone now.

Dealing with Errors in Rails

2009-04-17T21:15:00.000-07:00

Jeff Atwood's recent blog post on "Exception-Driven Development" has driven me to write a post that I've been thinking about writing for the last couple weeks.

I'm working on a Rails application in my spare time, and one of the concepts I wanted to drill into the codebase from the beginning was easy management of the system from within the system. This means I can log in and, given that I have the proper access, view errors that have happened in the site, migrate the database, restart the application, and perform various other actions that would otherwise require tedious activities like digging in log files, or obtaining shell access and executing various commands. While I am quite comfortable on the shell (I run Linux on all the machines that I use with any regularity), I would much rather have an easy to use GUI that gives me access in the context of when I need it (ie, when I am actually checking up on the website... from within it).

So, this post will basically show you how to add error logging within your Rails application, as I've done in mine.

First up: the database. Here's the database migration script you will need:

class CreateErrors < ActiveRecord::Migration
 def self.up
   create_table :errors do |t|
     t.string :level
     t.string :title
     t.boolean :handled
     t.text :description

     t.timestamps
   end
 end

 def self.down
   drop_table :errors
 end
end

The level column is so you can keep track of errors vs warnings, or have informational messages as well. It's not strictly necessary, but I want the flexibility, plus I'm used to the concept from all the logging frameworks that support multiple levels of messages.

The title and description should be self explanatory. In case you are slow though, the title is just a header so you can at a glance see what errors have cropped up, and the description provides a detailed look at the error (such as a backtrace).

The handled column is critical so you can keep track of which errors you have dealt with, and which need looking at. It acts as a soft delete.

Since you've seen the database end, you might as well peak at the model next:

class Error < ActiveRecord::Base
  def self.error(title, description)
    log :error, title, description
  end

  def self.warn(title, description)
    log :warn, title, description
  end

  def self.info(title, description)
    log :info, title, description
  end

  def self.log(level, title, description)
    Error.new(:level => level.to_s, :title => title,
        :description => description, :handled => false).save
  end
end

The error, warn and info methods are to easily log something at the given level, much like a typical logging library. All of those methods use the one logging method which will actually save the error as a row in the database. Pretty simple, as is the usual case with ActiveRecord classes! You could make a case for making the log method private, but I don't really want to, so I won't.

Next up, the errors controller:

class ErrorsController < ApplicationController
  title "Errors"
  require_admin

  def index
    @total = Error.count :all, :conditions => ["handled = ?", false]
    @errors = Error.find :all,
        :conditions => ["handled = ?", false],
        :order => "created_at DESC", :limit => 50
    @start = [1, @total].min
    @amount = @errors.size
  end

  def show
    @error = Error.find params[:id]
  end

  def handle
    @error = Error.find params[:id]
    @error.update_attributes :handled => true
    redirect_to errors_url
  end
end

The title and require_admin methods are some utility methods I baked into the base controller... I may publish those at some point, but they aren't the focus of this post, so not now. If I get any comments requesting the code, I will probably follow up with the code, so let me know if it is intriguing you! Their purpose are to set the title on all views associated with this controller, and check for administrator privileges respectively.

If you will notice, the index will only load the most recent 50 errors... I figured it would be possible for the application to get in a weird loop where you trigger tons and tons of errors. I wouldn't want to haplessly check out the current list of errors, only to watch in horror as I am downloading a list of 100,000 errors. The handle method is the only means to update an error... and it simple turns on the soft delete so the error won't show anymore. It might make sense to set up a mass handled action which would handle a group of selected errors all at once... but I don't really need that yet... I will build it when the need arises.

The rest in index should make sense as you check out the index.html.erb view:

<h1>Listing errors</h1>

<p>
  <%= @start %> - <%= @amount %> of <%= @total %> errors.
</p>

<table>
  <tr>
    <th>Date</th>
    <th>Level</th>
    <th>Title</th>
  </tr>

<% @errors.each do |error| %>
  <tr>
    <td><%= h format_datetime(error.created_at) %></td>
    <td><%= h error.level %></td>
    <td><%= h truncate(error.title, :length => 100) %></td>
    <td><%= h error.handled %></td>
    <td><%= link_to "Show", error %></td>
    <td><a href="/errors/handle/<%= error.id %>">Handled</a></td>
  </tr>
<% end %>
</table>

Pretty straightforward, huh? Well, here's show.html.erb

<p>
  <b>Title:</b>
  <%= h @error.title %>
</p>

<p>
  <b>Time:</b>
  <%= h format_datetime(@error.created_at) %>
</p>

<p>
  <b>Level:</b>
  <%= h @error.level %>
</p>

<p>
  <b>Description:</b>

  <br/>

  <%= format_multilines h(@error.description) %>
</p>

<a href="/errors">Back</a>

Also pretty simple. If you are curious about format_datetime and format_multilines, they are simple helpers I created to respectively format a date as I want them formatted, and take a string with newlines and convert the newlines to <br/> elements. Pretty simple helpers, but I like keeping as much logic off my views as possible.

That's it! Or wait... that's all the stuff that rails practically generates for you (tweaked for simplicity, but still). How can this stuff get put into practice? Well, I'll give you that code too... since you've gotten this far. In your base application controller:

class ApplicationController < ActionController::Base
  RENDER_LIMIT = 0.5
  around_filter :time_page

...

  private
  def time_page
    before = Time.new
    yield
    after = Time.new
    delta = after - before

    if delta >= RENDER_LIMIT
      title = "Page took #{delta} seconds to render"
      description = "Execution of #{request.path} took #{delta} seconds to render."
      Error.warn title, description
    end
  end

  def rescue_action(e)
    title = "Caught exception '#{e}'"
    description = "Exception was caught: '#{e}'\n\nBacktrace:\n#{e.backtrace.join("\n")}"
    Error.error title, description
    super
  end
end

With this addition, your new in-site error reporting mechanism will report any page that takes over half a second to load, or absolutely any exception that crops up. This includes failing to parse a Ruby file, actions or controllers that were browsed to that don't exist, or any random exception the user encounters. You may want to adjust the RENDER_LIMIT constant to the threshold of your liking. At some point I would like to add a separate timer for all database queries, but I haven't had the time or interest to do it yet... and besides, the render limit should encompass all queries anyways, so it would be slightly redundant (though I would like to have a much lower threshold of query time than render time).

And as a caveat, this will not report a failure to parse the application controller Ruby file... because... well... think about it and you should realize why it can't :-)

Go forth and practice Exception-Driven Development in Rails.

jQuery UI Theme... A Tribute

2009-04-14T01:54:00.000-07:00

Ok, I haven't blogged in a while, and I have been intending to start... with maybe little bite sized entries. I have the perfect one now, so here it is... my very own (and first custom) jQuery UI Theme.

In case you can't figure it out... I call it "Hot Dog Stand."

I am actually including it in a web app I am building.

Why Reddit is Better than Stack Overflow (WAAAAY Better)

2008-10-13T21:17:00.000-07:00

Ok, so to start, I am a recovering former addict of crackoverfl.... I mean stackoverflow.com. Honest! Look at my rank on the site (I go by "Mike Stone"). As I write this, I am 6th on the list of users sorted by reputation... and just a week or so ago I was holding steady at 3rd for quite a while. As you read this, I may not even be on that first page anymore.

There were times when I would be constantly looking for questions I could answer well... and other times I attempted to answer anything at all (that didn't go so well usually).

As a reformed addict, I think I can safely say a few things about the site. Whether they hold sway for you, or whether you agree or not... frankly I don't care much... because if you disagree, you are one of THEM....

So here it goes... Reddit has 1 thing going for it that I don't think Jeff Atwood will ever have... trust in the cloud.

That's it.

That's all there is to it.

The thing is, this is a touchy subject. In case you don't know what I'm talking about, I am alluding to the "close question" feature that Jeff so dogmatically believes is helping steer the users into asking the "right" questions. Instead, it is giving some users a power trip, because after all, they know what is good for the site. So they go ahead and close questions that clearly don't belong. Except some people find value in those questions, and it has a negative effect on the asker if they are powerless to disagree and change the decision.

If you are in the "we need to steer users towards our opinions" camp... I'll tell you right now that I hate you. Don't take it personally... what I really mean is I hate your stance, but it's more fun to just say I hate you. I don't like the power trip you feel, and I don't like that you are forcing your opinion down on my throat what should and should not be open for discussion. Yeah, the D word... I know Jeff doesn't like discussion on his precious baby of a site, but sometimes I want to go to the site for discussion. It's fun to answer answerable questions, and it's also fun to discuss interesting arguable questions with intelligent people (such as my most popular question). I want to do both, and I want to see both types of questions thrive.

But... Closing Questions is Working!

This is the argument I've seen from Jeff Atwood. He says closing questions is working, and he links to questions that have been closed as so-called proof. We have exchanged some emails, and when he sends me the links, I look at them. Every single time I've looked at his examples of questions that deserved to be closed, it is the same damn thing. The questions are either duplicates, have a negative vote total, have been deleted (likely because of offensive count), or they are gray area questions that more often than not I would rather see them on the site.

Duplicates Should be Closed, Right?

This is the single best reason for having the close question feature, except I think it is a problem best solved with a different, unimplemented feature. At least half the questions I see closed are marked as duplicates. Wouldn't it be neat if instead of closing duplicates, all the duplicates were congealed into 1 massive blob of a question? You go to one, and you see the blob... and all the questions are available, able to morph between the different forms via tabs (or some other mechanism). This way, the answers to all these similar questions can be viewed at once without having to scan for links to follow (and if you can't provide a link to the duplicate in question, don't close it as a duplicate... that angers me more than the close feature in general). The oldest one could even always be the first tab, so there is 1 canonical place to get the best answers, and all the others turn into alternative forms with possible nuggets of gold awaiting you to click its tab.

I have a secret about duplicate questions though... Jeff actually likes keeping duplicates around! If you don't believe me, here's a quote from an email he sent me:

There's no value in most closed questions, with the exception of edited duplicates which allow better search matching. People have the uncanny ability to ask the exact same question using totally different words, so we need these as breadcrumb trails.

So if duplicates are an important part of the ecosystem, and it's the only good use of closed questions (which I hope to show before you stop reading), then closing questions shouldn't be necessary at all.....

This One is Negatively Voted... Isn't That Proof Closed Questions ARE Working?

Not exactly. If it's negatively voted, this seems perfect proof that the voting system was a good system to implement. If a question is negatively voted, then people should be able to watch "hot" tabs and just not even see the negatively voted questions. There will be plenty of people watching all new questions who will vote it up if it was voted down inappropriately, and those same people will continue to vote it down if it really should stay down. Thus... why does it need to be closed?

If someone finds value in a negatively voted question, what is the harm in keeping it around? If the asker gets an answer he/she is happy with, who loses? If it is tagged appropriately, it probably won't even show up in the tag filters people set up (or the site automatically figures out) so that people who would rather not see it, won't see it.

So those questions don't REALLY need to be closed... voting dealt with them.

I Saw a REALLY Offensive Question Closed... That's Good Right?

These questions get marked offensive, and they get automatically deleted after 5 offensive marks. That is the purpose of the offensive feature, and it deals with those questions well... closing them is like closing a negatively voted question. Saying it is bad twice doesn't make it extra bad... just 1 way to say it is bad is all that is really needed (well, 1 way to say it's bad, and one to say it's REALLY bad... hense downvotes and offensive). I just laugh when I see a question that is negatively voted, marked offensive a couple times AND closed. I wonder if those people closing it think they actually did something valuable with their time....

What About The Rest?

So the rest are basically questions that are running dead even at 0 votes, or have positive votes. If this is the case... doesn't that mean someone is finding value in the question? This group is pretty varied. Some of them are polls on the best X, where X can range from programming comics to the best reference guide. Some people don't like polls, but I personally find them entertaining to participate in, and even sometimes find truly useful information.

You know who else likes polls on Stack Overflow? Joel Spolsky. That's right, the cofounder of the site talks about polls in his launch post, while Jeff grits his teeth when Joel mentions them on the podcast.

A lot of the other questions are discussions, which I've already mentioned I find value in. They help build the community, and provide outlets for your ideas or disagreements with your felow programmers.

Even more of the gray zones are borderline programming questions... usually they are somehow of interest to programmers, or IT people in general, so it seems to me they have a home there (or should, anyways). These range from Virtual Machine questions, to general troubleshooting problems with an OS or application that programmers will typically use. If they are tagged right, why can't they be on the site? If you are looking in your comfort zone of tags, you will never see the VirtualBox, or Ubuntu questions, so why can't someone get real use out of the site on such marginally programming related subjects?

So What Makes Reddit Better? I Heard it Sucks!

It seems a lot of people have lost faith in Reddit, and feel it has nothing left to offer. I find it quite the contrary... it is so addictive that I have to avoid it if I want to get any work done. The key to Reddit's success, I think, is the clever combination of subgroups and voting. The voting lets the interesting topics rise to the top. The subgroups keep those topics interesting for those looking at the group. That's why I always only go to programming.reddit.com, because every time I go there, I see really interesting articles about programming. It NEVER lets me down.

Stack Overflow could do the exact same thing with their implementation of tags and voting. Filter the interesting stuff to the top with voting, and keep it interesting to me with my focused set of tags. Think of each tag as a focused niche group where the stuff that rises with voting is going to be interesting to the members of that niche group. The only stuff that group of people would want to close are all stuck in other tags that don't even appear for them.

The next time you would close a question... reconsider leaving it open for someone in that niche group that would enjoy participating in the question.

So, that's the key, in my view. Reddit trusts the cloud and refines it with subgroups and it works. Jeff Atwood doesn't trust the cloud, and he turns people like me away who favor more open systems.

What of Proof?

Well, I'm not going to compile a long list of questions to prove my point... I've seen the proof with my eyes, and if I haven't convinced you that closing questions are unnecessary, I never will. I will, however, give you a short list of some questions I reopened because I felt strongly that they shouldn't have been closed. If you agree that any of these shouldn't have been closed, then maybe there are more that shouldn't have been closed.

The truly sad thing is that some questions that shouldn't be closed lose their chance at getting a good answer because someone was denied the ability to provide a good answer because of 1 selfish person that felt the need to push his/her opinion on everyone else.

The first was interesting to me because I actually want a low energy low cost linux box. Jeff Atwood has himself blogged about this on his programming blog, so it seems appropriate.

The second and third are VM questions... I use VMs daily for my programming work, so their effective use is of much interest to me (though I have not run into these particular issues).

The final one was something I wanted to thrive because the answers make for entertaining reads. Why can't we live with some humor now and then?

Final Thoughts

So, before I relinquish my control over you (as I apparently have some control, if you've gotten this far), I want to say a couple last minute things. I weened myself off Stack Overflow because I was sick of seeing questions closed that shouldn't be. I think it is a travesty to see people force their opinion of what to discuss on others. Though the number of closed questions may be small, it is unjust enough in my opinion that Stack Overflow is not worth my traffic, or my answers. Furthermore, it makes me depressed to see Jeff have so little faith in the system and community he built. Despite drastically reducing my usage, I reserve the right to ask questions when I have one I need answered... after all, I wouldn't be this impassioned if it weren't a good site!

If you don't like closed questions either, email Jeff Atwood (you can find his email on his blog if you look for it). Alternatively, open a uservoice ticket. The more people opening tickets the better... he might submit to pressure if he has to get rid of the tickets over and over. Just stick to 1 ticket though... unique users posting requests is a lot better than abuse of the uservoice system.

UPDATE:
cky brings up a great point in the comments that removing the close question feature will also do harm in causing other users to leave... a compromise between the 2 camps is probably the best solution. Some system of check/balance for the close question feature would be good, such as a simple majority vote. Other fair alternatives would be welcome in my book as well.

Till Death do we Rewrite

2008-10-12T23:09:00.000-07:00

It's been a while since it was written, but I recently read an interesting article from Joel Spolsky. It's from 2000, but it is of course still very relevant.

The article was about rewriting code, something he argues you should never do. It might be more fun and appealing to play devil's advocate, but I can't for this one. Joel is dead on here.

In my last job I was dealing with GUI code primarily. There were a few of us working on this section of the code, so it was by no means all my own work. One particular component of it was written by someone else who ended up moving on to another part of the project, so me and a few others inherited the work.

There came a time to expand on this component, and so I looked over what was done. It seemed somewhat awkwardly implemented, and I felt I could do a lot better if I started from scratch. There were events being raised and passed on 3 levels deep in the object hierarchy for no apparent reason, and the object structure itself seemed a bit awkward. It also got a bit slow under certain circumstances. Thus, I plowed ahead with a rewrite.

I started out strong. I had objects that correlated logically with what was being displayed on the screen. I had a simple structure that seemed a lot clearer and cleaner to me. It was working, and it seemed to even be faster. I was proud of what I had created (at least, at the time I was).

Then time went on and the GUI got more complicated. It sprouted the hairs that Joel talks of, and started getting slow for no apparent reason (but in different ways from the original implementation). All in all, the end result started looking a lot more like what had been there than I liked to admit.

The rewrite wasn't a total failure... it worked, and some of the details had definitely been improved, but at the cost of some new hairs that I would rather go back and fix. However, in retrospect, it would have been a much better idea to just dig in with the existing code and salvage what I could, refactoring what really needed improvement. I walked away with a valuable lesson though... whenever you want to rewrite code... don't. What is already there can work if you give it a chance. Refactor what doesn't work, and make the unclear sections more clear.

Learning to resist the urge to totally rewrite code is an important lesson I needed in order to become a better programmer.

Please Hold on the Interfaces

2008-10-09T23:57:00.001-07:00

I just got my first Notable Question badge over on stackoverflow.com for what seemed to turn into a rather controversial question. At the time, I was just ranting based on a question I had just read where all the answers suggested the use of an interface, when it didn't really seem to need one. Even more egregious, the interface was basically being proposed just to support testing.

To spare you a little reading, the basic premise of my question was: interfaces are overused, and it sucks when they are being overused, so what is it I'm missing, if anything?

This got some people to go NUTS over it! I mean, literally these people went wildly crazy, spouting nonsense... listen to this one, which was probably one of the worst responses I saw:

An issue with favouring mocking (or any kind of runtime substition) through subclassing, rather than through an interface, is that you can only override the behaviour of virtual members.

Invoking a virtual member requires an additional level of indirection via a Vtable and is slower than a final member. Whereas using an interface adds a compile-time overhead, but using virtual members adds a runtime overhead to every member invocation.

Surely, adding an interface **just to support mocking** is far preferable to slowing down your entire app **just to suport mocking**?

My preference, therefore, is to favour 'final by default'.

Talk about premature optimization!!!! This guy wants to avoid polymorphism because of the runtime costs?!?!? I can't believe how STUPID people can get sometimes! I mean, seriously! I try not to get angry when I'm on the Internet, because flame wars and trolling gets us nowhere, but c'mon now. A v-table call is going to be... what... nanoseconds? Even faster? So small we can't even reliably measure it??? And he thinks he has a net gain by avoiding that cost? I sure wouldn't want to be the one maintaining HIS code!

OK, I'm getting off topic. Interfaces. Those things that pretend to be a class, but have no details whatsoever. I don't mind them. Sometimes they can be quite helpful. However, like all tools, we must stop and analyze why we are going to use the tool, and what will we gain by doing it. If you pound on a screw with a hammer, you might just get it in there, but did you go about it the best way?

There are times when interfaces make a lot more sense in places they don't seem to belong, such as perhaps frameworks... though this guy rightfully pointed out that they also lock you in and restrict change, especially in frameworks (where you can't control who has implemented it).

So here's my biggest beef with interfaces: using one when you have a single implementation class.

That's it. That's the single most spine-curling, eye-gouging use of interfaces that will make me want to vomit all over you if you suggest it.

If you are a lover of interfaces, this may come as shocking news (as I sadly found out). Why is it so bad? You get to increase testability, right? You make your class more extensible, right? You are programming to an interface, right?

WRONG! No, no, no, no.... NO NO NO!

Testability

First of all, testability has nothing to do with whether you have an interface or not. It has to do with whether or not the dependencies you have in your class can be mocked out and altered so that you can ensure that the class works properly regardless of what those dependencies spew at you.

For example, let's say you have an object that represents a customer, which is represented in the database. Perhaps one of the worst things you can do is load the data for that object from the database from within the constructor. At this point, you can't get around constructing your object without depending on the database. Now, if you want to write a test for a DIFFERENT class that uses this customer object, you all of a sudden have to have the database up and running. This is bad because it makes the test a lot more complicated than it needs to be.

Now, if you are in the Interface Camp, you might be thinking... hah! If you define an interface for that class, you can get around that dependency! This is true, but it's also the easy way out. Singletons are also the easy way out to a lot of problems, but you don't hear many people clamoring for singletons everywhere, do you? The easy way out is oftentimes NOT the clean or best way out. Instead of shrouding your class in an evil interface, just move the database initialization to a protected method that the constructor calls. Now you can override the protected method in subclasses or mock classes and no longer depend on the database!

Extensibility...?

I just don't understand why people think classes are less extensible than interfaces. It just isn't true, damnit!! I've been extending classes left and right when I see the need, and I have never had a problem. The trick is, again, to avoid dependencies.

If you are doing something that can't be changed in a subclass, you might be doing something wrong. The constructor is the first place to look... is there behavior in there that does something that might be unwanted in an alternate version of this class? Is there a static initializer that is doing something fundamental to the class that can't be changed in a subclass? As long as you have only virtual methods (believe me, the "performance hit" is worth it... and it's why I love Java's default virtual and hate C#'s default final), and your constructors and static initializers are clean, you can reimplement the entire class as if it had been an interface all along! It's really not that hard, and if you are active with Test Driven Development, you will find the classes often come out that way anyways.

Programming to an INTERFACE....?!?!

Someone else said it better. I don't remember who, and I don't remember where. What do I look like, google? Find it for your own damn self! Well... the gist is, "programming to an interface" does not mean literally to a programming language construct called an interface... it means programming to a contract between you and users of your class... i.e. the publicly visible methods, fields and constructors agreed upon between you and clients of your class.

Just because "interface" is a keyword in some programming languages, doesn't mean we can't speak about "interface" as an English word used to describe what we are exposing.

Appropriate Uses??

I'm not going to lay out a bunch of rules on when you should use an interface. Deciding what tool to solve a particular problem is often somewhat of an art more than an algorithm, so you really have to look at the problem and just decide for yourself. You may end up using interfaces in an inappropriate time... but hey, we all make mistakes! Refactor it when you find a better way... you got all those tests to deal with it, right?

Without going into too much detail, I think a good rule of thumb for when to use an interface effectively is to use it when the thing you are trying to build doesn't lend itself to a default implementation, or you literally HAVE to do it... such as the case when you absolutely need to extend 2 classes (pick one, and extract an interface). However, there can be ways around needing the interface in these cases, but if the interface is cleaner, go for it.

You WILL Regret it Later!

No, you won't, but some claim you will, like this guy:

Sure, sometimes you may not ever have an alternate implementation of ISomeFoo in which case it might have been a waste for that particular component. But if you ever DO need to have an alternate implementation, going and changing those 50 references to concrete PetrolSomeFoo to concrete HydrogenSomeFoo will hurt really bad, especially if someone of the changes involve other applications or integration scenarios.

So, my argument is to let the IDE do the work for you. Extracting an interface and replacing all references is not all that hard of a task, especially given the help of a good IDE. If you are developing a framework where you don't control client code... this is one of the few cases where more interfaces might make sense, unless you have the balls to break backwards compatibility. If you have the ability to only use a tool when you need it, I say save on using that tool unless it really solves your immediate problem cleaner than other solutions.

Finish Already!

So the next time you reach for the interface keyword, stop and consider to yourself... is this really the best solution? Am I really solving this problem cleanly and elegantly with interfaces, or am I just hacking around a problem without considering how else I might solve it?

Why Java Generics Suck

2008-07-28T21:12:00.000-07:00

When I first saw Java's Generics, I thought it was a pretty neat addition (rather harsh syntax, but effective). You get stronger type safety without all the extra casts. Less code with less room for mistakes, right? Well, yes and no. The problem is, Sun wanted to add this new feature, but at the same time preserve bytecode compatibility. There are probably a lot of reasons that maintaining bytecode compatibility is a good thing, but I just don't buy it.

So, what am I talking about you ask? Good question. The way Generics are implemented is via a thing called Type Erasure. What this essentially means is that the type information is there only for the compiler. So, if you have a List<String>, you don't ACTUALLY have a list of strings. You have a List that will only let you store and retrieve strings, but ONLY as long as you know the type of that list. I will get to an example in a little bit. My best guess at why Sun did this is they wanted to add this neat feature that C# had and C++ had, but at the same time they didn't want to cause upgrade problems where clients of your code MUST be using the Java 1.5 runtime. Except, if you use any of the new API, your clients are going to have to upgrade to Java 1.5 anyways, so it seems like a myth that they are ACTUALLY benefiting anyone by being backwards compatible. By trying to appease 2 crowds (those wanting more features and those wanting to be able to run on old runtimes), they ended up creating a bit of a Frankenstein's Monster in the process.

So if that garbled mess of a paragraph didn't help you understand the problem (which it probably didn't... I was really just ranting), then hopefully a concrete example will help.

Have you ever seen the warning "Type safety: The cast from Object to List is actually checking against the erased type List" or something like it? You can get it from the following code:


public static void storeInt(Object object, int value) {
   List<Integer> list = (List<Integer>) object;
   list.add(value);
}

What this means is that you tried to take an object without Generic type information (Object for example), and you tried to extract the Generic type information from it (Integer)... except that information was erased the moment you stored your list into a reference of type Object. Why? Because in order to preserve backwards compatibility, the generic type of Integer is not ACTUALLY available at runtime. This means you can end up recasting your list to the wrong generic type, and add elements of the wrong type, which can later cause a ClassCastException. Consider the following code:


public static void storeString(Object object, String value) {
   List<String> list = (List<String>) object;
   list.add(value);
}

public static void storeInt(Object object, int value) {
   List<Integer> list = (List<Integer>) object;
   list.add(value);
}

public static void printList(Object object) {
   List<?> list = (List<?>) object;
      for (Object value : list) {
      System.out.println(value);
   }
}

public static void main(String[] args) {
   List<String> list = new ArrayList<String>();
   storeString(list, "Hello World");
   storeInt(list, 1);
   printList(list);
}

The above code will not only compile, it will also print 2 lines, the first saying "Hello World" the second saying "1"... even though we have clearly violated our initial list of strings and stored an integer in there. However, let's add some more code. After the printList call in the main method, try:


   for (String value : list) {
      System.out.println(value);
   }

Now you will finally see the runtime error you might have been expecting in the previous example. This time you will see "Hello World" followed by "1" followed by "Hello World" followed by a ClassCastException. This is because Java is trying to take the "1" and put it in the String value reference... but it's not a String, so we have a casting problem! If sun had implemented Generics right (which probably would have meant byte code incompatibility with 1.4), then the first time you tried to cast that list of strings as a list of integers, you would have gotten a ClassCastException. Furthermore you would have been able to do things like this:


public static <T> T createAndStore(List<T> list) {
   T value = new T();
   list.add(value);
   return value;
}

and this:


public static <T> T[] toArray(List<T> list) {
   T[] array = new T[list.size()];
   for (int i = 0; i < list.size(); i++) {
      array[i] = list.get(i);
   }
   return array;
}

But with type erasure those 2 examples are impossible, because the type is unknown at runtime.

You may think that I am just ranting about these and that the issues never come up, but I assure you they do come up. If you are working with any legacy code or APIs that were not built when Generics were around and did not upgrade to use Generics (or were built when they were around, but the author(s) didn't think to use them)... or if Generics just can't solve the problem that a simple list of Objects can, then there can be issues.

Case in point: servlets. Take a look at the servlet API, particularly HttpSession's getAttribute method. I can only store and retrieve Objects, which makes sense. I should be able to store anything I want in the Session. However, the moment I store an object with Generic type in there... especially one that needs to be modified, I have thrown type safety out the window.

Let's say I store a list of strings in the session, and I want to add a string to that list periodically throughout my user's session. This doesn't seem so far fetched, right? Well, now how do you suppose to add items to that list once you have removed the String generic type information?


public static void addString(Object object, String value) {
   List<String> list = (List<String>) object;
   list.add(value);
}

causes a warning at the casting time like we discussed before.


public static void addString(Object object, String value) {
   List list = (List) object;
   list.add(value);
}

also causes a warning, this time at the point where we call add... because Java thinks we should be using Generics... after all, they spent so much time designing the system, you should always use it!


public static void addString(Object object, String value) {
   List<?> list = (List<?>) object;
   list.add(value);
}

causes an actual compiler error on the add call... because Java can't give you any guarantees that you are safely calling add with the correct type, so they just don't let you.

The solution? Well, you have to just use the SuppressWarnings annotation on the method where you want to add that string! Seems like a hack to me, but maybe you can live with that. Personally, I would rather just ditch Java and go use Ruby. None of this is even remotely an issue when you go dynamic.

C# Regions and Style

2008-07-07T19:36:00.000-07:00

Often, Jeff Atwood has insightful and interesting things to say. I very often agree with his viewpoint, though every now and then it just doesn't quite sit well. That's probably important, to be sure that I am reading with a critical eye and not just taking his word on blind faith. The reason I start with this is his most recent post just didn't sit well with me.

I worked with C# at my last job. It had its perks and its problems, and it was a Microsoft product, which adds instant negative baggage (sorry Jeff, despite enjoying your writing and podcast material, I don't think I will ever understand your appreciation of Microsoft products). One thing I did like was #regions. C# is about as verbose as Java (at least it was before 3.0, which I never got to use but have heard they improved things a bit). As such, this often leads to large code files, especially in GUI code that has tons of event handlers and special setup code. I found at my last job that a small sprinkling of regions helped organize my code quite well. I could keep all methods of a particular type grouped together with the ability to toggle the code at will. Some of my favorite groupings were "Event handling", "Constructors" and "Utility methods". It just seemed cleaner to keep things together that are semantically similar. Though I prefer Java and (even more so) Ruby, Regions is one thing I wished I could take from the Microsoft world into the rest of the world.

He is completely right that it can be abused, though. I saw plenty of other code in the project that would simply have regions around every method definition, which just seemed redundant to me. I know Eclipse will allow this kind of code folding automatically, and I could swear Visual Studio did it also (but it has been long enough that I could just be thinking of Eclipse). At that point, the regions aren't grouping semantically similar concepts, and it ends up being quite a mess visually and organizationally. At least to me... I'm sure the author preferred it that way.

This leads me to conclude that the first part of his article is dead on. Despite my disagreement with the details of regions, I fully agree that a team must be aligned on code style. There is one class I missed in college that in retrospect I truly regret not taking. It was a class that everyone feared as being too hard, but everybody who took it ended up loving the professor. It was some kind of individual software project course, and the class was known for the professor enforcing a standard coding style on the students, with some rules purposefully contrary to common style guides. Thus, the student walks away learning the ability of bending one's will for the good of the team. Style is often a "religious" debate that has no answer. Both sides are valid, because it's just a matter of personal style. The style I consider beautiful is ugly to the next person, and likewise his or her style is ghastly to me.

The style I appreciate most of all though, is consistent style. If that means bending my style a little sometimes, then so be it.

Singleton Methods

2008-06-30T21:14:00.000-07:00

Ruby has a neat feature called Singleton Methods. The idea behind them is that you can define a method on an instance of an object, and that method only exists for that instance. Sounds pretty crazy the first time you learn about it, but it can prove quite powerful. This feature only makes sense in conjunction with duck typing (which I would like to dedicate another post about at some point). Don't expect a JSR for Singleton Methods for Java any time soon! Actually, you can do something similar using an anonymous class, but it would be rather hard to invoke, and you would have to define the method at the point of creation.

How could Singleton Methods possibly be useful? Well, my personal favorite is for unit tests. I prefer testing with actual classes, but sometimes that is not feasible. The object in question may not be built yet. It may be a legacy object that you don't want to touch because there aren't all that many tests for it, and it would just be easier to pretend that you have an instance of it. Perhaps the most likely scenario is that you are using a framework, and it is just plain annoying or infeasible to construct the real version of the object in question. Sockets are a prime candidate for this exact situation. Whatever the reason, you want to mock an object. What do you do in Java? You either construct an anonymous class, or you define a mock class which extends the object in question (or implements the required interface), and voila!

Except... anonymous classes are rather cumbersome, and an actual mock class separates your mock logic with the test that requires it. You could of course have a general purpose mock that is used everywhere, but you will have to go back and forth between the mock definition and your test class definition whenever you are developing tests that use the mock.

In comes Singleton Methods to the rescue! Your method only connects, and then does some extra work once it is connected, so you just need to mock the connect method. The following code would accomplish this goal:


mock = Object.new
def mock.connect()
end
invoke_my_method_with mock
# assert some conditions

Then, you should have a test in case the connect fails, right? Your object needs to do some other logic when the connection doesn't actually connect! So, the next test mocks the object differently:


mock = Object.new
def mock.connect()
   raise "Some exception"
end
invoke_my_method_with mock
# assert some other conditions

Because of duck typing, your method doesn't know that it didn't get an actual Socket, but you were able to keep the mock logic near the test logic (in a clean and concise manner), which makes it a lot clearer what the test is testing.

Making mocking easier is nice, but ever since I learned Singleton Methods, I have been itching to use it in another way. Let's say we have an entity in our system that has a set of actions possible. My favorite case is a character in a game. There are many possible actions, but our character is only able to use a few of these actions. He can kick, but he can't punch. Each command could be a different Singleton Method! Now, whenever the character wants to punch, just invoke punch with the actual character instance. Of course, this means you have to be prepared that the character can't ACTUALLY punch, so you must catch a method missing error and inform the character that it tried to do an action it didn't know how to do (or perhaps didn't even exist). I don't know how this type of design would actually pan out, but it sure seems like the perfect fit for the feature.

I bet there are other good uses for Singleton Methods, so don't let these ideas limit the possibilities!

Thoughts on Groovy

2008-06-26T00:55:00.000-07:00

So I have been working with Groovy for the last few weeks on a for fun project at home. Nothing special, just a game... one that will probably never see the light of day, but I will nonetheless have fun with it for a while. I am a bit disappointed with Groovy, probably because I came at it having first learned Ruby. Groovy is very heavily influenced from Ruby, but not quite enough in my opinion.

Before I get into some issues I have with it, I guess I should note that it isn't ALL bad. Namely, they did a great job of making (mostly) seamless Java and Groovy integration possible... in BOTH directions. This is the 1 thing I wish JRuby would borrow from Groovy. What is the problem you ask? Well, with Groovy, you compile down to classes, and the resulting class is more or less exactly how you would expect it to work. That is, you can construct the object from Java directly, then call methods directly on it, exactly as you defined in your Groovy code. If you specified types, you will have access to those type declarations from Java. This is really good if you have an existing code base you want to integrate with. If you are starting an application from scratch with no dependencies, well, you might as well just stick with JRuby or Ruby (or whatever inferior language you like), since you can control whether integrating with your language from the Java end is a big deal. I played a TINY bit with Rhino, and they seem to have a similar type if integration possible, but Groovy seemed a lot cleaner... with Rhino you have to compile the class to a given object or interface... so the code itself doesn't really have a definition of the type in this case. Seemed like a bit of a hack to me, but if you are primarily using it from the JavaScript side, it is really a non-issue. Oh, and I decompiled the results of all 3 of these languages (Rhino, JRuby and Groovy), and Groovy seemed to add the lease baggage in the code, but it still had baggage.

Also... if you are a big fan of static typing, then Groovy will be a good transition to dynamic typing because you can fall back on static typing whenever you want. Personally, I see no reason to, but I am one who loves dynamic typing.

Now some bad things. Perhaps as a direct consequence of the apparent design goal to integrate seamlessly in both directions with Java, the language feels very... inconsistent. They TRY to do things in a nice dynamic way, but then fall short in some cases that I think Ruby shines in. This leading to my biggest gripe... the distinction between fields and methods. If you are going to support fields of varying scope, you essentially can't have the kind of clean design you have in Ruby. What do I mean by this? Well, let's look at an example.

In Ruby, if you have a property named "property" then you would define it as such:


attr_accessor :property

Which is equivalent to:


def property()
   @property
end

def property=(value)
   @property = value
end

So what is going on here? Well... we still have the distinction of fields and methods, but in Ruby, your fields are ALWAYS private. You can't make them public, so when you call a method, it will never be ambiguous with accessing a field. This means, invoking a method with no arguments, you can just drop the parenthesis because they are redundant. In fact, you can even drop them when there ARE arguments, as long as it doesn't make it ambiguous in conjunction with other method calls or whatever. I think this makes Ruby as clean as it is, and makes it really look like you are reading English... almost.

What about Groovy though? Well, they support a similar paradigm. Defining the same property is 1 line as well, but it looks like this:


def property

And it is essentially equivalent to creating getProperty() and setProperty(value). BUT! The consequence of seamless integration with Java means you can't drop the parenthesis in every method case. They too support dropping them when there are arguments, and it doesn't make it ambiguous, except when there are no arguments and it isn't a getter. Thus, in Ruby you could have "stream.close" but in Groovy it would have to be "stream.close()". This may seem minor, but every time I have to add those parens in Groovy, I'm reminded that Ruby's grammar feels so much more internally consistent.

I guess I wrote a lot about that, but for some reason that minor detail really irks me. Before I just end with only a single gripe, I better add a couple more for good measure. I haven't tried nested classes in Ruby yet (though I've read they are possible), but Groovy definitely doesn't support them, at least not now. I think this is something they will eventually get to, but as it stands, this is a major limitation. This seriously cuts short the "seamless integration" with Java that I feel is one of the significant strengths of Groovy. Not much more I can say about that, but maybe it's also because it is getting late and I am tired.

One more issue? Null. Groovy tried so hard to be like Ruby, but they missed yet another thing that I really like about Ruby. In Ruby, nil (Java and Groovy's null) is an object, just like any other object. What does this mean? Well, null pointer exceptions... no such thing. Instead they become a method missing error. This was a very cool concept to me the first time I saw it. I'm not sure if it allows significantly more concise code, but it does seem to pretty up the code, at least to me. A list of method calls as conditionals is a lot easier for me to parse than a mix of == and method conditionals. Visually, calling methods makes each condition a single entity, unlike with an == or != conditional. For example:


if connection != null && !connection.is_connected


if !connection.nil? && !connection.connected?

Which brings me to another point that I won't go into too much details... but being able to add question marks and exclamation points to my method names has got me wishing I could do it in every language. Beyond cutting a couple characters from the method length, they allow you to add meta info about the method that is visible at a glance... such as question mark for asking a boolean question, and exclamation for indicating state will be changed (which are essentially the Ruby standards for their usage).

Ok, I have ranted enough. I haven't proofread this, because it is late. If anyone happens to read this, just enjoy or hate it as it is.

Mike

A JRuby Swing Library

2008-06-23T23:48:00.000-07:00

Well, I haven't really posted anything yet, mostly because there is always something else to do. So, in an effort to actually start writing something, I have decided to post a little Ruby "library" I wrote a while ago. I put in the sarcastic quotes because it's quite a tiny library, if you can even call it that, but I think it has potential to be quite useful. I recently ported it to groovy for another personal project, and maybe I will post that next.

At any rate, before I list the code, I just wanted to say what its purpose is. Call me crazy (I probably am), but I like Java's Swing. Why? Well, it's portable in the sense that Java is portable... that is, you can easily install Java anwhere and thus have Swing at your fingertips... you can't really say that for most other UI libraries I have used (admittedly a small list). Either your window library of choice is X platform only (I'm looking at you Microsoft), or is just not nearly as pervasive as Java. Swing also does an OK job of looking decent, in my non-graphical-arts-critical eye (so take that with a grain of salt), and it can mostly look like your native platform with a special toggling (somewhere buried in the API that I end up always looking up, even though it is 1 or 2 lines of code).

So, with the good comes bad. I'm sure you can site many other bad things that I just don't have the experience or interest to point out, but the leading problem I am aware of, both in experience writing Swing GUIs and in reading other people mention the problem, is that it is an inherently tree structure being expressed in a non-tree syntax. The solution? XML! Well, there are many other options besides XML that provide a tree structure and don't make you want to gouge your eyes out, but XML happens to be pretty easy to parse in that most languages have a pre-packaged XML library. Thus, XML! Besides, XML is a lowest common denominator, everyone basically understands it, right? Isn't that what the XML enthusiasts try to shove down our throats?

Thus SwingXML was born, my tiny "library" that will take an XML definition, and load it into memory as a Swing component tree. Beyond representing the components in their inherent tree structure, it has the added benefit of separating your view definition from your view logic. In the groovy version I added a feature that I have not yet added to the Ruby version.

Without further ado:


require "java"
require "rexml/document"

class SwingXml
  def initialize(xml)
    document = REXML::Document.new(xml)
    @widget_hash = {}
    @widget = handle document.root, nil
  end

  attr_reader :widget

  def [](widget_symbol)
    @widget_hash[widget_symbol.to_sym]
  end

  private
  def handle(element, parent)
    raise ArgumentError, "Element #{element.name} does not have an sxmlId!" unless element.attributes.has_key? "sxmlId"
    raise ArgumentError, "Duplicate key found on #{element.name} (#{element.attributes['sxmlId']})!" if @widget_hash.has_key? element.attributes["sxmlId"].to_sym

    widget = eval "#{element.name}.new"
    @widget_hash[element.attributes["sxmlId"].to_sym] = widget

    element.attributes.each do |name, value|
       eval "widget.set_#{name} #{value}" unless name.index("sxml") == 0
    end

    element.elements.each do |child|
       handle child, widget
    end

    if element.attributes.has_key? "sxmlAction"
       eval "parent.#{element.attributes['sxmlAction']} widget" unless parent.nil?
    else
       parent.add widget unless parent.nil?
    end

    widget
  end
end

So, that's it! Pretty small for doing seemingly quite a bit, right? If you are wondering how to use the above "library", think of the SwingXml object as the root of your Swing component tree, while it is also a hash of the contained components. Typically, you would have a JFrame at the root, but you could even break them up more fine grained and have them be a panel or canvas of some sort. The components contained within the tree are accessible via their sxmlId values as indexes, which are an attribute you must specify on each xml element. You can even add more complicated things like layouts and whatnot with an sxmlAction which specifies how the object should be added to its parent.

As a simple example:


frame = SwingXml.new "<javax.swing.JFrame sxmlId=\"frame\" default_close_operation=\"javax.swing.JFrame::EXIT_ON_CLOSE\"><java.awt.FlowLayout sxmlId=\"frameLayout\" sxmlAction=\"set_layout\"/><javax.swing.JButton sxmlId=\"helloButton\" text=\"'Hello World!'\"/><javax.swing.JButton sxmlId=\"goodbyeButton\" text=\"'Goodby World!'\"/></javax.swing.JFrame>"

listener = java.awt.event.ActionListener.new
def listener.actionPerformed(e)
  puts "Hello world!"
end
frame[:helloButton].add_action_listener listener

listener = java.awt.event.ActionListener.new
def listener.actionPerformed(e)
  puts "Goodbye world!"
end
frame[:goodbyeButton].add_action_listener listener

frame[:frame].pack
frame[:frame].set_visible true

For easier reading, here is the XML but not embedded in the Ruby code:


<javax.swing.JFrame sxmlId=\"frame\" default_close_operation=\"javax.swing.JFrame::EXIT_ON_CLOSE\">
   <java.awt.FlowLayout sxmlId=\"frameLayout\" sxmlAction=\"set_layout\"/>
   <javax.swing.JButton sxmlId=\"helloButton\" text=\"'Hello World!'\"/>
   <javax.swing.JButton sxmlId=\"goodbyeButton\" text=\"'Goodby World!'\"/>
</javax.swing.JFrame>

In the groovy port, I added some code to remove the need for javax.swing.JXXX and java.awt.XXX, but I didn't add this enhancement to the Ruby code, but it should be pretty trivial to add. That small addition makes the XML a lot less of a pain to deal with. Well... less of a pain anyways.

I hope you enjoyed this, and I hope to make these posts more frequent!

Mike

Introduction

2008-03-07T00:36:00.000-08:00

Hello everyone! I have been thinking about creating a blog for a long time, and I finally have. I have a lot I want to talk about, but for today I want to just start with who I am, and why I decided to start blogging.

So for starters, my title and blog name. I wanted my blog name to be ducktyping, after one of my favorite features of my new favorite programming language, Ruby, but alas someone beat me to it. The next best thing was an offbeat quote from Fry from my absolute favorite TV show, Futurama. It is from "Roswell That Ends Well" and is from a scene where fry puts a stove-top popcorn package in the microwave, it begins to emanate a blue glow, and Fry sniffs the air and says "Hey, what smells like blue?"

In case you are wondering who I am, well, my name is Mike Stone. I am a programmer, and my interest in computers and programming goes back to when I picked up a book on basic somewhere around the fourth grade. It has been my passion ever since, and I often spend free time reading about programming, thinking about programming, or actually programming. I graduated from Cal Poly San Luis Obispo in 2004 with a BS degree in Computer Science and have been working in the field primarily in C# since then. The first programming language I grew to really love was Java, though I initially approached it freshman year with doubts, seeing it more as a toy language. My classes proved me wrong, and I grew to appreciate what Java offered... a multi-platform managed programming language that made certain types of bugs impossible. I especially enjoyed the documentation, which I have felt has always been easy to find what I need, and easy to decipher how to use the corresponding classes. C# failed to displace Java as my favorite, and really only strengthened a dislike of Microsoft in me that you may learn to discover if you read my future blog posts.

Enter Ruby 3 and a half years out of college. During a Christmas vacation at the end of 2007, I decided to tackle Haskell, but was also itching to try Ruby. I abandoned Haskell before really learning about what Monads were all about... I enjoyed the language, but Ruby was secretly calling to me. When a language like Ruby calls, you listen. Ruby quickly proved why there was so much disdain for Java in the dynamic programming languages camp. I can't describe it as anything else but "fun." I have read many times that the language was DESIGNED to be "fun" and it all made sense once I had taken the leap.

I started blogging because I have been reading blogs from such prominent authors as Jeff Atwood, Steve Yegge, Joel Spolsky, and the hilariously painful articles presented by Alex Papadimoulis. These blogs have inspired me to put out my thoughts, opinions, and observations of programming out in the wild, for better or worse. I am still a young developer (3 and a half years experience), but I have a thirst for more that I sadly don't see in many coworkers (at least at my current job).

What should you expect to see in my blog? Right now I want to talk primarily about my experiences with Ruby and Java, and why you should stop claiming Java is the best language there is and reconsider those dynamic languages you might be bashing at various other blogs or forums. It's tired and repeated over and over from many people, but I think we will see that our collective programming future will move away from the likes of Java and C#, towards the likes of Ruby, and Python.