Accidental Complexity

I thought I had some interesting insights in my last post about Postgres and why I was going back to MySQL, at least for the time being. So much so, that I posted it on Reddit. Alas, it didn't go over well. It had a lot of downvotes, and most of the comments were fairly negative. I think I'm still right about what I did and why, but I think I can better articulate why I think so in two words:

Accidental Complexity.

It's definitely not a new concept, and I'm sure many people have talked about it before, but in reading the responses on Reddit, it became clear that this is precisely the reason why I switched to MySQL, and precisely why I probably won't switch back anytime soon.

Wikipedia defines accidental complexity as "complexity that arises in computer programs or their development process (computer programming) which is non-essential to the problem to be solved."

This perfectly describes the issues I was having. Think about if I could sudo apt-get install postgres, hook it up in Rails, and be on my way. I could focus nearly all of my time on essential complexity (ie developing features for my actual application), or at the very least, accidental complexity arising from other applications or from my coding choices.

The choice may bite me later on with other accidental complexity that Postgres tackles well, but right now I want to actually get the project to a point where I can start showing people, and maybe start getting some users. After all, getting users is the whole point. To that end, reducing the hassle being caused by the database is a Good Thing.

So, I really think this should be a lesson for all the applications I write, and all the applications you write as well. Make sure your application reduces the accidental complexity you are forcing on your users. If a significant portion of them are spending a lot of their time configuring your application instead of solving the problem they are trying to solve, you have failed them.

Don't take this to the extreme, though... you still have to see the big picture. Who knows, maybe the default user settings is good for the majority of users, and I just happen to fall in the unlucky minority, but comments like "pg_hba.conf can be a bitch, but updating it to allow transparent local access is just a couple of lines away" tells me that I'm not the only person who has had trouble configuring Postgres, even if the configuration changes are small.

Postgres... is it worth it?

In working on my side projects, I needed to pick a database. I decided to pick Postgres, and it has been nothing but problems since the beginning. It sucks, because half the problems don't really seem to be the fault of Postgres, though they all seem to stem from the issue that IS their fault.

So, when I first started with Postgres, right off the bat it is a pain. Every single time I need to install Postgres (usually on a new development machine), it is a pain in the butt! They provide a lot of flexibility for database user security. However, the last time I installed MySQL (which is admittedly a long time ago), you set it up, and it asks you for a root user password, and that's it. The rest seemed a snap, at least in the rose colored glasses of the past. With Postgres, each and every install requires me to search to figure out what settings to change in which config file just to get the damn users to be able to connect! Give me some reasonable defaults! If I try to connect to my database locally with a specific user, it would be awesome if the system would give full permissions to any databases I create with that user. Then things like rails would work out of the box, no configuration necessary. I will lock down the system to my hearts content if I need to, but make it JUST WORK first.

That was probably the extent of what I can actually blame Postgres for. It's a big fault in my opinion, though. Software that just works out of the box with no configuration necessary is nirvana. That is why Rails typically feels like heaven to me.

The next issue was probably HostMonster's fault. They give Postgres a back-seat treatment. They run an old Postgres version (I think 8.1), with the excuse that they need to wait till cPanel (their site management web app) upgrades what they support. Ok, I can live with that, but it sucks. When I actually tried to create a database, though, Rails couldn't connect to it! Grrr. HostMonster was on it though, and actually fixed my configuration issue. That was cool of them, go HostMonster! I have been a fan of their support, but that's not what this blog post is about. The issue had been a Rails configuration, I think changing how it was connecting... probably because HostMonster's user connection settings were set up a certain way, contrary to my Rails configuration.

Things were smooth until I needed to create a new database for a new website I wanted to create. It just didn't work. I couldn't create the database, and there was nothing I could do about it... possibly something HostMonster had done, but I didn't want to wait for a fix, so I took the dive and just used a MySQL database.

But I stuck with Postgres on my development machines. Why change? I still wanted it more than MySQL.

Until now.

I just upgraded to Karmic Koala on my netbook. I think it went well, except the upgrade from Postgres 8.3 to 8.4 didn't go so well (which was part of the Karmic upgrade). Both versions are now installed, and when I try to load Rails, I can't connect to the database. Running rake db:migrate even fails. No nirvana, no more Postgres. I'm fed up with the issues. I'm going to the dark side of MySQL, and I'm not going to come back until it is dead simple to set up a Postgres database with Rails.

Postgres intrigued me so much because of the recent drama surrounding MySQL... namely that the Evil Empire (Oracle) now owns them. Perhaps it's a bit superficial, but it's drama I didn't want to get caught up in. I want to know my database system will be open and free throughout the life of my products. However, MySQL makes things so easy that I just can't resist switching back. I will reconsider if it becomes easier to set up a database on Postgres out of the box.

Lesson? Make defaults that work towards achieving your users goals without them needing to dig into any documentation. If it's not easy to start, your users won't start, so you won't have users (at least not as many as you could have).

A Nerd's Proposal

I got out of the car, and then helped her out. I was a bit nervous as we walked towards the restaurant, arm in arm. She reached her arm around me, edging her hand close to my inner left breast pocket. Uh oh! I stiffened my arm and blocked a potential revelation on her part. Safe for now (so I thought, anyway).

We walked into the restaurant and took a booth. It was New Year's Day, and not really that full. I had wanted a more significant location, but my first choice place was closed that day... drat! After some idle chat and ordering our meal, I figured it was now or never... or at least now or some other day. I wanted to do it now, so I took out my phone.

"I found this really cool Will and Grace app!" Her Christmas present was the Will and Grace box set. She was a big fan, so I figured a Will and Grace app would be interesting to her.

"Check it out, it's pretty cool. Click through that splash screen." I reached down into my inner coat pocket and grabbed the box. I opened it and waited.

She clicked through, selected a season and episode. She hit the item to play the episode, and after a brief pause, "Can't Take My Eyes Off You" from the Jersey Boys musical started playing. That's our special song, so I figured it was a good pick. I pulled up the box and held it out as she looked at a graphic with a couple hearts with the words "Will you marry me?"

Each moment that passed after that was stretched out like an eternity. She finally looked up and simply said "yes." That's it! I'm now engaged. She's the most wonderful woman in the world, and now I get to share my life with her.

Now that you know how it went down, let me tell you what went wrong. I settled on the day a few weeks before, while I was at my parent's house for Christmas vacation. I realized my Android phone would be the perfect way for me to pop the question... I could write a customized little app just for her! Google images proved useful for the "Will you marry me" graphic. It was a snap to rip our song from my CD musical recording. The hardest part was gathering all the content for all the Will and Grace episodes. Episode names was all I needed, but I had to take them from IMDB and convert them into Java strings. It was time consuming, but once all the content was set up, the Java app itself was a snap.

I guess that was what went well. The blunders came closer to the day. First came when I was asking her for dinner on the first. "Let's have dinner New Year's Day to ring in the new year!" Whoops. Very bad choice of words, especially since we picked the ring together... she knew it was coming, just not when. Why the hell did I say THAT??? She was of course suspicious, but I assured her it was just for dinner... I totally hadn't meant to give her a hint so early, but what's done is done.

We decided on the perfect place for dinner. Our 6 month anniversary had been at a nice fondu place. Perfect! I called them up, hoping to make a reservation, but alas! They are closed on the first! I expressed my disappointment to her... perhaps too emphatically. We agreed on another place. Not ideal, but I was determined to do this on the first.

New Year's Eve was at her parent's place. It was a lot of fun, and we were all packed to head over to Sacramento, which was our destination. I spilled some drink or something on my pants and all I had to spare was my nice clothes prepared for dinner. She suggested I change, and when she saw I only had nice clothes, she looked at me odd "we are dressing up for dinner?" I gave an excuse that it was New Year's... while wishing I had planned this a little better. All my effort went into the program to propose, why hadn't I worked out these details a little better? I'm not exactly good at thinking ahead and planning things well. If I could do it all over again, you can bet I would figure it out a lot better, but in the end it's the marriage that excites me more than the proposal.

With all those hints, she knew it was coming. In fact, she confessed she was reaching around while we were walking in because she wanted to test my inner pocket for the box. Sneaky! She saw me fiddling under the table when I passed her the phone, and thought "this is it." Thankfully, my application fooled her enough to second guess her inclination, so I salvaged a bit of a surprise.

I'm almost embarrassed to post to the world how unromantic my proposal was, but I think all the blunders made it memorable in it's own right... and it's very much me. I focused on the part I liked most, the programming, and assumed the rest would just go smoothly... because... you know, I had wrote a PROGRAM to PROPOSE! ON MY PHONE! I'm now just excited to become a part of her wonderful family, and know that I can spend the rest of my life trying to make her as happy as she makes me.

Quick Lesson from DevDays

Ok, so I went to Stack Overflow DevDays in San Francisco this last Monday, and it was a lot of fun. I went with a friend from work, and we both walked away happy, stickers in hand.

I took some notes, and I would like to share what I thought of all the speakers and their presentations, but we'll save that for another day... if I actually end up writing it (post a comment if you are interested, and hopefully that will push me to write it up).

What I want to talk about is a small bit I learned from one of the presenters. If you don't know me, I'm a big Linux fan, I heart Google, I have an Android G1, I love open source and Ruby and Rails, and Microsoft is practically the devil. Given that, you might find it interesting that the thing that struck me most... enough to actually write about it... came from Microsoft. That's right, it came from the near-devil. Well, it came from Scott Hanselman, who was a pretty good speaker.

Scott talked about Microsoft .NET MVC. It seems like a relatively cool product, if you can bear to be in the Microsoft world, and if you can bear to deal with such a clunky language as C#. Ok, I gotta give props that C# is a decent language... for a statically typed language... but the platform restrictions (excluding Mono), and it not being nearly as awesome as Ruby, will keep me from ever dealing with it again (I worked in C# professionally for a few years). Yes, .NET provides access to a fleet of languages, but C# is still their flagship. Anyway, from the discussion, MVC seems like it does a decent job at mimicking Rails, though it still didn't seem nearly as elegant.

That was a long lead-up for a simple point I extracted from the Big M... <%: value %>. That's it. I have wished the default of scriptlets was to sanitize html output for a long time, and I'm pretty sure Jeff Atwood had a post on it at one point. That tag, if I understood Scott right, differs from <%= value %> in that it will sanitize the output automatically.

On the train ride back to the south bay, I set it upon myself to implement this auto sanitization in Rails for my current side project. Initially I thought I could do something like:

def :(arg)
 h arg
end

But alas, that doesn't work. Ruby won't let you define ":" as a method... it throws a syntax error... drat!

So, I figured, maybe I could monkeypatch the ERB handling so that <%: value %> works as I want, so I looked around the Rails source for how ERB compiles the views. It didn't take long to find erb.rb, which deals with the compilation. I inspected it for a bit, and toyed with some ways to deal with the problem, and ultimately came up with:

class ERB::Compiler::Scanner
  alias_method :initialize_without_sanitize, :initialize

  def initialize_with_sanitize(src, trim_mode, percent)
    initialize_without_sanitize src, trim_mode, percent
    @src.gsub! /<%:(.*?)%>/, "<%= h(\\1) %>"
  end

  alias_method :initialize, :initialize_with_sanitize
end

If the above code is cryptic for you, it basically overrides the initialize method (which is the constructor method for all you non-Rubyists) and uses a regex to replace all of the new sanitizing scriptlets with what would actually work to sanitize it. So, <%: value %> gets transformed to <%= h( value ) %>. I can't vouch for the performance (because I haven't done any performance testing), but it works.

Put the above snippet wherever you put your monkeypatches that you want run once. I have mine in a monkeypatches.rb file set to load after Rails has initialized (so it runs just once, at startup time).

Go forth in sanitized goodness.

Why Newspapers Can Go Away

Joel Spolsky and Jeff Atwood's most recent podcast struck a nerve with me. They go on for a period of time about Craig's List and the effect it has had on newspapers. Joel in particular tries to argue that Craig's List has essentially killed off the public good of investigative journalism, and by refusing to cash out on Craig's List in some way, has not returned the public with anything other than the marginal usefulness of free classified ads.

Normally I think very highly of Joel and his opinions, but in this instance, I think he has completely missed the boat. Jeff tries to poke holes in his argument, but he didn't articulate the argument that I most strongly feel....

Investigative journalism is now a distributed affair, and no longer needs the legwork of the newspapers that Joel alludes to. On a side note, I want to point out that the linking to eachother that Joel accuses bloggers of is quite prevalent in newspapers as well... aren't popular Associated Press articles picked up and run nationally all the time?

I just want to point out one recent case study to illustrate my point... the shooting of Oscar Grant on New Years Day in an Oakland BART station.

I admit, I haven't followed up on how the case against the officer panned out (or if it is even through yet), but the point remains that something like excessive police force cannot go unnoticed in todays world. We don't need newspapers to do the legwork anymore because the ubiquity of the web, combined with a computer in almost everyone's pocket in the form of a smart phone capable of snapping a photo, shooting a video, or recording a conversation and then posting it online immediately has put the power of investigative journalism into the hands of everyone, everywhere.

If something corrupt happens within earshot (or eyesight) of someone, there is a much greater chance nowadays that someone is listening and could easily publish the wrongdoings within minutes. This is not to say that we can do without real journalists going out and finding corruption and shining the light on it, but I firmly believe the Internet and technology is our salvation, not the newspapers.

Jeff, I think you are right that we are in a transition, and the spotlight on the bugs under the rock is being lit by everyone now.