Start Coding Like a Cowboy

Given that I've been blogging since 2002, you'd think I would've written about software development many times. As it stands, though, I've written very few posts about programming. I think a big reason for this is because when I'm coding away like a madman, I don't blog (or do much of anything else!) - and when I'm not coding, I don't feel like writing about it.

But I'm in the middle of another coding frenzy right now, and this time I'm going to open the kimono by talking about a "memetracker" feature I'm considering for a future version of FeedDemon (screenshot). I enjoy memetrackers like Tailrank and Techmeme because they let me know what people are talking about, but I'd like to have a memetracker that looks only in feeds I'm subscribed to in FeedDemon. Not only would this inform me about popular topics, but it would also let me read multiple posts about the same topic at once so I don't have to read them one-at-a-time as I come across them.

So, given that I'm supposed to be a seasoned, old school programmer, you probably think that I first sketched out some ideas for how this feature should work, and then followed some sort of formal design process before coding it.

Nope. I went straight to coding. No design. No plan. No nothing.

Now, before you label me a cowboy coder, let me explain. See, many times I've followed a more formal design process, only to run into some unforeseeable problem that stops me in my tracks once I start coding. More often than not, my logical design fails to predict the irrational behavior of some external process or service. For example, I'm sure I'm not the only aggregator developer who scrapped his initial design after discovering that a huge number of RSS feeds aren't valid XML ("but I planned to use an off-the-shelf validating XML parser, dammit!").

Rather than face that torment, I'll often start coding away for the sole purpose of running headlong into the cruel wall of experience. FeedDemon's memetracker is a good example of this, since I've already had to start over due to problems that I couldn't have anticipated.

Here's the fun part: a week or two from now, after I'm comfortable that the memetracker is working as intended, I'll then throw the code away and start a more formal design approach. Cowboy coding is a great way to find problems, but it's no good for commercial software - you've got to write well-thought-out code unless you're willing to be buried in bug reports. So the goal here isn't to complete a feature quickly (although that's often a side effect), but instead to discover the gotchas up-front so you can design the feature correctly.

PS: I should add that I have no idea when this memetracker feature will make it into FeedDemon - it's still in the hacking stages, so it may be a while before it appears.

Comments

[Chris Pirillo]

Whoa. Hack, man - hack.

[Nick Bradbury]

Chris, you'll be happy to hear that your subscriptions are serving as a nice test case for this feature - I figure if it can handle your set of feeds, it can handle anything :)

[Andrew]

Nick, I've read your blog for a long time now and I remember a post a while back with a screenshot of your dev setup. This thread got me to thinking while you are talking code you should give us a post on your dev setup. I'm interested to know. Seems like you were using a Borland product, but Newsgators stuff is .NET or MS C++.

[Nick Bradbury]

Andrew, check out http://nick.typepad.com/blog/2005/02/behind_the_scen.html

[Baher]

I've been waiting for this particular feature for a long time, see here:
http://technoogle.com/blog/2006/03/12/feedbucket-another-rss-reader/

It will transform the way we use Feed Readers, and will save a lot of time by offering a quick look at the important issues that is happening only among my trused list of sources.

I'm very glad that you're working on it right now, and hope it won't take a long time.

[Markus Pirchner]

Would it be possible to use the memetracker feature as an alternative way of navigation or structuring the feeds and posts? Like asking the user "Do you want to create a folder from this meme and have all corresponding posts stored in this folder automatically?" Similar to News Bin but more automatic ;-)

[Kevin Burton]

Yeah.... I do the same thing. I usually realize that I'll want to rewrite the first version anyway because I learn a lot in the process.

I've done this a LOT of times and its basically my primary coding method.

[Andrew Herron]

I wouldn't call this a hack, as long as you actually throw away the code - it's called prototype or exploratory coding, and usually leads to a rock solid design that can be used to write the final code :)

[Todd Cochrane]

Nick

My team has been in the hot and heavy coding mode since about Nov and with a couple of site launches behind us we are about to launch the mac daddy site we have been building to. Were gonna be dealing with a lot of RSS feeds and it was funny because I was asked a question today.

Should we only accept valid feeds. My response was in a perfect world yes but the way people have a lot of there data wacked no. My alternative was to set a database flag so we could follow up and look at the members listings to make sure there xml did not kill the site.

I have a feeling were gonna be looking at a lot of feeds. I instructed the team to only pull certain elements as I am pretty worried about the rss validation issue.

I would love to hear how you have overcome these issues and what some of the worst offenders were and if you had any guidelines. I'll be honest Id love to have one of my guys pick your brain for 15 mintues sometime as well on the issue

Todd...

[kosso]

I've had the same problem. SO MANY invalid rss feeds. It's a bloomin' disgrace!

Even so-called 'pro' podcasts appear to hand type their feeds and often break it.

Sometimes, when I get the time I send an email to the author to highlight the problem an explain a fix. rarely do I get a reply. I often see items in reverse order.

I'm tempted to set up a forum system where I can list those DATA PERPS and help them get it fixed. One post per broken feed - each reply to each will thread the fixing journey. They owe it to themselves. Doesnt scale too well on my own, so thought forum systems are suited.

When we index a feed, after jumping through hoops to understand the producer's intentions, to attempt to sanitise it do an extent - and chop down to the last 'x' items - as also some people archive over a year's worth of shows in a feed - still.

Advice - feed building tools - assistance - best practice : that's what's needed imho. The awful thing is - judging by some of these offending podcasts, some of these people are actually being paid somewhere along the line to create the feed - and not doing it correctly.

Cheers,
Kosso
http://podcast.com

[Shawn Oster]

I love the programming topics! I feel an irrational connection to your programming simply because a) I do 70% of my coding in Delphi (the rest is C# and Ruby) and b) I live in Denver where you've mentioned visiting because of the NewsGator offices.

I feel I've also come to the same style of development, a bit of cowboy code to do some discovery and then you can base your design on reality vs. the idealized world of how things "should work". I was rather glad to see that Ron Jeffries talked about doing a very similiar thing in his great book, "Extreme Programming Adventures in C#". The whole book is a great read because it's nice to see the development process, warts and all.

[Gabe]

What's a "memetracker"?

[Ryan Mack]

I do the same thing. I've just accepted the fact that there is no better planning than actually doing. I also regard programming as an art form. Few artist (novelists, painters, sculptors, take your pick) sit down and directly produce a finished product. They produce a rough draft and though successive refinements, finish the piece.

[Rick Klau]

Nick -

I'm already drooling! FWIW, one of the reasons I remained a SharpReader user for so long before committing once and for all to FeedDemon was one feature: threaded presentation of posts within my subscription list. I blogged about it a couple years ago here:

http://www.rklau.com/tins/archives/2004/04/16/sharpreader_-_threaded_rss.php

With a screen grab of what it did. I see your idea providing me much of the same visibility, which would make FeedDemon even more critical to my ability to sift through this info.

In an interesting twist, it would actually encourage me to subscribe to more feeds - not so much to read their content, but to see when they show up in the cloud along with other sites who are talking about topics I care about.

Great stuff, thanks for opening the kimono a bit.

--Rick

[Sterling Camden]

I guess WordPress doesn't pingback to your blog. More thoughts on code-first development: http://www.chipsquips.com/?p=374

[Karyn German]

Why Nick, this is simply the very agile concept of prototype/design/build/test...love it!! It's been a bit more challenging for us on our online features due to infrastructure issues, but I am a big fan of this technique.