a little madness

As programmers we tend to get attached to code we have written. After all, we spent all that effort dreaming it up, typing it in, and (sometimes) making it work. Many of us also like to look over how much we have churned out, to see that lines of code metric crack the 500,000 (or whatever) mark. These sorts of things make us loathe to delete code we have written. But we forget these important points:

1. Maintenance: someone has to maintain all of that code, and I’m sorry, but it’s you! The less code you have to maintain, the better. So next time you try and crack the million LOC, think about how fun it is going to be to maintain it all .
2. Dead code rots: leaving unused code around “just in case” is a particularly bad idea. If it isn’t exercised, it will become broken very soon. Its very presence distracts the reader from their real task.
3. Source control: think you might need that dead code again? Think that a bad refactor may bring you unstuck? That’s what source control is for: it has got your back.
4. Efficiency: every little bit of code slows everything down just that little bit. More disk space. Longer compile times. Slower IDE performance. Larger reports. Over time, all these little things add up.
5. Elegance: verbose code is often a sign of a non-optimal solution. When refactoring, you should be able to use your superior knowledge of the problem to simplify. Cutting back code in this way removes the cruft that builds up over time.

So, stop being so happy that you’ve produced so much code. You should be much happier to achieve the same result with a smaller codebase, even if it means throwing away yesterday’s hard work!

Fresh out of a Computer Science degree at uni, I hit the job market looking, naturally, for the best job I could find. I immediately ruled out moving into big business (think banking, insurance) and the consulting roles that tend to take you to such places. I wanted to work with technology I could enjoy, on new projects, churning out interesting “stuff”. That is what led me to the wonderful world of the VC-funded startup.

There are many great things about startups:

1. New projects: by their very nature, startups have brand new projects to work on. We all know that starting from scratch is much more stimulating and fun than maintaining a 30-year-old COBOL codebase.
2. Great people: because they have interesting projects and questionable job security, startups tend to attract people with a real interest in technology. You don’t get people who moved into “IT” as a career strategy, you get people who love what they do.
3. Great atmosphere: partly because of 1. and 2., but also because of their small size and youth, startups often have a great feel. They haven’t had time to develop the painful politics and beuraucracy of older, larger companies. Hopefully they never will!
4. No customers: when you’re just starting out, there are none of those annoying end-users to pester you with support questions and Real World problems. You can just code away in your own little world.

So I jumped aboard the VC-funded startup roller-coaster, and I had a great time. I lost my job a couple of times, but there were always new ventures to start on. I had a ton of fun and I learned heaps.

But something did start to bug me. I poured my efforts into these ventures and their projects, and even though the technology was fun, there was somewhat of a hollow feeling. The problem was that all of this effort was going to waste: without customers everything I created was consigned to obscurity. Of course the startups planned to get customers, and lots of them, but the business model was the classic boom-or-bust strategy. Either we would “disrupt” the world, or we would fade into oblivion. Of course, the odds lean heavily towards the latter.

So this supposed advantage of not having to worry about customers turned out to be the one great drawback of working for startup ventures. Jeff Atwood touches on it in his post Shipping Isn’t Enough:

How many users actually use your application? Now that’s the ultimate metric of success.

I achieved plenty in terms of development as a programmer, and I even managed to do some pretty good work (I hope!). But at the end of the day, I didn’t have the sense of achievement that only comes from creating something that other people use.

However, I have not given the startup game away . I realised that you can have all the advantages, just with a different strategy. So at Zutubi we’re not planning to “disrupt” anyone. We just make software that people love to use, and grow organically off the back of that. And for once, I have Real Customers that use my software, and I interact with them closely. Who would have thought it: I love dealing with customers! Not because they pay (though I thank them for that ), but because every time I learn of someone new using our software, I get that sense of achievement I had been looking for.

Even while I wrote this entry, an email landed from a customer commenting on how they loved the user experience of Pulse. Honestly, it does not get any better than that!

I would like to draw your attention to the recent release of the new Java Persistence API for the Berkeley DB. In short, the Berkeley DB is designed to be a very efficient embedded database. With no SQL query support, you instead talk directly to its internal BTree via a very simple API. A very good write up is available at the server side for those interested in the full details.

This style of persistence mechanism is certainly not for everyone. If you want adhoc query support for writing reports and extracting data, look elsewhere.

If you don’t, then Berkeley should be considered when defining the persistence architecture of your next project, just don’t tell your DBAs. Not only is it reportedly very fast, due largely to the lack of SQL style interface, but it also supports transactions, hot backups and hot failover, all of the things that help you sleep at night. However, what has me intrigued is idea of not having to deal with SQL schema migration.

I consider Schema migration to be one of the more tedious and yet non-trivial tasks that is required by any application that employs relational persistence. Yes, Hibernate makes this task somewhat easier to deal with. However, even with Hibernate, you will still need to roll up your sleeves and write some SQL to handle the migration of the data.

Managing schema migration within the Berkeley DB is different. Where as previously you extracted the data via SQL, converted it and then updated the DB via SQL, with Berkeley, you just convert the data in Plain Old Java. They have some examples in there javadoc that gives a reasonable idea of what is involved. Below is one of these examples, a case where the Person object’s address field is split out into a new Address object with 4 fields. The core of the work is done by the convert method:

Personally, I think this is a great improvement. Now, if only I had been aware of this at the start of this project, things might be a little different, and faster, and some other good stuff as well.

So are schema upgrades a thing of the past? Maybe not, but they don’t have to be a part of every project.

It appears that I am not the only one wondering if the pragmatic keep it real approach to software development will result in programs that are missing that something extra, those features that you can not do without the moment you find them.

Kathy writes:

Our users will tell us where the pain is. Our users will drive incremental improvements. But the user community can’t do the revolutionary innovation for us. That’s up to us.

Dave takes a more direct approach when he writes:

It does seem, though, that agile teams will be less likely to either prioritize or implement some of the more subtle touches of the kind that the Wired article discusses. When forced to choose between the features “Send email”, and “Implement graceful date column resizing”, guess which one is likely to get short shrift.

An excellent example of this type of feature, a feature that you didn’t know you needed until you saw it, is iTunes smart shuffle. It “allows you to control how likely you are to hear multiple songs in a row by the same artist or from the same album”.

I don’t know how I managed without it.

I can not help but wonder whether this feature would have been added by the agile keep it real approach? It is not an essential feature, and I would not have noticed if it was not there. But the fact that it is makes me a very happy user.

Every time that I have upgraded my monitor, I have marvelled at how much more screen realestate is available. Each time I have gone through a period of adjustment where I adjusted the layouts of my applications to take maximum advantage of the extra space. My latest move from a 19inch to one of the nice Dell 24inch screens has allowed me to move to a three column layout in IDEA with the project view on the left, the structure on the right, and still plenty of room for 120 char lines in the editor. Awesome. Moving on to Thunderbird has yielded a similarly pleasing outcome with its nice three column vertical layout. Nice. Even JEdit supports a configurable layout that allows me to make good use of the screen.

Unfortunately my amusement came to an end when I opened up my browser. Either one, it matters not. Where ever I go, whether it’s my favourite news or community website, any one of the blogs I try to keep up with, I end up with one of two things. Either a fixed width page with half my screen empty on either side (but all of the content nicely squeezed into the middle), or a 100% width page where the lines are so wide that they become difficult to read and large patches of whitespace spread around the page.

Now I understand that large screens are only just becoming affordable, and so I do not expect websites to suddenly rearrange their content to suit me and my dislike of unused pixels. It does however bring to attention one aspect of building web applications that can be very frustrating. The page layout. Now certainly some people are doing nice things with the tools that are available, namely css, but that can only go so far. What I would really like to see now is a good old fashioned trustworthy layout manager.

Only just last week I was trying to achieve a relatively simple task. Opening a popup window that is an appropriate size for the content it will display. Well, I had hoped that it would be a relatively simple task. After loading the content into an iframe, doing my best to work out what its visible width and height would be and then dropping it into a popup, I am still no closer to a solution that works in both Firefox and IE. With a slightly better API for handling page layouts, this type of task would become dead simple. Speaking of layout managers, those guys over at qooxdoo are doing some pretty interesting things. Now, if only it didn’t look so much like windows.

Sorry readers, I know you have all been waiting with bated breath for the final installment in the wiki series . Well, here goes:

6. Unreadable syntax for tables

   The problem is actually more generic: the simple wiki syntax works well for inline elements, but poorly for structured elements. Why? Mainly because whitespace is significant. This is convenient most of the time, but gives no way to format a structured entity such that it can be easily read. A possible solution is to allow insignificant whitespace so that structured elements can be formatted with indentation. Naturally, for regular paragraphs, significant whitespace would have to stay. I think the semantics for whitespace can still be clear if the insignificant whitespace is only allowed “outside” of the actual content, in the space between whatever syntax elements are used to delimit the table (or whatever it happens to be). Or you could just use a WYSIWYG editor, but you won’t catch me doing it .

7. Editing in a browser text area sucks

   A couple of readers actually pointed out some neat solutions for this one, such as the mozex extension for Firefox, which lets you use an external program for editing instead of the browser text area. Nice idea, but still a little awkward, as you can’t avoid interacting with the web UI for other actions (this is a wiki after all!). I think the real answer is the slow, painful progression towards browsers as a richer application platform. Many have tried to create the ubiquitous thin client technology for the web, but none can compete with the inertia of the trusty old web browser. Unfortunately for us users, this means we’ll be waiting a long time for browser technology to evolve to a state where web UIs are actually enjoyable to use. Just look at how excited we all get when AJAX lets us emulate two-decade-old desktop technology in our web apps.

8. Poor support for versioning

   Let’s start simple: a wiki that doesn’t have at least RCS-level versioning support (simple revisions, diffs, history) should not even be allowed to exist. A wiki that doesn’t have some form of merging support (with conflict resolution) isn’t worth using. Personally, my requirements are even more strict. I maintain documentation for software products, and these products have multiple release streams. I manage these streams in my code base using branches, and ideally would do the same with the documentation. I don’t know of a wiki that supports this. This seems a shame, because the version control technology has been around for a long time. Rather than reinventing a weaker versioning scheme, I don’t understand why more wikis don’t use a full-blown version control system (like Subversion) in the back end. There is a wealth of existing tools and technologies that just aren’t being leveraged here.

9. Lost edits due to browser crashes

   First off, for those wikis with autosave, this is much less of a problem. I would like to mention one insightful comment from reddit, however. There, poster rahul turned this problem back at the browser itself. A good point: why don’t the web browsers implement their own autosave for form data? They are already part of the way there by remembering form input, it is not much of a leap to get from there to autosaving for text areas. The obvious benefit of this approach is that once it is implemented in the browser, there is no longer a need for every web app to reimplement the same functionality.

10. Wiki discussions

    People, please just stop using flat wiki pages for discussions! It hurts us, precious! Some wikis have a more enlightened approach, and allow threaded comments on each page. This is a huge improvement, no doubt. Still, will they ever match dedicated forum solutions? Maybe wiki implementors should be looking again to adopt existing technologies. I wonder how feasible it would be to integrate an existing forum solution into the wiki. Perhaps a looser coupling would even be benefitial, allowing more diverse tools to be linked to a wiki page. The counterbalancing force is of course the benefit of tight integration.

So there you have it: some good can come of a rant after all. At the very least, I have picked up a few new ideas myself!

——-
Into continuous integration? Want to be? Try pulse.

Wow, I’m quite surprised by the coverage received by my previous post: 10 Things I Hate About Wikis. I was just blowing off a bit of steam, but somehow I managed to spark some discussion and learn a thing or two. So, I thought: why not take this discussion and turn it into something constructive? Like I say in my previous post, my beef with Wikis is not with the idea, but with execution. Of course, execution can (and hopefully will) improve over time. So I’ll go through my 10 points with some clarifications, ideas and contributions from others. To keep things sane, I’ll split it over a couple of posts. Here goes:

1. Wikis are the easiest way to create awful documentation.

   This is mostly a wiki maintainer issue. As many comments point out, it’s all about having discipline when creating and maintaining the wiki content. I won’t try to address all of the cultural issues involved in successful collaboration. My concern is what the wiki implementor can do to help. It is too easy for the implementor to throw their hands up and say it is out of their control, but this is only partly true. A wiki can encourage quality by making the easy way to do things also the right way. Given the constantly-evolving nature of a wiki, the implementation should also encourage frequent refactoring by making it easy to reorganise content. Features like moving/renaming pages are quite common, but not always convenient, especially when you have a lot of content to shuffle around. How about features like splitting/joining pages? Global search and replace? I’m sure people have loads of ideas.

2. WikiWords

   No software should restrict its users when it is so unnecessary. Wiki implementors: get rid of the restrictions, it’s really not that hard. Then I won’t have to hear about why the title of the page I am reading is inaccurate due to technical restrictions.

3. Every wiki has its own syntax.

   The only answer is to standardise the syntax somehow. Will this happen? I think it is likely a standard will appear. It is much harder to tell how widely it will be implemented. There are sure to be compatibility problems with the masses of content that already exists. I would like to think, for the sake of their users, that wiki implementors will consider it. If wikis are here to stay, and continue to spread, this will only become more important.

4. Wikis mark the return of the content management dark ages.

   This point was perhaps too brief to convey what I was really getting at. The facilities offered by wikis to create headings, lists and so on are fine. The markup is indeed semantic, as has been pointed out. What I miss (possibly through ignorance, wiki implementors please correct me) is an easy way to create my own styles. Not just custom HTML fragments, but the ability to create font and paragraph styles using CSS. I’d love to see existing solutions, I just haven’t come across one yet. For example, I can imagine a wiki allowing me to define a class in CSS syntax, and then apply that class to any part of my content by wrapping in the appropriate syntax. There’s no reason why the built in syntax can’t just be shorthand for the application of pre-defined classes that I can also modify if I wish.

5. Inexplicably poor navigation.

   I know wikis aren’t a traditional hierarchy, and nor should they be. The possibilities with a wiki are much greater than that, as linking is such a fundamental part of the system. The point about searching made in one comment is a good one, a powerful search is a must and a great way to find content. However, search only works when you know what you are looking for. Convenient navigation, on the other hand, allows you to discover related content. My problem is when useful navigation functionality is non-existent or hidden away. Of course, this varies from wiki to wiki. I think at the very least, I should be easily able to navigate to:

   • All ancestors of the current page
   • All children of the current page
   • All siblings of the current page
   • All pages that link to the current page

   When I say easily, I don’t mean navigating to another page which shows me the links. I want access to the links on the current page, although allowing them to be shown/hidden may be necessary to avoid clutter. In addition to this page-to-page discovery, wikis also need a convenient way to browse around the entire content. In my experience this functionality exists, although the UI could often be a lot more dynamic.

Stay tuned for part 2.

——-
Into continuous integration? Want to be? Try pulse.

OK, so wikis are a great idea. I have “embraced the wiki” as a great communication tool, and there are many, many benefits. But still they manage to get to me. It’s almost always down to one thing: execution. Execution by the implementors of the wiki, and execution by the people creating the content. So here it is:

10 Things I Hate About Wikis

1. Wikis are the easiest way to create awful documentation. Lowering the barrier to entry is good, but if I see another open source project throw up a wiki and think they now have documentation, I’m going to scream! Perhaps we should call it lowering the barrier to stupidity (arrogance mine ).
2. WikiWords. Not all wikis are affected by this blight, thankfully. But don’t you love those that are, especially the knots you tie yourself in to manufacture a WikiWord when you just want to use a single word!
3. Every wiki has its own syntax. Sure, HTML is too verbose to be convenient for editing wikis – it ruins the whole idea. Unfortunately, however, this has led to a proliferation of custom wiki syntaxes, each with their own quirks. Hence, working with multiple wikis is a pain, and every wiki has its own learning curve.
4. Wikis mark the return of the content management dark ages. Once upon a time, we formatted documents in our word processor with font sizes, bold text etc. Eventually we realised this was a Bad Idea, and styles were born. (OK, introduced. Back in your box now, Tex groupies. And make sure the troff monster stays in there.) These days, nobody in their right mind creates a significantly-sized document without using styles. Then there was the internet. Remember the early days? The <b> tags? Eventually we realise this was a Bad Idea, and stylesheets were born (easy there, troff monster). Now we are back where we started again. I hope there are no LISP programmers in the room, because they’re bound to mention they “told us so”…
5. Inexplicably poor navigation. Come on, wiki implementors, this should be one of your strongest points! Too often I find myself deep in the bowels of a wiki without a sensible way to navigate around. Sure, some of this is down to the wiki author, but there are so many opportunities for convenient navigation that are missed, by either not allowing the navigation or by hiding it away somewhere.
6. Could anything be harder to read than a table written in typical wiki syntax? This is where the simplicity of the syntax falls flat on its face. The syntax works for basic, inline elements, but start to create strutured data and you become lost in a sea of ascii art (and not the good kind).
7. Editing in a browser text area sucks! Possibly the only thing that sucks more is the half-assed rich text editing facilities wikis sometimes offer. Unfortunately, we’re pretty stuck with this one. Maybe advances in web UIs will help…
8. Poor support for versioning. One thing I always hated about creating documentation in word processors is the inability to track changes and merge documents (yes, I know Word has an “implementation” of this feature – if only it actually worked). On the face of it, wikis have both the opportunity (text-based format) and the motivation (strong chance of concurrent editing) to have strong versioning and merging support. However, most of them don’t. Wiki implementors: this is a (largely) solved problem! No excuses!
9. Losing 30 minutes of typing because my browser crashed, or I closed the tab, or some other minor tragedy occured. Thank god wikis are starting to implement autosave! Dragging themselves just a bit out of the dark ages .
10. Wiki discussions, e.g. those found in the original wiki. OK, so wikis were a cool new idea. That doesn’t override the fact that forums and newsgroups already existed as a much better medium for online discussion!

Phew, that felt good. Now wiki lovers everywhere, I’m ready for you to tell me how I “just don’t get it”.

——-
Into continuous integration? Want to be? Try pulse.