Hindsight and the 80/20 rule

Good perspective on the 80/20 rule and the obvious benefit of hindsight!

It’s easy to be cynical about the 80/20 rule. There are too many hucksters selling books and consulting services that boil down to saying “concentrate on what’s most productive.” Thanks. Never would have thought of that. Let me write you a check.

At one extreme is the belief that everything is equally important, or at least equally likely to be important. At the other extreme is the belief that 80/20 principles are everywhere an that it is possible to predict the “20″ part. Reality lies somewhere between these extremes, but I believe it is often closer to the latter than we think. In many circumstances acting as if everything were equally important is idiocy or sloth.

You can improve your chances of correctly guessing which activities are going to be most productive. Nobody is going to write a book that tells you how to do this in your particular circumstances. It takes experience and hard work. But you can get better at it over time.

Can you predict the “20” in 80/20?

The Unexpected Upgrade Experience

A couple of months ago I upgraded my hosting package during its annual renewal. Up to then I had an excellent experience with my provider and so when they made me an offer to move to their new hosting package for a similar price as my old package it seemed like a reasonable thing to do. My thinking was that since the old package was no longer available to new customers then it would only be a matter of time before support for the old package was reduced and eventually discontinued.

Since the migration to the new package, I have been plagued with issues. Nothing catastrophic but having to raise support tickets and the resultant email tag with the support team gets tiresome after a ticket or two. The root of the problem is that the majority of the help items and howtos relate to the old package and not the new one. That would be fine if they both operated similarly – but they don’t!

I’ll use my most recent issue as an example. On the old package, the WordPress application was installed for a domain using the application manager and the application was installed into the httpdocs directory area for that domain. I stupidly assumed that the same would be true for the new package – especially since the application manager offered a WordPress installation. So, I installed the application and imported the posts from my old package and everything seemed to be fine. That was until I tried to change the permalink format for the posts. When I tried this I got a .htaccess permissions error. It turns out there were two things wrong. The first was that the WordPress files in my domain’s httpdocs area were actually a copy from my old package that were copied across by my hosting provider during the migration and were completely independent of the current WordPress installation. The second was that the application install for WordPress installed into a hidden directory tree that was so hidden that you could not browse to it without knowing the exact path before hand – and that exact path contained a five digit object id as the hidden part of its pathname. Once the support team figured this out, we easily sorted it out. 

This sort of thing should not have happened in the first place. Where differences in hosting packages exist, there should be a cheat sheet available to help people migrate between packages. The howto documentation should also cover both packages, not just the old one. Having good tech support is great but having a great product that doesn’t require tech support is so much better!

Hopefully all my problems are finally resolved. I’m on the fence as to whether or not I’ll stick with my provider … one more issue will push me over!

Frequent Informal Code Reviews

The traditional code review model is to write a bunch of code and then gather the troops to review it once its done and ready. Following the review the author then spends a couple of hours/days in rework.
In contrast one of the benefits of pair programming is the just in time scrutiny of a second pair of eyes while the code is being developed. This just in time scrutiny helps pick up problems early and the cost of rework is minimal. Pair programming does however have some pitfalls and is not as prevalent as it supporters would like.
There is a middle ground that involves team members spending a few minutes each day informally reviewing any changes that were committed to the source code repository in the previous day. The informal review involves reading the changes and posting questions and comments to the author. This provides “almost in time” feedback and keeps everyone in tune with the changes and direction of the code base.

This approach has the following characteristics:

  • Collective code ownership
  • Frequent informal code reviews
  • Requires regular commits to the repository
  • Requires discipline from team members to frequently review new changes – otherwise thy just build up and we are back to where we began
  • Has effect of improving code quality  – people are less likely to commit rubbish with the intent of cleaning it up later since they know they will receive instant feedback

Note that some shops mandate a code review prior to each checkin. While this approach does help keep up the quality of committed code, it does introduce delays and bottlenecks in the checkin process.

Ensuring Success at Code Reviews

Volumes have been written on the subject of code reviews and various processes for conducting them. Most processes describe particular code review roles such as reader, moderator etc. and formats for conducting the code review meeting. These roles are certainly important to keep the code review meeting on track and to avoid the review degenerating into religious wars on commenting styles and magic numbers. However, I have witnessed many code reviews that had brilliant moderators and readers but failed to find any significant defects. It wasn’t because the code under review was defect free because serious defects were often found during testing. At one point a team was seriously considering scraping code reviews due to low return for a huge effort. So what went wrong?

I believe there were two general problems. The first was that the areas under review were rather specialist areas involving complex design and code. In general, most of the reviewers were not familiar with either the code or the details of the design prior to the review. The second problem was that in many cases the people reviewing the code did not have sufficient levels of experience in the area under review to be considered experts in that area. On average more than 50% of the reviewers were new developers ramping up on the technology area and in some cases even ramping up on the programming language itself. The code reviews were used as a ramp-up task for these developers and while it certainly provided an opportunity for these developers to ask questions about various parts of the code, they were not in a position to be able to critically review the code.

So what can we learn from this? My key takeaways are:

  1. Ensure all reviewers are sufficiently familiar with the design and the code prior to the review. A one hour presentation on the design is usually not enough to provide this familiarization.
  2. Ensure sufficient experts are involved in the review.
  3. Using code reviews as part of a newbie’s ramp up is a good idea, but the newbies shouldn’t be relied upon to provide the critical feedback for the review. As a guideline, newbies should not compromise more than 25% of the reviewers.
  4. If any of the above are not satisfied then postpone the review until they can be satisfied.

This blog has a new home

This blog has moved from it’s old home at http://stephendoyle.blogspot.com to its own domain at http://softwareramblings.com. The move should be transparent to subscribers of the feedburner feed – at most there will be a few duplicate entries in the feed as the old posts are imported into the new setup.

So why the move? The main reason was because I found the blogger interface a bit cumbersome for posts that involved source code. I was underwhelmed by the presentation of source code that resulted from <code> and <pre> tags. It was also a pain to have to remember to properly handle the angle brackets and ampersands in the code. I decided to move from blogger to WordPress so that I could take advantage of it’s syntax highlighting plugin which is based on the syntaxhighlighter Google Code project by Alex Gorbatchev. Check out the following snippet – cool or what?

#include

using namespace std;

int main()
{
    cout << "Hello World" << endl;     return 0; }[/sourcecode] Rather than moving the blog from stephendoyle.blogspot.com to something like stephendoyle.wordpress.com or softwareramblings.wordpress.com I decided to move the blog to it's own domain. Since the blog was entitled "Software Ramblings" and the domain of the same name was miraculously free, my mind was made up.

5 Generic Debugging Tips

These are some practical tips to help maximise your chances of success when debugging software problems – especially those hard to isolate problems. Most of these tips will seem like common sense but I am constantly amazed at how often they are overlooked! I’ve deliberately tried to keep these at a generic level and not tie them to a specific language so that they may be of relevance to most people.

  1. Think: If you can quickly build and test your application there is often the temptation to keep trying things until the bug goes away. It is easy to get caught up in a cycle of “make a change – build – test (fail) – make another change …” without really stopping to catch your breath. The danger with this approach is that when the bug goes away it is sometimes difficult to tell if the bug is fixed or if you have just solved a symptom. Although frustrating, it is sometimes a learning experience to have to debug an application or system that takes a long time to rebuild and test. This forces you to take a step back and to think carefully about what you can glean from your current data and what the next step should be. This will help you to better understand the nature of the problem and in doing so it will help to find the root cause more quickly.
  2. Baby Steps: This is a really obvious but often overlooked practice. When debugging always take baby steps and only change one thing between tests. It is often very tempting to make a couple of changes or to skip a step, but this invariably results in an inconclusive test result and you will end up backtracking to figure out which change caused a change in behavior.
  3. Simplify: Try and reproduce a suspect piece of code in a stand-alone environment. For example if a particular algorithm is miss-behaving, try and replicate this behavior in a simple application that only contains the algorithm and may be easier to control and debug. Similarly, if debugging issues in a kernel, it is often possible to replicate the code at a user level which is a much more friendly debug environment.
  4. Tools: Become intimately familiar with your tool chain. Take time to learn the power and quirks of your unit test environment, debugger, memory leak checker, compiler, etc. If you develop on multiple platforms and use different tool chains on those platforms then learn them all!
  5. Challenge Assumptions: Assumptions can be dangerous. How many times have you assumed that a particular piece of code is working only to find you many hours later that it contained the bug that was the root cause of the issue you were debugging or that it behaved subtly different than what you expected? In general it is good practice to assume nothing and to challenge all assumptions that you find yourself making.

Always an apprentice?

With the rate at which new software languages and technologies are appearing, its a tough job for a software engineer to keep up with the latest developments and trends as well as keeping existing skills sharp. Just as you feel you master one language or technology, a new one comes along and your apprenticeship starts over in this new field.

One of the best ways that I have found to both learn new languages/skills and to keep existing skills sharp is to practice each skill as much as possible. For programming languages this means regularly writing pieces of code in each language in the toolbox. I try to use each language that I know at least once a month for languages that I have become proficient in and at least once a week for languages that I am learning.

Dave Thomas describes this concept in greater detail in his CodeKata series.

I find the use of programming puzzles and challenges as excellent ways to find concise coding exercises that don’t soak up to much time but yet provide opportunities to hone & develop skills as well as often providing opportunities to learn about new libraries, packages, algorithms etc.

Some of the best coding problem / challenge sites that I use are:
http://codekata.pragprog.com/
http://www.pythonchallenge.com/
http://www.topcoder.com/