Technical Debt is a cancer

After a discussion about Technical Debt (aka TDebt) in Thoughtworks I realized something: Technical Debt is just like cancer.

There are several similarities between the disease and this "condition" that happens quite frequently in development projects.

There is a line in medicine that says that if you live long enough you will develop some sort of cancer because organisms have intrinsic failures that will manifest themselves if enough time is given. An IT project that lives long enough will present TDebt related problems.

Cancer can develop from external factors or congenital predispositions. Tobacco, sunlight or too much tuna can disturb your organism to a point of metastasis. Less than good design/coding decisions affect the general project’s health in similar way. If the team chooses to cut corners and develop bad code intentionally TDebt will build up. As for the congenital problems that may be circumvented by using some tools. From a Java angle it can be FindBug or PMD to continuously inspect congenital problems related to the technology stack of choice.

No one wants to discover a cancer but once you find it you need to take some action or it can have terrible effects. Tackling TDebt is hard and many choose to put it off until it's too late. However if the case is a terminal one, then nothing can be done. All you can hope for is for it to live long enough to fulfill its purpose. Different types of problems require different procedures. But seems that the most popular and successful approach is: isolate and remove.

Early detection and prevention are key. The sooner the problem is identified the easier it is to heal. Regular check-ups are a Good Idea™. If a potential problem is detected it should be examined to determine whether it is benign or malign. After that one can decide the course of action. Also there's no point in being a hypochondriac, this shouldn’t consume the team’s time away, it’s preventive not obsessive.

Some people get worse after they discover a cancer because their moral gets shot. Projects slow down because of TDebt, which gets devs very de-motivated. TDebt makes things unnecessarily complex turning simple tasks into major efforts. Anything that impairs people from living normal lives have devastating effects on their will.

When do you communicate to your family and friends? To what extend TDebt should became visible to the business? I'd say that the approach depends on the severity of the case. Less seriously cases can be treated internally without making a big fuss, but if more people, money and support are going to be required to reestablish the healthy state then stakeholders need to fully participate in the process.

Bottom line, have healthy habits (TDD, simple design), prophylaxis (refactoring), use tools to monitor congenital problems (FindBug/PMD) and frequently consult your doctor for a code health check-up.

HashMap temptation

When integrating systems it is very easy to fall into HashMap temptation. “Just put some details in a Map and pass it to the service layer and it gives back what you need”. Maybe,… but there are risks.

I consider passing Maps around to services an anti-pattern because interfaces become less intentful and creates coupling between layers.

Let’s consider a hypothetical example of a farm system. Every morning old McDonald wants to name roll call all animals to ensure they are safe and sound. Farmer McDonald knows he has animals in the pond and on the pasture.

On the first pass we create a service receiving a context Map, keyed by environment type and containing animals’ names. The service returns a List with farm Animal instances.


Old McDonald knows the animals’ names, where they live and expects them to say: “present”, in their own dialect, obviously. One client program could look like the one bellow:


With this implementation it’s impossible to have a pond roll call only. But that’s not the worst thing, knowing that the service needs to be aware of different environments and keep track of them is the creepiest part. One solution is to make the service layer more intentful and have old McDonald say explicitly the animals of which environment he is roll calling:


Now instead of a Map it’s a List that gets passed around, the service interface expresses *exactly* what’s to be done and code can be re-used. The downside is that an extra call needs to be made from the client. In this particular example it’s not a major hassle because it requires no extra information about the livestock:


Bottom line: avoid passing Maps around and put more effort into defining highly intentful interfaces. As a rule of thumb service layers are dumb, hence they don’t have to decide which method/logic is to be invoked based on states/keys of other objects. Service layers should only know about domain objects and which methods to invoke. Utilising HashMaps in similar ways to this example violates the Tell, Don’t Ask principle. This design forces service layers to understand what’s inside of Maps to decide which method to invoke and the client layer in its turn, must create Maps with all things the service needs in order to process the call, which smells like coupling.

Why Agile adoption fails?

Why some seem to succeed to produce quality software using lean methodologies and others don’t?

For starters there’s a giant lacuna between acknowledging that Agile is useful, fast, high quality, etc and been truly Agile. For Agile to work one must believe and *act* according to her beliefs!

I’ve noticed that teams fail to create software using lean approaches mainly because of their preconceived mindsets towards an apparently less predictable, although more precise, model.

Unfortunately no one can build software as if it was a house. In software construction you *can* have your own voltage, no gravity and wine instead of water running in the pipes. Therefore managers need to accept that will this level of freedom there is a proportional level of uncertainty that cannot be cleared upfront, regardless of the effort put into planning and estimating.

A forcefully preset release date on a spreadsheet means very little to software development. In the outset it only expresses a desire. Functionalities need to be sized, time must be derived for sizing and then, only then, a date estimated. Even after that, realize that that’s all it is, an estimated date.

Trying to re-estimate stories because they’re taking longer than expected is another contributing factor for disaster. Let velocity do the work of defining progress, story size cannot change speed.

Teams have a hard time accepting that there’s only so much that can be done to go faster in order to meet a deadline. Reducing complexity is one option, re-prioritizing and re-scoping too. Resizing your team can also have positive impact, but overtime not. By working long hours quality will inevitably be compromised and reducing quality is not an option in Agile.

Some say Agile is too informal, and even that it makes the office space look ugly with all of those cards on the walls! Software development has always been messy and Agile only exposes it so people can act upon it. Agile is formal, as formal as the organization using it wants it to be. And yes, there is planning too. I don’t have figures but with Agile there’s more planning effort than waterfall. The difference is that the planning is diluted in everyday work such as stand ups, technical huddles, iteration planning meetings, kick-offs, planning poker, etc..

Agile is highly dependent on talented developers. Roles such as team managers and architects have less or no impact in Agile teams. Traditional waterfall micro-management kills creativity and architects tend to do a lot of upfront design and create unacceptable resistance to design change over the course of stories. The devs need to be different too. Agile devs must be design savvy, have creativity and initiative.

Lastly, if all of this seems too daunting and you don’t want to take the risk of figuring out Agile heuristically, go and hire an Agile consultancy. Get people with experience to help you out in a few projects and inculcate the principles. This way the organization gains confidence, establishes a faster return of investment and creates enough in-house Agile champions to sustain the changing momentum throughout the company.

Swing Acceptance Testing with JFC

After many years developing for the Web I found myself having to do some Swing code and *obviously* had to do some testing for it. I went with JFC for the functional tests and here’s a little cookbook.

The test class extends from JFCTestCase

public class CurCalcTest extends JFCTestCase

The main JFC class to take heed is JFCTestHelper. Probably a Good Idea is to set it in the setup method:

protected void setUp() throws Exception {
super.setUp();
setHelper(new JFCTestHelper());
}

After that, my approach was to get the container under test/investigation:

JPanel curCalcPanel = (JPanel) frame.getContentPane().getComponent(0);

And then obtain each one of the elements I want to manipulate, such as comboBoxes, textFields, and etc.:

NamedComponentFinder amountFinder = new NamedComponentFinder(
JTextField.class, "jTextFieldAmount");
amountField = (JTextField) amountFinder.find(curCalcPanel, 0);

NamedComponentFinder countryComboFinder = new
NamedComponentFinder(JComboBox.class, "jComboBoxCountry1");
countryCombo = (JComboBox) countryComboFinder.find(curCalcPanel, 0);

NamedComponentFinder messagesFinder = new
NamedComponentFinder(JLabel.class, "messages");
messagesLabel = (JLabel) messagesFinder.find(curCalcPanel, 0);

Now I can insert my keystrokes/clicks using the helper:

getHelper().sendString(new StringEventData(this, amountField, "1"));
getHelper().enterClickAndLeave(new JComboBoxMouseEventData(this,
countryCombo, 1, 1));
getHelper().enterClickAndLeave(new MouseEventData(this, convertButton));

And assert my expectation(s):

Assert.assertEquals("0.97478773", messagesLabel.getText());

JFC is another fantastic tool by ThoughtWorks. It is a very useful framework but not friendly all of the time. For instance, if you forget to identify the elements properly via the setName() method, and try to use NamedComponentFinder no error is given, the thread just sits in there without complaining about anything.

New Year's resolution

Is it too late for a New Year resolution?? Probably not. Then here it goes. I'll learn Ruby in 2008. These are the reasons:

• I'm getting a bit tired of Java and Ruby seems to be the next logical move
• Now that JRuby is out, chances are that the Ruby will really make to corporate environments
• It has a strong community
• ThoughtWorks is supporting it wholeheartedly, BTW welcome to the team Ola Bini!
• Ruby is very flexible allowing some very fancy things, still lacks a bit of control thou
  
• Finally and most important, Ruby is Agile

So Ruby, here I go!

Notes on Selenium-grid

The fact that web tests, let’s call them here Selenium tests, are time consuming seems to hurt a lot of agile projects. Long build times are hard to live with.

In the early stages of the project there are a few Selenium tests, usually not very well structured and testing on a per story basis. As more stories are played more Selenium tests are created, usually with a certain degree of repetition, causing the build time to increase. That’s when it starts to sting.

In my last client the build time started off in the thirty minutes range and after four iterations it was costing the team a horrendous seventy minutes. Selenium tests alone were responsible for almost fifty minutes.

There were a few ways to reduce time. The first obvious one is ditching some tests. If you are playing by the book and doing TDD, really caring about your unit and integration tests, and have a decent test coverage, giving up a couple of redundant tests might not be so bad. Another option is to rewrite tests to eliminate redundancy and test using a scenario approach instead of the story one. Scenario based tests are a Good Idea™ but time consuming and hard to guarantee that everything that was tested before is still been tested using the conceived scenarios. A third option was to parallelize the tests and Selenium-grid allows doing just that.

Lawrence Song and I had Selenium-grid up and running with no traumas. It’s quite straight forward to install it but not all is roses. Web tests that run on a farm need to be more carefully constructed that those running on a single threaded environment.

Tests must run atomically and the rule of one behaviour per test should be strictly observed. For instance, if tests perform database operations on the same data they can have problems when running concurrently.

Don’t leave Selenium objects hanging in there. Selenium objects that aren’t closed properly causes the hub to stop sending RC commands until you have only one Selenium server available. So the Selenium lifecycle needs to be managed more tightly.

You need to think about the test list. In order to use the grid’s capabilities tests need to be run in a multi-thread fashion. TestNG does a better job that Parallel JUnit. Some Gargoyle like facility would help make it more manageable. As a last resource, which seems to be the sensible one especially when needing to avoid two or more tests running concurrently, is to use parallel ant task and pick your tests using a pattern.

Web tests are generally CPU intense and it’s a Good Idea™ to think about how many Selenium servers should be running on each box, especially if they are also running other apps like your container for example. We saw flaky results when boxes where running close to 100% CPU all the time.

Bottom line is that Selenium-grid is a great help to solve a common problem, in our client the build time dropped significantly and now takes around twenty five minutes with a few hours of effort. Yay! And we can always add more boxes. Selenium-grid has a very positive side effect which is to reinforce better test design.

Nevertheless I’d really enjoy if Selenium-grid could provide the community with the ability to utilise idle computers to run tests and a friendlier interface to manage the hub.