Does Java 5 Generics give you better code?

I think I start to understand at least part of the Java 5 Generics now. I'm really not convinced that the benefit is worth the syntax and reduced readability. I'll try to explain why...

I must confess that I see the point for the simple cases. In Java 1.4 the code for finding payments over a certain limit in a list of Payments would look like this:

    public List findAllPaymentsOverLimit(List payments,
                                         int limit)
    {
        List overLimitPayments = new ArrayList();
        for(Iterator each = payments.iterator(); each.hasNext(); ) {
           Payment payment = (Payment) each.next();
           if (payment.amount > limit)
                overLimitPayments.add(payment);
        }
        return overLimitPayments;
    }

And the bad thing is that we've all gotten used to that horrible casting syntax. With Java 5 we could change this function to something like this:

    public List<Payment> findAllPaymentsOverLimit(
                            List<Payment> payments,
                            int limit)
    {
        List<Payment> overLimitPayments 
                = new ArrayList<Payment>();
        for (Payment payment : payments) {
           if (payment.amount > limit)
                overLimitPayments.add(payment);
        }
        return overLimitPayments;
    }

And that's much better, right?

My problem is with the more advanced uses of this new tool. I'll admit that I haven't fully understood this, but if you want to create code that must accept a typed collection or even worse a map, you'll have to learn the more advanced syntax where

List<? extends Class>

or

List<? super Class>

This is where it gets hairy, and where I start to think it might just not be worth it. On the other hand, I could leave that problem to the framework makers.

What have I been doing?

As can be seen from the blog it has been quite a while since my last post. I could always say that I've been busy (which is true), or that I haven't had that much to say, but that would just be excuses.

In these past months I've been busy working on getting more agile practices accepted at work. Ever since our sessions with Eric Evens and his guys, I've been convinced that we must change the way we develop software. The full implications of this did not register until I experienced the talk by Robert C. Martin at JAOO last autumn. Since then I've been devouring everything I could find on agile development, and it has been my main focus. I've also tried to speed up the process of change in my company, but as all big corporations it is slow.

I have, however, also found the time for some coding. God forbid I should forget how to do that. At work I've been busy doing some integration work, so it has been a lot of Java and XML. At home I've found just a little time to play a little bit with Ruby and Java 5 (more on that later).

Well, lets hope the next post doesn't delay as long as this one. See you.

Great tutorial / story on development

I came across "The Craftsman" article series by Robert C. Martin (Uncle Bob), when I was looking for articles on agile development on the ObjectMentor web site.

This is a series written in monthly installments, with the latest installment (#40) published in the October issue of this year. The series is published in the magazine Software Developer and describes the story of a young developer called Alphonse and his transition from apprentice to craftsman programmer. He goes through a set of learning experiences in each of the articles that improves his skills. It is written in a scifi setting, but this shouldn't put anyone off as it only adds to the narrative of the series. It is extremely well written and (I find) entertaining.

Articles 1 through 31 can be found on the ObjectMentor web site, and the remaining can be found in the Software Developer magazine for those that have a subscription. (Installment 31 was published in January 2005.)

You should definitely check out this article series at is will teach you quite a bit, no matter what level of programmer you are.

JAOO wrapup

I want to summarize my impressions and thoughts from JAOO this year. I've barely had the time to adjust to the ordinary life and spent the weekend not thinking too much about these things.

Besides going to a lot if interesting talks (not to mention the tutorials), and getting a lot of new ideas from that, the main thing has been that I've met a great many interesting people. This is what is good about JAOO, the speakers are very interested in talking to the other participants, and the schedule facilitates this with the parties and other activities.

I already mentioned meeting Robert C. Martin (Uncle Bob), Dave Thomas and Kevlin Henney. Dave also introduced me to Brian Berry who was his co-godfather of Eclipse. I also had a brief chance to speak with Martin Fowler, but he's more like a rock star these days. Steve Freeman and Nat Pryce of jMock and NMock were also two people I had the pleasure to meet. I especially got to spend quite some time with Steve, and we had quite a few discussions on agile development and computer science fundamentals. Another interesting guy was Adam Bien, an independant consultant from Germany. He has some down to earth views on how to develop J2EE applications, and in his talk called Are Java Fat Clients Dead?, he discusses alternatives to the thin client which are very similar to what we're trying to do with TOPS.

Of course I also talked to a lot of other guys too, and all of the different people have something to add to the conversation. This is what makes JAOO such a good conference, this and the extremely good program and set of speakers.

I hope to be back next year.

Model doesn't mean UML

Everywhere in the software industry today we encounter the notion of model. Certainly the schedule at JAOO was filled with references to it; it even had a separate track on Model Driven Architecture.

I've always been very sceptical to the ideal that of executable UML and the whole "software through pictures" crowd, because I really think it isn't that simple. The thing is that this has given me a very negative association to anything that sounds like "Model Driven Something", because model has to be a set of UML diagrams, right?

Wrong!

When most well-established speakers and software professionals like Martin Fowler, Eric Evans, Dave Thomas, Andy Hunt, Kevlin Henney and the like, talk about models, they are talking about the mental concepts that we build during our investigation of the problem domain.

The main point is that a model is an abstraction of the problem area or domain for which we build the software. Now this can sometimes be represented by UML diagrams, but seldom completely and never in an absolute way.

So remember this: UML is a communication tool, the model is ultimately in your head (and hopefully in the head of your teammates...).

And whenever someone speaks about Model Driven Design or Model Driven Development, take a closer look and don't dismiss it even if you are (rightly) sceptical of Model Driven Architecture.

Domain Specific Languages

This is the last day of the conference part of JAOO, tomorrow is a full day tutorial.

Today I've been mostly listening to talks about Domain Specific Languages. This is a notion that kind of is the next step from Domain Driven Development, but also requires something more to quite get there. I think the stuff that we're trying to do with DDD is the closest thing we can get to an "internal" DSL in Java (to use Martin Fowlers classification), without actually making a full-blown DSL.

Now, what is a full-blown DSL? A DSL is a specific syntax, typically for a small, well defined domain, aimed at making the software more expressive. For a more through description read Martin Fowlers article on Language Workbenches. Usually, a DSL, whether internal or external, will imply some kind of transformation into executable code, so in that sense what we do directly in Java is not exactly DSL by strict definition. Even so it is possible to make the argument that it is.

Language Oriented Programming is a term used on the style of programming promoted by the Language Workbenches and similar technologies. LOP uses multiple DSLs to create full programs. This is analogous to Eric Evans notion of different domains within an application.

Another topic that propped up towards the end of the DSL track as a side effect is our use of the word Model and what this means to different people. And that will be the subject of my next post.

The fundamentals

If there is one recurring theme from all the conversations I've had with people at this conference, it is that it is vital to understand the fundamentals of a problem before we throw ourself into the technology discussions.

How can you choose one technology over another if you don't know the fundamental issues that should guide your decision?

Anyone interessted in the fundamentals of programming languages and language design should check out Abelsons and Sussman's lecture series Structure and Interpretation of Computer Programs.

Ivar Jacobson on Software Agents

Todays keynote by Ivar Jacobson was entitled Beyond Agile: Smart.

Ivar, of course, promotes the unified process, and especially the practice of making knowledge explicit. This is, he says what makes his process better than agile methods. Dave Thomas commented to me that it is easy to arrive at such conclusions when you don't have to do it in practice.

Anyway, the main thing that got my attention (of course) was what Ivar calles the next thing. His idea is that all the information gathered and formalized by (R)UP is a good thing as long as it stays out of our way and is available when we need it. This is an interessting proposition, albeit a bit far fetched. Ivar is actually talking about (and have started developing) intelligent agents to be responsible for this information distribution. He is using software agents as virual pairs to suggest options for you and learn from what you are doing. How about that?

He envisions a set of tools that will let agents play different roles, such as

• virtual pair project manager
• virtual pair architect
• virtual pair designer
• virtual pair developer and so on...

If you are more interessted in his work on this have a look at the website of his company JacZone.

As far as I can tell, this is all foilware, but it is interessting that these kinds of things pop up.

What are the key things with patterns?

A pattern is a description of a solution to a recurring problem in a given context. The format is not important, but it should include the following sections:

• Name
• Context
• Problem description
• Forces and tradeoffs
• Solution

This gives us the possibility to name recurring solutions and communicate the different aspects of them, which I think is a good thing that we should do more.

This is also a sensible (and perhaps the only usable) way of organizing any technical writing. If you don't cover at least the three last bullets in that order in your writing it will be very difficult to understand what you are trying to say. Providing a context helps and giving it a name adds to the communicativeity of the concept.

I think we should start looking into better ways of communicating across projects and organizational boundaries. Maybe using a common repository of short presise descriptions of our learning. Adding to this repository should be part of the project end life cycle.

Patterns in Software

I spent the rest of the day yesterday listening to different talks about patterns. Most of them talked about what has happend since the GoF book (Design Patterns, 1995), and what they think will be the future of patterns.

The track started with a talk from Ralph Johnson (one of the Gang of Four). He gave us his personal take on the history of patterns up to today. After him, Dragos Marinescu from ThoughtWorks talked about the new book Pattern-Languages of Program Development volume 5, where a new collection of Pattern Languages will be published later this year.

Markus Völter then talked about Patterns in Modern Software, Klaus Marquardt talked about how to improve the usability of patterns, and finally Kevlin Henney and Frank Buschmann talked about their take on the past, present and future of patterns.

The main trend in the pattern community is the move of focus from single patterns to pattern languages (interconnected patterns). Kevlin and Frank will publish a new book in Frank's series Pattern-Oriented Software Architecture, volume 5, later this year. In this book they will connect patterns from the GoF book, patterns from Fowlers EA Patterns book and other new pattern languages.

Those of you more interessted in patterns can check out the Hillside Group. They are the group that run the PLoP conferences.

To me patterns is a little bit off in my peripheral vision at the moment, especially when I'm still so geared up from the agile stuff on Sunday. I do however think that there are some ideas in the patterns movement that we should embrace in Statoil as well and start to use for better communication both internally and externally.

Opening keynote: Simon Phipps on Open Source

Simon Phipps is the Chief Officer for Open Source at Sun Microsystems, which makes him a guy to listen to and with power to change stuff in the open source space.

He talked about what he calls the "Participation Age" to describe the era we're in now. In this era open source is the model for building software. But open source is not enough. Standards must be in place to enable the companies to add value to the open source commons. Many business models are possible around open source.

Also interoperability is required for the users to be able to achieve the freedom that open source is all about for them.

A very interessting talk, that for me at least opened some new perspectives on the open source space.

First day of the JAOO conference starting

Yesterday was the pre-conference tutorial. Today the conference starts in earnest. Multiple tracks with many different interessting speakers. Today I will mostly follow the Patterns track, so we'll see what kind of ideas my head is filled with tonight...

Fundation for adopting agile methods

Dave Thomas made this point repeatedly: If you don't have an infrastructure for doing continuous integration and automated tests then you can forget about transitioning to agile methods.

This is good news for us, since we're there and have been for some time. I personally think it has been the right way to go, introducing the foundation and gradually maturing towards a better understanding. I think understanding must be there internally before we can bring in someone from the outside to show us the way. I feel that we are ready now to move further with agile methods and start to challenge our document driven ways.

So, what am I advocating here? The death of PROMIT, certainly not, but it needs to be reworked seriously. PROMIT is derived from PROMIS. Why do we have PROMIS? To manage huge, costly oil field development projects with huge amount of infrastrucure costs. The cost distribution of these projects are low in the specification phase and extremely high once the infratructure starts to be implemented. On these projects it makes sense to have the decision gates to control the costs before implementation, but this doesn't translate into software development.

Software development cost distribution is no different in the implementation phase than in any other time of the project. The cost depends on how many people and how long it runs. The best way to control this cost is to set a date (fix the time), set the teams (fix the people) and let the scope vary within reason.

Scaling Agile Development

Robert C. Martin (aka Uncle Bob) is a carismatic speaker. He is clear and consise in his descriptions and entertaining in his presentations. I've obtained an electronic version of his talk and can provide this to those who are interessted. Just send me an email.

The talk was split into two parts. The first half of the day uncle Bob spent going thorough the different principles and practices of agile development. In the second half, Dave Thomas talked about the management view. If you don't know Dave Thomas (not the pragmatic programmer), he is the one who managed the Eclipse project starting with IBM Visual Age, so you know he knows a thing or two about scaling agile development.

Bob started off by talking about measurements, under the banner in order to manage we need data. I'm a little embarrased to say that this surprised me, because that means that I have bought into the misconceptions that is going around about agile development that it lacks dicipline. Bob uses story points (or function points) to measure what a team can perform in a week, and then accompany this with a chart showing the remaining function points. With this you can estimate your delivery date. We'll come back to how he comes up with these function points.

He then went on to talk about the fact that in most projects the delivery date is what is fixed earliest, and the requirements never are and never should be.

The problem with the waterfall and other document centric approaches (apart its history) is that it lets you spend a lot of time in the start of the project without producing tangible results. Yes, a requirments document is tangible and sometimes even helpful, but if you spend the first seven months of a project doing requirements and only have three left for implementation it will fail no matter how good the implementation team is. The problem is really that there is no way to determine if a requirements document is done, or when a design document is done. It is however no problem to define when an implementation is done, and hence the problems in the end. Also the practice of putting in the quality at the end is doomed since you rearly have time to do a good job when the implementation fase slips.

He then proceeded to talk about agile development and the different practices or principles in it. Now I don't want this blog entry to turn into a tutorial on agile development, so please follow the link above for an overview of agile development ala Robert Martin.

Finally, Bob, did a programming excercise to show test driven development. He set out to write a bowling scoring game (from his book). The point of the excercise was to show that you will arrive at a different and simpler solution that with up-front design. First we did a collaborative design and arrived at classes like Game, Frame, Roll, and Tenth Frame (to handle the special 3 roll possibility of the last frame). Bob then starts from scratch with TDD approach. Simple creation test for Game, first test case all rolls scoring zero (gutter game), nest test all rolls scoring 1, next test one spare, next test one strike and final test all strikes. This produces one class Game with two methods roll(), which registeres the pins taken down in each roll and score() which very accurately captured a clear bowling scoring algorithm. It was interessting to watch. I actually didn't realize he had finished.

Then we had Dave Thomas come in to tell us about scaling and how to fit in with management. Basically his statement was, give them whatever you have to to keep them happy, generate requirements documents from your source or Fitness cases, generate the plan, fake it if you have to to keep them off your back. Internally teams scale into teams of teams with aggregation of the practices. This comes with the price of longer time intervals between control at the top level, so you need to be able to trust the people running the teams and that they can perform. The team structure can be evolved starting with a few people or architected from the start. Stories at the top level are broken down and divided amongst the team according to the structure. This is not so relevant to us, as we don't have to scale that high, but some of it could be useful.

To sum up, this sessions answered a lot of questions for me personally, and I'm going to take with me these new ideas when we look at how we can take a step furter and improve the way we run IT projects.

Agile Development according to Uncle Bob

Uncle Bob preaches pure agile development XP style, so those of you who know XP will recognize the practices:

Team level practices:

• Planning game
• Acceptance tests
• Small releases
• Whole Teams

Development practices:

• Simple design
• Test Driven Development
• Refactoring
• Pairing

In addition there are four intermediary practices:

• Continuous integration
• Collective ownership
• Sustainable pace
• Metaphor

An agile project starts with a meeting where the overall project parameters are defined. This most often starts with setting the delivery date, but the date is not always there from the start. In this meeting the overall functionality of the project will be discussed and start to be formalized as stories. Each story represent a function that has business value and that can be estimated and tested.

Bob use a deck of index cards, one for each story. Cards will be added and removed as the project moves on. First each story gets an estimate. The estimates are arbitrary numbers sigifying the relative complexity of a story. These numbers constitute the function points or story points used to plan and measure project progress.

An initial plan is then made laying the stories out in weekly iterations. The initial number of story points in an iteration is a guess and depends on team experience amongst other tings. If a team starts out trying to do 50 points in a week and on wednesday only 22 are done, the customer is promptly told that he can expect only 44 points to be finished by the iteration and so on. When the iteration ends, the final number of points done is counted, put in a graph to show progress and a new plan is created based on this new knowledge. The customer will select the stories that are most important at the beginning of each iterations, and stories can be added or taken away. The project is done when the customer cannot find any new stories that are worth implementing or the delivery date, whichever comes first.

They also do an initial two weeks of analysis into the problem domain. This is consistant with what Eric Evans recommends. This is when the requirement harvesting begins, and this phase could even produce an initial object model. The important thing is that this model will be discarded when the implementation starts.

So when is a story done? This is down to the acceptance tests. When a story passes all its acceptance tests its done. Acceptance tests are automated using Fitnesse or another framework, and run automatically. Ideally they would be written by the customer in Fitness, but most often it will be Business Analysts writing the "happy path" and QA people and testers writing the exceptions and boundary conditions.

How do they handle project slippage? When you measure function points done per week and function points remaining, after around five weeks the number of points done per week in average will be acurate enough for predicting a delivery date. This is still pretty early in the project, about seven weeks into the schedule we can tell the customer that the end date is unrealistic. This is unheardof in waterfall projects, where the norm is to have this information in the end at best when the implementation phase is reestimated. Of course the customers will still complain about slippage, but when you discover this early it's possible to do something about it. Most often the date will stay fixed, adding more people can be an option this early, and it is never a good idea to compromise on the quality. What is left is the scope, and the story deck needs to be reshuffled and reprioritized.

User demonstrations after each weekly iterations and fully functional, ready-for-production releases every six weeks. This means that if the project is cancelled after one of these releases, it will provide value to the customer. Also it allows the customer to terminate the project before the end delivery date if he cannot find any more stories worth implementing.

Whole teams refer to the practice of co-locating the whole team in one room. This includes business analysts and QA people, project manager and testers.

The lower level practices are more known to us and we have implemented most of them in the course of our work with Eric Evans on Domain Driven Design. To me these team level practices are what we need to complete the picture and help us avoid project failures in the future.

Why do we do the waterfall?

The reason that the waterfall and its spawn document centric, specify then build processes has become popular has mainly to do with a paper by Dr. Winston Royce in 1970 where he began with an ideal proposition saying "wouldn't it be great if we could do this", and basically went on to describe the waterfall model. This is the most cited paper from people talking about the waterfall or similar processes.

What they mostly fail to mention is that Royce went on to saying that he didn't think that this was possible and that he instead recommended an iterative approach. Because what he found is that there will be feedbacks from development into design and from design into spec once you start working with the implementation.

The reason the waterfall turned into such an international hit is that the US Department of Defense asked a tech writer to compile the standard known as 2167, and when that was approved, countries and companies worldwide said "if the DoD is doing it it must be good."

Since then both Royce and the tech writer has, according to Bob Martin, spent their entire time trying to convince people that this was not their intention and appologizing for their errors. Even Royces son has been plagued by this, said uncle Bob.

Robert C. Martin on Agile Development

I'm really happy that I signed up for this tutorial.

Bob Martin is describing his take on agile development, and in addition to being very inspirational, it really provides some missing pieces of the puzzle for me.

I will write a thorough summary of this later today, but for now I'll just say that this means a lot of changes to how we should run our projects. It effectively means the death of the requirements face as we know it. If we don't do this we will not be able to deliver our projects properly...

I'll leave you with that for now, and hopefully I'll be able to justify this statement in my post this evening.

JAOO 2005 conference has started

I've just finished my breakfast, and am waiting for the start of the pre-conference tutorials. Registration should open any minute, and I've spent the time getting up and running on the conference wireless zone.

Today's session should be good, it's with Robert C. Martin and Dave Thomas (not the pragmatic programmer, the other one...) on Scaling Agile Methods. Now if you follow this blog you can expect me to rant a lot about agile methods in the next couple of posts, so if you don't know anything about it please have a look at the Agile Manifesto. I really think everybody should know at least what's on there ;-)

If you look at that site and see the names of the people involved, you'll find Robert C. Martin in addition to other well known names such as Martin Fowler, Kent Beck, Alistair Cockburn, Ward Cunningham, Dave Thomas (the pragmatic one, actually) and Andrew Hunt.

JAOO 2005

It's time for JAOO 2005. There is a lot of interesting talks and sessions, and I want to experience as many of them as possible.

I will do my best to report on the different sessions that I go to and any interesting events that I vitness.

I will attend a tutorial with Robert Martin (aka. Uncle Bob) on agile methods, and definetly the track on Domain Specific Languages on Wednesday. I'm also signed up for a tutorial on JDO by the creator Craig Russel, which should be good. Other than that I'll play it by ear, and pick what catches my attention and time permits.

Interesting individuals @ JavaZone

One cool thing about going to a conference as a speaker (besides getting the chance to speak in front of people ;-) is that you get to meet a lot of interesting speakers. This year I've met both old friends of mine from school like Knut Magne Risvik, and more famous guys like Kevlin Henney and Tom Gilb.

Also I would like to say that the level of people speaking at JavaZone is generally very high, and for the record I'm now talking about the other guys, not me. The local guys have a lot of insight into the different subjects that they contribute.

The thing about people like Tom Gilb, Kevlin Henney, Eric Evans, Bruce Tate, Rod Johnson and our own Einar Landre is that they have a more fundamental understanding of the problems we face in software development. This allows them to think beyond the mere use of tools or methods, and see the "big picture". Now I'm decidedly nowhere near their caliber, but I find listening to their discussions both inspirational and educational.

Einar and I had a brief discussion with Tom Gilb about his talk on Agile methods and what is missing from them, namely metrics and improvment processes. His talk was unfortunately parallel with mine so I couldn't participate, but I got his slides, as well as a copy of his new book titled "Competitive Engineering" and I'm looking forward to reading them both.

Another thing that strikes me after having met these people is that they are generally extremly pleasant and forthcoming people. I'm just sorry I had to come home tonight and couldn't stay for the second day. Well life's full of choices and I really need to spend a little time with my family now. So JavaZone 2005 is over for me, we'll just have to wait for the videotaped sessions and presentations to become available.