The State of Agile Development

A few days ago I came across interesting news (I've been digesting it a bit), following up on my first post about Agile Software Development and the Enterprise World (SCRUM + CMMI).

A survey conducted by VersionOne entitled "The State of Agile Development" (PDF) reports on various indicators regarding Agile Development and its impact on organizations, both positive and negative outcomes. The survey reached nearly 1700 individuals across 71 countries. Vishal Sharma's blog is the source where I found this, he has posted some highlights about the survey.

As my perception is that many companies still don't benefit from Agile development and management, I was curious to see who within the organizations has driven and supported most the transition (corresponding to the question "What role most closely identifies the initial champion of Agile development in your organization"):

  • Team Lead / Dev Management: 22%
  • VP / Director of Development: 19%
  • C-Level Executive: 18%
  • Project Manager: 15%
Maybe because upper level management perceives too many risks or is misinformed and developers simply don't have the necessary influence, it seems that middle management plays a key role when it comes to adopting Agile practices.

But ultimately the executives have to be convinced it's worth it to adopt Agile methodologies, so here are some figures for them, regarding what has improved or significantly improved since the adoption of Agile practices by the surveyed companies:
  • Enhanced ability to manage changing priorities: 92%
  • Improved project visibility: 83%
  • Improved team morale: 82%
  • Increased productivity: 80%
  • Enhanced software quality: 77%
  • Reduced project risk: 75%
[This does not mean that productivity increased 80%, it means that 80% of cases productivity increased or increased significantly.]

Incidentally, I read today an article titled "Welcome to the First World, Now Your Labor Rates are Too High" by Michael Hugos. The vision expressed is part of what attracts me so much into Agile development and management practices: more and more, timing is key to deliver successfully, as reducing waste is important to keep costs not too high. We're not in the days when delaying a project or product by months is an option anymore (except maybe for very big companies, but it costs them!).

Companies must empower teams and individuals if they plan to stay competitive. So, what are you waiting for?


Extending the Timetable Combinator

I received quite some feedback on the Timetable Combinator, and the number of accesses was quite motivating:

  • Saturday 15:00 - 0 hits
  • Sunday 16:00 - ~1200 hits
  • Monday 10:45 - ~2400 hits

So, I took some time yesterday to implement stuff.

Firstly, I listened to a request I received a few times: the possibility to toggle theoretical classes. It was quite easy, just inserting a parameter in a few methods.

The second improvement has more to do with jQuery and is performance-related. I noticed that when having too many URLs to load, the browser would eat up too much processing power and sometimes the server would refuse to serve some pages. So, I thought about implementing some kind of AJAX request queueing.

Then, I thought again and googled quickly for some implementation of AJAX request queueing for jQuery. It was very easy to find and worked like a charm. However, now the resources were being wasted, because one request at a time would take to long to complete if the URL list was too long. It took me some minutes to implement support for multiple queues, just check the source code if you're interested.


Experimenting with jQuery

Today I concluded a little exploration journey through jQuery.

I already had some experience with Prototype (and Script.aculo.us, building the website of the Portuguese Museum of Ancient Art), so I had some expectations regarding jQuery. I have to admit that jQuery blew them away quite easily.

jQuery is more intuitive than Prototype in many aspects, allows developers to do more with less coding and already contains many effects that can be found in Script.aculo.us. The documentation is very complete (including useful examples) and, of course, supports plugins and provides AJAX to your applications.

The main advantage is precisely its "querying" abilities: it's very easy to find multiple elements on the DOM and manipulate them in various ways (including attributes and CSS). jQuery supports CSS expressions, XPath expressions and even some custom expressions (for example, to select all even nodes that match some expression).

To experiment with jQuery, I picked up something I did some time ago and turned it into what is now known as the Timetable Combinator :)

I'm sorry, I opened it with IE, Opera and Safari for Windows but I think it only works in Firefox. I think it wouldn't be too hard to cross-browserize it, but I don't have the time. If you want to take it to the next level, drop me a line.


Agile or Waterfall?

At school / university, they mostly teach and (we) practise software development by the book. I mean, requirements before design, before architecture, before development, before testing, ... Waterfall. Any project that requires documentation values that phased way of building software.

Management is oriented pretty much by that same principles. Documentation and well-defined process. After all, this same style of management has always served the various engineering disciplines with proved results, so it must be right, right?.

When companies are competing in the enterprise market, they abide by these rules. It shows how reliable and professional they are. There are even the CMMI and other kinds of enterprise certifications, which seem to push the company in the more-process-and-documentation direction, which in turn makes them more enterprise-ish.

But there's the flipside of the coin

In the past few months, I've been reading pretty much everything I could find about agile methodologies. Heck, I've even used agile management practises and an agile development process in the last project (namely, Feature Driven Development, FDD). However, as it was pretty much self-instructed, we always felt like there was something not quite right there (nevertheless, the project was very successful from the client's point of view and development run pretty smoothly).

Looking for defined and more complete agile methodologies, I've been reading about SCRUM lately. I've read soft-evangelism and user experiences, mostly. The various articles, testimonies and other resources on the Web were a very good source of information, of course, but it was too much to process in such a short time. I'll be digesting it for the next months. Maybe I'll have a chance to try it in a class I'm taking this semester, called Agile Software Development Methodologies.

Summarizing, SCRUM advocates short development iterations, nearly constant user and/or customer interaction, daily team synchronization (yes, management can be present), among other practises. It doesn't exclude things like Extreme Programming (XP) or other development practises, but upwards (in the management layers) some action must be taken to accommodate SCRUM.

And that brings me back to the initial subject. Management likes status reports, documentation and organizations like processes. By this perspective, SCRUM probably brings additional risks to a project. And many times, it's the case that a proposition must be made with a closed budget, for which the agile method might not seem so suitable. So, it's not easy to convince management. Some of the articles I've read about SCRUM mention this transition difficulty, and recommend starting by the small projects, not upfront disrupting the existing practises (they wouldn't let you, anyway).

Adopting SCRUM would be a whole other topic, and besides that, I don't think I could provide meaningful knowledge about it, as I've never been there doing that (I actually met someone at BarCamp Portugal who does it since February -- Ricardo Mestre, we didn't talk much but he promised that next year he would make a talk or workshop about that).

Why can't they co-exist?

However passionate or reluctant opinions I read, I was never completely convinced that one approach or the other was completely right or completely wrong. I believe there are cases where Waterfall might be the best option, but I was mostly convinced by Agile approaches.

I was always someone who tries to bring different perspectives together, especially when their representatives are passionate about it, because it means that there must be some value in their ideas. So, today I was thrilled when I read a post by Richard Banks stating that SCRUM and CMMI not only can work together, but yield extraordinary results! The organization in question, Systematic Software Engineering, is actually CMMI level 5 certified! And not only are they using SCRUM but also Lean. (The original report on the subject is from Jeff Sutherland, co-creator of SCRUM.)

I believe this is not a coincidence or simply bright people being in the right place and time. CMMI 5 indicates an organization with continuous process improvement and high process maturity. Perhaps only a very mature organization is able to change itself into adopting agile practises, at such a large scale. But wouldn't it be better if your organization changed before it reached such a large scale?

It's hard to list in a logical way all the stuff I've been reading on this matter. If you're interested, just explore my del.icio.us bookmarks regarding the agile and scrum categories.

Buzzwords / Acronyms:

Update: Added the name of the person I met at BarCamp that uses SCRUM.


Presentation and Code Examples are Online

Regarding my BarCamp presentation, I've just uploaded the presentation (pdf or view online, strange fonts) and a very stripped down development version of OncologiaPediatrica.org packaged in a zip file, with just one model and a bit of additional documentation. I call it a development version because some settings are set to 'debug' and stuff like that, I just wanted to release this ASAP.

Maybee I should explain a little better what's going on under the hood, but I think that if people get the big picture and really consider CodeIgniter (CI) as a valid alternative, they'll get to know CI really well really fast.

Fell free to contact me about this.



In the latest months (or perhaps one or two years), I've read lots of interesting stuff and met some very interesting persons. Also, I've gained a lot of personal and, above all, professional experience in the areas of software development (both Web and standalone), teamwork, management and related.

I don't think I could ever list everything that influenced and inspired me during that time, but along with the Project Management Laboratory class I took last semester and my experience at JuniFEUP, two events played a major role lately: TakeOff 2007 and BarCamp Portugal 2007. The key factor was the diversity of interesting and inspiring people that I met there (or simply watched and heard, in some cases).

There are quite some reviews around, but since TakeOff is probably old news, I'll just leave a link to Alcides' Blog, which has the most complete review of BarCamp I could find, split through several posts. I agree with most of what he wrote and I like that he has a mostly impartial view on what happened -- it's in Portuguese, since this was BarCamp Portugal :P

And there I did it: went to BarCamp, had an active role there, and blogged about it! [These were the rules I remembered when I finished reading the concept of BarCamp] What was the active role? I made a presentation about CodeIgniter and my experience using it to build OncologiaPediátrica.org. Details will be posted shortly at the BarCamp wiki.

The above are most of the main reasons that compelled me to restart my blog. The reasons that prevented me from doing it until now are gone (fear of not having anything to write about, mainly) so, it was natural evolution, I guess :)

Also, this date is quite meaningful, since it was exactly two years ago that I wrote the last post here. The old posts are still available, but they're in Portuguese and not related to anything that will follow.