Tuesday, December 01, 2009

Distributed Agile

Background

Over the last few years the software development process has been changing a lot. Organizations have started moving away from traditional methods and many companies are adopting one of the other form of Agile like Extreme Programming(XP), Scrum, lean development etc. With more and more people adopting Agile, tools and vendors have also come up with products which suite this form of software development. One of the fundamental requirement for a team to be agile is to communicate properly between different stakeholders. I have been working on a team which is developing software in different geographic locations. Here are  some of the challenges we face in day today life.

My Experiences

One of the fundamental principle of Agile is communication between team members. There is also a school of thought which suggests that Agile works best when team members are co located. This improves communication which results in faster turnaround time and improved velocity. In many cases teams are seated in different locations. On such occasion team in one location could be ahead or behind their counterparts in a different time zone. The difference in time zones could lead to delay in communication. One team might be about to finish their day when the other team is just beginning theirs.

Another problem I have personally felt when teams are distributed is that its not always possible to get all the stakeholders in one place at the right time. In such cases if key stakeholders are missing from certain meetings it could delay certain key decisions being taken. This increases the risk of project delivery and could impact timelines.

There are different ways to reduce the delay in getting the right information. We can use scrum of scrum calls to get updates from all the scrum teams. The medium of communication used for scrum of scrums calls becomes important. I have used tele conference calls. At times it become difficult to catch all the communication and many a times these calls end up being just a status update calls. We can use other methods like Telepresence. This is a viable option but unfortunately getting access to telepresence rooms everyday is a Herculean task. Another alternative to use is email communication. The problem I see with this form is people tend to ignore it if they are busy with some other task.

We tend to use email communication a lot to clarify doubts. That adds a risk because if the information transmitted through emails is not always updated in the respective documents like business requirements document (BRD) or functional requirements etc. Things which can be answered by product / feature owners if they were face to face takes longer duration because things can be misinterpreted over emails. At times there are communication gaps because people don't understand certain language. It can also be irritating for a developer to reply to someone over an email for some trivial tasks which can be completed in minutes if they were sitting in the same location. Having the teams co-located also helps in building relationships among team members which is very difficult to build otherwise. Distributed teams also increase the risk of duplication of effort for certain tasks.

These are some of the issues which I have experienced personally during last few months. In order to address these issues I can think of some measures. To reduce dependencies on business folks and to get answers to business  clarifications we can have a representative of the product owner sitting with the offshore team. We can also have a business analyst (BA) replacing the actual customer or product owner. But from my experience I can say that the BA merely acts as a  proxy who interacts with business folks to help development team answer their questions. Having a business representative working closely with development team also reduces lot of effort and adds much more value to the whole development cycle. If we have the business representative working with development team it also enables them to get early feedback at the end of the sprint which can help in improving the overall product quality.

In order to reduce dependencies between teams there needs to a high level release plan at a project level. This plan should highlight the various dependencies between various features of the product. This will help the product owner as well as development teams to build features in such a way that the dependencies are sorted out in time before stating something which is totally dependent on some other feature. This is where the active participation of product owner is required in day to day activities. Because if there is a delay in implementing certain feature product owner can re-prioritize the product backlog so that timelines are not impacted.

Conclusion

My experiences for past few months suggest that Agile doesn't work really well in a distributed environment. We loose some of the advantages of agile if teams are not co-located. Having seen all the pain and being at the center stage of many of the decisions we had to take during past few weeks I would suggest people to be very careful before going in for Agile teams in distributed locations.

My experiences might not be really good, but I would love to hear from people who might have had success working on distributed Agile teams. I would like to know what methods or tools they used to succeed in their projects.

 

Submit Apache Spark job from IntelliJ to HDInsight cluster

Background This is 2nd part of the Step by Step guide to run Apache Spark on HDInsight cluster. In the first part we saw how to provision t...