This is the story of my last project. A story that proves that agile works well in a distributed team, although most things are slightly harder. A story that shows all the obstacles we’ve faced and how we solved them by using proven techniques and processes. I’m writing this to inspire others to actively change and enhance the way they work, and also share our experiences with the world.
I’m sure you’re familiar with the painting The Scream, by Edvard Munch. That painting is full of agony, a bit like how I felt around a year ago. I was just leaving one of the very last meetings in my current project, were I worked as a Java consultant here in Stockholm Sweden, when a manager stopped me. He was very excited, apparently the department was about to start up a new project. The manager told me the project had acctually been running for three months at another department in Copenhagen Denmark, but the development team hadn’t really delivered anything and were more or less stuck. Our department had good reputation of succeding with high quality software development projects and it had been decided that we would take over. Fun, I said. The manager continued, of course it was a very high prioritized project (as always) and it was very urgent that we finished within around 9-10 months. The scope was big and a lot of features needed to be done in the first release. Tough, but definitely doable I said. Just before the manager was done, he said: And yeah, we have to use the off-shoring so it’s going to be a distributed team. Now I felt like the painting I mentioned in the beginning… Eeeeeh, challenging I said.
It sure was a challenge, but as a good consultant I decided to see it as an opportunity. A chance to learn new things. I hadn’t worked in distributed team before and the only off-shoring experience I had wasn’t something I considered a success. Either we make it and will have learned a lot or we fail and I still will have learned a lot.
In this blog post I’ll recap this past year and describe the obstacles we’ve faced, the solutions we’ve tried and how we in the end succeeded and also created friendship across nations.
Getting the right people
Together with one other Swedish developer and two Indian developers located in Mumbai India I was the development team. A Project manager, a Product owner and a Test manager from Denmark were also in the project. The Scandinavian people met for a sprint zero planning meeting where scope, initials stories and tasks etc were discussed. After only a few days it became clear that the other Swedish developer lacked the motivation needed to deliver this type of project. Therefore, a tough decision had to be taken immediately and switch him against another developer. I was one of the initiators of this switch and even though feelings were hurt it was clear to me that we needed the right people onboard to succeed with this project.
Key personal characteristics that we looked for in all the developers were ambition to learn and a wish to continuously evolve. If you also had very good technical knowledge and software craftsmanship abilities its was a very good combination. The latter is not always possible, especially in off-shoring situations where a ”cost reduce” perspective often takes the upper hand over quality and skills. But if every team member wants to be successful than I think you have a pretty good ground to build upon. After a while the development team grew to be four Indian developers and three Swedish.
Story point baseline
I still remember the first real sprint planning meeting. We developers still had very limited knowledge about the business domain, and we were still trying to get to know each other. Unfortunately, the company we worked for hadn’t spent much time or money on getting the technical foundation up and running to accomplish off-shoring, and we had to settle for phone meetings and screen sharing to communicate with Denmark and India. When it was time for reviews and planning meetings the PM and PO flew in from Denmark to prevent us being more distributed than necessary. The PO walked us through the proposed user stories and I must confess we were all pretty confused. But we ended up with a bunch of stuff to do, both big and small user stories.
Some of the people in the team had only worked with hourly estimates and I recommended we use story points instead. Mainly because it removes unnecessary low level detailed discussions when estimating and also because it’s a consistent way to estimate work size even if things turn out to go faster or slower than expected. We organized all stories in size and decided that the smallest we could find should represent 1 story point of work. A story roughly twice the 1 pointer was estimated to 3 and so on. Soon we had been able to give story points to all stories. We didn’t use planning poker during the first planning meeting, since we didn’t have a common view on the story points and therefore needed to find our baseline. I think planning poker works very well, and we introduced it during next planning, but for the first meeting it worked better to have an open discussion instead.
Lay the foundation for efficiency
According to me there’s one thing that stands out from the rest when it comes to creating efficient software development. No matter what you work with you have to know the business domain. If you don’t you either spend too much time waiting for answers or making incorrect assumptions and therefore risk building the incorrect solution. It was probably even more important in our setup, were half the development team were located in another time zone on the other side of the world. For several hours each day they had no business representative available. We invested a huge amount of time during planning meetings to get everyone onboard, and especially the PO had to answer a lot of questions. Both good ones and not so good ones.
After a while we got the hang of it and became more forward-leaning in discussions. After all, we are the ones with the experience in developing easy to use, yet advanced software applications of high quality. We must use our technical knowledge and recommend solutions that will make the application better. We became more self-sufficient and could get more work done quickly. We gained true efficiency in my eyes.
None of us had much experienced working in a distributed manner and we had to try a lot of different setups, and then inspect the result and adapt our way of developing software. Two weeks sprints, visualising progress and continuously evaluate the development process is an awesome way of working and most problems becomes visible and available for the team to solve. We used retrospectives after each spring and dealt with several problems. Thanks to recurring tuning of the process we could keep a high, constant velocity in almost all sprints.
I had the opportunity to be moderator at the first retrospective. I experimented with different tools to use in order to handle the fact that all of us weren’t in the same room, but I ended up with a simple Google doc. We did the Mad, Sad, Glad in combination with 5 Whys in order to find the root causes to things that was problematic for us. After an open discussion we could define a few SMART goals for next sprint. The KISS principle worked great in this setting, and also in most other meetings. Instead of letting advanced tools rule our way you communicate and cooperate a simple route like phone and screen sharing gave the meeting enough flow. Luckily after a while a video conference was installed which gave us the possibility to see each other.
To be done with something
A great thing about agile development is that you continuously show the PO, user, or customer your progress and get instant feedback and verification. Developing the right stuff, that also works, is probably the best way to earn trust from PO and create confidence within the team. Another benefit, it’s brutally obvious if something is not working. After a few sprints the application foundation felt solid and user stories after user story were finished. That’s when we saw our first real problem. Stories we said were done, thought were done, weren’t really done. Small stuff in many stories weren’t working as the PO expected. As a consequence of this these user stories had to be included in the next sprint as well, since they obviously weren’t done.
But what is done? Could we get a clear picture of that in some way? Apparently we had different opinions about the meaning of done and needed to talk about it. Scrum has its Definition of Done which I think is great. Some teams I’ve been part of have created rigorous DoD before going into development. That can work, but not if you create a big, abstract definition that is hard to grasp and difficult to relate to. Another approach which we used was to let us fall into some pitfalls, and than deal with the situation. My hope was that these experiences would create greater coherence and commitment to the problem, and the Definition of Done as the solution. We talked about what Done meant to us in this project, what good code quality is and created a short list things we thought had to be fulfilled in order to be Done with a story. We revisited that list several times during the project and made changes, but only when there was a problem which could be related to our view of Done.
For us this strategy was a winner. All Indian developers were pretty new to Agile and Scrum and needed a lot of guidance and help in order to understand the principles behind this way of developing software. What better way to learn than to be faced with a real problem and solve it by using proven, simple techniques. The same approach was used for the code, the quality of it and what principles and guidelines to follow when developing it. We’ve had a lot of discussions about Clean code, patterns etc and used pair programming and TDD as much as possible. Letting the problems bubble up to surface, dealing with them and learn from the situation was a tremendous help when we created a united team working for one common goal.
Creating a united team
One thing I was sure about when going into this project was that all team members had to meet in person at least once during the project. I know I couldn’t take a visit to Mumbai for granted but I told the PM and PO the importance of meeting in person and the benefits that would come after we’ve gotten to know each other. Fortunately they were onboard and after a few months of work the whole Scandinavian part of the team spend 1,5 week in Mumbai working together with the Indian developers.
The focus of that sprint wasn’t to develop a ton of user stories but instead to get to know each other and create one awesome team. A lot of pair programming and great discussions took place, both technical ones and domain related ones with the PO. I’m glad to hear that all the team members afterwards consider that trip to be a turning point in the project, were we went from being an ordinary team to a great team. Communications via chat, phone and video conference was a lot easier after this trip.
Time to accelerate
Although we worked hard the scope was still big, to big in retrospect for one release if you ask me, and time wasn’t something we could spare. We couldn’t afford to relax. I’ve seen it happen in many projects, after a while a tendency to start compromising on different things such as code quality, good developer practices and good sprint planning starts to sneak in. To some extent it happened to us as well. People worked more on their own, did more stuff at the same time and sometimes cut corners on code quality. Thanks to Scrum all progress was visible and it was impossible for us to ignore any problems.
What could we do about the fact that we had a lot of tasks ongoing at the same time? What could we do about the fact that we had stories so big that it could last until the very end of the sprint before they were finished. Sometimes we didn’t have time to finish them at all during the sprint and they had to be included in next sprint as well. We decided to introduce a part time Scrum master. A SM role that each of us developers took turn at, and its primary focus was to pay attention to the big picture, the flow and lead times of tasks, number of items in progress and also to coach other developers to pair program, use TDD and other well proven practices. I started as part time SM and my first move was to test WIP limits, which would directly stop us from doing to much at the same time. We also forbid stories over a certain size to keep the stories small. This would increase our chances of finishing stories early and fast. If stories were estimated to high we split them up into several smaller ones. These changes would help us avoid having too much stuff hanging at the end of the sprint. A WIP limit that was lower than the number of developers in the team would encourage us to pair up and do more things together. Developing together, for instance using pair programming, would also help us go those extra miles when it comes to code quality.
We introduced team agreements were we stated, among other things, that we wouldn’t start developing a story by ourself but instead be at least two people discussion the problems and potential solutions. All were pretty simple adjustments, but for us it worked very well. We gained predictability in what we could deliver per sprint and the velocity was kept high and at a consistent level.
Things were looking good, but after yet another couple of sprints we thought we could do even better. We dug deeper into the subject and tried to look at the big picture and the common goal. There were certain areas we found we could improve more on. We still had waiting times when it came to getting all our questions answers by the PO. Questions this far into the project weren’t always easy to answer, which of course took a lot of time. Our PO was awesome, had a lot of experience within the business domain and was very interested in building the best solution. But since she was only one person and there were seven developers she could became a real bottleneck at the end of the project. She started to work more or less 100% together with us, sitting next to us discussing and explaining things for us. This gave us immediate feedback and a possibility to correct any misunderstandings as early as possible. We developers notified her and the Test manager as soon as a new story was developed and available in a test environment. We gained even more chances to fix potential defects early and close to development, when everything was fresh in our minds.
The role as SM was also perfect for deciding in which order it was optimal to develop the user stories in the sprint backlog, with consideration taken to business priority, dependencies and potential technical or personal bottle necks.
These additional tunings made us even more efficient, and small stories and tasks floated nicely through the system without any major disturbances. We became such a tight team that in the end of the project you couldn’t barely see who was a developer and who was a business person.
We can do it
Only a few sprints left and the goal of shipping a feature complete application on time was definitely within reach. But hard work was remaining and it was clear that it was going to be tight. I always work hard and do my best, that’s all I promise. I don’t work overtime for long periods, it’s not possible to combine with being a family father with three kids (2y, 4y and 6y), and it also goes against everything I stand for. If a project plan requires developers to work a lot of overtime than the plan is not aligned with reality and should therefore be adjusted. Fortunately this project had so far been well planned with almost no overtime.
But, although nothing was required it was definetly encouraged to put in extra hours in the last sprints to get this application completed. The PO was of course well aware of this, so she started challenging us and offering small bonuses if those challenges were met. For instance if we completed all stories in a full packed sprint we went out for a nice meal or watched a movie together. This gave a boost and made us put in extra energy at the end. I don’t recommend anyone using this teqchnique from the start in a project, it shouldn’t be the norm of course, but for us the ”sprint end treats” was an inspiration to work just a little bit harder in the end. It also had a positive impact on a personal level within the team. We fought together and we celebrated together.
We made it. Except for a few ”nice to haves” we shipped a feature complete application into production which the users been very happy about. When working in our application they are able to do five times more stuff than before. They do it in an elegant, fast and intuitive yet advanced application. The feedback we gotten since first release is exclusively positive, both regarding the application and the team work, and discussions for future versions have already started.
To sum up, what did we do to complete this project with more all less the complete scope, on time and on budget?
- We continuously showed progress and got quick feedback from end users and PO. We knew immediately if we were on the wrong track.
- We created one team with one common goal. In this team everyone had an opinion that mattered and everyone had to contribute. This wasn’t done by itself as you now know, but once we got there the benefits were huge.
- We worked hard to become better and better of what we do, technical, process and domain wise, every week. We knew there would be a lot of problems on the way, but we took it easy and dealt with them. The problems only made us stronger.
- We had fun together. When something good happened we celebrated.
What could have been better?
- We could have done more and better code reviews. It was very hard to do it well due to the distributed setup, and the fact that junior developers were located on another place of the earth.
- We could have taken time and money from the project to find and invest in better communication tools than the ones we had at our disposal.
- In the beginning, we developers could have argued even harder for smaller and more releases instead of one big bang.
All and all this project was a huge success and each of us within the team became good friends. The memories and experiences I’ve taken with me from this project are some of the bests in my career. 🙂