Lessons learned from an agile scrum shop - Why Agile Business Analysts, con't.

We last entertained the concept of bringing Business Analysts (BA) into an agile methodology process.  First I want to reiterate that this suggestion is just one of many options in an agile shop.  My opinion is that the need for this type of dedicated role increases with the volume of differentiating product lines in a given business model.  As a business that tries to manage many different product lines simultaneously, the tendency is often to dilute your product owners into managing several products at a time versus hiring a multitude of product owners.  This makes the day to day task of grooming the backlog and refining user story conformance usually a lower priority to actual product market analysis.

BA’s can be a good choice in this scenario but are they truly called BA’s?  Our preference is to call these roles ‘Agile Business Analysts’ (ABA) as the responsibility changes from the traditional sense.   Primary responsibility of the ABA is to extend off of the product owner responsibilities.  Key difference is that the ultimate decision must be maintained by the product owner and a good product owner will insure checkpoints in this process to govern that responsibility.

One option that we experimented with was to assign this role to lead system engineers on the scrum teams.  Seemed like the natural path at the time.  We tasked these individuals with working directly with the product owners in grooming backlog prior to any release planning.  They worked with various subject matter experts or customers to refine epic user stories into actual sprint-able stories.  They often spent their days talking with software architects or user experience designers to help gather conformance for the stories.

Impacts experienced from this involved the need to cut back these individuals sprint capacity because of the ever increasing demand upon their time as an ABA.   We found some lead engineers could only be tasked at 20% development capacity due to the ABA role taking upwards of 80%.  Even then we still found product backlog not being as prepared as we liked coming into sprint planning.  Unhappy developers also were encountered as our best developers were not the ones doing the coding and experience taught us that most senior engineers really like to develop code.  Along with inconsistency across different product lines of ABA work and the difficulty in finding lead engineers that also wanted to be Business Analysts, we next evolved to a different path.

Lessons learned from an agile scrum shop - Why Agile Business Analysts, con't.

So just one of the benefits from agile is that it has a tendency to bubble up existing issues in your software development process and make them glaringly visible. My opinion is that it does this by focusing your entire development life-cycle into small incremental windows.  When you increase the velocity of your development life-cycle into such a short period, you leave no room for inadequate processes.  The scrum process we adopted at Mercury Payment Systems, for example, started with two week sprint cycles.  This allowed visibility of an entire development life-cycle within just two weeks.  We quickly identified several areas of our development process that could stand improvement.

One of the mistakes we had made at Mercury was to start our agile process without the dedicated role of a product owner.  We thought we could just multi-task key staff members to function in that capacity without the training and passion that a true product management focus can bring.  Without the authority of just one person to call the shots, so to speak, product decisions quickly got tied up into group debates.  This slowed down our velocity and also introduced conflicting requirements that sometimes clashed with the greater picture of the overall company vision.

It didn't take us long after implementing a product owner role to see the benefits of a more highly focused product vision.  We had a clearer overall picture of where our product was intended to evolve and what the user looked like.  This helped tremendously with solving conflicting requirements and made decisions easier but somehow still did not improve our scrum velocity.

We found that epic user stories were coming into sprint planning sessions and with very little design or architecture discussed.  Software engineers spent a fair portion of their sprint capacity acting as business analysts (BA).  While I'm sure some agilists will profess that this is the beauty of agile and that it is more advantageous to put each developer directly in front of the customer for BA type functions,  we found the consistency to be greatly lacking.  What was also uncovered by having these epic requirements being architect-ed in the sprints,  budget roadblocks were often stopping the sprints as IT purchasing needs were uncovered.  Or worse case scenarios were sometimes occurring in that technical debt was being chosen as quick solutions to get around budget or time constraints.  

What agile bubbled up to us next was the discovery that not all software developers make good business analysts.  While one team produced good velocity and rock solid design and architecture, others spun their wheels almost an entire sprint to get requirements designed and clarified and even then sometimes missed the conformance.  For those of us with experience in waterfall methodology,  we remembered the benefit that came from a strong business analysts.  In my experience,  the ones that I found the most successful were those that came up from a development background and had strong design and architectural skills.  It was never an easy task to also find those skills coupled with a strong  business understanding and capability to translate business needs into technical specifications and vice-versa.  But how do you bring these roles into an agile scrum methodology?

Keep following and that is where we will pick up next.

Why Agile Business Analysts

What are agile business analysts (ABA)?  To head down that path, it helps to put the context in place of what a traditional business analyst (BA) is. 
Historically, BA’s have been used in waterfall software development practices  to assist with requirements documents that sometimes take months to complete and exceed hundreds of pages.  These development practices have been criticized for lengthy development stages and requirements that often change by the time the software is deployed.  Agile is just one answer to these problems by shortening deployment cycles so that customers can get changes more frequently and more important feature sets can be prioritized above the less important requirements.  Requirements documents are replaced by user stories that simply state out testing conformance so typically achieves the feature request in just a few paragraphs.
So where would a BA fit in this methodology that prides itself on producing only the necessary documentation to achieve quality results?  ABA's come into the agile process as a way to extend the work being done by product owners.  They still are instrumental in getting good requirements and their most important goal in agile terms is to get epic user stories into stories that are sprint-able prior to involving the scrum team.  The benefit of using these resources in your agile scrum is to increase scrum team velocity and increase customer satisfaction for the product being developed.
Over the next several entries, I will describe in detail how this can be accomplished along with some of the lessons learned from our business along the journey.