The Fourth Noble Truth of Software Development

Buddha’s fourth noble truth discusses how there is a path to the cessation of suffering. This is explained
in further by what is detailed in something called the eightfold path. In a nutshell (enlightenment
for dummies), I like to think of it as the middle ground. Not too self-indulgent but also not too self-
mortifying either. Or simply put ‘don’t fall into the trappings of compulsive living’. It also describes
the nature at which craving, clinging, ignorance, delusions and such will gradually lesson as you make
progress on the path. This path also covers many lifetimes so patience is a virtue.

This describes a perfection that is strived for but not necessarily obtained within this lifetime. Not too
different than the agile process when you look at from a philosophical view. One should visualize their
adaption of agile as something that has not yet achieved nirvana but should be moving along the path
towards it. It will take time to get there so once you accept that suffering is inevitable along the journey,
peace will follow.

Agile acknowledges that things may not always go smoothly by planning a retrospective meeting after
every sprint. This meeting provides the chance for the team to improve the process that makes them
work more efficiently one sprint at a time. To make this meeting strive towards nirvana, allow the
team to focus on an inward introspection versus an outward one. What I mean by this is that too many
organizations get wrapped up by what is written in text books or labeled by the word ‘repeatable-process’.
What should be happening, in my opinion, is introspection into what makes the most sense for the
individual company or team regardless of what is going on externally. This can facilitate a true root-
cause-analysis of what are some of the stumbling blocks to this team based upon the conditions specific
to that team and how to solve them. What is a solution for one individual or team to get to perfection may
not always be the same for others as they are each at a different juncture on the path.

What can be achieved by allowing the team the autonomy to improve their own process through creativity
and innovation is increased efficiency and velocity in my opinion.  Some might even call that nirvana....

The Third Noble Truth of Software Development

Buddha’s third noble truth states that ‘The cessation of suffering is attainable’.  This expresses the idea that suffering can be ended by achieving dispassion, the unmaking of  sensual craving and conceptual attachment.  The result is a state of Nirvana, which can translate to freedom from all worries, troubles, complexes, fabrications and ideas.  Sounds great, huh.  I don’t know about you but I’m ready for Nirvana right now.  

The phrase ‘fabrications and ideas’ is interesting to me here.  My first inclination was that this meant ‘design is bad’ as one should let go of having any idea of the future.  Upon further examination, my opinion is that this is not necessarily translating into ‘design is bad’ but is more in line with the second noble truth in that the future is constantly changing.  One can only truly design for today.  Design is ‘good’ if it is simply a construct of how the product should evolve from a technical perspective but allow for a shifting future.  This is one of the things I like about SOLID architecture principles in that my opinion is that it allows for this easily changing components.  So now you have a ‘basic’ design in place using architectural principles such as SOLID to allow for shifting futures, how do you manage the day to day development team with the thought that cessation of suffering is attainable?  

The beauty of Kanban is that by limiting throughput of feature requests or change in general,  you limit the wandering of the mind into things of tomorrow that may never come to be.  Suffering from a developers viewpoint occurs when time is wasted on events (or designs) that never come to fruition.  The ‘worries’ of tomorrow can be shielded from developers by limiting the throughout of the backlog.  The team only looks at what is truly prioritized for the immediate cycle.  This acknowledges that the future is constantly shifting so why plan on things that you may never do. The product owner simply moves the next important requirement at the time into the backlog when one is needed for replacement. Only plan for what can be managed in the short-term of a release duration.  

Developers pull an item from the backlog into the ‘in-progress’ bucket when they are ready for work.  The in-progress bucket is also limited in throughout so that it reduces distractions as of the result of working on too many tasks at one time.   The length of time for task completion can also be reduced by allowing for this mindful focus.  Developers get the satisfaction of getting more frequent visualization of completed tasks and spend less time in wasteful futures planning.

So can Nirvana be achieved in software development?  I like to think so and believe Kanban is the noble path to achieve this which leads us to the fourth noble truth which talks about the path. We will get into that discussion next.

The Second Noble Truth of Software Development

We previously talked about my lack of mindfulness while in meditation and the similarities of Buddhist wisdom to software development challenges.  I am continuing my work on a wandering mind (will most likely be a lifelong journey) but do recognize how you can put this wisdom into everyday practice including that of software development.

Buddha’s second noble truth states that ‘The origin of suffering is attachment’.  In a nutshell,  this basically points us to the recognition that suffering results from an attachment to transient things and the ignorance thereof.  Transient things by nature are impermanent which Buddha tells us all things are.  Ignorance comes from the lack of understanding of how our mind is attached to these impermanent things.  This can be often demonstrated through the actions of craving or clinging.

In software development terms, this can easily be demonstrated by what happens when too many requirements are presented for a product.  The ‘craving’ for too many features in one release can lead to suffering for all on the team, including the customer themselves.  Too many requirements in one release can lead to lengthy release iterations.   It can also lead to distracted developers from the looming nature of what lies ahead which will often take them into pondering tomorrow more than they are thinking about what needs to be done today.  The drawback from this approach comes from  the transient nature of our world and how fast things change.  Often, too much time is spent pondering requirements that will never come to release or used by customers.  Shorter iterations can help address the ‘now’ mentality, meaning state of the world as it is today and lead to higher product satisfaction (i.e. less suffering).

This is one of the reasons I like Kanban as an agile approach.  By keeping the team focused on only a few features at a time  (limit work in progress) and deciding as you go when the product is releasable as a product owner,  you have the ability to adapt to this transient world more in harmony with its true nature.  The team stays focused on what needs to get done ‘now’ versus designing the for a changing future.

Next we will discuss the similarities between the third noble truth which states that ‘The cessation of suffering is attainable’  and my version which states that ‘The cessation of suffering is attainable through Kanban’.

The Four Noble Truth’s of Software Development

You may recognize the similarity here to that of most Buddhist practice. I would never want to plagiarize Buddha so must give full credit where credit is due.  I was in a meditation class the other day and the teacher was talking about the suffering caused by the nature of wanting.  Now I recognize the irony here from a wandering mind while in meditation but I immediately connected what she was talking about to the suffering also visible in software development.  For the sake of the meditation practice I did file the thought away for another day and went back to my oms.  So here I am rethinking the concept but this time sitting at a desk and not on a mat.

Let’s start with the first noble truth which states ‘Life means Suffering’.  My summarization of this noble truth is simply that we are never able to keep permanently what we strive for.  In technical terms this is best translated to mean processes continuously change.  Even processes that are good one day will inevitably become bad another so you should anticipate this constant change.

Let’s break this analogy down a little further for the sake of the skeptical mind (which could also translate to someone who denies that change is inevitable).    The world around us is constantly changing so therefore our processes must also change to adapt to the need at the time.  Suffering manifests within those individuals that do not acknowledge this truth and resist a constantly evolving process.

The beauty of agile methodology in respect to this noble truth comes inherently through the usage of sprint retrospectives.  If used properly,  the positives and delta’s are adapted into the agile process so that it is constantly evolving.  The happy agilist embraces this constant change and also recognizes that the good times are also fleeting moments.  The changing world will throw challenges into your process and you should simply acknowledge the trying times as the sign to evolve.  Don’t suffer from these challenges but simply recognize them as part of the ebb and flow that is this world then retrospect on how to make tomorrow another ‘happy’ day.

Next we will discuss the similarities between the second noble truth which states that ‘The origin of suffering is attachment’  and my version which states that the origin of product suffering is too many requirements.

Lessons learned from an agile scrum shop - Why Agile Business Analysts, con't.

An interesting statement came the other day from an agilest with the Mile High Denver conference.  They discounted the use of BA’s in an agile shop as something that only a bureaucratic company would do and not many mainstream agile shops.  I found that compelling and would love to hear from other development shops as to their experience on this.

My thoughts are that every development shop has someone that is filling the role of the business analyst in their scrum.  The agile process allows for scrum team members to rise to the occasion and fill the void that is presented which is the beauty of agile.   My challenge to this concept comes not from a bureaucratic company but one that has an ever increasing volume of products to support.    It is easy to say that those experienced can rise to the role of BA when necessary until you have spread those key resources (lead engineers, lead QA Engineers,  Product Owners, etc.) too thin for all the active products going simultaneous.  Not all junior team members have mastered the concept of Business Analysis and are often trained through pairing with lead team members.

Having a dedicated BA that serves as part of the scrum team can greatly facilitate an increase in overall scrum team velocity.  This came as a result of developers having very well vetted and designed requirements at the start of a sprint.  We found that tasking these BA resources as part of the scrum team can also facilitate the knowledge transfer amongst the entire team.  Similar to the scrum master or product owner roles, we find that we can task BA’s at only part capacity (depending on the product) so that they can take on more than one product.

Before we added BA’s to our agile role definitions, we had struggling velocity with some sprints actually accomplishing nothing but requirement vetting by sprints end.  After BA’s came into play, we found sprint velocity and overall customer satisfaction increasing.  I’m not saying we couldn’t have done this in a multiple of ways.  Some shops hire more senior QA engineers that often fill this role or hire only experienced developers.  For a smaller company (we are less than 25 developers), hiring a couple of BA’s to facilitate the agile process was a very cost effective way for us to operate with almost immediate results (within 1 – 2 months ramp up).

Anyone else out there have any similar experience?

Lessons learned from an agile scrum shop - Why Agile Business Analysts, con't.

We last entertained the concept of bringing Business Analysts (BA) into an agile methodology process.  First I want to reiterate that this suggestion is just one of many options in an agile shop.  My opinion is that the need for this type of dedicated role increases with the volume of differentiating product lines in a given business model.  As a business that tries to manage many different product lines simultaneously, the tendency is often to dilute your product owners into managing several products at a time versus hiring a multitude of product owners.  This makes the day to day task of grooming the backlog and refining user story conformance usually a lower priority to actual product market analysis.

BA’s can be a good choice in this scenario but are they truly called BA’s?  Our preference is to call these roles ‘Agile Business Analysts’ (ABA) as the responsibility changes from the traditional sense.   Primary responsibility of the ABA is to extend off of the product owner responsibilities.  Key difference is that the ultimate decision must be maintained by the product owner and a good product owner will insure checkpoints in this process to govern that responsibility.

One option that we experimented with was to assign this role to lead system engineers on the scrum teams.  Seemed like the natural path at the time.  We tasked these individuals with working directly with the product owners in grooming backlog prior to any release planning.  They worked with various subject matter experts or customers to refine epic user stories into actual sprint-able stories.  They often spent their days talking with software architects or user experience designers to help gather conformance for the stories.

Impacts experienced from this involved the need to cut back these individuals sprint capacity because of the ever increasing demand upon their time as an ABA.   We found some lead engineers could only be tasked at 20% development capacity due to the ABA role taking upwards of 80%.  Even then we still found product backlog not being as prepared as we liked coming into sprint planning.  Unhappy developers also were encountered as our best developers were not the ones doing the coding and experience taught us that most senior engineers really like to develop code.  Along with inconsistency across different product lines of ABA work and the difficulty in finding lead engineers that also wanted to be Business Analysts, we next evolved to a different path.

Lessons learned from an agile scrum shop - Why Agile Business Analysts, con't.

So just one of the benefits from agile is that it has a tendency to bubble up existing issues in your software development process and make them glaringly visible. My opinion is that it does this by focusing your entire development life-cycle into small incremental windows.  When you increase the velocity of your development life-cycle into such a short period, you leave no room for inadequate processes.  The scrum process we adopted at Mercury Payment Systems, for example, started with two week sprint cycles.  This allowed visibility of an entire development life-cycle within just two weeks.  We quickly identified several areas of our development process that could stand improvement.

One of the mistakes we had made at Mercury was to start our agile process without the dedicated role of a product owner.  We thought we could just multi-task key staff members to function in that capacity without the training and passion that a true product management focus can bring.  Without the authority of just one person to call the shots, so to speak, product decisions quickly got tied up into group debates.  This slowed down our velocity and also introduced conflicting requirements that sometimes clashed with the greater picture of the overall company vision.

It didn't take us long after implementing a product owner role to see the benefits of a more highly focused product vision.  We had a clearer overall picture of where our product was intended to evolve and what the user looked like.  This helped tremendously with solving conflicting requirements and made decisions easier but somehow still did not improve our scrum velocity.

We found that epic user stories were coming into sprint planning sessions and with very little design or architecture discussed.  Software engineers spent a fair portion of their sprint capacity acting as business analysts (BA).  While I'm sure some agilists will profess that this is the beauty of agile and that it is more advantageous to put each developer directly in front of the customer for BA type functions,  we found the consistency to be greatly lacking.  What was also uncovered by having these epic requirements being architect-ed in the sprints,  budget roadblocks were often stopping the sprints as IT purchasing needs were uncovered.  Or worse case scenarios were sometimes occurring in that technical debt was being chosen as quick solutions to get around budget or time constraints.  

What agile bubbled up to us next was the discovery that not all software developers make good business analysts.  While one team produced good velocity and rock solid design and architecture, others spun their wheels almost an entire sprint to get requirements designed and clarified and even then sometimes missed the conformance.  For those of us with experience in waterfall methodology,  we remembered the benefit that came from a strong business analysts.  In my experience,  the ones that I found the most successful were those that came up from a development background and had strong design and architectural skills.  It was never an easy task to also find those skills coupled with a strong  business understanding and capability to translate business needs into technical specifications and vice-versa.  But how do you bring these roles into an agile scrum methodology?

Keep following and that is where we will pick up next.

Why Agile Business Analysts

What are agile business analysts (ABA)?  To head down that path, it helps to put the context in place of what a traditional business analyst (BA) is. 
Historically, BA’s have been used in waterfall software development practices  to assist with requirements documents that sometimes take months to complete and exceed hundreds of pages.  These development practices have been criticized for lengthy development stages and requirements that often change by the time the software is deployed.  Agile is just one answer to these problems by shortening deployment cycles so that customers can get changes more frequently and more important feature sets can be prioritized above the less important requirements.  Requirements documents are replaced by user stories that simply state out testing conformance so typically achieves the feature request in just a few paragraphs.
So where would a BA fit in this methodology that prides itself on producing only the necessary documentation to achieve quality results?  ABA's come into the agile process as a way to extend the work being done by product owners.  They still are instrumental in getting good requirements and their most important goal in agile terms is to get epic user stories into stories that are sprint-able prior to involving the scrum team.  The benefit of using these resources in your agile scrum is to increase scrum team velocity and increase customer satisfaction for the product being developed.
Over the next several entries, I will describe in detail how this can be accomplished along with some of the lessons learned from our business along the journey.