When focusing on Minimum Viable Product is the wrong thing to do #Agile #MVP

mvp

There isn’t too many things more sacred than Minimum Viable Product in the Agile circles. Maybe Planning Poker, Automated Testing, and Continuous Integration. But usually Minimum Viable Product is also right up there. What if I told you that focusing on Minimum Viable Product is sometimes the wrong thing to do? Would I be branded a heretic?

Well it is – sometimes.

Client Focus

I sometimes find that we in the Agile community are falling into the old traps that haunted the traditional Software Development Professional – the fact that we know what is best for the client. I see comments like we shouldn’t create any documents or create any estimates because they are waste. Usually these positions are written from the point of view of the Software Development team. But we as developers may define value and waste differently than our clients. We need to understand the context of the project we are in and the situation the client is in and apply our processes accordingly.

For some clients not creating documents may be a waste as they will need to create them for regulatory purposes and it may take them longer to create them after. (In addition to being more error-prone) For some clients, estimates may provide them great value in their forecasting and budgeting process. So while we may think that creating estimates in wasteful, the clients may not share the same view. Ultimately, their viewpoint is the one that matters if we want to be client focused and remain in the service industry.

Our job as Software Development professionals is to provide our expert opinion and highlights the impacts of the decision to the client. If we can’t convince the client of the value of our position, then our position could not have been a strong one to start with.

Our job as Software Development Professionals to to provide all the information to the clients so they can make the most well-informed decision possible. Not to decide for them. Ever.

Minimum Viable Product

So where does that leave us with Minimum Viable Product? How could organizing the project in a way that you could go live during the project be wrong?

Well, what if going live early has no value to the client?

In the diagram above, what if the client definition of value is only realized by being able to transport a family of 5? Being able to structure a project to continue to deliver Minimum Viable Products comes with a cost. If you look at the example above, you can see that so each step up in the viable product chain, we need to swap out some frameworks and functionality for other frameworks and functionality. Structuring a project in this way doesn’t come without a cost.

Now there are many great technical reasons why to want to structure a project focusing on a chain of Minimum Viable Products. They help to address risk and provide lessons learned to improve the product and project as you go along. But don’t assume that it is always what the client wants, needs, or values. In exceptional circumstances, the extra effort to structure a project focusing on Minimum Viable Product may actually be most wasteful that structuring the project in a traditional way.

There might be a series of Minimum Viable Products that could be created to address technical risk, but not to provide any early client value. Perhaps it might be just one prototype early on to address new technology. The important thing is don’t assume it has value for the client.

Summary

We must always remind ourselves that the processes and methods we use must return more value for the client than other options and not just fall into the habit of executing every project using all the Agile tools in the tool belt.

After all, isn’t that what got us in the Detailed Work Breakdown Structure, Big Design Up Front mess before?

 

 

 

Birth of #Agile #Agile2014

Reading through all of the Agile 2014 tweets, my memory flipped back to my university years where I consider the seeds of Agile were sown. Although I am sure people at that time didn’t think that was the CASE. (pun intended)

I was taking my Advanced System Analysis course and the final chapter was on CASE tools. If you want a refresher, the wikipedia page is actually quite good.

I remember at that time in the late 80’s there was major discussion/movement on code generation and CASE tools. I guess it was the professions response to all the challenges with Software Development at that time. We had challenges completing projects successfully and within our estimates, so the CASE tools were going to ride to the rescue and generate all the code once we got all the requirements specified. In that way we would remove all the challenges and problems with the troublesome development activities. Which I guess were identified as the problem. If anything, this was the last gasp of the big requirements up front movement. If anything, these CASE tools required even more detailed requirements up front than were required before. A proliferation of modeling tools arose out of this movement to greater and lesser degrees of success. But thankfully none of these tools really provided an answer to the problems.

In my limited use of CASE tools, I was amazed with the amount of work required to define the requirements to be able to generate code. And then it seemed that unless the code was totally mundane, there would always be issues with the code. And don’t get me started on the performance of the code.

Two Solutions

What I thought was interesting was that I saw two solutions arise out of the CASE rubble.

1) Off-Shoring – In the spirit of sub-optimization, this solution followed in line. If we can’t design tools to generate the code automatically, lets figure out a way to ship the code creation to the cheapest possible resources. In many ways, this was a solution in-line with the principles of CASE. We need to create a detailed specification document that we can then take and generate code from,  either by algorithm or people.

2) Agile – although we didn’t call it Agile at the time, the other movement that arose was a recognition that sub-optimizing the development activities was the wrong approach. What we needed was an integrated, cross-functional team, not more requirement documents. We also needed to deliver more frequently to the end client and work side by side with them. This was the absolute opposite of what was typically envisioned with the CASE solutions. In hindsight, I really feel that this was the start of many of the flavours of Agile. (although they wouldn’t officially should up on the scene until years later)

Observations

What I find most interesting is when I look back, the CASE movement was all about trying to solve a problem with more and more technology. Which we know hardly ever works. Now there is additional technology that makes Agile solutions possible, but at its heart it is a system with less technology not more. It is about communicating and working together to solve a business problem. Whether we be business people, analysts, developers, or yes, even project managers.

If I only had the foresight back them to buy stock in 3M for all the stickies that the industry would successfully use over the next decades, I would be a rich man.

Rest in Peace CASE, Long Live Agile.

 

#1 skill for #agile and #traditional project team members – “Enough-ness Sense” #pmot #baot

I’ve had the good fortune on being on small, medium, and large traditional and agile teams. As I’ve worked on Agile teams recently, I’ve come to realize that both types of teams require one non-technical skill above all others. (In my humble opinion) Interestingly enough, this skill is required on both traditional and agile projects but in contrasting ways.

The one skill

What is the one skill I refer to? Enough-ness Sense. 🙂

What is Enough-ness Sense? It is the skill to know whether the work has been done to the appropriate level for the circumstance.

Funny enough, (har-har) this Enough-ness Sense plays contrasting roles on traditional projects versus agile projects. On traditional projects, the Enough-ness Sense is used to sense when we have done enough planning and analysis and we should move to developing and other doing activities. Without this sense of Enough-ness Sense, sometimes traditional projects get stuck in analysis-paralysis and an inability to move until all the intricate details are known. Professionals with Enough-ness Sense know the difference between an important factor that needs to be defined and unimportant factors that can be determined at a later time. This can be seen in how detailed the analysis and design documents are. Chances are if your documents are greater than 50+ pages you don’t have enough Enough-ness Sense.

On Agile projects, the effect of Enough-ness Sense is the reverse. Here the Enough-ness Sense is used to ensure we don’t start work until we have enough of an overall solution and design. I have seen some Agile projects jump into development too early without an end-to-end thought out design. These projects then usually suffer from large rework and scope increases later as key elements were not considered and planned for. If you start coding without a User Story Map or some general design work, you probably don’t have enough Enough-ness Sense.

In both scenarios, this Enough-ness Sense is crucial. And it is not specific to one role. The Enough-ness Sense is used to determine is the plan, requirements, code, and tests are at an appropriate level? It is a critical skill for a person to be able to look at the risk inherent in the situation and determine the appropriate level of thought required.

The negative side of Enough-ness Sense

So what happens if you don’t have enough Enough-ness Sense on your project team?’

On a traditional project, this will result in waste. There will be too much planning, too much analysis, and too much design. Funny enough, it usually doesn’t result in too much coding as that is the last activity being done. But in some cases of gold-plating it can result in too much code.

On an agile project, this will result in rework. There will not have been enough planning, analysis, and design to envision a cohesive solution. As such, changes occur on the project not due to changes client requirements but rather because the project team did not think something through. It is a fine line to embrace change but to not use it as an excuse to not think something through. Agile is about embracing client change,  not a lack of thoroughness on the team.

Summary

Enough-ness Sense is critical for success on both traditional and agile projects. Unfortunately I don’t know how much Enough-ness Sense is enough. (har-har)

When is a plan good enough? Provisional Project Planning

As I’ve taken the journey along the Agile path I’ve come to learn two things I am certain about:

1) A detailed Work Breakdown Structure is one of the largest wastes and mistakes that can exist in a project

2) That not creating a temporal plan at some level can also lead to rework and failing late

On the surface, these comments may seem to be contradictory. I seem to be supporting neither traditional or Agile. And I would agree that it can appear in this way. But like most good solutions, the optimal solution is a compromise between extremes.

In my blog about whether Flow can be used on projects, I emphasize the need to plan. (if you are interested, you can find the blog here) I then continue the planning discussion in my Agile Chicken Little blog. (which can also be found here)

My intent of both posts was to communicate my thoughts on the fact that some level of planning is crucial. But when do you know when the plan is good enough? When is it excessive?

Too Much

I must admit I was someone who used to create detailed work breakdown structures. I even used to assign multiple people to one deliverable and have multiple items in progress. Looking back now I see that this was excessive planning for little value. The detailed project plan was also not an accurate depiction of how the actual project would execute.

But even when I started working on more Agile projects, I still sought out a moderate detail plan to try to comfort myself. I thought that we needed a full prioritized backlog of items. I held frustrating meetings with clients asking them if item 101 was more or less important that item 102. (If you were in one of these sessions with me, I apologize and owe you a beer)

Too Little

But as I stated in my prior posts, the creation of a temporal plan is critical to allow the team to think through the project process, dependencies, and possible conflicts. To start to do the project using just Flow may not allow you to fail fast. You may discover a dependency or conflict late in the process. It is possible that an initial plan would have highlighted this issue.

But creating a project plan at the start can create assumptions and issues all on its own. So what to do?

Just right – Provisional Project Plan

It the game of golf, there is a wonderful concept of a provisional ball. In the case where you have already launched a ball into the woods, you hit a provisional in case you cannot find the ball. It is seen as a time-saver to keep the game moving.

I use this analogy when I create my high-level project plans.

My provisional project plans are:

  • Only one way out of many to execute the project – it is not the one correct way
  • It is high level to get me to when I can look for more detail to refine the plan. (i.e. finding my first ball)
  • It sets the expectations with the clients that the plan will change, but that we have at least thought through one way that the project could be delivered.

We then use flow to prioritize the next group of stories in the project. Once they are prioritized we use flow to execute.

Summary

In my experience, these Provisional Project Plans provide the right level of planning. They allow the team to think through the execution of the project without committing to ‘the’ plan. It also then provides the ability to execute in a flow-based structure within the Provisional Project Plan.

Special thanks to Luke Hohmann who illustrated the waste of creating a full prioritized backlog through the use of Innovation Games. By prioritizing and executing in groups of 10-20 items, we are able to efficiently work in a flow mode. (And update our Provisional Project Plan as we execute)

#Agile Enterprise Data Model – #holistic and #iterative

I’ve recently been working on a Data Warehouse project that I am trying to do in an Agile way. Unlike Agile Software Development, Agile Data projects have some complexities that cause you to adapt the common Agile methods and practices. I am a big fan of User Story Mapping and creating a User Story Map to guide your development of an application and to create and prioritize your backlog. But I have felt that a Data Model tends to not allow stories to be sliced as thinly as application functionality. In many cases a group of tables must be taken together to make any sense at all. Unlike application components that can be segmented easier, it sometimes is quite difficult to segment a data model into parts that can be worked on independently.

I have been involved in other Data Modelling projects where intense analysis was done to try to understand all the intricacies of the data before the application development started. This analysis sometimes lasted 3 months, 6 months, or even years. It goes without saying that this is not an efficient was to organize a project.

If you try to segment application user stories, individual reports, and data requirements as stories they end up being a bit too fine-grained from database point of view. Creating a data model from a single user story or report can lead to a data model that has to undergo a great amount of re-factoring as new user stories, reports, and data requirements are considered. Frequently a new user story that enhances application functionality requires an entirely different data model. 😦

Then I started to think that this can happen to most Agile projects. They slice the User Stories thin from an application point of view and that works as the application components can be somewhat independent, but what happens when you try to line-up the data components?

I am sure this impedance has led to the more drastic database re-factoring that has caused it to have a bad reputation and caused people to write books on the subject.

Epiphany

Then it hit me. Although the User Story Mapping exercise does allow for an end to end view of the application that can then be iterated on, it does not create an end-to-end view of the Data Model or the Data Domain. In fact, some critical components of the Data Model may not be encountered until much later. This can cause significant rework based upon the design that has been done earlier.

So what could be the solution?

Agile Enterprise Data Model

An Agile Enterprise Data Model is a thin data model of the Data Domain that can be done as a two-week technical spike. This Agile Enterprise Data Model provides a User Story Map for the data. It provides a Data Model with placeholders for discussion and further refinement. The concept it not to do detailed Data Modelling but to understand all the data components at a high level that will need to be considered by the project. Used in conjunction with a User Story Map, it can be a valuable tool to allow the application and data model to be worked on in true iterations.

Typically an Agile Enterprise Data Model is a conceptual data model with 20-30 entities. There are minimal attributes and discussion and understanding revolves more around entities and relationships than anything. The Agile Enterprise Data Model can then be refined to create the application database, Operational Data Store, and Data Warehouse.

Summary

I have used the Agile Enterprise Data Model with very good success. It also works best in a Data Domain you are unfamiliar with as it allows you to understand a Data Domain holistically initially and then deliver in iterations. The best of both worlds.

Top 10 practices for estimating Agile projects successfully

I’ve had multiple posts in the past that provided my passionate opinion on why I believe we should estimate Agile projects. (Both from the perspective of the client and the team) But rather than get into that discussion once again, I thought it would be more valuable if I shared the 10 practices that we have used over the past 6 years that have allowed our teams to successfully estimate Agile projects.

Context

I work for a company that bids competitively on projects against other companies. In almost all situations we have to create estimates to provide to the client or else we would not be successful in winning the business. So estimating really is a fact of life for the type of business we are in. Although it might be nice to proceed on a project without an upfront estimate, that really isn’t a luxury we are provided. I believe we are in a similar boat to the majority of people out there.

The Practices

Over the past 20 years I have honed my estimating skills with the experiences on projects. Over the past 6 years, this estimating has been predominantly for Agile projects. Although we have not been perfect in my estimating, I am quite proud of our track record and feel that we have been correct far more than we have been incorrect. Where we have been incorrect, we have augmented the estimating process so we won’t be incorrect in that way again.

The practices are not in order of priority as I had a real problem trying to do that as I think they are all important. So without further ado:

1) Remember to estimate Project Management and Technical Management

We frequently ran across missing these factors on early estimating attempts. This is usually done by everyone as they work on estimating as the work is easy to forget. It is a critical part of the estimate to ensure that these estimates are included so that we don’t scrimp on these tasks and cause larger issues for the project.

2) Don’t plan on your Technical or Application Architects being any more than 20% on construction tasks

This one continues to plague projects I estimate. No matter what the work is, the Architects will always want to work on their fair share of construction tasks. It is a noble desire and will return great benefit if they can work more on construction tasks. But I wouldn’t bet on it. In fact, I would even consider reducing this to zero depending on team size and complexity of the solution. If the Architects can work on more construction tasks, just consider that a bonus.

3) Remember to estimate for meetings and other factors in your schedule estimate

OK, now you have the estimates so lets just plan the hours out on the calendar right? Not quite. Remember to reserve some time every day for meetings, stand ups, and other non-construction activities. In addition, remember to plan for at least some time for vacations, holidays, and sick time. Use your statistics from your company to generate a rough factor for the project. It will just reserve a percentage of your schedule at the start, but then you will have the wiggle room when people start dropping like flies during flu season.

4) Hold a risk workshop and include the weighted estimates in the overall estimate

Mention Risk Workshops and most people yawn before you have completed the sentence. But they have great value. They are a great team building exercise at the start of the project as well. But ultimately, the output from the Risk Workshop should be used to generate an estimate and apply that estimate to the overall project estimate. By calculating the impact and probability of Risks occurring you will generate an estimate of the Risks that will likely occur. My experience is that if you don’t include these estimate, you will frequently address these risks occurring with Change Requests. The project can be much smoother if the estimate are included up front and you don’t surprise the clients. Yes, we won’t know all the Risks at the start, but it is better than assuming there are no risks at all. As Dr. Phil says “How is not estimating any Risks at all working for you?”

5) Estimate at multiple levels and triangulate

Probably the most important practice. We never just create one estimate. We typically estimate bottom-up (object level), deliverable (mid level), and top-down (schedule level). We use metrics for the bottom up and mid level estimates. (how long does a simple CRUD screen take, moderate report, etc..) Then we compare all three estimates and rationalize the differences. It is amazing how this rationalization uncovers estimating oversights.

We also then use Planning Poker sessions as we execute the project to confirm requirements and plan our iterations.

6) Proactively get information required to provide an estimate

Many times there are factors that may cause the estimate to change greatly. (Data Conversions, Legacy interfaces, unknown technology) Ask these questions early on and factor them into your estimates.

7) Say No when you don’t have information to provide an estimate. But be proactive and say what you need to provide an estimate.

If you don’t have the required information to properly estimate, then don’t. 🙂 But don’t just say you can’t estimate. Provide the details of what information you need to be able to provide an estimate. I know on past projects we have guessed many times on estimates where in hindsight, the client would have gladly provided all the information we required.

8) Track estimates and actuals and use the metrics for future estimating. (both within the project and for future projects)

Yep. We need to track estimates. Sorry. These metrics will then be used to determine on average how long certain objects will take and the percentage that certain areas of the project will require. (Like Project Management!)

We also hold retrospectives and review iteration estimates and actuals so that we can revise our estimates as soon as possible within the project and taken them into consideration for future iterations.

9) Factor educated contingency into your estimates.

We have 17 contingency factors (and counting) that are weighted and applied. (at the team’s discretion) If nothing else, it is a least a checkpoint for the team to think about whether certain situations exist that may affect estimating. (new team? data conversion? new technology? complex interfaces?) Don’t just apply a standard 10% blindly to estimates. It has to be based on reality.

Like Risk, these contingency factors should then we applied to the estimate. I would also recommend that you don’t show what part of the estimate is Risk versus Contingency versus base estimate. Clients will then say that the contingencies won’t happen so just subtract that estimate. <sigh> The estimate is holistic.

10) Execute in a manner that respects that estimates are not actuals. They will be incorrect.

And after all the estimates are done, don’t execute in a manner that makes Captain Bligh look like a level fellow. Don’t micro-manage, realize that these were only estimates – some will be higher and some will be lower. Very rarely will any be totally correct. Let the clients trade requirements and stories as priorities change! You could not possibly know all the scope up front. But also don’t just agree to add new scope. A lot of issues about estimating do come down to good old-fashioned scope control. The estimate should be a guide and a placeholder for a discussion, not a roadmap.

Summary

My experience is that is we do all of these practices, our estimates will still be wrong. But much less wrong. More importantly, these practices help us to discover better ways of estimating. Discovering better ways doesn’t only apply to development practices, but all project practices. Ultimately these estimates also help the team members to feel better about their work as well. No one like to miss estimates and managing projects in a manner that respects that the estimates are not actuals will alleviate many of the negative connotations with estimates.

<SoapboxMode>

The correct thing is to fix how the estimates are managed, not to stop doing estimates.

</SoapboxMode>

P.S. I forgot one! Here is a bonus practice 11) Estimate that you will be wrong – remember to estimate for refactoring, defect fixing, and some rework. We are all good, but not that good. 🙂

 

#Agile’s Sub-Optimal Value Stream

As I review proposals from people in the Agile community to push the boundaries of Agile, I believe that Agile is in danger of becoming more and more Sub-Optimal.

Before I proceed, let me just review my definition of Optimal and Sub-Optimal Systems:

“An Optimal system is a system where the optimization of the system or the elimination of waste has been undertaken based upon the value stream of the entire system”

“A Sub Optimal system is a system where the optimization of the system or the elimination of waste has been undertaken based upon the value stream of part of the system. These efforts may actually cause the entire system to be less efficient”

Project Value Stream

Agile has rightly focused on the Project Value Stream and I don’t believe anyone would argue that the Project Value Stream is maximized when executing the project in an Agile manner. But by optimizing ONLY the Project Value Stream, have we sacrificed other Value Streams?

What about the value external to the project? What about the Program and Company value?

Program Value Stream

If I follow Agile by the consensus of opinions I hear proposed most frequently, the following two types of project information are lacking:

Project Memory – Information about how the project was planned and executed.

Solution Memory – Information about the technical or functional solution for the project.

If I use User Stories on stickies, manage via a Kan Ban board, and my retrospectives, application and automated test cases are primarily my documentation at the end, can I answer the following questions?

  1. How can I search prior projects for leverage opportunities? Must I read through the code and tests and somehow determine leverage opportunities?
  2. How can I review prior projects history for a history of how estimating worked so my estimates can be better this time?
  3. Since we acknowledge that I there is never enough time to create all the automated test cases, how do I know if a new requested change is implementing behaviour that is beyond the intent of the application and will cause downstream effects that I can’t anticipate because we don’t have an automated test to ensure its correctness currently.
  4. How can I ensure my solutions are consistent across projects if they don’t have high level knowledge of each other?
  5. Do I have adequate documentation to maintain these applications for the next 10 years if I have a complete turnover in employees?
  6. How can I manage a department where the reality is that many people will not be able to be dedicated for the lifetime of the project? Do I require more documentation in a situation like this?

By it’s very nature, Agile Software Development projects are silo-ed projects with primarily full-time members and with little governance breaking down the barriers between projects. This causes an even greater need for attention to the Program Value Stream.

Company Value Stream

Perhaps one of the most troubling trends lately is the statement that estimates should not be created anymore as they are bound to be wrong and a waste of time. It is proposed that the project should be started and after two or three iterations the project will be able to confirm the velocity and inform the client as to an accurate estimate on what it will take to complete the project.

This statement is from the point of view of the project entirely. Every project that starts needs to be backed with a business case that confirms that for $X I will return value Y and that is acceptable to the business. If we are not estimating projects anymore, we run the risk of initiating the wrong projects and wasting project money until we determine that the project will either spend too much money or return too little value.

Stating we don’t need to estimate is ludicrous if we truly care about the long-term viability of the company. Implied in my statement is my belief that User Story Points must be translated to hours. In discussions I have had with developers, they have asked for the hours for each User Story as a means for ensuring they were on track. (when I didn’t provide it, they did the math themselves)

Although I like the use of User Stories and User Story Points to estimate, if we do not translate the User Story Points to hours we are Sub-Optimizing. We are placing the development process ahead of meeting the business case. By not giving the developers the context of the estimates in relation to the project’s business case, we are creating a layer of isolation between the business case and the programmers. We are making the project iterations more efficient, but also possibly sacrificing the ability to meet the business case.

And isn’t the business case the reason the project exists?

Three ways to ensure your Agile Project is not Sub-Optimal

  1. Estimate your project – Make sure you estimate your project and ensure that estimate is incorporated into the Business Case to validate the project raison d’etre. These estimates must be given the care and attention they deserve and not just estimates for the sake of getting the project approved. My recommendation for how to estimate an Agile project is a topic for another day.
  2. Create Lightweight Solution Architecture Deliverables – Ensure that your project defines the Solution Architecture at a high level and everyone shares that vision of the solution. These deliverables can then be used for project governance and for ensuring consistency and leverage opportunities across projects in a program.
  3. Translate User Story estimates to hours and track your time – User Stories being translated to hours sets the duration expectation with developers and also provides context back to the business case to ensure we are on track. Tracking time, the bane of everyone’s daily work life actually does have great value for the program and company. Although knowing a project velocity is a great predictor and planning tool, it doesn’t let me know the following:
  • What types of work do we estimate poorly or take longer than expected?
  • What types of work do we estimate well or take less time than expected?
  • What types of work do we forget to estimate?
  • What issues do we encounter that add to the time in take to complete a task?

Summary

“Are we sacrificing long-term Enterprise objectives by developing projects in an Agile way?” I don’t believe that anyone argues that Agile is not the best way to execute a project. But I do think that sometimes Agile’s sole focus on value for the project and the client within the project may cause us to lose focus on the value the Program and the Company requires.

We need to ensure we keep the entire value stream in mind when we are optimizing processes. Otherwise we will have a string of very successful projects at a number of failed companies.