Agile Maturity and KPIs

The problem with Maturity Models

Maturity models can be useful to kick off a reflection on where your team or organization is right now on its agile path.

However, often times these models get confused with metrics. They provide a number which then suggests something got measured. In reality, though, it’s just a subjective value attached to a high level, abstract question like “On a scale from 1 to 5, how much is your HR a catalyst for agile transformation?”. Uh? 3…-ish?

Our approach is different.

First, in our understanding the word “agile” is only defined by the Agile Manifesto, i.e. its 4 value pairs and 12 principles.

Second, we ask our clients these quantitative questions, referencing the Agile Manifesto:

“Outcome (e.g. working & valuable software), delivered in short cycles”

  1. What is your project’s definition of “delivery”?
  2. When/how often has your project delivered working software in the last 3 months?
  3. Was it actually shipable at those times as well?
  4. What is the cycle time of a feature in your project (measured from “came up with idea” to “delivered”)?
  5. When/how often have you integrated (into) the full system in the last 3 months?
  6. How many pieces of actual user feedback have you received in the last 3 months?
  7. How many of these feedback items were turned into backlog items?

“Welcome Change”

  1. When did you introduce the last 3 changes in your project/product?
    How many where there in the last 3 months?
  2. How long did it take each of these changes from ideation to delivery?

“Face-to-Face, Business & Development collaboration”

  1. When/how often have developers and business people had face-to-face conversations in the last 4 weeks?
  2. When have developers collaborated with stakeholders, customers, business people, domain experts etc. outside their team in the last 3 months?

“Technical excellence”

  1. How do you assess the technical excellence in your project and what metrics/evidence is this based on?
  2. When you think of the last 3 changes (see above), how much time was spent on the actual implementation?
  3. Of that time spent, how much was spent on
    1. understanding the existing code base,
    2. refactoring the existing code,
    3. working around issues in the code base?

“Sustainability”

  1. What has been the weekly work time for people over the last 3 months?
  2. How many people on the team CANNOT be replaced by a successor or compensated for by the remaining team within 4 weeks?

“Self-organization”

  1. How many decisions has the team made in the last 3 months?
  2. How many of them were actually made by individual people, e.g. Scrum Master, Product Owner, Project Manager?
  3. How many of them had to be approved by someone outside the team before they became effective, e.g. the Line Manager, Director of XYZ, CEO etc?
  4. How many of them were opposing or enhancing company guidelines?
  5. How many of them were opposing the Product Owner’s / Project Manager’s, Line Manager’s etc. or any outside stakeholder’s opinion?

“Reflection”

  1. How often has the team met for reflection sessions in the last 3 months?
  2. Has the team or any outside stakeholder had the impression that the project was in a bad situation at least 1 time in the last 3 months?
    If yes, did the team meet for a reflection session / stop-the-line meeting outside their regular meeting schedule?
  3. How many and what changes to the way of working has the team tried in the last 3 months?
  4. How many proposed changes to the way of working were NOT tried in the last 3 months, e.g. they
    1. were turned down by the Product Owner or Scrum Master or an individual lead role on the team,
    2. were turned down by an outside stakeholder, e.g. Head of Department,
    3. were not pursued by the team because they feared rejection or wanted to avoid the need for outside approval,
    4. lacked the time, budget, resources?

Image source: Tumisu from Pixabay