SPACE Metrics: What They Are and Why They're Important

Many engineering leaders are already familiar with DORA (DevOps Research and Assessment program) Metrics. These metrics help engineering leaders track and measure the performance (i.e. the development velocity and quality of their organization). DORA suggests four primary metrics, including Software Delivery, Lead Time and Deployment Frequency, to measure the Dev Velocity, and Mean Time to Restore (MTTR) and Change Failure Percentage to track the output quality.

But simply measuring the output of the teams is not enough. To improve the output, you must also understand and optimize the multitude of factors that potentially impact developer productivity. Other metrics (e.g. Business Alignment, Developer Happiness, Motivation, etc.) are often used in tandem with DORA for complete coverage. SPACE metrics strive for a more holistic approach to measuring Dev Productivity. They seek to close the gap left by DORA in the human and emotional aspect of dev teams. 

What is the SPACE framework?

The concept of SPACE to measure engineering efficiency was developed by researchers at GitHub, the University of Victoria, and Microsoft. The intention is to capture a 360 view of developer productivity.

SPACE dispels pervasive myths around tech production that have been prevailing even after decades of research and practical development experience, namely that it can be measured by using siloed windows into individual performance. It moves aways from measuring development productivity with simple metrics or even worse, trying to encapsulate it with “one metric that matters”, and It embraces different factors that impact productivity at the individual, team, and organizational levels. SPACE summarizes 5 different dimensions for Productivity.

1. Satisfaction and Well-being 

Satisfaction is a measurement of how fulfilled developers feel with their work, team, tools, and culture at a high level. Well-being (how happy and healthy workers are) goes hand-in-hand with this. More often than not, productivity and satisfaction are positively correlated. 

A familiar example to most of us - many companies noted an increase in productivity (measured by the number of pull requests, tickets closed, or other project tracking metrics) after the pandemic forced workers to use a remote model. This increase makes sense - many studies have found that remote workers are significantly more likely to report being satisfied at their jobs. 

However, this quantitative data is not necessarily an accurate representation of these same employees' well-being. After all, many people were struggling with their mental and physical health while quarantined. Thus, take caution in using quantitative productivity measurements without considering the context of developer well-being.  Satisfaction and Well-being can be measured in four aspects: 

Satisfaction: Are they happy with the current processes in place 

Efficacy: Do they feel they have appropriate tools to do their job

Engagement: Do they feel engaged in their jobs

Burnout: Whether they feel affected by exhaustion or prolonged stress

Developer happiness can be impacted considerably by many different variables. Here are some ways you can help.

‍
Reduce Mundane Process Overhead through Automation

The SDLC requires the execution of several mundane yet crucial tasks, which can cause process overhead. Missing any of the steps can stall the SDLC process. A well-designed automation to detect missed steps and gently “nudge” the developer enables action without any human judgment.

Provide Public Recognition

Many developers, especially in today’s remote working situations, have an emotional need to be recognized for their efforts. Measure and recognize developers’ accomplishments with as little subjective bias as possible and ensure kudos are public and equitable.

Task Diversity

Repeating the same type of work leads to fatigue and unhappiness. Balancing developer tasks between new feature development, bug fixes, peer-reviewing code, and handling escalations. Variety of tasks rejuvenate the developers’ interest and stem unhappiness.

Regular Conversations

Have regular 1:1 conversations to gauge individual sentiment. Consider providing an extra mental health day to avoid burnout, or invest in new tooling if efficacy is an issue.

Surveys

Surveys provide very valuable insights that tooling and telemetry cannot provide. Surveys can provide more qualitative information on whether developers feel that they have the right resources and tools to do their jobs. Surveys can give a better sense of how developers are feeling in their roles and how satisfied they are.

2. Performance

Performance is a metric many of us are already familiar with. SPACE metrics, however, broaden the traditional definition of "performance", or delivery speed. An individual's performance becomes less relevant. What truly matters is the sum of all individual contributions and the impact of the outcome rather than the quantity of output.Performance then becomes a combination of Velocity, Quality and Business Impact.


The four DORA metrics provide a good gauge of the development velocity and quality. They are a good set of North Star metrics for engineering leaders to measure the tempo, rhythm and responsiveness of their organization. These metrics are:

Velocity: Speed of new releases, frequent updates, etc

• Software Delivery Lead Time (also known as Lead Time for Changes): time from when the development team starts working on a task to the time the customer gets the feature.
• Deployment Frequency: Number of times code or software is deployed to production or “shipped”.

Quality: Reliability, absence of bugs, maintainability and ongoing health

• Mean Time To Restore (MTTR) : time it takes to restore a failure in production.
• Change Failure Percentage: the percentage of deployment causing a failure in production. 

Business Value: Customer satisfaction, adoption, retention, usage and cost reduction

• Customer Escalations
• Customer Satisfaction 
• Net Promoter Score
• Customer Churn & Retention

For developers, this directly translates to feedback. Understanding the impact of their work is deeply satisfying to developers. Product Management is ideally suited to provide feedback and kudos on how developers work has benefited customers. It also serves as self-motivation for the next task/project.

3. Activity

The third SPACE metric is activity, a quantitative measurement determined by the count of actions or outputs completed in a predetermined unit of time. If measured correctly, a team's activity can provide valuable insights into the efficiency of the systems in place, both technical and non-technical.

One of the most considerable complications in accurately capturing a developer’s activity is that it is nearly impossible to be comprehensive. After all, not all developer activity can be quantified. Some activities that can, however, be easily measured are:

Design and coding-related activity

• Volume or count of design documents and specs, work items, pull requests, commits, PR reviews as well as PRs Reviewed, story points delivered, epics closed, etc.

CI-CD activity

• Total count of builds, tests, deployments/releases, and infrastructure utilization.

Operational activity

• Volume of incidents/issues based on their severity including on-call participation and incident mitigation.

Activity can be planned to a certain extent, but it’s difficult to plan for every task that must be accomplished in a sprint – particularly when production issues arise. To avoid ad-hoc issues as much as possible, however, be prepared to put more time into your sprint planning 

It is important metrics be set in place to measure the success of the Sprint, as well as to determine any potential issues. The right metrics can point out issues or bottlenecks early on in the Sprint, so that those can be rectified quickly. Here are some considerations for Sprint Planning.

‍

Like all other metrics, it's strongly advised not to use any independent activity view in isolation from others. For example, should the amount of design and coding be lower than usual, this does not indicate that developer activity is down. Instead, it's likely the team used time in other areas of activity. While it's hardly possible to capture every effort in a sprint, do your best to be as comprehensive as possible. 

4. Collaboration and Communication 

Intuitively, Collaboration & Communication measure how well people and teams work together. To measure this properly requires a high level of transparency and awareness of team member activities, team dynamics, and task priorities.

For this, various activities can be measured such as:

• How does information flow within and across groups of people? 
• What is the quality and quantity of appropriate documentation? 
• Are developers waiting for others (PM, PR Reviews, CI-CD, Security, etc)
• Are there a lot of Bounce Backs (back-and-forth communication) 
• How is the collaboration between the teams? 
• Are there buddy systems in place for PR Reviews?
• What is the quality of PR reviews? 
• Are engineers set up and incentivized to review other’s code?
• What's the onboarding time for new team members?

In Dev teams, these communications will involve Product Management, Quality Assurance, and Security teams. This communication is important. But excessive or avoidable communication creates interruptions to work, so engineering leaders need to have processes and tools to have a balance. 

5. Efficiency and Flow

Engineers frequently describe their productivity as a "flow." Flow is the state of the brain when you are “in the zone” (i.e. extremely focused and productive). Interruptions when you are in Flow can be detrimental to productivity, and also cause irritation. Oftentimes, developers who report being unsatisfied with their work can link process overhead, interruptions, or communication overload to the root cause. 

This SPACE metric Efficiency & Flow attempts to capture the ability to complete work or make progress with minimal interruptions and delays. 

Engineering leaders must ensure that their teams are able to get an adequate amount of Sustained Maker Time. Maker time refers to the time the developer is able to spend coding and creating software. A fragmented coding time interrupted by non-coding tasks  causes aggravation and frustration leading to unhappiness.  Engineering leadership must create conditions that reduce interruption by non-coding tasks, context switching, back and forth across multiple tools to complete their work. This can include:

• Checks & Balances in place to ensure that the team has everything they need to get started - for example, detailed requirements from PMs
• Tools that are integrated with their existing toolset, i.e. their existing IDEs, CI-CD pipeline or communication tools like Slack 
• Processes in place to review requests from Customer Escalations to sift out escalations that are not code related but related to documentation, configuration, etc.
• Automation to make nudges and reminders system generated so developers don't have to get out of their Flow to interact with others

Conclusion

The SPACE framework is more than just a set of metrics - it provides a holistic view into the state of the engineering organization. It takes into account the well-being of the individuals, as well as the overall satisfaction with the company and their role. It measures the various dimensions across teams, individuals and the organization and takes into account the collaboration, communication and overall productivity.

The SPACE framework provides engineering leaders data-led insights into overall developer happiness and productivity. With the right set of metrics, engineering leaders can build more sustainable and balanced practices for their Dev organizations. 

‍