Measuring & Maximizing Developer Productivity

It seems that nowadays, everything is measured - and it’s no different for software engineers. Team members are now assessed by productivity metrics such as lines of code, number of commits, pull requests, code reviews, and in Continuous Integration (CI - such as number of builds/tests) and Continuous Deployment (CD - such as number of deployments/releases). Almost anything can be measured - you won’t be looking at a single metric to gauge productivity - it’ll be a whole slew of ‘em!

The focus is shifting clearly to remove all possible distractions and interruptions and getting the most out of a developer's time. But the issue is complex - there are micromanagement implications when performance gets reviewed so critically, and there are certainly right and wrong ways to measure developer productivity. In this article, we’ll go over what developer productivity is, how the SPACE framework can help developers, ways to improve productivity, and how we, at Harness, measure our software development teams’ productivity. Let’s dig in!

What Is Developer Productivity?

Developer productivity is, quite simply, how productive a developer is in any given timeframe/metric. A company would create objectives/metrics they want to track (for example, bugs fixed and code reviews done) and get a baseline of what’s acceptable. Then, developers would be assessed based on those results to get a good gauge on their productivity. If Ian and Mark are deploying 5 times per day, why is Steve only deploying 3 times? Could there be inefficient processes at the company level, behaviors that could be addressed, or productivity tools that could be implemented to help Steve? 

In addition to these metrics, there are multiple ways you can measure a software developer’s productivity by employing different frameworks - for example, one could use the SPACE framework, which we’ll go over below, or one could use OKRs.

Ah, OKRs. There are two issues we see repeatedly with those. 

Firstly, let’s just throw it out there: technical debt is sadly a problem at most companies. We’ve seen countless examples of new features being the main, well, feature of OKRs. Technical debt gets left behind, slowly piling up and skyrocketing developers’ blood pressure. If you want to do OKRs the right way, allow time and space for technical debt remediation/elimination. 

Secondly, with many companies wanting to move as fast as Google and Microsoft without actually being a Google or a Microsoft, a lack of resources and proper tooling can put immense pressure on developers and DevOps teams. It’s ok to not be a Google. Scale as you go. Don’t become part of the ‘churn and burn’ problem amongst software engineers!

What is SPACE?

Developer productivity isn’t something that can be defined by a single metric. For example, if you tried to evaluate developer productivity using just the number of outputs per developer, you would miss crucial context. Maybe that developer is working later hours, maybe the outputs aren’t high quality, maybe the outputs are simple features with low customer value. In order to get a clear picture of productivity, several different story points need to be mapped. Nicole Forsgren, along with other researchers from GitHub and Microsoft, tackled this issue by creating the SPACE framework:

S: Satisfaction and well-being

P: Performance

A: Activity

C: Communication and collaboration

E: Efficiency and flow

This framework helps provide a holistic view of developer productivity and gives managers the tools to evaluate changes. 

Satisfaction and Well-Being

Developers are, first and foremost, people. Their productivity will be affected by how fulfilled their work, team, and culture makes them feel. Taking it one step further, someone’s productivity will be affected by their overall happiness and health. 

To evaluate satisfaction and well-being, it's important to understand what matters to developers. Working on impactful projects, seeing code being used by customers, and working with colleagues that help them feel inspired, are all examples of things that can lead to more satisfaction in the workplace.

Some key measurements include:

• Employee Satisfaction: How satisfied developers are with their work.
• Developer Efficacy: Access to the tools and resources needed to complete their job. 
• Burnout: Exhaustion related to work activities.  

Performance

Developer performance is the outcome of their work. This can be difficult to measure because business outcomes often can’t be drawn directly back to developer output. Different factors like sales, marketing, and customer satisfaction come into play with business outcomes. Because of this, the outcomes that are focused on need to be both business and developer specific. 

Developer Specific: quality, reliability, absence of bugs, overall service health.

Business Specific: customer satisfaction, adoption, feature usage, cost reduction.

Activity

Developer activity is the count of actions or outputs performed during work. There are definitely insights to be gained here, but they might be more limited. For instance, it is virtually impossible to measure all the actions a developer takes in a day, and it's difficult to determine if those actions were of high quality. 

Examples of measurable activity include:

• Volume or count of design docs and specs, pull requests, commits, code reviews, build counts, deployment frequency, incident mitigation, etc. 

Communication and Collaboration

Developers need to communicate and collaborate effectively in order to produce the highest quality productivity. Software development is inherently a creative process that requires constant feedback and ideation from colleagues. Communication and collaboration can also play a big role in overall developer satisfaction. 

Companies can try to assess communication and collaboration by looking at developer onboarding experience, qualifications of reviews, and seeing how connected developers are with each other. 

Efficiency and Flow

Developer efficiency and flow is a reference to how much progress a developer can make continuously without being interrupted. We’ve all either experienced or heard about “getting into a state of flow.” Getting into a flow state requires the space to consecutively do work with minimal distractions. 

In addition to getting into a flow state, the flow of information and knowledge is equally as important. Key talents can become lost in knowledge silos that hamper a company's ability to perform basic tasks. An example would be a single developer with the knowledge of software delivery, operating in a silo giving everyone else zero context into the health of software delivery. 

Metrics that can be used to capture a picture of efficiency and flow include:

• Number of handoffs required in a process.
• Ability to quickly and efficiently complete work.
• Time required throughout a system, time between processes, etc. 

The Difficulty With Measuring Developer Impact & Productivity

Developers are always building and shipping new software. As with any facet of business, there needs to be productivity measures in place, and the same goes for the development side of the house. But how do you measure 1) the productivity level of a developer; and 2) the impact of what they build?

Measuring productivity level might come in a few dimensions, such as velocity, regressions, git commits, or planned releases on the schedule. Deciding what is the most appropriate way of doing it ends up coming down to what makes the most sense for the organization. So where does that objective come from?

Most often, the measurements stem from the tail-end of the process: impact. Theoretically, if we know what impact we want to create, we can back into how we measure productivity. And there’s the million dollar question: What is impact, and how do we measure it? 

For different organizations, impact can look very different. For a mission-critical API, it could be the SLA impact and making the infrastructure airtight; the productivity measure might be how often SLA breaches occur or improvements in the SLA number. For a SaaS product, it might be the level of customer satisfaction measured by the number of rollbacks required or how many features get shipped in a sprint.

Regardless of what needs to be measured, the fact remains that measurement needs to occur. And the difficulty in doing this really comes from the wide variety of things that could be important and the lack of standardization in software delivery.

Current Ways People Are Measuring Developer Productivity

As the old adage goes: You get out of a person what you measure them on. So getting the methods of measurement correct is important to create the right incentives for developers. Let’s look at some ways that organizations measure developer productivity today.

Measuring Individuals

This is the most obvious way to measure productivity, because we are looking at the individual developer. The biggest issue with doing it this way is a lack of standardization across developers - how do you measure everyone in the same way when they have different tasks?

Keep in mind that while individual measures themselves say something, they’re best looked at as a holistic picture. Beyond that, they also need to be tied to the individual contribution that the developer is tasked with on a team.

Here are some ways that individual developers can be (and are) measured:

• Number of commits.
• Average commit size.
• Time to review.
• Number of code reviews.
• Frequency of code reviews.
• ...and the list goes on.

Measure Teams, Not Individuals

Because individuals can have drastically different tasks on a team, it’s tough to measure each team member by their output. For example, there might be devs that ship new features, those that act as “janitors” and make sure things run smoothly, and those who may not code as much but whose inputs are vital to others doing their jobs well. Measuring by teams in this scenario makes much more sense.

Things that could be measured at the team level:

• Output per sprint relative to previous sprints.
• Number of JIRA tickets closed.
• Consistent production of useful software.
• “Velocity budget” based on tasks multiplied by complexity.

No Hard Measures

Some organizations thrive without any hard measures of developer productivity! The organizations that do have these measures oftentimes are tasked with creating as much efficiency as possible because of issues like engineering bloat. Other organizations tie engineering to product releases, or to customer satisfaction, or to high-level strategic business objectives.

Ways to Improve Developer Productivity

Let’s start with the chicken and the egg: The first step to improving developer productivity is measuring developer productivity. In all seriousness, though, there are a multitude of ways that developer productivity can be improved, both by the individual developer and by the organization.

Work Within Existing Workflows

Consider that developers already have to deal with multiple tools, languages, and frameworks to do the basics of their jobs. What happens when a new tool or a new requirement is added that they have to incorporate into their daily jobs? The answer is that productivity takes a hit.

By putting things into the existing tool stack or finding ways to integrate into that stack, it saves developers time and stress, allowing them to at the least maintain their productivity. And, depending on what’s being added, it could be a benefit to them!

Decouple From Other Developer Workstreams

Before any release is pushed to production, developers have to go through a set of verifications and tests to ensure that dependencies are met, no regressions happen, and much more. This is to be expected, because organizations want to push highly-performant code to production and ensure that end users don’t take a hit. 

However, this means that a developer saying a feature is “ready” doesn’t always really mean it’s ready. It can often mean that their branch is ready in isolation, or that they’ve run some tests, but if the whole suite of new features or changes isn’t ready to be tested against, it can slow down the process and hamper productivity. One solution is to implement feature flags, which allow developers essentially their own development lanes that can be decoupled from all the other changes that may come out in a release.

Focus On Developer Experience

Ultimately, it comes down to making the developer’s life easier. This can be through clever tooling or performance measures, and it can also be through finding ways to reduce their load or the stress that they’re under. Without getting into the psychology of it all, the basic premise is that a happier developer is a more productive developer.

In solving for developer experience, the aforementioned options to improve productivity are two questions that organizations should consider asking as a starting point. Does what is being added create value without requiring more effort?

How Harness Measures Developer Productivity

Here at Harness, we adhere to the fast-moving startup ideology: Shipping code fast while keeping the quality of the code high. To do so, we measure our developer productivity in a multitude of ways, with some key metrics being the number of features delivered per quarter, the number of lines of code per developer per month, and the number of sprint commits. This helps engineering managers be aware of the quantity of code that is produced via the development work, be able to see if the workload of the sprint is appropriate, and be aware if there are signs of burnout from the developers.

How Harness Can Improve Developer Productivity

The Harness platform was created with developer experience at the forefront. It was created as a simple, smart, and scalable tool that developers would want to use, therefore hopefully improving developer productivity. The CI/CD modules of the platform make it easy for developers to create pipelines that are essential for the delivery of code. They take away the toil of scripting pipelines and allow developers to use pipeline as code to configure these pipelines rather than writing thousands of lines of code. Harness allows the developers to focus on their actual jobs: Writing code. This helps their productivity because there is less context switching between the actual source code itself and the pipeline script - and there is less burnout from developers having to write and maintain these scripts. 

Also, within the Harness CI module, a custom dashboard feature was added to allow for transparency and for a visual representation of the commits, builds, runs, etc. that were created in the CI process. Managers can use this dashboard to measure developer productivity, and in turn, see the trends before burnout actually happens.

Conclusion

Developer productivity is a key metric to check the quality of the code as well as the levels of developer burnout. This allows managers to manage developers and make sure that their work is up to par - and to make sure there is retention within the workforce. Harness, like every other software company, measures developer productivity to ensure engineers are happy and satisfied. To promote Harness’ commitment to productivity, in the platform itself, there is a major focus on developer experience. Take a demo and let us prove it to you!