Data-Led Actionability for VP of Engineering
As engineering leaders. we always look for methods to improve our team's output and impact. One approach to achieve this is using data-led insights to guide our decisions. Data-led insights provide a quantifiable and objective way to track the team's impact, prioritize actions, make optimal decisions, and measure the post-decision impact in a consistent way. In this blog, we will discuss some key engineering metrics to track and the actionability that is driven by the data and insights from DevOps tools.
Actionability driven by data and insights is a key concern for engineering leaders at all levels. However, the specific actions engineering leaders take will vary widely depending on the level of the engineering leader. For example, a VP of engineering may be concerned with high-level metrics such as customer satisfaction or engineering efficiency. Meanwhile, a director or manager may focus more on data-led decisions such as estimating the risk of fixing specific bugs, reducing lead time, or optimizing specific processes.
Ultimately, actionability is determined by the specific goals and priorities of the engineering leader in question. This blog is intended for the VP of Engineering or Head of Engineering. However, there are some commonalities across all levels of engineering management. For example, all leaders need to effectively use data to prioritize their actions, ensure their data-led actions are aligned to business goals, and quantify and communicate the impact of actions to their teams, peers, and management.
For a VP of Engineering, actionability is making data-led decisions that have broad impact across their organization, with peer organizations, and is of concern to rest of business executives since their decisions either directly or indirectly impact them. The following five areas of data-led actionability have been commonly observed among highly successful engineering leaders.
Business Alignment: A basic metric for engineering leaders is alignment to the business. This means knowing how much effort is spent on new features, unplanned work, bug fixes and customer escalations. The actionability driven by this data is to balance and set the right targets for different teams, and monitor how effort is allocated against those targets. Too often, this exercise is left to quarterly business reviews (QBRs) and annual reviews where engineering leaders can only look back and wonder. A data-led approach enables engineering leaders to flip this paradigm. Going from a reactive approach to a more proactive or kinetic stance to actively manage the trade-offs month over month. Thus ensuring that the alignment goals are actually met.
Engineering Execution (DORA): DevOps Research and Assessment (DORA) metrics are considered the north star metrics for engineering team performance. One of the first tasks that data-led engineering leaders take on is to measure this metric. Yes, it is great to know that our Lead Time to Change is 23 days. Now what? Here the actionability on the VP of Engineering is manifold.
- Identify the most important metric to improve. This will largely depend on the business priorities for the company and could also vary by team or product.
- Set a course of action on how to improve these metrics. Sometimes it is a change to the development process, at other times it maybe improved communication and collaboration, or small additional investments in infrastructure. These should be conducted in discussion with Directors and Managers who manage their teams.
- Measure the impact of decisions and adjust as needed. Too often we make decisions without the ability to draw a quantifiable inference on the impact of the measure. This "action" by leaders not only ensures they hold themselves accountable for decisions and choices but also helps the articulate to executive management the improvements and changes they are making. Note that within an executive team, engineering leaders are surrounded by numbers-driven peers - CMO, CRO and CEO are all held accountable to delivering specific numbers. When engineering leaders present their impact in quantifiable and easily defensible ways, their seat and voice at the table takes on much more meaning and value.
Outsourced Team Contribution and Vendor Selection: It is common for engineering organizations to use one or more outsourced vendors for engineering work. One of the most critical decisions an engineering leader faces each year is which vendors to continue investing in and which vendors to drop. Today, this decision is largely based on gut, intuition, and lets face it - personal relationships. A data-led approach ensures that this critical action/decision is driven by objective metrics and is consistently applied across vendors.
Customer Satisfaction: This is a broad topic and has many aspects depending on the type of business. For engineering organizations and leaders, customer satisfaction is typically measured by production adoption, feedback, and escalations. Engineering leaders must not only measure the nature and frequency of critical customer escalations but also measure the escalations that were incorrectly sent to engineering organizations. Engineers being the most expensive resource in an organization, any avoidable escalation must be removed. Actionability here is understanding the frequency, severity, and categories/patterns of escalations and their trends. Leaders must put in place steps to reduce escalations and eliminate systemic issues from recurring.
Developer Happiness: Prior metrics were centered around aspects that are important to the business. Keeping developers happy and engaged is an objective that engineering leaders never lose sight of. Indeed there are methods to measure and quantify developer happiness. Broadly speaking these are divided into improving productivity - a productive developer is a happy developer - and certain human factors such as task variety that also impact productivity.
