Introducing Harness AI Productivity Insights

AI-based coding Assistants like Google Gemini Code Assist, GitHub Copilot, and others are becoming increasingly popular. However, the efficacy of these tools is still unknown. Engineering leaders want to understand how effective these tools are and how much they should invest in them. 

Harness AI Productivity Insights

Harness AI Productivity Insights is a new (beta) capability in Software Engineering Insights that helps engineering leaders understand the productivity gains unlocked by leveraging AI coding tools.

This targeted solution empowers engineering leaders to generate comprehensive comparison reports across diverse developer cohorts. It facilitates insightful analyses, such as evaluating the impact of AI Coding Tools on productivity by comparing developers who leverage these tools against those who don't. Additionally, it allows for comparisons between different points in time, tracking how developers' performance evolves as they adopt and grow their proficiency with AI Coding tools.

• Assess the productivity impact of AI Coding Tools used in your organization
• One-click integration with your Source Code Management systems to unlock metrics across velocity and quality 
• Productivity analysis across diverse developer cohorts
• Analyze productivity differences between developers utilizing AI Coding Tools and those who are not
• Conduct Developer Experience Surveys to capture qualitative data on the impact of AI Coding Tools

A Closer Look at AI Productivity Insights

Customers can choose different types of comparison reports. The most common reports are comparing cohorts of developers who use coding assistants and those who don’t. Other supported types of comparison reports include comparing cohorts of developers with different metadata, for example senior engineers versus junior engineers, or comparing the same set of developers at different points in time. 

For every report, customers can flexibly define the comparison cohorts either through manual selection or by utilizing existing metadata filters. 

Customers can run multiple reports at any time. Reports will be saved and available to share within the organization.

Each report analyzes the productivity scores of both cohorts, calculating the productivity gain of the second cohort relative to the first. The analysis encompasses various facets of performance, including velocity and quality metrics. Additionally, the solution offers the option to gather qualitative insights through surveys distributed to all cohort members, enriching the quantitative data with user feedback.

Data Sources

AI Productivity Insights relies on source code management (SCM) systems for metrics collection. Customers can seamlessly integrate their preferred SCM platforms through convenient one-click integrations. To gain insights into AI Coding Tool usage, the solution also offers one-click integrations with these tools, enabling comprehensive data collection and analysis across the development ecosystem.

Interested to learn more?

Let us know you are interested. We'd love to show you more and hear your feedback.