Overview

Sensormatic Solutions is the leading global retail solutions portfolio of Johnson Controls. It optimizes retail operations at scale, while enabling smart and connected shopper engagement. Our intelligent digital operating platform, Sensormatic IQ, combines the full Sensormatic Solutions portfolio, including third-party data to deliver unmatched insights into shopper experience, inventory intelligence, loss prevention and operational excellence with advanced technologies like AI and Machine Learning. The platform empowers retailers to act confidently with prescriptive and predictive data-driven insights and outcomes, when making key business decisions. Sensormatic Solutions prioritizes responsible retail practices, leveraging innovative and sustainable technology to meet retailers' evolving needs.

One of the key objectives for Sensormatic Solutions was to enhance the overall quality of the product and establish appropriate key performance indicators (KPIs) and metrics to monitor the organization's progress in terms of quality and efficiency improvements.

‍

The Challenge 

Sensormatic Solutions employs a diverse range of tools and processes across its engineering teams. In early 2021, an internal dashboard solution was initially developed and proved effective for a specific team. However, scalability challenges were encountered when attempting to extend its usage across multiple teams and varied tool sets. This, combined with a lack of integrated insights, starting from ticket creation through coding and deployment, prevented a comprehensive view of the Software Development Life Cycle (SDLC) and hindered the team’s identification of bottlenecks.

The primary objectives of the Sensormatic Solutions engineering teams revolved around enhancing their delivery efficiency, quality, and cost-effectiveness. The organization sought comprehensive insights into its current quality status, process bottlenecks, change failure rates, and overall process hygiene. This necessitated the integration of insights from various tools utilized in its SDLC.

Furthermore, the teams were in the process of transitioning towards a unified, agile framework and desired a benchmark to assess its agile maturity, as well as a distinct framework for identifying process gaps, and effectively tracking adherence, to ensure successful agile implementation.

‍

The Solution

Sensormatic Solutions utilized the SEI (Software Engineering Insights) to gain valuable and unique insights into agile methodologies, quality enhancement, and efficiency optimization. By leveraging this resource, it was able to standardize and normalize these insights across various teams, enabling data-driven improvements across the organization.

‍

The Approach

DORA Benchmarking

Sensormatic Solutions recognized the importance of DevOps Research and Assessment (DORA) insights in achieving standardized Key Performance Indicators (KPIs). The organization prioritized obtaining insights from DORA, considering them essential elements for optimizing its software development processes.

Sensormatic Solutions strategically made DORA insights accessible at both the team and executive levels. This involved consolidating data from various teams and integrating Jira and Bitbucket with Harness SEI. The seamless integration allowed Sensormatic Solutions to gain out-of-the-box DORA insights, empowering all teams to comprehensively track four key metrics crucial for their DevOps journey:

• Lead Time for Change: The duration from the initiation of a change request to its implementation.
• Deployment Frequency: The frequency of deploying changes to production, a critical measure of agility.
• Mean Time to Recover (MTTR): The average time taken to recover from failures, a crucial metric for system resilience.
• Change Failure Rate: The percentage of changes that fail, impacting system stability.

Agile Standardization and Maturity

As Sensormatic Solutions embarked on bolstering its agile delivery process, a set of foundational metrics was introduced to guide teams towards standardization and maturity. These metrics, tracked monthly, aimed to elevate the organization's agile practices and ensure a consistent, high-quality delivery process.
‍

Hygiene Score for Process Conformance:

The hygiene score became a pivotal metric, assessing process conformance and maturity. Teams were scored based on their adherence to fundamental agile tenets, including:

• Acceptance Criteria for Stories: Ensuring that user stories had well-defined acceptance criteria, fostering clarity and understanding.
• Story Pointing: Accurately assigning story points to tasks, facilitating better planning and resource allocation.
• Story Right Sizing: Ensuring that stories were appropriately sized, preventing overcommitment, and optimizing delivery timelines.
• Ticket Assignment: Clearly defining and assigning tasks to team members, enhancing accountability and collaboration.
• Work Allocation: A refined process has been introduced to bolster work allocation reporting, providing valuable insights into feature development, technical debt management, and bug resolution. This systematic approach ensures alignment between allocated and actual percentages, serving as a critical checkpoint to maintain team accountability and project accuracy.

This quantified approach provided teams with a clear goal and a checklist, empowering them to actively work towards achieving and maintaining agile process maturity.
‍

Scope Creep Measurement and Threshold:

Recognizing the impact of ad hoc requests on engineering planning and delivery, Sensormatic Solutions  implemented a metric to measure and control scope creep.

A predefined threshold was set to ensure that ad hoc requests remained within acceptable limits. This strategic threshold aimed at maintaining consistency and safeguarding the quality of the delivery process.

By actively monitoring and limiting scope creep, Sensormatic Solutions aimed to foster a more controlled and predictable development environment, minimizing disruptions and ensuring the integrity of project timelines.
‍

Commit to Done:

Completing committed work would be a key indicator of a team's comprehensive understanding of their tasks and accurate estimation of capacity.

Teams were encouraged to signal the completion of committed tasks, indicating a thorough understanding of the work and an ability to estimate their capacity realistically. This metric aimed to enhance delivery consistency by ensuring teams fulfilled their commitments, contributing to a culture of accountability and reliability in the agile delivery process.
‍

Quality Scorecard

The team, recognizing the nuanced nature of software quality, delved into a meticulous exploration of multiple data points to construct a comprehensive quality scorecard. This scorecard comprised two sets of quality signals, capturing both leading and lagging indicators to provide a holistic view of the software development process.
‍

Leading Indicators: Predictors of Quality Pre-Deployment

• Initiative Hygiene or Process Hygiene: Assessment of the overall hygiene in the development process, ensuring that agile and development practices align with established standards.
• Coding Hygiene: Evaluation of the codebase for adherence to best practices, readability, and maintainability, fostering a foundation for robust software.
• Code Smells and Vulnerabilities: Identification and measurement of undesirable code patterns (code smells) and potential security vulnerabilities, mitigating risks early in the development lifecycle.
• Unit Test Coverage: Measurement of the percentage of code covered by unit tests, ensuring comprehensive testing and early detection of potential issues.
• Dev to QA Ratio for Features: Evaluation of the balance between development and quality assurance efforts, promoting an efficient workflow and minimizing bottlenecks.
  ‍

Lagging Indicators: Quality Implications Post-Deployment

• Defect Escape Rate: Calculation of the percentage of defects that escape detection during testing and manifest in the production environment, providing insights into the effectiveness of testing processes.
• Defect Escapes per Story Point: Measurement of the number of defects discovered in the production environment relative to the complexity of the implemented features, offering a refined understanding of the impact of defects on delivered functionality.
• Change Failure (DORA): Utilization of DORA (DevOps Research and Assessment) metrics as a lagging indicator, assessing the success or failure of changes post-deployment.

SEI Integration for Streamlined Insights and Reporting:

Leveraging Harness SEI, the team seamlessly integrated these quality signals, enabling the automatic generation of insights "out-of-the-box." This integration empowered Sensormatic Solutions to not only provide teams with valuable insights immediately, but also to ScoreCard each team based on these metrics. The scorecard, complete with Key Performance Indicators (KPIs), served as a tangible target for teams, fostering accountability and providing leadership with an executive dashboard for holistic quality assessment across all teams.

‍

Results

Improved Coding Hygiene

Facing a challenge with Peer Review approvals, Sensormatic Solutions employed the power of Harness SEI for a transformative solution. Initially, 35% of Peer Reviews lacked formal approval, posing risks to code quality and architecture. With SEI’s implementation, teams focused on establishing a robust Peer Review approval process.

Within months, Peer Review hygiene surged from 35% to an impressive 95%, while unapproved Peer reviews dwindled to just 5%. This turnaround not only mitigated the risk of merging suboptimal code, but also allowed early detection and resolution of architectural issues. The heightened attention to Peer Reviews instilled a proactive quality assurance culture, aligning with Sensormatic Solutions’ commitment to delivering best-in-class software products.

Agile Process Standardization

Implementing a standardized set of Agile Hygiene Criteria proved pivotal for Sensormatic Solutions, catalyzing a significant evolution in their agile processes. As the engineering teams embarked on this journey, the majority initially scored in the low 30s out of 100. However, in a matter of months, there was a remarkable surge, with most teams achieving scores in the 80s.

This leap in Agile Hygiene scores was indicative of teams successfully honing their agile process skillsets. Notable achievements included:

• Acceptance Criteria Enhancement: Streamlined communication between Development and QA teams by reducing unnecessary back-and-forth. Well-defined acceptance criteria led to a shared understanding, minimizing ambiguities, and optimizing workflow.‍
• Consistent Story Points: The establishment of consistent story points facilitated uniformity in planning and delivery. Teams experienced enhanced predictability and efficiency in their agile workflows.‍
• Done to Commit Ratio Exceeding 75%: Adherence to process hygiene translated into a remarkable improvement in the Done to Commit ratio across all teams. This surge to over 75% indicated accelerated and more consistent product shipments.

The disciplined focus on Agile Hygiene Criteria not only standardized processes but also catalyzed a cultural shift, fostering faster and more reliable product deliveries. Sensormatic Solutions’ journey exemplifies how strategic process standardization can elevate agile practices, leading to enhanced team efficiency and consistent project success.
‍

Lead Time Efficiency Gains

During DORA benchmarking at Sensormatic Solutions, inefficiencies surfaced in the SDLC process, specifically in the testing phase. Notably, testing time per feature emerged as a critical bottleneck. To address this, the team implemented a strategic approach, combining efforts in coding hygiene with a dedicated focus on automating testing processes.

The outcome was a substantial 30% reduction in testing time, resulting in a corresponding lead time gain for shipping features. Crucially, this efficiency enhancement occurred without compromising product quality. The streamlined testing process not only accelerated feature deployment but also underscored Sensormatic Solutions' commitment to delivering high-quality software.
‍

Quality Benchmarking

Armed with well-defined leading and lagging indicators for quality, Sensormatic Solutions implemented a robust quality benchmarking initiative. The organization successfully introduced a standardized Quality ScoreCard, meticulously assessing each team on a scale of 1-100 across various quality dimensions. This comprehensive evaluation extended to both leading indicators, reflecting pre-deployment processes, and lagging indicators, gauging post-deployment outcomes.

The standardized executive reporting, facilitated by the quality scorecard, empowered leadership with a clear overview of each team's performance. This not only streamlined decision-making processes but also enabled teams to pinpoint specific areas for immediate improvement based on their scores. The scorecard served as a dynamic tool for adopting targeted quality initiatives, fostering a culture of continuous improvement across the organization.
‍

Investment Analysis

With the introduction of a cohesive software delivery process, Sensormatic Solutions achieved a profound understanding of its investment priorities, categorized into growth, sustainability, and efficiency. This newfound clarity empowered the team to enact necessary changes based on actionable insights derived from the process.

Harnessing the capabilities of the SEI platform, Sensormatic Solutions delved into the intricacies of each module within the software delivery process, encompassing workshop, discovery, development, testing, and deployment phases. The platform played a pivotal role in gauging the punctuality of deliverables, rendering teams accountable and instilling a sense of ownership across the entire development lifecycle.

The data-driven approach facilitated not only the celebration of successes,  it focused attention to areas requiring improvement. Sensormatic Solutions' strategic utilization of the SEI platform underscores its commitment to efficiency, adaptability, and continuous improvement in the realm of software delivery.

‍

Conclusion: Elevating Software Delivery Excellence

Sensormatic Solutions' journey with Harness SEI exemplifies a commitment to excellence in software delivery. From enhancing agile processes to streamlining testing, and fostering coding best practices, the transformative impact is evident.

The organization's adept use of SEI insights for investment prioritization, coupled with a unified software delivery approach, demonstrates a forward-thinking stance on growth, sustainability, and efficiency that will positively impact its employees and customers going forward.

Teams, armed with punctuality metrics and a sense of ownership, are celebrating their successes and continuing to tackle areas for improvement with haste and precision.