Many engineering leaders are already familiar with DORA (DevOps Research and Assessment program) Metrics. These metrics help engineering leaders track and measure the performance (i.e. the development velocity and quality of their organization). DORA suggests four primary metrics, including Software Delivery, Lead Time and Deployment Frequency, to measure the Dev Velocity, and Mean Time to Restore (MTTR) and Change Failure Percentage to track the output quality.
But simply measuring the output of the teams is not enough. To improve the output, you must also understand and optimize the multitude of factors that potentially impact developer productivity. Other metrics (e.g. Business Alignment, Developer Happiness, Motivation, etc.) are often used in tandem with DORA for complete coverage. SPACE metrics strive for a more holistic approach to measuring Dev Productivity. They seek to close the gap left by DORA in the human and emotional aspect of dev teams.
The concept of SPACE to measure engineering efficiency was developed by researchers at GitHub, the University of Victoria, and Microsoft. The intention is to capture a 360 view of developer productivity.
SPACE dispels pervasive myths around tech production that have been prevailing even after decades of research and practical development experience, namely that it can be measured by using siloed windows into individual performance. It moves aways from measuring development productivity with simple metrics or even worse, trying to encapsulate it with “one metric that matters”, and It embraces different factors that impact productivity at the individual, team, and organizational levels. SPACE summarizes 5 different dimensions for Productivity.
Satisfaction is a measurement of how fulfilled developers feel with their work, team, tools, and culture at a high level. Well-being (how happy and healthy workers are) goes hand-in-hand with this. More often than not, productivity and satisfaction are positively correlated.
A familiar example to most of us - many companies noted an increase in productivity (measured by the number of pull requests, tickets closed, or other project tracking metrics) after the pandemic forced workers to use a remote model. This increase makes sense - many studies have found that remote workers are significantly more likely to report being satisfied at their jobs.
However, this quantitative data is not necessarily an accurate representation of these same employees' well-being. After all, many people were struggling with their mental and physical health while quarantined. Thus, take caution in using quantitative productivity measurements without considering the context of developer well-being.
Satisfaction and Well-being can be measured in four aspects:
Satisfaction: Are they happy with the current processes in place
Efficacy: Do they feel they have appropriate tools to do their job
Engagement: Do they feel engaged in their jobs
Burnout: Whether they feel affected by exhaustion or prolonged stress
Developer happiness can be impacted considerably by many different variables. Here are some ways you can help.
Reduce Mundane Process Overhead through Automation
The SDLC requires the execution of several mundane yet crucial tasks, which can cause process overhead. Missing any of the steps can stall the SDLC process. A well-designed automation to detect missed steps and gently “nudge” the developer enables action without any human judgment.
Provide Public Recognition
Many developers, especially in today’s remote working situations, have an emotional need to be recognized for their efforts. Measure and recognize developers’ accomplishments with as little subjective bias as possible and ensure kudos are public and equitable.
Repeating the same type of work leads to fatigue and unhappiness. Balancing developer tasks between new feature development, bug fixes, peer-reviewing code, and handling escalations. Variety of tasks rejuvenate the developers’ interest and stem unhappiness.
Have regular 1:1 conversations to gauge individual sentiment. Consider providing an extra mental health day to avoid burnout, or invest in new tooling if efficacy is an issue.
Surveys provide very valuable insights that tooling and telemetry cannot provide. Surveys can provide more qualitative information on whether developers feel that they have the right resources and tools to do their jobs. Surveys can give a better sense of how developers are feeling in their roles and how satisfied they are.
Performance is a metric many of us are already familiar with. SPACE metrics, however, broaden the traditional definition of "performance", or delivery speed. An individual's performance becomes less relevant. What truly matters is the sum of all individual contributions and the impact of the outcome rather than the quantity of output.Performance then becomes a combination of Velocity, Quality and Business Impact.
The four DORA metrics provide a good gauge of the development velocity and quality. They are a good set of North Star metrics for engineering leaders to measure the tempo, rhythm and responsiveness of their organization. These metrics are:
Velocity: Speed of new releases, frequent updates, etc
Quality: Reliability, absence of bugs, maintainability and ongoing health
Business Value: Customer satisfaction, adoption, retention, usage and cost reduction
For developers, this directly translates to feedback. Understanding the impact of their work is deeply satisfying to developers. Product Management is ideally suited to provide feedback and kudos on how developers work has benefited customers. It also serves as self-motivation for the next task/project.
The third SPACE metric is activity, a quantitative measurement determined by the count of actions or outputs completed in a predetermined unit of time. If measured correctly, a team's activity can provide valuable insights into the efficiency of the systems in place, both technical and non-technical.
One of the most considerable complications in accurately capturing a developer’s activity is that it is nearly impossible to be comprehensive. After all, not all developer activity can be quantified. Some activities that can, however, be easily measured are:
Design and coding-related activity
Activity can be planned to a certain extent, but it’s difficult to plan for every task that must be accomplished in a sprint – particularly when production issues arise. To avoid ad-hoc issues as much as possible, however, be prepared to put more time into your sprint planning
It is important metrics be set in place to measure the success of the Sprint, as well as to determine any potential issues. The right metrics can point out issues or bottlenecks early on in the Sprint, so that those can be rectified quickly. Here are some considerations for Sprint Planning.
Like all other metrics, it's strongly advised not to use any independent activity view in isolation from others. For example, should the amount of design and coding be lower than usual, this does not indicate that developer activity is down. Instead, it's likely the team used time in other areas of activity. While it's hardly possible to capture every effort in a sprint, do your best to be as comprehensive as possible.
Intuitively, Collaboration & Communication measure how well people and teams work together. To measure this properly requires a high level of transparency and awareness of team member activities, team dynamics, and task priorities.
For this, various activities can be measured such as:
In Dev teams, these communications will involve Product Management, Quality Assurance, and Security teams. This communication is important. But excessive or avoidable communication creates interruptions to work, so engineering leaders need to have processes and tools to have a balance.
Engineers frequently describe their productivity as a "flow." Flow is the state of the brain when you are “in the zone” (i.e. extremely focused and productive). Interruptions when you are in Flow can be detrimental to productivity, and also cause irritation. Oftentimes, developers who report being unsatisfied with their work can link process overhead, interruptions, or communication overload to the root cause.
This SPACE metric Efficiency & Flow attempts to capture the ability to complete work or make progress with minimal interruptions and delays.
Engineering leaders must ensure that their teams are able to get an adequate amount of Sustained Maker Time. Maker time refers to the time the developer is able to spend coding and creating software. A fragmented coding time interrupted by non-coding tasks causes aggravation and frustration leading to unhappiness. Engineering leadership must create conditions that reduce interruption by non-coding tasks, context switching, back and forth across multiple tools to complete their work. This can include:
The SPACE framework is more than just a set of metrics - it provides a holistic view into the state of the engineering organization. It takes into account the well-being of the individuals, as well as the overall satisfaction with the company and their role. It measures the various dimensions across teams, individuals and the organization and takes into account the collaboration, communication and overall productivity.
The SPACE framework provides engineering leaders data-led insights into overall developer happiness and productivity. With the right set of metrics, engineering leaders can build more sustainable and balanced practices for their Dev organizations.
Enjoyed reading this blog post or have questions or feedback?
Share your thoughts by creating a new topic in the Harness community forum.