When managing developer organizations focused on complex financial applications, the stakes are high, and mistakes are costly. In this environment, conventional productivity metrics such as lines of code committed tend to miss the point. Instead, developers need a mix of autonomy and accountability. That qualitative approach creates higher-quality results. In this Q&A, Arcesium’s Matt Katz shares his view on best practices for development in a rapidly changing environment.

Q1: When managing complex financial applications, how should we define developer success and move beyond ineffective traditional metrics?

From a leadership perspective, the best approach for managing complex financial applications is to focus more on outcomes from teams than on individual developers.

A naive approach to productivity focuses on individual developers at a personal level. There really are outstanding and underwhelming individual contributors — but most outcomes have to do with the performance of a team. Focusing on team or organizational productivity emphasizes the outcomes that matter to clients or stakeholders and ultimately impact the bottom line.

The worst metric that you could use would be lines of code committed. For example, a developer might submit an outstanding merge request that consists entirely of deletions. They figured out a way to remove an entire section of code that was no longer needed. Based on lines of code committed, they seem negatively productive — but if all code has bugs then they have simplified your technical debt.

In general, look to outcomes. How long does it take to go from an idea or request to a return on investment for that request? How many critical support requests does a team field after a deployment? How many alerts does an application generate?

Along the way, use qualitative measures instead. They help teams self-identify what's blocking them from getting a return, such as organizational process issues, like being bottlenecked on senior developers for solution designs and code reviews. What does the team think is most important? Does your team think they are fast? Do they think they uphold high standards? Do they think they provide seamless change? Then ask if they think they are meeting what’s important. Would their stakeholders agree on these goals? These seem like simple questions, but they are often unasked, and the answers are assumed by everyone in a frustrating environment.

Q2: Why is tailoring management practices to the context of the development group essential for high-stakes environments, instead of using a rigid, organization-wide standard?

The working situation or the working context of different development groups requires different practices. For instance, the infrastructure layer supports all the other applications, so the kind of testing and the level of provability that we need are extraordinarily high. The stakes raise the level of rigor required from infrastructure team members, and the need to conduct experiments before implementing sweeping changes.

Up a level at the application layer, a rolling deployment could be more appropriate. After you have adequately tested it, you can deploy to a subset of users to check, and based on feedback, you can roll it out further. Of course, you don’t want to wait too long to fully deploy or fully roll back because that increases your support risks and complexity.

Ultimately, for highly targeted custom solutions, tailoring to an individual client or specific use case is the most crucial aspect. When the potential overall impacts are low, we can accept that change has a limited “blast radius.” That mindset lets us concentrate on speed.

So, a primary goal is not to have an overarching idea that “this is the one way across the whole org.” Teams need room to define their strategy, given the customer's specific needs for their application. Then you can measure against those outcomes to determine whether they have performed well or have room for improvement.

Q3: How does fostering developer autonomy and responsibility lead to better organizational results compared with relying on top-down control?

As a technology leader, you must pay attention to both outputs and outcomes, while also being responsible for career development. People are happy and productive when they experience autonomy, growth, and mastery. Developers want to be able to make decisions and be effective in them. They want to grow and improve, and they strive for mastery in a specific domain.

I love checklists with all my heart, but sometimes these turn into “check-the-box” exercises that obscure responsibility for outcomes. The best kind of checklist is one that people write for themselves and buy into — a technology leader helps them think of what they need to cover in that checklist.

Organizations have the same goals. When you implement top-down control instead of aligning incentives within organizations, you end up losing the context of why teams and individuals are doing what they do.

The alternative works better. Wherever you can say, "I'm going to align your incentives, and I'm going to allow you to confront your direct circumstances and be responsible for your outcomes in a way that you can understand what changes you need to make," you find that people on the ground make fairly smart moves. You can assist development groups rather than dictate to them.

Organizations also find success because having development teams empowered to make changes, pursue goals, and self-organize is more effective than navigating extensive bureaucracy. This approach is faster, leading to results and incremental returns on investment as they develop. It also encourages diversity of thought, which translates to strength, as you see many different ways to solve problems. Best practices tend to percolate throughout the organization.

Q4: What measurable, outcome-based metrics truly support high-stakes development, and how do they encourage responsibility?

Metrics that empower a development organization are outcome-focused and contextual. For instance, you can measure things like how long it takes to address a bug between the time it’s reported and the time that you can release a change. How long does it take for a user to achieve a goal dependent on the application? Similarly, you can consider the time it should take to recover from an incident or outage.

When you use that outcome-based approach with metrics, you can track it to see when teams are doing the right thing and when they aren’t hitting their mark and need help.

Another effective technique is to provide people with a budget. They can define what good looks like within a budget of any kind, not just financial resources. How many alerts should a good application raise in a week? If an application raises lots of alerts that on-call staff are just bulk closing, an alert budget helps focus the team to understand what truly needs an alert versus what’s something that could be reviewed monthly. It helps a team know if they need to prioritize resiliency, observability, or technical debt. When you provide people with that kind of context and a budget or goal, they usually rise to meet it.

Q5: How do you balance individual responsibility with high-reliability development environments that require coordination with external financial entities?

The question posits a tension or tradeoff that is assumed but is not real. In highly integrated development environments, the two key factors are that developers have context, and the organization has context and expertise in its work. There is no substitute for having the context, of understanding the expectations of counterparties and clients, as well as understanding what the application does.

In areas like investment management, context and domain expertise add significant value. Development isn't just a generic activity. If you treat your development as widget production, you will find lots of failure cases where things seem good enough but are not. Knowing that a counterparty can accept amendments or needs a full replacement snapshot makes a lot of difference to how you produce data or track changes. Knowing the industry and players in the domain allows you to move quickly and safely.

Responsibility frees developer organizations to be more autonomous and faster safely. Diffused responsibility doesn’t work for anyone — having an accountable owner means someone who can make a decision. For developers, it should be their name next to their merge request. For a release, the application owner is accountable for results. Giving responsibility and ownership (along with guidance and resources) leads to high reliability and good coordination.

If you look at tools as ways to enable a human being to get tasks done and to improve the abilities of a human being, then a human being is responsible for the output of those tools. All of the tooling in the world — continuous integration, unit testing, code completion, templates, and newer AI-based techniques — amounts to enabling humans to be more productive at achieving their own goals.

When you make it human-centered and domain-centered, you'll see great results and fewer problems. If a firm views a developer as a generic cog, the alternative is spending more time and money in review at the business layer, because someone has to have the domain knowledge and the context to say, “Yes, this is ready to go.” The more you can put that into the hands of the people who are close to the code, close to the application, and the more that they're involved in it, the fewer issues you're going to see.