How Chase UK went slow with Harness to get quick migration of continuous applications delivery
Chase UK, the London end of the famous US banking behemoth, was founded towards the end of 2021 as a brand new cloud-native operation from the very start. Despite – perhaps even because of – the existential dramas of the pandemic, it has grown significantly while also undertaking a significant migration, two events that individually would leave many business managers nervous wrecks.
The story of that migration was delivered to the delegates attending the recent Harness Unscripted conference in London. Telling the tale was Petrut Iordachescu, who, as the Platform Lead on CI/CD and Observability for Chase UK, played a major part in driving the migration through.
The bank’s start was fast, and is continuing so. According to Iordachescu, it has already been able to attract some two million customers, over $20 billion in deposits and, of more specific interest to him, it already has about 500 micro-services running in production. He described the storyline as a `journey from tools to ecosystems’, which may not sound quite as drama-laden as moving from a Cobol batch-processing legacy environment to a cloud-native world.
But in practice, changing the way in which Continuous Deployment of applications, applets and micro-services is delivered is, for a modern bank, just as important. How well such systems work is the cornerstone on which all customer experience is now based. Good results there can be gold dust for every modern bank operation, he noted:
The Continuous Deployment (CD) journey from using tools, to running a SaaS environment represents the bank’s transformative journey in advancing its CD processes from a tool-centric approach to a rich Software Development Lifecycle (SDLC) ecosystem.
Harness’ capabilities in application development and delivery were assisted by AI-based tools well before it became a ‘thing’ about a year ago. It has used other types of AI to improve continuous verification and test intelligence for some time. These help speed up software development lifecycles through steps such as shortlisting appropriate tests for code updates. Earlier this year the company introduced a new LLM (Large Language Model) co-pilot to improve the user interface and streamline the governance of IT, security, and cost management.
Iordachescu’s first point was that the migration involved a significant change not just in technology used, but also in fundamental approach – a move from using tools to working with a complete ecosystem. He thinks that tools are now a thing of the past, particularly for an organization like the bank, working as it does in a regulated industry where certain checks and toll gates have to be met.
The original architecture was based on tools from Spinnaker, which he described as “complex”, particularly in the way they had to work together. This involved such activities as Kubernetes workloads having to deploy to and work with other Kubernetes deployments. As a hub-and-spoke model it also involved a lot of reporting up and down the lines, said Iordachescu:
“The thing is you have a complex architecture that your developers need to remember, and it means that somebody has to go into different accounts and maintain things and set up different endpoints, which you know you have to remember to set up all of those accounts for context. In the Chase UK environment this totals more than 50 accounts on AWS, which means a lot of different dependencies:
One of the things I think of with the cloud is I think of east/west traffic, which is traffic local between your accounts, and north/south traffic, which is traffic to the internet or other locations. The tool ecosystem introduces an interesting east/west challenge because you have the central account and you then have to make sure that the communication between these two is set up, plus a hub-and-spoke model is pretty antiquated in the context of cloud.
Enter, ecosystems
This simplifies life by utilizing a delegate to communicate, via a proxy, with the Harness control plane. This where most tasks happen. As the Harness environment is open source, it transpires that several of the Harness processes were originated by developers working for Chase UK, giving a high level of confidence in its operation, observability and security levels.
Chase decided to go live with the newest Harness platform for multiple reasons. One was that it seemed to be easier. In particular, Iordachescu saw little point in performing the migration and going into production with the older, established platform and then have to do it all again with the migration to the new platform:
The delegate rollout initially took two months. We deploy over 50 accounts, and I think initially it took us two months to deploy the first version of our delegates to all of those accounts and make sure that they're green and reporting in the Harness console. We then took a serial approach to it, because we wanted to run a bunch of tests to make sure that the validations work. It now takes 38 minutes.
One issue the migration team faced was the desire amongst the developer team to have feature parity with some of the old Spinnaker tools. So far there have been 20 feature requests accommodated, together with some critical improvements. There was close collaboration with Harness to make sure that the developers saw little or no friction or barriers
Last mile
Iordachescu now sees Harness as the last mile of the bank’s Software Development Lifecycle, effectively the point where the team can do some checkpoints and tollgating. So it was the point he chose to integrate with some internal tooling, factors that the delegate has sight of, but are not always seen at the backbone level. For example this has included the creation of a compliance and automation micro-service, he explained:
So the OPA engine inside of Harness checks that all of the different things that we want to put in as tollgates have been checked. And once it's green, it will allow a tollgate stage. So effectively, we use the OPA engine in the regular language in order to build that last mile.
The bank has also introduced ‘shift left’ – compliance by default. Instead of developers having to ensure that they have properly executed all necessary controls and tollgates, they can now be free the develop the code without that restraint. With compliance by default, developers know that, before they go to production, the checks that are mandatory are going to get done automatically.
One factor that is inevitable, and important to most users, is the ability to make changes such as additional services. As he observed, the financial community in particular is always prone to living with the reality of having `one more compliance requirement that has to be met’.
To help build awareness of changes, Harness allows users to set warnings in the user interface that action will be coming for about a month before it needs to be enforced. This gives developers and users time to interact with it and to push back in cases it is unnecessary. It also allows for spotting the need for short term exemptions.
One lesson learned out of the migration, especially if moving from a tools-based environment to an ecosystem, is to understand the touch points in the organization where specific interactions occur. It is wise to inform of each other's roadmap at every one of those points, for it will help in ensuring the development and delivery ecosystem, in this case Harness, can be exploited to its best potential. Iordachescu advised:
So go slow in prepping, so you can go faster in migration, because we have internal checks, and there's some regulatory checks and compliances that we have to make sure that we abide by. So it took a while, but the speed of adoption meant that we could go fast when it mattered and when we migrated.