These capabilities allow you to update or ingest additional metadata in the Software Catalog for components and use it to customize your Catalog pages UI, build new Scorecard checks and provide dropdown pickers in the Workflows UI.
TL;DR
Harness IDP now enables dynamic metadata updates in the Software Catalog without manual YAML edits, facilitating real-time test coverage tracking and ServiceNow CMDB integration, enhancing developer productivity and operational efficiency.
Lately we have been working hard to support some advanced use-cases for our customers. I am glad to write about an exciting set of new capabilities in Harness IDP (Internal Developer Portal) across Software Catalog, Scorecards and Workflows. I will take you through a couple of use-cases and show how this can be useful for you too.
These capabilities allow you to update or ingest additional metadata in the Software Catalog for components and use it to customize your Catalog pages UI, build new Scorecard checks and provide dropdown pickers in the Workflows UI.
One of our main considerations when building this has been the ease of use - that this should not require manual and expensive YAML updates from developers. We have heard you loud and clear.
Now let us dive into these features by taking a couple of use-cases.
Use Case - View and measure test coverage score for microservices
Assume you have a system which keeps track of test coverage scores for your microservices and other software components. This could be information that you may be getting from your testing tools or systems like Sonarqube. We do not care about the data source or whether you process the data yourself e.g. you might want to generate a mean score based on unit test coverage as well as end to end tests.
For our use-case, let’s assume you want to accomplish two goals -
- Show on the Software Catalog the test coverage score for any given component e.g. payment-service
- Build a new Scorecard check in your existing or a new Service Maturity scorecard
As you might be aware, every software component is ingested into the catalog using a catalog-info.yaml definition file. The challenge is that there are hundreds or thousands of microservices and the test coverage score is computed multiple times a day. So there is no way a developer or a platform engineer can keep all the YAMLs up to date.
This leads to the need of updating the component’s definition without updating its YAML definition file. As the Catalog in Harness IDP is powered by Backstage and Backstage does not have an API to update an entity, we had to create a new one!
Using the new Catalog Metadata Ingestion APIs, you can add new metadata fields like metadata.testCoverage for any or all software components in the Software Catalog. This is updated directly in the Catalog without updating its YAML definition file in git. Check out the docs to explore the endpoint in detail.
The entity data is now updated in Catalog, great! But how do we see it? Especially, how would developers find their test coverage score for their services, websites and libraries that they own?
This is where our new Entity Additional Info Card comes to the rescue. Without writing any custom plugin or JS/TS - you can update the Layout of the Catalog page from Admin -> Layout section, add this new Card and start displaying the test coverage score on the Catalog Overview page for the components. This is what it will look like -
Brilliant! There’s just one last thing - we need to update our default “Service Maturity” scorecard as well and utilize test coverage score as a check. Let’s assume that the architects in your organization have decided that for backend components, the score should be more than 70% and for UI components, it should be more than 60%.
Scorecards support Catalog as a native data source, where you can query the entity definition to write your own check. In this case, we’ll use the JEXL expression data point and build a check for backend components (filtered using catalog’s tags) to compare the score with the desired number.
And that’s it! The last missing piece is that you can now run the ingestion as per your preference and the data will keep on updating.
ProTip - You can use Harness Pipeline triggers and run the Catalog Ingestion as an hourly or daily cron job, using the Run step in IDP Stage.
This approach can be used for many other similar use-cases, for example tracking team leads and on-call members in catalog or tracking cost budget utilization per service, team or system. Now let’s take a look at another example.
Use Case - Integrating Harness IDP with ServiceNow CMDB
Many of our customers use the ServiceNow CMDB (Configuration Management Database System) to keep track of their software assets. Each application has a unique business application ID which is their unique identifier. It is used anytime developers have to make a new request to the central IT or DevOps teams.
Using the Catalog Ingestion APIs and Additional Info Card as described above, we can ingest and display the Business Application IDs in the IDP Software Catalog as well. However, when it comes to IDP Workflows, these IDs have a role to play there as well, for example when submitting an infrastructure provisioning request.
This is why we have launched a new Autocomplete Picker UI field which allows developers to pick from all available catalog entities and any of its metadata. In this case, we would like developers to pick from all values of metadata.serviceNowAppId.
This is known as EntityFieldPicker. Check out our documentation to learn more.
Thank you for reading! Checkout these additional resources on the features described above.
- Catalog Ingestion APIs
- Catalog Additional Info Card
- Scorecard Catalog based checks
- EntityFieldPicker in Workflows UI
If you would like to get started with Harness IDP, start here.
