The Internal Developer Portal (IDP) market is only 5 or 6 years old, but we’ve seen a huge amount of growth in that time, and many different options emerging.
According to Gartner, IDPs are reported as the most frequently piloted technology in their 2022 - 2024 Technology Adoption Roadmap Survey, and 75% of organizations with platform engineering teams will provide IDPs by 2026.
With this growth, 3 main categories of Internal Developer Portal products have emerged:
Open-source IDPs are exactly what you think. The code is open-source and you host and maintain it yourself.
Proprietary IDPs are typically software as a service startups funded by venture capital. You purchase access to them just like you purchase access to PagerDuty or GitHub Enterprise.
Hybrid IDPs are a combination of open-source and proprietary. They’re built on an open-source foundation, but come with commercial support and proprietary features.
In this article, I’m going to dive into each category one by one, and learn the pros and cons of each.
I’ll also explain why I believe the Hybrid model brings the best of both worlds. The community, extensibility and lack of vendor lock-on of the open-source model, alongside the low cost, ease of use, and fast deployment of the proprietary model.
Open-source
While technically not an IDP, and technically not the only open-source IDP, you can’t talk about open-source IDPs without the conversation focussing on Backstage.
from Capital One, which predates it by almost 4 years. Even after Backstage appeared, a third contender emerged with a strong start. Ride-hailing company Lyft threw their hat into the ring when they released
Benefits of the open-source model for IDPs
In many ways the open-source model is perfect for building an Internal Developer Portal. IDPs are a window into all of the tools that developers use. They’re the venerable “single pane of glass”.
Developers use a lot of tools though, and the developer tool landscape is thousands of tools strong and growing. Even the , with more than 4,000 contributions from end user companies.
Benefits of the open-source model for IDPs
In many ways the open-source model is perfect for building an Internal Developer Portal. IDPs are a window into all of the tools that developers use. They’re the venerable “single pane of glass”.
Developers use a lot of tools though, and the developer tool landscape is thousands of tools strong and growing. Even the , with more than 4,000 contributions from end user companies.
Highly extensible
Underlying all these plugins is a framework which is designed from the ground up to be maximally extensible. , adopters can make wholesale changes to Backstage if they like.
The result is that many organizations have customized Backstage heavily. Take a look at this screenshot of Sunrise,
This completely custom UI shows the progress of changes as they flow through CICD on their way to production. Very little of what you see here comes out of the box with Backstage. Instead, it’s wired together from the basic building blocks of Backstage plugins.
Lack of vendor lock-in
Because Backstage is open-sourced and backed by large end user companies and the CNCF, adopters can feel comfortable that it will be around for a long time, and will continue to meet their needs into the future.
The IDP market is still very early. The majority of the players have been founded in the past 5 years. We will likely see some consolidation of this market as players get acquired and wound down. Any customers of these companies will be forced to migrate to a different solution.
We’ve seen this play out in the CICD market already. There was a time when Travis CI was a dominant provider of continuous integration. In 2019, Travis CI was . In early 2020, Travis CI .
Companies can avoid this fate in the IDP market by building on an open-source project instead of a commercial vendor with it’s own data model and lock-in.
Drawbacks of the open-source model of IDPs
It’s expensive to build
Backstage is a larger undertaking than most people realize before they get into it. To quote Gartner in their at the Autodesk Developer Productivity Summit.
All that I’ve showed you was done through the contributions of many teams, but the core team was three engineers and a engineering manager.
That team has been working on Sunrise since 2020.
And that’s where my team is focussed on, actually like, the discovery part of our application landscape, and where we actually decided to, a few years ago in the beginning of the Backstage era in 2020 when Backstage was released, to actually onboard ourselves into Backstage.
Let’s do the math. Assume an engineer or engineering manager costs $250k per year, therefore the team of 4 costs $1 million per year. 4 years have passed since 2020, so they’ve spent $4 million in total on their core Backstage team. Add in the “contributions of many teams” and we could easily be talking about a $6 million outlay.
Now you’re probably saying “yeah but we’re not as big as Zalando”, but trust me, even small companies will spend a half a million dollars pretty easily.
It requires uncommon skills
Backstage is typically owned and managed by platform teams or DevOps teams. These teams are probably skilled in YAML and kubernetes-native languages like Go.
Backstage is written in TypeScript. The backend runs on NodeJS, and the frontend is written in React.
Organizations who are considering self-hosted Backstage should consider whether or not they have the skills to develop in these languages. As mentioned above, Backstage is a set of libraries that users combine together to make a developer portal. That means you need to write TypeScript to make use of it. You need to understand frontend development, HTML and CSS to customize it. It may not make sene make sense to build up these skills in a Platform and Infrastructure organization because they will not be easily transferrable to other projects.
The second set of uncommon skills that a Backstage deployment needs are project management and developer relations. Once it’s live, you need to evangelize it. You have to go out to the rest of the organization and teach them about it and how to use it. Are your platform developers really going to want to do this work?
It’s got a long time to value
The typical Backstage rollout has a few phases. It starts with an initial deployment, which typically takes a few months to get a basic Backstage instance into production. Some parts of Backstage will work straight out of the box at this point. You’ll be able to use the scaffolder to create new software projects from predefined templates, for example.
Other parts of Backstage require an adoption phase. The software catalog is usually one of them. In order to have a complete software catalog, teams must first put their software into it.
In 2023, at BackstageCon North America, I
I then shared some quotes from adopters who had followed this path
The catalog never really worked for us. We struggled with adoption and rate limiting. Half the info in the catalog was wrong when they first pushed adoption.
2,000 engineers, 2 years experience
The thing we struggled with the most was getting teams to create the YAML file. We spent 6 months actively encouraging teams to register their components with Backstage. After 1 year, only 30 or 40% of the active repos were there.
120 engineers, 1 year experience
Without a lot of product development and project management, this catalog population process takes a long time. There are many Backstage adopters who are at less than 50% catalog completeness after two years with the tool. Even Spotify admit that
This reality is brought on by the combination of overly familiar Material UI libraries and community contributed code that can be hit-or-miss in terms of how it looks and feels.
Backstages plugins can be contributed by anyone, so there’s no guarantee that they will look good, or look consistent. It’s very likely that different plugins will work completely differently even though they are installed in the same Backstage instance.
More guidance
Proprietary IDPs are more opinionated in how they should be used. When you work with a vendor you should get access to solutions engineering and customer success resources who can help to share lessons from other customers.
Working with the open source can sometimes feel like being handed a box of car parts and asked to build a car. Sure you can figure it out eventually, but you’re going to make mistakes along the way, and it probably won’t be the most efficient path.
Having a partner to work alongside and share lessons is worth a lot, and helps to cut the time to value for your IDP project.
Drawbacks of the proprietary model for IDPs
Vendor lock-in
Proprietary IDPs have their own in-house APIs, their own data model, and their own way of doing things. Many of the options on the market are early stage startups. There’s a good chance some of them will disappear over the next few years. If you’re a customer, your IDP will go with them.
When they do survive, you’ll likely be forced to choose between paying up for price hikes, or completing an expensive migration project to a different solution.
Limited extensibility
It’s not possible to change the code of a proprietary IDP. If there’s something about the way it works that doesn’t suit the needs of your company, your best hope is to contact support and hope that they have the desire and the roadmap space to change it.
Most development organizations have some homegrown tools that have sprung up over the years internally. These tools only exist inside the company they were created, and no IDP vendor will have an out-of-the box integration for them. For these tools, you need to create your own plugins if you wish them to be part of your single pane of glass IDP. Some proprietary IDPs do support custom plugin development, but you’ll need to do so using only the concepts and libraries that the vendor provides, and the plugins you create will not be transferrable to any other solution.
Proprietary IDPs have fewer integrations than the massive Backstage community. Even the most well funded proprietary vendor has 48 plugins listed on their integrations page, less than a quarter of the number that are available for Backstage. No matter how big this vendor grows their team, they won’t be able to compete with the 500+ companies who have contributed to Backstage.
Limited speed of execution
The tech landscape is constantly shifting and evolving. Before serverless we had Kubernetes, before Kubernetes we had virtualization, before virtualization we had bare metal. In between and around these major platform shifts we had and continue to have a massive number of different types of serverless, containerization and virtualization options.
A developer portal is supposed to wrap all of these shifts and technologies in order to give the best possible coverage over your internal engineering landscape. Proprietary IDPs may struggle to capture all of these technology shifts into the future, leading to gaps in the portal that platform teams put in front of their internal users.
Hybrid
Last, but not least, we have the Hybrid model. This model takes an open-source foundation with hundreds of plugins and integrations available, and makes it available in an out-of-the-box format which is easy to get started with.
This is what Roadie is. It’s the only standalone IDP based on Backstage, and it offers the best of both worlds - extensibility and speed of execution.
Huge number of integrations
The vast majority of plugins that have been created for Backstage can work on Roadie. and open-sourced them.
Highly extensible
Roadie strives to support as much customization of our IDP as possible. In addition to theming and branding, we support a custom data model, custom Backstage plugins, custom proxies, layouts, scaffolder actions and on and on. You name it,
Inexpensive
In many cases, Roadie is 5 times cheaper than self-hosting Backstage.
People tend not to realize this up front, but the most highly developed Backstage deployments in the world have cost millions of dollars. We talk to companies all the time who have built a team of 5+ engineers around Backstage and run those teams for 3+ years. The most expensive Backstage deployment we’ve seen cost $7.5 million dollars over 3 years.
Roadie doesn’t require such an extensive team to be built around it. We take away all of the operations and deployment effort, and much of the customer support that has to happen. We also give you out-of-the box features like that you would otherwise have to build yourself to realize the potential of your IDP.
Lastly, we charge based on usage. You only pay for the engineers who write code which is tracked in the catalog. Everyone else gets to log in for free. This means you can ramp gradually throughout your org, rather than paying for the developement of the entire IDP up front.
Requires few skills
Roadie doesn’t require any TypeScript skills. You just drag and drop plugins where you want them, and configure the product in our administration panels. It’s dead simple. This is perfect for platform teams who are trained on cloud-native technologies. We even have no-code plugins that you can use to display lists or charts inside the UI without writing any code.
If you do need to make something completely custom and you don’t want to write a Backstge plugin yourself, we can sort you out with professional services to suit your needs.
Short time to value
We’ve built a ton of features into Roadie to help shorten the time to value. For example, we help teams build out their catalog quickly and we see customers reaching a
Image by
SOCIAL SHARE CARD GENERATOR