How to build a commercial open source semiconductor company.
Or: How I learned to stop worrying and love FPGAs.
Is it possible to build an open source semiconductor company?
Or: How I learned to stop worrying and love FPGAs.
Y Combinator recently put out its 2024 Requests for Startups. Two things caught my eye: the focus on hard-tech and deep-tech, and a specific request for commercial open source companies. Normally, these two categories are distinct; most deep tech companies build their moats with patents and trade secrets, and the idea of leveraging the software-focused open source community for a hardware company seems far-fetched. However, as a semiconductor founder and a major proponent of open source software, I’ve been thinking about a plan to build something that nobody has ever built before: an open source semiconductor company.
Now, I know about eFabless and Google’s Open Source Silicon project. I know about TinyTapeout, and even volunteered at their workshops. I really love these projects, but they’re primarily aimed at education, research, and rapid prototyping, rather than commercial use. What I’m proposing is a company focused on selling hardware to enterprises to solve real-world problems, but entirely open source.
How do we do it? Well, it starts with picking a problem to tackle.
Step 1: Pick a computational problem
If you want to build a company, you have to pick a problem that actually matters. If you want to build an open-source semiconductor company, though, there are a few more constraints. Here’s what my rough list would be:
There needs to be a real, significant market for the solution to this problem. This is important for any startup, but especially for a chip startup, if you eventually plan to manufacture custom silicon. To justify the investment, you need a big market.
Existing chips, like CPUs and GPUs, can’t be very good at solving this problem. At the very least, existing chips need to be beatable with clever architectures. An example of a bad problem to choose would be graphics, because NVidia will always beat you.
Whatever you’re doing needs to be appealing to the open source community. One of the biggest reasons to develop a commercial open source company is to move faster with the support of community devs. So if you’re doing something too boring or too evil, this may not be the route for you.
Problems like accelerating data analytics, video encoding, or deep packet inspection are all potentially good fits for this sort of company structure, though admittedly I’m not an expert in any of those fields. Problems like AI acceleration might be a challenge, unless you have a fundamental architectural insight that blows GPUs and existing AI chips out of the water.
Why is it important that we pick a problem that existing chips struggle with? Well, our open-source solution is not going to be able to out-perform closed-source, optimized chips, like NVidia GPUs for matrix multiplication. But if we pick a problem for which no optimized chip exists, our open-source hardware implementation can outperform software running on a CPU.
This is important because we’re going to be starting out by working with FPGAs. At a high level, an FPGA is a chip with a large number of arbitrary logic gates on it. As a designer, you can write special software that connects these gates in complex ways; essentially, you get to build custom hardware, without having to manufacture custom hardware. Using an FPGA results in lower performance than an ASIC, but higher performance than software on a CPU. It’s perfect for building open-source hardware to solve computational problems for which custom ASICs don’t yet exist.
Step 2: Write a ton of Verilog
After we determine what problem we want to solve, we need to design the hardware to solve it. In theory, this is straightforward, but in practice, it’s not. The best way to develop FPGA architectures for large-scale computing is with AWS F1 FPGA instances, but that development environment is not particularly user-friendly. In theory, you could use one of their high-level synthesis flows, but those usually result in significantly worse performance than writing your own Verilog.
Because the goal of the company is to make this design open source, it’s critical to make sure your code is clean, well-written, and documented. Commercial open source companies win over users by building technically amazing products, and that starts from day one. To quote the YC RFS (emphasis mine):
Very technical founders are at a strong advantage here, as the sales motion relies more on the technical merits of the project rather than strong sales tactics.
Hopefully, with some elbow grease and a lot of assign statements, you should be able to get a basic proof-of-concept up and running on an F1 FPGA. Now, it’s time to start building the open source community around your project.
Step 3: Build excitement around the project
This theoretical open-source hardware company has one challenge that companies like GitLab and Supabase never had to face: you’re not building a company around existing, well-loved open source designs. Instead, you need to build a community around your new architecture in order to attract contributors, suggestions, and your first users.
Write articles, tweet a lot, and post on HackerNews and other forums. There aren’t too many open-source devs who know Verilog, so getting their attention will require a lot of smart PR. You do have one advantage, though: there aren’t too many cool open-source Verilog projects for the developers who would love to work on them.
Being open source gets you many advantages compared to your closed-source counterparts. To quote YC again (yet again, emphasis mine):
Open source companies move more quickly than closed source companies. For developer tools, being open source is a powerful way to gain developer adoption. But it’s also a great way for startups to become mature and sell to enterprises a lot sooner. [...] It’s more natural for technical founders to talk to users who are engineers just like them, and they can iterate faster since they’ll get feedback from the open source community.
The open source model is even better for hardware companies, though. If you’re staying closed source and your first deliverable is an ASIC, you have to spend years building before you ever put something in customers’ hands. With the open source FPGA approach, you can start deploying to beta users within weeks and start getting real feedback on your design. You can iterate significantly faster than a company starting with a closed source ASIC, and ultimately deliver better products.
If people are getting really excited about your project, you can reserve an F1 instance or two and let people pay to get access to a turnkey version of your design! They’re still welcome to build the Verilog themselves and run their own servers, but like I said, AWS F1 deployment is not very user friendly. Besides, you and your team are the world experts anyways. While some of your core contributors will probably handle building and development themselves, customers who care more about the application than how cool your hardware is will probably be happy to pay for a nice, hosted environment to run their applications and get amazing performance.
Step 4: Now you can make a chip
At this point, not only should you have a hardware architecture that’s fully open-source and supported by the community, but you should hopefully have some paying customers who are using your hosted FPGA product to run real-world applications. You’ve achieved something many semiconductor companies fail to do: actually deploy their designs at scale and start making money. You have validation from your customers that people actually care about the workloads you’re accelerating, and you have technical proof that your design works like a charm.
Best of all, you didn’t need to spend too much money to get there! With community support for hardware development, and relatively affordable AWS F1 FPGA hosting, your margin structure looks much more like a software company than a hardware company. But open source also comes with a cost: competitors might be trying to deploy your architecture as well.How do you defend against that and put up a moat?
Build an ASIC!
You already have a design and you have customers who love it. All a VC needs to do at this point is put in the money to take that Verilog you and your community wrote, and tape it out. It’s the most no-brainer funding round out there. All the commercialization challenges that normally plague early stage semiconductor companies are solved. You build a chip, build infrastructure around it, and migrate your existing customers to your new infrastructure, which is faster, better, and offers you higher margins.
But don’t sunset the FPGA versions! Keep working with your community to build out new features that you’ll put in your next generation silicon. Customers who want those new features can run their workloads on the FPGA implementation, and trade some performance to stay on the bleeding edge. This means that you can have a tighter feedback loop with your customers and make sure that your second generation ASICs aren’t just faster, but fulfill even more customers needs than your first generation.
The Dawn of Open Source Silicon
So, why hasn’t anybody done this yet? What are the biggest obstacles stopping startups from following this roadmap?
The first problem is simply that there aren’t all that many digital design engineers and computer architects who are both able and willing to embark on a project like this. Top universities graduate far more computer scientists than electrical engineers; only a small set of those electrical engineers are digital chip designers, and only a subset of those are interested in starting a company or contributing to an open source project like this.
Luckily, Verilog is easy to learn! Resources like TinyTapeout and TinyFPGA make it easy to learn about Verilog and chip design, and the more we support and invest in these projects, the more engineers are going to be able to learn these languages and eventually contribute to the open source hardware projects of tomorrow.
The second problem is more difficult to solve: there’s a limited set of important computational problems where an FPGA can easily outpace a GPU. Plus, it seems like the bigger an FPGA’s advantage is, the smaller the market for that problem is. The most exciting emerging compute market, AI and ML acceleration, is already dominated by GPUs and specialized ASICs, so it seems hard for FPGAs to keep up.
FPGAs have one key advantage here too, though: because you can rapidly prototype, you can try much more innovative techniques to get great performance. The 4-step playbook outlined above works just fine for a normal, digital design that, for example, accelerates video encoding, but I think it works even better for a design leveraging asynchronous logic, or emulating probabilistic spin logic. Not only will these esoteric architectures drive community involvement (because they’re cool as hell), but deploying esoteric architectures with multi-phase or asynchronous clocking schemes is easier on a FPGA, where an error costs you a few hours re-synthesizing rather than millions of dollars going back through the ASIC manufacturing process.
If you’re working on an open source FPGA project, I’d love to hear about it! Feel free to tweet at me at @blip_tm.