Not startups per say, but rumor has it that OpenAI and xAI are both working on training ASICs! There's also Amazon's Tranium and of course the TPU, which are both decent at training. Google did some incredible work on the networking capabilities of the recent generations of TPUs, probably for this very reason.
Great read as always Zach. Curious as to what your thoughts on Etched are considering how they are solely focused on inference for transformers, which feels like a bold bet considering there are murmurs about exploring alternatives to transformers
interesting observation, didn't think about this trend at all. any top of mind second-gen chip startups working on training?
Not startups per say, but rumor has it that OpenAI and xAI are both working on training ASICs! There's also Amazon's Tranium and of course the TPU, which are both decent at training. Google did some incredible work on the networking capabilities of the recent generations of TPUs, probably for this very reason.
Great read as always Zach. Curious as to what your thoughts on Etched are considering how they are solely focused on inference for transformers, which feels like a bold bet considering there are murmurs about exploring alternatives to transformers