🔥 Did you know? Groq is rivaling giants like Google Gemini and Amazon Bedrock in LangChain’s API usage!
Daniel Newman & Patrick Moorhead catch Groq CEO Jonathan Ross to get his take on AI’s exponential growth and why Groq is poised to dominate the inference market.
🎧 Tune into this segment of The View From Davos for more on:
- AI’s Exponential Growth: AI is evolving at an incredible pace, driven by the doubling of chips and transistors. Groq is uniquely positioned to capitalize on this with their data center-level approach
- Energy Efficiency: Groq’s architecture uses 1/3 the energy of traditional GPUs, a major advantage in a world increasingly focused on sustainability
- Massive Capacity: Groq is ramping up production to 2 million chips this year, rivaling NVIDIA’s output
- Global Inference Network: With deployments in Saudi Arabia and beyond, Groq is building a truly global inference network
- Jonathan also hinted at a potential 4nm chip in the works… 👀
Learn more at Groq.
Watch the video below, and be sure to subscribe to our YouTube channel, so you never miss an episode.
Or listen to the audio here:
Disclaimer: The View From Davos is for information and entertainment purposes only. Over the course of this webcast, we may talk about companies that are publicly traded and we may even reference that fact and their equity share price, but please do not take anything that we say as a recommendation about what you should do with your investment dollars. We are not investment advisors, and we ask that you do not treat us as such.
Transcript:
Patrick Moorhead: The Six Five is On the Road with a View from Davos. We are here at the World Economic Forum having some incredible conversations. It’s one of the few places where you’ve got the intersection of tech, politics, regulation, sustainability, all wrapped into one. Dan, it’s been a good show so far.
Daniel Newman: It has been. It is the morning and the Magic Mountain looks beautiful, but you really can’t get through a day here without just having some of the most inspirational and interesting conversations. But look, AI, Pat, is of course, well, every year there’s a theme. There was years where it was sustainability. There’s years where it’s been diversity and inclusion. This year is really the year I think it’s coming together. The last few years have been a lot about infrastructure and build up. This year it’s about seeing it in action, seeing AI work and driving enterprise and business and government value.
Patrick Moorhead: Very true ROI. One of the keys to this is obviously the inference, the processing of all that data. I can’t imagine a better person to have a conversation with than the CEO of Groq, Jonathan Ross. Good to see you.
Jonathan Ross: Sorry, did someone say inference? We’re doing LPU chips. They’re the fastest for inference and we’re going to ramp up quite a bit this year. Some of the numbers are unbelievable, but we’ve been hitting them.
Patrick Moorhead: Well, it’s incredible. I mean, we saw you over the holidays loading 747s off to a far end country, super exciting though. But I have to ask you though, what do you want to get out of this event? Is it a customer event for you? Are you listening to officials? Is it all the above?
Jonathan Ross: It’s meetings. We’ve been pretty heads down trying to deliver on what we had to deliver. Now I’m just bumping into people in the streets. In the coffee line this morning, someone recognized me and these are people who we need to get back in touch with because we’ve been so focused on delivering for this customer. But we now measure our deliveries in 747s.
Daniel Newman: I think that’s actually great. I mean, look, both Pat and I have been supporters of your journey from the very beginning. We watched the pivot. We both are on your cap table. Full disclosure, we’re invested and fully believe in what you’re doing, but we’ve also seen a pretty substantial pivot. Remember in the beginning you sort of had a chip that could do a few different things and then at some point you kind of said, we think it’s inference. You went to the LPU and then you kind of went to the cloud. You’re like, “Look, if people aren’t going to buy this and build it, we’ll build it. We’ll deploy it.” I mean, talk a little bit about some of these, what you can. I know you can’t share everything, but I mean these 747s are going somewhere. Customers are starting to consume this stuff. I mean you raised money recently. There must be some evidence now that you can maybe share out there about how this is sort of evolving.
Jonathan Ross: Well, actually, one of the biggest ones, LangChain recently announced their top API users or whatever are providers, and of course, OpenAI. Then a local thing, which isn’t really an API, then Anthropic, then OpenAI again on Azure and then Groq, and then Google Gemini, and then Amazon Bedrock. We’re actually above those in terms of usage. What we’ve lacked is total amount of capacity for AI and we’ve just added a whole bunch more. We just delivered a whole bunch of 747s worth of our systems to the Kingdom of Saudi Arabia. That’s our second region, but it’s almost on the opposite side of the planet, which has brought our latency down almost no matter where you are. Now we’re looking to deploy all over the world and build out a global inference network. But as for the pivot, we actually always wanted to sort of build this and make it easy for people to use from an API point of view and cloud. We never thought we would be able to get the capital to start doing this ourselves. We started selling systems, but we found it was just easier to get people using us when we provide an API. We launched in March and we’re now at over 800,000 developers.
Daniel Newman: Wow.
Patrick Moorhead: No, that is a staggering number. Every time I do a double take, every one of the emails I get from you guys or you or Sonny is kind of flaunting it on there. I mean, congratulations there. One of the questions I get a lot that it’s funny. I do think I know the answer to this, but I think our viewers need to hear is architecturally, I mean everybody’s like NVIDIA, NVIDIA, NVIDIA, NVIDIA, training, training, training. What is it about the Groq chip in services that makes it special just for the layperson?
Jonathan Ross: Well, for the layperson, I’ll give it a shot, a little complicated, but all right. Think of it this way. Everyone’s heard of Moore’s Law and so the number of transistors doubles roughly every 18 to 24 months. What we realized was that AI was actually getting better, faster than exponential. But if you look at the amount of data being created, it wasn’t exponential. The algorithms, they were improving, but it was big jumps. It wasn’t this sort of exponential growth. Where was this double exponential coming from? What we realized when we started Groq was that the number of chips was also doubling every 18 to 24 months. If you’re going to get an advantage, why focus at the chip level? Why not focus at the data center level? What we did that’s very different than GPUs. GPUs have a lot of external memory. They will do part of the problem, bring stuff in from memory and then do part of the problem and bring stuff in from memory. It’s very slow, very energy intensive. We use about a third of the energy versus a GPU because what we do is we’ll take hundreds or thousands of our chips, lay the problem out completely in those chips so we don’t touch any external memory. It’s like an assembly line. We’ll just go through that very quickly. We take advantage of that double exponential and we’re the first to do this.
Daniel Newman: I know we only have a minute with you Jonathan. By the way, thank you so much for taking the time. Now I do want to point out for everybody out there, you’ve been doing this on 14 nanometer, which you don’t have to, I mean you got to be a little technical to understand what that actually means, but there’s a lot of opportunity to improve. You’ve been these incredible speeds on like 14 nanometer. You could go to four, three, two and start to even get more processing power, transistor density. It’s pretty exciting stuff. As you kind of finish up your first journey to Davos, I know what is sort of the one thing you want people to be thinking about Groq as we head into the full, what are?
Jonathan Ross: Let me give you two. I’ll give you two for the price of one.
Daniel Newman: Two?
Jonathan Ross: The first one is going to the 14 nanometer. To give a sense of that, most of you will have heard of what a Blackwell is or an H100. Blackwell is four nanometer. Then you get the H100, which is older. Then you have, I think it was the Volta and then you have the Pascal. Now you’re at somewhere around 12 nanometer. We’re 14 nanometer and we’re running circles around these four nanometer chips. Rumors are, we may have taped out a four nanometer chip that may be coming soon. I’m not going to confirm or deny. That’s one. Number two, the amount of capacity, because we’re not capacity constrained because we’re not using HBM, we’re going to ramp and build an enormous number of chips this year. Our intention is to get close to 2 million this year. If we do that, keep in mind that NVIDIA is trying to do two to 3 million this year. Next year we plan to do a lot more.
Daniel Newman: That’s incredible and makes me really proud to just sit there, let the shares appreciate Jonathan. I think I’m glad that we bought early and believed early. I continue to believe. I’m really glad we had the chance to sit here in Davos. I know we’ve done it in other parts of the country in the US. It’s just been great to watch your explosive growth, dimensions on All-In. We’re all fans here. Pat and I actually, we went to the original All-In Summit. We actually did it the first year. Those guys are great. They’re super interesting. I love how the kind of truth telling and that they just say it straight. I know you’re a guy that always likes to say it straight. Have a great Davos, thanks for joining us and let’s talk again soon.
Jonathan Ross: Thanks. All right.
Daniel Newman: Thank you everybody for tuning in. Stay part of our Six Five On the Road, a View from Davos. So many great conversations here. This is a great way to start day number two. Hit subscribe. We’ll talk to you all later. Bye-bye.
