Public Intelligence

A call for AI models with open weights, open source, and open corpus.

I applaud Jensen Huang and industry leaders for coming out in support of open large language models. The entire tech sector minus Anthropic has now signed on. Now that the movement has momentum, I urge the signatories to endorse something even more radical that would truly separate us in our economic competition with China: The USA should heavily invest in models not just with open weights, but with open source and open corpus.

Let me back up in case you missed it. Last Friday, Huang pasted a short letter on Twitter calling for American investment in “a strong, open ecosystem” around open source artificial intelligence. “Open source…created a shared foundation of knowledge on which generations of American engineers and entrepreneurs built their institutional sovereignty,” Huang predicts that a similar thing will happen with the embrace of open source AI.

Part of this letter was a plea to the Trump administration, which had been sending signals about banning open source software in a protectionist move to squash insurgent Chinese language machines. Seeing that this would benefit Anthropic and OpenAI and no one else, the CEOs of all of the big and small tech firms quickly signed on to Huang’s letter on Friday.

This is an amazing, positive development, and I want to air some points that Huang left out. I’ve been passionately calling for a deeper investment in open source language models on this blog for years. In fact, this post from last July makes a bunch of points about what I think is necessary for this to happen, and I don’t think any of it has changed.

You should go read that post, but let me summarize what I said for the present moment. What is needed to build a good language model is people, compute, and data. It’s not clear how much of each of these you need, but the answer seems to be a lot. We’ve always had the people, though the promise of impossible riches has lured many smart young minds away from more ethical paths. But now that we have the backing of the entire tech industry, I think this base is covered.

What about compute? Hyperscalers and their economic bedfellows always want you to believe that their massive datacenter buildout is a moat from us open source plebes. But the Chinese models have complicated this story and shown you can get by on the cheap. No one has precise estimates, but models from Chinese companies DeepSeek and Moonshot were arguably trained using under ten million dollars. Dozens of AI startup “NeoLabs” with few concrete ideas are currently being given 10x that by venture firms. Good VCs could make a small bet on the open ecosystem to juice their other investments. Compute for open models will require a massive industrial consortium, but it would cost each member of the consortium a minuscule fraction of their war chests. Let’s work together to build computing agreements for open source development.

Once these models are built out in the open, there is no doubt that they will only get dramatically more efficient. We’ve seen time and time again that machine learning is a field that innovates through “frictionless reproducibility.” Research, code, and data out in the open, evaluated by competitive benchmarking, rapidly improve machine learning systems. It also makes them more efficient. In last year’s post, I wrote about how high-quality ImageNet models went from something only trainable at Google to something you could build on a desktop in less than a year. These sorts of efficiency gains happened throughout the 2010s. The secrecy of labs in the 2020s has harmed the broader engineering field, even though the artifacts produced have been beyond impressive. Put everything out in the open, and we’ll figure out how to make it faster and more efficient. It’s guaranteed.

That brings us to the actual hard part. The data. As my friend and colleague Alyosha Efros loves to remind us, “It’s all about the data,” and language models are trained on unfathomable amounts of it. Discovery in lawsuits has revealed that companies trained these models on pirated libraries of books, academic papers, and copyrighted imagery. The models are trained on collaborative knowledge bases like Wikipedia, countless volunteer forums like Reddit, and all of the public code on GitHub. They are trained on the transcript of every video you post to YouTube. All of this collective work by human society gets slurried into proprietary software so a few zealots can get rich. This is a bad outcome!

The companies are all up front about this use. They have been found liable in court. They have admitted it in papers they have written. They claim this is all fair use because the training was transformative of the texts. Fine, if that’s the case, then it’s fair use to take the outputs of their models and build new ones. This is called “distillation.” Chatbot terms of service agreements do not negate my argument. That the Trump administration is flirting with banning distillation is a travesty.

But I want something bigger. The biggest step to making competitive open source models is allowing the broader community fair use access to the same material the companies used. This is the open corpus. If closed models can exist, then open models should be allowed a level playing field. This will require a long overdue rethinking of intellectual property and what we owe the people who create it.

I think all of these challenges are surmountable. We’re going to have to (a) get young people to care about open source instead of becoming impossibly rich. (b) get billionaires to collaborate in a non-winner-take-all fashion, and (c) have a long hard conversation about copyright law, intellectual property, and fair use standards. But we’ve thrived in an open ecosystem before ChatGPT. It hasn’t even been half a decade of closedness. With the blessing of the entire tech industry, it’s time to open things up again.

---

The title of this post is a coinage of Kevin Kelly. His dedication also inspired me to write this post.