Thread
š„ If you read one thing about AI this week, let it be Googleās leaked doc
āWe have no moat, and neither does OpenAIā
āWe have no moat, and neither does OpenAIā
Open-source is innovating faster and cheaper than us, or OpenAI, ever can.
Examples:
Examples:
Large expensive models are still slightly better, but the gap is quickly closing. 
Metaās LLaMA leaked, and the open-source community tan with it. 
Some innovations are now coming from single people, tinkering on their laptop āin their garageā 
After StableDiffusion it shouldnāt have come as a surprise. (Anyone remember DALL-E? many people sure donāt..) 
Seems like having a lot of money to train large models might cause them to pay less attention to far cheaper methods (inversely, thatās what the tinkerer at home works with) 
And maybe those small, cheap, incremental updates can outrun progress on large, expensive models. 
To save money, ātinkerersā are training on smaller, but better curated data-sets. Outputs from ChatGPT that have been shared by users because they were cool are used. 
Highly specialized individuals are making highly specialized products. 
And Meta might be benefiting hereā¦ because the leaked model was theirs, they can now use all the innovation thatās happening on top of it š„² 
Is Google about to open-source a lot more stuff? āIf you canāt beat ām, join āmā? 
And why be secretive? āOpenAI already knows everything we do because they hire our people.ā 
Additional thoughts after 24h:
1. We donāt know if the doc is real, but I havenāt heard anyone argue with the analysis + we know that there are sometimes well written white-papers circulating within Google.
2. It does conveniently signal to the regulator āwe are not a monopolyā
1. We donāt know if the doc is real, but I havenāt heard anyone argue with the analysis + we know that there are sometimes well written white-papers circulating within Google.
2. It does conveniently signal to the regulator āwe are not a monopolyā
3. Talking about regulations: open-source is definitely harder to regulate.
4. Apple and Amazon have been relatively absent in the recent AI race ā There have been rumors that Apple has vastly underestimated LLMs potential. Might they now benefit from open-source?
4. Apple and Amazon have been relatively absent in the recent AI race ā There have been rumors that Apple has vastly underestimated LLMs potential. Might they now benefit from open-source?
At Cradle.bio we are using AI to design better proteins to ultimately replace traditional farms š and factories š
Read more in this threadš§µ
(we are hiring)
Read more in this threadš§µ
(we are hiring)
Mentions
See All
Dave Troy @davetroy
Ā·
May 5, 2023
Important thread here; leaked doc from Google argues that big companies are not advantaged in the AI arena because the proliferation of lightweight open source AI models allows for faster, more creative and wider innovation. May be a serious inflection point for big tech.
