Can a new transformer architecture with 2 million 🚀 tokens fix the token window problem? Or are there no free lunches? What do we gain, what do we give up? New paper from @MikhailBurtsev, Aydar Bulatov and @yurakuratov Let's break it down👇 arxiv

Matt Clifford @matthewclifford · Apr 24, 2023

Great thread on an approach to scaling token windows to >1m tokens. New important ideas emerging every day…

Tweet Apr 24, 2023

by Hrishi Olickel

Recommended by 1 person

1 mention