CWIC
Compute Where It Counts. A new state-of-the-art method for creating sparse transformers that automatically decide when to use more or less compute.
Links
I developed CWIC with the team at Crystal AI, so I will just point you to the official release here.
You can also view a more recently revised version of our paper here.