Research

Ongoing research projects and notes that evolve over time.

AI
Attention
Flash Attention
Updated:
Self Attention
Updated:
Papers
Big Bird
Updated:
Linformer
Updated:
Longformer
Updated:
Performer
Updated:
Reformer
Updated:
Questions

Changelog

2026-01-26 — Added initial content on how context windows got longer in transformers.