MoDeGPT for MoE-adjacent compression: modular decomposition without recovery fine-tuning
MoDeGPT compresses Transformer modules with joint low-rank decomposition, avoiding recovery fine-tuning while still reporting 90–95% zero-shot performance at 25–30% compression and up to 46% throughput gain — but the gains come from a training-free, module-level reformulation that is not the same as universally safe pruning for every layer or model family.