Word to the wise: as someone who actually works in the field, trust NO claims until you can verify them with real code.
Papers very often contain the very uppermost bound of what's _theoretically_ possible when it comes to benchmarks. Researchers rarely have the skill to realize those gains in practice, so any performance numbers in papers should be assumed theoretical and unverified unless you can actually download code and benchmark them yourself or unless they come from a research organization known for competent benchmarking (e.g. Google Brain). In particular any "sparse" approach is deeply suspect as far as its practical performance or memory efficiency claims: current hardware does not deal with sparsity well unless things are _really_ sparse (like 1/10th or more) and sparsity is able to outweigh architectural inefficiencies.
Run on a single machine by logically partitioning GPUs. Don't get me wrong, I'm not disputing that this could work or that it could be a "breakthrough". I'm just saying that unless it's independently replicated and confirmed, it's just a paper like a million others.
It's an interesting premise nonetheless. Perhaps another similar approach would be one from mathematical manifolds, where they have charts and atlases, and I believe they build the atlas by having overlapping charts.
Papers very often contain the very uppermost bound of what's _theoretically_ possible when it comes to benchmarks. Researchers rarely have the skill to realize those gains in practice, so any performance numbers in papers should be assumed theoretical and unverified unless you can actually download code and benchmark them yourself or unless they come from a research organization known for competent benchmarking (e.g. Google Brain). In particular any "sparse" approach is deeply suspect as far as its practical performance or memory efficiency claims: current hardware does not deal with sparsity well unless things are _really_ sparse (like 1/10th or more) and sparsity is able to outweigh architectural inefficiencies.