ai-research

research papers regarding AI.

HPC

• tpuv4
• slimfly
• efficient ampi all reduce
• flattened butterfly
• hamming mesh
• hyperx topology
• hyperx vs fat tree at scale
• rail optimized network

Optical

• direct connect topologies
• Massively scalable Kerr comb-driven silicon photonic link
• Peta-Scale Embedded Photonics Architecture for Distributed Deep Learning Applications

networking

• route packets not wires

LLM

• tree of thought
• toolformer

Training

• advancing MOE inference
• megablocks advancing moe experts
• Reflexion paper
• CASSINI: Network-Aware Job Scheduling in Machine Learning Clusters
• Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity

AI generic

• gshard all to all moe
• switch transformers

neuroscience

• ‘Hierarchy’ in the organization of brain networks
• interesting relating to network mapping on brain
• Prediction
• segregation of brain and networking in context to cognition