Generalist Neural Algorithmic Learner(Report)
The Generalist Neural Algorithmic Learner is a project aimed at developing a graph neural network capable of learning and executing a wide range of algorithms. Inspired by existing research, our goal was to create a single model that can solve multiple algorithmic tasks, generalizing beyond the distribution it was trained on. This project represents a significant step forward in demonstrating how reasoning abilities can be applied to diverse control flows and tasks.
- Ashwin Daswani, Boston University ([email protected])
- Duc Minh Nguyen, Boston University ([email protected])
- Rohan Sawant, Boston University ([email protected])
The Generalist Neural Algorithmic Learner is designed to learn the execution trace of algorithms given an intermediate state. Initially, the model is trained on single-task models for various algorithmic tasks. Subsequently, it is trained on multi-algorithmic tasks to evaluate its performance against the single-task models.
- Graph Neural Networks: Utilizes GNNs to learn and execute classical algorithmic tasks.
- Single and Multi-task Learning: Supports both single-task and multi-task experiment settings.
- Model Improvements: Includes various enhancements such as randomized position scalar, static hint elimination, encoder initialization, gradient clipping, and soft hint propagation.
To run the project, ensure you have the following dependencies installed:
- JAX (Note: JAX is not supported on Windows)
- NumPy
- TensorFlow
- CLRS from DeepMind
Install the required packages using the following command:
pip install -r requirements.txtTo use the Generalist Neural Algorithmic Learner, navigate to the /src directory and follow the instructions provided there for running the various experiments.
This project was inspired by the work on the generalist neural algorithmic learner by DeepMind and other related research in the field of neural algorithmic reasoning.
This project is open-source and available under the MIT license.