This repository accompanies a hands-on training event to introduce data scientists (and ML-ready developers / technical leaders) to core model training and deployment workflows with Amazon SageMaker.
Like a "101" course in the academic sense, this will likely not be the simplest introduction to SageMaker you can find; nor the fastest way to get started with advanced features like optimized SageMaker Distributed training or SageMaker Clarify for bias and explainability analyses.
Instead, these exercises are chosen to demonstrate some core build/train/deploy patterns that we've found help new users to first get productive with SageMaker - and to later understand how the more advanced features fit in.
An interactive walkthrough of the content with screenshots is available at:
Sessions in suggested order:
- builtin_algorithm_hpo_tabular: Explore some pre-built algorithms and tools for tabular data, including SageMaker Canvas, SageMaker AutoML APIs, the XGBoost built-in algorithm, and automatic hyperparameter tuning
- This module also includes a quick initial look at SageMaker Feature Store, SageMaker Model Registry, and the AutoGluon built-in algorithm - but you don't need to dive deep on these topics.
- custom_script_demos: See how you can train and deploy your own models on SageMaker with custom Python scripts and the pre-built framework containers
- (Optional) Start with sklearn_reg for an introduction if you're new to deep learning but familiar with Scikit-Learn
- See huggingface_nlp (preferred) for a side-by-side comparison of in-notebook versus on-SageMaker model training and inference for text classification - or alternatively the custom CNN-based keras_nlp or pytorch_nlp examples.
- migration_challenge: Apply what you learned to port an in-notebook workflow to a SageMaker training job + endpoint deployment on your own
- Choose the sklearn_cls, keras_mnist or pytorch_mnist challenge, depending which ML framework you're most comfortable with.
The recommended way to explore these exercises is to onboard to SageMaker Studio. Once you've done this, you can download this repository by launching a System terminal (From the "Utilities and files" section of the launcher screen inside Studio) and running git clone https://github.com/aws-samples/sagemaker-101-workshop.
If you prefer to use classic SageMaker Notebook Instances, you can find a CloudFormation template defining a simple setup at .simple.cf.yaml. This can be deployed via the AWS CloudFormation Console.
You can refer to the "How Are Amazon SageMaker Studio Notebooks Different from Notebook Instances?" docs page for more details on differences between the Studio and Notebook Instance environments.
Depending on your setup, you may be asked to choose a kernel when opening some notebooks. There should be guidance at the top of each notebook on suggested kernel types, but if you can't find any, Data Science 3.0 (Python 3) (on Studio) or conda_python3 (on Notebook Instances) are likely good options.
Some of the examples depend on ipywidgets and ipycanvas for interactive inference demo widgets (but do provide code-only alternatives).
We also usually enable some additional JupyterLab extensions powered by jupyterlab-lsp and jupyterlab-s3-browser to improve user experience. You can find more information about these extensions in this AWS ML blog post
ipywidgets should be available by default on SageMaker Studio, but not on Notebook Instances when we last tested. The other extensions require installation.
To see how we automate these extra setup steps for AWS-run events, you can refer to the lifecycle configuration scripts in our CloudFormation templates. For a Notebook Instance LCC, see the AWS::SageMaker::NotebookInstanceLifecycleConfig in .simple.cf.yaml. For a SageMaker Studio LCC, see the Custom::StudioLifecycleConfig in .infrastructure/template.sam.yaml.
See CONTRIBUTING for more information.
This library is licensed under the MIT-0 License. See the LICENSE file.
One major focus of this workshop is how SageMaker helps us right-size and segregate compute resources for different ML tasks, without sacrificing (and ideally accelerating!) data scientist productivity. For more information on this topic, see this post on the AWS Machine Learning Blog: Right-sizing resources and avoiding unnecessary costs in Amazon SageMaker
For a workshop that starts with a similar migration-based approach, but dives further into automated pipelines and CI/CD, check out aws-samples/amazon-sagemaker-from-idea-to-production.
As you continue to explore Amazon SageMaker, you'll also find many more useful resources in:
- The official Amazon SageMaker Examples repository: with a broad range of code samples covering SageMaker use cases from beginner to expert.
- The documentation (and maybe even the source code) for the SageMaker Python SDK: The high-level, open-source PyPI library we use when we
import sagemaker. - The Amazon SageMaker Developer Guide: documenting the SageMaker service itself.
More advanced users may also find it helpful to refer to:
- The boto3 reference for SageMaker and the SageMaker API reference: in case you have use cases for SageMaker where you want (or need) to use low-level APIs directly, instead of through the
sagemakerlibrary. - The AWS Deep Learning Containers and SageMaker Scikit-Learn Containers source code: For a deeper understanding of the framework container environments.