Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add fp8 support for SD/Update notebook paths #8387

Closed
wants to merge 66 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
37ac5a3
[tutorial] fixed missing RIR scripts file. (#8257)
XuesongYang Jan 29, 2024
7b2415a
add values to en tts dict (#7879)
mgrafu Jan 30, 2024
85d8756
Add Bert HF checkpoint converter (#8088)
yaoyu-33 Jan 31, 2024
f6e6485
Pin lhotse version to 1.19.2 (#8291)
pzelasko Jan 31, 2024
a4f1f1c
Fix documentation build (#8308)
artbataev Feb 1, 2024
5fdd12e
Cache Aware Streaming tutorial notebook (#8296) (#8311)
github-actions[bot] Feb 2, 2024
d10726d
Attention encoder-decoder models for multiple speech-to-text tasks …
pzelasko Feb 3, 2024
a5448f3
"Loop labels" greedy decoding: faster implementation (#8286)
artbataev Feb 3, 2024
5e22ff4
updated online sample mapping (#8181)
arendu Feb 5, 2024
dced14d
Fix memory leak caused by context parallelism hanging references by o…
github-actions[bot] Feb 5, 2024
d95624c
Fixing bug in tutorials. (#8335)
tbartley94 Feb 5, 2024
c2ea202
Support uploading NeMo models to HF via `push_to_hf_hub()` (#8263)
titu1994 Feb 6, 2024
9940ec6
add check for distributed optimizer which is unsupported for PEFT (#8…
cuichenx Feb 6, 2024
b0ddfa0
Remove asr webapp (#8347) (#8348)
github-actions[bot] Feb 6, 2024
4afc277
ASR Transcription Refactor (#8167)
titu1994 Feb 6, 2024
0fb851c
remove _target_ at model level in aed config (#8351) (#8352)
github-actions[bot] Feb 6, 2024
d3237d5
Update HF hub (#8349)
titu1994 Feb 7, 2024
2bc2e97
Change default (#8371) (#8372)
github-actions[bot] Feb 8, 2024
5a65505
Unfinished checkpoints handling (#7952)
jbieniusiewi Feb 8, 2024
c84121a
Improve communication overlapping in FP8 distributed optimizer (#8221)
timmoon10 Feb 8, 2024
b100cd1
Add AudioCodecModel to documentation (#8376)
anteju Feb 8, 2024
0bb9e66
Add longform infer for MultitaskAED models (#8355)
stevehuang52 Feb 9, 2024
6865c39
bug fix in fast-conformer-aed.yaml and adding jenkins test for speech…
github-actions[bot] Feb 9, 2024
220f2fa
Add fp8 support for SD/Update notebook paths
Victor49152 Feb 9, 2024
35f2fb3
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Feb 9, 2024
8a08b00
Reintroduce dictionaries for data prefixes in GPT (#8362)
jbaczek Feb 12, 2024
fe0fe23
Add Finetuning tutorial with HF Datasets (#8356) (#8393)
github-actions[bot] Feb 12, 2024
8349d63
Fixes for MoE parameter passing & use of AutoTokenizer/Model for mist…
github-actions[bot] Feb 12, 2024
0bfac69
Context-biasing by CTC-based Word Spotter (CTC-WS) (#8223)
andrusenkoau Feb 13, 2024
4e7293a
Fix Canary chunked infer on short audios (#8382)
stevehuang52 Feb 13, 2024
3d0e5ca
revert changes (#8410) (#8411)
github-actions[bot] Feb 13, 2024
1f519a9
Update NFA video download link (#8406) (#8408)
github-actions[bot] Feb 13, 2024
05c051b
updated link to pubmed (#8402) (#8407)
github-actions[bot] Feb 13, 2024
3a76b9d
Mcore customization doc (#8298) (#8405)
github-actions[bot] Feb 13, 2024
03a7e4f
Script for estimating Lhotse dynamic duration buckets (#8237)
pzelasko Feb 13, 2024
a06835f
Add Canary support for decoding with return_hypotheses=True (#8338)
stevehuang52 Feb 13, 2024
6e96e9c
[TTS] Add modules for mel spectrogram codec (#8238)
rlangman Feb 14, 2024
478ec6b
coldfix (#8412)
Jorjeous Feb 14, 2024
5a6bca2
Fixed errors in the CTM gen functions (#8416) (#8420)
github-actions[bot] Feb 14, 2024
14cbdc8
Add change_vocabulary and save_tokenizers() support to Multitask ASR …
github-actions[bot] Feb 14, 2024
21990e4
fix path location and branch (#8314)
github-actions[bot] Feb 15, 2024
36323cf
Add TP comm overlap knobs to AutocastTransformerLayer (#8290)
minitu Feb 15, 2024
71e9494
add deallocate pipeline output optimization (#8279) (#8318)
github-actions[bot] Feb 16, 2024
5a86625
remove assertion (#8302) (#8321)
github-actions[bot] Feb 16, 2024
b5ce971
Keep max_seqlen and cu_seqlens_argmin for later micro-batches when PP…
github-actions[bot] Feb 16, 2024
52f5611
Enable megatron core loggers for GPT pretraining (#8354) (#8384)
github-actions[bot] Feb 16, 2024
a2adae4
Fix dreambooth data sampler issue (#8400) (#8413)
github-actions[bot] Feb 16, 2024
536f573
add ensemble decoding fix (#8427) (#8433)
github-actions[bot] Feb 16, 2024
977f139
NeVA Tutorial Notebook (#8217)
PannuMuthu Feb 16, 2024
54ed7b1
mcore customization doc minor fix (#8421) (#8437)
github-actions[bot] Feb 16, 2024
4432962
Add `loop_labels` algorithm for TDT greedy decoding (#8215)
artbataev Feb 16, 2024
960b6a7
Add dist ckpt support for regular optimizers (#7749) (#8293)
github-actions[bot] Feb 16, 2024
a4b204c
Multimodal r1.23.0 bug fix (#8315) (#8339)
github-actions[bot] Feb 16, 2024
1e8aa8c
mcore ds fix (#8283) (#8385)
github-actions[bot] Feb 16, 2024
87589a7
MCore dataset compatibility for tokenizers (#8390) (#8397)
github-actions[bot] Feb 16, 2024
5c1b8d1
Canary: inference tokenization improvements; preserving custom keys w…
pzelasko Feb 16, 2024
8222634
add sbert to IR (#8445)
JRD971000 Feb 17, 2024
df5a395
Update readme (#8440)
ericharper Feb 17, 2024
e57abf5
NeMo-Mistral to HF converter bugfix. (#8353) (#8442)
github-actions[bot] Feb 20, 2024
9c07053
Fixing mcore bert for TP, PP and SP (#8336) (#8443)
github-actions[bot] Feb 20, 2024
81f56ea
Add LoRA support to all linear layers (#7988)
ertkonuk Feb 21, 2024
59aa22d
Add Neva Template for NV-DPO Models (#8358)
HuiyingLi Feb 21, 2024
5c3387d
Account for mpirun use case in get_rank (#8429)
janekl Feb 22, 2024
69e48ec
Add settings to suppress bf16 compile errors in CI on V100 (#8481) (#…
github-actions[bot] Feb 22, 2024
88e332d
fix canary chunk infer bug (#8449)
stevehuang52 Feb 22, 2024
3cfd019
Merge branch 'main' into mingyuanm/add_fp8_support_for_sd
Victor49152 Feb 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.

ARG BASE_IMAGE=nvcr.io/nvidia/pytorch:23.12-py3
ARG BASE_IMAGE=nvcr.io/nvidia/pytorch:24.01-py3

# build an image that includes only the nemo dependencies, ensures that dependencies
# are included first for optimal caching, and useful for building a development
Expand Down Expand Up @@ -66,19 +66,19 @@ WORKDIR /workspace/
# We leave it here in case we need to work off of a specific commit in main
RUN git clone https://github.com/NVIDIA/Megatron-LM.git && \
cd Megatron-LM && \
git checkout 27cbe46714a50c43ed290f1b1472db8d2780c55c && \
git checkout 240a8ef7a21df201e47b5b2ae33cc5f4c5486849 && \
pip install .

# Apex bugfix for PyTorch 23.11 container: https://github.com/NVIDIA/apex/pull/1760
# Performance optimizations for distributed optimizer: https://github.com/NVIDIA/apex/pull/1771
RUN git clone https://github.com/NVIDIA/apex.git && \
cd apex && \
git checkout c07a4cf67102b9cd3f97d1ba36690f985bae4227 && \
git checkout f058162b215791b15507bb542f22ccfde49c872d && \
pip install -v --no-build-isolation --disable-pip-version-check --no-cache-dir --config-settings "--build-option=--cpp_ext --cuda_ext --fast_layer_norm --distributed_adam --deprecated_fused_adam" ./

# Transformer Engine 1.2.0
RUN git clone https://github.com/NVIDIA/TransformerEngine.git && \
cd TransformerEngine && \
git fetch origin 4f9662fbe621671f5f905e772fc1138953af77f6 && \
git fetch origin da30634a6c9ccdbb6c587b6c93b1860e4b038204 && \
git checkout FETCH_HEAD && \
git submodule init && git submodule update && \
NVTE_FRAMEWORK=pytorch NVTE_WITH_USERBUFFERS=1 MPI_HOME=/usr/local/mpi pip install .
Expand Down
Loading
Loading