Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

New mcore transformer block spec #8925

Merged
merged 6 commits into from
Apr 24, 2024
Merged

New mcore transformer block spec #8925

merged 6 commits into from
Apr 24, 2024

Conversation

jbaczek
Copy link
Collaborator

@jbaczek jbaczek commented Apr 15, 2024

What does this PR do ?

This PR leverages new mcore gpt spece handling. (not yet merged to mcore)

Collection: [Note which collection this PR will affect]

Changelog

  • Add specific line by line info of high level changes in this PR.

Usage

  • You can potentially add a usage example below
# Add a code snippet demonstrating how to use this

Jenkins CI

To run Jenkins, a NeMo User with write access must comment jenkins on the PR.

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you add or update any necessary documentation?
  • Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
    • Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

  • New Feature
  • Bugfix
  • Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.

Additional Information

  • Related to # (issue)

@github-actions github-actions bot added the NLP label Apr 15, 2024
github-advanced-security[bot]
github-advanced-security bot found potential problems Apr 15, 2024
View reviewed changes
nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py
return ModuleSpec(module=TETransformerLayerAutocast)
num_layers = get_num_layers_to_build(transformer_config)
return TransformerBlockSubmodules(
layer_specs=[ModuleSpec(module=TETransformerLayerAutocast)] * num_layers, layer_norm=FusedLayerNorm

Check failure

Code scanning / CodeQL

Wrong name for an argument in a class instantiation

Keyword argument 'module' is not a supported parameter name of [ApexGuardDefaults.__init__](1).
@jbaczek jbaczek changed the base branch from main to r2.0.0.rc0.beta April 19, 2024 19:08
@jbaczek jbaczek force-pushed the jbaczek/new_mcore_transformer_block_spec branch from bdcad3c to dcd9fe5 Compare April 21, 2024 13:36
@github-actions github-actions bot added the CI label Apr 21, 2024
@jbaczek jbaczek force-pushed the jbaczek/new_mcore_transformer_block_spec branch from dcd9fe5 to 5f40024 Compare April 23, 2024 12:52
@github-actions github-actions bot removed the CI label Apr 23, 2024
@jbaczek jbaczek changed the title [draft] New mcore transformer block spec New mcore transformer block spec Apr 23, 2024
@jbaczek jbaczek force-pushed the jbaczek/new_mcore_transformer_block_spec branch from 89fe163 to ee457eb Compare April 24, 2024 14:20
@pablo-garay pablo-garay merged commit 8f95646 into r2.0.0.rc0.beta Apr 24, 2024
10 checks passed
@pablo-garay pablo-garay deleted the jbaczek/new_mcore_transformer_block_spec branch April 24, 2024 22:36
github-actions bot pushed a commit that referenced this pull request Apr 24, 2024
@jbaczek @ericharper
@pablo-garay @pre-commit-ci @web-flow
New mcore transformer block spec (#8925)
979bc45 
* update package info (#8793)

Signed-off-by: eharper <[email protected]>

* update mcore (#8917)

Signed-off-by: Jan Baczek <[email protected]>

* Use new mcore transformer block config handling

Signed-off-by: Jan Baczek <[email protected]>

* API fixes

Signed-off-by: Jan Baczek <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Revert chages to CI and Dockerfile

Signed-off-by: Jan Baczek <[email protected]>

---------

Signed-off-by: eharper <[email protected]>
Signed-off-by: Jan Baczek <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Pablo Garay <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
jbaczek added a commit that referenced this pull request Jul 19, 2024
@jbaczek @ericharper
@pablo-garay @pre-commit-ci
New mcore transformer block spec (#8925)
a7ae71d 
* update package info (#8793)

Signed-off-by: eharper <[email protected]>

* update mcore (#8917)

Signed-off-by: Jan Baczek <[email protected]>

* Use new mcore transformer block config handling

Signed-off-by: Jan Baczek <[email protected]>

* API fixes

Signed-off-by: Jan Baczek <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Revert chages to CI and Dockerfile

Signed-off-by: Jan Baczek <[email protected]>

---------

Signed-off-by: eharper <[email protected]>
Signed-off-by: Jan Baczek <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Pablo Garay <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Jan Baczek <[email protected]>
jbaczek added a commit that referenced this pull request Jul 26, 2024
@github-actions @jbaczek
@ericharper @pablo-garay @pre-commit-ci
New mcore transformer block spec (#9035)
7506f98 
* New mcore transformer block spec (#8925)

* update package info (#8793)

Signed-off-by: eharper <[email protected]>

* update mcore (#8917)

Signed-off-by: Jan Baczek <[email protected]>

* Use new mcore transformer block config handling

Signed-off-by: Jan Baczek <[email protected]>

* API fixes

Signed-off-by: Jan Baczek <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Revert chages to CI and Dockerfile

Signed-off-by: Jan Baczek <[email protected]>

---------

Signed-off-by: eharper <[email protected]>
Signed-off-by: Jan Baczek <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Pablo Garay <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Jan Baczek <[email protected]>

* Adjust function calls after branch rebase

Signed-off-by: Jan Baczek <[email protected]>

---------

Signed-off-by: eharper <[email protected]>
Signed-off-by: Jan Baczek <[email protected]>
Co-authored-by: jbaczek <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Pablo Garay <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jan Baczek <[email protected]>
BoxiangW pushed a commit to BoxiangW/NeMo that referenced this pull request Jul 30, 2024
@github-actions @jbaczek
@ericharper @pablo-garay @pre-commit-ci @BoxiangW
New mcore transformer block spec (NVIDIA#9035)
10a6dfe 
* New mcore transformer block spec (NVIDIA#8925)

* update package info (NVIDIA#8793)

Signed-off-by: eharper <[email protected]>

* update mcore (NVIDIA#8917)

Signed-off-by: Jan Baczek <[email protected]>

* Use new mcore transformer block config handling

Signed-off-by: Jan Baczek <[email protected]>

* API fixes

Signed-off-by: Jan Baczek <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Revert chages to CI and Dockerfile

Signed-off-by: Jan Baczek <[email protected]>

---------

Signed-off-by: eharper <[email protected]>
Signed-off-by: Jan Baczek <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Pablo Garay <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Jan Baczek <[email protected]>

* Adjust function calls after branch rebase

Signed-off-by: Jan Baczek <[email protected]>

---------

Signed-off-by: eharper <[email protected]>
Signed-off-by: Jan Baczek <[email protected]>
Co-authored-by: jbaczek <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Pablo Garay <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jan Baczek <[email protected]>
Signed-off-by: Boxiang Wang <[email protected]>
xuanzic pushed a commit to xuanzic/NeMo that referenced this pull request Aug 1, 2024
@github-actions @jbaczek
@ericharper @pablo-garay @pre-commit-ci
New mcore transformer block spec (NVIDIA#9035)
d2a0b46 
* New mcore transformer block spec (NVIDIA#8925)

* update package info (NVIDIA#8793)

Signed-off-by: eharper <[email protected]>

* update mcore (NVIDIA#8917)

Signed-off-by: Jan Baczek <[email protected]>

* Use new mcore transformer block config handling

Signed-off-by: Jan Baczek <[email protected]>

* API fixes

Signed-off-by: Jan Baczek <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Revert chages to CI and Dockerfile

Signed-off-by: Jan Baczek <[email protected]>

---------

Signed-off-by: eharper <[email protected]>
Signed-off-by: Jan Baczek <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Pablo Garay <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Jan Baczek <[email protected]>

* Adjust function calls after branch rebase

Signed-off-by: Jan Baczek <[email protected]>

---------

Signed-off-by: eharper <[email protected]>
Signed-off-by: Jan Baczek <[email protected]>
Co-authored-by: jbaczek <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Pablo Garay <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jan Baczek <[email protected]>
Signed-off-by: Vivian Chen <[email protected]>
mmarcinkiewicz pushed a commit that referenced this pull request Sep 1, 2024
@jbaczek @ericharper
@pablo-garay @pre-commit-ci @mmarcinkiewicz
New mcore transformer block spec (#8925)
4d9a435 
* update package info (#8793)

Signed-off-by: eharper <[email protected]>

* update mcore (#8917)

Signed-off-by: Jan Baczek <[email protected]>

* Use new mcore transformer block config handling

Signed-off-by: Jan Baczek <[email protected]>

* API fixes

Signed-off-by: Jan Baczek <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Revert chages to CI and Dockerfile

Signed-off-by: Jan Baczek <[email protected]>

---------

Signed-off-by: eharper <[email protected]>
Signed-off-by: Jan Baczek <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Pablo Garay <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
monica-sekoyan pushed a commit that referenced this pull request Oct 14, 2024
@github-actions @jbaczek
@ericharper @pablo-garay @pre-commit-ci @monica-sekoyan
New mcore transformer block spec (#9035)
5632801 
* New mcore transformer block spec (#8925)

* update package info (#8793)

Signed-off-by: eharper <[email protected]>

* update mcore (#8917)

Signed-off-by: Jan Baczek <[email protected]>

* Use new mcore transformer block config handling

Signed-off-by: Jan Baczek <[email protected]>

* API fixes

Signed-off-by: Jan Baczek <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Revert chages to CI and Dockerfile

Signed-off-by: Jan Baczek <[email protected]>

---------

Signed-off-by: eharper <[email protected]>
Signed-off-by: Jan Baczek <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Pablo Garay <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Jan Baczek <[email protected]>

* Adjust function calls after branch rebase

Signed-off-by: Jan Baczek <[email protected]>

---------

Signed-off-by: eharper <[email protected]>
Signed-off-by: Jan Baczek <[email protected]>
Co-authored-by: jbaczek <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Pablo Garay <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jan Baczek <[email protected]>
hainan-xv pushed a commit to hainan-xv/NeMo that referenced this pull request Nov 5, 2024
@github-actions @jbaczek
@ericharper @pablo-garay @pre-commit-ci
New mcore transformer block spec (NVIDIA#9035)
700d6a8 
* New mcore transformer block spec (NVIDIA#8925)

* update package info (NVIDIA#8793)

Signed-off-by: eharper <[email protected]>

* update mcore (NVIDIA#8917)

Signed-off-by: Jan Baczek <[email protected]>

* Use new mcore transformer block config handling

Signed-off-by: Jan Baczek <[email protected]>

* API fixes

Signed-off-by: Jan Baczek <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Revert chages to CI and Dockerfile

Signed-off-by: Jan Baczek <[email protected]>

---------

Signed-off-by: eharper <[email protected]>
Signed-off-by: Jan Baczek <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Pablo Garay <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Jan Baczek <[email protected]>

* Adjust function calls after branch rebase

Signed-off-by: Jan Baczek <[email protected]>

---------

Signed-off-by: eharper <[email protected]>
Signed-off-by: Jan Baczek <[email protected]>
Co-authored-by: jbaczek <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
Co-authored-by: Pablo Garay <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jan Baczek <[email protected]>
Signed-off-by: Hainan Xu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ShriyaPalsamudram ShriyaPalsamudram ShriyaPalsamudram approved these changes

Assignees
No one assigned
Labels
NLP
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@jbaczek @ShriyaPalsamudram @pablo-garay @ericharper