v0.8.0

The 0.8.0 release focuses on generative LLM features in KerasNLP.

Summary

• Added the Mistral and Gemma models.
• Allow passing dtype directly to backbone and task constructors.
• Add a settable sequence_length property to all preprocessing layers.
• Added enable_lora() to the backbone class for parameter efficient fine-tuning.
• Added layer attributes to backbone models for easier access to model internals.
• Added AlibiBias layer.

# Pass dtype to a model.
causal_lm = keras_nlp.MistralCausalLM.from_preset(
    "mistral_instruct_7b_en",
    dtype="bfloat16"
)
# Settable sequence length property.
causal_lm.preprocessor.sequence_length = 128
# Lora API.
causal_lm.enable_lora(rank=4)
# Easy layer attributes.
for layer in causal_lm.backbone.transformer_layers:
    print(layer.count_params())

What's Changed

• Fix test for recent keras 3 change by @mattdangerw in #1400
• Pass less state to jax generate function by @mattdangerw in #1398
• Add llama tokenizer by @mattdangerw in #1401
• Add Bloom Model by @abuelnasr0 in #1382
• Try fixing tests by @mattdangerw in #1411
• Revert "Pass less state to jax generate function (#1398)" by @mattdangerw in #1412
• Bloom tokenizer by @abuelnasr0 in #1403
• Update black formatting by @mattdangerw in #1415
• Add Alibi bias layer by @abuelnasr0 in #1404
• Pin to tensorflow-hub 0.16.0 to fix CI error by @sampathweb in #1420
• Update TF Text and remove TF Hub deps by @sampathweb in #1423
• Pin Jax Version in GPU CI by @sampathweb in #1430
• Add Bloom preprocessor by @abuelnasr0 in #1424
• Add layer attributes for all functional models by @mattdangerw in #1421
• Allow setting dtype per model by @mattdangerw in #1431
• Add a Causal LM model for Mistral by @tirthasheshpatel in #1429
• Fix bart by @mattdangerw in #1434
• Add a settable property for sequence_length by @mattdangerw in #1437
• Add dependabot to update GH Actions and Python dependencies by @pnacht in #1380
• Bump the github-actions group with 1 update by @dependabot in #1438
• Add 7B presets for Mistral by @tirthasheshpatel in #1436
• Update byte_pair_tokenizer.py to close merges file properly by @divyashreepathihalli in #1440
• bump version to 0.8 by @mattdangerw in #1441
• Update our sampler documentation to reflect usage by @mattdangerw in #1444
• Add Gemma model by @mattdangerw in #1448
• Version bump for dev release by @mattdangerw in #1449
• Version bump to 0.8.0 by @mattdangerw in #1450

New Contributors

• @dependabot made their first contribution in #1438
• @divyashreepathihalli made their first contribution in #1440

Full Changelog: v0.7.0...v0.8.0