Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
Ag2S1 authored Sep 30, 2022
1 parent a611deb commit f5105e1
Showing 1 changed file with 4 additions and 1 deletion.
5 changes: 4 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
# Mengzi-Retrieval-LM

"Retrieval" is a crucial way to improve the performance of language models. This repository is an experimental implementation of the retrieval-enhanced language model. **Currently, it only supports retrieval fitting on GPT-Neo.**
At Langboat Technology, we focus on how to enhance pre-trained models to make them lighter to satisfy real industry needs.
And retrieval is an important method to achieve this goal.

This repository is an experimental implementation of the retrieval-enhanced language model. **Currently, it only supports retrieval fitting on GPT-Neo.**

We forked [Huggingface Transformers](https://github.com/huggingface/transformers) and [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) to add retrieval support. The indexing part is implemented as an HTTP server to decouple retrieval and training better.

Expand Down

0 comments on commit f5105e1

Please sign in to comment.