Generalized streaming control stop_sequences

[ArthurBook]

• Description:
  Introduces StopSequenceAwareStreamer that enhances streaming control by allowing end users to specify arbitrary stop sequences spanning multiple tokens, without knowledge of the tokenizer. This generalizes current functionality, adding capabilities without changing API. End users' current implementations will work the same while allowing longer stop sequences.
• Issue: -
• Dependencies: None
• Twitter handle: https://twitter.com/TheRealABook

Happy Friday <:)
This is still a draft in the sense that I would welcome your thoughts on: 1) where this logic would best be centralized. 2) which LLMs that we want to implement this for.
I've tested this out locally but will happily add unit tests for the components once we firm up the above two points!

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
langchain	⬜️ Ignored (Inspect)	Visit Preview		Jan 31, 2024 4:46pm

[ArthurBook]

Hi @hwchase17 ! Happy new year!

I think this feature could be useful while providing a convenient layer of abstraction that centralizes the streaming logic. Who would be best suited for giving it a look?

Many thanks in advance.

[ArthurBook]

Hi @baskaryan, @eyurtsev, @hwchase17.
Following up on this - let me know if there is any more clarifying examples that I would provide to speed up the review.

In a nutshell, this PR enables arbitrary stop_sequences to be passed and centralizes the streaming logic into Streamer(ABC).

[ArthurBook]

@hwchase17 Hi! Noticed that you closed this PR.
Do you have any feedback on why? Think this could be useful both as a feature and centralization of streaming logic.

[baskaryan]

not intentional, we can't reopen at the moment but will do so as soon as we can :/ See #16796

[ArthurBook]

Hi! @hwchase17 @baskaryan
Wanted to bump this back up.

[efriis]

howdy! This is in the deprecated huggingface community integration - could you confirm this is still an issue in the partner package, and if so, submit an issue and/or PR against that?