Support for streaming messages

[jtpio]

Problem

Currently messages are added to the chat in one go. See this screencast as an example:

jupyterlab-codestral-demo-lite.webm

However most of the AI chat UIs stream responses to the user, for example:

mistral-ai-chat.webm

Proposed Solution

Add optional support for streaming messages, which could be enabled via the settings editor or by extensions.

Additional context

This would be useful for AI extensions using jupyter-chat for interacting with some online AI providers, such as https://github.com/jtpio/jupyterlab-codestral

[jtpio]

This might already be possible in some way by a plugin, by calling the updateMessage method:

jupyter-chat/packages/jupyter-chat/src/model.ts

Lines 76 to 85 in 5cf3955

	/**
	* Optional, to update a message from the chat panel.
	*
	* @param id - the unique ID of the message.
	* @param message - the updated message.
	*/
	updateMessage?(
	id: string,
	message: IChatMessage
	): Promise<boolean | void> | boolean | void;

[brichet]

This might already be possible in some way by a plugin, by calling the updateMessage method:

Yes, this is possible but will call the RendermimeMarkdown for the whole content on every update.

To avoid this, we could find a way to tell the message component to append the new content to the DOM, using the appendContent flag:

jupyter-chat/packages/jupyter-chat/src/components/rendermime-markdown.tsx

Lines 78 to 80 in 5cf3955

	(appendContent ? (
	<div ref={node => node && node.appendChild(renderedContent)} />
	) : (

[jtpio]

Linking to jupyterlab/jupyter-ai#228 as related.

[dlqqq]

@brichet Can this issue be closed now? Seems like the update message PR makes this possible.

I'm not sure if there's a use-case for disabling message streaming; I think we can open another issue to track that if needed.

[brichet]

Yes, this issue can be closed.

I'm not sure if there's a use-case for disabling message streaming; I think we can open another issue to track that if needed.

Currently, there is no concept of streaming in the chat extension. There is only a way to update a message.
I think that this concept is really related to generative AI stuff (correct me if I'm wrong), so a feature to disable message streaming would make more sense in jupyter-ai.