Prefix is not removed from inline completions when streaming

[krassowski]

Description

Using GPT 4 from Open AI:

No streaming:

Streaming:

Reproduce

1. In inline completer settings change "streaming" to "Always"
2. Attempt to type something
3. See the prefix repeated

Expected behavior

No prefix.

Context

2.17.0

[krassowski]

For bulk completion the post-processing method is always applied:

jupyter-ai/packages/jupyter-ai-magics/jupyter_ai_magics/providers.py

Lines 452 to 458 in 02d1966

	async def generate_inline_completions(
	self, request: InlineCompletionRequest
	) -> InlineCompletionReply:
	chain = self._create_completion_chain()
	model_arguments = completion.template_inputs_from_request(request)
	suggestion = await chain.ainvoke(input=model_arguments)
	suggestion = completion.post_process_suggestion(suggestion, request)

For streaming it is applied conditionally:

jupyter-ai/packages/jupyter-ai-magics/jupyter_ai_magics/providers.py

Lines 488 to 497 in 02d1966

	async for fragment in chain.astream(input=model_arguments):
	suggestion += fragment
	if suggestion.startswith("```"):
	if "\n" not in suggestion:
	# we are not ready to apply post-processing
	continue
	else:
	suggestion = completion.post_process_suggestion(suggestion, request)
	elif suggestion.rstrip().endswith("```"):
	suggestion = completion.post_process_suggestion(suggestion, request)

Some more refined logic is needed so that the prefix which may be added after opening ``` is also trimmed. It needs to keep track if trimming was performed to avoid removing the prefix-like string second time.