Segmentation fault (core dumped) with gpt4all models during concurrent execution

[krassowski]

Description

If the gpt4all models are downloaded they work ok as long as you wait for the previous prompt to finish. If you send two messages simultaneously the app crashes with:

Segmentation fault (core dumped)

It might be that there is some non-safe multi-threading involved. I saw this with a few models but I did not try all.

Reproduce

Send two messages without waiting for reply, e.g. message a and b:

See:

> Entering new ConversationChain chain...
Prompt after formatting:
You are Jupyternaut, a conversational assistant living in JupyterLab to help users.
You are not a language model, but rather an application built on a foundation model from GPT4All called orca-mini-3b-gguf2-q4_0.
You are talkative and you provide lots of specific details from the foundation model's context.
You may use Markdown to format your response.
Code blocks must be formatted in Markdown.
Math should be rendered with inline TeX markup, surrounded by $.
If you do not know the answer to a question, answer truthfully by responding that you do not know.
The following is a friendly conversation between you and a human.

Current conversation:

Human: test
AI:

> Finished chain.
[I 2023-11-18 22:10:51.075 ServerApp] Default chat handler resolved in 100958 ms.


> Entering new ConversationChain chain...
Prompt after formatting:
You are Jupyternaut, a conversational assistant living in JupyterLab to help users.
You are not a language model, but rather an application built on a foundation model from GPT4All called orca-mini-3b-gguf2-q4_0.
You are talkative and you provide lots of specific details from the foundation model's context.
You may use Markdown to format your response.
Code blocks must be formatted in Markdown.
Math should be rendered with inline TeX markup, surrounded by $.
If you do not know the answer to a question, answer truthfully by responding that you do not know.
The following is a friendly conversation between you and a human.

Current conversation:
Human: test
AI:  Hello! How can I assist you today?
Human: a
AI:


> Entering new ConversationChain chain...
Prompt after formatting:
You are Jupyternaut, a conversational assistant living in JupyterLab to help users.
You are not a language model, but rather an application built on a foundation model from GPT4All called orca-mini-3b-gguf2-q4_0.
You are talkative and you provide lots of specific details from the foundation model's context.
You may use Markdown to format your response.
Code blocks must be formatted in Markdown.
Math should be rendered with inline TeX markup, surrounded by $.
If you do not know the answer to a question, answer truthfully by responding that you do not know.
The following is a friendly conversation between you and a human.

Current conversation:
Human: test
AI:  Hello! How can I assist you today?
Human: b
AI:
Segmentation fault (core dumped)

Expected behavior

No crash

Context

• Python 3.11
• Operating System and version: Ubuntu 22
• Browser and version: Chrome
• JupyterLab version: 4.0.9

gpt4all                        2.0.2
jupyter_ai                     2.6.0
jupyter_ai_magics              2.6.0
langchain                      0.0.318
langsmith                      0.0.65

[dlqqq]

@krassowski Thanks for reporting this issue. This should be solvable by a new provider attribute like concurrency, which defaults to True in BaseProvider. Then the GPT4All provider can define this as False. The implementation would live somewhere in the RootChatHandler class.

[dlqqq]

Wait, we already have an allows_concurrency property. See the BedrockProvider class for an example. It should be straightforward to open a PR fixing this.

[kaosbeat]

is there any update on this? I'm experiencing the same behavior, but I do not understand how to implement this fix with the above instructions.

[dlqqq]

@kaosbeat Sorry to hear that! I've opened the fix for you just now. We'll try to include this in a patch release soon.