Developed a healthcare chatbot using the open-source large language model: mistralai/Mistral-7B-v0.1. The chatbot is fine-tuned on a healthcare dataset (lavita/ChatDoctor-HealthCareMagic-100k) and responds to user queries. The project includes data preprocessing, model fine-tuning, chatbot interface development and performance evaluation.
- medical_chatbot.ipynb: Main script of the project
- requirements.txt: List of dependencies.
- evaluation_metrics.txt: File to save evaluation results.
- Python 3.10 or higher
- PyTorch
- Transformers library from Hugging Face
- Datasets library from Hugging Face
- Gradio for the chatbot interface
- Additional libraries as listed in requirements.txt
I used the ChatDoctor-HealthCareMagic-100k dataset, containing around 110K+ rows of patient queries and doctor responses. For this project due to the limited GPU capability, I sampled 5000 rows for fine-tuning the model.
I formatted the dataset into a specific structure to be fed into the LLM and tokenized it using Huggingface’s AutoTokenizer. The input format includes patient queries and doctor responses, with padding tokens added to the beginning of the sequence.
I used quantized low-rank adaptation (QLoRA) to load the model in 4-bit mode, significantly reducing memory usage without sacrificing performance.
Further, I used Parameter Efficient Fine-Tuning (PEFT) and LoRA to freeze most of the model parameters and fine-tune a small subset. This allows efficient fine-tuning even with limited data and computational resources.
I trained the model for 500 steps, saving checkpoints along the way. This helps in restarting the fine-tuning process of the model from the point it stops.
I initially attempted to perform the evaluation using the "lm evaluation harness," but due to compatibility issues with my environment, I was unable to execute it successfully. Consequently, I evaluated the model by computing BLEU, ROUGE, and METEOR scores and comparing them to the responses provided in the "truthful_qa" reference dataset from Hugging Face.
The final fine-tuned model can be used to generate responses to medical queries through a user-friendly interface built with Gradio.
Here are some example queries you can try with the chatbot:
- "Hi Doctor, I am feeling feverish. What should I do?
- "How can I maintain my insulin spikes?"
