Question regarding the training of the llama2 version

[JunZhan2000]

Thank you for your work!
Is the training of the mini-gpt4 in the llama2 version the same as the vicuna version, including aspects like training data and duration? In my actual usage, I've noticed that the llama2 version has many enhancements.
If there are any differences, please let me know what they are. Thank you very much.

[TsuTikgiau]

Hello! The training data and training strategy is exactly the same as the vicuna version. One different thing is, in the old vicuna version, we use the blip-2's q-former. In the llama2 version, we remove it. The linear layer now directly map the output of clip's vision encoder to LLM's input.

[TsuTikgiau]

BTW, may I ask what are these enhancements you found in the LLama2 version?

[JunZhan2000]

Thank you for your reply, sorry I didn't see it in time.
The most obvious is that the ability to follow instructions is much stronger. For example, I let it only do true or false questions. The first version often does not follow instructions or gibberish, while the second version usually answers yes or no according to instructions. I think it might be llama2, which is also seen in other multimodal models.

[Shivtej8446]

Can anyone guide step by step running the project in vs code