Request for LLM support

[HeldvonKosmos]

Dear Sir or Madam,

i know that there are only a few supporters for this project, but it has a big potential.
I have tried to use it for converting the new Llama3.2 Model, because i can't find any LLM that is written in Julia.
Can you please help, to add the missing Operations for converting LLM's?
I would like to contribute, but haven't the time to understand the code, because it looks very complex.

Thanks in advance!
Jonas

[DrChainsaw]

Hi,

The code is a bit of a mess, but it is (hopefully) designed to make adding new operations easy without having to tackle the mess.

Before looking further into that, have you tried using https://github.com/FluxML/ONNX.jl? It is the more "official" and well maintained project when it comes to importing ONNX models.

There is also https://github.com/jw3126/ONNXRunTime.jl which will allow you to run LLMs in Julia through onnxruntime.

Assuming you want to take a stab at implementing the new OPs:

What I would do is to just try to load the model, note down which OP OnnxNaiveNASflux complains about, look it up here and try to implement it.

For deserializing (i.e import), it shall be added to this file with test cases going in here (look at existing testcases and here to get an idea of how to download to ONNX testdata).

For serialization (i.e export), it shall be added to this file with test cases going in here.

I will of course provide assistance as needed.

After the OP is added, then try to load the model again and repeat the procedure :)

[HeldvonKosmos]

Thank you for the guide, i will have a look and will reply again, when there are any further questions.