Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OnnxTransformer GetNamedOnnxValueUnknownSize Index Error #7348

Open
Eddie-Hartman opened this issue Dec 19, 2024 · 0 comments
Open

OnnxTransformer GetNamedOnnxValueUnknownSize Index Error #7348

Eddie-Hartman opened this issue Dec 19, 2024 · 0 comments
Labels
untriaged New issue has not been triaged

Comments

@Eddie-Hartman
Copy link

System Information (please complete the following information):

  • OS & Version: Windows 11
  • ML.NET Version: 3.0.1 (Also applicable for 4.0)
  • .NET Version: .NET 8.0

Describe the bug
Splitter/consolidator worker encountered exception while consuming source data error, but seems like the underlying ML.NET library code is the cause of the issue.
In OnnxTransform.cs, there is a _zeroIndex variable that looks for the tensorShape IndexOf(0). When not found, that will return -1, but then it tries to use that value in assignment for GetNamedOnnxValueUnknownSize() method, thus causing an index out of bounds error.

To Reproduce
Steps to reproduce the behavior:

  1. See below about getting access to repo and code.
  2. Unfortunately, other than that I'm not going to be really helpful other than pointing out where in the library it's occurring. The size is looking for a 0 in the shape, but there isn't one. I'm also a bit confused by the "UnknownSize" part in general since in my example I know the size of each input and output at each step.

Expected behavior
Error not to be thrown.

Screenshots, Code, Sample Projects
I can give temporary access to git repo to a Microsoft employee or contributor who'd look into this, but not making it public. The repo shows a model that runs successfully without using ML.NET pipelines (straight ONNX runtime) and has a separate method where this issue is occurring where I was trying to set up the ML.NET pipeline to work.

Here's a really trimmed down version where I'm just trying to get an EfficientNet model running. Last line is what throws the error.

        var pipeline = mlContext.Transforms.ResizeImages("resizedImage", classificationModelImageWidth, classificationModelImageHeight, nameof(ImageFile.Input), ImageResizingEstimator.ResizingKind.Fill)
            .Append(mlContext.Transforms.ExtractPixels("tensored_image", "resizedImage", ImagePixelExtractingEstimator.ColorBits.Rgb,
                ImagePixelExtractingEstimator.ColorsOrder.ARGB, true, scaleImage: 1f / 255f))
            .Append(mlContext.Transforms.CustomMapping<ImagePixelInput, ImagePixelOutput>((input, output) =>
            {
                //Normalize the values.
                //Since we are in CHW format, we will go through all R, G, then B pixels in that order.
                //So for each third us the appropriate value.
                output.NormalizedPixels = new float[input.ImagePixels.Length];
                float[] means = [0.485f, 0.456f, 0.406f];
                float[] stds = [0.229f, 0.224f, 0.225f];
                int colorLength = input.ImagePixels.Length / 3; //The number of values for a single color
                for (int c = 0; c < 3; c++)
                {
                    for (int i = 0; i < colorLength; i++)
                    {
                        output.NormalizedPixels[c * colorLength + i] = (input.ImagePixels[c * colorLength + i] - means[c]) / stds[c];
                    }
                }

                Console.WriteLine($"First 10 normalized pixels: {string.Join(", ", output.NormalizedPixels.Take(10))}");
            },
            "NormalizeImage"))
            .Append(mlContext.Transforms.ApplyOnnxModel("output", "input", classificationFilePath));

        var model = pipeline.Fit(data);

        // Read image into stream.
        using MemoryStream memoryStream = new(imageBytes);

        IEnumerable<ImageFile> images = [new(memoryStream)];
        IDataView imageDataView = mlContext.Data.LoadFromEnumerable(images);
        IDataView scoredData = model.Transform(imageDataView);

        var schema = scoredData.Schema;
        foreach (var column in schema)
        {
            Console.WriteLine($"{column.Name}: {column.Type}");
        }

        var test = scoredData.GetColumn<float[]>("input").First();

        // EDDIE current issue seems to be that _zeroIndex in get GetNamedOnnxValueUnknownSize in OnnxTransformer.cs

        var test2 = scoredData.GetColumn<float[]>("output").ToArray();

Additional context
I stumbled upon this after getting one ONNX model working fine with a transformer pipeline, but when I tried to do something similar with a different ONNX model some of the same code throws errors. I then set everything up to run as an InferenceSession instead and that works fine. So revisiting and working backwards to make sure everything was working the same in the transformers is what lead me to dig deeper and currently believe this is a bug.

To be clear, I'm new to this library and not super knowledgeable about this subject overall, so take everything I'm saying with a grain of salt.

@dotnet-policy-service dotnet-policy-service bot added the untriaged New issue has not been triaged label Dec 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
untriaged New issue has not been triaged
Projects
None yet
Development

No branches or pull requests

1 participant