Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GetTempPath hot in some benchmarks in Microsoft.ML.PerformanceTests #7277

Open
alexcovington opened this issue Oct 22, 2024 · 0 comments
Open
Labels
untriaged New issue has not been triaged

Comments

@alexcovington
Copy link

I have noticed that depending on the length of the temporary path, some benchmarks see a significant performance drop because of the need to convert from a Windows short name to a long name.

If I use the temporary directory path C:\tempdir\, the benchmark gives these performance results on my machine:

Microsoft.ML.PerformanceTests.exe --filter '*CreatePredictionEngine' --envVars TEMP:C:\tempdir TMP:C:\tempdir


| Method                 | Mean     | Error   | StdDev  | Extra Metric |
|----------------------- |---------:|--------:|--------:|-------------:|
| CreatePredictionEngine | 249.5 us | 4.80 us | 4.93 us |            - |

If I use a long temporary directory C:\Users\username\AppData\Local\Temp\ML.NET\A Very Very Long File Name That Will Be Converted To A Short Path\Temp (which has the short name C:\Users\username\AppData\Local\Temp\ML.NET\AVERYV~1\Temp), the benchmark shows significantly worse performance on the same machine:

 .\Microsoft.ML.PerformanceTests.exe --filter '*CreatePredictionEngine' --envVars TEMP:C:\Users\username\AppData\Local\Temp\ML.NET\AVERYV~1\Temp TMP:C:\Users\username\AppData\Local\Temp\ML.NET\AVERYV~1\Temp

| Method                 | Mean     | Error    | StdDev   | Extra Metric |
|----------------------- |---------:|---------:|---------:|-------------:|
| CreatePredictionEngine | 10.53 ms | 0.082 ms | 0.077 ms |            - |

Looking at this under PerfView, I see a lot of extra time spent here.

I'm wondering if it is possible to cache this lookup? It seems misleading for the benchmark performance to fluctuate so much because of the temporary directory path.

@dotnet-policy-service dotnet-policy-service bot added the untriaged New issue has not been triaged label Oct 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
untriaged New issue has not been triaged
Projects
None yet
Development

No branches or pull requests

1 participant