Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed printing to stdout when MIGRAPHX_TRACE_MLIR is enabled #2131

Closed

Conversation

ravil-mobile
Copy link
Contributor

@ravil-mobile ravil-mobile commented Aug 29, 2023

While working with migraphx + rocmlir, I noticed some gibberish in stdout which is set to on. For example,

$ head -n 20 before.out
Compiling ...
Reading: /MIGraphXDeps/resnet50-v1-7.onnx
mlir_main:pointwise4:x1.0 = @param:x1.0 -> float_type, {1, 64, 1, 1}, {64, 1, 1, 1}, target_id=0
mlir_main:pointwise4:@1 = multibroadcast[out_lens={1, mlir_main:pointwise2:x1.0mlir_4:y1 = mlir_main:pointwise6:x3.0 = @param = :@param64@param:, x1.056 -> , float_type56:x3.0 -> float_type, {},out_dyn_dims={}](mlir_main:pointwise4:x1.0) -> float_type, {1, 64, 56, 56}, {, 64, 1, 0, 0}, target_id=0
mlir_main:pointwise4:y1 = @param:y1 -> float_type, {64, 64, 3, 3}, {576, 9, 3, 1}, target_id=0
y1mlir_main:pointwise0:x1.0mlir_main:pointwise4:y0{ =  -> float_type, {@param:y0 -> float_type, {1, 64, 56, 56}, {1, 256, 1, 1}, {200704, 3136, 56, 1256, 1, 1, 1}}1, 64, 1, 1, target_id=}, , target_id={00
64, 1, 1, 1mlir_main:pointwise4:@4 = 256, 64, 1, 1convolution
}, mlir_main:pointwise6:x1.0 = @param:x1.0 -> float_type, {1, 256, 1, 1}, {256, 1, 1, 1}, target_id=0
mlir_main:pointwise10:x1.0 = @param:x1.0 -> mlir_main:pointwise6:@2 = float_typemultibroadcast{[out_lens=, {{1, 256, 56, 56},out_dyn_dims={}64, 1, 1, 1]} = (mlir_main:pointwise6:x3.0) -> float_type, {@param:1, 256, 56, 56}, {x1.0256, 1, 0, 0[, target_id=}0mlir_main:pointwise26:x1.0padding = 1, 64, 1, 1@param, target_id=
}, {mlir_4:y0 = @param:y0 -> float_type, {01, 64, 56, 56
mlir_main:pointwise6:x2 = @param:x2 -> float_type, {64, 1, 1, 11, 256, 56, 56}, {802816, 3136, 56, 1}, target_id=0
 -> float_typemlir_main:pointwise6:@4 = multibroadcast[out_lens={1, 256, 56, }, , {{56},out_dyn_dims={}]}(mlir_main:pointwise6:x1.0200704, 3136, 56, 11, 64, 1, 1}}, =) -> float_typemlir_main:pointwise32:x1.0, target_id=, target_id={1, 1, 1,  = @param:x1.0 -> {0
64, 1, 1, 1}, target_id=0
mlir_main:pointwise0:@1 = multibroadcast[out_lens={1, 64, 112, 112},out_dyn_dims={}](mlir_main:pointwise0:x1.0) -> float_type, {1, 64, 112, 112}, {64, 1, 0, 0}, target_id=0
mlir_main:pointwise0:y1 = @param:y1 -> float_type, {64, 3, 7, 7}, {147, 49, 7, 1}, target_id=0
mlir_main:pointwise0:y0 = @param:y0 -> float_type, {1, 3, 224, 224}, {150528, 50176, 224, 1}, target_id=0
mlir_main:pointwise0:@4 = convolution[padding={3, 3, 3, 3},stride={2, 2},dilation={1, 1},group=1,padding_mode=0](mlir_main:pointwise0:y0,mlir_main:pointwise0:y1) -> float_type, {1, 64, 112, 112}, {802816, 12544, 112, 1}, target_id=0
mlir_main:pointwise0:@5 = add(mlir_main:pointwise0:@4,mlir_main:pointwise0:@1) -> float_type, {1, 64, 112, 112}, {802816, 12544, 112, 1}, target_id=0
mlir_main:pointwise0:@6 = relu(mlir_main:pointwise0:@5) -> float_type, {1, 64, 112, 112}, {802816, 12544, 112, 1}, target_id=0
mlir_main:pointwise0:@7 = @return(mlir_main:pointwise0:@6), target_id=0

The problem is solved by surrounding all std::cout with lock guards. Here is the result

$ head -n 20 after.out
Compiling ...
Reading: /MIGraphXDeps/resnet50-v1-7.onnx
mlir_main:pointwise6:x3.0 = @param:x3.0 -> float_type, {1, 256, 1, 1}, {256, 1, 1, 1}, target_id=0
mlir_main:pointwise6:x1.0 = @param:x1.0 -> float_type, {1, 256, 1, 1}, {256, 1, 1, 1}, target_id=0
mlir_main:pointwise6:@2 = multibroadcast[out_lens={1, 256, 56, 56},out_dyn_dims={}](mlir_main:pointwise6:x3.0) -> float_type, {1, 256, 56, 56}, {256, 1, 0, 0}, target_id=0
mlir_main:pointwise6:x2 = @param:x2 -> float_type, {1, 256, 56, 56}, {802816, 3136, 56, 1}, target_id=0
mlir_main:pointwise6:@4 = multibroadcast[out_lens={1, 256, 56, 56},out_dyn_dims={}](mlir_main:pointwise6:x1.0) -> float_type, {1, 256, 56, 56}, {256, 1, 0, 0}, target_id=0
mlir_main:pointwise6:y1 = @param:y1 -> float_type, {256, 64, 1, 1}, {64, 1, 1, 1}, target_id=0
mlir_main:pointwise6:y0 = @param:y0 -> float_type, {1, 64, 56, 56}, {200704, 3136, 56, 1}, target_id=0
mlir_main:pointwise6:@7 = convolution[padding={0, 0, 0, 0},stride={1, 1},dilation={1, 1},group=1,padding_mode=0](mlir_main:pointwise6:y0,mlir_main:pointwise6:y1) -> float_type, {1, 256, 56, 56}, {802816, 3136, 56, 1}, target_id=0
mlir_main:pointwise6:@8 = add(mlir_main:pointwise6:@7,mlir_main:pointwise6:@4) -> float_type, {1, 256, 56, 56}, {802816, 3136, 56, 1}, target_id=0
mlir_main:pointwise6:@9 = add(mlir_main:pointwise6:@8,mlir_main:pointwise6:x2) -> float_type, {1, 256, 56, 56}, {802816, 3136, 56, 1}, target_id=0
mlir_main:pointwise6:@10 = add(mlir_main:pointwise6:@9,mlir_main:pointwise6:@2) -> float_type, {1, 256, 56, 56}, {802816, 3136, 56, 1}, target_id=0
mlir_main:pointwise6:@11 = relu(mlir_main:pointwise6:@10) -> float_type, {1, 256, 56, 56}, {802816, 3136, 56, 1}, target_id=0
mlir_main:pointwise6:@12 = @return(mlir_main:pointwise6:@11), target_id=0

mlir_main:pointwise17:x1.0 = @param:x1.0 -> float_type, {1, 64, 1, 1}, {64, 1, 1, 1}, target_id=0
mlir_main:pointwise17:@1 = multibroadcast[out_lens={1, 64, 56, 56},out_dyn_dims={}](mlir_main:pointwise17:x1.0) -> float_type, {1, 64, 56, 56}, {64, 1, 0, 0}, target_id=0
mlir_main:pointwise17:y1 = @param:y1 -> float_type, {64, 256, 1, 1}, {256, 1, 1, 1}, target_id=0
mlir_main:pointwise17:y0 = @param:y0 -> float_type, {1, 256, 56, 56}, {802816, 3136, 56, 1}, target_id=0

Closes ROCm/rocMLIR#1216

Copy link
Contributor

@manupak manupak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@codecov
Copy link

codecov bot commented Aug 29, 2023

Codecov Report

Merging #2131 (835c724) into develop (ac96370) will not change coverage.
The diff coverage is n/a.

❗ Current head 835c724 differs from pull request most recent head 967cc5f. Consider uploading reports for the commit 967cc5f to get more accurate results

@@           Coverage Diff            @@
##           develop    #2131   +/-   ##
========================================
  Coverage    91.43%   91.43%           
========================================
  Files          422      422           
  Lines        15771    15771           
========================================
  Hits         14420    14420           
  Misses        1351     1351           

@krzysz00 krzysz00 self-requested a review August 30, 2023 14:33
@ravil-mobile ravil-mobile force-pushed the ravil/mlir-trace-print-bug-fix branch from 4ae6ed5 to 7f1be9e Compare August 31, 2023 15:34
@causten
Copy link
Collaborator

causten commented Aug 31, 2023

Can resubmit this PR without being a fork? It seems our CI environment can't handle this request

@ravil-mobile
Copy link
Contributor Author

ravil-mobile commented Sep 4, 2023

Can resubmit this PR without being a fork? It seems our CI environment can't handle this request

@causten Yes, sure. Please, have a look at #2148

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

MIGraphX + rocMLIR: gibberish in the standard output when MIGRAPHX_TRACE_MLIR=1
5 participants