Could not open file model.jl

[sama2689]

I am able to import the alexnet ONNX file with the code

using Flux, ONNX
cd("/home/pratiksamant/JuliaCodes/AlexNEt/bvlc_alexnet")
ONNX.load_model("model.onnx") 
weights=ONNX.load_weights("weights.bson")
weights

The weights load just fine with no issues.

However when I try to open the associated .jl file using
model=include("model.jl")
I get the error
'could not open file .../model.jl
include at boot.jl:328 [inlined]
include_relative(::Module, ::String) at loading.jl:1105
include(::Module, ::String) at Base.jl:31
include(::String) at client.jl:424
top-level scope at LoadAlexNEt.jl:6'

I have confirmed that the file does exist, but julia is not able to load it.

For reference, the alexnet model I am trying to load can be found here: https://github.com/onnx/models/tree/master/vision/classification/alexnet

[ayush-1506]

Are you sure you're in the same directory as the model.jl file? If not, you might have to specify the entire path to the file to open it.

[sama2689]

Yes I am sure that I am in the same directory, indeed readdir() even shows that "model.jl" is present in the directory. This seems to be an issue with the file itself, which ONNX is creating but cannot seem to read.

Here is the output of readdir:

julia> readdir()
12-element Array{String,1}:
 "model.jl"       
 "model.onnx"     
 "test_data_0.npz"
 "test_data_1.npz"
 "test_data_2.npz"
 "test_data_set_0"
 "test_data_set_1"
 "test_data_set_2"
 "test_data_set_3"
 "test_data_set_4"
 "test_data_set_5"
 "weights.bson" 

[ayush-1506]

This is really strange, I'm not quite sure what's happening here. I can see that you're doing cd from inside Julia. Could you cd to the directory before and try loading it? (Try running julia from the directory that contains model.onnx, so you don't have to cd).
Apart from this I can't think of a reason behind this. Doesn't look to be ONNX.jl's fault though.

[sama2689]

I tried cd-ing to the directory before but no luck :( I am certain the directory is correct, not sure what the issue could be especially as the weights load just fine

[stevengj]

include is not relative to the current working directory, it is relative to the directory of the file you are including from. So doing cd has no effect on it.

Just do

model=include("/home/pratiksamant/JuliaCodes/AlexNEt/bvlc_alexnet/model.jl")

(See the include documentation.)

[stevengj]

If you are doing this a lot, you could also define a function to include relative to the current working directory:

cwd_include(path::AbstractString) = include(joinpath(pwd(), path))

[sama2689]

I have tried model=include("/home/pratiksamant/JuliaCodes/AlexNEt/bvlc_alexnet/model.jl" but now I get the error
LoadError: UndefVarError: pad=(0, 0, 1, 1) not defined
in expression starting at /home/pratiksamant/JuliaCodes/AlexNEt/bvlc_alexnet/model.jl:4
top-level scope at model.jl:5
include at boot.jl:328 [inlined]
include_relative(::Module, ::String) at loading.jl:1105
include(::Module, ::String) at Base.jl:31
include(::String) at client.jl:424
top-level scope at LoadAlexNEt.jl:7

[iuliancioarca]

Maybe ONNX.jl does not yet support asymmetrical padding?

[ayush-1506]

I think I remember adding support for asymmetric padding. Are you on the latest Flux?

[iuliancioarca]

I tried to reproduce OPs error, using Flux 0.9 (latest?). Here's how lines 5 and 6 of the model.jl looks like:
c_1 = MaxPool((3, 3), var"pad=(0, 0, 1, 1)", var"stride=(2, 2)")
c_2 = CrossCor(weights["conv5_w_0"], weights["conv5_b_0"], var"stride=(1, 1)", var"pad=(1, 1, 1, 1)", var"dilation=(1, 1)")

[ayush-1506]

Thanks for that. Not sure where var is coming from, it wasn't there before. I should get time to fix it this weekend. Hopefully it's not a big change.

[sama2689]

Thank you so much! Please keep me posted.

[sama2689]

Hi Ayush,
Did you have time to fix it this weekend?
Thanks

[ayush-1506]

Not yet. However, if you just want to use the model, you can manually remove var from the model.jl file for now. I hope to look at this in the coming days.

[sama2689]

Thanks Ayush, how would I remove var from a model file?

[DrChainsaw]

@sama2689: The model.jl file should just be (poorly formatted and quite compact) julia code, so you can open it up in an editor and edit it like you would edit any code.

If you are in need of a quick fix you can try using this: https://github.com/DrChainsaw/ONNXmutable.jl

I think that alexnet model is one of the models I tested it with.

[sama2689]

Thanks, I have removed var from the model.jl file but now I get the error LoadError: UndefVarError: LRNorm not defined

The original model.jl is shown below

using Statistics 
Mul(a,b,c) = b .* reshape(c, (1,1,size(c)[a],1)) 
Add(axis, A ,B) = A .+ reshape(B, (1,1,size(B)[1],1)) 
begin
    c_1 = MaxPool((3, 3), var"pad=(0, 0, 1, 1)", var"stride=(2, 2)")
    c_2 = CrossCor(weights["conv5_w_0"], weights["conv5_b_0"], var"stride=(1, 1)", var"pad=(1, 1, 1, 1)", var"dilation=(1, 1)")
    c_3 = CrossCor(weights["conv4_w_0"], weights["conv4_b_0"], var"stride=(1, 1)", var"pad=(1, 1, 1, 1)", var"dilation=(1, 1)")
    c_4 = CrossCor(weights["conv3_w_0"], weights["conv3_b_0"], var"stride=(1, 1)", var"pad=(1, 1, 1, 1)", var"dilation=(1, 1)")
    c_5 = MaxPool((3, 3), var"pad=(0, 0, 0, 0)", var"stride=(2, 2)")
    c_6 = LRNorm(1.0f0, 5, 0.0001f0, 0.75f0)
    c_7 = CrossCor(weights["conv2_w_0"], weights["conv2_b_0"], var"stride=(1, 1)", var"pad=(2, 2, 2, 2)", var"dilation=(1, 1)")
    c_8 = MaxPool((3, 3), var"pad=(0, 0, 0, 0)", var"stride=(2, 2)")
    c_9 = LRNorm(1.0f0, 5, 0.0001f0, 0.75f0)
    c_10 = CrossCor(weights["conv1_w_0"], weights["conv1_b_0"], var"stride=(4, 4)", var"pad=(0, 0, 0, 0)", var"dilation=(1, 1)")
    c_11 = broadcast(Int64, Tuple(reverse(weights["OC2_DUMMY_1"])))
    c_12 = 1 * weights["fc6_b_0"]
    c_13 = 1 * weights["fc7_b_0"]
    c_14 = 1 * weights["fc8_b_0"]
    (x_15,)->begin
            edge_16 = reshape(c_1(relu.(c_2(relu.(c_3(relu.(c_4(c_5(c_6(relu.(c_7(c_8(c_9(relu.(c_10(x_15))))))))))))))), c_11)
            edge_17 = 1 * permutedims(edge_16, reverse(range(1, ndims(edge_16)))) * weights["fc6_w_0"]
            edge_18 = identity(relu.(broadcast(+, permutedims(edge_17, reverse(range(1, ndims(edge_17)))), c_12)))
            edge_19 = 1 * permutedims(edge_18, reverse(range(1, ndims(edge_18)))) * weights["fc7_w_0"]
            edge_20 = identity(relu.(broadcast(+, permutedims(edge_19, reverse(range(1, ndims(edge_19)))), c_13)))
            edge_21 = 1 * permutedims(edge_20, reverse(range(1, ndims(edge_20)))) * weights["fc8_w_0"]
            softmax(vec(broadcast(+, permutedims(edge_21, reverse(range(1, ndims(edge_21)))), c_14)))
        end
end

my edited version of model.jl is shown below

using Statistics
Mul(a,b,c) = b .* reshape(c, (1,1,size(c)[a],1))
Add(axis, A ,B) = A .+ reshape(B, (1,1,size(B)[1],1))
begin
    c_1 = MaxPool((3, 3))
    c_2 = CrossCor(weights["conv5_w_0"], weights["conv5_b_0"])
    c_3 = CrossCor(weights["conv4_w_0"], weights["conv4_b_0"])
    c_4 = CrossCor(weights["conv3_w_0"], weights["conv3_b_0"])
    c_5 = MaxPool((3, 3))
    c_6 = LRNorm(1.0f0, 5, 0.0001f0, 0.75f0)
    c_7 = CrossCor(weights["conv2_w_0"], weights["conv2_b_0"])
    c_8 = MaxPool((3, 3))
    c_9 = LRNorm(1.0f0, 5, 0.0001f0, 0.75f0)
    c_10 = CrossCor(weights["conv1_w_0"], weights["conv1_b_0"])
    c_11 = broadcast(Int64, Tuple(reverse(weights["OC2_DUMMY_1"])))
    c_12 = 1 * weights["fc6_b_0"]
    c_13 = 1 * weights["fc7_b_0"]
    c_14 = 1 * weights["fc8_b_0"]
    (x_15,)->begin
            edge_16 = reshape(c_1(relu.(c_2(relu.(c_3(relu.(c_4(c_5(c_6(relu.(c_7(c_8(c_9(relu.(c_10(x_15))))))))))))))), c_11)
            edge_17 = 1 * permutedims(edge_16, reverse(range(1, ndims(edge_16)))) * weights["fc6_w_0"]
            edge_18 = identity(relu.(broadcast(+, permutedims(edge_17, reverse(range(1, ndims(edge_17)))), c_12)))
            edge_19 = 1 * permutedims(edge_18, reverse(range(1, ndims(edge_18)))) * weights["fc7_w_0"]
            edge_20 = identity(relu.(broadcast(+, permutedims(edge_19, reverse(range(1, ndims(edge_19)))), c_13)))
            edge_21 = 1 * permutedims(edge_20, reverse(range(1, ndims(edge_20)))) * weights["fc8_w_0"]
            softmax(vec(broadcast(+, permutedims(edge_21, reverse(range(1, ndims(edge_21)))), c_14)))
        end
end

[DrChainsaw]

Hi,

It means that LRNorm is not implemented.

It seems like the package author left a hook for it so that people can implement it themselves if really needed.

From what I understand, the DL community has basically stopped using that type of layer as it turned out to not provide any value. This may explain why noone has prioritized getting an implementation into Flux yet. It will not work with my package either btw. Perhaps the model will perform roughly the same if you remove the LRNorms (perhaps with some retraining).

Does it have to be alexnet or could you use VGG or a resnet instead? I belive they don't use LRNorm.

[ayush-1506]

That's right; while I was working on this package, local response normalisation wasn't implemented in Flux. Initially I replaced all LRNorm layers with identity to merely bypass the input, but I remember the results were a bit inconsistent, so for the moment I've added a "hook", which means you'd need to define the forward pass of LRNorm yourself.