TH1_3 not defined #168
Comments
|
Lines 79 to 85 in 7d37122 I think we just don't have bootstrap for that type here |
|
idk how this works, looks like it does literally nothing, but if you want, try adding these two lines struct TH1_3 <: TH1 end
function readfields!(io, fields, T::Type{TH1_3}) end |
|
I added the two lines from Moelf in bootstrap.jl which fixed this issue but I need to do the same sort of thing for many other types (TAxis_ , TAttAxis_ TNamed_ , ...) and at the end I got the following error: (geant4-v11.0.1 , OS: Ubuntu 20.04.4 LTS) |
|
can you send an example file? |
|
Hi Jerry,
Here is this very simple root file having only one TH1D histogram which I can’t read with UnROOT
… Le 11 mai 2022 à 17:33, Jerry Ling ***@***.***> a écrit :
can you send an example file?
—
Reply to this email directly, view it on GitHub <#168 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AKNFOYO5JTEIRLMGXPFAQP3VJPHL5ANCNFSM5VSACI5A>.
You are receiving this because you authored the thread.
|
|
sorry I can't see your file |
|
What do you mean by you can’t see the file ?
I attached it again in this email.
… Le 11 mai 2022 à 19:50, Jerry Ling ***@***.***> a écrit :
sorry I can't see your file
—
Reply to this email directly, view it on GitHub <#168 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AKNFOYNCWXTOLJS3AGG6NXTVJPXNJANCNFSM5VSACI5A>.
You are receiving this because you authored the thread.
|
|
you're not sending email to me directly, see: #168 |
|
Here is the file concerned |
|
Oups sorry.
I just send a .gz file
… Le 11 mai 2022 à 20:03, Jerry Ling ***@***.***> a écrit :
you're not sending email to me directly, see: #168 <#168>
—
Reply to this email directly, view it on GitHub <#168 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AKNFOYIZKBWUTD66RTGCEYDVJPY6DANCNFSM5VSACI5A>.
You are receiving this because you authored the thread.
|
|
looks like this is the difference between # this issue
In [3]: f = up.open("./dedx2COMET.root")
In [4]: f.keys()
Out[4]: ['h1;1']
In [5]: f["h1"]
Out[5]: <TH1D (version 1) at 0x7f07b281ce20>
In [6]: f["h1"]
Out[6]: <TH1D (version 1) at 0x7f07b281ce20>
# what UnROOT.jl knows how to handle
In [4]: f["myTH1D"]
Out[4]: <TH1D (version 2) at 0x7fa9a22892b0>I don't have experience in hunting down different internal version of ROOT object, asking @tamasgal |
|
In case it helps, you could try to copy the current structs and modify them based on these differences root [27] TH1::Class()->GetStreamerInfo(3)->ls()
StreamerInfo for class: TH1, version=3, checksum=0x5668b3d7
TNamed BASE offset= 0 type=67 The basis for a named object (name, title)
TAttLine BASE offset= 64 type= 0 Line attributes
TAttFill BASE offset= 80 type= 0 Fill area attributes
TAttMarker BASE offset= 96 type= 0 Marker attributes
Int_t fNcells offset=112 type= 3 number of bins(1D), cells (2D) +U/Overflows
TAxis fXaxis offset=120 type=61 X axis descriptor
TAxis fYaxis offset=336 type=61 Y axis descriptor
TAxis fZaxis offset=552 type=61 Z axis descriptor
Short_t fBarOffset offset=768 type= 2 (1000*offset) for bar charts or legos
Short_t fBarWidth offset=770 type= 2 (1000*width) for bar charts or legos
Stat_t fEntries offset=776 type= 8 Number of entries
Stat_t fTsumw offset=784 type= 8 Total Sum of weights
Stat_t fTsumw2 offset=792 type= 8 Total Sum of squares of weights
Stat_t fTsumwx offset=800 type= 8 Total Sum of weight*X
Stat_t fTsumwx2 offset=808 type= 8 Total Sum of weight*X*X
Double_t fMaximum offset=816 type= 8 Maximum value for plotting
Double_t fMinimum offset=824 type= 8 Minimum value for plotting
Double_t fNormFactor offset=832 type= 8 Normalization factor
TArrayD fContour offset=840 type=62 Array to display contour levels
TArrayD fSumw2 offset=864 type=62 Array of sum of squares of weights
TString fOption offset=888 type=65 histogram options
TList* fFunctions offset=912 type=63 ->Pointer to list of functions (fits and user)
i= 0, TNamed type= 67, offset= 0, len=1, method=0
i= 1, TAttLine type= 0, offset= 64, len=1, method=0
i= 2, TAttFill type= 0, offset= 80, len=1, method=0
i= 3, TAttMarker type= 0, offset= 96, len=1, method=0
i= 4, fNcells type= 3, offset=112, len=1, method=0
i= 5, fXaxis type= 61, offset=120, len=1, method=0
i= 6, fYaxis type= 61, offset=336, len=1, method=0
i= 7, fZaxis type= 61, offset=552, len=1, method=0
i= 8, fBarOffset type= 22, offset=768, len=2, method=0 [optimized]
i= 9, fEntries type= 28, offset=776, len=8, method=0 [optimized]
i=10, fContour type= 62, offset=840, len=1, method=0
i=11, fSumw2 type= 62, offset=864, len=1, method=0
i=12, fOption type= 65, offset=888, len=1, method=0
i=13, fFunctions type= 63, offset=912, len=1, method=0root [28] TH1::Class()->GetStreamerInfo(8)->ls()
StreamerInfo for class: TH1, version=8, checksum=0x1c3740c4
TNamed BASE offset= 0 type=67 The basis for a named object (name, title)
TAttLine BASE offset= 64 type= 0 Line attributes
TAttFill BASE offset= 80 type= 0 Fill area attributes
TAttMarker BASE offset= 96 type= 0 Marker attributes
int fNcells offset=112 type= 3 number of bins(1D), cells (2D) +U/Overflows
TAxis fXaxis offset=120 type=61 X axis descriptor
TAxis fYaxis offset=336 type=61 Y axis descriptor
TAxis fZaxis offset=552 type=61 Z axis descriptor
short fBarOffset offset=768 type= 2 (1000*offset) for bar charts or legos
short fBarWidth offset=770 type= 2 (1000*width) for bar charts or legos
double fEntries offset=776 type= 8 Number of entries
double fTsumw offset=784 type= 8 Total Sum of weights
double fTsumw2 offset=792 type= 8 Total Sum of squares of weights
double fTsumwx offset=800 type= 8 Total Sum of weight*X
double fTsumwx2 offset=808 type= 8 Total Sum of weight*X*X
double fMaximum offset=816 type= 8 Maximum value for plotting
double fMinimum offset=824 type= 8 Minimum value for plotting
double fNormFactor offset=832 type= 8 Normalization factor
TArrayD fContour offset=840 type=62 Array to display contour levels
TArrayD fSumw2 offset=864 type=62 Array of sum of squares of weights
TString fOption offset=888 type=65 histogram options
TList* fFunctions offset=912 type=63 ->Pointer to list of functions (fits and user)
int fBufferSize offset=920 type= 6 fBuffer size
double* fBuffer offset=928 type=48 [fBufferSize] entry buffer
TH1::EBinErrorOpt fBinStatErrOpt offset=968 type= 3 option for bin statistical errors
TH1::EStatOverflows fStatOverflows offset=972 type= 3 per object flag to use under/overflows in statistics
i= 0, TNamed type= 67, offset= 0, len=1, method=0
i= 1, TAttLine type= 0, offset= 64, len=1, method=0
i= 2, TAttFill type= 0, offset= 80, len=1, method=0
i= 3, TAttMarker type= 0, offset= 96, len=1, method=0
i= 4, fNcells type= 3, offset=112, len=1, method=0
i= 5, fXaxis type= 61, offset=120, len=1, method=0
i= 6, fYaxis type= 61, offset=336, len=1, method=0
i= 7, fZaxis type= 61, offset=552, len=1, method=0
i= 8, fBarOffset type= 22, offset=768, len=2, method=0 [optimized]
i= 9, fEntries type= 28, offset=776, len=8, method=0 [optimized]
i=10, fContour type= 62, offset=840, len=1, method=0
i=11, fSumw2 type= 62, offset=864, len=1, method=0
i=12, fOption type= 65, offset=888, len=1, method=0
i=13, fFunctions type= 63, offset=912, len=1, method=0
i=14, fBufferSize type= 6, offset=920, len=1, method=84435080
i=15, fBuffer type= 48, offset=928, len=1, method=920
i=16, fBinStatErrOpt type= 23, offset=968, len=2, method=0 [optimized]and similar for TAxis 6 vs 10. I didn't get further than that because of an EOF error :( |
|
I'm currently experiencing the same situation with TH1_6 not defined and wanted to check if there has been any progress / new input regarding this. |
|
Can you easily check how old those root files are? I feel like we must haven't experience this because we have been testing with new files |
|
They are at least from 2013 since they are part of the supplementary material of a statistics book we are working with that got published in that year I can ask if there's a more exact date but I doubt it, so your guess is likely correct since you mentionted most tests were made with NanoAOD files. |
|
Sorry for the very late response, I was on holidays and then had a lot of teaching to do. As @aminnj wrote, the |
|
Thanks for responding anyway and taking the time to look at it, since I'm
currently working with a workaround it's not urgent but I appreciate it!
Tamas Gal ***@***.***> schrieb am Di., 21. Juni 2022, 09:53:
… Sorry for the very late response, I was on holidays and then had a lot of
teaching to do.
As @aminnj <https://github.com/aminnj> wrote, the GetStreamerInfo() is
very helpful but I also used uproot in the past successfully to create
the parser logic and compare between versions. That being said, I'd propose
to check how uproot handles it. There are still some unresolved mysteries
regarding old files, so maybe we are on hitting some higher barriers there
;) I will try to free up some time and look at your file.
—
Reply to this email directly, view it on GitHub
<#168 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ALY43GIYNLBQX6PTXXOVBBLVQFYI7ANCNFSM5VSACI5A>
.
You are receiving this because you commented.Message ID: <JuliaHEP/UnROOT.
***@***.***>
|
|
I have a similar issue that's probably related. I'm just attaching the file here. |
|
Thanks Jan. We really need a bit more contributors 🙈 I'll try to look at this tomorrow (I have another issue as draft for a week or so...) |
|
Thanks! I'd be interested in the workaround that @Momox357 has mentioned. At the moment I'm a bit stuck. |
|
You might be a bit disappointed in that case! Since I only had a few histograms that had this problem I wrote a ROOT script that exported the relevant data (bin content and error) to a .txt and imported that into a new histogram that I could continue working with. |
|
I just had a look at the file @Momox357 has provided and I even bump into a parsing error in uproot: I think that also uproot tries to simply parse data with possibly incompatible streamer versions and either it works -- which means it should be OK since there are several sanity checks in the ROOT format itself -- or produces some errors which indicate that some versions are not compatible. I tried to teach UnROOT that TH1_3 is TH1_8 but I think the problem is much more deeper. After working myself through all the streamer versions I encountered (like Which can only be some parsing error, unless the file is from the post-alien era 😉 |
|
OK I have a bit more information. I am now working with the file @jstrube provided in #168 (comment) When I add julia> using UnROOT
julia> f = ROOTFile("t.root");
julia> ENV["JULIA_DEBUG"] = "UnROOT";
julia> f["Primaries/px1"]
┌ Debug: Splitting path 'Primaries/px1' and getting items recursively
└ @ UnROOT ~/Dev/UnROOT.jl/src/root.jl:152
┌ Debug: Retrieving Primaries ('TTree')
└ @ UnROOT ~/Dev/UnROOT.jl/src/root.jl:157
┌ Debug: TTree: UnROOT.TKey32(2965, 2, 22758, 0x00000000, 73, 1, 5381164, 64, "TTree", "Primaries", "Step-by-step energy deposition")
└ @ UnROOT ~/Dev/UnROOT.jl/src/bootstrap.jl:899
┌ Debug: Compression type: ZL
└ @ UnROOT ~/Dev/UnROOT.jl/src/types.jl:154
┌ Debug: Compressed/uncompressed size in bytes: 2883 / 22758
└ @ UnROOT ~/Dev/UnROOT.jl/src/types.jl:155
┌ Debug: fIOFeatures is missing
└ @ UnROOT ~/Dev/UnROOT.jl/src/bootstrap.jl:955
┌ Debug: Reading branches
└ @ UnROOT ~/Dev/UnROOT.jl/src/bootstrap.jl:959
ERROR: EOFError: read end of file |
|
So I guess this is where to search first: Line 944 in 57e49d3 but this might not be the end of the story ;) |
|
Well, this looks like a different issue to me, somehow a recursion error when inside the Maybe the problem is that these branches are sitting at the top of the ROOT file, without any |
|
Hi, is there maybe any update on this topic? |
|
Hi @Cornelius-G ! I have not made any progress but if you could provide a small sample file, that would already increase the sample size greatly ;) |
|
hi @Cornelius-G , I think we're mainly much more ergonomic when dealing with TTree or RNTuple. Btw, you can try using oh and it would be nice to have a sample |
|
Unfortunately, the original file is quite large and should be looped over. |
|
I'm a bit confused what is as far as I can tell the only known version is |
|
Well, I have absolutely no idea about root files and the parser. So I'm sorry if I'm just saying things that you already know.... I just played around a bit with the With this I end up with an Not sure how to get further from this point. But I guess that was also your problem 😅 |
|
The problem is probably already happening before reaching the AttrMarker. I think there is some systematics that the version is parsed as "1". Maybe something right before is not parsed correctly. The ROOT structure is fairly complex and the only source of information we have is code, more or less ;) If uproot can parse the file, then we have some inconsistencies in the streamer parsing. The good news is that it's at the beginning of the chain. If you have time and are eager, you can look up what uproot does and compare with UnROOT. I have little time for this at the moment, very sorry |
|
Is there an update on this? This is what I've done. I saved the file here https://scikit-hep.org/uproot3/examples/hepdata-example.root locally and tried to read the histograms using the following snippet |
|
yeah sorry about that, we don't know what does the internal of |
|
I will do another reverse engineering session to see if I can figure out something... |
|
So I am pretty sure we are missing something. Now I am a bit surprised why which is a doubly linked list ( |
|
I got further (cf #309) but still struggling to fully parse the TH1 with class version 7. There are a bunch of graphics formatting things saved in this file, so it's not as easy as parsing the histogram. There are |
|
Coming back to the original issue ( Btw. I added a debug function to compare streamers. It's very rudimentary but does the job. Here I compare the streamers for the file with julia> f1 = ROOTFile("/Users/tamasgal/Downloads/dedx2COMET.root")
ROOTFile with 1 entry and 53 streamers.
/Users/tamasgal/Downloads/dedx2COMET.root
└─ h1 (TH1D)
julia> f2 = ROOTFile("test/samples/histograms1d2d.root")
ROOTFile with 5 entries and 14 streamers.
test/samples/histograms1d2d.root
├─ myTH1F (TH1F)
├─ myTH1D (TH1D)
├─ myTH2F (TH2F)
├─ myTH2D (TH2D)
└─ myTH1D_nonuniform (TH1D)
julia> UnROOT.Debug.streamerdiff(f1, f2, "TH1")
==========
A: class version 3 (checksum: 1449702359)
B: class version 8 (checksum: 473383108)
Common dependencies: TNamed, TAttMarker, TAttFill, TAttLine
--------------
A: TNamed
B: TNamed
Mismatch of field values in version:
A: 2
B: 4
Mismatch of field values in fMaxIndex:
A: Int32[0, 0, 0, 0, 0]
B: Int32[0, -541636036, 0, 0, 0]
--------------
A: TAttLine
B: TAttLine
Mismatch of field values in version:
A: 2
B: 4
Mismatch of field values in fMaxIndex:
A: Int32[0, 0, 0, 0, 0]
B: Int32[0, -1811462839, 0, 0, 0]
Mismatch of field values in fBaseVersion:
A: 1
B: 2
--------------
A: TAttFill
B: TAttFill
Mismatch of field values in version:
A: 2
B: 4
Mismatch of field values in fMaxIndex:
A: Int32[0, 0, 0, 0, 0]
B: Int32[0, -2545006, 0, 0, 0]
Mismatch of field values in fBaseVersion:
A: 1
B: 2
--------------
A: TAttMarker
B: TAttMarker
Mismatch of field values in version:
A: 2
B: 4
Mismatch of field values in fMaxIndex:
A: Int32[0, 0, 0, 0, 0]
B: Int32[0, 689802220, 0, 0, 0]
Mismatch of field values in fBaseVersion:
A: 1
B: 2
--------------
A: fNcells
B: fNcells
Mismatch of field values in version:
A: 2
B: 4
Mismatch of field values in fTypeName:
A: Int_t
B: int
--------------
A: fXaxis
B: fXaxis
Mismatch of field values in version:
A: 2
B: 4
Mismatch of field values in fSize:
A: 0
B: 216
--------------
A: fYaxis
B: fYaxis
Mismatch of field values in version:
A: 2
B: 4
Mismatch of field values in fSize:
A: 0
B: 216
--------------
A: fZaxis
B: fZaxis
Mismatch of field values in version:
A: 2
B: 4
Mismatch of field values in fSize:
A: 0
B: 216
--------------
A: fBarOffset
B: fBarOffset
Mismatch of field values in version:
A: 2
B: 4
Mismatch of field values in fTypeName:
A: Short_t
B: short
--------------
A: fBarWidth
B: fBarWidth
Mismatch of field values in version:
A: 2
B: 4
Mismatch of field values in fTypeName:
A: Short_t
B: short
--------------
A: fEntries
B: fEntries
Mismatch of field values in version:
A: 2
B: 4
Mismatch of field values in fTypeName:
A: Stat_t
B: double
--------------
A: fTsumw
B: fTsumw
Mismatch of field values in version:
A: 2
B: 4
Mismatch of field values in fTypeName:
A: Stat_t
B: double
--------------
A: fTsumw2
B: fTsumw2
Mismatch of field values in version:
A: 2
B: 4
Mismatch of field values in fTypeName:
A: Stat_t
B: double
--------------
A: fTsumwx
B: fTsumwx
Mismatch of field values in version:
A: 2
B: 4
Mismatch of field values in fTypeName:
A: Stat_t
B: double
--------------
A: fTsumwx2
B: fTsumwx2
Mismatch of field values in version:
A: 2
B: 4
Mismatch of field values in fTypeName:
A: Stat_t
B: double
--------------
A: fMaximum
B: fMaximum
Mismatch of field values in version:
A: 2
B: 4
Mismatch of field values in fTypeName:
A: Double_t
B: double
--------------
A: fMinimum
B: fMinimum
Mismatch of field values in version:
A: 2
B: 4
Mismatch of field values in fTypeName:
A: Double_t
B: double
--------------
A: fNormFactor
B: fNormFactor
Mismatch of field values in version:
A: 2
B: 4
Mismatch of field values in fTypeName:
A: Double_t
B: double
--------------
A: fContour
B: fContour
Mismatch of field values in version:
A: 2
B: 4
Mismatch of field values in fSize:
A: 0
B: 24
--------------
A: fSumw2
B: fSumw2
Mismatch of field values in version:
A: 2
B: 4
Mismatch of field values in fSize:
A: 0
B: 24
--------------
A: fOption
B: fOption
Mismatch of field values in version:
A: 2
B: 4
Mismatch of field values in fSize:
A: 0
B: 24
--------------
A: fFunctions
B: fFunctions
Mismatch of field values in version:
A: 2
B: 4
Mismatch of field values in fSize:
A: 0
B: 8
Missing in A: fBufferSize, fBuffer, fBinStatErrOpt, fStatOverflows
julia>
|
|
Also interesting there is no streamer for julia> f1 = ROOTFile("/Users/tamasgal/Downloads/dedx2COMET.root")
ROOTFile with 1 entry and 53 streamers.
/Users/tamasgal/Downloads/dedx2COMET.root
└─ h1 (TH1D)
julia> f2 = ROOTFile("test/samples/histograms1d2d.root")
ROOTFile with 5 entries and 14 streamers.
test/samples/histograms1d2d.root
├─ myTH1F (TH1F)
├─ myTH1D (TH1D)
├─ myTH2F (TH2F)
├─ myTH2D (TH2D)
└─ myTH1D_nonuniform (TH1D)
julia> UnROOT.streamerfor(f1, "TH1D")
UnROOT.StreamerInfo(UnROOT.TStreamerInfo{UnROOT.TObjArray}("TH1D", "", 0x0b288969, 1, UnROOT.TObjArray("", 0, Any[UnROOT.TStreamerBase
version: UInt16 0x0002
fOffset: Int64 0
fName: String "TH1"
fTitle: String "1-Dim histogram base class"
fType: Int32 0
fSize: Int32 0
fArrayLength: Int32 0
fArrayDim: Int32 0
fMaxIndex: Array{Int32}((5,)) Int32[0, 0, 0, 0, 0]
fTypeName: String "BASE"
fXmin: Float64 0.0
fXmax: Float64 0.0
fFactor: Float64 0.0
fBaseVersion: Int32 3
, UnROOT.TStreamerBase
version: UInt16 0x0002
fOffset: Int64 0
fName: String "TArrayD"
fTitle: String "Array of doubles"
fType: Int32 0
fSize: Int32 0
fArrayLength: Int32 0
fArrayDim: Int32 0
fMaxIndex: Array{Int32}((5,)) Int32[0, 0, 0, 0, 0]
fTypeName: String "BASE"
fXmin: Float64 0.0
fXmax: Float64 0.0
fFactor: Float64 0.0
fBaseVersion: Int32 1
])), Set(Any["TArrayD", "TH1"]))
julia> UnROOT.streamerfor(f2, "TH1D")
missing |
|
that's wild |
|
Yeah... Well, in the end it's just reading the streamers, composing and inherit fields from the bases (which are superclasses) and then append the remaining fields. After that it's just a recursive thing to read everything. This really needs time and work 😬 |
I use for the first time and I try to read a TH1F histogram produced with GEANT4
But I got the following issue:
julia> f = ROOTFile("/Users/lebrun/Desktop/dedx2COMET.root")
ROOTFile with 1 entry and 53 streamers.
/Users/lebrun/Desktop/dedx2COMET.root
└─ h1 (TH1D)
julia> f["h1"]
ERROR: UndefVarError: TH1_3 not defined
Did I make a trivial error ?
The text was updated successfully, but these errors were encountered: