fix python hierarchy so jupyter kernel restart is not required

[nicolasaunai]

On jupyter :
so far if the h5 file is loaded, changes and is reloaded (via GetRun and thus via hierarchy_from), reloading has no effect and the hierarchy still contains data before the file has changed. only kernel restart makes reload effective.
The goal of this issue is to change that

[PhilipDeegan]

There's a PR for this
I don't use jupyter a lot, but we should try it out

[nicolasaunai]

Will investigate

[nicolasaunai]

H5 persistence

This notebook aims to check to what extent re-reading a h5 file that has changed really read the file or keep old data.
#527

import h5py as h5
import numpy as np

#from ipython import add_h5_weak_ref

def read_data():
    f = h5.File("test.h5", "r")
    datas = []
    for k in f.keys():
        print("k = {}".format(k))
        datas += [ f[k] ]
    return datas
    

!cat datasets.py

#! /bin/env python

import sys
import h5py as h5
import numpy as np
import os

def create_file():
    f = h5.File("test.h5","w")    
    data = np.arange(100)
    fd = f.create_dataset("data", data=data)
    f.close()

def create_second():
    f = h5.File("test2.h5", "a")
    fd = f.create_dataset("data", data=np.arange(100))
    fd = f.create_dataset("data2", data=np.arange(100)+1000)
    os.rename("test2.h5", "test.h5")
    f.close()

def main():
    if sys.argv[1] == "create":
        create_file()
    elif sys.argv[1] == "replace":
        create_second()
    else:
        raise ValueError("either create or add")



if __name__ == "__main__":
    main()

We simulate the creation of data

!python datasets.py create

!h5dump test.h5

HDF5 "test.h5" {
GROUP "/" {
   DATASET "data" {
      DATATYPE  H5T_STD_I64LE
      DATASPACE  SIMPLE { ( 100 ) / ( 100 ) }
      DATA {
      (0): 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
      (19): 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34,
      (35): 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50,
      (51): 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66,
      (67): 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82,
      (83): 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98,
      (99): 99
      }
   }
}
}

class Hierarchy:
    def __init__(self):        
        self.data = read_data()
        #f.close()

h = Hierarchy()
h.data

k = data





[<HDF5 dataset "data": shape (100,), type "<i8">]

!python datasets.py replace

h = Hierarchy()
h.data

k = data
k = data2





[<HDF5 dataset "data": shape (100,), type "<i8">,
 <HDF5 dataset "data2": shape (100,), type "<i8">]

[nicolasaunai]

the above code (from a notebook) mimics the creation, reading, update and re-reading of an hdf5 file, as if the simulation had progressed since last hierarchy construction and the data file was replaced by a newer version then re-read.
I see the second file is read correctly without doing any trick as proposed in #527.

I propose to keep this is issue open but assign lower priority and "still needed" tag in case we see the described behavior in the future. And if not we could close.