Python error during synthesis process

[andyb87]

Dear eqasim developers,

First of all, thank you for the great project and the work.
I am currently trying to recreate the San Francisco scenario, following the documentation meticulously.
Unfortunately I run into a Python error and since I am not that familiar with Python I would like to ask if you can help me?

It happens always in the 22 stage where "synthesis.population.spatial.by_person.primary_locations_education" is executed:

INFO:synpp:Pipeline progress: 22/28 (78.57%)                                                                                                                                                    
INFO:synpp:Executing stage synthesis.population.spatial.by_person.primary_locations_education__9d115bd22684d7fa5b85e5ab5e91a0df ...                                                             
INFO:synpp:Loading cache for synthesis.population.sociodemographics__1fb23cc29a732c02767b3d80ad838e71 ...                                                                                       
INFO:synpp:Loading cache for synthesis.population.spatial.by_person.primary_zones__9d115bd22684d7fa5b85e5ab5e91a0df ...                                                                         
INFO:synpp:Loading cache for synthesis.population.spatial.by_person.primary_locations__9d115bd22684d7fa5b85e5ab5e91a0df ...                                                                     
INFO:synpp:Loading cache for synthesis.destinations__276e498090d2b7b804c663738c6038d1 ...                                                                                                       
INFO:synpp:Loading cache for data.hts.cleaned__4a0d7dfd1ef8a95f682308468477cd08 ...                                                                                                             
INFO:synpp:Loading cache for synthesis.population.trips__1fb23cc29a732c02767b3d80ad838e71 ...                                                                                                   
multiprocessing.pool.RemoteTraceback:                                                                                                                                                           
"""                                                                                                                                                                                             
Traceback (most recent call last):                                                                                                                                                              
  File "/home/bunsenuser/.conda/envs/california/lib/python3.7/multiprocessing/pool.py", line 121, in worker                                                                                     
    result = (True, func(*args, **kwds))        
  File "/home/bunsenuser/.conda/envs/california/lib/python3.7/multiprocessing/pool.py", line 44, in mapstar                                                                                     
    return list(map(*args))                                                                                                                                                                     
  File "/home/bunsenuser/workspace/andyb/project_22cities/matsim-california/california/synthesis/population/spatial/by_person/primary_locations_education.py", line 53, in impute_education_locations_same_zone                                 
    indices_cp, distances_cp = tree.query_radius(home_coordinates_cp, r=random_from_cdf_cp, return_distance = True, sort_results=True)                                                          
  File "sklearn/neighbors/_binary_tree.pxi", line 1452, in sklearn.neighbors._kd_tree.BinaryTree.query_radius                                                                                   
  File "/home/bunsenuser/.conda/envs/california/lib/python3.7/site-packages/sklearn/utils/validation.py", line 556, in check_array                                                              
    "if it contains a single sample.".format(array))                                            
ValueError: Expected 2D array, got 1D array instead:                                                                                                                                            
array=[].                                       
Reshape your data either using array.reshape(-1, 1) if your data has a single feature or array.reshape(1, -1) if it contains a single sample.                                                   
"""        
The above exception was the direct cause of the following exception:                            
                                                                                                
Traceback (most recent call last):                                                              
  File "/home/bunsenuser/.conda/envs/california/lib/python3.7/runpy.py", line 193, in _run_module_as_main                  
    "__main__", mod_spec)                                                                       
  File "/home/bunsenuser/.conda/envs/california/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/bunsenuser/.conda/envs/california/lib/python3.7/site-packages/synpp/__main__.py", line 14, in <module>
    synpp.run_from_yaml(config_path)
  File "/home/bunsenuser/.conda/envs/california/lib/python3.7/site-packages/synpp/pipeline.py", line 818, in run_from_yaml
    Synpp.build_from_yml(path).run_pipeline()
  File "/home/bunsenuser/.conda/envs/california/lib/python3.7/site-packages/synpp/pipeline.py", line 842, in run_pipeline
    ensure_working_directory=True)
  File "/home/bunsenuser/.conda/envs/california/lib/python3.7/site-packages/synpp/pipeline.py", line 742, in run
    result = stage["wrapper"].execute(context)
  File "/home/bunsenuser/.conda/envs/california/lib/python3.7/site-packages/synpp/pipeline.py", line 63, in execute
    return self.instance.execute(context)
  File "/home/bunsenuser/workspace/andyb/project_22cities/matsim-california/california/synthesis/population/spatial/by_person/primary_locations_education.py", line 156, in execute
    educ_19_100 = parallelize_dataframe(hts_trips_educ, df_agents, df_candidates, df_trips, 18,  100, 2, "/nas/balacm/educ16100.png",impute_education_locations_same_zone, 6)
  File "/home/bunsenuser/workspace/andyb/project_22cities/matsim-california/california/synthesis/population/spatial/by_person/primary_locations_education.py", line 106, in parallelize_datafram
e
    df_locations = pd.concat(pool.map(prod_x, df_split))
  File "/home/bunsenuser/.conda/envs/california/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/home/bunsenuser/.conda/envs/california/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
ValueError: Expected 2D array, got 1D array instead:
array=[].
Reshape your data either using array.reshape(-1, 1) if your data has a single feature or array.reshape(1, -1) if it contains a single sample.             

I have the suspicion that it could be due to changed data sources and have therefore already checked the input files several times and tried to modify the Pyhon script but without success. I would appreciate it very much if you could give me a hint here? :)

Thank you and many greetings
Andy