Question about "YearBuilt" recognition

[amaccarini]

I have a question regarding how BRAILS recognizes the "YearBuilt" parameter. Does the built-in ML algorithm only identify the building age (“YearBuilt”) for Berkeley? I ask because I ran the example file for a random Italian town (named Alvito) and the resulting .geojson file only contained building footprints, without any attributes related to “YearBuilt,” “BuildingHeight,” or similar parameters. This made me wonder if the ML algorithm might have been trained exclusively on images of buildings in Berkeley and therefore cannot recognize buildings in other locations.

[bacetiner]

Thank you for your interest in BRAILS.

The current YearBuilt computer vision model in BRAILS was trained on data from all across the US, so it is expected to give more accurate results for query areas in the US. That said, it generates predictions for anywhere in the world by design.

Could you please share the code snippet you used to create the building inventory you referred to? I would like to investigate the issue further by reproducing the problem you described.

[amaccarini]

Sure, please find below the code:

#import InventoryGenerator:
from brails.InventoryGenerator import InventoryGenerator

# Initialize InventoryGenerator:
invGenerator = InventoryGenerator(location='Alvito, Italy',
                                  fpSource='osm',
                                  lengthUnit='m',
                                  outputFile='BaselineInvAlvito.geojson')

# View a list of building attributes that can be obtained using BRAILS:
invGenerator.enabled_attributes()

# Run InventoryGenerator to generate an inventory for the entered location:
# To run InventoryGenerator for all enabled attributes set attributes='all':
invGenerator.generate(attributes=['erabuilt','buildingheight'],
                      GoogleAPIKey='mykey',
                      nbldgs=10,
                      outputFile='BaselineInvAlvito.geojson')

# View generated inventory:
invGenerator.inventory

This code takes inspiration from the example code shown in the README at https://github.com/NHERI-SimCenter/BRAILS

Please note that in addition to the general question mentioned in the comment above (about the computer vision model), this code appears to have two issues:

1. More than 10 buildings are exported in the geojson file
2. Although the code appears to successfully export the GeoJSON file, it still generates an error:

Traceback (most recent call last):
  File "C:\home\My_working\tests\brails_test2.py", line 15, in <module>
    invGenerator.generate(attributes=['erabuilt','buildingheight'],
  File "C:\Users\me\AppData\Local\Programs\Python\Python311\Lib\site-packages\brails\InventoryGenerator.py", line 637, in generate
    facadeParserModel.predict(image_handler)
  File "C:\Users\me\AppData\Local\Programs\Python\Python311\Lib\site-packages\brails\modules\FacadeParser\FacadeParser.py", line 400, in predict
    install_default_model(self.model_path)
  File "C:\Users\me\AppData\Local\Programs\Python\Python311\Lib\site-packages\brails\modules\FacadeParser\FacadeParser.py", line 200, in install_default_model
    torch.hub.download_url_to_file(model_url, model_path,
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\me\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\hub.py", line 622, in download_url_to_file
    u = urlopen(req)
        ^^^^^^^^^^^^
  File "C:\Users\me\AppData\Local\Programs\Python\Python311\Lib\urllib\request.py", line 216, in urlopen
    return opener.open(url, data, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\me\AppData\Local\Programs\Python\Python311\Lib\urllib\request.py", line 525, in open
    response = meth(req, response)
               ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\me\AppData\Local\Programs\Python\Python311\Lib\urllib\request.py", line 634, in http_response
    response = self.parent.error(
               ^^^^^^^^^^^^^^^^^^
  File "C:\Users\me\AppData\Local\Programs\Python\Python311\Lib\urllib\request.py", line 557, in error
    result = self._call_chain(*args)
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\me\AppData\Local\Programs\Python\Python311\Lib\urllib\request.py", line 496, in _call_chain
    result = func(*args)
             ^^^^^^^^^^^
  File "C:\Users\me\AppData\Local\Programs\Python\Python311\Lib\urllib\request.py", line 749, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\me\AppData\Local\Programs\Python\Python311\Lib\urllib\request.py", line 525, in open
    response = meth(req, response)
               ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\me\AppData\Local\Programs\Python\Python311\Lib\urllib\request.py", line 634, in http_response
    response = self.parent.error(
               ^^^^^^^^^^^^^^^^^^
  File "C:\Users\me\AppData\Local\Programs\Python\Python311\Lib\urllib\request.py", line 563, in error
    return self._call_chain(*args)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\me\AppData\Local\Programs\Python\Python311\Lib\urllib\request.py", line 496, in _call_chain
    result = func(*args)
             ^^^^^^^^^^^
  File "C:\Users\me\AppData\Local\Programs\Python\Python311\Lib\urllib\request.py", line 643, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: NOT FOUND

Hope you can help!

EDIT: I just realized that the error seems related to another ongoing discussion. If I understand correctly, you are already working on solving this issue.

[amaccarini]

@bacetiner Hi, I was wondering if you've had time to look into this issue. Thanks!

[bacetiner]

Hello and thank you for your patience. We are aware of this issue, and it is currently a high priority that will be addressed soon.

In the meantime, if you are interested in just the year of construction predictions of BRAILS, you can use the script below. Please note that the YearBuiltClassifier module was primarily intended for hurricane applications in the US, and it makes its predictions for relevant eras of construction. To be specific, the eras this model is able to predict are 1) Pre-1970, 2) 1970 - 1979, 3) 1980 - 1989, 4) 1990 - 1999, 5) 2000 - 2009, and 6) Post-2010, denoted with 1960, 1975, 1985, 1995, 2005, and 2015 respectively.

"""Script demonstrating the use of YearBuiltClassifier, NFloorDetector, 
and OccupancyClassifier."""

import random
import pandas as pd
from brails.modules import (YearBuiltClassifier, NFloorDetector,
                            OccupancyClassifier)
from brails.workflow.FootprintHandler import FootprintHandler
from brails.workflow.ImHandler import ImageHandler

random.seed(20)  # Random seed is set to 20 for reproducibility of results

NBLDGS = 10  # Can also set to 'all'

# Get footprint data for Alvito, Italy from OSM (can also set fpSource='ms'):
fp_handler = FootprintHandler()
fp_handler.fetch_footprint_data('Alvito, Italy',
                                fpSource='osm',
                                lengthUnit='m')
footprints = fp_handler.footprints
if NBLDGS == 'all':
    NBLDGS = len(footprints)

# Randomly sample from the extracted footprints. Please note that random
# seed is set to 20 for reproducibility of results:
footprints = random.sample(footprints, NBLDGS)

# Get street-level imagery for Alvito by using the footprint locations:
image_handler = ImageHandler('AIzaSyBJoY6tFIBZHjthJqOqdJPi1XsFzoP5nLo')
image_handler.GetGoogleStreetImage(fp_handler.footprints)
imstreet = [im for im in image_handler.street_images if im is not None]

# Initialize the era of construction classifier:
year_model = YearBuiltClassifier()

# Call the classifier to determine the era of construction for
# each building:
year_model.predict(imstreet)

# Get the prediction results:
preds = year_model.results_df.copy(deep=True)

# Prepare preds for merging with building inventory:
preds_era = preds.rename(
    columns={'prediction': 'YearBuilt',
              'image': 'street_images'}).drop(
                  columns=['probability'])

# Initialize the floor detector:
nfloor_model = NFloorDetector()

# Call the floor detector to determine number of floors of
# buildings in each image:
nfloor_model.predict(imstreet)

# Get predictions and prepare them for merging with the building inventory:
preds_nfloor = pd.DataFrame(
    list(zip(nfloor_model.system_dict['infer']['images'],
              nfloor_model.system_dict['infer']['predictions'])),
    columns=['street_images', 'NumberOfStories'])

# Initialize the occupancy classifier object:
occupancy_model = OccupancyClassifier()

# Call the occupancy classifier to determine the occupancy
# class of each building:
occupancy_model.predict(imstreet)

# Write the prediction results to a DataFrame:
preds_occ = pd.DataFrame(occupancy_model.preds, columns=[
    'street_images', 'OccupancyClass'])

# Write prediction results into the inventory DataFrame:
inventory_df = pd.DataFrame(
    pd.Series(fp_handler.footprints, name='Footprint'))
inventory_df.Footprint = inventory_df.Footprint.apply(str)
lon = []
lat = []
for pt in fp_handler.centroids:
    lon.append(pt.x)
    lat.append(pt.y)
inventory_df['Latitude'] = lat
inventory_df['Longitude'] = lon
inventory_df['PlanArea'] = fp_handler.attributes['fparea']

inventory_df['street_images'] = image_handler.street_images
inventory_df = inventory_df.merge(preds_era,
                                  how='left',
                                  on='street_images')
inventory_df = inventory_df.merge(preds_nfloor,
                                  how='left',
                                  on='street_images')
inventory_df['NumberOfStories'] = inventory_df['NumberOfStories'].astype(dtype='Int64')
inventory_df = inventory_df.merge(preds_occ,
                                  how='left',
                                  on='street_images')
print(inventory_df)

[amaccarini]

Hi, thanks for sending the code about the era prediction. I just tested it and it runs smoothly!
But I understand that I can't associate a polygon with its corresponding era directly, correct? I believe that's handled by the InventoryGenerator function (which you're working on), is that right?

[bacetiner]

Correct, InventoryGenerator has built-in utilities for merging AI prediction results into a building inventory.

Alternatively, you can achieve this merging through some pandas magic. I updated my response above to include these additional pandas lines.

I hope you find this helpful!

[amaccarini]

Thank you so much, that worked perfectly!
I do have one last issue that I hope you can help me with. In addition to determining the era, I’d also like to output the number of floors. I’ve tried experimenting with the code you shared and attempted to integrate the NFloorDetector module, but I haven’t had any success so far.

[bacetiner]

I am glad to hear that the provided code was helpful! I updated the sample script above to include the lines for generating number of floors predictions.

[amaccarini]

Great, it works! Thank a lot.
Just out of curiosity: is it normal for the floor detection calculation to take quite a long time (a couple of minutes for 20 buildings)? I’m a bit surprised since the age detector is much faster.

[bacetiner]

Fantastic!

The pre-trained model for floor detection is considerably more complicated than the one trained for predicting the era of construction. So, longer processing times are expected for floor detection. That said, a couple of minutes for 20 buildings sounds quite a bit longer than I would anticipate (for reference, it takes about 7 seconds on my work laptop). This likely occurs because BRAILS is unable to detect a Pytorch-supported GPU on your system, causing the calculations to run on your CPU instead.

If your system is equipped with a CUDA-enabled Nvidia GPU or ROCm-enabled AMD GPU, I might be able to provide limited support in getting you set up (I do not have as much experience with Apple M GPUs). Alternatively, running the code on Google Colab with a GPU runtime could be a good option.

I hope this helps!