Airtable Schema Generator and Patches

The Airtable Schema Generator is a package developed by Blueprint to aid prior projects. We use it here in this project. It generates helper functions accordance with your Airtable Schema on top of the basic Airtable Javascript client. Overall, it simplifies abstractions between data in Airtable and in the application. The details on how it works and all the settings available can be found here.

Overview

Whenever the schema on Airtable is changed (ie any changes to the columns), it's necessary to run the schema generator in order to keep the local project data and files in sync with Airtable. Since schema changes affect every branch immediately, schema updates should be merged separately from a development branch.

The command npnm run generate-schema regenerates the schema. You must fill in the AIRTABLE_EMAIL and AIRTABLE_PASSWORD sections of your own .env file and allow password login on Airtable. These credentials are not saved anywhere except on your computer. This allows the schema generator to scrape your schema from Airtable. If you're curious what is happening, change auto-headless in the schema generator's settings in package.json to auto.

The schema generator's settings are in package.json. For the most part, they will not need to be edited. Details are below.

Generated Files

The schema generator outputs 3 files, and we've added a 4th:

airtable.js

This file interacts with the airtable client and exposes the core CRUD functions. It also contains the logic that translates the airtable column names to JS-formatted names. "Tariff Plans" becomes "tariffPlans" etc.

request.js

This file contains helper functions for every crud operation for every table. It's a convenience layer over airtable.js

schema.js

This file is a schema mapping that describes all the table and column names in a JS format. The app should never use this file but it is necessary for airtable.js to translate records to JS-readable variable names. It should be used as a reference to know what a column name has been translated to.

interface.ts

This file is not auto-generated and must be maintained by the developer. Whenever a column is added or removed from a table, the developer should update the interface.ts of the corresponding table record interface so that they are consistent. This file helps type all the records that are pull in from Airtable.

meepanyar

Schema Generator Settings:

"airtable-schema-generator": {
    "mode": "auto-headless",
    "output": "./src/lib/airtable",
    "airlock": true
}

• mode signifies how to fetch the schema. The options are detailed on the schema generator repo, but auto-headless will scrape the schema for you with no unnecessary windows
• output signifies where to save the generated files
• airlock signifies that we want the airlock version of the airtable.js file.

meepanyar-node

"airtable-schema-generator": {
    "mode": "auto-headless",
    "output": "./airtable"
}

• mode signifies how to get the files
• output signifies where to output generated files

Patches

We modify some of the generated files (both request.js and schema.js) in order to account for offline functionality, hit non-standard endpoints, etc... To save work re-doing this code every time the Airtable Schema Generator is run, we utilize a patch system on the frontend that stores the diffs between the base generated files and what's currently in main so that they can be reapplied after each schema regeneration. You can apply patches by running the command npm run apply-schema-patches. This is usually done after running the schema generator by running the command npm run generate-schema.

How to update patches

We'll walk through an example to understand how to update a patch to reinforce what patches are and why they're necessary. In this example, I'll be adding some comments to the Sites object of schema.js so that it's better documented. There are many ways to do this, but the important idea is to get a diff

1. We make the change to schema.js directly. In the screenshot, I updated the comments in the Sites object.

2. We commit the change we intend to make directly so that the our working branch becomes clean.

3. Now, our objective is to diff our current working branch with the base generated schema so that we can update our patch file. We can generate the base schema files again by running npm run generate-schema

4. At this point, a git status should show that our schema.js and request.js have been modified, and if we look at the area we had just made the change, our change is gone. This is because the base schema files do not contain the added comments or custom-objects that we've added over time.

5. Since we made our change only in schema.js and have no intention of updating the request.js patch, we can check out the request.js file so that it's consistent with main. We can do this by running from the meepanyar/ directory git checkout src/lib/airtable/request.js. After this, a git status should show only schema.js being modified.

6. To obtain our intended diff between the base generated schema.js and the schema.js we committed in step 2, we can commit the base generated schema.js and diff the two commits (assuming that we only changed schema.js in step 1). We add the base generated schema.js file and commit it.

7. A git log should show something like that following for your latest 2 commits (depending on your commit comments)

8. Now, we create our intended patch by diff-ing the contents of HEAD to HEAD^1. Make sure to diff in the correct way, in this case our diff should contain a lot of "+"s. The command to run to diff is git diff HEAD HEAD^1, which gives a diff between the contents of HEAD and the contents of the commit before HEAD. The result is something like the following

9. Finally, after verifying the screenshot, we can write our diff to a patch file by running from the meepanyar/ directory the following command: git diff HEAD HEAD^1 > src/lib/airtable/patches/patch-schema.patch

10. A git status should show that the file src/lib/airtable/patch-schema.patch has been modified.

11. With the patch fully updated, you can return to the state of step 2 except with our updated patch files by running npm run generate-schema && npm run apply-schema-patches which will rerun the base schema and then apply the patches in the src/lib/airtable/patches/ directory.

12. Verify that the changes from step 2 have reappeared. If so, we can commit all the changes and push, open a PR, etc...