For preprocessing progress tracking or other tasks that takes lots of time to run

[fmohebi]

If the preprocess function takes too long to run, feel free to add a progress bar so that we are keep updated on the progress:
First install the tqdm package:
%pip install tqdm
And don’t forget to load it to your notebook:
from tqdm import tqdm
Then update the preprocess function, change the apply function to:
df[new_col] = df[text_col].progress_apply(process_text)
Finally add the following line before we run the preprocess function:
tqdm.pandas()
df = preprocess(df, text_col='selftext', new_col='pp_text')