deploy: a0090b0

_downloads/819e10305ddd6839cd7da05935b17060/mass-inference.txt

@@ -1,17 +1,17 @@
-2024-04-16 17:51:46 (INFO): Project root: /home/runner/work/ocp/ocp
+2024-04-16 21:46:24 (INFO): Project root: /home/runner/work/ocp/ocp
 /opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/torch/cuda/amp/grad_scaler.py:126: UserWarning: torch.cuda.amp.GradScaler is enabled, but CUDA is not available.  Disabling.
   warnings.warn(
-2024-04-16 17:51:47 (WARNING): Detected old config, converting to new format. Consider updating to avoid potential incompatibilities.
-2024-04-16 17:51:47 (INFO): amp: true
+2024-04-16 21:46:26 (WARNING): Detected old config, converting to new format. Consider updating to avoid potential incompatibilities.
+2024-04-16 21:46:26 (INFO): amp: true
 cmd:
-  checkpoint_dir: ./checkpoints/2024-04-16-17-50-56
-  commit: 3c11f14
+  checkpoint_dir: ./checkpoints/2024-04-16-21-45-36
+  commit: a0090b0
   identifier: ''
-  logs_dir: ./logs/tensorboard/2024-04-16-17-50-56
+  logs_dir: ./logs/tensorboard/2024-04-16-21-45-36
   print_every: 10
-  results_dir: ./results/2024-04-16-17-50-56
+  results_dir: ./results/2024-04-16-21-45-36
   seed: 0
-  timestamp_id: 2024-04-16-17-50-56
+  timestamp_id: 2024-04-16-21-45-36
 dataset:
   a2g_args:
     r_energy: false
@@ -122,29 +122,29 @@ test_dataset:
 trainer: ocp
 val_dataset: null
 
-2024-04-16 17:51:47 (INFO): Loading dataset: ase_db
-2024-04-16 17:51:47 (INFO): rank: 0: Sampler created...
-2024-04-16 17:51:47 (INFO): Batch balancing is disabled for single GPU training.
-2024-04-16 17:51:47 (INFO): rank: 0: Sampler created...
-2024-04-16 17:51:47 (INFO): Batch balancing is disabled for single GPU training.
-2024-04-16 17:51:47 (INFO): Loading model: gemnet_t
-2024-04-16 17:51:49 (INFO): Loaded GemNetT with 31671825 parameters.
-2024-04-16 17:51:49 (WARNING): Model gradient logging to tensorboard not yet supported.
-2024-04-16 17:51:49 (INFO): Loading checkpoint from: /tmp/ocp_checkpoints/gndt_oc22_all_s2ef.pt
-2024-04-16 17:51:49 (INFO): Overwriting scaling factors with those loaded from checkpoint. If you're generating predictions with a pretrained checkpoint, this is the correct behavior. To disable this, delete `scale_dict` from the checkpoint. 
-2024-04-16 17:51:49 (WARNING): Scale factor comment not found in model
-2024-04-16 17:51:49 (INFO): Predicting on test.
+2024-04-16 21:46:26 (INFO): Loading dataset: ase_db
+2024-04-16 21:46:26 (INFO): rank: 0: Sampler created...
+2024-04-16 21:46:26 (INFO): Batch balancing is disabled for single GPU training.
+2024-04-16 21:46:26 (INFO): rank: 0: Sampler created...
+2024-04-16 21:46:26 (INFO): Batch balancing is disabled for single GPU training.
+2024-04-16 21:46:26 (INFO): Loading model: gemnet_t
+2024-04-16 21:46:28 (INFO): Loaded GemNetT with 31671825 parameters.
+2024-04-16 21:46:28 (WARNING): Model gradient logging to tensorboard not yet supported.
+2024-04-16 21:46:28 (INFO): Loading checkpoint from: /tmp/ocp_checkpoints/gndt_oc22_all_s2ef.pt
+2024-04-16 21:46:28 (INFO): Overwriting scaling factors with those loaded from checkpoint. If you're generating predictions with a pretrained checkpoint, this is the correct behavior. To disable this, delete `scale_dict` from the checkpoint. 
+2024-04-16 21:46:28 (WARNING): Scale factor comment not found in model
+2024-04-16 21:46:28 (INFO): Predicting on test.
 device 0:   0%|                                           | 0/3 [00:00<?, ?it/s]/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/torch_geometric/data/collate.py:145: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly.  To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
   storage = elem.storage()._new_shared(numel)
 /opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/torch_geometric/data/collate.py:145: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly.  To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
   storage = elem.storage()._new_shared(numel)
 /opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/torch/amp/autocast_mode.py:250: UserWarning: User provided device_type of 'cuda', but CUDA is not available. Disabling
   warnings.warn(
-device 0:  33%|███████████▋                       | 1/3 [00:02<00:04,  2.09s/it]device 0:  67%|███████████████████████▎           | 2/3 [00:04<00:02,  2.35s/it]device 0: 100%|███████████████████████████████████| 3/3 [00:06<00:00,  2.01s/it]device 0: 100%|███████████████████████████████████| 3/3 [00:06<00:00,  2.09s/it]
+device 0:  33%|███████████▋                       | 1/3 [00:02<00:05,  2.99s/it]device 0:  67%|███████████████████████▎           | 2/3 [00:05<00:02,  2.88s/it]device 0: 100%|███████████████████████████████████| 3/3 [00:07<00:00,  2.39s/it]device 0: 100%|███████████████████████████████████| 3/3 [00:07<00:00,  2.54s/it]
 /home/runner/work/ocp/ocp/ocpmodels/trainers/ocp_trainer.py:510: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
   predictions[key] = np.array(predictions[key])
 /home/runner/work/ocp/ocp/ocpmodels/trainers/base_trainer.py:840: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
   np.array(gather_results[k])[idx]
-2024-04-16 17:51:56 (INFO): Writing results to ./results/2024-04-16-17-50-56/ocp_predictions.npz
-2024-04-16 17:51:56 (INFO): Total time taken: 6.417162895202637
-Elapsed time = 12.7 seconds
+2024-04-16 21:46:36 (INFO): Writing results to ./results/2024-04-16-21-45-36/ocp_predictions.npz
+2024-04-16 21:46:36 (INFO): Total time taken: 7.780270338058472
+Elapsed time = 14.1 seconds

_sources/core/fine-tuning/fine-tuning-oxides.md

@@ -209,7 +209,7 @@ yml = generate_yml_config(checkpoint_path, 'config.yml',
                    update={'gpus': 1,
                            'task.dataset': 'ase_db',
                            'optim.eval_every': 1,
-                           'optim.max_epochs': 5,
+                           'optim.max_epochs': 4,
                            'optim.batch_size': 4,
                             'logger':'tensorboard', # don't use wandb!
                            # Train data

_sources/core/inference.md

@@ -126,7 +126,7 @@ d = results[0].split(':')[-1].strip()
 
 ```{code-cell} ipython3
 import numpy as np
-results = np.load(f'{d}/s2ef_predictions.npz', allow_pickle=True)
+results = np.load(f'{d}/ocp_predictions.npz', allow_pickle=True)
 results.files
 ```
 

_sources/core/model_training.md

@@ -171,7 +171,7 @@ Next, run this model on the test data:
 python main.py --mode predict --config-yml configs/s2ef/2M/schnet/schnet.yml \
         --checkpoint checkpoints/[TIMESTAMP]/checkpoint.pt
 ```
-The predictions are stored in `[RESULTS_DIR]/s2ef_predictions.npz` and later used to create a submission file to be uploaded to EvalAI.
+The predictions are stored in `[RESULTS_DIR]/ocp_predictions.npz` and later used to create a submission file to be uploaded to EvalAI.
 
 ## Training OC20 models with total energies (IS2RE/S2EF)
 

_sources/tutorials/advanced/fine-tuning-in-python.md

@@ -81,7 +81,7 @@ yml = generate_yml_config(checkpoint_path, 'config.yml',
                    update={'gpus': 1,
                            'task.dataset': 'ase_db',
                            'optim.eval_every': 1,
-                           'optim.max_epochs': 5,
+                           'optim.max_epochs': 4,
                            'optim.batch_size': 4,
                            'logger': 'tensorboard', # don't use wandb unless you already are logged in 
                            # Train data

core/fine-tuning/fine-tuning-oxides.html

@@ -769,7 +769,7 @@ <h1>Fine tuning a model<a class="headerlink" href="#fine-tuning-a-model" title="
   warnings.warn(
 </pre></div>
 </div>
-<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Elapsed time 68.0 seconds.
+<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Elapsed time 68.8 seconds.
 </pre></div>
 </div>
 <img alt="../../_images/92bd7f94dd548c8cfc2744eb5890cd23fada1ff98e8dc907657e2eb109af0402.png" src="../../_images/92bd7f94dd548c8cfc2744eb5890cd23fada1ff98e8dc907657e2eb109af0402.png" />
@@ -921,7 +921,7 @@ <h2>Setting up the configuration yaml file<a class="headerlink" href="#setting-u
                    <span class="n">update</span><span class="o">=</span><span class="p">{</span><span class="s1">&#39;gpus&#39;</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span>
                            <span class="s1">&#39;task.dataset&#39;</span><span class="p">:</span> <span class="s1">&#39;ase_db&#39;</span><span class="p">,</span>
                            <span class="s1">&#39;optim.eval_every&#39;</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span>
-                           <span class="s1">&#39;optim.max_epochs&#39;</span><span class="p">:</span> <span class="mi">5</span><span class="p">,</span>
+                           <span class="s1">&#39;optim.max_epochs&#39;</span><span class="p">:</span> <span class="mi">4</span><span class="p">,</span>
                            <span class="s1">&#39;optim.batch_size&#39;</span><span class="p">:</span> <span class="mi">4</span><span class="p">,</span>
                             <span class="s1">&#39;logger&#39;</span><span class="p">:</span><span class="s1">&#39;tensorboard&#39;</span><span class="p">,</span> <span class="c1"># don&#39;t use wandb!</span>
                            <span class="c1"># Train data</span>
@@ -1075,7 +1075,7 @@ <h2>Setting up the configuration yaml file<a class="headerlink" href="#setting-u
   load_balancing: atoms
   loss_energy: mae
   lr_initial: 0.0005
-  max_epochs: 5
+  max_epochs: 4
   mode: min
   num_workers: 2
   optimizer: AdamW
@@ -1134,10 +1134,7 @@ <h2>Running the training job<a class="headerlink" href="#running-the-training-jo
 <div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>^C
 </pre></div>
 </div>
-<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Elapsed time = 1200.5 seconds
-</pre></div>
-</div>
-<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>
+<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Elapsed time = 1800.5 seconds
 </pre></div>
 </div>
 </div>

core/gotchas.html

@@ -929,7 +929,7 @@ <h1>I get wildly different energies from the different models<a class="headerlin
   warnings.warn(
 </pre></div>
 </div>
-<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>1.6831002235412598
+<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>1.6853339672088623
 </pre></div>
 </div>
 </div>
@@ -1433,7 +1433,7 @@ <h1>To tag or not?<a class="headerlink" href="#to-tag-or-not" title="Link to thi
   warnings.warn(
 </pre></div>
 </div>
-<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>-0.4297373294830322
+<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>-0.42973682284355164
 </pre></div>
 </div>
 </div>
@@ -1483,17 +1483,17 @@ <h1>Stochastic simulation results<a class="headerlink" href="#stochastic-simulat
   warnings.warn(
 </pre></div>
 </div>
-<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>1.2139861583709717 1.530342725360986e-06
-1.2139849662780762
-1.213989019393921
+<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>1.2139856100082398 1.5959804507426796e-06
+1.2139854431152344
 1.2139837741851807
-1.2139866352081299
-1.2139861583709717
-1.2139859199523926
-1.2139866352081299
+1.2139830589294434
+1.2139854431152344
 1.2139878273010254
-1.2139866352081299
-1.2139840126037598
+1.2139887809753418
+1.2139854431152344
+1.2139859199523926
+1.2139849662780762
+1.2139854431152344
 </pre></div>
 </div>
 </div>
@@ -1536,7 +1536,7 @@ <h1>The forces don’t sum to zero<a class="headerlink" href="#the-forces-don-t-
   warnings.warn(
 </pre></div>
 </div>
-<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>array([ 0.00848317,  0.01409542, -0.05882776], dtype=float32)
+<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>array([ 0.00848235,  0.01409314, -0.05882859], dtype=float32)
 </pre></div>
 </div>
 </div>
@@ -1549,7 +1549,7 @@ <h1>The forces don’t sum to zero<a class="headerlink" href="#the-forces-don-t-
 </div>
 </div>
 <div class="cell_output docutils container">
-<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>array([-2.44472176e-09,  1.06636435e-07,  2.38418579e-07], dtype=float32)
+<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>array([ 1.1816155e-08,  6.8917871e-08, -2.3841858e-07], dtype=float32)
 </pre></div>
 </div>
 </div>