We recently discovered that the IS2RE results using the Relaxation approaches in Table 4 of our paper are erroneous. For reference, we approached the IS2RE task using two methods:
- Direct: Directly predicting relaxed energies from initial states; or more broadly - trained on the IS2RE dataset (~460k).
- Relaxation: Iteratively estimate atomic forces and update atomic positions until a relaxed state is reached. Report the energy of the found relaxed state; trained on the S2EF dataset (~134M).
The errors were a result of a bug later found in the denormalization of the model outputs in the custom script that was used to generate the final outputs. The corrected numbers are provided in the updated table below. Note, the Relaxation-based approaches now produce the best-performing numbers. We have updated the leaderboard with the new results. We will update the paper shortly with these results and another bug affecting relaxations with the CHCOH adsorbate and a few bulk surfaces (<1% of the overall dataset resulting in small changes in accuracy). Additionally, we plan to add an extra column to the IS2RE leaderboard to indicate whether entries were trained using the IS2RE dataset (direct) or S2EF dataset (relaxation) since these approaches are notably different.
We’re sorry for any trouble or confusion this error may have caused. Please reach out if you have any questions or concerns.