The OCP evaluation server and leaderboards are now available!
Evaluation server: eval.ai/web/challenges/challenge-page/712/overview
Leaderboards:
— Structure to energy and forces (S2EF): opencatalystproject.org/leaderboard_s2ef.html
— Initial structure to relaxed structure (IS2RS): opencatalystproject.org/leaderboard_is2rs.html
— Initial structure to relaxed energy (IS2RE): opencatalystproject.org/leaderboard_is2re.html
Instructions for how to create EvalAI submissions files are available here: https://github.com/Open-Catalyst-Project/ocp/blob/master/TRAIN.md.
We allow max 10 submissions to the S2EF and IS2RE servers, and max 5 submissions to the IS2RS server for each approach. The limit for IS2RS is lower since we run DFT evaluations (which are compute-heavy) for each submission if it’s above 20.26% on the Average Distance within Threshold (ADwT) metric. Complete breakdown of daily / monthly / overall submission limits is available here.
Please refer to our paper for a complete description of all metrics.
As an additional note, we recently updated the dataset due to minor bugs (affecting < 1% of the data). Please re-download the data if you downloaded it before Feb 10, 2021. More details here.
As always, please feel free to ask any questions about the models / evaluation server here or open an issue on github for questions about the code.