Hi everyone, it looks like the Open Catalyst eval server for the leaderboard is down, meaning new models can’t be submitted to the leaderboard currently. I wanted to flag this. Thanks!
Hi - Was there something specific that wasn’t working that you tried? https://eval.ai/web/challenges/challenge-page/712/overview look accessible on my end. It’s possible EvalAI was down for maintenance.
Edit - I see what you mean now. Things are stuck in submitted. We’ll look into it.
@akr Thanks for flagging this! Ultimately, we are going to have to migrate our evaluation server — this is going to take some time. In the interim, we will do evaluations on request, so please let us know.