Submisions silently freezes or failing ❄️ #2236
Description
Hello! I am the organizer of the competition https://www.codabench.org/competitions/12842/
Thank you for creating and supporting the codabench platform!
But we have a problem similar to the one described in issues:
Some of our members' submissions without any logs either froze in the state "Running" for a long time (fixed for up to several hours) or failed. None of the suspended submissions came out of the state "Running". We recorded this behavior both on the current task of the competition and on the previous one. Both tasks do not carry much computational complexity, and the first one instantly displayed a fictitious result for any submission. Considering the above and the fact that some submissions still work out quickly and the first time, I conclude that the problem should not be hidden in the code of our task.
Restarting the submissions is currently helping to solve the problem. Usually, after the first restart, the submissions works correctly, but there has already been a case that required 2 restarts. Restarts of the submissions cause confusion among the participants. At the same time, I saw in other issues that you were planning to update the system for running submissions, which may fix the error.
An illustrative example is the case where it took 2 restarts. Submission 588298 crashed, was restarted in submission 589675. Submission 589675 froze, was canceled and restarted in submission 590524. Submission 590524 completed correctly.
Did I understand correctly that my problem relates to the one you already know? Are you planning to make updates to the system in the near future that can solve this problem?
Thank you in advance for your reply!