Intermitent SNOPT inform test failure

[A-CGray]

Description

We are seeing TestTP109.test_snopt_informs fail intermitently on our ARM docker image builds. This test sets a time limit of 1e-15 on an optimization and then checks that SNOPT exits with a 30-34 "time limit reached" exit code. However, for some reason the test occasionally fails with SNOPT returning a 0-1 exit code. Seems like SNOPT is ignoring the time limit and successfully solving the problem. The only explanation I could think of for this is that the time limit option key used in the test is "Time Limit", while the correct key is "Time limit", but I can't tell why this would be non-deterministic though.

Steps to reproduce issue

1. …
2. …
3. …

Current behavior

     about to run test_tp109.py:TestTP109.test_snopt_informs (isolated) /home/***/repos/pyoptsparse/tests/test_tp109.py:TestTP109.test_snopt_informs  ... FAIL (00:00:0.10, 133 MB)
Traceback (most recent call last):
  File "/home/***/repos/pyoptsparse/tests/test_tp109.py", line 180, in test_snopt_informs
    self.assert_inform_equal(sol, 34)
  File "/home/***/.pyenv/versions/3.9.16/lib/python3.9/site-packages/pyoptsparse/testing/pyOpt_testing.py", line 165, in assert_inform_equal
    self.assertEqual(sol.optInform.value, optInform)
AssertionError: 1 != 34

Expected behavior

SNOPT should return exit code 34 and test should pass.

Code versions

• Operating System: Ubuntu 24.04
• Python: 3.9.16
• OpenMPI: 4.1.6
• PETSc:
• Compiler: gcc
• This repository: main (6cc1756)

[A-CGray]

It seems like in both failing and passing cases, the time limit is being passed to SNOPT but not enforced properly (assuming SNOPT cannot get through 28 major iterations in 1e-15 seconds), below are snippets for SNOPT_print.out files

Failing test

         ==============================
         S N O P T  7.7.1    (Nov 2018)
         ==============================
      Print file 18
      Summary file 19
      Minor print level               0
      Minimize
      Derivative level               0
      Iterations limit        10000000
      Minor iterations limit           10000
      Proximal iterations limit           10000
      Total character workspace             500
      Total integer workspace           21771
      Total real workspace           12003
      Time Limit  1.00000000E-15
.
.
.
     90    71      2 1.0E+00     74 (8.2E-11) 1.8E-04  5.3620692E+03      34            1 1.2E+02 3.7E+00 _      c
     91    72      1 1.0E+00     75 (2.2E-12)(2.5E-08) 5.3620692E+03      29            1 1.2E+02 3.7E+00 _      c
1

 SNOPTC EXIT   0 -- finished successfully
 SNOPTC INFO   1 -- optimality conditions satisfied

Passing test

         ==============================
         S N O P T  7.7.1    (Nov 2018)
         ==============================
      Print file 18
      Summary file 19
      Minor print level               0
      Minimize
      Derivative level               0
      Iterations limit        10000000
      Minor iterations limit           10000
      Proximal iterations limit           10000
      Total character workspace             500
      Total integer workspace           21771
      Total real workspace           12003
      Time Limit  1.00000000E-15
.
.
.
     45    27      1 1.2E-02     30  8.1E-03  5.9E+04  5.4624235E+03      29            1 1.1E+02 3.7E+00 _
     46    28      1 1.2E-02     31  8.0E-03  5.9E+04  5.4602265E+03      29            1 1.1E+02 3.7E+00 _
1

 SNOPTC EXIT  30 -- resource limit error
 SNOPTC INFO  34 -- time limit reached

[ewu63]

Hm a more reliable approach is probably to artificially slow down the objcon with some time.sleep() and passing a more reasonable time limit , would you mind trying that?

[A-CGray]

Hm a more reliable approach is probably to artificially slow down the objcon with some time.sleep() and passing a more reasonable time limit , would you mind trying that?

Slowing down the function evaluation by 1ms was enough to fix this. I think the issue is not how small the time limit is, but the precision (or lack of it) with which SNOPT measures time depending on your fortran compiler.

I opened #478 to fix this.