Possible regression: cblas_dspmv test fails in v0.3.32

[hmenke]

Observed on Ubuntu 24.04 running on aarch64 as part of another build: https://github.com/flathub-infra/vorarbeiter/actions/runs/23469172794/job/68288026035

 cblas_dspmv  PASSED THE TESTS OF ERROR-EXITS

 ******* FATAL ERROR - COMPUTED RESULT IS LESS THAN HALF ACCURATE *******
           EXPECTED RESULT   COMPUTED RESULT
       1      0.667230          0.667230    
       2      -1.16658          -1.16658    
       3      0.210145          0.210145    
       4      0.543602          0.543602    
       5     -0.675359         -0.675359    
       6       1.33709           1.33709    
       7      0.668852          0.668852    
       8       1.70144           1.70144    
       9     -0.375509         -0.375509    
      10       1.21933           1.21933    
      11      -1.99305          -1.99305    
      12      0.306078          0.306078    
      13       1.00700           1.00700    
      14       1.29749           1.29749    
      15      0.503632          0.503632    
      16      0.624915          0.624915    
      17      -1.05779          -1.05779    
      18       3.71865           3.71865    
      19      -1.45335          -1.45335    
      20       2.15959           2.15959    
      21      -1.31750          -1.31750    
      22     -0.331946         -0.331946    
      23     -0.397639         -0.397639    
      24     -0.898512         -0.898512    
      25     -0.143124         -0.143124    
      26      0.763038          0.763038    
      27     -0.184042         -0.184042    
      28      -1.33639          -1.33639    
      29      -1.81842          -1.81842    
      30     -0.639056         -0.639056    
      31      -1.73327          -1.73327    
      32     -0.603902         -0.603902    
ERROR STOP 

Error termination. Backtrace:
      33     -0.602854          0.273770    
      34     -0.638333         -0.231240    
      35     -0.985313         -0.418380    
      36     -0.386861         -0.189558    
      37      -2.50400          -1.97703    
      38     -0.100883          0.456060    
      39     -0.482794         -0.195581    
      40      0.279730           1.12638    
      41     -0.869296         -0.192472    
      42      0.548802E-01      0.262173    
      43     -0.304101          0.132961    
      44      -1.02007         -0.652936    
      45       1.02721           1.35438    
      46     -0.275883          0.812599E-01
      47      0.605179          0.692592    
      48     -0.218912E-01      0.495092    
      49      0.751895E-01      0.722043    
      50     -0.489880         -0.482387    
      51     -0.759316         -0.522053    
      52      0.139411          0.306744    
      53     -0.798163         -0.145958E-02
      54     -0.303303         -0.175930    
      55     -0.172739          0.713874    
      56       1.07911           1.39629    
      57      0.393113E-01      0.486364    
      58       1.35710           1.63432    
      59     -0.688482          0.118211    
      60      0.847893           1.81443    
      61      0.881102           1.47801    
      62       5.50897           6.43554    
      63     -0.112485          0.844059    
 ******* cblas_dspmv  FAILED ON CALL NUMBER:
   1445: cblas_dspmv (    CblasUpper, 63, 1.0, AP, X, 1, 0.0, Y, 1) .
 ******* cblas_dspmv  FAILED ON CALL NUMBER:
      2: cblas_dspmv (    CblasUpper,  1, 0.0, AP, X, 1, 0.0, Y, 1) .

 ******* FATAL ERROR - TESTS ABANDONED *******
#0  0xff7b4415fc07 in ???
#1  0xff7b44160973 in ???
#2  0xff7b4416214f in ???
#3  0xaaf9666bf013 in ???
#4  0xaaf9666b4fc3 in ???
#5  0xff7b43ee61d3 in ???
#6  0xff7b43ee62b7 in ???
#7  0xaaf9666b50af in _start
	at ../sysdeps/aarch64/start.S:98
#8  0xffffffffffffffff in ???

[martin-frbg]

Very curious as there weren't that many changes in 0.3.32, and if I read your log correctly, the job ran on Neoverse N1 hardware, which is covered by OpenBLAS' CI . Which compiler is your build using, please ?

[hmenke]

The compiler is GCC 14.3.0 (part of freedesktop-sdk 24.08).

[martin-frbg]

Seems to be connected to the DDOT kernel - sometimes it works, sometimes it doesn't, workaround appears to be to compile it at -O1 maximum. This would suggest it it connected to the very recent fix (#5691) for SDOT malfunctions on NEOVERSEV2, where the dot product was apparently optimized away unless the return variable was declared as volatile. Now that very same declaration appears to cause malfunction on other cpus when the same code is used for double precision ... there's something crazily wrong with this.

Perhaps there was something wrong with the much earlier change from #3845 that "removed unnecessary instructions" from that assembly kernel, though the change was proposed by an ARM guy @Mousius

[martin-frbg]

For a quick temporary fix - either change DDOTKERNEL in kernel/arm64/KERNEL.NEOVERSEN1 to "../arm/dot.c" (the generic C implementation), or edit kernel/arm64/dot_kernel_asimd.c to change the single declaration of "volatile RETURN_TYPE dot = 0.0;" into

#ifndef DOUBLE
volatile
#endif
  RETURN_TYPE dot = 0.0;