stockfish 18 issue

[Djahmo]

my app run perfectly with s17-lite but when i just try to replace by s18-lite, stockfish dont work (but s18-lite single work normaly)

[Djahmo]

edit : i have benchmark 17 and 18

with 17 :

⏱️ Eval started after 0ms, now waiting 180ms for depth 20 useStockfish.ts:458:15
✅ Got 3 lines after timeout, depth 14 useStockfish.ts:475:17
🔄 Clearing activeSearchRef and scheduling next search useStockfish.ts:357:17

start in 0 ms

with 18 :

activeSearchRef and scheduling next search useStockfish.ts:357:17
⏱️ Eval started after 790ms, now waiting 180ms for depth 20 useStockfish.ts:458:15
✅ Got 3 lines after timeout, depth 14 useStockfish.ts:475:17
🔄 Clearing activeSearchRef and scheduling next search useStockfish.ts:357:17

start int 800ms

[nmrugg]

Can you give me a script to reproduce this?

[Djahmo]

@nmrugg i make that for you.

https://github.com/Djahmo/StockfishTest

[Djahmo]

@nmrugg have you seen my message ?

[JsonBorn64]

First of all, I would like to thank you for the excellent work. However, I have also noticed some degradation.

I have been successfully using the multithreaded Stockfish build from this repository on my website since version 16. In version 18, a strange startup delay of up to 3–4 seconds appeared. In addition, some users have started reporting that it does not launch at all on their iPhone devices, although version 17.1 was working correctly.

[nmrugg]

FYI: I'm looking into it.

[nmrugg]

I've been looking to this for a while. There are several issues that are being confused together here. Let's go through them:

my app run perfectly with s17-lite but when i just try to replace by s18-lite, stockfish dont work (but s18-lite single work normaly)

If only the single threaded engine is working, then are almost certainly not setting the correct headers. There has been no change between 17 and 18 in this regard. Perhaps you are using the single threaded 17. Frankly, the single-threaded engine is what you should be using unless you have an niche use-case. Its performance virtually identically to the multi-threaded engine, and it's much easier to use.

edit : i have benchmark 17 and 18

The benchmarks tell a different story. It appears that 18 does run, just slower.

However, it does not actually run slower. It just appears to run slower. depth is not a good measurement for an engine's performance because depth doesn't actually have a meaning. It's just a number. It does not correspond to how many moves deep the engine has analyzed. (See seldepth, compare pv length, etc.) We also cannot compare depths from one engine (or version) to another. Despite appearing slower, from my tests, and others, Stockfish 18 is actually stronger.

Start up time is another story, but still a bit complicated.

start in 0 ms

Starting in 0 ms makes no sense, and it's not what I'm seeing.

From my tests (and using your test repo), Stockfish 18 Lite is a little slower at starting up than Stockfish 17.1 Lite (by around 100ms or so). On the other hand, the full Stockfish 18 starts up faster than Stockfish 17.1 for me (by about 200ms). I'm not sure why that is. I may try to investigate that further. But I am not seeing anything significant in the start up times.

Also, simply measuring the readyok response is not a great real-world test. You should at least make sure it has fully started (like run go nodes 1 and wait for bestmove).

I have been successfully using the multithreaded Stockfish build from this repository on my website since version 16. In version 18, a strange startup delay of up to 3–4 seconds appeared. In addition, some users have started reporting that it does not launch at all on their iPhone devices, although version 17.1 was working correctly.

Stockfish 16 did not enable NNUE weights on initialization or by default. It does start much faster and it is much smaller if you are using the classical eval. However, if you take into consideration loading and processing the NNUE weights, it's much closer to Stockfish 18.

Stockfish is certainly getting larger consistently. It is also getting stronger. So, pick what you want.

Web browsers don't like dealing with a hundred megabyte binary blobs. So, if you're site does not need the absolute strongest engine on earth and the older versions run faster, I'd recommend using an older version of Stockfish, like 16 or even earlier. It's still far stronger than any human on earth will ever be.

If you need the strongest version, you may have to put up with larger file sizes and slower start up times. Or look for creative solutions. Most of that is outside of the scope of this project, which is trying to keep in sync with Stockfish development, which is not concerned about those things.

In summary, depth is not a good performance indicator, and I'm not seeing significant start up time degradation. Closing for now. Feel free to open or create a new issue if you have some specifics and something for me to reproduce.

Thanks.