-
-
Notifications
You must be signed in to change notification settings - Fork 4.8k
Description
⚠️ This issue respects the following points: ⚠️
- This is a bug, not a question or a configuration/webserver/proxy issue.
- This issue is not already reported on Github OR Nextcloud Community Forum (I've searched it).
- Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
- I agree to follow Nextcloud's Code of Conduct.
Bug description
After a burst of file writes and deletes, an old deleted file is restored.
A snippet from the log file shows
/extra/reference/nextcloud/bobw/files/Experiment
-rw-r--r-- 1 bobw warren 463 2026-03-20 17:24:22 nextcloud_stress_000.txt
-rw-r--r-- 1 bobw warren 464 2026-03-20 17:24:22 nextcloud_stress_001.txt
/extra/client/bobw/Nextcloud/Experiment
-rw-r--r-- 1 bobw warren 463 2026-03-20 17:24:22 nextcloud_stress_000.txt
-rw-r--r-- 1 bobw warren 464 2026-03-20 17:24:22 nextcloud_stress_001.txt
-rw-rw-r-- 1 bobw warren 339 2026-03-20 17:23:02 nextcloud_stress_002.txt
where it is evident that a file "nextcloud_stress_002.txt" which was deleted from the reference
and has been restored on the client
Steps to reproduce
- Set up a Nextcloud Server (which emulates a remote connection with ~150ms delay)
- Set up a Nextcloud Client and Add Folder Sync Connection to folder "Experiment" (not virtual)
- Run a script which generates a burst of file writes and deletes on one client
- then loops waiting for the server and other clients to catch up
- Observe a file restored to the client after it has been deleted
Standalone nextcloud environment for demonstrating issues.
Expected behavior
Deleted files remain deleted
Nextcloud Server version
32
Operating system
Other
PHP engine version
PHP 8.4
Web server
Apache (supported)
Database engine version
MariaDB
Is this bug present after an update or on a fresh install?
None
Are you using the Nextcloud Server Encryption module?
None
What user-backends are you using?
- Default user-backend (database)
- LDAP/ Active Directory
- SSO - SAML
- Other
Configuration report
Server:
sudo -u apache php -f /usr/share/nextcloud/console.php config:list system | grep version\.:
"version": "32.0.6.1",
Client:
nextcloud --version
Nextcloud version 3.17.4daily
Git revision 371e4b9311e851dbb8d6ae6f7c3f77af958e1a33
Using Qt 6.10.1, built against Qt 6.10.1
Using Qt platform plugin 'xcb'
Using 'OpenSSL 3.5.4 30 Sep 2025'
Running on Fedora Linux 43 (Forty Three), x86_64
php -v
PHP 8.4.18 (cli) (built: Feb 10 2026 17:48:03) (NTS gcc x86_64)
Copyright (c) The PHP Group
Built by Fedora Project
Zend Engine v4.4.18, Copyright (c) Zend Technologies
with Zend OPcache v8.4.18, Copyright (c), by Zend TechnologiesList of activated Apps
not relevantNextcloud Signing status
None relevantNextcloud Logs
----
I don't think these very large log files are helpful
ls -lh /extra/server/nextcloud/nextcloud.log*
-rw-r----- 1 apache apache 58M 2026-03-21 12:35 /extra/server/nextcloud/nextcloud.log
-rw-r----- 1 apache apache 101M 2026-03-11 12:55 /extra/server/nextcloud/nextcloud.log.1
veriicon(432)% tail -5 /extra/server/nextcloud/nextcloud.log
{"reqId":"ab6QXDZn64HOfHIp0FIA8AAAAAY","level":1,"time":"March 21, 2026 12:34:37","remoteAddr":"127.0.0.1","user":"bobw","app":"no app in context","method":"GET","url":"/nextcloud/ocs/v1.php/cloud/user?format=json","scriptName":"/nextcloud/ocs/v1.php","message":"The user config key files/quota is not defined in the config lexicon","userAgent":"Mozilla/5.0 (Linux) mirall/3.17.4daily (Nextcloud, fedora-6.19.7-200.fc43.x86_64 ClientArchitecture: x86_64 OsArchitecture: x86_64)","version":"32.0.6.1","clientReqId":"071e2eb9-6e9e-4118-8849-7c2fc997de0a","data":[]}
{"reqId":"ab6QbnRAHlXrGeEwefr-bwAAANU","level":1,"time":"March 21, 2026 12:34:55","remoteAddr":"192.168.0.13","user":"bobw","app":"no app in context","method":"GET","url":"/nextcloud/ocs/v1.php/cloud/user?format=json","scriptName":"/nextcloud/ocs/v1.php","message":"The user config key files/quota is not defined in the config lexicon","userAgent":"Mozilla/5.0 (Windows) mirall/4.0.6 (build 20260122) (Nextcloud, windows-10.0.19045 ClientArchitecture: x86_64 OsArchitecture: x86_64)","version":"32.0.6.1","clientReqId":"7582df10-f3f7-40e5-bd42-fa51d730b120","data":[]}
{"reqId":"ab6QdeBDb7gmCPUTEfkPTAAAAEw","level":1,"time":"March 21, 2026 12:35:02","remoteAddr":"192.168.0.15","user":"bobw","app":"no app in context","method":"GET","url":"/nextcloud/ocs/v1.php/cloud/user?format=json","scriptName":"/nextcloud/ocs/v1.php","message":"The user config key files/quota is not defined in the config lexicon","userAgent":"Mozilla/5.0 (Windows) mirall/4.0.6 (build 20260122) (Nextcloud, windows-10.0.19045 ClientArchitecture: x86_64 OsArchitecture: x86_64)","version":"32.0.6.1","clientReqId":"08c23978-6058-4b1b-b2f0-f3fe02a0a21e","data":[]}
{"reqId":"ab6QfzZn64HOfHIp0FIA9wAAAAM","level":1,"time":"March 21, 2026 12:35:12","remoteAddr":"127.0.0.1","user":"bobw","app":"no app in context","method":"GET","url":"/nextcloud/ocs/v1.php/cloud/user?format=json","scriptName":"/nextcloud/ocs/v1.php","message":"The user config key files/quota is not defined in the config lexicon","userAgent":"Mozilla/5.0 (Linux) mirall/3.17.4daily (Nextcloud, fedora-6.19.7-200.fc43.x86_64 ClientArchitecture: x86_64 OsArchitecture: x86_64)","version":"32.0.6.1","clientReqId":"3956895a-bb3a-448d-b1cb-43b59cbdff3b","data":[]}
{"reqId":"ab6QkOBDb7gmCPUTEfkPUgAAAFY","level":1,"time":"March 21, 2026 12:35:29","remoteAddr":"192.168.0.13","user":"bobw","app":"no app in context","method":"GET","url":"/nextcloud/ocs/v1.php/cloud/user?format=json","scriptName":"/nextcloud/ocs/v1.php","message":"The user config key files/quota is not defined in the config lexicon","userAgent":"Mozilla/5.0 (Windows) mirall/4.0.6 (build 20260122) (Nextcloud, windows-10.0.19045 ClientArchitecture: x86_64 OsArchitecture: x86_64)","version":"32.0.6.1","clientReqId":"42f963fc-255e-477d-a8f5-4a9fd2707f82","data":[]}Additional info
I suspect this behaviour is the cause real problems in my production environment.
It is easily reproducable, but may take thousands of attempts to capture.
The file write burst is generated from a script with an inner loop which is something like:
rm "${filename_prefix}"_${delete_filename_idx}.txt
xxd -l ${size} -p -c 0 /dev/urandom > "${filename_prefix}"_${filename_idx}.txt
where for each iteration
- size is incremented
- delete_idx is 2 behind filename_idx
- filename_idx is incremented modulo 3
(the actual script starts the file size at 0, then increments the file size by 1 in each iteration)
Because each file is a unique size and consecutive it is easy to spot if a previously deleted file
has been restored. In the log snippet above, the most recently deleted file would have size 463, but the fallaciously restored file has size 339 meaning it was deleted several iterations in the past.
The monitored paths are listed near the top of the log file, where
- client_0 path is the reference
- client_1 path is the primary client, file writes and deletes only to this location
- client_9 path is the raw server storage (only observed for investigative purposes)
- plus various other clients including webdav and davfs
The full log file is attached.
nextcloud_stress_260320_2111.log
It shows multiple failure modes:
- Deleted files are restored (which is detected after a catchup timeout has expired)
- Conflicts
- Incorrect mtime (on the server storage, only observed for investigative purposes)
Metadata
Metadata
Assignees
Labels
Type
Projects
Status