Support context menu modifications for third-party file managers (XYplorer, Total Commander, Double Commander and possibly more)

[geezmolycos]

I implemented an context menu shell extension in Nilesoft Shell to retrieve the selected file list, so that it works in most file managers where Shell Extensions are allowed to run in context menu. In my test, in XYplorer, Total Commander, Double Commander, Everything context menu modifications are working correctly, in Directory Opus and OneCommander, they are not working because they don't use native context menu.

In the beginning I discovered that Nilesoft Shell doesn't work properly with third-party file managers. It only changes the menu appearence, item addition/modification are not working, like so:

On the Internet I found out lots of discussions on this problem:
https://www.xyplorer.com/xyfc/viewtopic.php?p=226677#p226677
https://www.xyplorer.com/xyfc/viewtopic.php?t=26686
https://web.archive.org/web/20250715220407/https://www.xyplorer.com/xyfc/viewtopic.php?t=27737
https://resource.dopus.com/t/how-can-i-make-the-directory-opus-use-my-windows-context-menu/53508/4
https://www.reddit.com/r/OneCommander/comments/1hf7q6l/request_support_for_nilesoft_shell_integration/

Because I feel like Nilesoft Shell fits well with modern Windows 11 UI, and replacement solutions are not as easy to use, I decided to somehow implement third-party file manager support for Nilesoft Shell. I reseached into the code of Nilesoft Shell and come up with my solution.

The details of this implementation is that:

• I added a dummy context menu Shell Extension that retrieves selected files required by Selection::QuerySelected.
• It gets called whenever a shell context menu is generated, and puts the file list in a global variable g_ShellContext.
• In ContextMenu::Initialize it checks for the dummy context menu item it generated to judge whether a menu is indeed a context menu.
• If it is a context menu and the original method to retrieve selection list is failed, QuerySelected reads selected file list stored in g_ShellContext.

Instead of Windows File Explorer hook that only works in Explorer, this method can retrieve the selected file list wherever Shell Extension runs (in third-party file managers). The original implementation requires a ShellBrowser to be available in Selection::QueryShellWindow, and used in Selection::QuerySelected, but third-party managers implement file browser by themselves, they don't have a ShellBrowser exposed. My implementation doesn't completely change how it works, it only tries to use the Shell Extension retrieved file selection list when the original method fails.

Below are some screenshots showing it work in different file managers. In shell.nss the excluded condition should be changed. like:

XYplorer
exclude.where = !process.is_explorer && (process.name != "XYplorer") && (process.name != "notepad") && (process.name != "dopus") && (process.name != "dopusrt") && (process.name != "doublecmd") && (process.name != "OneCommander")

Total Commander 64-bit

Double Commander

Directory Opus (not working)

Everything (working)

If it's not working, it can be added in "excluded" condition, to remove the "ShellExtSelectionRetriever Placeholder" item in context menu.