Skip to content

Prefetching (non-slop)#40

Open
markovejnovic wants to merge 54 commits intomainfrom
marko/mes-696-lookup-pre-fetch-post-readdir-2
Open

Prefetching (non-slop)#40
markovejnovic wants to merge 54 commits intomainfrom
marko/mes-696-lookup-pre-fetch-post-readdir-2

Conversation

@markovejnovic
Copy link
Collaborator

No description provided.

@blacksmith-sh

This comment has been minimized.

@mesa-dot-dev
Copy link

mesa-dot-dev bot commented Feb 18, 2026

Mesa Description

This PR introduces a significant architectural refactoring by extracting core filesystem and caching logic into a new, reusable git-fs library. The existing MesaFS implementation has been refactored to use this new library, replacing a bespoke, tightly-coupled system with a generic, asynchronous, and multi-layered caching architecture. This change establishes a more modular and maintainable foundation, paving the way for future performance enhancements like prefetching.

The previous filesystem logic, including inode management and caching, was deeply integrated within the mescloud modules. This made the system difficult to extend and improve. This refactoring was motivated by the need to create a clean, reusable abstraction layer that could support more advanced features and improve concurrency.

Key Changes:

  • New git-fs Library (lib/)

    • A new library was created to house generic, reusable filesystem components.
    • AsyncFs: An asynchronous, cached filesystem layer that manages inode lifecycles, attribute caching, and directory entry caching.
    • FutureBackedCache: A new concurrent cache designed to deduplicate in-flight asynchronous operations, ensuring that data-fetching logic for a given key is executed only once, even with many concurrent requests.
    • DCache: A dedicated, concurrent cache for directory entries to speed up readdir operations.
    • Core Abstractions: Introduced fundamental traits like FsDataProvider and FileReader to decouple the filesystem implementation from the underlying data source (e.g., the Mesa API).
  • Refactoring of MesaFS and Daemon

    • RepoFs, OrgFs, and CompositeFs were re-implemented to use the new git-fs traits, acting as data providers rather than managing their own complex state and caching.
    • The main daemon was updated to initialize and use the new, generic FuserAdapter from git-fs, simplifying the bridge between the FUSE kernel interface and our filesystem logic.
  • Deletion of Legacy Code

    • The entire src/fs/icache module, which contained the previous custom inode caching implementation, has been removed.
    • The old Fs trait and its associated data structures (src/fs/trait.rs) have been deleted and replaced by the more robust abstractions in the new library.

Description generated by Mesa. Update settings

Copy link

@mesa-dot-dev mesa-dot-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Performed full review of e2d3738...b4fa0dc

Analysis

  1. Incomplete architecture for new asynchronous filesystem caching - the AsyncFs component lacks critical interfaces for table population and write-through to the directory cache, making these data structures currently unusable.

  2. Background task orchestration framework is added but not connected to the rest of the system, leaving the foundational infrastructure without clear integration paths.

  3. Significant regression in telemetry configuration - removal of Trc::with_telemetry and hard-coding of OTLP exporter means application-level telemetry settings are now ignored.

  4. The PR attempts to introduce multiple significant architectural changes simultaneously (async filesystem caching, background tasks, telemetry rework) without complete implementation paths for each.

Tip

Help

Slash Commands:

  • /review - Request a full code review
  • /review latest - Review only changes since the last review
  • /describe - Generate PR description. This will update the PR body or issue comment depending on your configuration
  • /help - Get help with Mesa commands and configuration options

0 files reviewed | 9 comments | Edit Agent SettingsRead Docs

@markovejnovic markovejnovic force-pushed the marko/mes-696-lookup-pre-fetch-post-readdir-2 branch 2 times, most recently from 4bb43c2 to 6d0510c Compare February 19, 2026 00:13
@markovejnovic markovejnovic force-pushed the marko/mes-696-lookup-pre-fetch-post-readdir-2 branch from 176d4d3 to c8b0bd4 Compare February 19, 2026 00:24
These items are still used by OrgFs — they were incorrectly annotated
as dead after removing RepoFs's usage.
Rename CompositeFs -> CompositeFsLegacy and ChildSlot -> ChildSlotLegacy
to make room for the new CompositeFs implementation that uses
InodeBridge + DCache instead of the icache/HashMapBridge layer.
Add the new CompositeFs<Inner> that replaces the two-generic-parameter
CompositeFsLegacy<R, Inner> with a simpler design:

- Uses FutureBackedCache + DCache + InodeBridge instead of MescloudICache
  + HashMapBridge + FileTable
- Owns refcounts, open-file table, and inode-to-slot mappings directly
- Implements delegated_lookup with DCache fast path, delegated_readdir
  with readdir_populated guard, delegated_getattr, delegated_statfs,
  delegated_open/read/release, and delegated_forget
- Provides add_child, cache_inode, cache_inode_and_inc_rc helpers

All new code is behind #[expect(dead_code)] until MesaFS/OrgFs migrate.
MesaFS now wraps CompositeFs<OrgFs> and OrgFs wraps
CompositeFs<RepoFs>, replacing CompositeFsLegacy and its
MesaResolver/OrgResolver. The legacy icache infrastructure
(MescloudICache, HashMapBridge, FileTable, InodeFactory, etc.)
is now dead code, annotated with #[allow(dead_code)] for removal
in a follow-up cleanup task.
Delete the entire legacy inode cache infrastructure that became unused
after migrating MesaFS and OrgFs to the new CompositeFs:

- src/fs/icache/ module (AsyncICache, IcbResolver, IcbLike, HashMapBridge,
  FileTable, InodeFactory) — 1,637 lines
- src/fs/mescloud/icache.rs (MescloudICache, InodeControlBlock,
  make_common_file_attr, blocks_of_size) — 470 lines
- CompositeFsLegacy and ChildSlotLegacy from composite.rs — 323 lines

Net: -2,430 lines removed.
Replace the generic Fs trait with the INode-native ChildFs trait across
the entire filesystem stack. CompositeFs, OrgFs, and MesaFS now work
directly with lib types (INode, InodeAddr, FsDirEntry). FuserAdapter
becomes concrete over MesaFS with INode-to-FileAttr conversion only at
the FUSE boundary. Delete the now-unused trait.rs.
@blacksmith-sh
Copy link
Contributor

blacksmith-sh bot commented Feb 20, 2026

Found 2 test failures on Blacksmith runners:

Failures

Test View Logs
test_directory_layout/test_directory_layout_matches_clone[github-samples/planventure] View Logs
test_directory_layout/test_directory_layout_matches_clone[kelseyhightower/nocode] View Logs

Fix in Cursor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

Comments