Skip to content

Initialize the newer core filesystem design#887

Open
jaybosamiya-ms wants to merge 16 commits into
mainfrom
jayb/fs-via-backend-part1
Open

Initialize the newer core filesystem design#887
jaybosamiya-ms wants to merge 16 commits into
mainfrom
jayb/fs-via-backend-part1

Conversation

@jaybosamiya-ms

@jaybosamiya-ms jaybosamiya-ms commented May 30, 2026

Copy link
Copy Markdown
Member

This PR introduces some of the key components the new filesystem design. Concretely, it sets up a new Backend abstraction, which is used by the Resolver to provide a FileSystem. This PR itself is part 1 of the changes towards this new file system design. For now, I have only migrated the devices sub-filesystem over to the new model.

Few points of interest:

  1. the model is somewhat inspired by, but not the same as Linux's vfs system; nonetheless, it should give us a lot of the benefits of centralizing the walking + permission checks
  2. not all elements of resolving/backend are done in this PR; there are some other layers needed to compose different backends together, support mounting, etc.; those will be in future PRs
  3. the interfaces for the new design should be largely stable, but might have some shifts happen as I migrate more of our existing file systems over
  4. rather than just migrate devices over, I also fixed up some of the unimplemented!s in it along the way
  5. some litebox shim tests needed fixes, since they were previously not accounting for /dev showing up

@jaybosamiya-ms jaybosamiya-ms force-pushed the jayb/fs-via-backend-part1 branch from 1fbc7ce to 0bea5df Compare May 30, 2026 02:28
@jaybosamiya-ms jaybosamiya-ms marked this pull request as ready for review May 30, 2026 02:37
@jaybosamiya-ms jaybosamiya-ms force-pushed the jayb/fs-via-backend-part1 branch from 4189660 to 89cd30b Compare May 30, 2026 02:39
Comment on lines +251 to +252
let read_allowed = access_mode == OFlags::RDONLY || access_mode == OFlags::RDWR;
let write_allowed = access_mode == OFlags::WRONLY || access_mode == OFlags::RDWR;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to check if the user has the read or write permission before we say it's allowed?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Weidong, when improving this, I realized I could improve the whole walking situation a bit more, and simplify open itself, making it much cleaner and easier to work with. The latest version has that nicer design, and the user permission checking is included (and the type system will remind us now).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to check executable permission at some place?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For files, executable permissions don't quite matter when opening, it only matters if you are trying to execveing or similar, so it would that the shim's job to use fd_file_status or similar.

Comment thread litebox/src/fs/resolver.rs Outdated

@wdcui wdcui left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for staring this sequence of PRs to improve the fs code. I left a couple of comments below.

@jaybosamiya-ms jaybosamiya-ms force-pushed the jayb/fs-via-backend-part1 branch from 89cd30b to 76ff35d Compare June 4, 2026 23:15
@jaybosamiya-ms jaybosamiya-ms force-pushed the jayb/fs-via-backend-part1 branch from 76ff35d to 7cd8d12 Compare June 4, 2026 23:26
@jaybosamiya-ms jaybosamiya-ms requested a review from wdcui June 4, 2026 23:31
@github-actions

github-actions Bot commented Jun 4, 2026

Copy link
Copy Markdown

🤖 SemverChecks 🤖 ⚠️ Potential breaking API changes detected ⚠️

Click for details
--- failure struct_missing: pub struct removed or renamed ---

Description:
A publicly-visible struct cannot be imported by its prior path. A `pub use` may have been removed, or the struct itself may have been renamed or removed entirely.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.48.0/src/lints/struct_missing.ron

Failed in:
  struct litebox::fs::devices::FileSystem, previously in file /home/runner/work/litebox/litebox/target/semver-checks/git-main/84f28ec743c369a9c8d81f9f732e00d61463d575/litebox/src/fs/devices.rs:68

@wdcui wdcui left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I left a couple of minor comments. Thanks.

Comment thread litebox/src/fs/devices.rs
rdev: core::num::NonZeroUsize::new(0x109),
};
#[derive(Debug, Clone, Copy)]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Do we want to add this empty line?

@jaybosamiya-ms jaybosamiya-ms Jun 15, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was weird for the enum to be sitting right up against the existing consts from a grouping perspective, so I had added a newline here.

Comment on lines +251 to +252
let read_allowed = access_mode == OFlags::RDONLY || access_mode == OFlags::RDWR;
let write_allowed = access_mode == OFlags::WRONLY || access_mode == OFlags::RDWR;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to check executable permission at some place?

@wdcui

wdcui commented Jun 13, 2026

Copy link
Copy Markdown
Member

GPT-5.5 found the following issues. You can decide if they should be addressed in this PR.

  1. litebox/src/fs/resolver.rs:361-365 — High — O_CREAT | O_EXCL can run backend open side effects before failing.
    In the existing-file path, the resolver calls self.backend.open_file_at(outcome.last, name, flags)? before checking O_CREAT | O_EXCL and returning
    OpenError::AlreadyExists. Since open_file_at receives the full flags (litebox/src/fs/backend.rs:82-87) and O_TRUNC is accepted by the resolver
    (resolver.rs:292-304), a backend that honors open-time truncation could modify the file before exclusive-create failure is returned.
    Nuance: the only current Backend impl appears to be Devices, and its O_TRUNC handling does not mutate anything (devices.rs:270-277, 356-358). So this is
    a real generic resolver/contract bug, but likely not observable today with only the devices backend. The TODO at resolver.rs:361-362 explicitly confirms
    the intended fix.

  2. litebox/src/fs/resolver.rs:329-336, litebox/src/fs/resolver.rs:348-355 — Medium — O_TRUNC is silently ignored for directories.
    The resolver opens root as a directory fd immediately at resolver.rs:329-336, and does the same for a resolved directory like /dev at
    resolver.rs:348-355. Neither branch checks O_TRUNC, even though O_TRUNC is accepted in the supported flags at resolver.rs:292-304.
    This is a behavior regression versus the existing in-memory filesystem: in_mem.rs:284-291 applies truncation after open, and in_mem.rs:436-448 returns
    TruncateError::IsDirectory when the fd is a directory. OpenError also explicitly supports TruncateError at errors.rs:29-30. So directory opens with
    O_TRUNC should not silently succeed unless that semantic change is intentional.

  3. litebox/src/fs/resolver.rs:292-304, litebox/src/fs/resolver.rs:309, litebox/src/fs/resolver.rs:324, litebox/src/fs/resolver.rs:439-441,
    litebox/src/fs/resolver.rs:478-480, litebox/src/fs/resolver.rs:516-518, litebox/src/fs/resolver.rs:568-570, litebox/src/fs/resolver.rs:675-677 — Medium —
    Accepted O_PATH fds can panic on normal APIs.
    O_PATH is explicitly accepted by open at resolver.rs:292-304, recorded as path_only at resolver.rs:309, and stored in ResolverEntry at resolver.rs:324.
    Several fd operations then detect path_only and call unimplemented!, which panics instead of returning an error: read at resolver.rs:439-441, write at
    resolver.rs:478-480, seek at resolver.rs:516-518, truncate at resolver.rs:568-570, and read_dir at resolver.rs:675-677.
    Nuance: some calls may return an earlier error first depending on handle type or access mode, e.g. directory reads can return ReadError::NotAFile, and
    O_PATH without write access can return NotForWriting before reaching the panic. But valid O_PATH fds can still hit these unimplemented! paths, so
    accepting O_PATH while leaving these operations panicking is a real issue.

  4. litebox/src/fs/devices.rs:138-143, litebox/src/fs/inode_allocator.rs:19-23, litebox/src/fs/inode_allocator.rs:38-44,
    litebox/src/fs/devices.rs:371-390 — Medium — / and /dev report the same inode.
    Devices::new allocates /dev with allocator.next() at devices.rs:138-143. InodeAllocator::for_device starts its counter at 0 (inode_allocator.rs:19-23),
    and next() returns the current counter before incrementing (inode_allocator.rs:38-44), so /dev gets inode 0.
    But root directory status also hard-codes ino: 0 while using the same dev as /dev at devices.rs:371-382. /dev status then returns self.dev_dir_inode at
    devices.rs:385-390. So stat("/") and stat("/dev") can both report the same (dev, ino).
    This is exposed through directory listing too: read_dir("/") reports the /dev entry using self.dev_dir_inode at devices.rs:285-291, while
    file_status("/")/file_status("/dev") go through resolver.rs:700-729 and surface the colliding dir_status values.

@jaybosamiya-ms jaybosamiya-ms left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Weidong! For the GPT-5.5 discovered issues:

  1. I already have an existing TODO to account for this in a future PR (GPT-5.5 even recognizes that I have a TODO for it); I did not want to block this PR on figuring out this currently-unobservable detail.
  2. Does not apply right now, because I've only migrated the devices file system which is read-only; it is a hazard, and will be handled in future PRs with mutable file systems.
  3. unimplemented!() is an explicit signal of "we will implement this later", the panic there is intentionally indicating "we need to solve this in a future PR"
  4. This is a known issue due to the ongoing migration of the different backends, will be resolved in future PRs (there are technically even more collisions possible due to the STDIO_NODE_INFO and such constants), yes the inode allocation needs improvement to make consistent throughout the codebase, but that will come as one of the last few PRs in the series, because it needs coordination across all backends (that's why I'm migrating everything over to the inode allocator design that this PR introduces).

Comment on lines +251 to +252
let read_allowed = access_mode == OFlags::RDONLY || access_mode == OFlags::RDWR;
let write_allowed = access_mode == OFlags::WRONLY || access_mode == OFlags::RDWR;

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For files, executable permissions don't quite matter when opening, it only matters if you are trying to execveing or similar, so it would that the shim's job to use fd_file_status or similar.

Comment thread litebox/src/fs/devices.rs
rdev: core::num::NonZeroUsize::new(0x109),
};
#[derive(Debug, Clone, Copy)]

@jaybosamiya-ms jaybosamiya-ms Jun 15, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was weird for the enum to be sitting right up against the existing consts from a grouping perspective, so I had added a newline here.

@jaybosamiya-ms

Copy link
Copy Markdown
Member Author

In prep for the next PR in the series, I've discovered that it may be helpful to slightly tweak the new Backend abstraction that this PR introduces. I don't think we should block merging this on that change, but just making a note that there may be a change to the Backend trait in a future PR if it leads to a simpler overall design (I still need to play around with the implications of the tweaked design I came up with this weekend, thus I don't want to tweak this PR just yet).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants