FSKit and Network File Systems?

Hi folks! I’ve been paying attention to FSKit for moment to develop a network file system designed for source control-use cases (à la Eden or Google’s CITC). The design goal is support instant clones, even of massive repositories, by lazily fetching files as needed.

Based on this thread, it seems like macOS 27 has added some of the requisite APIs needed to support inode invalidation for network/shared file systems like the cache coherency APIs. For example, I’m thinking that for this file system, I’d want to bypass the kernel’s own caching and have my file system be entirely responsible for it, as it‘d have a better understanding/picture of what is up-to-date and what isn’t and there won’t need to be multiple layers of cache invalidation/coherency. Am I correctly reading the intent of these new APIs?

Answered by Apple Staff in 891627022

That's correct, it's designed to manage cache coherency for network file systems and other kinds of file systems where an outside actor might modify the data outside of the kernel's normal data flow. By adopting this protocol, such an actor can inform the kernel that its cached data is now stale, or ask the kernel to push out any dirty cached data it might have in order to be safely read or modified by someone else.

It's essentially a generalized form of "leasing".

Based on the responses in this thread, it looks like FSVolume.DataCacheHandler is designed for network file systems. I can't wait to dig in!

That's correct, it's designed to manage cache coherency for network file systems and other kinds of file systems where an outside actor might modify the data outside of the kernel's normal data flow. By adopting this protocol, such an actor can inform the kernel that its cached data is now stale, or ask the kernel to push out any dirty cached data it might have in order to be safely read or modified by someone else.

It's essentially a generalized form of "leasing".

https://developer.apple.com/documentation/fskit/fsfilesystem still says

 The current version of FSKit supports only FSUnaryFileSystem, not FSFileSystem.

I used to think FSUnaryFileSystem only supports physical partitions, and FSFileSystem is needed for virtual or networked file systems.

Are the docs out of date (that note has been there since Sequoia) or is my understanding wrong (it's pretty naive) or is something else going on?

(I am not sure if Apple folks are still taking questions, if not I might ask on the main forum)

FSKit and Network File Systems?
 
 
Q