Add NFSv3 client support via bundled libnfs - #19269
Conversation
Mirror the SMB VFS/menu path with nfs:// browsing, settings, Apple and qb builds, and fix Ozone OSK word_wrap overlap crash on macOS.
e67b56c to
90b58a2
Compare
|
Would this work on android too? |
Match HAVE_BUILTINSMBCLIENT so webOS packages ship the read-only NFS browser without needing a system libnfs.
libsmb2's include/slist.h was winning on the -I path, so LIBNFS_LIST_* macros stayed undefined and the webOS link failed. Match Makefile.
|
it should, same logic as of userspace smb for userspace nfs (both libs same author) |
Keepalive setsockopt failures and app UIDs were aborting NFSv3 mounts on sandboxed Linux targets; prefer IPv4, normalize export paths, and show libnfs errors when Browse fails.
|
in my case was nfs server setup issue forgot this is working setup |
|
In linux I usually have to mount the nfs server on boot and I want to share a external drive. This is difficult to do after mounting of the rest of external drives is complete (systems does that as a given I think sysv, not so much). So what I do is mount /media. I recommend you add crossmnt if you want this scheme and look up what fsid=0 and all_squash do (respectively, treat /media as a "root point" (you can use /media/$user if its a real directory instead of a system created thing) and not have to add /media to the uri, and all_squash maps all logged in users to a opaque user with only permissions to that place. You can also have the inner network (your 192.168.1.0/24 ) read-write and the rest (signified as |
|
right now it works my problem was here |
Copy nfs:// and smb:// files into Cache/vfs_stage so cores can load without fopen on VFS URLs, skip async prefetch for those schemes, and keep export path stripping aligned with mount normalization.
Move VFS URL early-outs after declarations for linux-c89, and gate HAVE_NFSCLIENT to macosx so iOS does not link without bundled libnfs.
Remove dead buf declarations from mount XDR stubs and avoid redefining _GNU_SOURCE when the build already provides it.
nfs_open cannot open zip#rom paths; copy the archive alone and reattach the entry suffix so local extract/load still works.
|
finally loads on webos |
xcconfig only accepts // comments; /* */ broke iOS and macOS CI parses.
Drop the forced read-only mount flag and wire open/write paths so saves and other writers can use nfs:// like local files.
Resolve materialui NFS settings label to MENU_ENUM_LABEL_*_STR style.
theromis
changed the title
|
now NFSv3 fully supported: read/write/create/truncate |
Expose HAVE_NFSCLIENT on all Apple SDKs and compile libnfs sources into the RetroArch_iOS13 iOS/tvOS targets.
Compile vendored libnfs as separate TUs and prefer its include path so slist.h resolves correctly alongside libsmb2.
Fall back to vendored libnfs when system libnfs is not available.
Compile bundled libnfs separately from griffin, and add win32_compat for MSVC/MinGW.
NDK builds need major()/minor() from sys/sysmacros.h when compiling nfs_v3.
|
This point right here:
Can we turn that off generally and for selected playlists? I'm not very enthusiastic about making my playlist a delay machine where I have to copy 600mb of chd (a streamable format) before it even starts. |
Fixes linux-nomenu (--disable-menu) when bundled libnfs is enabled by default.
|
Right now NFS/SMB content is always staged to the local cache before load (same for VFS and non-VFS cores). That was the safe default so cores that only fopen() local paths still work. You’re right that for streamable formats like CHD this is painful — a 600 MB copy before start is not ideal. An option to skip staging when the core reports VFS support (globally and/or per-playlist) would make sense; I can add implementation. |
socket.c includes it on WIN32; without the win32 include path MSVC/UWP fail.
Gate bundled libnfs on NETWORKING, relax C89 pedantic flags for vendored libnfs TUs, and define HAVE_UTIME_H for Win32/MinGW.
RetroArch MSVC projects use CompileAsCpp globally; libnfs uses an export identifier and void* conversions that break as C++20.
linux-c89 builds vfs_implementation_nfs.o under OBJDIR/./libretro-common.
Switch, Vita, Wii U, Wii, 3DS flip HAVE_BUILTINNFSCLIENT; PS3 gets griffin-side libnfs objects plus vendored ps3_compat. Stub major/minor for non-Linux/Darwin targets that lack sys/sysmacros.h.
There are even some cores like dosbox-pure that make zips... sort of random access and handle zip loading internally, what it does is to make zip create file offset information to resume into the offsets of single internal files the first time a zip is open. Its a good play when romsets insist on zip. But not all cores are like that most dump the rom into memory if small enough (note, copying to client to disk would still be unnecessary caching in this case), or let retroarch cache zip mechanism handle it. I was assuming youre implementing that for feature parity but when I saw "all content" not just #game.rom content I though it better to interject. I suspect you could also get speedups and less disk usage if on supported cores while reading a zip/7z #content you setup a option to cache the rom to memor (generally). Although not all cores support memory loading. |
VFS-capable cores can open network URLs directly (stream CHD). Keep staging for non-VFS need_fullpath cores and for archive#entry paths.
Switch/Vita/Wii/WiiU/3DS/PS3 CI fails on missing headers and POSIX APIs; revert enablement and drop unused ps3_compat until ported.
|
Agreed on the zip/memory point too. Staging default is updated: nfs:// / smb:// are only copied to cache when the core needs a full path and lacks VFS, or when the path is archive#entry (nfs/smb can’t open # directly). VFS cores should stream (e.g. CHD) without a full local copy. Caching zip/7z #entry into memory on supported cores is a separate, more general improvement — happy to look at that as a follow-up; it’s not NFS-specific. |
|
I'm ok with that last. I'd suggest that if possible the least code duplication between smb/nfs and retroarch itself zip#game.rom caching mechanism happen, ideally theyd be the same so improvements to one would apply to the other (such as the proposal to force zip#game.rom content to memory) I myself only keep playlists with zips of max 200mb or so (nds), so I'd like this to save wear on the phone ssd. Per playlist would be safer than general for people with some small zips but some large zips playlists. Also some cores like name, dosbox-pure, fbneo etc all cores that use zip itself as a allowed extension dont need this since they want their own handling. It seems obvious but just in case to avoid confusion. |
|
Another reason to share the mechanism is that some core requirements aren't obvious. Sometimes you can extract just the zip#rom referenced. Sometimes that "single" rom is just a entry point to the rest of the files. I think retroarch currently extracts the whole file (I'm not sure, but its the safest alternative, disregarding the people that zip up whole sets), but in a future improvement it may only do so for "index" files and their dependencies (.scummvm, .m3u, .cue, .toc etc etc). Respectively those files need the current directory in the zip, all the files (including other index files and contents) referenced inside, the referenced cue and track files inside it, ditto etc etc. The requirements can be different although you can safely assume when finding a index file that you want to extract the whole zip, disregarding people with zipped up romsets again, who just can't win with doing this and using index files without extreme care to specify all index file requirements and parsing them which probably won't happen). Then there are nes pallete files, sbi files, softpatched roms, and those also apply to all files referenced and retroarch itself has a stacked softpatch convention/extension (.ips.ips1,.ips2,.bps3 etc), or chd parent/child dependencies so thats fun. Generally supporting "multiple games in a single zip" is somewhat complicated and if it ever happens, it needs heuristics in a single place. |
|
Sounds good. Staging default is already the VFS-aware path we discussed (stream when the core supports VFS; cache only for non-VFS On the zip/ I’m fine taking that shared path as a next step after this lands. |
Use one content_path_needs_stage gate and a single cache copy before memory load/extract, preserving VFS-aware streaming for CHD etc.
|
Quick update on the shared path: staging for Optional memory cache for zip |
|
This is going to sound kind of stupid, but if youre recognizing "files inside archives" by the Any filename + # + something else in the filename part of a path, be aware that # is not a illegal windows or Unix character. So what you actually want is ".zip#anything" one for each supported compression type. I didnt see the code so I dont know if it has this problem, but just in case, I wouldn't put it past a rom or two to have a official name with # in it. |
|
Good catch — RetroArch already does it that way. |
|
So for that |
2 participants
Summary
nfs://) backed by vendoreddeps/libnfs, mirroring the existing SMB client for browse, load, and write/create/truncate.Makefile, Apple, Android, Windows, Linux, webOS, and networked consoles (HAVE_NFSCLIENT/HAVE_BUILTINNFSCLIENT).nfs:///smb://to local cache only when needed: cores without VFS that require a full path, andarchive#entry(nfs/smb cannot open#paths). VFS-capable cores stream URLs directly (no full CHD pre-copy).word_wrap/ overlapping buffers).Test plan
zip#rom)HAVE_BUILTINNFSCLIENTinMakefile.webosnfs://starts without copying the whole file to cacherwNFS export (saves or manual path)./configurebuild smoke test (builtin libnfs auto)