Storage migration is unreasonably slow for worlds with a lot of files

[personalized-advertising]

Spark Profile

Not possible to spark this early on, visual vm screenshot below

Description of issue

For a world that is 500gb for example world migration just takes so long

shouldn't it just be renaming a few folders?

Plugin and Datapack List

N/A

Server config files

N/A

Paper version

26.1.2 76d2ac7

Other

No response

[lynxplay]

The migration logic spins up a copy-on-write FS in a temporary directory to prevent your world from possibly being left corrupted if the server dies during the migration.

Spinning up that COW FS on large large folders sadly is not very fast but I don't think paper should remove such precautions and possibly leave worlds in a broken state if the server process fails during the process.

[Malfrador]

Are you testing this on Windows by any chance?
With a Windows file system Files#isWritable, which checkAttributes mostly is, will resolve the full path, check per-file security settings, ACLs, read-only volumes. Additionally Windows Defender or other AVs can also intercept this and do their stuff unless the directory is excluded.
This will be rather slow.

On a Unix file system, it just checks the file writable bit, which would be a single syscall and pretty fast.

Though I am not entirely convinced that isWritable on every single file is really needed here, checking just the directory should be enough. Given most servers run on Linux systems, its probably not worth changing if its a Windows-only issue though.

[se27213]

Not looking forward to this, overworld is over 4TB. Actually not even sure how to do this since we use symlinks for all dimensions atm.