Capture failure due to newline parsing in files with CRLF line endings

[ctm22396]

Describe the bug

nvim-orgmode capture finalization for custom datetree target fails with Index out of bounds when destination .org file has CRLF line endings and has not yet been opened in the current Neovim session. This can be considered a Windows issue, but is more specifically a consequence of file formats that use CRLF line endings.

Here is the traceback:

E5108: Lua: ...l/lazyvim-data/lazy/orgmode/lua/orgmode/capture/init.lua:369: Index out of bounds
stack traceback:
[C]: in function 'nvim_buf_get_lines'
...l/lazyvim-data/lazy/orgmode/lua/orgmode/capture/init.lua:369: in function 'is_line_empty'
...l/lazyvim-data/lazy/orgmode/lua/orgmode/capture/init.lua:374: in function '_get_destination_range_without_empty_lines'
...l/lazyvim-data/lazy/orgmode/lua/orgmode/capture/init.lua:202: in function 'action'
...cal/lazyvim-data/lazy/orgmode/lua/orgmode/files/file.lua:152: in function 'update'
...cal/lazyvim-data/lazy/orgmode/lua/orgmode/files/file.lua:162: in function 'update_sync'
...l/lazyvim-data/lazy/orgmode/lua/orgmode/capture/init.lua:194: in function '_refile_from_capture_buffer'
...l/lazyvim-data/lazy/orgmode/lua/orgmode/capture/init.lua:130: in function <...l/lazyvim-data/lazy/orgmode/lua/orgmode/capture/init.lua:123>

I also suspect the same same bug may affect agenda navigation. I have found that selecting an item from the agenda sometimes jumps to the bottom of the target buffer instead of the correct headline, suggesting that the target_line may be greater than the total number of lines in the file.

checkhealth

Orgmode ~

• ✅ OK Treesitter grammar installed (version 2.0.4)
• ✅ OK Setup called
• ✅ OK org_agenda_files configured
• ✅ OK org_default_notes_file configured
• ⚠️ WARNING shellslash is not set. This might cause issues with file paths in links. Set vim.opt.shellslash = true in your configuration.

Steps to reproduce

I reproduced this with a minimal config using only lazy.nvim + nvim-orgmode. Within the minimal config I have defined the following custom template

Steps:

1. Start Neovim with the minimal repro config.
2. Trigger capture. Org capture
3. Press m to select the custom-defined capture template
4. Finalize with C-c C-c.

Expected behavior

The capture should be inserted into the correct datetree location in the path meetings_file

Emacs functionality

No response

Minimal init.lua

local tmp_dir = vim.env.TMPDIR or vim.env.TMP or vim.env.TEMP or "/tmp"
local nvim_root = tmp_dir .. "/nvim_orgmode"
local lazy_root = nvim_root .. "/lazy"
local lazypath = lazy_root .. "/lazy.nvim"

for _, name in ipairs({ "config", "data", "state", "cache" }) do
	vim.env[("XDG_%s_HOME"):format(name:upper())] = nvim_root .. "/" .. name
end

-- Install lazy.nvim if not already installed
if not vim.uv.fs_stat(lazypath) then
	vim.fn.system({
		"git",
		"clone",
		"--filter=blob:none",
		"https://github.com/folke/lazy.nvim.git",
		"--branch=stable", -- latest stable release
		lazypath,
	})
end
vim.opt.rtp:prepend(lazypath)

local repro_dir = vim.fn.stdpath("state") .. "/org_capture_repro"
local meetings_file = repro_dir .. "/meetings.org"
vim.fn.mkdir(repro_dir, "p")

require("lazy").setup({
	{
		"nvim-orgmode/orgmode",
		event = "VeryLazy",
		ft = { "org" },
		config = function()
			require("orgmode").setup({
				org_agenda_files = { repro_dir .. "/*.org" },
				org_default_notes_file = meetings_file,
				org_capture_templates = {
					m = {
						description = "Work Meeting Note",
						template = "*** %? Meeting at %<%H:%M>\t:WORK:MEETING:\nCreated %U",
						target = meetings_file,
						datetree = {
							tree_type = "custom",
							tree = {
								{
									format = "%Y-%m",
									pattern = "^(%d%d%d%d)%-(%d%d)$",
									order = { 1 },
								},
								{
									format = "%Y-%m-%d %A",
									pattern = "^(%d%d%d%d)%-(%d%d)%-(%d%d) (Monday|Tuesday|Wednesday|Thursday|Friday|Saturday|Sunday)$",
									order = { 1, 2 },
								},
							},
						},
					},
				},
			})
		end,
	},
}, {
	root = lazy_root,
	lockfile = nvim_root .. "/lazy.json",
	install = {
		missing = false,
	},
})

require("lazy").sync({
	wait = true,
	show = false,
})

Screenshots and recordings

No response

nvim-orgmode version

f32a06d

OS / Distro

Windows 11

Neovim version/commit

v0.12.4

Additional context

The problem appears to be coming from

destination_headline, target_line = Datetree:new({ files = self.files }):create(opts.template)

I traced it back to the OrgFile:load functions and finally to the root cause: utils.readfile found in utils/init.lua (link below). The issue seems to be the consequence of how this functions splits the file data into lines:

orgmode/lua/orgmode/utils/init.lua

Line 34 in f32a06d

local lines = vim.split(data, '[\r\n]')

The pattern in the sep argument means that \r and \n get split separately instead of together, so CRLF (\r\n) gets treated as two separators instead of one line ending. This appears to double line counts in unopened CRLF files (e.g. fileformat=dos) and then propagates to headling locations/ranges and ultimately to the target line for the capture.

A safer approach that solves this problem could be to be normalize line endings first, and then split on plain \n, for example:

local normalized = data:gsub('\r\n', '\n'):gsub('\r', '\n')
local lines = vim.split(normalized, '\n', { plain = true })

Potential workaround for other users experiencing this issue

Convert all org files to unix file format.

For example in neovim, open up an org file. Then run the command :set ff=unix. Finally save the file.