Response to `applyEdit` never received

[ja-he]

I encountered an issue with my language server stalling after sending an applyEdit request to the client from inside of the processing of an executeCommand request. I believe this is a common pattern (code actions give the client command names, client calls executeCommand for the selected action, in the executeCommand we compute the edit and call applyEdit).

The server ends up infinitely waiting in jsonrpc2/conn.go:118 on the response to the applyEdit.
Thus, this issue might actually belong to jsonrpc2 (or of course be me not using either of the packages right).

I am testing mostly using Neovim, and in Neovim's log (~/.cache/nvim/lsp.log) the requests and responses all look appropriate (until my stalled language server stops participating, after sending applyEdit).

I have implemented a minimal working example (see below) that shows the behavior.
I have also implemented a similar small example in Rust using tower-lsp, which works as expected.

Repro

• compile the code below, of course
• set up with editor (I used Neovim) as client
• open any document with the filetype that starts the server, etc.
• invoke code action (you should see Foo)
• select Foo code action
• it properly (calls executeCommand, which calls applyEdit, which) inserts 'FOO'
• the client's response to the applyEdit is never received by the server, forever waiting on case resp := <-rchan

MWE

full main.go

package main

import (
	"context"
	"fmt"
	"io"
	"os"

	"go.lsp.dev/jsonrpc2"
	lsp "go.lsp.dev/protocol"
	"go.uber.org/zap"
)

type Backend struct {
	client              lsp.Client
}

func (b *Backend) Initialize(ctx context.Context, params *lsp.InitializeParams) (result *lsp.InitializeResult, err error) {
	return &lsp.InitializeResult{
		Capabilities: lsp.ServerCapabilities{
			CodeActionProvider:     true,
			ExecuteCommandProvider: &lsp.ExecuteCommandOptions{Commands: []string{"foo"}},
		},
		ServerInfo: &lsp.ServerInfo{
			Name:    "mycoolserver",
			Version: "0.1.0",
		},
	}, nil
}
func (b *Backend) Initialized(ctx context.Context, params *lsp.InitializedParams) (err error) {
	return nil
}

func (b *Backend) Shutdown(ctx context.Context) (err error) { return nil }
func (b *Backend) Exit(ctx context.Context) (err error)     { return nil }

func (b *Backend) WorkDoneProgressCancel(ctx context.Context, params *lsp.WorkDoneProgressCancelParams) (err error)                                                              { panic("Unimplemented: WorkDoneProgressCancel") }
func (b *Backend) LogTrace(ctx context.Context, params *lsp.LogTraceParams) (err error)                                                                                          { panic("Unimplemented: LogTrace") }
func (b *Backend) SetTrace(ctx context.Context, params *lsp.SetTraceParams) (err error)                                                                                          { panic("Unimplemented: SetTrace") }
func (b *Backend) CodeLens(ctx context.Context, params *lsp.CodeLensParams) (result []lsp.CodeLens, err error)                                                                   { panic("Unimplemented: CodeLens") }
func (b *Backend) CodeLensResolve(ctx context.Context, params *lsp.CodeLens) (result *lsp.CodeLens, err error)                                                                   { panic("Unimplemented: CodeLensResolve") }
func (b *Backend) ColorPresentation(ctx context.Context, params *lsp.ColorPresentationParams) (result []lsp.ColorPresentation, err error)                                        { panic("Unimplemented: ColorPresentation") }
func (b *Backend) Completion(ctx context.Context, params *lsp.CompletionParams) (result *lsp.CompletionList, err error)                                                          { panic("Unimplemented: Completion") }
func (b *Backend) CompletionResolve(ctx context.Context, params *lsp.CompletionItem) (result *lsp.CompletionItem, err error)                                                     { panic("Unimplemented: CompletionResolve") }
func (b *Backend) Declaration(ctx context.Context, params *lsp.DeclarationParams) (result []lsp.Location /* Declaration | DeclarationLink[] | null */, err error)                { panic("Unimplemented: Declaration") }
func (b *Backend) Definition(ctx context.Context, params *lsp.DefinitionParams) (result []lsp.Location /* Definition | DefinitionLink[] | null */, err error)                    { panic("Unimplemented: Definition") }
func (b *Backend) DidChange(ctx context.Context, params *lsp.DidChangeTextDocumentParams) (err error)                                                                            { panic("Unimplemented: DidChange") }
func (b *Backend) DidChangeConfiguration(ctx context.Context, params *lsp.DidChangeConfigurationParams) (err error)                                                              { panic("Unimplemented: DidChangeConfiguration") }
func (b *Backend) DidChangeWatchedFiles(ctx context.Context, params *lsp.DidChangeWatchedFilesParams) (err error)                                                                { panic("Unimplemented: DidChangeWatchedFiles") }
func (b *Backend) DidChangeWorkspaceFolders(ctx context.Context, params *lsp.DidChangeWorkspaceFoldersParams) (err error)                                                        { panic("Unimplemented: DidChangeWorkspaceFolders") }
func (b *Backend) DidClose(ctx context.Context, params *lsp.DidCloseTextDocumentParams) (err error)                                                                              { panic("Unimplemented: DidClose") }
func (b *Backend) DidOpen(ctx context.Context, params *lsp.DidOpenTextDocumentParams) (err error)                                                                                { panic("Unimplemented: DidOpen") }
func (b *Backend) DidSave(ctx context.Context, params *lsp.DidSaveTextDocumentParams) (err error)                                                                                { panic("Unimplemented: DidSave") }
func (b *Backend) DocumentColor(ctx context.Context, params *lsp.DocumentColorParams) (result []lsp.ColorInformation, err error)                                                 { panic("Unimplemented: DocumentColor") }
func (b *Backend) DocumentHighlight(ctx context.Context, params *lsp.DocumentHighlightParams) (result []lsp.DocumentHighlight, err error)                                        { panic("Unimplemented: DocumentHighlight") }
func (b *Backend) DocumentLink(ctx context.Context, params *lsp.DocumentLinkParams) (result []lsp.DocumentLink, err error)                                                       { panic("Unimplemented: DocumentLink") }
func (b *Backend) DocumentLinkResolve(ctx context.Context, params *lsp.DocumentLink) (result *lsp.DocumentLink, err error)                                                       { panic("Unimplemented: DocumentLinkResolve") }
func (b *Backend) DocumentSymbol(ctx context.Context, params *lsp.DocumentSymbolParams) (result []interface{} /* []SymbolInformation | []DocumentSymbol */, err error)           { panic("Unimplemented: DocumentSymbol") }
func (b *Backend) FoldingRanges(ctx context.Context, params *lsp.FoldingRangeParams) (result []lsp.FoldingRange, err error)                                                      { panic("Unimplemented: FoldingRanges") }
func (b *Backend) Formatting(ctx context.Context, params *lsp.DocumentFormattingParams) (result []lsp.TextEdit, err error)                                                       { panic("Unimplemented: Formatting") }
func (b *Backend) Hover(ctx context.Context, params *lsp.HoverParams) (result *lsp.Hover, err error)                                                                             { panic("Unimplemented: Hover") }
func (b *Backend) Implementation(ctx context.Context, params *lsp.ImplementationParams) (result []lsp.Location, err error)                                                       { panic("Unimplemented: Implementation") }
func (b *Backend) OnTypeFormatting(ctx context.Context, params *lsp.DocumentOnTypeFormattingParams) (result []lsp.TextEdit, err error)                                           { panic("Unimplemented: OnTypeFormatting") }
func (b *Backend) PrepareRename(ctx context.Context, params *lsp.PrepareRenameParams) (result *lsp.Range, err error)                                                             { panic("Unimplemented: PrepareRename") }
func (b *Backend) RangeFormatting(ctx context.Context, params *lsp.DocumentRangeFormattingParams) (result []lsp.TextEdit, err error)                                             { panic("Unimplemented: RangeFormatting") }
func (b *Backend) References(ctx context.Context, params *lsp.ReferenceParams) (result []lsp.Location, err error)                                                                { panic("Unimplemented: References") }
func (b *Backend) Rename(ctx context.Context, params *lsp.RenameParams) (result *lsp.WorkspaceEdit, err error)                                                                   { panic("Unimplemented: Rename") }
func (b *Backend) SignatureHelp(ctx context.Context, params *lsp.SignatureHelpParams) (result *lsp.SignatureHelp, err error)                                                     { panic("Unimplemented: SignatureHelp") }
func (b *Backend) Symbols(ctx context.Context, params *lsp.WorkspaceSymbolParams) (result []lsp.SymbolInformation, err error)                                                    { panic("Unimplemented: Symbols") }
func (b *Backend) TypeDefinition(ctx context.Context, params *lsp.TypeDefinitionParams) (result []lsp.Location, err error)                                                       { panic("Unimplemented: TypeDefinition") }
func (b *Backend) WillSave(ctx context.Context, params *lsp.WillSaveTextDocumentParams) (err error)                                                                              { panic("Unimplemented: WillSave") }
func (b *Backend) WillSaveWaitUntil(ctx context.Context, params *lsp.WillSaveTextDocumentParams) (result []lsp.TextEdit, err error)                                              { panic("Unimplemented: WillSaveWaitUntil") }
func (b *Backend) ShowDocument(ctx context.Context, params *lsp.ShowDocumentParams) (result *lsp.ShowDocumentResult, err error)                                                  { panic("Unimplemented: ShowDocument") }
func (b *Backend) WillCreateFiles(ctx context.Context, params *lsp.CreateFilesParams) (result *lsp.WorkspaceEdit, err error)                                                     { panic("Unimplemented: WillCreateFiles") }
func (b *Backend) DidCreateFiles(ctx context.Context, params *lsp.CreateFilesParams) (err error)                                                                                 { panic("Unimplemented: DidCreateFiles") }
func (b *Backend) WillRenameFiles(ctx context.Context, params *lsp.RenameFilesParams) (result *lsp.WorkspaceEdit, err error)                                                     { panic("Unimplemented: WillRenameFiles") }
func (b *Backend) DidRenameFiles(ctx context.Context, params *lsp.RenameFilesParams) (err error)                                                                                 { panic("Unimplemented: DidRenameFiles") }
func (b *Backend) WillDeleteFiles(ctx context.Context, params *lsp.DeleteFilesParams) (result *lsp.WorkspaceEdit, err error)                                                     { panic("Unimplemented: WillDeleteFiles") }
func (b *Backend) DidDeleteFiles(ctx context.Context, params *lsp.DeleteFilesParams) (err error)                                                                                 { panic("Unimplemented: DidDeleteFiles") }
func (b *Backend) CodeLensRefresh(ctx context.Context) (err error)                                                                                                               { panic("Unimplemented: CodeLensRefresh") }
func (b *Backend) PrepareCallHierarchy(ctx context.Context, params *lsp.CallHierarchyPrepareParams) (result []lsp.CallHierarchyItem, err error)                                  { panic("Unimplemented: PrepareCallHierarchy") }
func (b *Backend) IncomingCalls(ctx context.Context, params *lsp.CallHierarchyIncomingCallsParams) (result []lsp.CallHierarchyIncomingCall, err error)                           { panic("Unimplemented: IncomingCalls") }
func (b *Backend) OutgoingCalls(ctx context.Context, params *lsp.CallHierarchyOutgoingCallsParams) (result []lsp.CallHierarchyOutgoingCall, err error)                           { panic("Unimplemented: OutgoingCalls") }
func (b *Backend) SemanticTokensFull(ctx context.Context, params *lsp.SemanticTokensParams) (result *lsp.SemanticTokens, err error)                                              { panic("Unimplemented: SemanticTokensFull") }
func (b *Backend) SemanticTokensFullDelta(ctx context.Context, params *lsp.SemanticTokensDeltaParams) (result interface{} /* SemanticTokens | SemanticTokensDelta */, err error) { panic("Unimplemented: SemanticTokensFullDelta") }
func (b *Backend) SemanticTokensRange(ctx context.Context, params *lsp.SemanticTokensRangeParams) (result *lsp.SemanticTokens, err error)                                        { panic("Unimplemented: SemanticTokensRange") }
func (b *Backend) SemanticTokensRefresh(ctx context.Context) (err error)                                                                                                         { panic("Unimplemented: SemanticTokensRefresh") }
func (b *Backend) LinkedEditingRange(ctx context.Context, params *lsp.LinkedEditingRangeParams) (result *lsp.LinkedEditingRanges, err error)                                     { panic("Unimplemented: LinkedEditingRange") }
func (b *Backend) Moniker(ctx context.Context, params *lsp.MonikerParams) (result []lsp.Moniker, err error)                                                                      { panic("Unimplemented: Moniker") }
func (b *Backend) Request(ctx context.Context, method string, params interface{}) (result interface{}, err error)                                                                { panic("Unimplemented: Request") }

func (b *Backend) CodeAction(ctx context.Context, params *lsp.CodeActionParams) (result []lsp.CodeAction, err error) {
	return []lsp.CodeAction{
		{
			Title: "Foo",
			Kind:  "source",
			Edit:  &lsp.WorkspaceEdit{},
			Command: &lsp.Command{
				Title:     "Foo",
				Command:   "foo",
				Arguments: []interface{}{params.TextDocument.URI},
			},
			Data: nil,
		},
	}, nil
}

func (b *Backend) ExecuteCommand(ctx context.Context, params *lsp.ExecuteCommandParams) (result interface{}, err error) {
	switch params.Command {
	case "foo":
		uri := lsp.URI(params.Arguments[0].(string))
		b.client.ApplyEdit(
			ctx,
			&lsp.ApplyWorkspaceEditParams{
				Label: "insert foo at the top of the document",
				Edit: lsp.WorkspaceEdit{
					Changes: map[lsp.URI][]lsp.TextEdit{
						uri: {
							{
								Range:   lsp.Range{Start: lsp.Position{0, 0}, End: lsp.Position{0, 0}},
								NewText: "FOO\n",
							},
						},
					},
					DocumentChanges:   []lsp.TextDocumentEdit{},
					ChangeAnnotations: map[lsp.ChangeAnnotationIdentifier]lsp.ChangeAnnotation{},
				},
			},
		)
		return nil, nil
	default:
		panic(fmt.Sprint("unimplemented command:", params.Command))
	}
}

func main() {
	logger, err := zap.NewDevelopment()
	if err != nil {
		panic(err)
	}

	ctx := lsp.WithLogger(context.Background(), logger)

	stdio := struct {
		io.ReadCloser
		io.Writer
	}{
		os.Stdin, os.Stdout,
	}
	conn := jsonrpc2.NewConn(jsonrpc2.NewStream(stdio))

	client := lsp.ClientDispatcher(conn, logger)
	server := &Backend{client}
	handler := lsp.ServerHandler(server, jsonrpc2.MethodNotFoundHandler)

	conn.Go(ctx, handler)
	select {
	case <-ctx.Done():
		fmt.Fprintf(os.Stderr, "context done")
		conn.Close()
	case <-conn.Done():
		fmt.Fprintf(os.Stderr, "jsonrpc conn done")
	}
}

Log

lsp.log snippet

[DEBUG][2022-03-24 15:07:24] .../lua/vim/lsp.lua:962	"LSP[spdx_lsp]"	"client.request"	1	"textDocument/codeAction"	{  context = {    diagnostics = {}  },  range = {    end = <1>{      character = 0,      line = 0    },    start = <table 1>  },  textDocument = {    uri = "file:[redacted path]"  }}	<function 1>	1
[DEBUG][2022-03-24 15:07:24] .../vim/lsp/rpc.lua:347	"rpc.send"	{  id = 2,  jsonrpc = "2.0",  method = "textDocument/codeAction",  params = {    context = {      diagnostics = {}    },    range = {      end = <1>{        character = 0,        line = 0      },      start = <table 1>    },    textDocument = {      uri = "file:[redacted path]"    }  }}
[DEBUG][2022-03-24 15:07:24] .../vim/lsp/rpc.lua:454	"rpc.receive"	{  id = 2,  jsonrpc = "2.0",  result = { {      command = {        arguments = { "file:[redacted path]" },        command = "foo",        title = "Foo"      },      edit = vim.empty_dict(),      kind = "source",      title = "Foo"    } }}
[ERROR][2022-03-24 15:07:24] .../vim/lsp/rpc.lua:420	"rpc"	"[redacted binary path]"	"stderr"	"2022-03-24T15:07:24.997+0100\tDEBUG\tprotocol@v0.12.0/server.go:160\ttextDocument/codeAction\n"
[DEBUG][2022-03-24 15:07:29] .../lua/vim/lsp.lua:962	"LSP[spdx_lsp]"	"client.request"	1	"workspace/executeCommand"	{  arguments = { "file:[redacted path]" },  command = "foo",  title = "Foo"}	<function 1>	1
[DEBUG][2022-03-24 15:07:29] .../vim/lsp/rpc.lua:347	"rpc.send"	{  id = 3,  jsonrpc = "2.0",  method = "workspace/executeCommand",  params = {    arguments = { "file:[redacted path]" },    command = "foo",    title = "Foo"  }}
[ERROR][2022-03-24 15:07:29] .../vim/lsp/rpc.lua:420	"rpc"	"[redacted binary path]"	"stderr"	"2022-03-24T15:07:29.161+0100\tDEBUG\tprotocol@v0.12.0/client.go:371\tcall workspace/applyEdit\n"
[DEBUG][2022-03-24 15:07:29] .../vim/lsp/rpc.lua:454	"rpc.receive"	{  id = 1,  jsonrpc = "2.0",  method = "workspace/applyEdit",  params = {    edit = {      changes = {        ["file:[redacted path]"] = { {            newText = "FOO\n",            range = {              end = {                character = 0,                line = 0              },              start = {                character = 0,                line = 0              }            }          } }      }    },    label = "insert foo at the top of the document"  }}
[DEBUG][2022-03-24 15:07:29] .../vim/lsp/rpc.lua:464	"server_request: callback result"	{  result = {    applied = true  },  status = true}
[DEBUG][2022-03-24 15:07:29] .../vim/lsp/rpc.lua:347	"rpc.send"	{  id = 1,  jsonrpc = "2.0",  result = {    applied = true  }}
[nothing more happens here; if we invoke an action, the client sends the proper data, but the server remains stalled]

[ja-he]

Maybe to ask the alternative question: Is the idea of the library that requests to the client need to be run in separate goroutines?

For notifications of course it's fine, but from what I can tell for requests such as applyEdit no separate goroutine is started and then we block in the request (as I describe above) waiting for a message on a channel that won't be sent until we receive it (i.E. deadlock).

[dan1994]

The log does seem to indicate the response to applyEdit is sent back (last line). FYI, I found a bug[1] in the unmarshalling of the result of applyEdit so it may be related but I'm not sure.
Another possible avenue for debugging this is to try to use lsp.NewServer. This wraps lsp.ServerHandler with a couple of other handlers (CancelHandler, AsyncHandler and ReplyHandler). This is what I've done, and it worked for me.

[1] #38

[ja-he]

That sounds pretty much like it would behave the way I was expecting it to out of the box, thanks!

By now I've settled into the synchronous mode but I'll play with this when I find the time.