Skip to content

Log corruption and content interleaving when using Logger and Async TSVLogger simultaneously #115

@youjiajia

Description

@youjiajia

Library Version: v1.0.121

Description: I encountered a log corruption issue when using a synchronous Logger and an asynchronous TSVLogger (wrapped with AsyncWriter) in the same test case. Even though the two loggers are configured to write to different files, the output in the asynchronous log file becomes interleaved and corrupted.

Specifically, fragments of the JSON format from the synchronous logger (e.g., {"time":) appear inside the TSV logger's output file, and the messages are truncated or mixed together.

Code:

func TestSyncAndAsyncLoggerConsistency(t *testing.T) {
	syncFile := "consistency_sync.log"
	asyncFile := "consistency_async.log"

	// 1. Define a synchronous JSON logger
	syncLogger := Logger{
		Writer: &FileWriter{Filename: syncFile},
	}

	// 2. Define an asynchronous TSV logger
	asyncLogger := TSVLogger{
		Writer: &AsyncWriter{
			ChannelSize:   4096,
			DisableWritev: false,
			DiscardOnFull: false,
			Writer:        &FileWriter{Filename: asyncFile},
		},
	}

	for i := 0; i < 1000; i++ {
		msg := "hello file writer"
		// Write JSON log
		syncLogger.Info().Str("line", msg).Msg("")
		// Write TSV log
		asyncLogger.New().Str(msg).Msg()
	}

	if err := syncLogger.Writer.(io.Closer).Close(); err != nil {
		t.Fatalf("close sync logger: %+v", err)
	}
	if err := asyncLogger.Writer.(io.Closer).Close(); err != nil {
		t.Fatalf("close async logger: %+v", err)
	}

	// Helper to check content consistency
	LinesEqual := func(path string, content string) error {
		f, err := os.Open(path)
		if err != nil {
			return err
		}
		defer f.Close()

		scanner := bufio.NewScanner(f)
		for scanner.Scan() {
			// In this case, we expect the async log to only contain "hello file writer"
			if scanner.Text() != content {
				return errors.New("content mismatch: " + scanner.Text() + " != " + content)
			}
		}
		return scanner.Err()
	}

	err := LinesEqual(asyncFile, "hello file writer")
	if err != nil {
		t.Fatalf("read async file: %+v", err)
	}
}

Actual Output in consistency_async.log: The log file contains corrupted lines like:

hello file writer
hello file writer
{"time":le writer
{"time":le writer
hello file writer
{"time":le writer

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions