Rewrite Qwen analyzer

[mike1858]

Closes #190.

Summary by CodeRabbit

Release Notes

• New Features

  • Added support for Qwen Code session parsing from JSONL, with more accurate token, usage, and cost calculations (including cached input handling).
  • Improved session hashing and message extraction, including tool call statistics and better handling of timestamps and billable records.
  • Expanded data discovery to scan ~/.qwen/projects/*/chats/ for both JSONL and JSON, with improved deduplication.

• Tests

  • Added JSONL fixture and new tests validating glob patterns, parsing results, message roles, and computed usage/cost fields.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: fede14fe-91f1-4f0c-91c0-05d8e0d53607

📥 Commits

Reviewing files that changed from the base of the PR and between 2ea825e and 5d507af.

📒 Files selected for processing (1)

• src/analyzers/qwen_code.rs

🚧 Files skipped from review as they are similar to previous changes (1)

• src/analyzers/qwen_code.rs

---

📝 Walkthrough

Walkthrough

The Qwen Code analyzer is refactored to parse JSONL session logs instead of a JSON session-blob format. New structs model JSONL records, parts, function calls, and usage metadata. Cost calculation, tool stats, path detection, glob patterns, discovery, and deduplication are all updated accordingly. A public parse_jsonl_session_file function is added, and tests plus a JSONL fixture are introduced.

Changes

Qwen Code JSONL Migration

Layer / File(s)	Summary
JSONL record model and parsing helpers src/analyzers/qwen_code.rs	Defines JsonlRecord, JsonlMessage, JsonlPart, FunctionCall, and UsageMetadata structs with tolerant UTC timestamp deserialization, visible-text extraction, tool-call collection, and per-tool stat computation from functionCall parts.
Tool stats, cost calculation, and path predicate src/analyzers/qwen_code.rs	Reworks per-tool stat estimation (read/edit/write/list_directory with add/delete line splits), adds calculate_qwen_cost using usageMetadata fields (prompt - cached for non-cached input), introduces is_qwen_code_chat_path predicate, and adds <session_context> filtering and hash computation.
JSONL session parser: ConversationMessage construction src/analyzers/qwen_code.rs	Implements parse_jsonl_session_file: reads lines tolerantly, computes global_hash/conversation_hash (incorporating uuid when present), builds session name from first non-context user text, and constructs ConversationMessage entries for user/assistant records with usageMetadata-derived token counts; ignores other record types.
Discovery, glob patterns, and deduplication wiring src/analyzers/qwen_code.rs	Updates glob patterns to ~/.qwen/projects/*/chats/*.jsonl and *.json, rewires WalkDir discovery to filter via is_qwen_code_chat_path, wires availability and parallel parsing to parse_jsonl_session_file, deduplicates by global_hash, and delegates is_valid_data_path to is_qwen_code_chat_path.
Tests and JSONL fixture src/analyzers/tests/qwen_code.rs, src/analyzers/tests/source_data/qwen_code.jsonl	Updates imports and smoke tests; adds test_qwen_code_glob_patterns_use_projects_dir and test_parse_sample_qwen_code_session with the new fixture asserting message roles, unique global_hash values, per-message token fields, and non-zero aggregated totals.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~50 minutes

Poem

🐇 Hop, hop, through the JSONL stream,
Each line a record, each record a dream.
No more blob, just rows in a file,
The rabbit parses them all with a smile.
Cached tokens counted, tool stats aligned,
A cleaner Qwen Code session defined! ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 30.77% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'Rewrite Qwen analyzer' clearly describes the main change—refactoring the Qwen Code analyzer from single-JSON to JSONL format with improved token usage calculation.
Linked Issues check	✅ Passed	The PR successfully addresses issue #190 by refactoring the Qwen Code analyzer to correctly parse JSONL session files and compute token usage from usageMetadata fields, resolving the token usage display bug.
Out of Scope Changes check	✅ Passed	All changes are directly related to fixing Qwen Code token usage: JSONL parsing implementation, cost calculation updates, test coverage, and test fixtures remain within scope.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix-qwen-code

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/analyzers/qwen_code.rs`:
- Around line 250-260: The is_qwen_code_chat_path function currently accepts
files with any ancestor named "chats", which is too permissive and allows nested
files like .../chats/archive/foo.jsonl to pass validation. Instead of using
ancestors().skip(1).any() to check if "chats" exists anywhere in the path
hierarchy, verify that the direct parent directory of the file is named "chats"
by checking path.parent() and its file_name(). This ensures only files directly
under a "chats" directory are accepted, matching the intended glob pattern of
projects/*/chats/*.
- Around line 209-213: In the edit statistics handling block within qwen_code.rs
(lines 209-213), the current code uses max operations to set lines_added and
lines_deleted, which replaces existing values from write_file operations instead
of accumulating them. Change the logic to add (accumulate) the edit-derived
estimates to the existing stats.lines_added and stats.lines_deleted values
rather than taking the maximum. This ensures that when a turn contains both
write_file and edit/replace operations, both contributions are retained in the
final statistics.
- Around line 276-291: The fallback hash for UUID-less QwenCodeRecord entries
uses only file path and timestamp, causing collisions when multiple records
share the same timestamp. To fix this, modify the loop that iterates through
content.lines() to use enumerate() to capture the line index, then update the
None branch of the global_hash match statement to include the line index as a
stable per-line discriminator in the hash formula (e.g., combine file_path_str,
the current line index, and the timestamp when uuid is absent).
- Around line 237-247: In the calculate_qwen_cost function, the
thoughtsTokenCount (usage.thoughts) is incorrectly being billed at the input
token rate. Remove usage.thoughts from the total_input_tokens calculation on
line 241 so it only includes non_cached_input, then modify the
calculate_output_cost call on line 244 to include both usage.candidates and
usage.thoughts combined, since Qwen bills reasoning tokens at the output rate
not the input rate.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 01a3f1da-b8f9-4329-b0f1-698b50e5b580

📥 Commits

Reviewing files that changed from the base of the PR and between 6ec3e68 and 2ea825e.

📒 Files selected for processing (3)

• src/analyzers/qwen_code.rs
• src/analyzers/tests/qwen_code.rs
• src/analyzers/tests/source_data/qwen_code.jsonl