Use token usage as threshold for summarization#17
Conversation
| If the path doesn't exist, this function returns "File not found". | ||
| """ | ||
| path: str = Field(description="The relative path of the file on the VFS. IMPORTANT: Do NOT include a leading `./` it is implied") | ||
| range: FileRange | None = Field(description="If set, (start, end) indicates to return lines starting from line `start` (lines are 1 indexed) until `end` (exclusive). If unset, the entire file is returned.", default=None) |
There was a problem hiding this comment.
This ... shouldn't be removed. The agent has the option to read the entire file if it needs to. There are genuine reasons to avoid splatting the whole file into the context, and to let the agent read only selected parts of said file. If you're seeing an agent trying to read files incrementally with ranges, that should be addressed at the prompt level not by forcing the agent to read entire files every time.
There was a problem hiding this comment.
To wit, we can certainly copy some of the instructions used from claude code's system prompt:
Tool description body:
▎ Reads a file from the local filesystem. You can access any file directly by using this tool.
▎ Assume this tool is able to read all files on the machine. If the User provides a path to a file assume that path is valid. It is okay to read a file that does not exist; an error will be returned.
▎
▎ Usage:
▎ - The file_path parameter must be an absolute path, not a relative path
▎ - By default, it reads up to 2000 lines starting from the beginning of the file
▎ - When you already know which part of the file you need, only read that part. This can be important for larger files.
▎ - Results are returned using cat -n format, with line numbers starting at 1
▎ - This tool allows Claude Code to read images (eg PNG, JPG, etc). When reading an image file the contents are presented visually as Claude Code is a multimodal LLM.
▎ - This tool can read PDF files (.pdf). For large PDFs (more than 10 pages), you MUST provide the pages parameter to read specific page ranges (e.g., pages: "1-5"). Reading a large PDF without the pages parameter will fail. Maximum 20
▎ pages per request.
▎ - This tool can read Jupyter notebooks (.ipynb files) and returns all cells with their outputs, combining code, text, and visualizations.
▎ - This tool can only read files, not directories. To list files in a directory, use the registered shell tool.
▎ - You will regularly be asked to read screenshots. If the user provides a path to a screenshot, ALWAYS use this tool to view the file at the path. This tool will work with all temporary file paths.
▎ - If you read a file that exists but has empty contents you will receive a system reminder warning in place of file contents.
▎ - Do NOT re-read a file you just edited to verify — Edit/Write would have errored if the change failed, and the harness tracks file state for you.
Parameter descriptions (the key bit):
- offset: "The line number to start reading from. Only provide if the file is too large to read at once"
- limit: "The number of lines to read. Only provide if the file is too large to read at once."
We likely need to emphasize that the range parameter should only be used to select a known range of the file when that range is already known; present it as an optimization as opposed to the "happy path" of passing in null.
| to_ret._summary_config = config | ||
| return to_ret | ||
|
|
||
| def with_default_summarizer(self, *, max_messages: int = 20, enabled: bool = True) -> "Builder[_BStateT, _BContextT, _BInputT]": |
There was a problem hiding this comment.
why not let the user configure the token threshold here?
There was a problem hiding this comment.
Why yes? the threshold is a function of the model being used, not of how long we expect the agent to run.
|
|
||
| if summary_config is not None: | ||
| model_name = getattr(unbound_llm, "model", "?") | ||
| model_name = getattr(unbound_llm, "model", "") |
There was a problem hiding this comment.
do I not understand python? Can we not use None as the default here? I guess it doesn't matter, but I'd still prefer to use the type system to our advantage if we can.
There was a problem hiding this comment.
You could, but then to keep pyright happy you'd need to make the function on the next line accept str | None which seemed wrong to me
2 participants
A few changes to reduce the number 'get_file' tool usages during a run.