Personal Knowledge Agent 🧠

An AI agent that lives on your personal knowledge corpus — ask your past in plain language. Built from scratch with LangChain + LangGraph, starting with a real corpus of 217 journal entries (2025-09 ~ 2026-07).

English | 简体中文

It is two things at once: a knowledge assistant its author actually uses daily (CLI + web UI), and a hands-on curriculum — grown stage by stage, each stage shipping docs, runnable demos, and tests.

What it can do

• 🔍 Ask your past — "Which days last month did I mention insomnia?" The agent picks the right tools itself and searches the full corpus
• 🧠 Semantic retrieval (RAG) — "What was going on when I felt anxious?" Meaning-based recall even when your wording differs from what's written (pgvector + bge-m3)
• 💬 Multi-turn memory — remembers earlier conversation turns; past monthly reviews are auto-injected as background knowledge
• ✍️ Guided writing — it asks questions, you answer (LangGraph interrupt, human-in-the-loop); it follows up if your answers are thin, then produces a draft
• 📊 Monthly retrospective — map-reduce compresses a month of entries into one structured review
• 🔁 Hot-swappable models — DeepSeek deepseek-v4-flash (default) / Agnes; flip one environment variable and it applies globally

Quick start

# 1. Install dependencies
pip install -r requirements.txt

# 2. Configure keys: copy the template and fill in your key
cp .env.example .env
#    Required: DEEPSEEK_API_KEY (https://platform.deepseek.com)
#    Optional: AGNES_API_KEY, SILICONFLOW_API_KEY (for semantic retrieval, free)

# 3. Put your journals here (gitignored, never committed)
#    日记/2026/2026-07/2026-07-03.docx  — "日记" means "journals"; folders by year/month, files named by date

# 4. Run it
python scripts/diary_agent.py        # CLI entry point
python -m streamlit run web/app.py   # web UI entry point

Semantic retrieval (RAG) takes two extra steps:

docker compose up -d                 # start PostgreSQL with pgvector
python scripts/index_diaries.py      # embed & index all journals (takes a few minutes on first run)

How it was built: six incremental stages

This codebase wasn't dumped out in one shot. It grew through six stages, each shipping three deliverables: a written tutorial (docs/), runnable demos (scripts/), and tests (tests/) that pin down behavior with fake models — zero real API calls.

Stage	Topic	Milestone
0	Environment & model factory	Dual providers wired; src/llm.py as the single entry point
1	LangChain basics	One journal entry → structured JSON analysis
2	LangGraph	State graphs, conditional edges, loops, interrupt
3	Tool calling	Hand-written agent loop; the model autonomously queries journals
4	Memory	Checkpointer session memory + map-reduce monthly summaries
5	RAG	Chunking → embedding → pgvector → semantic Q&A
6	Integration	Final agent + CLI + web UI + evaluation

📖 Full tutorials start at docs/README.md (written in Chinese) — each covers concepts, code walkthroughs, exercises, and common pitfalls.

🔬 Engineering deep-dives worth reading:

• docs/08-RAG改进路线.md — three retrieval failure cases pinned down with data (top-k has no relevance threshold; key sentences sink to the bottom of rankings; literal matches invisible to dense retrieval), root-cause analysis, and a hybrid-retrieval + rerank improvement roadmap
• docs/07-代码评审与改进清单.md — self code review and improvement checklist

Common commands

python scripts/parse_diary.py               # analyze the most recent journal entry
python scripts/diary_graph_demo.py          # watch data flow through a LangGraph state machine
python scripts/diary_writer.py              # interactive guided journaling
python scripts/monthly_review.py 2026-07    # generate a monthly retrospective (change the month)
python scripts/agent_with_tools.py          # hand-written agent loop with visible tool calls
python scripts/rag_query.py "question"      # RAG semantic Q&A
python scripts/eval_rag.py                  # run the eval set for baseline scores
python -m pytest tests/ -q                  # full test suite (Docker-dependent cases auto-skip)

Project structure

├── docs/               # six-stage tutorials (the project's core textbook)
├── AGENTS.md           # ground rules for AI coding assistants working on this repo
├── 日记/                # raw journal data ("journals") — gitignored, never committed
├── src/
│   ├── llm.py          # single model entry point get_llm()
│   ├── diary/          # DiaryEntry data model + docx parsing
│   ├── chains/         # LangChain chains + LangGraph graphs
│   ├── tools/          # @tool functions (search / date lookup / stats)
│   ├── memory/         # session memory + monthly summaries
│   ├── rag/            # embedding + pgvector + retrieval-augmented generation
│   └── agent/          # final orchestration + evaluation
├── scripts/            # runnable demo per stage
├── web/app.py          # Streamlit web UI
├── tests/              # tests with fake models — no real API calls
└── docker-compose.yml  # PostgreSQL + pgvector

Privacy & safety

• .env (keys), 日记/ (raw journals), and memory_cache/ (summaries derived from journals) are all excluded via .gitignore — they never enter git
• Keys are read only from .env via os.getenv() — zero hardcoded secrets anywhere
• Tests use fake models / temp files only: they never touch real journals and cost zero API credits
• Journal content is sent only to the LLM API you configure yourself (DeepSeek / Agnes) — that's the product doing its job, nothing else

Tech stack

Python 3.12 · LangChain 1.x · LangGraph 1.x · Streamlit · PostgreSQL + pgvector (Docker) · SiliconFlow bge-m3 (embeddings) · pytest