# CLAUDE.md — PRism ## What is this project? PRism is a Python pipeline that extracts business logic from ~3,500 merged GitHub PRs, enriches them with Jira ticket context, analyzes them using the Anthropic API (Claude Sonnet), and generates comprehensive business documentation organized by domain. The target codebase is 500K+ lines with substantial business logic embedded in implementation code. **Accuracy is the top priority** — cost and speed are secondary concerns. ## Architecture The pipeline runs in sequential phases: ``` Phase 0A: phase0_collect_github.py → Download all merged PRs (metadata + diffs + full files) Phase 0B: phase0_collect_jira.py → Batch-fetch all referenced Jira tickets + parent Stories Phase 0C: phase0_collect_timeline.py → Build chronological timeline + collect direct/non-PR changes Phase 1: phase1_codebase_overview.py → Generate structural map of the codebase (run once) Phase 2: phase2_analyze_prs.py → Per-PR deep analysis: code + Jira → LLM → JSON Phase 2B: phase2b_normalize_domains.py → Canonicalize domain labels Phase 3: phase3_cross_validate.py → Group by domain → reconcile → verify against current code Phase 4: phase4_generate_docs.py → Produce final business documentation per domain ``` Each phase reads from and writes to the `data/` directory. Phases are idempotent and support resume — they skip already-processed items. ## Project structure ``` CLAUDE.md # This file IMPLEMENTATION_PLAN.md # Detailed plan with full code for all phases config.py # Central configuration (env vars, paths, limits) phase0_collect_github.py # GitHub PR collection phase0_collect_jira.py # Jira ticket collection phase0_collect_timeline.py # Timeline build + direct/non-PR change capture phase1_codebase_overview.py # Codebase structural analysis phase2_analyze_prs.py # Main LLM analysis pipeline phase2b_normalize_domains.py # Domain label normalization phase3_cross_validate.py # Cross-PR domain validation phase4_generate_docs.py # Documentation generation run_all.sh # Pipeline runner (all phases) utils/ ├── token_budget.py # Token counting, truncation, budget management └── progress.py # Progress tracking, resume support, logging prompts/ ├── system_pass1.txt # System prompt for Phase 2 (per-PR analysis) ├── system_pass2.txt # System prompt for Phase 3 (cross-validation) └── system_pass3.txt # System prompt for Phase 4 (doc generation) data/ # All pipeline data (gitignored) ├── prs/ # One JSON per PR: {number}.json ├── jira/ # One JSON per ticket: {KEY}.json + _resolved_index.json ├── codebase_overview.md # Phase 1 output ├── pass1_results/ # Phase 2 outputs: {source_key}.json ├── pass2_results/ # Phase 3 outputs: {domain}.json └── docs/ # Phase 4 outputs: {domain}.md + INDEX.md run_all.sh # Runs all phases sequentially requirements.in # Human-managed runtime deps (floor pins) requirements-dev.in # Human-managed dev/test deps (floor pins) requirements.txt # Pinned runtime deps (generated by pip-compile) requirements-dev.txt # Pinned dev/test deps (generated by pip-compile) .env # Secrets (not committed) .env.example # Template for .env .gitignore ``` ## Key technical decisions - **Model:** Claude Sonnet (`claude-sonnet-4-20250514`) via Anthropic API. Do not switch to smaller or open-source models — accuracy on complex code comprehension is the priority. - **Temperature:** Always `0` for all LLM calls. We want deterministic, consistent extraction. - **Context strategy:** Each Phase 2 call includes the codebase overview + PR metadata + Jira context + full file content (not just diffs). This is critical for accuracy. - **Chunking:** PRs with more than `MAX_FILES_PER_CHUNK` files are split by directory so related changes stay together. Each chunk is analyzed separately, then results are merged. - **Resume support:** Every phase checks for already-processed items and skips them. Results are saved incrementally per unit of work, not in bulk at the end. - **Source-aware provenance:** Persisted Phase 2 results carry `source_id`, `source_kind`, `source_pr_number`, `source_sha`, and `timeline_order`. Phase 3 rule provenance must use `introduced_in_source` / `last_modified_in_source` / `removed_in_source`, not PR-only numeric references. - **Jira trust hierarchy:** Code > PR review comments > PR description > Jira acceptance criteria > Jira description > Jira comments. When code and Jira conflict, document both and flag the discrepancy. ## Commands ```bash # Install runtime dependencies pip install -r requirements.txt # Install runtime + dev/test dependencies pip install -r requirements-dev.txt # To update pinned versions after editing .in files: # pip-compile requirements.in -o requirements.txt --strip-extras # pip-compile requirements-dev.in -o requirements-dev.txt --strip-extras # Set up environment cp .env.example .env # Edit .env with your tokens # Run individual phases python phase0_collect_github.py python phase0_collect_jira.py python phase0_collect_timeline.py python phase1_codebase_overview.py python phase2_analyze_prs.py # main work, takes hours python phase2_analyze_prs.py --start 500 --end 1000 # process a range python phase2b_normalize_domains.py python phase3_cross_validate.py python phase4_generate_docs.py # Run everything bash run_all.sh ``` ## Development guidelines ### Code style - Python 3.11+. Use type hints everywhere (including `str | None` union syntax, not `Optional[str]`). - No classes unless they genuinely manage state (like `ProgressTracker`). Prefer plain functions. - Every module has a docstring at the top explaining what it does, what it reads from, what it writes to, and how to run it. - Use `pathlib` or `os.path` consistently — the project currently uses `os.path`. ### Error handling - All external API calls (GitHub, Jira, Anthropic) must use `tenacity` retry decorators with exponential backoff. - Never let a single PR failure kill the pipeline. Catch exceptions per-PR, log the error, save an `{pr_number}_error.json` file, and continue. - JSON parse failures from LLM responses should save the raw text for debugging, not silently discard. ### Data flow rules - Phase outputs are JSON files, one per unit of work (one per source item in Phase 2, one per Jira ticket, one per domain). - Never hold all data in memory. Process one item at a time, write to disk, move on. - The `data/` directory is the single source of truth between phases. Phases communicate only through files, not in-memory state. ### Prompt engineering rules - System prompts live in `prompts/*.txt`, not hardcoded in Python files. - The output format is always JSON with a documented schema. The system prompt specifies the exact JSON structure. - When modifying prompts, be explicit about what to extract and what to ignore. Vague instructions produce vague results across 3,500 PRs. - Always include the trust hierarchy (code > PR comments > PR description > Jira) in any prompt that receives Jira context. ### Adding new functionality - If adding a new phase, follow the naming pattern: `phaseN_descriptive_name.py`. - New prompts go in `prompts/`. New utilities go in `utils/`. - Any new configuration goes in `config.py` with a sensible default and a comment explaining what it controls. - New runtime dependencies go in `requirements.in` (floor pins). New dev/test dependencies go in `requirements-dev.in`. Then regenerate pinned files with `pip-compile`. ### Testing and validation - After Phase 2 completes, spot-check 20–30 random results across different domains. Read the PR yourself, then compare with the LLM's extraction. This catches prompt quality issues early. - Phase 3 cross-validation is itself a quality check. If it finds many contradictions, the Phase 2 prompt may need tuning. - After Phase 4, do a final validation: pick a domain doc, read the current source files, and check for rules in the code that are missing from the documentation. ## Configuration All configuration lives in `config.py` and reads from environment variables via `.env`. Key settings: | Variable | Purpose | |---|---| | `GITHUB_TOKEN` | GitHub personal access token with repo read access | | `GITHUB_REPO` | Target repository in `owner/repo` format | | `LOCAL_REPO_PATH` | Absolute path to a local git clone of the repo (used for `git show` to get file content at specific commits) | | `PRIMARY_BRANCH_NAMES` | Mainline branch aliases ordered oldest→newest (example: `master > main`) | | `JIRA_SERVER` | Jira instance URL | | `JIRA_ENDPOINT` | Alias for `JIRA_SERVER` | | `JIRA_BROWSE_ENDPOINT` | Jira browse endpoint used for link-based key extraction | | `JIRA_EMAIL` / `JIRA_API_TOKEN` | Jira authentication | | `ANTHROPIC_API_KEY` | Anthropic API key | | `JIRA_TICKET_PATTERN` | Regex for Jira ticket keys in PR titles (default: `[A-Z]{2,10}-\d{1,6}`) | | `CODE_EXTENSIONS` | Set of file extensions to analyze (skip lock files, images, etc.) | | `SKIP_PATTERNS` | File path patterns to always exclude from analysis | ### Settings that may need tuning per project - `CODE_EXTENSIONS`: Add project-specific extensions (`.erb`, `.hbs`, `.blade.php`, etc.) if business logic lives in templates. - `SKIP_PATTERNS`: Review whether `migrations/` should be included — some projects encode business rules in migrations. - `JIRA_TICKET_PATTERN`: Adjust if your project uses non-standard ticket key formats. - `MAX_FILES_PER_CHUNK`: Increase if most PRs are large and you want fewer chunks (costs more per call but better context). - Jira custom field IDs for acceptance criteria in `phase0_collect_jira.py`: These vary per Jira instance. Find yours via `GET /rest/api/2/field`. ## Important context for implementation - **Only merged PRs are analyzed.** The GitHub collection phase filters on `state=closed` + `pr.merged == True`. Closed-but-not-merged PRs are excluded. - **Jira ticket hierarchy matters.** Many PR titles contain Jira keys like `SBR-219`. These tickets may be subtasks. The real business logic is almost always in the parent Story ticket, not the subtask. Epic tickets are too high-level and must not be used as primary business logic sources. - **Code is the ultimate source of truth.** Jira describes intent. Code describes reality. When they conflict, document both and flag the discrepancy. Some decisions happen outside Jira entirely. - **The codebase overview is included in every Phase 2 call.** This gives the LLM architectural context when reading individual PRs. It's generated once in Phase 1 and reused. ## The implementation plan The file `IMPLEMENTATION_PLAN.md` in this repository contains the full implementation plan with complete code for every phase, every utility module, every prompt, and every configuration option. **Start there.** It includes rationale for every design decision.