1
0
Fork 0
mirror of https://github.com/SeriousBug/dotfiles synced 2026-08-17 06:46:16 -05:00

Compare commits

..

No commits in common. "1aea5cf6599a685e866571c7bf1b1aa93770a9af" and "a0c3471053f5ab985bba88f48bc555e4b9b90d4b" have entirely different histories.

19 changed files with 19 additions and 1297 deletions

View file

@ -65,7 +65,6 @@ depends = []
[claude.files] [claude.files]
"claude/CLAUDE.md" = "~/.claude/CLAUDE.md" "claude/CLAUDE.md" = "~/.claude/CLAUDE.md"
"claude/settings.json" = "~/.claude/settings.json"
[claude.variables] [claude.variables]

7
.gitignore vendored
View file

@ -7,10 +7,3 @@
# Mudpuppy skills (not tracked) # Mudpuppy skills (not tracked)
claude/skills/mudpuppy-implementation-review/ claude/skills/mudpuppy-implementation-review/
claude/skills/mudpuppy-pr-review/ claude/skills/mudpuppy-pr-review/
# Python bytecode cache
__pycache__/
*.pyc
# htop runtime history
htop/htop_history

View file

@ -1,9 +1,5 @@
# Dotfiles Configuration Notes # Dotfiles Configuration Notes
## Commit Directly to Main
Commit directly to `main` in this repo. It reflects the current state of my machine, so feature branches don't make sense here. Do not create branches or open PRs for changes.
## Dotter Template Handling ## Dotter Template Handling
Dotter automatically treats files with `{{` as templates. For files that contain `{{` in their code (like Lua tables), disable templating by setting `type = "symbolic"` in `.dotter/global.toml`. Dotter automatically treats files with `{{` as templates. For files that contain `{{` in their code (like Lua tables), disable templating by setting `type = "symbolic"` in `.dotter/global.toml`.

View file

@ -1,14 +1,5 @@
# Personal Claude Code Instructions # Personal Claude Code Instructions
## Do not modify my machine without permission
**Never install software, packages, or tools on my machine without explicit permission** — no `brew install`, `npm install -g`, `pip install`, `apt`, downloading binaries, or anything that changes global system state. This is absolutely disallowed. If a task needs a tool I don't have, stop and ask first, or find a way that doesn't touch my machine (e.g. run it in Docker — if Docker is not available then ask me).
## Agents
If work is parallel or cleanly scoped out, consider handing off to subagents to preserve context.
Do **not** use fork subagents. Only separate agents with clear scope or a clear goal.
## Notifying me / sharing images via Discord ## Notifying me / sharing images via Discord
The `discord-send` CLI is on my PATH. Use it whenever I ask you to notify me when something is done, or to show me an image or video — send it to my Discord and I'll see it there. The `discord-send` CLI is on my PATH. Use it whenever I ask you to notify me when something is done, or to show me an image or video — send it to my Discord and I'll see it there.
@ -22,14 +13,6 @@ discord-send --attach ./screenshot.png 'Here is the rendered page'
When asked to respond to reviews in a PR, check the PR description and all comments. Check out the PR branch. Decide if any changes are required or not, make changes if needed, commit and push once done. Always push the PR branch if responding to a PR, do not push to a different branch. When asked to respond to reviews in a PR, check the PR description and all comments. Check out the PR branch. Decide if any changes are required or not, make changes if needed, commit and push once done. Always push the PR branch if responding to a PR, do not push to a different branch.
## Commits
Commit often. It's better to commit things than to leave changes hanging for a long time. You don't have to commit after every change, but once you're done working just go ahead and commit.
## Comments in code
Only comment what is necessary to explain **why** the code is doing something. Do not write comments for **what** the code is doing, that should be obvious from the code itself.
Do not write comments for anything obvious, anything that an expert can be expected to know, or anything already documented elsewhere.
Comments are permanent part of the code! Only comment what should be permanently stored along with the code, moment to moment coding decisions, what old versions of the code were before refactors etc. do not belong in comments.
## Writing Style ## Writing Style
Following rules are for your writing. Follow these when responding to the user, when writing documentation and code comments, and when writing commit messages or PR descriptions. Following rules are for your writing. Follow these when responding to the user, when writing documentation and code comments, and when writing commit messages or PR descriptions.
@ -45,13 +28,11 @@ Following rules are for your writing. Follow these when responding to the user,
- Do not use em-dashes. No "It's not X, it's Y". Avoid overusing intricate, underscore, highlight, showcase, journey to, step into. Do not use "You're absolutely right!". - Do not use em-dashes. No "It's not X, it's Y". Avoid overusing intricate, underscore, highlight, showcase, journey to, step into. Do not use "You're absolutely right!".
- Avoid sentence fragments for emphasis. Bad: "Not X. Not Y. Just Z." - Avoid sentence fragments for emphasis. Bad: "Not X. Not Y. Just Z."
- Do use emphasis when necessary: "This is a destructive operation. All data will be lost. Should I proceed?" - Do use emphasis when necessary: "This is a destructive operation. All data will be lost. Should I proceed?"
- Avoid flowery language like "no ceremony", "this is more than X", "you're right to poke it", "this is load bearing", "dissolves your original concern"
Examples: Examples:
- Bad: Sure! I'd be happy to help you with that. The issue you're experiencing is likely caused by a problem with the authentication middleware. I will investigate this problem now. - Bad: Sure! I'd be happy to help you with that. The issue you're experiencing is likely caused by a problem with the authentication middleware. I will investigate this problem now.
- Bad: Not a bug. Not a feature. A fundamental design flaw. - Bad: Not a bug. Not a feature. A fundamental design flaw.
- Bad: It's worth noting that this approach has limitations. - Bad: It's worth noting that this approach has limitations.
- Bad: Just implementation, no ceremony.
- Good: Sure. Bug may be in auth middleware. I'll check. - Good: Sure. Bug may be in auth middleware. I'll check.
- Good: This is a design flaw. - Good: This is a design flaw.
- Good: This approach has limitations. - Good: This approach has limitations.

View file

@ -1,79 +0,0 @@
{
"attribution": {
"sessionUrl": false
},
"includeCoAuthoredBy": false,
"permissions": {
"allow": [
"mcp__claude_ai_Willet__list_organizations",
"mcp__claude_ai_Willet__list_organization_members",
"mcp__claude_ai_Willet__list_projects",
"mcp__claude_ai_Willet__list_project_members",
"mcp__claude_ai_Willet__list_tasks",
"mcp__claude_ai_Willet__list_tags",
"mcp__claude_ai_Willet__search_tasks",
"mcp__claude_ai_Willet__get_task",
"mcp__claude_ai_Willet__get_task_graph",
"mcp__claude_ai_Willet__get_project_stats",
"mcp__claude_ai_Willet__render_dependency_graph",
"mcp__claude_ai_Willet__render_task_board",
"mcp__claude_ai_Willet__create_task",
"mcp__claude_ai_Willet__update_task",
"mcp__claude_ai_Willet__complete_task",
"mcp__claude_ai_Willet__cancel_task",
"mcp__claude_ai_Willet__reopen_task",
"mcp__claude_ai_Willet__start_task",
"mcp__claude_ai_Willet__link_tasks",
"mcp__claude_ai_Willet__unlink_tasks",
"mcp__claude_ai_Willet__add_comment"
],
"defaultMode": "auto"
},
"worktree": {
"baseRef": "fresh"
},
"enableWorkflows": false,
"workflowKeywordTriggerEnabled": false,
"statusLine": {
"type": "command",
"command": "~/.claude/ccline/ccline",
"padding": 0
},
"enabledPlugins": {
"frontend-design@claude-plugins-official": true,
"honeycomb@honeycomb-plugins": true,
"caveman@caveman": false
},
"extraKnownMarketplaces": {
"mcp-apps": {
"source": {
"source": "github",
"repo": "modelcontextprotocol/ext-apps"
}
},
"honeycomb-plugins": {
"source": {
"source": "github",
"repo": "honeycombio/agent-skill"
}
},
"caveman": {
"source": {
"source": "github",
"repo": "JuliusBrussee/caveman"
}
}
},
"alwaysThinkingEnabled": true,
"effortLevel": "medium",
"promptSuggestionEnabled": false,
"awaySummaryEnabled": false,
"showClearContextOnPlanAccept": true,
"tui": "fullscreen",
"autoMemoryEnabled": false,
"editorMode": "vim",
"fileCheckpointingEnabled": false,
"remoteControlAtStartup": true,
"agentPushNotifEnabled": true,
"skipAutoPermissionPrompt": true
}

View file

@ -16,13 +16,8 @@ The helper script lives at `~/.claude/skills/cc-compact/compact_session.py`.
## Step 1: Resolve the session ## Step 1: Resolve the session
The argument is one of three forms: The argument is one of two forms:
- **Nothing** — no id or title given. This is the common case: the user ran
`/clear` and then `cc-compact` to reload the session they just cleared. Run
the script with no selector; it auto-picks the most recently active session
in the current project, **excluding this session** (the fresh one `/clear`
created). This emulates Claude Code's built-in `/compact`.
- **A session id** — when invoked like `/resume claude --resume <id>` or - **A session id** — when invoked like `/resume claude --resume <id>` or
`cc-compact <id>`, the user already gave you the UUID. Pass it as `--id`. `cc-compact <id>`, the user already gave you the UUID. Pass it as `--id`.
- **A session name / title** — free text. Pass it as `--title`; the script - **A session name / title** — free text. Pass it as `--title`; the script
@ -32,9 +27,6 @@ The argument is one of three forms:
Run the script exactly once with the matching form: Run the script exactly once with the matching form:
```sh ```sh
# Latest session before the /clear (default, no selector):
python3 ~/.claude/skills/cc-compact/compact_session.py
# By id: # By id:
python3 ~/.claude/skills/cc-compact/compact_session.py --id <session-uuid> python3 ~/.claude/skills/cc-compact/compact_session.py --id <session-uuid>
@ -44,8 +36,7 @@ python3 ~/.claude/skills/cc-compact/compact_session.py --title "the session name
## Step 2: Read the output ## Step 2: Read the output
Everything is wrapped in a `<compacted-session-chain>` element containing one or It prints a bounded, XML-tagged report:
more `<session depth="N">` blocks. Each block is a bounded, XML-tagged report:
- header: project, git branch, time span, record/prompt counts - header: project, git branch, time span, record/prompt counts
- the first few exchanges (user prompt + the agent's reply) — the original intent - the first few exchanges (user prompt + the agent's reply) — the original intent
@ -55,14 +46,6 @@ more `<session depth="N">` blocks. Each block is a bounded, XML-tagged report:
- the last several tool calls — what the agent was physically doing last - the last several tool calls — what the agent was physically doing last
- the final assistant message — what it was saying / waiting on last - the final assistant message — what it was saying / waiting on last
`depth="0"` (`role="requested"`) is the session you asked for. If that session
was itself resumed from an earlier one via `/clear` → cc-compact, the script
follows that lineage backwards and emits each older session at `depth="1"`, `2`,
… (`role="ancestor"`), summarized more tightly at each step so the total stays
bounded. It walks up to `--max-depth` ancestors (default 10) and guards against
cycles. Read the deeper blocks as fading background: the further back, the
terser. You do not need to do anything to trigger this — it happens on its own.
## Step 3: Pick up the work ## Step 3: Pick up the work
The output is context for *you* — treat it like a resumed session, not something The output is context for *you* — treat it like a resumed session, not something

View file

@ -18,10 +18,6 @@ Session resolution (pick one):
--id UUID find <UUID>.jsonl under the projects dir --id UUID find <UUID>.jsonl under the projects dir
--title TEXT find the session whose ai-title contains TEXT --title TEXT find the session whose ai-title contains TEXT
(case-insensitive substring; newest match wins) (case-insensitive substring; newest match wins)
--latest the most recently active session in the current project,
excluding the caller's own session — this is the default
when no selector is given, so `cc-compact` right after
`/clear` picks up the session you just cleared
Sessions live at: ~/.claude/projects/<encoded-cwd>/<session-id>.jsonl Sessions live at: ~/.claude/projects/<encoded-cwd>/<session-id>.jsonl
""" """
@ -31,7 +27,6 @@ import glob
import json import json
import os import os
import random import random
import re
import sys import sys
PROJECTS_DIR = os.path.expanduser("~/.claude/projects") PROJECTS_DIR = os.path.expanduser("~/.claude/projects")
@ -89,87 +84,7 @@ def is_genuine_prompt(rec):
return not s.startswith(skip_prefixes) return not s.startswith(skip_prefixes)
def current_session():
"""(project_dir, session_id) for the session invoking this script.
Claude Code exports CLAUDE_CODE_SESSION_ID; its log lives at
<project_dir>/<id>.jsonl. Fall back to the encoded cwd when the env var is
missing (script run by hand outside a session). Either value may be None."""
sid = os.environ.get("CLAUDE_CODE_SESSION_ID")
if sid:
matches = glob.glob(os.path.join(PROJECTS_DIR, "**", f"{sid}.jsonl"), recursive=True)
if matches:
return os.path.dirname(matches[0]), sid
# Claude Code encodes the cwd into the project dir name by replacing every
# non-alphanumeric char with a dash (/Users/kaan/Code/Veery -> -Users-kaan-Code-Veery).
encoded = re.sub(r"[^A-Za-z0-9]", "-", os.getcwd())
cand = os.path.join(PROJECTS_DIR, encoded)
return (cand if os.path.isdir(cand) else None), sid
def resolve_latest():
"""Newest session log in the current project, excluding the caller's own
session, so compacting right after `/clear` lands on the just-cleared one."""
project_dir, sid = current_session()
if project_dir:
pool = glob.glob(os.path.join(project_dir, "*.jsonl"))
else:
pool = glob.glob(os.path.join(PROJECTS_DIR, "**", "*.jsonl"), recursive=True)
if sid:
pool = [p for p in pool if os.path.basename(p) != f"{sid}.jsonl"]
if not pool:
sys.exit("No previous session found to compact in this project")
latest = max(pool, key=os.path.getmtime)
sys.stderr.write(f"Auto-selected latest session: {latest}\n")
return latest
# A compacted session's log records the cc-compact run it did on its own
# predecessor: the helper command (carrying --id/--file) and the report it
# printed (whose <file> tag and "Auto-selected" line hold the resolved absolute
# path). Following those references walks the /clear -> cc-compact lineage back.
_ANCESTOR_PATH_RES = [
re.compile(r"Auto-selected latest session:\s*(\S+\.jsonl)"),
re.compile(r"<file>([^<]+\.jsonl)</file>"),
re.compile(r"compact_session\.py[^\n\"]*?--file[= ]+(\S+\.jsonl)"),
]
_ANCESTOR_ID_RE = re.compile(r"compact_session\.py[^\n\"]*?--id[= ]+([0-9a-fA-F-]{36})")
def find_ancestor(path, visited):
"""Return the session log `path` was compacted from, or None.
Scans for references to another session and returns the first that resolves
to an existing file not already in `visited` (guards against cycles)."""
self_real = os.path.realpath(path)
try:
fh = open(path, encoding="utf-8", errors="replace")
except OSError:
return None
with fh:
for line in fh:
if "compact_session.py" not in line and "<file>" not in line and "Auto-selected" not in line:
continue
candidates = []
for pat in _ANCESTOR_PATH_RES:
candidates += pat.findall(line)
for sid in _ANCESTOR_ID_RE.findall(line):
candidates += glob.glob(os.path.join(PROJECTS_DIR, "**", f"{sid}.jsonl"), recursive=True)
for cand in candidates:
cand = os.path.expanduser(cand)
if not os.path.isfile(cand):
continue
real = os.path.realpath(cand)
if real == self_real or real in visited:
continue
return cand
return None
def resolve_file(args): def resolve_file(args):
if args.latest:
return resolve_latest()
if args.file: if args.file:
return os.path.expanduser(args.file) return os.path.expanduser(args.file)
@ -216,83 +131,20 @@ def resolve_file(args):
def main(): def main():
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
g = ap.add_mutually_exclusive_group() g = ap.add_mutually_exclusive_group(required=True)
g.add_argument("--file", help="path to a session .jsonl") g.add_argument("--file", help="path to a session .jsonl")
g.add_argument("--id", help="session UUID") g.add_argument("--id", help="session UUID")
g.add_argument("--title", help="substring of the session's ai-title") g.add_argument("--title", help="substring of the session's ai-title")
g.add_argument("--latest", action="store_true",
help="newest session in the current project, excluding the caller's own "
"(the default when no selector is given)")
ap.add_argument("--first", type=int, default=5, help="how many opening exchanges") ap.add_argument("--first", type=int, default=5, help="how many opening exchanges")
ap.add_argument("--last", type=int, default=10, help="how many closing exchanges") ap.add_argument("--last", type=int, default=10, help="how many closing exchanges")
ap.add_argument("--middle", type=int, default=5, help="how many exchanges randomly sampled from the middle") ap.add_argument("--middle", type=int, default=5, help="how many exchanges randomly sampled from the middle")
ap.add_argument("--top-files", type=int, default=10, help="how many most-edited files") ap.add_argument("--top-files", type=int, default=10, help="how many most-edited files")
ap.add_argument("--maxlen", type=int, default=800, help="max chars per quoted message") ap.add_argument("--maxlen", type=int, default=800, help="max chars per quoted message")
ap.add_argument("--seed", type=int, default=1, help="seed for the middle random sample") ap.add_argument("--seed", type=int, default=1, help="seed for the middle random sample")
ap.add_argument("--max-depth", type=int, default=10,
help="how many ancestor sessions to follow back through the "
"/clear -> cc-compact chain (0 = just this session)")
ap.add_argument("--decay", type=float, default=0.6,
help="per-depth shrink factor for every limit (tighter summaries deeper in the chain)")
args = ap.parse_args() args = ap.parse_args()
if not (args.file or args.id or args.title):
args.latest = True
path = resolve_file(args) path = resolve_file(args)
out = sys.stdout.write
# Walk the lineage: the requested session at depth 0, then each older
# session it was compacted from, summarized ever more tightly.
visited = {os.path.realpath(path)}
chain = [path]
cur = path
for _ in range(max(0, args.max_depth)):
anc = find_ancestor(cur, visited)
if not anc:
break
chain.append(anc)
visited.add(os.path.realpath(anc))
cur = anc
out(f'<compacted-session-chain sessions="{len(chain)}" '
'note="depth 0 is the session you asked for; deeper entries are older '
'sessions it was compacted from, summarized more tightly">\n')
for depth, p in enumerate(chain):
params = scale_params(args, depth)
summarize(p, depth, params, out)
out("</compacted-session-chain>\n")
# Floors keep even the deepest summary useful without letting it grow.
_FLOORS = {"first": 1, "last": 2, "middle": 0, "top_files": 3, "maxlen": 200, "final": 300, "tools": 3}
_BASE_FINAL = 2000 # final-agent-message clip at depth 0
_BASE_TOOLS = 8 # last-tool-calls shown at depth 0
def scale_params(args, depth):
"""Depth-0 uses the CLI limits; each deeper level multiplies every limit by
args.decay ** depth, floored so summaries stay non-empty."""
f = args.decay ** depth
def s(base, floor):
return max(floor, int(round(base * f)))
return {
"first": s(args.first, _FLOORS["first"]),
"last": s(args.last, _FLOORS["last"]),
"middle": s(args.middle, _FLOORS["middle"]),
"top_files": s(args.top_files, _FLOORS["top_files"]),
"maxlen": s(args.maxlen, _FLOORS["maxlen"]),
"final": s(_BASE_FINAL, _FLOORS["final"]),
"tools": s(_BASE_TOOLS, _FLOORS["tools"]),
"seed": args.seed,
}
def summarize(path, depth, p, out):
"""Load one session log and emit its bounded, XML-tagged report under a
<session depth="N"> element. `p` holds the per-depth limits."""
turns = [] # conversational turns: {"prompt": str, "reply": [text,...]} turns = [] # conversational turns: {"prompt": str, "reply": [text,...]}
edit_counts = {} # file path -> edit count edit_counts = {} # file path -> edit count
last_assistant_text = [] # final text blocks last_assistant_text = [] # final text blocks
@ -351,17 +203,16 @@ def summarize(path, depth, p, out):
# sampled only from what's left between them. If the regions collide # sampled only from what's left between them. If the regions collide
# (short session), the overlap is simply dropped. # (short session), the overlap is simply dropped.
n = len(turns) n = len(turns)
first_idx = list(range(min(p["first"], n))) first_idx = list(range(min(args.first, n)))
last_start = max(len(first_idx), n - p["last"]) last_start = max(len(first_idx), n - args.last)
last_idx = list(range(last_start, n)) last_idx = list(range(last_start, n))
mid_pool = list(range(len(first_idx), last_start)) mid_pool = list(range(len(first_idx), last_start))
if p["seed"] is not None: if args.seed is not None:
random.seed(p["seed"]) random.seed(args.seed)
mid_idx = sorted(random.sample(mid_pool, min(p["middle"], len(mid_pool)))) mid_idx = sorted(random.sample(mid_pool, min(args.middle, len(mid_pool))))
# ---- emit a bounded, XML-tagged report (newlines preserved) ---- # ---- emit a bounded, XML-tagged report (newlines preserved) ----
role = "requested" if depth == 0 else "ancestor" out = sys.stdout.write
out(f'\n<session depth="{depth}" role="{role}">\n')
out("<session-summary>\n") out("<session-summary>\n")
out(f" <file>{esc(path)}</file>\n") out(f" <file>{esc(path)}</file>\n")
@ -380,9 +231,9 @@ def summarize(path, depth, p, out):
t = turns[i] t = turns[i]
reply = "\n".join(t["reply"]).strip() reply = "\n".join(t["reply"]).strip()
out(f' <exchange n="{i + 1}">\n') out(f' <exchange n="{i + 1}">\n')
out(f" <user>\n{clip(t['prompt'], p['maxlen'])}\n </user>\n") out(f" <user>\n{clip(t['prompt'], args.maxlen)}\n </user>\n")
if reply: if reply:
out(f" <agent>\n{clip(reply, p['maxlen'])}\n </agent>\n") out(f" <agent>\n{clip(reply, args.maxlen)}\n </agent>\n")
else: else:
out(" <agent note=\"no text reply — tool calls only\" />\n") out(" <agent note=\"no text reply — tool calls only\" />\n")
out(" </exchange>\n") out(" </exchange>\n")
@ -392,22 +243,20 @@ def summarize(path, depth, p, out):
emit_exchanges("sampled-middle", mid_idx) emit_exchanges("sampled-middle", mid_idx)
emit_exchanges("last", last_idx) emit_exchanges("last", last_idx)
out(f'\n<most-edited-files top="{p["top_files"]}">\n') out(f'\n<most-edited-files top="{args.top_files}">\n')
for fp, c in sorted(edit_counts.items(), key=lambda kv: -kv[1])[: p["top_files"]]: for fp, c in sorted(edit_counts.items(), key=lambda kv: -kv[1])[: args.top_files]:
out(f' <file edits="{c}">{esc(fp)}</file>\n') out(f' <file edits="{c}">{esc(fp)}</file>\n')
out("</most-edited-files>\n") out("</most-edited-files>\n")
out("\n<last-tool-calls>\n") out("\n<last-tool-calls>\n")
for name, arg in recent_tools[-p["tools"]:]: for name, arg in recent_tools[-8:]:
out(f' <call tool="{esc(name)}">{esc(arg)}</call>\n') out(f' <call tool="{esc(name)}">{esc(arg)}</call>\n')
out("</last-tool-calls>\n") out("</last-tool-calls>\n")
out("\n<final-agent-message>\n") out("\n<final-agent-message>\n")
out(clip("\n".join(last_assistant_text), p["final"]) + "\n") out(clip("\n".join(last_assistant_text), 2000) + "\n")
out("</final-agent-message>\n") out("</final-agent-message>\n")
out("</session>\n")
if __name__ == "__main__": if __name__ == "__main__":
main() main()

View file

@ -1,111 +0,0 @@
---
name: compact-file
description: >
Compress natural language memory files (CLAUDE.md, todos, preferences) into shortened format
to save input tokens. Preserves all technical substance, code, URLs, and structure.
Compressed version overwrites the original file. Human-readable backup saved as FILE.original.md.
Trigger: /compact-file FILEPATH or "compact memory file"
---
# Compact File
## Purpose
Compress natural language files (CLAUDE.md, todos, preferences) into caveman-speak to reduce input tokens. Compressed version overwrites original. Human-readable backup saved as `<filename>.original.md`.
## Trigger
`/compact-file <filepath>` or when user asks to compress a memory file.
## Process
1. The compression scripts live in `scripts/` (adjacent to this SKILL.md). If the path is not immediately available, search for `scripts/__main__.py` next to this SKILL.md.
2. From the directory containing this SKILL.md, run:
python3 -m scripts <absolute_filepath>
3. The CLI will:
- detect file type (no tokens)
- call Claude to compress
- validate output (no tokens)
- if errors: cherry-pick fix with Claude (targeted fixes only, no recompression)
- retry up to 2 times
- if still failing after 2 retries: report error to user, leave original file untouched
4. Return result to user
## Compression Rules
### Remove
- Articles: a, an, the
- Filler: just, really, basically, actually, simply, essentially, generally
- Pleasantries: "sure", "certainly", "of course", "happy to", "I'd recommend"
- Hedging: "it might be worth", "you could consider", "it would be good to"
- Redundant phrasing: "in order to" → "to", "make sure to" → "ensure", "the reason is because" → "because"
- Connective fluff: "however", "furthermore", "additionally", "in addition"
### Preserve EXACTLY (never modify)
- Code blocks (fenced ``` and indented)
- Inline code (`backtick content`)
- URLs and links (full URLs, markdown links)
- File paths (`/src/components/...`, `./config.yaml`)
- Commands (`npm install`, `git commit`, `docker build`)
- Technical terms (library names, API names, protocols, algorithms)
- Proper nouns (project names, people, companies)
- Dates, version numbers, numeric values
- Environment variables (`$HOME`, `NODE_ENV`)
### Preserve Structure
- All markdown headings (keep exact heading text, compress body below)
- Bullet point hierarchy (keep nesting level)
- Numbered lists (keep numbering)
- Tables (compress cell text, keep structure)
- Frontmatter/YAML headers in markdown files
### Compress
- Use short synonyms: "big" not "extensive", "fix" not "implement a solution for", "use" not "utilize"
- Fragments OK: "Run tests before commit" not "You should always run tests before committing"
- Drop "you should", "make sure to", "remember to" — just state the action
- Merge redundant bullets that say the same thing differently
- Keep one example where multiple examples show the same pattern
CRITICAL RULE:
Anything inside ``` ... ``` must be copied EXACTLY.
Do not:
- remove comments
- remove spacing
- reorder lines
- shorten commands
- simplify anything
Inline code (`...`) must be preserved EXACTLY.
Do not modify anything inside backticks.
If file contains code blocks:
- Treat code blocks as read-only regions
- Only compress text outside them
- Do not merge sections around code
## Pattern
Original:
> You should always make sure to run the test suite before pushing any changes to the main branch. This is important because it helps catch bugs early and prevents broken builds from being deployed to production.
Compressed:
> Run tests before push to main. Catch bugs early, prevent broken prod deploys.
Original:
> The application uses a microservices architecture with the following components. The API gateway handles all incoming requests and routes them to the appropriate service. The authentication service is responsible for managing user sessions and JWT tokens.
Compressed:
> Microservices architecture. API gateway route all requests to services. Auth service manage user sessions + JWT tokens.
## Boundaries
- ONLY compress natural language files (.md, .txt, .typ, .typst, .tex, extensionless)
- NEVER modify: .py, .js, .ts, .json, .yaml, .yml, .toml, .env, .lock, .css, .html, .xml, .sql, .sh
- If file has mixed content (prose + code), compress ONLY the prose sections
- If unsure whether something is code or prose, leave it unchanged
- Original file is backed up as FILE.original.md before overwriting
- Never compress FILE.original.md (skip it)

View file

@ -1,9 +0,0 @@
"""Caveman compress scripts.
This package provides tools to compress natural language markdown files
into caveman format to save input tokens.
"""
__all__ = ["cli", "compress", "detect", "validate"]
__version__ = "1.0.0"

View file

@ -1,3 +0,0 @@
from .cli import main
main()

View file

@ -1,80 +0,0 @@
#!/usr/bin/env python3
from pathlib import Path
import sys
# Support both direct execution and module import
try:
from .validate import validate
except ImportError:
sys.path.insert(0, str(Path(__file__).parent))
from validate import validate
try:
import tiktoken
_enc = tiktoken.get_encoding("o200k_base")
except ImportError:
_enc = None
def count_tokens(text):
if _enc is None:
return len(text.split()) # fallback: word count
return len(_enc.encode(text))
def benchmark_pair(orig_path: Path, comp_path: Path):
orig_text = orig_path.read_text()
comp_text = comp_path.read_text()
orig_tokens = count_tokens(orig_text)
comp_tokens = count_tokens(comp_text)
saved = 100 * (orig_tokens - comp_tokens) / orig_tokens if orig_tokens > 0 else 0.0
result = validate(orig_path, comp_path)
return (comp_path.name, orig_tokens, comp_tokens, saved, result.is_valid)
def print_table(rows):
print("\n| File | Original | Compressed | Saved % | Valid |")
print("|------|----------|------------|---------|-------|")
for r in rows:
print(f"| {r[0]} | {r[1]} | {r[2]} | {r[3]:.1f}% | {'' if r[4] else ''} |")
def main():
# Direct file pair: python3 benchmark.py original.md compressed.md
if len(sys.argv) == 3:
orig = Path(sys.argv[1]).resolve()
comp = Path(sys.argv[2]).resolve()
if not orig.exists():
print(f"❌ Not found: {orig}")
sys.exit(1)
if not comp.exists():
print(f"❌ Not found: {comp}")
sys.exit(1)
print_table([benchmark_pair(orig, comp)])
return
# Glob mode: repo_root/tests/caveman-compress/
# __file__ lives at <repo_root>/skills/caveman-compress/scripts/benchmark.py
# Walk up four dirs: scripts → caveman-compress → skills → repo_root.
tests_dir = Path(__file__).resolve().parents[3] / "tests" / "caveman-compress"
if not tests_dir.exists():
print(f"❌ Tests dir not found: {tests_dir}")
sys.exit(1)
rows = []
for orig in sorted(tests_dir.glob("*.original.md")):
comp = orig.with_name(orig.stem.removesuffix(".original") + ".md")
if comp.exists():
rows.append(benchmark_pair(orig, comp))
if not rows:
print("No compressed file pairs found.")
return
print_table(rows)
if __name__ == "__main__":
main()

View file

@ -1,85 +0,0 @@
#!/usr/bin/env python3
"""
Caveman Compress CLI
Usage:
caveman <filepath>
"""
import sys
# Force UTF-8 on stdout/stderr before any code can print. Windows consoles
# default to cp1252 and crash on the ❌ glyphs in error/validation branches,
# masking the real error and leaving the user with a half-compressed file.
for _stream in (sys.stdout, sys.stderr):
reconfigure = getattr(_stream, "reconfigure", None)
if callable(reconfigure):
try:
reconfigure(encoding="utf-8", errors="replace")
except Exception:
pass
from pathlib import Path
from .compress import backup_dir_for, compress_file
from .detect import detect_file_type, should_compress
def print_usage():
print("Usage: caveman <filepath>")
def main():
if len(sys.argv) != 2:
print_usage()
sys.exit(1)
filepath = Path(sys.argv[1])
# Check file exists
if not filepath.exists():
print(f"❌ File not found: {filepath}")
sys.exit(1)
if not filepath.is_file():
print(f"❌ Not a file: {filepath}")
sys.exit(1)
filepath = filepath.resolve()
# Detect file type
file_type = detect_file_type(filepath)
print(f"Detected: {file_type}")
# Check if compressible
if not should_compress(filepath):
print("Skipping: file is not natural language (code/config)")
sys.exit(0)
print("Starting caveman compression...\n")
try:
success = compress_file(filepath)
if success:
print("\nCompression completed successfully")
backup_path = backup_dir_for(filepath) / (filepath.stem + ".original.md")
print(f"Compressed: {filepath}")
print(f"Original: {backup_path}")
sys.exit(0)
else:
print("\n❌ Compression failed after retries")
sys.exit(2)
except KeyboardInterrupt:
print("\nInterrupted by user")
sys.exit(130)
except Exception as e:
print(f"\n❌ Error: {e}")
sys.exit(1)
if __name__ == "__main__":
main()

View file

@ -1,342 +0,0 @@
#!/usr/bin/env python3
"""
Caveman Memory Compression Orchestrator
Usage:
python scripts/compress.py <filepath>
"""
import os
import re
import shutil
import subprocess
import sys
from pathlib import Path
from typing import List
OUTER_FENCE_REGEX = re.compile(
r"\A\s*(`{3,}|~{3,})[^\n]*\n(.*)\n\1\s*\Z", re.DOTALL
)
# YAML frontmatter: starts at file start with --- on its own line, ends with --- on its own line.
# Captures the entire block (including delimiters and trailing newline) and the body after.
FRONTMATTER_REGEX = re.compile(
r"\A(---\r?\n.*?\r?\n---\r?\n)(.*)", re.DOTALL
)
def split_frontmatter(text: str):
"""Split YAML frontmatter from body. Returns (frontmatter, body).
Memory files (and many other markdown docs) start with a YAML frontmatter
block delimited by `---` lines. The compression LLM has a habit of stripping
or rewriting these despite preserve-structure rules in the prompt so we
surgically remove the frontmatter before compression and prepend it back
verbatim to the output. Files without frontmatter pass through unchanged.
"""
m = FRONTMATTER_REGEX.match(text)
if m:
return m.group(1), m.group(2)
return "", text
# Filenames and paths that almost certainly hold secrets or PII. Compressing
# them ships raw bytes to the Anthropic API — a third-party data boundary that
# developers on sensitive codebases cannot cross. detect.py already skips .env
# by extension, but credentials.md / secrets.txt / ~/.aws/credentials would
# slip through the natural-language filter. This is a hard refuse before read.
SENSITIVE_BASENAME_REGEX = re.compile(
r"(?ix)^("
r"\.env(\..+)?"
r"|\.netrc"
r"|credentials(\..+)?"
r"|secrets?(\..+)?"
r"|passwords?(\..+)?"
r"|id_(rsa|dsa|ecdsa|ed25519)(\.pub)?"
r"|authorized_keys"
r"|known_hosts"
r"|.*\.(pem|key|p12|pfx|crt|cer|jks|keystore|asc|gpg)"
r")$"
)
SENSITIVE_PATH_COMPONENTS = frozenset({".ssh", ".aws", ".gnupg", ".kube", ".docker"})
SENSITIVE_NAME_TOKENS = (
"secret", "credential", "password", "passwd",
"apikey", "accesskey", "token", "privatekey",
)
def backup_dir_for(filepath: Path) -> Path:
"""Resolve the out-of-tree backup directory for a given source file.
Backups must live OUTSIDE the source directory so skill auto-loaders
(Claude Code rules/, opencode instructions/, etc.) stop re-ingesting the
`.original.md` copies as live files. Base dir is platform-aware:
- Windows: %LOCALAPPDATA%\\caveman-compress\\backups
- else: $XDG_DATA_HOME/caveman-compress/backups if set,
else ~/.local/share/caveman-compress/backups
The source file's parent-dir name is mirrored under the base to reduce
cross-project collisions (e.g. two `task.md` files in different repos).
"""
if os.name == "nt" or sys.platform == "win32":
local_appdata = os.environ.get("LOCALAPPDATA")
base = Path(local_appdata) if local_appdata else Path.home() / "AppData" / "Local"
base = base / "caveman-compress" / "backups"
else:
xdg = os.environ.get("XDG_DATA_HOME")
base = Path(xdg) if xdg else Path.home() / ".local" / "share"
base = base / "caveman-compress" / "backups"
return base / filepath.parent.name
def is_sensitive_path(filepath: Path) -> bool:
"""Heuristic denylist for files that must never be shipped to a third-party API."""
name = filepath.name
if SENSITIVE_BASENAME_REGEX.match(name):
return True
lowered_parts = {p.lower() for p in filepath.parts}
if lowered_parts & SENSITIVE_PATH_COMPONENTS:
return True
# Normalize separators so "api-key" and "api_key" both match "apikey".
lower = re.sub(r"[_\-\s.]", "", name.lower())
return any(tok in lower for tok in SENSITIVE_NAME_TOKENS)
def strip_llm_wrapper(text: str) -> str:
"""Strip outer ```markdown ... ``` fence when it wraps the entire output."""
m = OUTER_FENCE_REGEX.match(text)
if m:
return m.group(2)
return text
from .detect import should_compress
from .validate import validate
MAX_RETRIES = 2
# ---------- Claude Calls ----------
def call_claude(prompt: str) -> str:
"""Send a prompt to Claude.
Prefers the Anthropic SDK when ANTHROPIC_API_KEY is set; otherwise falls
back to the ``claude --print`` CLI (which handles desktop auth).
On Windows the CLI subprocess decoding defaults to the system codepage
(cp1251 / cp1252) and crashes on UTF-8 output see issue #152. Pinning
``encoding="utf-8"`` with ``errors="replace"`` matches the CLI's actual
native I/O and prevents the UnicodeDecodeError before validation can
report. Windows users with non-ASCII content can also set
``ANTHROPIC_API_KEY`` to route through the SDK and skip the subprocess.
"""
api_key = os.environ.get("ANTHROPIC_API_KEY")
if api_key:
try:
import anthropic
client = anthropic.Anthropic(api_key=api_key)
msg = client.messages.create(
model=os.environ.get("CAVEMAN_MODEL", "claude-sonnet-4-5"),
max_tokens=8192,
messages=[{"role": "user", "content": prompt}],
)
return strip_llm_wrapper(msg.content[0].text.strip())
except ImportError:
pass # anthropic not installed, fall back to CLI
# Fallback: use claude CLI (handles desktop auth).
# Resolve binary via shutil.which so Windows .cmd/.bat shims (e.g.
# %APPDATA%\npm\claude.CMD) work without shell=True. On POSIX,
# shutil.which returns the same absolute path as the implicit lookup,
# so this is a no-op there. Falls back to bare "claude" if not found
# on PATH so subprocess raises a clear FileNotFoundError.
claude_bin = shutil.which("claude") or "claude"
try:
result = subprocess.run(
[claude_bin, "--print"],
input=prompt,
text=True,
capture_output=True,
check=True,
encoding="utf-8",
errors="replace",
)
return strip_llm_wrapper(result.stdout.strip())
except subprocess.CalledProcessError as e:
raise RuntimeError(f"Claude call failed:\n{e.stderr}")
def build_compress_prompt(original: str) -> str:
return f"""
Compress this markdown into caveman format.
STRICT RULES:
- Do NOT modify anything inside ``` code blocks
- Do NOT modify anything inside inline backticks
- Preserve ALL URLs exactly
- Preserve ALL headings exactly
- Preserve file paths and commands
- Return ONLY the compressed markdown body do NOT wrap the entire output in a ```markdown fence or any other fence. Inner code blocks from the original stay as-is; do not add a new outer fence around the whole file.
Only compress natural language.
TEXT:
{original}
"""
def build_fix_prompt(original: str, compressed: str, errors: List[str]) -> str:
errors_str = "\n".join(f"- {e}" for e in errors)
return f"""You are fixing a caveman-compressed markdown file. Specific validation errors were found.
CRITICAL RULES:
- DO NOT recompress or rephrase the file
- ONLY fix the listed errors leave everything else exactly as-is
- The ORIGINAL is provided as reference only (to restore missing content)
- Preserve caveman style in all untouched sections
ERRORS TO FIX:
{errors_str}
HOW TO FIX:
- Missing URL: find it in ORIGINAL, restore it exactly where it belongs in COMPRESSED
- Code block mismatch: find the exact code block in ORIGINAL, restore it in COMPRESSED
- Heading mismatch: restore the exact heading text from ORIGINAL into COMPRESSED
- Do not touch any section not mentioned in the errors
ORIGINAL (reference only):
{original}
COMPRESSED (fix this):
{compressed}
Return ONLY the fixed compressed file. No explanation.
"""
# ---------- Core Logic ----------
def compress_file(filepath: Path) -> bool:
# Resolve and validate path
filepath = filepath.resolve()
MAX_FILE_SIZE = 500_000 # 500KB
if not filepath.exists():
raise FileNotFoundError(f"File not found: {filepath}")
if filepath.stat().st_size > MAX_FILE_SIZE:
raise ValueError(f"File too large to compress safely (max 500KB): {filepath}")
# Refuse files that look like they contain secrets or PII. Compressing ships
# the raw bytes to the Anthropic API — a third-party boundary — so we fail
# loudly rather than silently exfiltrate credentials or keys. Override is
# intentional: the user must rename the file if the heuristic is wrong.
if is_sensitive_path(filepath):
raise ValueError(
f"Refusing to compress {filepath}: filename looks sensitive "
"(credentials, keys, secrets, or known private paths). "
"Compression sends file contents to the Anthropic API. "
"Rename the file if this is a false positive."
)
print(f"Processing: {filepath}")
if not should_compress(filepath):
print("Skipping (not natural language)")
return False
original_text = filepath.read_text(errors="ignore")
# Store backup outside the source directory so skill auto-loaders don't
# re-ingest the `.original.md` copy as a live file. Mirror the source's
# parent-dir name + stem under a platform-aware base to reduce collisions.
backup_dir = backup_dir_for(filepath)
backup_dir.mkdir(parents=True, exist_ok=True)
backup_path = backup_dir / (filepath.stem + ".original.md")
if not original_text.strip():
print("❌ Refusing to compress: file is empty or whitespace-only.")
return False
# Check if backup already exists to prevent accidental overwriting
if backup_path.exists():
print(f"⚠️ Backup file already exists: {backup_path}")
print("The original backup may contain important content.")
print("Aborting to prevent data loss. Please remove or rename the backup file if you want to proceed.")
return False
# Split YAML frontmatter off before compression. Claude tends to strip or
# rewrite frontmatter despite preserve-structure rules; we keep it verbatim
# by removing it from the input and re-prepending it to the output.
frontmatter, body = split_frontmatter(original_text)
if frontmatter:
print(f"Detected YAML frontmatter ({len(frontmatter)} chars) — preserving verbatim")
if not body.strip():
print("❌ Refusing to compress: body is empty after frontmatter removal.")
return False
# Step 1: Compress (body only, frontmatter excluded)
print("Compressing with Claude...")
compressed_body = call_claude(build_compress_prompt(body))
if compressed_body is None or not compressed_body.strip():
print("❌ Compression aborted: Claude returned an empty response.")
print(" Original file is untouched (no backup created).")
return False
# Compare the BODY (not the whole file) — frontmatter is preserved verbatim
# and would never change, so identity must be judged on the compressible part.
if compressed_body.strip() == body.strip():
print("❌ Compression aborted: output is identical to input.")
print(" Likely causes: Claude refused, returned the prompt verbatim, or the file is")
print(" already in caveman form. Original file is untouched (no backup created).")
return False
# Reassemble: frontmatter (verbatim) + compressed body
compressed = frontmatter + compressed_body
# Save original as backup, then verify the backup readback before
# touching the input file. If the filesystem dropped bytes (encoding,
# antivirus, disk full), unlink the bad backup and abort instead of
# leaving the user with a corrupt backup + compressed primary.
backup_path.write_text(original_text)
backup_readback = backup_path.read_text(errors="ignore")
if backup_readback != original_text:
print(f"❌ Backup write verification failed: {backup_path}")
print(" In-memory original differs from on-disk backup. Aborting before touching the input file.")
try:
backup_path.unlink()
except OSError:
pass
return False
filepath.write_text(compressed)
# Step 2: Validate + Retry
for attempt in range(MAX_RETRIES):
print(f"\nValidation attempt {attempt + 1}")
result = validate(backup_path, filepath)
if result.is_valid:
print("Validation passed")
break
print("❌ Validation failed:")
for err in result.errors:
print(f" - {err}")
if attempt == MAX_RETRIES - 1:
# Restore original on failure
filepath.write_text(original_text)
backup_path.unlink(missing_ok=True)
print("❌ Failed after retries — original restored")
return False
print("Fixing with Claude...")
compressed = call_claude(
build_fix_prompt(original_text, compressed, result.errors)
)
filepath.write_text(compressed)
return True

View file

@ -1,121 +0,0 @@
#!/usr/bin/env python3
"""Detect whether a file is natural language (compressible) or code/config (skip)."""
import json
import re
from pathlib import Path
# Extensions that are natural language and compressible
COMPRESSIBLE_EXTENSIONS = {".md", ".txt", ".markdown", ".rst", ".typ", ".typst", ".tex"}
# Extensions that are code/config and should be skipped
SKIP_EXTENSIONS = {
".py", ".js", ".ts", ".tsx", ".jsx", ".json", ".yaml", ".yml",
".toml", ".env", ".lock", ".css", ".scss", ".html", ".xml",
".sql", ".sh", ".bash", ".zsh", ".go", ".rs", ".java", ".c",
".cpp", ".h", ".hpp", ".rb", ".php", ".swift", ".kt", ".lua",
".dockerfile", ".makefile", ".csv", ".ini", ".cfg",
}
# Patterns that indicate a line is code
CODE_PATTERNS = [
re.compile(r"^\s*(import |from .+ import |require\(|const |let |var )"),
re.compile(r"^\s*(def |class |function |async function |export )"),
re.compile(r"^\s*(if\s*\(|for\s*\(|while\s*\(|switch\s*\(|try\s*\{)"),
re.compile(r"^\s*[\}\]\);]+\s*$"), # closing braces/brackets
re.compile(r"^\s*@\w+"), # decorators/annotations
re.compile(r'^\s*"[^"]+"\s*:\s*'), # JSON-like key-value
re.compile(r"^\s*\w+\s*=\s*[{\[\(\"']"), # assignment with literal
]
def _is_code_line(line: str) -> bool:
"""Check if a line looks like code."""
return any(p.match(line) for p in CODE_PATTERNS)
def _is_json_content(text: str) -> bool:
"""Check if content is valid JSON."""
try:
json.loads(text)
return True
except (json.JSONDecodeError, ValueError):
return False
def _is_yaml_content(lines: list[str]) -> bool:
"""Heuristic: check if content looks like YAML."""
yaml_indicators = 0
for line in lines[:30]:
stripped = line.strip()
if stripped.startswith("---"):
yaml_indicators += 1
elif re.match(r"^\w[\w\s]*:\s", stripped):
yaml_indicators += 1
elif stripped.startswith("- ") and ":" in stripped:
yaml_indicators += 1
# If most non-empty lines look like YAML
non_empty = sum(1 for l in lines[:30] if l.strip())
return non_empty > 0 and yaml_indicators / non_empty > 0.6
def detect_file_type(filepath: Path) -> str:
"""Classify a file as 'natural_language', 'code', 'config', or 'unknown'.
Returns:
One of: 'natural_language', 'code', 'config', 'unknown'
"""
ext = filepath.suffix.lower()
# Extension-based classification
if ext in COMPRESSIBLE_EXTENSIONS:
return "natural_language"
if ext in SKIP_EXTENSIONS:
return "code" if ext not in {".json", ".yaml", ".yml", ".toml", ".ini", ".cfg", ".env"} else "config"
# Extensionless files (like CLAUDE.md, TODO) — check content
if not ext:
try:
text = filepath.read_text(errors="ignore")
except (OSError, PermissionError):
return "unknown"
lines = text.splitlines()[:50]
if _is_json_content(text[:10000]):
return "config"
if _is_yaml_content(lines):
return "config"
code_lines = sum(1 for l in lines if l.strip() and _is_code_line(l))
non_empty = sum(1 for l in lines if l.strip())
if non_empty > 0 and code_lines / non_empty > 0.4:
return "code"
return "natural_language"
return "unknown"
def should_compress(filepath: Path) -> bool:
"""Return True if the file is natural language and should be compressed."""
if not filepath.is_file():
return False
# Skip backup files
if filepath.name.endswith(".original.md"):
return False
return detect_file_type(filepath) == "natural_language"
if __name__ == "__main__":
import sys
if len(sys.argv) < 2:
print("Usage: python detect.py <file1> [file2] ...")
sys.exit(1)
for path_str in sys.argv[1:]:
p = Path(path_str).resolve()
file_type = detect_file_type(p)
compress = should_compress(p)
print(f" {p.name:30s} type={file_type:20s} compress={compress}")

View file

@ -1,213 +0,0 @@
#!/usr/bin/env python3
import re
from collections import Counter
from pathlib import Path
URL_REGEX = re.compile(r"https?://[^\s)]+")
FENCE_OPEN_REGEX = re.compile(r"^(\s{0,3})(`{3,}|~{3,})(.*)$")
HEADING_REGEX = re.compile(r"^(#{1,6})\s+(.*)", re.MULTILINE)
BULLET_REGEX = re.compile(r"^\s*[-*+]\s+", re.MULTILINE)
# crude but effective path detection
# Requires either a path prefix (./ ../ / or drive letter) or a slash/backslash within the match
PATH_REGEX = re.compile(r"(?:\./|\.\./|/|[A-Za-z]:\\)[\w\-/\\\.]+|[\w\-\.]+[/\\][\w\-/\\\.]+")
class ValidationResult:
def __init__(self):
self.is_valid = True
self.errors = []
self.warnings = []
def add_error(self, msg):
self.is_valid = False
self.errors.append(msg)
def add_warning(self, msg):
self.warnings.append(msg)
def read_file(path: Path) -> str:
return path.read_text(errors="ignore")
# ---------- Extractors ----------
def extract_headings(text):
return [(level, title.strip()) for level, title in HEADING_REGEX.findall(text)]
def extract_code_blocks(text):
"""Line-based fenced code block extractor.
Handles ``` and ~~~ fences with variable length (CommonMark: closing
fence must use same char and be at least as long as opening). Supports
nested fences (e.g. an outer 4-backtick block wrapping inner 3-backtick
content).
"""
blocks = []
lines = text.split("\n")
i = 0
n = len(lines)
while i < n:
m = FENCE_OPEN_REGEX.match(lines[i])
if not m:
i += 1
continue
fence_char = m.group(2)[0]
fence_len = len(m.group(2))
open_line = lines[i]
block_lines = [open_line]
i += 1
closed = False
while i < n:
close_m = FENCE_OPEN_REGEX.match(lines[i])
if (
close_m
and close_m.group(2)[0] == fence_char
and len(close_m.group(2)) >= fence_len
and close_m.group(3).strip() == ""
):
block_lines.append(lines[i])
closed = True
i += 1
break
block_lines.append(lines[i])
i += 1
if closed:
blocks.append("\n".join(block_lines))
# Unclosed fences are silently skipped — they indicate malformed markdown
# and including them would cause false-positive validation failures.
return blocks
def extract_urls(text):
return set(URL_REGEX.findall(text))
def extract_paths(text):
return set(PATH_REGEX.findall(text))
def count_bullets(text):
return len(BULLET_REGEX.findall(text))
def extract_inline_codes(text):
text_without_fences = re.sub(r"^```[\s\S]*?^```", "", text, flags=re.MULTILINE)
text_without_fences = re.sub(r"^~~~[\s\S]*?^~~~", "", text_without_fences, flags=re.MULTILINE)
return re.findall(r"`([^`]+)`", text_without_fences)
# ---------- Validators ----------
def validate_headings(orig, comp, result):
h1 = extract_headings(orig)
h2 = extract_headings(comp)
if len(h1) != len(h2):
result.add_error(f"Heading count mismatch: {len(h1)} vs {len(h2)}")
if h1 != h2:
result.add_warning("Heading text/order changed")
def validate_code_blocks(orig, comp, result):
c1 = extract_code_blocks(orig)
c2 = extract_code_blocks(comp)
if c1 != c2:
result.add_error("Code blocks not preserved exactly")
def validate_urls(orig, comp, result):
u1 = extract_urls(orig)
u2 = extract_urls(comp)
if u1 != u2:
result.add_error(f"URL mismatch: lost={u1 - u2}, added={u2 - u1}")
def validate_paths(orig, comp, result):
p1 = extract_paths(orig)
p2 = extract_paths(comp)
if p1 != p2:
result.add_warning(f"Path mismatch: lost={p1 - p2}, added={p2 - p1}")
def validate_bullets(orig, comp, result):
b1 = count_bullets(orig)
b2 = count_bullets(comp)
if b1 == 0:
return
diff = abs(b1 - b2) / b1
if diff > 0.15:
result.add_warning(f"Bullet count changed too much: {b1} -> {b2}")
def validate_inline_codes(orig, comp, result):
c1 = Counter(extract_inline_codes(orig))
c2 = Counter(extract_inline_codes(comp))
if c1 != c2:
lost = set(c1.keys()) - set(c2.keys())
added = set(c2.keys()) - set(c1.keys())
for code, count in c1.items():
if code in c2 and c2[code] < count:
lost.add(f"{code} (lost {count - c2[code]} of {count} occurrences)")
if lost:
result.add_error(f"Inline code lost: {lost}")
if added:
result.add_warning(f"Inline code added: {added}")
# ---------- Main ----------
def validate(original_path: Path, compressed_path: Path) -> ValidationResult:
result = ValidationResult()
orig = read_file(original_path)
comp = read_file(compressed_path)
validate_headings(orig, comp, result)
validate_code_blocks(orig, comp, result)
validate_urls(orig, comp, result)
validate_paths(orig, comp, result)
validate_bullets(orig, comp, result)
validate_inline_codes(orig, comp, result)
return result
# ---------- CLI ----------
if __name__ == "__main__":
import sys
if len(sys.argv) != 3:
print("Usage: python validate.py <original> <compressed>")
sys.exit(1)
orig = Path(sys.argv[1]).resolve()
comp = Path(sys.argv[2]).resolve()
res = validate(orig, comp)
print(f"\nValid: {res.is_valid}")
if res.errors:
print("\nErrors:")
for e in res.errors:
print(f" - {e}")
if res.warnings:
print("\nWarnings:")
for w in res.warnings:
print(f" - {w}")

View file

@ -16,10 +16,9 @@ If there are uncommitted changes:
## Step 2: Switch to main and pull latest ## Step 2: Switch to main and pull latest
- Attempt to check out `main`, or failing that `master`. - Attempt to check out `main`.
- Then pull the latest changes. - If that fails because the branch does not exist, check out `master` instead.
- Then pull the latest changes:
You **must** do this. Unless instructed to continue on the same branch, make sure you start from latest main.
## Step 3: Create a new branch ## Step 3: Create a new branch

View file

@ -1,32 +0,0 @@
---
name: sqlite
description: Recommended SQLite defaults (PRAGMAs and STRICT tables). Use when setting up a new project that uses SQLite, or when adding new SQLite tables to an existing project.
---
Apply these when starting a new SQLite project or adding tables. They are the sane defaults SQLite ships without.
## Connection PRAGMAs
Run these on **every new connection** (except `journal_mode`, see below). They are per-connection state, not stored in the DB file. If your library has a "connection init" hook, put them there.
```sql
PRAGMA journal_mode = WAL; -- set once, persists in the DB file
PRAGMA busy_timeout = 5000; -- wait 5s on a locked DB instead of erroring
PRAGMA synchronous = NORMAL; -- safe to lower from FULL only under WAL
PRAGMA foreign_keys = ON; -- SQLite ignores FK constraints unless you enable this
```
## Tables
Create tables as `STRICT` so column types are enforced:
```sql
CREATE TABLE user (id INTEGER PRIMARY KEY, name TEXT NOT NULL) STRICT;
```
## Gotchas
- **`foreign_keys`, `busy_timeout`, `synchronous` reset per connection.** Missing them on any connection silently drops the behavior (e.g. FK constraints stop being enforced). Set them on connection open.
- **`journal_mode = WAL` is different**: it persists in the DB file, so set it once. WAL needs the DB on **local disk** (breaks on network filesystems) and adds `-wal` and `-shm` sidecar files next to the DB.
- **`synchronous = NORMAL` durability**: an OS crash or power loss can lose the last few committed transactions. The DB stays consistent (no corruption) — only the tail is lost. `NORMAL` is almost always the right choice; `FULL` is rarely needed. If you genuinely cannot afford to lose any committed write, you shouldn't be on SQLite at all — use a DB replicated across regions.
- **STRICT allows only `INT`, `INTEGER`, `REAL`, `TEXT`, `BLOB`, `ANY`.** No `VARCHAR`, `DATETIME`, `BOOLEAN`, etc. Use `ANY` for a column that needs to hold mixed types.

View file

@ -1,6 +1,2 @@
.history/ .history/
## Panda
styled-system
styled-system-studio

1
htop/htop_history Normal file
View file

@ -0,0 +1 @@
python