wtf is doc-lok?

doc-lok is a Node.js library and command-line tool that reduces the number of tokens your Markdown documents consume when you feed them to an AI model. It works by detecting every external URL in a Markdown file, checking whether the content at that URL has changed since you last ran it, and replacing any unchanged links with a tiny HTML comment placeholder. Only links that have actually changed are left visible to the model. The token savings come from the fact that URLs and the text around them take up space in the AI's context window. If a documentation link or external reference has not changed since your last prompt, the model does not need to see it again. doc-lok caches the content hash and HTTP ETag of each URL in a local lockfile, then checks freshness with a lightweight network request before deciding whether to compress a link. The README claims up to 99.5% reduction per cached link. The restore command reverses the operation, turning the compact placeholders back into full links. This round-trip is lossless because each placeholder embeds the hash needed to look up the original URL in the lockfile. You can use it from the command line by piping a Markdown file through it before sending the result to an LLM, or as a TypeScript library inside a larger automation. It also outputs JSON for machine-readable integration with AI agent frameworks. A Windsurf workflow skill file is included. The library has zero runtime dependencies and requires Node.js 18 or later. The license is MIT.