Threadstone is offline terminal chat for local MLX language models on Apple Silicon. It starts a local mlx-vlm server for the model you choose, streams the answer into your terminal, keeps the conversation usable across crashes, and shuts the server down when you leave.

The useful part is ownership. The network is only for setup and model downloads. After that, chat runs against local snapshots with Hugging Face offline flags, no telemetry, no hosted API, and no account.

Requirements: macOS, Apple Silicon, Python 3.13+, Homebrew, and about 12 GB for the default model set.

git clone https://github.com/gabrimatic/threadstone.git
cd threadstone
./setup.sh

setup.sh creates ~/mlx-env, installs the MLX runtime, downloads the configured Qwen model snapshots, installs the threadstone CLI, and writes shell helpers for oracle, forge, and quench.

Start a chat:

oracle
oracle 4B
oracle "be terse" 2B

Use the installed CLI directly:

threadstone --list-models
threadstone --doctor
threadstone "answer like a systems engineer" 9B

Threadstone uses the network during setup. Runtime chat stays on localhost and local model files.

Offline guard:

HF_HUB_OFFLINE=1
TRANSFORMERS_OFFLINE=1
HF_DATASETS_OFFLINE=1
HF_HUB_DISABLE_TELEMETRY=1
HF_HUB_DISABLE_IMPLICIT_TOKEN=1
HF_TOKEN=
DO_NOT_TRACK=1
DISABLE_TELEMETRY=1
ANONYMIZED_TELEMETRY=0

Inside chat:

Shell helpers from setup.sh:

Default models are 4-bit MLX snapshots from mlx-community.

Each model has its own default port, memory estimate, context threshold, and response budget in config.py.

Threadstone owns the server lifecycle for normal chat.

• Port recovery: if the default port is busy, Threadstone scans forward and starts the model on the next available localhost port.
• RAM guard: startup checks free and reclaimable memory before launching a model, including other reachable model servers.
• Crash recovery: if the server disappears during a turn, Threadstone restarts it and resends the pending message.
• Thinking models: reasoning streams dimmed until </think>, then the final answer prints normally and only the final answer is sent back in later history.
• Attachments: /read accepts text files and directory listings, rejects non-regular files, rejects binary-looking content, and caps file payloads at 50 KB.
• Context trimming: old turns are trimmed when the approximate context crosses the configured threshold while preserving valid role alternation.

oracle / threadstone
    |
    v
threadstone.py
    |-- argument parsing, doctor, REPL, streaming, history
    |-- ServerManager starts and monitors mlx_vlm.server
    |-- /read attaches bounded local text context
    |
    v
config.py
    |-- model registry, ports, limits, memory estimates
    |-- snapshot resolution from the Hugging Face cache
    |-- offline environment guard

Files:

python3 -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"
python3 -m unittest discover -s tests -t . -v
python3 -m build --sdist --wheel
python3 -m twine check dist/*

Run the CLI without starting a model:

threadstone --version
threadstone --list-models

Run the local setup check after ./setup.sh:

threadstone --doctor --all-models

Threadstone is a Python package, so the package registry is PyPI, not pub.dev. pub.dev is for Dart and Flutter packages.

Release path:

1. Update CHANGELOG.md and pyproject.toml.
2. Run tests, build, and twine check.
3. Create a GitHub release tag such as v1.1.0.
4. The release workflow builds the sdist and wheel, then publishes to PyPI through Trusted Publishing.

PyPI Trusted Publishing must be configured for:

Runtime chat is local by design, but model files and dependencies still come from external package and model hosts during setup.

Report vulnerabilities through GitHub private vulnerability reporting. Do not open a public issue for security reports.