Fix autothink and thinkdeeper for transformers>=5 (#320)

* Fix autothink and thinkdeeper for transformers>=5

* Keep device transfer when unwrapping BatchEncoding, cover deepconf, bump 0.3.22

Builds on the transformers>=5 fix: apply_chat_template now returns a
BatchEncoding, so the tensor must be unwrapped via .input_ids.

Two corrections to the original change:

1. Preserve the device transfer. `tokens.to(device)` was called without
   assigning the result, but Tensor.to() is not in-place -- it returns a new
   tensor. As written the tokens stayed on CPU, so on CUDA/MPS the subsequent
   model(input_ids=tokens, ...) would fail with a device mismatch. A CPU-only
   repro hides this. Chain it instead: .input_ids.to(self.model.device).

2. Apply the same fix to optillm/deepconf/processor.py, which has the identical
   apply_chat_template(return_tensors="pt") -> model(input_ids=tokens) pattern
   and was still passing a BatchEncoding.

Also updates the README badges (drop the GitHub stars badge, use the pepy.tech
downloads badge which reports actual counts) and bumps the version to 0.3.22.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Asankhaya Sharma <codelion@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Fix PyPI wheel deps: add math-verify, cap transformers <5.13 (bump 0.…

…3.21) (#318)

The published wheel is built from pyproject.toml [project.dependencies], which
had drifted from requirements.txt. As a result `pip install optillm` shipped a
broken package:

- math-verify was missing entirely, so `import optillm` failed with
  ModuleNotFoundError (optillm.cepo imports math_verify at import time) -- the
  package would not even start.
- transformers was unpinned, so the mlx-lm cap added in 0.3.19 (requirements.txt
  only) never reached PyPI; installs pulled transformers 5.13.0 and broke mlx-lm
  on Apple silicon.

Sync the runtime deps into pyproject.toml:
- add math-verify
- transformers>=5.0.0,<5.13.0
- outlines[transformers]>=1.2.3

Verified by building the wheel and installing it into a clean venv (only its
declared deps) and importing optillm successfully.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Honor model eos_token_id and bound local generation, bump to 0.3.20 (#…

…317)

Two fixes for runaway local generation with models whose ChatML end token
differs from their tokenizer EOS (e.g. dhara-250m: chat ends at <|im_end|>=49154
but tokenizer eos is <|end_of_text|>=1). optillm forced eos to the tokenizer's
id and defaulted max_new_tokens to 4096, so such a model never stopped and
generated 4096 tokens (~800s at ~5 tok/s) on every call that omitted max_tokens.

1. Resolve EOS from the model's generation_config.eos_token_id (merging the
   tokenizer eos as a fallback) instead of hardcoding tokenizer.eos_token_id.
   Applied to both PyTorch generate paths.
2. Make the default max_new_tokens env-configurable via OPTILLM_MAX_TOKENS
   (default 4096), covering the config builders and the InferenceClient.create()
   request paths. An explicit request max_tokens still wins. Set
   OPTILLM_MAX_TOKENS=128 in the CI jobs that run the small test model.

Adds unit tests (no model load) for both helpers. Verified end to end: an
unbounded request with OPTILLM_MAX_TOKENS=64 stops at 64 tokens instead of 4096.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Cap transformers <5.13 to keep mlx-lm importable (macOS), bump to 0.3…

….19 (#316)

transformers 5.13.0 tightened AutoModel/AutoTokenizer.register() to require a
class (it does key.__module__). mlx-lm 0.31.3 registers a custom tokenizer by
string name (tokenizer_utils.py: AutoTokenizer.register("NewlineTokenizer", ...)),
so `import mlx_lm` — and therefore `import optillm` — crashes with
"AttributeError: 'str' object has no attribute '__module__'".

This only affects Apple silicon, where optillm installs mlx-lm
(platform_machine=="arm64" and sys_platform=="darwin"); Linux CI never installs
mlx-lm so it was unaffected. The transformers 5.13.0 HF path itself is fine
(dhara loads and generates); the incompatibility is purely mlx-lm's.

Pin transformers>=5.0.0,<5.13.0 (resolves to 5.12.1) until mlx-lm ships a
transformers-5.13-compatible release, then lift the cap.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Add opt-in file persistence to the memory plugin (#111) (#315)

* Add opt-in file persistence to the memory plugin (#111)

The memory plugin builds a fresh in-RAM Memory() per request, so extracted
memories never survive across calls. This adds an opt-in file-backed store,
gated on the OPTILLM_MEMORY_FILE env var:

- Memory(persist_path=...) loads saved items on init and writes after each add()
- run() reads the path from OPTILLM_MEMORY_FILE; unset => behaviour unchanged
- loads degrade gracefully on missing/corrupt/non-list files (logged, no raise)
- saves are atomic (temp file + os.replace) and bounded by max_size
- README documents the env var; unit test covers round-trip, corrupt/missing
  files, max_size truncation, and the default no-I/O path

* Bump version to 0.3.18

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Add Frame SAST scan of PR-changed files to CI

Adds a security-scan workflow that runs the Frame neuro-symbolic SAST tool
(lambdasec/frame, pinned) on the Python files changed by a pull request.

Scanning only the PR's added/modified files surfaces issues introduced by the
change without failing on pre-existing findings elsewhere in the tree. The job
fails only on high/critical severity, so lower-confidence categories
(e.g. insecure_random on algorithmic sampling) do not block merges.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Asankhaya Sharma <codelion@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

fix: read MCTS params per-request instead of mutating shared global (#…

…304) (#314)

* fix: read MCTS params per-request instead of mutating shared global (#304)

proxy() wrote the request's mcts_depth/mcts_exploration/mcts_simulations
into the module-level server_config on every request, and
execute_single_approach read them back out of that same global. Under
Flask's threaded mode (or gunicorn/uWSGI), concurrent requests overwrite
each other's MCTS parameters between the write and the read, silently
corrupting results. The write also leaked one request's params into every
subsequent request.

Resolve the params from the per-request request_config (with the server
defaults as fallback) and stop writing them into the shared global, so
each request uses its own values with no cross-request interference.

Add regression tests covering the per-request read, the CLI-default
fallback, and an end-to-end proxy check that the global is no longer
mutated by a request.

Fixes #304

* Bump version to 0.3.17

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Asankhaya Sharma <codelion@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Merge pull request #311 from SuperMarioYL/fix/response-validation-rem…

…aining-approaches

Guard against empty/None/truncated provider responses in rto, self_consistency, reread, leap

Merge pull request #305 from GoDiao/feature/compact-plugin

Add compact plugin for auto context compression

Merge pull request #297 from algorithmicsuperintelligence/fix-spacy-v…

…ersion-constraint

Fix spacy version constraint

Merge pull request #294 from algorithmicsuperintelligence/fix-mps-com…

…patibility-and-version-bump

Fix macOS MPS compatibility and bump version to 0.3.13