points by Confiks 1 day ago

Get into a venv, and run:

> pip3 install git+https://github.com/ml-explore/mlx-lm.git

> ./venv/bin/mlx_lm.generate --model "$MODEL" --temp 1.0 --top-p 0.95 --top-k 64 --max-tokens 128000 --prompt "Hello world"

Where $MODEL is an unsloth model like:

- unsloth/gemma-4-E4B-it-UD-MLX-4bit

- unsloth/gemma-4-26b-a4b-it-UD-MLX-4bit