Last active
October 7, 2025 03:42
-
-
Save do-me/34516f7f4d8cc701da823089b09a3359 to your computer and use it in GitHub Desktop.
A single line to try out mlx-community/Qwen3-Next-80B-A3B-Instruct-8bit on MacOS with mlx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import argparse | |
| from mlx_lm import load, generate | |
| # Parse CLI arguments | |
| parser = argparse.ArgumentParser() | |
| parser.add_argument("--prompt", type=str, default="hello", help="Custom prompt text") | |
| parser.add_argument("--max-tokens", type=int, default=1024, help="Maximum number of tokens to generate") | |
| args = parser.parse_args() | |
| # Load model | |
| model, tokenizer = load("mlx-community/Qwen3-Next-80B-A3B-Instruct-8bit") | |
| prompt = args.prompt | |
| # If the tokenizer supports chat templates, wrap input | |
| if tokenizer.chat_template is not None: | |
| messages = [{"role": "user", "content": prompt}] | |
| prompt = tokenizer.apply_chat_template( | |
| messages, add_generation_prompt=True | |
| ) | |
| # Generate response | |
| response = generate( | |
| model, | |
| tokenizer, | |
| prompt=prompt, | |
| verbose=True, | |
| max_tokens=args.max_tokens | |
| ) |
Update: this is a better option to actually chat with the model:
uv run --with git+https://github.com/ml-explore/mlx-lm.git mlx_lm.chat --model mlx-community/Qwen3-Next-80B-A3B-Instruct-8bit --max-tokens 10000
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Run Qwen3-Next-80B-A3B-Instruct-8bit with one line on Mac!
Install uv, then just run this line to try it out (note that as of 12 September there is a bug in the latest mlx-lm release, so it must be used from source):
Disclaimer: You should never run remote code like this from random folks on the internet. This example is meant for educational purposes. Better do this and double check the code before running:
... or do it safer with 2 lines!
curl -sL https://gist.githubusercontent.com/do-me/34516f7f4d8cc701da823089b09a3359/raw/b24c55bc0b80e95254a3028c92f6b2e875ef1c1b/prompt.py -o prompt.py uv run --with git+https://github.com/ml-explore/mlx-lm.git python prompt.py --prompt "What is the meaning of life?" --max-tokens 2048Response on Mac M3 Max (shell output)
Response (rendered markdown)
The question “What is the meaning of life?” is one of the most profound and enduring inquiries in human history — explored by philosophers, theologians, scientists, artists, and everyday people across cultures and centuries. There is no single, universally accepted answer, but here are several perspectives:
1. Philosophical Perspectives
2. Religious and Spiritual Views
3. Scientific and Biological View
From a biological standpoint, life’s “purpose” is survival and reproduction — passing on genes. But humans, uniquely self-aware, transcend mere biology. Science doesn’t assign meaning, but it helps us understand how we came to be — leaving the “why” to us.
4. Psychological Perspective
Psychologists like Viktor Frankl (a Holocaust survivor and founder of logotherapy) argued that the primary human drive is not pleasure (Freud) or power (Adler), but meaning. He wrote:
5. Personal and Humanistic View
Many find meaning in:
In Summary:
It’s not a single answer written in the stars, but a living question — one you answer every day through your choices, your compassion, your curiosity, and your courage to keep going.
As philosopher Albert Camus put it:
So — what does your life mean to you? 🌱
Prompt: 15 tokens, 86.336 tokens-per-sec
Generation: 708 tokens, 46.862 tokens-per-sec
Peak memory: 84.834 GB