Perplexity CLI in pure shell

p.sh

#!/usr/bin/env bash
function p() {
	jq -n \
		--arg content "$*" \
		'{
      "model": "pplx-7b-online",
      "messages": [
        {
          "role": "system",
          "content": "Be precise and concise."
        },
        {
          "role": "user",
          "content": $content
        }
      ],
      "stream": true
    }' | curl --silent \
		--request POST \
		--url https://api.perplexity.ai/chat/completions \
		--header 'accept: application/json' \
		--header "authorization: Bearer $PERPLEXITY_API" \
		--header 'content-type: application/json' \
		--data @- | jq --unbuffered --raw-input -j 'gsub("^data: "; "") | gsub("\r$"; "") | select(. != null and . != "") | fromjson | .choices[0].delta.content'
}

Shipped some improvements by @TooTallNate

Does this work well for both of you?

Yes. @nwaughachukwuma did you run into issues?

Ok @rauchg. I wanted to be sure it worked well for you both. I modified the code above and used together-api, and wanted to share.

#!/usr/bin/env bash

function together() {
  local body_json=$(jq -n \
    --arg prompt "[INST]$1[/INST]"  \
    '{
      "model": "mistralai/Mixtral-8x7B-Instruct-v0.1",
      "max_tokens": 512,
      "prompt": $prompt,
      "request_type": "language-model-inference",
      "temperature": 0.7,
      "top_p": 0.7,
      "top_k": 50,
      "repetition_penalty": 1,
      "stream_tokens": true,
      "stop": [
        "[/INST]",
        "</s>",
        "<|im_end|>",
        "<|im_start|>"
      ]
    }')

  # Make the API call and process the output line by line
  while IFS= read -r line; do
    if [[ $line == data:* ]]; then
      # Remove the 'data:' prefix
      json_line=${line#data: }

      if echo "$json_line" | jq empty 2> /dev/null; then
        output=$(echo "$json_line" | jq -r '.choices[0].text')
        
        if [[ $output == 'null' ]]; then
          continue
        fi
        printf "%s" "$output"
      else
        continue
      fi
    fi

  done < <(curl --silent \
    -X POST https://api.together.xyz/v1/completions \
    -H 'accept: application/json' \
    -H 'Content-Type: application/json' \
    -H "Authorization: Bearer $TOGETHER_API_KEY" \
    -d "$body_json"
    )

  echo ""
}

## MAIN
together "$1"

@nwaughachukwuma We modified it to not use read and instead use a singular jq process to parse the output. This ended up being a lot faster.

@nwaughachukwuma We modified it to not use read and instead use a singular jq process to parse the output. This ended up being a lot faster.

@TooTallNate Could you give us the whole script, please 🥺 🙏 ?

Not sure what you mean. It's up at the top. @rauchg updated the gist already.

Is there a version -- or another tool -- that provides CLI access without registration? (Like running as "Unregistered" in a browser)