Perplexity CLI in pure shell

p.sh

#!/usr/bin/env bash
function p() {
	jq -n \
		--arg content "$*" \
		'{
      "model": "pplx-7b-online",
      "messages": [
        {
          "role": "system",
          "content": "Be precise and concise."
        },
        {
          "role": "user",
          "content": $content
        }
      ],
      "stream": true
    }' | curl --silent \
		--request POST \
		--url https://api.perplexity.ai/chat/completions \
		--header 'accept: application/json' \
		--header "authorization: Bearer $PERPLEXITY_API" \
		--header 'content-type: application/json' \
		--data @- | jq --unbuffered --raw-input -j 'gsub("^data: "; "") | gsub("\r$"; "") | select(. != null and . != "") | fromjson | .choices[0].delta.content'
}

yep... as follows:

curl --silent [CURL STUFF] |
jq -j 'gsub("^data: "; "") | .choices[0].delta.content'

The gsub function can be used to globally substitute the "data:" prefix with an empty string on each line.

Gotcha, it's not there in the original snippet, so I'm curious about how it was working.

In all fairness, I didn't actually try the updated version against the actual API! 🙂

I've made some changes to produce a version that works for me. The changes are as follows. First, saves the request and response in intermediate local variables instead of piping them, so you can debug by just printing them. Second, does not ask the endpoint to stream and take the sixth output (?), but just takes one result. Third, does not just define the function p but also invokes it, so that the resulting script itself can be set to executable and invoked. Fourth, adds a small usage message if it is called without a command line argument.

#!/usr/bin/env bash
if [ "$#" -eq 0 ]; then
    echo "Usage: $(basename $0) promt_to_send_to_perplexity"
    echo ""
    echo "  Requirements: jq must be installed, and PERPLEXITY_API defined"
    exit 1
fi

function p() {
    local json_request
    json_request=$(jq -n \
		--arg content "$*" \
		'{
      "model": "pplx-7b-online",
      "messages": [
        { "role": "system",  "content": "Be precise and concise." },
        { "role": "user",  "content": $content }
      ],
      "stream": false
    }')
    # echo $json_request # uncomment to debug
    
    local json_response
    json_response=$(curl --silent \
		--request POST \
		--url https://api.perplexity.ai/chat/completions \
		--header 'accept: application/json' \
		--header "authorization: Bearer $PERPLEXITY_API" \
		--header 'content-type: application/json' \
		--data "$json_request")
    # echo $json_response  # uncomment to debug

    echo "$json_response" | jq --raw-output .choices[0].message.content
}
p "$*"

Shipped some improvements by @TooTallNate

Does this work well for both of you?

Yes. @nwaughachukwuma did you run into issues?

Ok @rauchg. I wanted to be sure it worked well for you both. I modified the code above and used together-api, and wanted to share.

#!/usr/bin/env bash

function together() {
  local body_json=$(jq -n \
    --arg prompt "[INST]$1[/INST]"  \
    '{
      "model": "mistralai/Mixtral-8x7B-Instruct-v0.1",
      "max_tokens": 512,
      "prompt": $prompt,
      "request_type": "language-model-inference",
      "temperature": 0.7,
      "top_p": 0.7,
      "top_k": 50,
      "repetition_penalty": 1,
      "stream_tokens": true,
      "stop": [
        "[/INST]",
        "</s>",
        "<|im_end|>",
        "<|im_start|>"
      ]
    }')

  # Make the API call and process the output line by line
  while IFS= read -r line; do
    if [[ $line == data:* ]]; then
      # Remove the 'data:' prefix
      json_line=${line#data: }

      if echo "$json_line" | jq empty 2> /dev/null; then
        output=$(echo "$json_line" | jq -r '.choices[0].text')
        
        if [[ $output == 'null' ]]; then
          continue
        fi
        printf "%s" "$output"
      else
        continue
      fi
    fi

  done < <(curl --silent \
    -X POST https://api.together.xyz/v1/completions \
    -H 'accept: application/json' \
    -H 'Content-Type: application/json' \
    -H "Authorization: Bearer $TOGETHER_API_KEY" \
    -d "$body_json"
    )

  echo ""
}

## MAIN
together "$1"

@nwaughachukwuma We modified it to not use read and instead use a singular jq process to parse the output. This ended up being a lot faster.

@nwaughachukwuma We modified it to not use read and instead use a singular jq process to parse the output. This ended up being a lot faster.

@TooTallNate Could you give us the whole script, please 🥺 🙏 ?

Not sure what you mean. It's up at the top. @rauchg updated the gist already.

Is there a version -- or another tool -- that provides CLI access without registration? (Like running as "Unregistered" in a browser)