Azure Search Export

azure-search-export.sh

#! /bin/bash

# This script will export the json contents of an Azure Search instance into a JSON array.
# The script creates local files under the directory it is executed. The result is saved to a newly created local file.
# The script depends on `curl` and `jq` utilities.
# Arguments: $1 : azure search service name, $2: azure search index name, $3: azure search admin auth key.

set -e -o pipefail

serviceName="$1"
indexName="$2"
adminKey="$3"
apiVersion="2015-02-28"
link="https://${serviceName}.search.windows.net/indexes/${indexName}/docs?api-version=${apiVersion}&search=*"

while [ -n "$link" ]; 
do
    echo "getting ${link}"
    curl "${link}" -s \
    -H "Accept: application/json, text/plain, */*" \
    -H "api-key: ${adminKey}" \
    > ./tmp

    link=$(jq -r '.["@odata.nextLink"] // empty' ./tmp)
    jq -r '.value[]' ./tmp >> ./agg
done
rm ./tmp
jq '[inputs]' ./agg > ./res
rm ./agg
echo 'done! result at ./res'

@itaysk Would you know how to do this in python ? I want to do this midway through my pipeline

Note to self: you may need to run

echo "nameserver 8.8.8.8" | sudo tee /etc/resolv.conf > /dev/null

when using WSL2

Legend, thanks! Worked like a charm.