Skip to content

Instantly share code, notes, and snippets.

@fbatroni
Created January 27, 2015 17:37
Show Gist options
  • Save fbatroni/bf47200fb8bd3fb208bb to your computer and use it in GitHub Desktop.
Save fbatroni/bf47200fb8bd3fb208bb to your computer and use it in GitHub Desktop.
Dynamically add mongo collections to ElasticSearch mongodb river
#!/bin/bash
PWD=`pwd`
echo "configuring es_river(s)"
# return a list of collections except the system tables
collections=`mongo --host ${MONGODB_SERVER} --username ${MONGODB_USER} --password ${MONGODB_PASSWORD} ${MONGODB_DATABASE} --eval 'db.getCollectionNames().filter(function (c) { return /^((?!system).)*$/.test(c) && /^((?!sessions).)*$/.test(c) })' | sed -n '3p' | sed 's/,/ /g'`
echo "collections: ${collections}"
echo "configuring indices"
for collection_name in $collections
do
read -r -d '' CONFIG <<-EOF
{ \
"type": "mongodb", \
"mongodb": { \
"servers": [ \
{ \
"host": "${MONGODB_SERVER}", \
"port": "27017" \
} \
], \
"db": "${MONGODB_DATABASE}", \
"options": { \
"drop_collection": "true" \
}, \
"collection": "${collection_name}", \
"credentials": [ \
{ \
"db": "admin", \
"user": "${MONGODB_ADMIN_USER}", \
"password": "${MONGODB_ADMIN_PASSWORD}" \
} \
] \
}, \
"index": { \
"name": "${MONGODB_DATABASE}", \
"type": "${collection_name}" \
} \
}
EOF
echo "curl -XPUT 'http://localhost:9200/${MONGODB_DATABASE}/$collection_name/'"
echo curl -XPUT "http://localhost:9200/${MONGODB_DATABASE}/${collection_name}/"
echo "configuring river for collection: ${collection_name}"
echo $CONFIG > $PWD/es_river.tmp
echo "curl -XPOST -H \"Content-Type: application/json\" \"http://localhost:9200/_river/${collection_name}/_meta\" -d @${PWD}/es_river.tmp"
curl -XPOST -H "Content-Type: application/json" "http://localhost:9200/_river/${collection_name}/_meta" -d @${PWD}/es_river.tmp
echo "\n"
rm -f $PWD/es_river.tmp
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment