Last active
November 3, 2022 16:18
-
-
Save mderazon/8201991 to your computer and use it in GitHub Desktop.
Export all of Mongodb collections as csv without the need to specify fields
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
OIFS=$IFS; | |
IFS=","; | |
# fill in your details here | |
dbname=DBNAME | |
user=USERNAME | |
pass=PASSWORD | |
host=HOSTNAME:PORT | |
# first get all collections in the database | |
collections=`mongo "$host/$dbname" -u $user -p $pass --eval "rs.slaveOk();db.getCollectionNames();"`; | |
collections=`mongo $dbname --eval "rs.slaveOk();db.getCollectionNames();"`; | |
collectionArray=($collections); | |
# for each collection | |
for ((i=0; i<${#collectionArray[@]}; ++i)); | |
do | |
echo 'exporting collection' ${collectionArray[$i]} | |
# get comma separated list of keys. do this by peeking into the first document in the collection and get his set of keys | |
keys=`mongo "$host/$dbname" -u $user -p $pass --eval "rs.slaveOk();var keys = []; for(var key in db.${collectionArray[$i]}.find().sort({_id: -1}).limit(1)[0]) { keys.push(key); }; keys;" --quiet`; | |
# now use mongoexport with the set of keys to export the collection to csv | |
mongoexport --host $host -u $user -p $pass -d $dbname -c ${collectionArray[$i]} --fields "$keys" --csv --out $dbname.${collectionArray[$i]}.csv; | |
done | |
IFS=$OIFS; |
I'm unable to retrieve the Id, because of the following error
"2022-02-08T13:30:02.767-0500 I CONTROL [thread1] machdep.cpu.extfeatures unavailable _id"
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
@adamtaiti here is code with fix for exporting dates:
keys=
mongo $dbname --host "$host" --port "$port" -u "$user" -p "$pass" --authenticationDatabase "$authdb" --ssl --eval "function z(c,e){if(c===null || c === undefined || c instanceof Date){return e};var a=[];var d=Object.keys(c);for(var f in d){var b=d[f];if(b != undefined && c != undefined && typeof c[b]==='object'){var g=[],h=z(c[b],e+'.'+b);a=g.concat(a,h);}else a.push(e+'.'+b);}return a;}var a=[],b=db['$col'].findOne({}),c=Object.keys(b);for(var i in c){var j=c[i];if(typeof b[j]==='object'&&j!='_id'){var t1=[],t2=z(b[j],j);a=t1.concat(a,t2);}else a.push(j);}a.join(',');" --quiet;