Skip to content

Instantly share code, notes, and snippets.

View alexbevi's full-sized avatar
🙀

Alex Bevilacqua alexbevi

🙀
View GitHub Profile
@alexbevi
alexbevi / measureIndexUsage.js
Last active March 23, 2020 20:13
Measure the index utilization for all collections in a database before/after an operation
/**
* measureIndexUsage(block)
*
* Measure the index utilization before/after a block is run. The block can contain
* one to many operations, however the index utilization will only measure the collections
* associated with the current `db` instance.
*
* NOTE: this script does not "isolate" the operation(s) run so if running in a busy
* environment the results may represent other operations index utilization.
*
@alexbevi
alexbevi / getBackingFilenames.js
Created March 23, 2020 15:22
Return a summarized list of all collection and index uris for a MongoDB cluster
/**
* getBackingFilenames(filter)
*
* Function to return a summarized list of all collections and indexes within all databases.
* Passing *filter* will optionally filter the results by either the collection uri or
* the uri of any of the indexes within a collection.
*
* @samples
* // unfiltered
* printjson(getBackingFilenames());
// Print index details
print(["Namespace", "Index Name", "Index Size", "Index Keys", "Usage Count", "Last Used"].join('\t'));
db.getMongo().getDBNames().forEach(function (dbname) {
db.getSiblingDB(dbname).getCollectionNames().forEach(function (cname) {
var coll = db.getSiblingDB(dbname).getCollection(cname);
var stats = coll.stats();
coll.aggregate([ { $indexStats: {} } ]).forEach(function(ix) {
var ixname = ix.name;
var ns = dbname + "." + cname;
var ixsize = stats.indexSizes[ixname];
/*
* Print storage details for all collections and indexes.
* Supports sharded clusters
*
* @author [email protected]
* @version 1.3
* @updated 2022-11-21
*
* History:
* 1.3 - Filter out admin, local and config databases
{
"set": "replset",
"date": ISODate("2019-12-04T05:12:52.835Z"),
"myState": 5,
"term": NumberLong(3),
"syncingTo": "m2.example.net:27017",
"syncSourceHost": "m2.example.net:27017",
"syncSourceId": 1,
"heartbeatIntervalMillis": NumberLong(2000),
"majorityVoteCount": 2,
@alexbevi
alexbevi / initialSyncProgress.js
Created February 14, 2020 15:59
Script to help identify the progress of an intial sync of a MongoDB SECONDARY node
/*
* initialSyncProgress
* @author Alex Bevilacqua <[email protected]>
*
* Can be run against a MongoDB 3.4+ mongod that is in STARTUP2 (intitial sync) state to gain some
* insight into how the sync is progressing. This script WILL NOT tell you how long until the sync
* is complete, but based on how the script reports progress can be used to estimate this.
*
* usage:
* mongo --quiet --eval "load('initialSyncProgress.js'); initialSyncProgress();"

This script uses bsondump (twice), jq (twice) and ruby

bsondump --quiet metrics.2019-12-22T17-34-22Z-00000 | jq -s '.[] | select( .type | ."$numberInt" == "1")' | jq -s 'first | .data ."$binary" .base64' -Mc | ruby -rzlib -rbase64 -e 'd = STDIN.read; print Zlib::Inflate.new.inflate(Base64.decode64(d)[4..-1])' | bsondump --quiet | jq | less
@alexbevi
alexbevi / README.md
Last active February 24, 2020 14:29
Variation of collStats command that can be run in a sharded environment through the Mongo shell and assemble the result in Javascript to bypass BSON size limitations

As a result of SERVER-44891 which was closed as Won't Fix I created this workaround. This script can be verified as follows using mtools, m and mgeneratejs:

First, initialize the cluster:

mkdir shardedCluster
m 4.0.9
mlaunch init --replicaset --nodes 1 --sharded 60 --csrs --binarypath $(m bin 4.0.9) --dir shardedCluster --auth
require 'open-uri'
require 'nokogiri'
GITHUB_URL = "https://github.com/%s/network/dependents"
repos = [
{ name: "Ruby Driver", repo: "mongodb/mongo-ruby-driver" },
{ name: "Mongoid ODM", repo: "mongodb/mongoid" },
{ name: "C Driver", repo: "mongodb/mongo-c-driver" },
{ name: "C++ Driver", repo: "mongodb/mongo-cxx-driver" },
@alexbevi
alexbevi / template.json
Last active May 31, 2021 19:28
Mgeneratejs template
{
"a": {
"$integer": {
"min": 1,
"max": 100
}
},
"b": {
"$integer": {
"min": 1,