Final: Question 1: Distributed vs. Co-Located Replica Sets
For the following problem, you would like to use a 3-member replica set, and are considering the following deployment options:
Option 1:
All 3 servers are co-located in a data center in New York Option 2:
The primary is located in New York One secondary is located in Beijing One secondary is located in Paris What advantages/disadvantages will affect the dispersed replica set in option 2, relative to the single data center scenario in Option 1? Check all that apply.
You will have more read preference options Your writes using j : true will be faster () Clients far from New York are likely to find lower-latency round trips to servers when using $nearest In the absence of network issues, your secondaries' oplog will stay less stale () Your deployment will remain available if a single data center goes down
Final: Question 2: Config Servers
In this assignment, you will fix a problem that sometimes arises with config servers in a sharded cluster.
Start by running:
$ mongo --nodb
cluster = new ShardingTest( { shards: 1 , rs : { nodes : [ { } ] }, config : 3 } ) Next, click the "Initialize" button in MongoProc. This will create the problem you will need to fix.
You can see the issue if you try to split a chunk in the 'testDB.testColl' collection with:
db.adminCommand( { split : "testDB.testColl", middle : { _id : 7 } } ) This will fail. Your job is to track down the problem and solve it. When you think you've got it, test it with MongoProc and submit when finished.
Hint: You will almost certainly need to spin up your own config server separately from the ShardingTest. Make sure you use all of the same parameters used by ShardingTest when starting config servers. You can see these parameters by typing
ps ax | grep mongo in a Linux shell.
Final: Question 3: Replica Set
Suppose you have a 3 node replica set where 2 of the nodes are in a data center that goes down and stays down indefinitely. Your job is to bring your third server to a place where it can accept writes.
For this problem, we'll only need a replica set.
Start by running the following lines and waiting for each to complete.
Note: You must wait for rst.startSet() to finish before executing rst.initiate(). You will know when it is finished when the log shows: "[rsStart] replSet can't get local.system.replset config from self or any seed (EMPTYCONFIG)".
If you are seeing log messages appear you can still paste in each command.
$ mongo --nodb
options = { nodes: 3, startPort: 27017, nodeOptions: { smallfiles: "", noprealloc: "", nopreallocj: "" } }; rst = new ReplSetTest(options); rst.startSet(); rst.initiate(); Then click "Initialize" in MongoProc. This will tear down the servers on ports 27018 and 27019. Your job is to get the server on 27017 up, running, and able to receive writes.
Please do not spin up a server on localhost:27018 or localhost:27019 or access anything in the directories where those servers pointed before they were shut down. We want to simulate a situation where a data center disappears and it is impossible to access anything there. Generally, we prefer to give you a setup where we can eliminate "wrong" ways of solving a problem, but that is difficult to do in this case.
Final: Question 4: Fixing a Bad Shard Key
Suppose you have a sharded collection final_exam.m202 in a cluster with 3 shards. To set up the problem, first initialize a cluster with 3 single-server shards (not replica sets) in the mongo shell using ShardingTest:
$ mongo --nodb
config = { d0 : { smallfiles : "", noprealloc : "", nopreallocj : ""}, d1 : { smallfiles : "", noprealloc : "", nopreallocj : "" }, d2 : { smallfiles : "", noprealloc : "", nopreallocj : ""}}; cluster = new ShardingTest( { shards : config } ); Then click "Initialize" in MongoProc, which will verify that you have 3 shards and then insert test data into final_exam.m202.
Unfortunately, even though your shard key {otherID: 1} has high cardinality, the data is not being distributed well among the shards. You have decided that a hashed shard key based on the _id field would be a better choice, and want to reconfigure the collection to use this new shard key. Once you have the cluster using the new shard key for final_exam.m202, test in MongoProc and turn in when correct.
Final: Question 5: Failed replica set initiation
Please examine the attached log files. In this scenario, a replica set is initiated on one server with rs.initiate(). Once it successfully initialized, a second member was spun up and added to the replica set with rs.add().
Before it was successfully added, there was a connection problem. What was it?
A network partition One of the members went down while the second member was being added to the set The replica set names in the configs for the two members were different
- There were too many connections to one of the members The replica set config points to the wrong IP address Final: Question 6: Diagnosing an Event with Mtools
For this problem, start over with your original, unfiltered log file. It has been included again as a handout for your convenience.
The log file has an event where there was a sharp spike in incoming connections. Use mplotqueries to plot a chart that shows the connection churn in one-second intervals (buckets). Find the exact second when this connection spike begins. If you click on the first green bar of the connection spike, you can read the time printed in the shell. You can also zoom in close enough to read the second at the x-axis.
Once you've found the start of the spike, use mlogfilter to slice out a time windows from the log file, beginning at the time you found, and lasting 5 seconds. Store the matching log lines in a temporary file (by appending > tempfile.log ), or pipe it to the next command.
Use mplotqueries to create a scatter plot of the 5-second log window. Look at that highest spike you see. Click on some of the operations to get details in the shell.
What is the write concern of these operations?
{ w : 0 } { w : 1 }
- { w : 2 } { w : 3 } { w : 'majority' } { j : true }
Final: Question 7: Automatic chunk splitting
In a sharded cluster, which of the following can keep large chunks from being split as part of MongoDB's balancing process? Check all that apply.
- Frequent restarts of all mongos processes relative to the frequency of writes
- If there are not enough distinct shard key values
- If one of the config servers is down when a mongos tries to do a split If the average size of the documents in the chunk is too small If the number of shards in the cluster is too small
Final: Question 8: Traffic Imbalance in a Sharded Environment
In this problem, you have a cluster with 2 shards, each with a similar volume of data, but all the application traffic is going to one shard. You must diagnose the query pattern that is causing this problem and figure out how to balance out the traffic.
To set up the scenario, run the following commands to set up your cluster. The config document passed to ShardingTest will eliminate the disk space issues some students have seen when using ShardingTest.
mongo --nodb config = { d0 : { smallfiles : "", noprealloc : "", nopreallocj : ""}, d1 : { smallfiles : "", noprealloc : "", nopreallocj : "" } }; cluster = new ShardingTest( { shards: config } ); Once the cluster is up, click "Initialize" in MongoProc one time to finish setting up the cluster's data and configuration. If you are running MongoProc on a machine other than the one running the mongos, then you must change the host of 'mongod1' in the settings. The host should be the hostname or IP of the machine on which the mongos is running. MongoProc will use port 30999 to connect to the mongos for this problem.
Once the cluster is initialized, click the "Initialize" button in MongoProc again to simulate application traffic to the cluster for 1 minute. You may click "Initialize" as many times as you like to simulate more traffic for 1 minute at a time. If you need to begin the problem again and want MongoProc to reinitialize the dataset, drop the m202 database from the cluster and click "Initialize" in MongoProc.
Use diagnostic tools (e.g., mongostat and the database profiler) to determine why all application traffic is being routed to one shard. Once you believe you have fixed the problem and traffic is balanced evenly between the two shards, test using MongoProc and then turn in if the test completes successfully.
Note: Dwight discusses the profiler in M102.
Submitted