-
-
Save BruceZu/6054cc22ad8a298999013f3edee14330 to your computer and use it in GitHub Desktop.
| mongo replica set 3.4.4 local docker engine (bridge) |
BruceZu
commented
May 25, 2017
•
Cases of accessing MongoDB replica set using PyMongo
As reference for upgrading existing MongoDB settings and application code to access MongoDB Replica set
Using PyMongo 3.4. It supports MongoDB 3.4
$ python -c "import sys; print(sys.version)"
3.5.2 (default, Nov 17 2016, 17:05:23)
[GCC 5.4.0 20160609]
$ python -c "import pymongo; print(pymongo.version); print(pymongo.has_c())"
3.4.0
True
Reference case for update Settings and application code
Case1: Access AWS MongoDB replica set from EC2 'sshhop'
Access db 'foo'
(bzuvenv) ubuntu@ip-172-31-8-191:~/tmp$ python
Python 3.5.2 (default, Nov 17 2016, 17:05:23)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pymongo
>>> primary_private_dns = 'ip-172-31-4-46.us-west-2.compute.internal'
>>> secondary_1_private_dns = 'ip-172-31-13-173.us-west-2.compute.internal'
>>> replica_set_name = 'ft-mongo-replica-set'
>>> uri = 'mongodb://{},{}/?replicaSet={}'.format(primary_private_dns, secondary_1_private_dns,replica_set_name)
>>> client = pymongo.MongoClient(uri)
>>> db = client.foo
>>> db.name
'foo'Operation on db 'foo'
>>> db.my_collection.insert_one({"x":10}).inserted_id
ObjectId('5927b623ebe8ad792c83d5c3')
>>> db.my_collection.find_one()
{'x': 10, '_id': ObjectId('5927b623ebe8ad792c83d5c3')}Check replica set nodes and the one the client is connecting
>>> print(client.nodes)
frozenset({('ip-172-31-13-173.us-west-2.compute.internal', 27017), ('ip-172-31-13-124.us-west-2.compute.internal', 27017), ('ip-172-31-4-46.us-west-2.compute.internal', 27017), ('ip-172-31-2-3.us-west-2.compute.internal', 27017)})
>>> db.client.address
('ip-172-31-4-46.us-west-2.compute.internal', 27017)Check read preference and change the db 'foo' read preference to be ReadPreference.SECONDARY
>>> client.read_preference
Primary()
>>> from pymongo import ReadPreference
>>> db = client.get_database('foo', read_preference=ReadPreference.SECONDARY)
>>> db.read_preference
Secondary(tag_sets=None, max_staleness=-1)
>>>Case2: Access local docker dev environment MongoDB replica set from local 'worker' Docker container
Check 'worker' Docker container Python and Pymongo version
$ docker exec -it $(docker ps | grep worker | cut -d" " -f 1) python
Python 3.5.3 (default, May 11 2017, 22:09:56)
[GCC 4.9.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pymongo
>>> pymongo.version
'3.4.0'Access Mongo replica set
>>> from pymongo import MongoClient, ReadPreference
>>> uri = 'mongodb://{},{}/?replicaSet={}'.format('mongo_primary', 'mongo_secondary_1', 'ft-mongo-replica-set')
>>> client = MongoClient(uri, read_preference=ReadPreference.PRIMARY_PREFERRED)
>>> client.nodes
frozenset({('mongo_arbiter', 27017), ('mongo_secondary_2', 27017), ('mongo_primary', 27017), ('mongo_secondary_1', 27017)})
>>> db = client.foo
>>> db.name
'foo'
>>> db.client.address
('mongo_primary', 27017)Common operations
>>> db.test_collection.insert_one({"bruce":1}).inserted_id
ObjectId('5928876a9559d4006ed890da')
>>> db.test_collection.find_one()
{'_id': ObjectId('592887609559d4006ed890d9'), 'bruce': 1}
>>> Application code need care
1> ConnectionFailure
Check if the server is available like this:
from pymongo.errors import ConnectionFailure
client = MongoClient()
try:
# The ismaster command is cheap and does not require auth.
client.admin.command('ismaster')
except ConnectionFailure:
print("Server not available")2> AutoReconnect exception
"When this exception is raised our application code needs to decide whether to retry the operation or to simply continue, accepting the fact that the operation might have failed.
On subsequent attempts to run the query we might continue to see this exception. Eventually, however, the replica set will failover and elect a new primary (this should take no more than a couple of seconds in general). At that point the driver will connect to the new primary and the operation will succeed:"
3> Secondary Reads
options of MongoClient parameters: Read Preference:
- primary,
- primaryPreferred,
- secondary,
- secondaryPreferred,
- nearest.
Defaults to primary. To use secondaries for queries we have to change the read preference:
(bzuvenv) ubuntu@ip-172-31-8-191:~/tmp$ python
Python 3.5.2 (default, Nov 17 2016, 17:05:23)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pymongo import MongoClient, ReadPreference
>>> uri = 'mongodb://{},{}/?replicaSet={}'.format('ip-172-31-4-46.us-west-2.compute.internal', 'ip-172-31-13-173.us-west-2.compute.internal', 'ft-mongo-replica-set')
>>> client = MongoClient(uri, read_preference=ReadPreference.PRIMARY_PREFERRED)
>>> client.read_preference
PrimaryPreferred(tag_sets=None, max_staleness=-1)
>>> Reference
http://api.mongodb.com/python/current/api/pymongo/mongo_client.html
=================
on sshhop prepare python environment same as the test environment : 3.5
$ sudo apt install virtualenv
$ pip install --upgrade virtualenv
$ pip install --upgrade pip
$ sudo apt install python3-venv
Reference
http://api.mongodb.com/python/current/examples/index.html
http://api.mongodb.com/python/current/api/pymongo/mongo_client.html
https://github.com/mongodb/mongo-python-driver?jmp=docs
https://stackoverflow.com/questions/22035257/django-non-rel-connecting-to-multiple-hosts-in-a-replica-set
connect mongo arbiter in docker
$ docker exec -it forticloudplatform_mongo_1 mongo mongodb://forticloudplatform_mongo_arbiter_1run python from local 'worker' Docker container
$ docker exec -it $(docker ps | grep worker | cut -d" " -f 1) pythonfind out all place where the code need update: configuration and code
(.venv) bzu@bruce-laptop:~/project/FortiCloudPlatform$ grep -Rr "MONGODB" . --exclude-dir=".venv" --exclude-dir=".idea" --exclude-dir=".git"