Mongo replset in docker. Python access and verify Mongo replset.

gistfile1.txt

mongo replica set 3.4.4 local docker engine (bridge) 

Cases of accessing MongoDB replica set using PyMongo

As reference for upgrading existing MongoDB settings and application code to access MongoDB Replica set

Using PyMongo 3.4. It supports MongoDB 3.4

$ python  -c "import sys; print(sys.version)"
3.5.2 (default, Nov 17 2016, 17:05:23) 
[GCC 5.4.0 20160609]
$ python  -c "import pymongo; print(pymongo.version); print(pymongo.has_c())"
3.4.0
True

Reference case for update Settings and application code

Case1: Access AWS MongoDB replica set from EC2 'sshhop'

Access db 'foo'

(bzuvenv) ubuntu@ip-172-31-8-191:~/tmp$ python
Python 3.5.2 (default, Nov 17 2016, 17:05:23) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pymongo
>>> primary_private_dns = 'ip-172-31-4-46.us-west-2.compute.internal'
>>> secondary_1_private_dns = 'ip-172-31-13-173.us-west-2.compute.internal'
>>> replica_set_name = 'ft-mongo-replica-set'
>>> uri = 'mongodb://{},{}/?replicaSet={}'.format(primary_private_dns, secondary_1_private_dns,replica_set_name)
>>> client = pymongo.MongoClient(uri)
>>> db = client.foo
>>> db.name
'foo'

Operation on db 'foo'

>>> db.my_collection.insert_one({"x":10}).inserted_id
ObjectId('5927b623ebe8ad792c83d5c3')
>>> db.my_collection.find_one()
{'x': 10, '_id': ObjectId('5927b623ebe8ad792c83d5c3')}

Check replica set nodes and the one the client is connecting

>>> print(client.nodes)
frozenset({('ip-172-31-13-173.us-west-2.compute.internal', 27017), ('ip-172-31-13-124.us-west-2.compute.internal', 27017), ('ip-172-31-4-46.us-west-2.compute.internal', 27017), ('ip-172-31-2-3.us-west-2.compute.internal', 27017)})
>>> db.client.address
('ip-172-31-4-46.us-west-2.compute.internal', 27017)

Check read preference and change the db 'foo' read preference to be ReadPreference.SECONDARY

>>> client.read_preference
Primary()
>>> from pymongo import ReadPreference
>>> db = client.get_database('foo', read_preference=ReadPreference.SECONDARY)
>>> db.read_preference
Secondary(tag_sets=None, max_staleness=-1)
>>>

Case2: Access local docker dev environment MongoDB replica set from local 'worker' Docker container

Check 'worker' Docker container Python and Pymongo version

$ docker exec -it $(docker ps | grep worker | cut -d" " -f 1) python
Python 3.5.3 (default, May 11 2017, 22:09:56) 
[GCC 4.9.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pymongo
>>> pymongo.version
'3.4.0'

Access Mongo replica set

>>> from pymongo  import MongoClient, ReadPreference
>>> uri = 'mongodb://{},{}/?replicaSet={}'.format('mongo_primary', 'mongo_secondary_1', 'ft-mongo-replica-set')
>>> client = MongoClient(uri, read_preference=ReadPreference.PRIMARY_PREFERRED)
>>> client.nodes
frozenset({('mongo_arbiter', 27017), ('mongo_secondary_2', 27017), ('mongo_primary', 27017), ('mongo_secondary_1', 27017)})
>>> db = client.foo
>>> db.name
'foo'
>>> db.client.address
('mongo_primary', 27017)

Common operations

>>> db.test_collection.insert_one({"bruce":1}).inserted_id
ObjectId('5928876a9559d4006ed890da')
>>> db.test_collection.find_one()
{'_id': ObjectId('592887609559d4006ed890d9'), 'bruce': 1}
>>> 

Application code need care

1> ConnectionFailure

Check if the server is available like this:

from pymongo.errors import ConnectionFailure
client = MongoClient()
try:
    # The ismaster command is cheap and does not require auth.
    client.admin.command('ismaster')
except ConnectionFailure:
    print("Server not available")

2> AutoReconnect exception

"When this exception is raised our application code needs to decide whether to retry the operation or to simply continue, accepting the fact that the operation might have failed.

On subsequent attempts to run the query we might continue to see this exception. Eventually, however, the replica set will failover and elect a new primary (this should take no more than a couple of seconds in general). At that point the driver will connect to the new primary and the operation will succeed:"

3> Secondary Reads

options of MongoClient parameters: Read Preference:

• primary,
• primaryPreferred,
• secondary,
• secondaryPreferred,
• nearest.

Defaults to primary. To use secondaries for queries we have to change the read preference:

(bzuvenv) ubuntu@ip-172-31-8-191:~/tmp$ python
Python 3.5.2 (default, Nov 17 2016, 17:05:23) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pymongo import MongoClient, ReadPreference
>>> uri = 'mongodb://{},{}/?replicaSet={}'.format('ip-172-31-4-46.us-west-2.compute.internal', 'ip-172-31-13-173.us-west-2.compute.internal', 'ft-mongo-replica-set')
>>> client = MongoClient(uri, read_preference=ReadPreference.PRIMARY_PREFERRED)
>>> client.read_preference
PrimaryPreferred(tag_sets=None, max_staleness=-1)
>>> 

Secondary Reads

Reference
http://api.mongodb.com/python/current/api/pymongo/mongo_client.html

=================
on sshhop prepare python environment same as the test environment : 3.5

$ sudo apt install virtualenv
$ pip install --upgrade virtualenv
$ pip install --upgrade pip
$ sudo apt install python3-venv

Reference
http://api.mongodb.com/python/current/examples/index.html
http://api.mongodb.com/python/current/api/pymongo/mongo_client.html
https://github.com/mongodb/mongo-python-driver?jmp=docs
https://stackoverflow.com/questions/22035257/django-non-rel-connecting-to-multiple-hosts-in-a-replica-set

connect mongo arbiter in docker

$ docker exec -it forticloudplatform_mongo_1    mongo mongodb://forticloudplatform_mongo_arbiter_1

run python from local 'worker' Docker container

$ docker exec -it $(docker ps | grep worker | cut -d" " -f 1) python

find out all place where the code need update: configuration and code

(.venv) bzu@bruce-laptop:~/project/FortiCloudPlatform$ grep -Rr "MONGODB" .  --exclude-dir=".venv" --exclude-dir=".idea" --exclude-dir=".git"