-
Notifications
You must be signed in to change notification settings - Fork 479
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ServerSelectionTimoutError
on Remote Replica Set
#390
Comments
I see you are using --ssl-certificate-policy optional. Is your replica set actually running with TLS enabled? If not, can you connect to it without the ssl options? I'd like to know if this is an SSL/TLS problem, or a problem connecting through your proxy. |
We're running MongoDB hosted on Compose. Our instance has SSL enabled, but we only have access to a public key. From terminal, we can connect using the |
I'm not sure I understand why you would need --ssl-ca-certs or --ssl-certificate-policy. If the compose instance you are connecting to uses TLS I would assume the server's cert is signed by a well known certificate authority. If you are using PyMongo 3.0+ ssl=true should be all you need. If this is not the case you may want to ask compose for help. |
We're using mongo-connector 2.2 which is using PyMongo 3.2.1 right now. Without Compose doesn't provide a private key for SSL unfortunately, all we have to work with is a public key. Using Mongo's command line or native Node driver, we pass this public key into an option called |
You can configure all of these options using the MongoDB URI, not just ssl=true. See the docs here: |
The second-last option on that page is pretty much what I was trying to accomplish with my client = pymongo.MongoClient('example.com',
... ssl=True,
... ssl_ca_certs='/path/to/ca.pem') The equivalent works via command line, so I'm not sure what's going on in between the CLI and the actual MongoClient call. I'll try to debug and see. |
It's suggested in this stackoverflow thread that passing the arg Does mongo-connector support passing this |
That stackoverflow question is irrelevant for this discussion. mongo-connector doesn't fork any child processes. Your problem appears to be related to TLS. |
Using PyMongo's MongoClient alone, I can connect to the database like so: from pymongo import MongoClient
host = 'mongodb://<user>:<password>@<hostname>:<port>/<database>'
cert = '/path/to/cert'
MongoClient(host, ssl_ca_certs=cert) Any tips on replicating this with mongo-connector? |
The URI we use to connect to the MongoDB server is actually for a Mongos/Haproxy router. It looks like upon connection, mongo-connector detects the replicaSet behind the Mongos router and tries to connect directly to those hosts (which isn't allowed by Compose). This may be the core issue at hand. |
That explains the problem. You won't be able to use mongo-connector. It has to be able to connect to the shards directly, since that's where the oplogs live. |
I can access the oplog while connected to the Mongos router using pymongo's MongoClient. My use case is technically possible, just not implemented in the connector right now. |
@behackett is correct. mongo-connector needs to be able to connect to each of the shards directly in order to read the local oplog on each replica set. If connecting to the shards directly isn't possible, then mongo-connector cannot function. |
I guess I'm requesting a feature then |
Hello! Unfortunately the feature you are requesting is not possible. Mongo-connector cannot run without access to the oplog of each shard, and it is not possible to access the local database of shards through the mongos. How exactly have you been able to access oplog while connected to the Mongos router using pymongo's MongoClient? |
@aherlihy my script looks like this (sensitive info redacted): import ssl
from pymongo import MongoClient
host = 'mongodb://oploguser:[email protected]:12345/local?authSource=admin'
cert = '/path/to/cert.pem'
client = MongoClient(host, ssl_ca_certs=cert)
client.admin.command('isdbgrid') # confirms we are connected to a mongos, see: https://docs.mongodb.org/manual/reference/command/isdbgrid/
print client['local']['oplog.rs'].find()
# Proceeds to print documents from the oplog collection |
What version of MongoDB are you using? This is what I get from MongoDB 3.0: pymongo.errors.OperationFailure: database error: can't use 'local' database through mongos It is not possible to access the oplogs of the shards through mongos. |
I'm using Mongo 3.2 with no issues |
Using MongoDB 3.2.4.
|
I am facing the following issue: Exception in thread Thread-2: I tried to observe using "netsata -a" if I could figure out the ports where there might be issue in communicating, but didn't get any success. Any help for resolving the issue is highly appreciated. |
Is there now a suggested workaround or a solution to @letsgolesco 's query regarding mongo-connector timing out connecting to replicaSet host directly? |
The original ticket is asking for mongo-connector to support Compose's Mongos oplog proxy. This proxy looks like a mongos but only proxies a single replica set and is specific to Compose only. It might be possible to support it by checking if the "local.oplog.rs" collection is readable through the mongos connection. If we get There might be other changes required too. For example, can mongo-connector read all collections through the proxy? @pratyus you are welcome to look into these issues and provide a pull request. The "local.oplog.rs" check shouldn't be hard to implement. See https://github.com/mongodb-labs/mongo-connector/blob/master/mongo_connector/connector.py#L362-L365. |
Thanks @ShaneHarvey. I was able to verify read access for "local.oplog.rs". Also, verified mongo-connector with my compose deployment. Works like a charm. I have sent #751 to enable this. Let me know what you think, or if I am missing anything, happy to iterate and add appropriate tests. |
Friendly ping to take a look at the PR. Thanks |
@pratyus Nice job with the fix, implemented it for my app's architecture and it works fine. |
I'm trying to connect from a remote MongoDB instance to a local Elasticsearch instance. The MongoDB instance is a replica set behind a proxy, and I'm connecting with SSL. I've tried tweaking things with varying degrees of success, the closest I've gotten is when I hit this error:
ServerSelectionTimeoutError: <ip_address_1>:27017: timed out,<ip_address_2>:27017: timed out,<ip_address_3>:27017: timed out
The 3 IP addresses it lists are correct for the replica set members, which indicates to me that it's actually connecting/authenticating properly, but some issue occurs afterward.
The command I'm using looks like this:
mongo-connector -c mongo-connector-config.json -m mongodb://<ip_address>:<port>/?ssl=true -t 192.168.99.100:9200 -d elastic_doc_manager --ssl-ca-certs my-cert.pem --ssl-certificate-policy optional
The only things I'm specifying in my config are
noDump: true
and a few namespaces.The text was updated successfully, but these errors were encountered: