-
Committer:
Tarmac
-
Author(s):
Brad Crittenden
-
Date:
2014-03-19 12:59:41 UTC
-
mfrom:
(494.1.2 mongo-client)
-
Revision ID:
tarmac-20140319125941-iu0wiks8dcxehn92
Improve pymongo connection management.
Despite code in MongoTestBase to try to intelligently manage mongo
connections, sockets were being created and left open at a rate of about one
per test. A default system has a limit of 1024 open connections. As our test
suite exceeded that number failures were seen.
Adding a call to 'conn.disconnect' in tearDown did not help.
Research showed that we are using the deprecated call pymongo.Connection() to
create our connections. The preferred connection class is now
pymongo.MongoClient.
This branch converts all connection instantiation to use the new method.
By itself, this change was not very helpful. But, when the call to
'disconnect' was added, the result was better. Now the full suite uses about
250 open descriptors.
It is unclear why are all of these sockets are still open. Some are shown to
be ESTABLISHED while many are in CLOSE_WAIT.
QA:
Run the tests in one window.
In another, find the pid of the test runner.
% ps auxww |grep nosetest
Then, keep an eye on the number of open file descriptors with
% watch 'lsof -p 39007 -Ff | tail -10'
Be careful as the results will be erased as soon as the tests finish. You
should see something less than 300.
https://codereview.appspot.com/77700043/
R=bac, rharding.
Approved by Juju Gui Bot.