11

If I call boto.connect_xxx, where xxx is some service (dynamodb, s3, etc) multiple times, does it create a new connection pool each time? What I'd like to do is something like this (example in Flask):

@app.before_request
def before_request():
    g.db = connect_dynamodb()

to make sure I always connect, but I don't want to do this before each request if it will create new security tokens, etc, the whole rigamarole, each time. Is it safe to just call connect_xxx() once when the application starts, and rely on boto to generate new connections as needed, etc.?

1 Answer 1

12

The best approach is to call the connect_xxx method once when your application starts and rely on boto to manage the connection from then on. The only exception to that rule is if you are using multiple threads. In that case, each thread should create it's own connection since boto uses httplib which is not threadsafe.

Even if you call the connect_xxx method before each request, you really should be okay. Boto pools connection at the class level and should handle this in a pretty efficient manner.

5
  • 1
    Thanks for the answer! And thanks for boto itself, I couldn't live without it. Your mention of threads prompted another question on essentially the same topic, here. Commented Jul 24, 2012 at 22:08
  • Are there any documentation references available for this information? Commented Aug 25, 2014 at 16:42
  • The connect_xxx methods return an instance of a subclass of AWSAuthConnection which is indeed thread-safe. You should only do this once per process (and connection arguments) – not in every thread. The connection pool is not stored on the class level.
    – malthe
    Commented Sep 18, 2014 at 11:34
  • @malthe Got any references to point to about that (AWSAuthConnection being thread-safe)? Commented Nov 10, 2014 at 11:13
  • 2
    @TuukkaMustonen since this patch the connection object uses a pool such that each connection is used only by a single thread at a time.
    – malthe
    Commented Nov 10, 2014 at 12:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.