Bug #2800
closed
[Crunch] Remove global state in Python SDK
Description
It should be easy for a Python program to communicate with API and Keep storage servers at multiple sites in one process. For example:
a = arvados.api('v1', host='localhost', token='12345', insecure=True)
b = arvados.api('v1', host='qr1hi.arvadosapi.com', token='xyzzy')
x = a.collections().list().execute()
w1 = CollectionWriter(client=a)
w2 = CollectionWriter(client=b)
w1.write('foo')
w2.write('bar')
Updated by Tom Clegg almost 10 years ago
- Target version set to Arvados Future Sprints
Updated by Tom Clegg over 9 years ago
- Status changed from New to In Progress
- Assigned To set to Tom Clegg
Updated by Tom Clegg over 9 years ago
On 2800-pgs branch at c5f8a89
Interface- Accept host, token, and insecure arguments in the
api()call. If provided, use them rather than looking up environment variables.- This change obsoletes some awkward maneuvers with environment variables (some of this removed from
tests/test_api.py).
- This change obsoletes some awkward maneuvers with environment variables (some of this removed from
arvados.api()no longer returns a cached connection object with different credentials, endpoint, or "insecure" flag than the connection being requested. (Caching code used to assume only the requested version could change between calls toapi().)- Example: this change makes it unnecessary for
tests/test_api.pyto clear the connection cache to prevent accidental future use of its fake credentials. Removed.
- Example: this change makes it unnecessary for
- If the caller says
cache=False, don't use an existing connection (as before), but don't put the newly created connection object in the cache either.
- The credentials supplied to (or looked up by)
arvados.api()are used for the life of the connection it returns. Previously, CredentialsFromEnv looked up the auth token in the global os.environ during each API method call.
test_api.pyuses a mock for API requests but still needs a discovery document. Now it usesrun_test_server.pyinstead of relying on qr1hi.arvadosapi.com. This makes the test suite slower but more meaningful.
Updated by Tom Clegg over 9 years ago
- Target version changed from Arvados Future Sprints to 2014-08-27 Sprint
(The issue of using Keep services at multiple sites (global_client_object in keep.py) hasn't been addressed yet.)
Updated by Brett Smith over 9 years ago
Just a few small comments on c5f8a89
- We can skip the step of hashing strings for a cache key. We can just create a tuple of all the values we care about, and use that directly as the lookup key. What do you think of that approach?
- Please keep imports sorted alphabetically.
- Please retain the comment in
test_api.pyabout needing a real discovery document, to explain why we're running a test server.
Thanks.
Updated by Tom Clegg over 9 years ago
Brett Smith wrote:
- We can skip the step of hashing strings for a cache key. We can just create a tuple of all the values we care about, and use that directly as the lookup key. What do you think of that approach?
Ah, yes, I was limiting myself to string keys for no reason. Now using tuple as dictionary key.
- Please keep imports sorted alphabetically.
Fixed (by not importing hashlib)
- Please retain the comment in
test_api.pyabout needing a real discovery document, to explain why we're running a test server.
Indeed. Restored.
Thanks... now at 1e36870
Updated by Brett Smith over 9 years ago
Tom Clegg wrote:
Brett Smith wrote:
- Please keep imports sorted alphabetically.
Fixed (by not importing hashlib)
This is still an issue in test_api.py. Everything else looks good, so please go ahead and merge with that change. Thanks.
Updated by Brett Smith over 9 years ago
- Assigned To changed from Tom Clegg to Brett Smith
2800-pysdk-no-global-keep-client-wip is up for review to deprecate the global Keep client.
Updated by Brett Smith over 9 years ago
- Status changed from In Progress to Resolved
- % Done changed from 33 to 100
Applied in changeset arvados|commit:05bea2c50474edeb9d0e3fb8daaf838b58ea9a54.
Updated by Ward Vandewege over 9 years ago
- Subject changed from Remove global state in Python SDK to [Crunch] Remove global state in Python SDK