Feature #17014

Add Container Requests endpoint to controller

Added by Nico César 6 days ago. Updated 1 day ago.

Status:
In Progress
Priority:
Normal
Assigned To:
Category:
API
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:
(Total: 0.00 h)
Story points:
-

Description

In an effort to embrace container requests in controller and federation, we need to be able to have the endopoint arvados/v1/container_requests in controller.

as part of #16462 we encounter a problem with federation tests in f6ccc08c3f6b1ad42f2c827b19df0300f2c3c3db

We dedided to split out the task of doing first the container request endpoints and later doing the forcasting features (#16462 et al)


Subtasks

Task #17036: ReviewNew


Related issues

Blocks Arvados - Feature #16462: Expand arvados-controller to expose forecast featuresIn Progress

Associated revisions

Revision 70242f2d (diff)
Added by Nico Cesar 6 days ago

Initial commit with tests that are failing

this is the verbose log of failing test:

What next? test lib/controller/federation ======= test lib/controller/federation
ok git.arvados.org/arvados.git/lib/controller/federation 1.178s coverage: 48.1% of statements ======= test lib/controller/federation -- 3s
Pass: lib/controller/federation tests (3s)
All test suites passed.
What next? test lib/controller ======= test lib/controller

----------------------------------------------------------------------
FAIL: federation_test.go:590: FederationSuite.TestCreateContainerRequestBadToken

time="2020-10-16T17:27:40.958155049Z" level=info msg=request RequestID=req-33wrdjmne2bs1nwkc9z8 remoteAddr="192.0.2.1:1234" reqBytes=24 reqForwardedFor= reqHost=example.com reqMethod=POST reqPath=arvados/v1/container_requests reqQuery="cluster_id=zzzzz"
time="2020-10-16T17:27:40.958812397Z" level=info msg=response RequestID=req-33wrdjmne2bs1nwkc9z8 remoteAddr="192.0.2.1:1234" reqBytes=24 reqForwardedFor= reqHost=example.com reqMethod=POST reqPath=arvados/v1/container_requests reqQuery="cluster_id=zzzzz" respBody="{\"errors\":[\"badly formatted token\"]}\n" respBytes=37 respStatus="Internal Server Error" respStatusCode=500 timeToStatus=0.000649 timeTotal=0.000655 timeWriteBody=0.000006
federation_test.go:599:
c.Check(resp.StatusCode, check.Equals, http.StatusForbidden)
... obtained int = 500
... expected int = 403

federation_test.go:602:
c.Check(e["errors"], check.DeepEquals, []string{"invalid API token"})
... obtained []string = []string{"badly formatted token"}
... expected []string = []string{"invalid API token"}

----------------------------------------------------------------------
FAIL: federation_test.go:630: FederationSuite.TestCreateRemoteContainerRequestCheckRuntimeToken

time="2020-10-16T17:27:41.013982673Z" level=info msg=request RequestID=req-1cdwy5iwe0qw3jgxdjf0 remoteAddr="192.0.2.1:1234" reqBytes=165 reqForwardedFor= reqHost=example.com reqMethod=POST reqPath=arvados/v1/container_requests reqQuery="cluster_id=zmock"
time="2020-10-16T17:27:41.015464942Z" level=info msg=response RequestID=req-1cdwy5iwe0qw3jgxdjf0 remoteAddr="192.0.2.1:1234" reqBytes=165 reqForwardedFor= reqHost=example.com reqMethod=POST reqPath=arvados/v1/container_requests reqQuery="cluster_id=zmock" respBytes=726 respStatus=OK respStatusCode=200 timeToStatus=0.001473 timeTotal=0.001477 timeWriteBody=0.000004
federation_test.go:659:
c.Check(json.NewDecoder(s.remoteMockRequests0.Body).Decode(&cr), check.IsNil)
... value *json.SyntaxError = &json.SyntaxError{msg:"invalid character 'c' looking for beginning of value", Offset:1} ("invalid character 'c' looking for beginning of value")

federation_test.go:660:
c.Check(strings.HasPrefix(cr.ContainerRequest.RuntimeToken, "v2/zzzzz-gj3su-"), check.Equals, true)
... obtained bool = false
... expected bool = true

----------------------------------------------------------------------
FAIL: federation_test.go:664: FederationSuite.TestCreateRemoteContainerRequestCheckSetRuntimeToken

time="2020-10-16T17:27:41.020613425Z" level=info msg=request RequestID=req-12j635qfrmsbx1jludwq remoteAddr="192.0.2.1:1234" reqBytes=193 reqForwardedFor= reqHost=example.com reqMethod=POST reqPath=arvados/v1/container_requests reqQuery="cluster_id=zmock"
time="2020-10-16T17:27:41.022068784Z" level=info msg=response RequestID=req-12j635qfrmsbx1jludwq remoteAddr="192.0.2.1:1234" reqBytes=193 reqForwardedFor= reqHost=example.com reqMethod=POST reqPath=arvados/v1/container_requests reqQuery="cluster_id=zmock" respBytes=726 respStatus=OK respStatusCode=200 timeToStatus=0.001450 timeTotal=0.001453 timeWriteBody=0.000003
federation_test.go:690:
c.Check(json.NewDecoder(s.remoteMockRequests0.Body).Decode(&cr), check.IsNil)
... value *json.SyntaxError = &json.SyntaxError{msg:"invalid character 'c' looking for beginning of value", Offset:1} ("invalid character 'c' looking for beginning of value")

federation_test.go:691:
c.Check(cr.ContainerRequest.RuntimeToken, check.Equals, "xyz")
... obtained string = ""
... expected string = "xyz"

----------------------------------------------------------------------
FAIL: federation_test.go:694: FederationSuite.TestCreateRemoteContainerRequestRuntimeTokenFromAuth

time="2020-10-16T17:27:41.030750822Z" level=info msg=request RequestID=req-m6xowfzgvbv2zi2sg8jo remoteAddr="192.0.2.1:1234" reqBytes=165 reqForwardedFor= reqHost=example.com reqMethod=POST reqPath=arvados/v1/container_requests reqQuery="cluster_id=zmock"
time="2020-10-16T17:27:41.031717846Z" level=info msg=response RequestID=req-m6xowfzgvbv2zi2sg8jo remoteAddr="192.0.2.1:1234" reqBytes=165 reqForwardedFor= reqHost=example.com reqMethod=POST reqPath=arvados/v1/container_requests reqQuery="cluster_id=zmock" respBytes=726 respStatus=OK respStatusCode=200 timeToStatus=0.000961 timeTotal=0.000963 timeWriteBody=0.000002
federation_test.go:719:
c.Check(json.NewDecoder(s.remoteMockRequests0.Body).Decode(&cr), check.IsNil)
... value *json.SyntaxError = &json.SyntaxError{msg:"invalid character 'c' looking for beginning of value", Offset:1} ("invalid character 'c' looking for beginning of value")

federation_test.go:720:
c.Check(cr.ContainerRequest.RuntimeToken, check.Equals, arvadostest.ActiveTokenV2)
... obtained string = ""
... expected string = "v2/zzzzz-gj3su-077z32aux8dg2s1/3kg6k6lzmp9kj5cpkcoxie963cmvjahbt2fod9zru30k1jqdmi"

----------------------------------------------------------------------
FAIL: handler_test.go:321: HandlerSuite.TestGetObjects

handler_test.go:351:
s.CheckObjectType(c, "/arvados/v1/"+url, arvadostest.AdminToken, skippedFields)
handler_test.go:316:
c.Errorf("%s's key %q missing on controller's response.", direct["kind"], k)
... Error: arvados#containerRequest's key "expires_at" missing on controller's response.

handler_test.go:351:
s.CheckObjectType(c, "/arvados/v1/"+url, arvadostest.AdminToken, skippedFields)
handler_test.go:312:
c.Check(val, check.DeepEquals, direct[k],
check.Commentf("RailsAPI %s key %q's value %q differs from controller's %q.", direct["kind"], k, direct[k], val))
... obtained string = ""
... expected = nil
... RailsAPI arvados#containerRequest key "log_uuid"'s value %!q(<nil>) differs from controller's "".

handler_test.go:351:
s.CheckObjectType(c, "/arvados/v1/"+url, arvadostest.AdminToken, skippedFields)
handler_test.go:312:
c.Check(val, check.DeepEquals, direct[k],
check.Commentf("RailsAPI %s key %q's value %q differs from controller's %q.", direct["kind"], k, direct[k], val))
... obtained string = ""
... expected = nil
... RailsAPI arvados#containerRequest key "output_uuid"'s value %!q(<nil>) differs from controller's "".

handler_test.go:351:
s.CheckObjectType(c, "/arvados/v1/"+url, arvadostest.AdminToken, skippedFields)
handler_test.go:316:
c.Errorf("%s's key %q missing on controller's response.", direct["kind"], k)
... Error: arvados#containerRequest's key "container_count" missing on controller's response.

handler_test.go:351:
s.CheckObjectType(c, "/arvados/v1/"+url, arvadostest.AdminToken, skippedFields)
handler_test.go:312:
c.Check(val, check.DeepEquals, direct[k],
check.Commentf("RailsAPI %s key %q's value %q differs from controller's %q.", direct["kind"], k, direct[k], val))
... obtained float64 = 0
... expected = nil
... RailsAPI arvados#containerRequest key "container_count_max"'s value %!q(<nil>) differs from controller's %!q(float64=0).

handler_test.go:351:
s.CheckObjectType(c, "/arvados/v1/"+url, arvadostest.AdminToken, skippedFields)
handler_test.go:316:
c.Errorf("%s's key %q missing on controller's response.", direct["kind"], k)
... Error: arvados#containerRequest's key "filters" missing on controller's response.

handler_test.go:351:
s.CheckObjectType(c, "/arvados/v1/"+url, arvadostest.AdminToken, skippedFields)
handler_test.go:312:
c.Check(val, check.DeepEquals, direct[k],
check.Commentf("RailsAPI %s key %q's value %q differs from controller's %q.", direct["kind"], k, direct[k], val))
... obtained string = ""
... expected = nil
... RailsAPI arvados#containerRequest key "output_name"'s value %!q(<nil>) differs from controller's "".

handler_test.go:351:
s.CheckObjectType(c, "/arvados/v1/"+url, arvadostest.AdminToken, skippedFields)
handler_test.go:312:
c.Check(val, check.DeepEquals, direct[k],
check.Commentf("RailsAPI s key %q's value %q differs from controller's %q.", direct["kind"], k, direct[k], val))
... obtained map[string]interface {} = map[string]interface {}{"API":interface {}(nil), "keep_cache_ram":0, "ram":123, "vcpus":1}
... expected map[string]interface {} = map[string]interface {}{"ram":123, "vcpus":1}
... RailsAPI arvados#containerRequest key "runtime_constraints"'s value map["ram":
!q(float64=123) "vcpus":%!q(float64=1)] differs from controller's map["API":<nil> "keep_cache_ram":%!q(float64=0) "ram":%!q(float64=123) "vcpus":%!q(float64=1)].

handler_test.go:351:
s.CheckObjectType(c, "/arvados/v1/"+url, arvadostest.AdminToken, skippedFields)
handler_test.go:312:
c.Check(val, check.DeepEquals, direct[k],
check.Commentf("RailsAPI s key %q's value %q differs from controller's %q.", direct["kind"], k, direct[k], val))
... obtained map[string]interface {} = map[string]interface {}{"max_run_time":0, "partitions":interface {}(nil), "preemptible":false}
... expected map[string]interface {} = map[string]interface {}{}
... RailsAPI arvados#containerRequest key "scheduling_parameters"'s value map[] differs from controller's map["max_run_time":
!q(float64=0) "partitions":<nil> "preemptible":%!q(bool=false)].

handler_test.go:351:
s.CheckObjectType(c, "/arvados/v1/"+url, arvadostest.AdminToken, skippedFields)
handler_test.go:312:
c.Check(val, check.DeepEquals, direct[k],
check.Commentf("RailsAPI %s key %q's value %q differs from controller's %q.", direct["kind"], k, direct[k], val))
... obtained string = ""
... expected = nil
... RailsAPI arvados#containerRequest key "description"'s value %!q(<nil>) differs from controller's "".

handler_test.go:351:
s.CheckObjectType(c, "/arvados/v1/"+url, arvadostest.AdminToken, skippedFields)
handler_test.go:312:
c.Check(val, check.DeepEquals, direct[k],
check.Commentf("RailsAPI %s key %q's value %q differs from controller's %q.", direct["kind"], k, direct[k], val))
... obtained string = ""
... expected = nil
... RailsAPI arvados#containerRequest key "requesting_container_uuid"'s value %!q(<nil>) differs from controller's "".

----------------------------------------------------------------------
FAIL: integration_test.go:43: IntegrationSuite.SetUpSuite

refs #17014

Arvados-DCO-1.1-Signed-off-by: Nico Cesar <>

History

#1 Updated by Peter Amstutz 1 day ago

  • Status changed from New to In Progress

#2 Updated by Peter Amstutz 1 day ago

  • Target version changed from 2020-10-21 Sprint to 2020-11-04 Sprint

#3 Updated by Nico César 1 day ago

  • Blocks Feature #16462: Expand arvados-controller to expose forecast features added

Also available in: Atom PDF