Project

General

Profile

Actions

Story #9053

open

Port API server to Go

Added by Tom Clegg about 6 years ago. Updated 5 months ago.

Status:
New
Priority:
Normal
Assigned To:
-
Target version:
-
Start date:
01/01/2023
Due date:
05/31/2023 (Due in about 11 months)
% Done:

0%

Estimated time:
Story points:
-
Release:
Release relationship:
Auto

Description

Background

We want to replace our Rails server with a Go server. However:
  • Porting the entire code base to Go, and cutting over everything at once, seems unlikely to go smoothly.
  • In the time it takes to port, the Rails functionality will change. Keeping them in sync will be challenging.
  • Porting the functionality and the corresponding tests at once is likely to result in unnoticed errors and omissions.

Development approach

The first version of the Go service is just an HTTP proxy: it forwards each request to a configured upstream Rails API server. Nginx configs in the install docs (and existing deployments) are updated to send API requests to this new server. #13497

Next, port some functional/integration tests from the Rails server code base to the Go code base. These tests will pass because Go server is already proxying the relevant requests to the Rails server. (Mocking won't be portable, though!)

Each test that has been ported to Go can be skipped in the Rails code base.

When enough Rails tests are skipped to remove an API from Rails without making the remaining tests fail, that API can be ported to Go. The relevant tests are already in place in the Go code base, so they should be a reasonable way to confirm that the new Go port is correct.

Repeat until all tests are ported to Go, and they pass even with the proxy-to-Rails feature removed.

Caveats / TBD

Rails tests are fast (at least faster than Workbench tests, for example) because they use transaction rollback to reset database state. This option won't be available when tests run in a different (Go) process. Practically, this means the Go tests will need to use the "database reset" API, and they will run slowly.
  • Perhaps we can start by implementing "database reset" in Go? It will have direct access to the Postgres database, and it could take a snapshot of the database at startup instead of reading the Rails YAML fixtures.
  • Is there a better way around this?

Related issues

Related to Arvados - Story #9502: [API] Update permissions cache as needed after select writesResolved06/28/2016

Actions
Related to Arvados - Feature #13493: Federated record retrievalResolvedPeter Amstutz06/28/2018

Actions
Related to Arvados - Story #13995: [Controller] Port "collection update" functional tests from Rails API to controller test suiteNew

Actions
Related to Arvados - Story #13997: [Controller] Port "collection update" API from RailsNew

Actions
Related to Arvados - Feature #17014: Add Container Requests endpoint to controllerResolvedNico César11/10/2020

Actions
Blocked by Arvados - Story #13497: [API] Initial "arvados-controller" server that proxies API endpoints to Rails serverResolvedTom Clegg06/15/2018

Actions
Actions #1

Updated by Tom Clegg about 6 years ago

  • Description updated (diff)
Actions #2

Updated by Tom Clegg about 6 years ago

  • Description updated (diff)
Actions #3

Updated by Tom Clegg about 4 years ago

Actions #4

Updated by Tom Clegg about 4 years ago

  • Subject changed from [API] Deployable Go API server that proxies unimplemented APIs to Rails to [Epic] Port APIs to Go
  • Description updated (diff)
Actions #5

Updated by Tom Clegg about 4 years ago

  • Blocked by Story #13497: [API] Initial "arvados-controller" server that proxies API endpoints to Rails server added
Actions #6

Updated by Tom Clegg almost 4 years ago

  • Description updated (diff)
Actions #7

Updated by Tom Clegg almost 4 years ago

  • Related to Story #13995: [Controller] Port "collection update" functional tests from Rails API to controller test suite added
Actions #8

Updated by Tom Clegg almost 4 years ago

  • Related to Story #13997: [Controller] Port "collection update" API from Rails added
Actions #9

Updated by Tom Morris over 3 years ago

  • Release deleted (12)
Actions #10

Updated by Peter Amstutz over 2 years ago

  • Subject changed from [Epic] Port APIs to Go to Port API server to Go
  • Project changed from Arvados to Arvados Epics
Actions #11

Updated by Peter Amstutz over 2 years ago

  • Start date set to 08/01/2020
  • Due date set to 02/01/2021
Actions #12

Updated by Peter Amstutz over 2 years ago

  • Release set to 28
Actions #13

Updated by Peter Amstutz about 2 years ago

  • Start date changed from 08/01/2020 to 09/01/2020
  • Due date changed from 02/01/2021 to 03/01/2021
Actions #14

Updated by Peter Amstutz about 2 years ago

  • Start date changed from 09/01/2020 to 10/01/2020
  • Due date changed from 03/01/2021 to 04/01/2021
Actions #15

Updated by Peter Amstutz about 2 years ago

  • Start date changed from 10/01/2020 to 02/01/2021
  • Due date changed from 04/01/2021 to 07/30/2021
Actions #16

Updated by Peter Amstutz almost 2 years ago

  • Start date changed from 02/01/2021 to 04/01/2021
  • Due date changed from 07/30/2021 to 09/30/2021
Actions #17

Updated by Nico César over 1 year ago

  • Related to Feature #17014: Add Container Requests endpoint to controller added
Actions #18

Updated by Peter Amstutz over 1 year ago

  • Start date changed from 04/01/2021 to 09/01/2021
  • Due date changed from 09/30/2021 to 03/31/2022
Actions #19

Updated by Peter Amstutz 12 months ago

  • Start date changed from 09/01/2021 to 01/01/2022
  • Due date changed from 03/31/2022 to 06/30/2022
Actions #20

Updated by Peter Amstutz 9 months ago

  • Start date changed from 01/01/2022 to 06/01/2022
  • Due date changed from 06/30/2022 to 12/31/2022
Actions #21

Updated by Peter Amstutz 5 months ago

  • Start date changed from 06/01/2022 to 01/01/2023
  • Due date changed from 12/31/2022 to 05/31/2023
Actions

Also available in: Atom PDF