Arvados

h1. Service containers

Concept: Containers launched via the Crunch infrastructure, but provide a network port that things can connect to.

Arvados epic: https://dev.arvados.org/issues/17207

h2. Uses cases

* Applications providing an API

** a bunch of data needs to be loaded into RAM before it can be used, queried, or computed on

** e.g. large language models, databases, function-as-a-service

** Makes sense when the time spent on any given query is much much smaller than the loading time

* User facing web applications

** e.g. Integrative Genomics Viewer (IGV), Jupyter notebooks

** Also includes web applications that interact with an API (first bullet)

h2. Fundamental requirement

Crunch launches a container and makes it possible for an outside client to communicate with the container.

h2. Discussion points

h3. Communication between containers

Exposing services to outside vs communication between containers on the inside have different requirements.

Outside: Must be able to connect from outside.  Because containers are on a private network, some kind of proxying or NAT is required.

Inside: Assuming containers are on the same private network and can route to each other, can communicate using direct TCP connections.

h3. HTTP only, or arbitrary TCP connections?

HTTP only: Can proxy HTTP requests using wildcard DNS and "Host:" headers, we have a lot of machinery to do that already.  Can apply Arvados authentication to requests, e.g. clients have to provide an Arvados token and can only communicate with containers that have read access to.  Cannot host things that don't use HTTP.

Arbitrary TCP: Would need to apply NAT or connection tunneling to connections on an arbitrary external port that is associated with the container.  We don't have machinery to do this.  Authentication is left up to the service.  Can host things that have their own protocols, such as postgresql or ssh.

Container shell uses connection tunneling, it makes a HTTP connection and doing a connection upgrade to SSH.  This requires special cooperation between arvados-client and ssh, which doesn't generalize.

Internal-only connections (between containers) may be a bit easier to orchestrate arbitrary TCP connections (no tunneling required).  Authentication is still left up to the container, or requires fiddling with firewall rules on the fly to control who can access the container.

h2. 

* Running container can expose one or more TCP service ports.

* Internal crunch containers should be able to connect to the service

* External clients should be able to connect to the service.