Project

General

Profile

Keep real world performance numbers » History » Version 9

Ward Vandewege, 07/29/2020 04:01 PM

1 1 Ward Vandewege
h1. Keep real world performance numbers
2
3 6 Ward Vandewege
h2. AWS/S3 numbers
4
5
Here are some benchmark numbers against AWS S3, as measured in July 2020. They were generated with the _keep-exercise_ tool in an Arvados installation with one dedicated Keepstore node (c5n.2xlarge) and one dedicated node for running _keep-exercise_ (c5n.2xlarge). The Keepstore node was backed by one S3 bucket, in a VPC with an S3 endpoint installed. Versioning, Server Access logging, Static website hosting, Object-level logging and Default encryption were disabled. Object lock, Transfer acceleration and Requester pays were also disabled. There were no Active notifications. Each test consisted of 4 one minute runs, which were averaged into one number. The tests were repeated 3 times, and of those 3 runs, the highest average speed was selected and included in the table below.
6
7 7 Ward Vandewege
The *goamz* driver is the default Keep S3 driver as of Arvados 2.0. In Arvados 2.1, a new S3 driver is added based on the *aws-sdk-go-v2* driver. It offers better read performance but has not had as much production use.
8 6 Ward Vandewege
9
||_. goamz |_. aws-sdk-go-v2 |_. command line|
10
|single thread read performance (average)|32.53 MiB/s|79.48 MiB/s|keep-exercise -repeat 4 -run-time 60s -vary-request -use-index -rthreads 1 -wthreads 1|
11
|single thread write performance (average)|39.75 MiB/s|41.05 MiB/s|keep-exercise -repeat 4 -run-time 60s -vary-request -use-index -rthreads 1 -wthreads 1|
12
13
When using 20 concurrent read or write threads, the numbers from both drivers are:
14
15
||_. goamz |_. aws-sdk-go-v2 |_. command line|
16
|20 thread read performance (average)|585.60 MiB/s|898.93 MiB/s|keep-exercise -repeat 4 -run-time 60s -vary-request -use-index -rthreads 20 -wthreads 0|
17
|20 thread write performance (average)|610.40 MiB/s|688.25 MiB/s|keep-exercise -repeat 4 -run-time 60s -vary-request -use-index -rthreads 0 -wthreads 20|
18
19 8 Ward Vandewege
When using 40 concurrent read or write threads, the numbers from both drivers are:
20
21 9 Ward Vandewege
||_. goamz |_. aws-sdk-go-v2 |_. command line|_. comment|
22
|40 thread read performance (average)|1005.30 MiB/s|1005.35 MiB/s|keep-exercise -repeat 4 -run-time 60s -vary-request -use-index -rthreads 40 -wthreads 0||
23
|40 thread write performance (average)|653.05 MiB/s|974.925 MiB/s|keep-exercise -repeat 4 -run-time 60s -vary-request -use-index -rthreads 0 -wthreads 40|goamz is cpu bound on the keepstore (800%)|
24 8 Ward Vandewege
25 6 Ward Vandewege
h2. Historical AWS/S3 numbers
26 1 Ward Vandewege
27 5 Tom Morris
|Name|Network performance|CPU|RAM|Keepstore|Data Payload Throughput|CPU load|Date tested|Tested by|
28 4 Ward Vandewege
|c5.9xlarge|10 Gigabit|36 cores|72 GiB|0.1.20171212165144.296aa66-1|Saturates RX and TX (1+ GiB/sec)|~4|2018-01-18|Ward|
29 1 Ward Vandewege
|r4.8xlarge|10 Gigabit|32 cores|244 GiB|0.1.20171212165144.296aa66-1|Saturates RX and TX (1+ GiB/sec)|~4|2018-01-17|Ward|
30 3 Ward Vandewege
|r4.large|"up to" 10 Gigabit|2 cores|15 GiB|0.1.20171212165144.296aa66-1|~400 MiB/sec but long periods of cap at ~90 MiB/sec||2018-01-16|Ward|