Arvados

Postgres seems to be creating a lot of fragmentation in the materialized_permissions table in a context with a lot of users (~1000) and large groups. Reported the table size went from 200MB to 2GB to 20 GB in the span of a few days (? or even faster).

The group sync script is very slow in this context while there are other updates ongoing.

Normal usage (e.g. navigating a large project with lots of subprojects in a read-only arv-mount) is also very slow when the table is fragmented that way.

Vacuum full helps but can often not complete because normal usage patterns prevent it from completing.

This has been observed on Postgres 9.5 and 13.