[Spike] Test migration on production database
#5 Updated by Eric Biagiotti about 2 months ago
RecomputeFileNamesIndex migration (#13752) in 1.3.1 on e51c5 took 77 min to run #15008. The
RecomputeFileNamesIndex migration is similar to #14484 in that each collection is modified. The difference being that #14484 also uses the Ruby sdk to parse manifest text and retrieve the file count and total file size.
#6 Updated by Eric Biagiotti about 1 month ago
- Status changed from New to In Progress
Ran the migration on my machine using the production database from e51c5 with 4228696 collections (2555374 distinct pdhs).
The migration took 291 minutes. Approximately 871896 collections per hour.
CPU(s): 8 Thread(s) per core: 2 Core(s) per socket: 4 Socket(s): 1 Model name: Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz
Additionally, as a very rough estimate, parsing took ~.003 milliseconds per manifest, while each database commit took ~6 milliseconds.