Idea #15180
closed
[Spike] Test migration on production database
Added by Tom Morris almost 5 years ago.
Updated almost 5 years ago.
Assigned To:
Eric Biagiotti
Release relationship:
Auto
- Private changed from Yes to No
- Parent task deleted (
#15093)
- Tracker changed from Task to Idea
- Target version changed from To Be Groomed to 2019-05-08 Sprint
- Blocks Idea #15093: Work with Ops to decide best DB migration strategy for collection file count & size added
The RecomputeFileNamesIndex migration (#13752) in 1.3.1 on e51c5 took 77 min to run #15008. The RecomputeFileNamesIndex migration is similar to #14484 in that each collection is modified. The difference being that #14484 also uses the Ruby sdk to parse manifest text and retrieve the file count and total file size.
- Status changed from New to In Progress
Ran the migration on my machine using the production database from e51c5 with 4228696 collections (2555374 distinct pdhs).
The migration took 291 minutes. Approximately 871896 collections per hour.
My machine:
CPU(s): 8
Thread(s) per core: 2
Core(s) per socket: 4
Socket(s): 1
Model name: Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz
Additionally, as a very rough estimate, parsing took ~.003 milliseconds per manifest, while each database commit took ~6 milliseconds.
- Status changed from In Progress to Resolved
Also available in: Atom
PDF
Updated by Tom Morris 
Updated by Eric Biagiotti