Story #15849

Migrate collection & project properties to new vocabulary format

Added by Tom Morris 7 months ago. Updated 5 months ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
-
Target version:
Start date:
11/26/2019
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
-
Release relationship:
Auto

Description

Customer databases which use literal text labels for property/value pairs need to be converted to the new ID format in the database.

Input to this will be a new format vocabulary file which matches the original vocabulary file used in creating the database.

This will be an example script which accesses the database through the API.


Subtasks

Task #15861: Review 15849-vocab-migration-exampleResolvedPeter Amstutz

Associated revisions

Revision c884807d
Added by Lucas Di Pentima 6 months ago

Merge branch '15849-vocab-migration-example'
Closes #15849

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <>

History

#1 Updated by Lucas Di Pentima 7 months ago

  • Assigned To set to Lucas Di Pentima

#2 Updated by Tom Morris 7 months ago

  • Description updated (diff)

#3 Updated by Lucas Di Pentima 6 months ago

  • Status changed from New to In Progress

#4 Updated by Lucas Di Pentima 6 months ago

Updates at 8e1ca7677 - branch 15849-vocab-migration-example

Adds vocabulary migration example script to the documentation's admin section with an explanation on how to use it.

#5 Updated by Peter Amstutz 6 months ago

You can use required=True instead of default=None and then checking the value later.

opts.add_argument('--vocabulary-file', type=str, metavar='PATH', default=None,

list_all should use a stable ordering, like "created_at"

I know I said to put the script into the docs, but it occurred to me that you could put it in tools/vocabulary_migrate/vocabulary_migrate.py and have a symlink from docs/_include/_vocabulary_migrate_py.liquid to ../../tools/vocabulary_migrate/vocabulary_migrate.py

The {% comment %} makes it not directly executable as Python. A regular '#' comment is fine.

The migration script tag / values label matching is case sensitive. I'm not quite sure what to do about it, thought I'd mention it.

#6 Updated by Lucas Di Pentima 6 months ago

Updates at c6b259220

Addressed above comments. Added a note on the documentation about case sensitive matching.

#7 Updated by Peter Amstutz 6 months ago

Lucas Di Pentima wrote:

Updates at c6b259220

Addressed above comments. Added a note on the documentation about case sensitive matching.

One more comment, if you do --dry-run and there is nothing to do, it exits silently, whereas an actual run will say "Done, total object migrated: 0"

  • should be "objects"
  • for dry-run it should say something like "nothing to do"

otherwise LGTM.

#8 Updated by Lucas Di Pentima 6 months ago

Fixed at b0f2bbde0, thanks!

#9 Updated by Lucas Di Pentima 6 months ago

  • Status changed from In Progress to Resolved

#10 Updated by Peter Amstutz 5 months ago

  • Release set to 22

Also available in: Atom PDF