Metadata » History » Version 2

Peter Amstutz, 10/21/2016 03:29 PM

1 1 Peter Amstutz
h1. Metadata
2 1 Peter Amstutz
3 1 Peter Amstutz
Input and output documents gain parallel "input metadata" and "output metadata" documents.
4 1 Peter Amstutz
5 1 Peter Amstutz
The input metadata is parallel to the input document, except that it describes metadata about each parameter.  For example, for this input:
6 1 Peter Amstutz
7 1 Peter Amstutz
<pre>
8 1 Peter Amstutz
foo:
9 1 Peter Amstutz
  class: File
10 1 Peter Amstutz
  location: keep:abc+123/foo.bam
11 1 Peter Amstutz
</pre>
12 1 Peter Amstutz
13 1 Peter Amstutz
the metadata might be:
14 1 Peter Amstutz
15 1 Peter Amstutz
<pre>
16 1 Peter Amstutz
foo:
17 1 Peter Amstutz
  xyz:sample: sample1122333
18 1 Peter Amstutz
  xyz:sequencer: illumina
19 1 Peter Amstutz
</pre>
20 1 Peter Amstutz
21 1 Peter Amstutz
On workflow start, reverse lookup PDH to UUID and find all tags.  Create input metadata document.
22 1 Peter Amstutz
23 1 Peter Amstutz
On workflow end, create output metadata document based on "arv:PropagateMetadata".
24 1 Peter Amstutz
25 1 Peter Amstutz
<pre>
26 1 Peter Amstutz
hints:
27 1 Peter Amstutz
  arv:PropagateMetadata:
28 1 Peter Amstutz
    setMetadata:
29 1 Peter Amstutz
      - outputParameter: bar
30 1 Peter Amstutz
        valueFrom: $(metadata.foo)
31 1 Peter Amstutz
</pre>
32 1 Peter Amstutz
33 1 Peter Amstutz
Resulting in output:
34 1 Peter Amstutz
35 1 Peter Amstutz
<pre>
36 1 Peter Amstutz
bar:
37 1 Peter Amstutz
  class: File
38 1 Peter Amstutz
  location: keep:abc+123/result.vcf
39 1 Peter Amstutz
</pre>
40 1 Peter Amstutz
41 1 Peter Amstutz
with output metadata:
42 1 Peter Amstutz
43 1 Peter Amstutz
<pre>
44 1 Peter Amstutz
bar:
45 1 Peter Amstutz
  xyz:sample: sample1122333
46 1 Peter Amstutz
  xyz:sequencer: illumina
47 1 Peter Amstutz
</pre>
48 1 Peter Amstutz
49 2 Peter Amstutz
Arvados then creates tags to the output collection based on output metadata document:
50 2 Peter Amstutz
51 2 Peter Amstutz
<pre>
52 2 Peter Amstutz
link_class: tag
53 2 Peter Amstutz
name: xyz:sample
54 2 Peter Amstutz
head_uuid: zzzzz-4zz18-zzzzzzzzzzzzzzz
55 2 Peter Amstutz
properties: 
56 2 Peter Amstutz
  value: sample1122333
57 2 Peter Amstutz
  path: result.vcf
58 2 Peter Amstutz
</pre>
59 2 Peter Amstutz
60 2 Peter Amstutz
<pre>
61 2 Peter Amstutz
link_class: tag
62 2 Peter Amstutz
name: xyz:sequencer
63 2 Peter Amstutz
head_uuid: zzzzz-4zz18-zzzzzzzzzzzzzzz
64 2 Peter Amstutz
properties: 
65 2 Peter Amstutz
  value: illumina
66 2 Peter Amstutz
  path: result.vcf
67 2 Peter Amstutz
</pre>