Project

General

Profile

Bug #5298

Updated by Brett Smith about 9 years ago

A user just encountered an issue where their Crunch script had surprising behavior because an input manifest defined multiple files in the same stream with one file per line, like this: 

 <pre>. [locator] [data block] 0:3:foo 
 . [locator] [data block] 0:6:bar 
 …</pre> 

 Currently, CollectionReader.all_streams() iterates lines in the manifest.    Using this method in a for loop, the user expected to find all of the files listed above in a single iteration.    However, that's not the behavior all_streams() actually presented. 

 I believe our general expectation is that the SDK handles all the work of presenting manifests logically, so I think the method should be changed to iterate over "logical" streams rather than physical lines in the manifest.    As long as the final list of files is correct, I believe this would be backward compatible: since writing the manifest on one or multiple lines is functionally indistinguishable, presenting it as such to the SDK client should be indistinguishable, too.

Back