Bug #5298
Updated by Brett Smith almost 10 years ago
A user just encountered an issue where their Crunch script had surprising behavior because an input manifest defined multiple files in the same stream with one file per line, like this: <pre>. [locator] [data block] 0:3:foo . [locator] [data block] 0:6:bar …</pre> Currently, CollectionReader.all_streams() iterates lines in the manifest. Using this method in a for loop, the user expected to find all of the files listed above in a single iteration. However, that's not the behavior all_streams() actually presented. I believe our general expectation is that the SDK handles all the work of presenting manifests logically, so I think the method should be changed to iterate over "logical" streams rather than physical lines in the manifest. As long as the final list of files is correct, I believe this would be backward compatible: since writing the manifest on one or multiple lines is functionally indistinguishable, presenting it as such to the SDK client should be indistinguishable, too.