-
Leverage stream to do enrollment more efficiently
-
Move progress counter after gallery output, this should hopefully give slightly more accurate initial ETAs. When doing the finalize step in stream projectUpdate, don't push empty sets through the rest of the transforms (this reduces spurious calls to project, making progress counter work slightly better).
-
Conflicts: openbr/core/core.cpp
-
Merge Datasource/DataSourceManager, leaving DataSource as the primary interface for opening a template list. Introduce an alternate hieararchy for things that are done to individual templates. Rename FirstStage->ReadStage. Drop LastStage as a unique class, it can be represented as a single threaded stage with a transform that just collects the templates it receives. Introduce a separate class contaiing the stream mode enum. This avoids a lot of circular dependencies, giving us a more sane file layout.
-
This change consolidates the previous 'read' and 'noDuplicates' flags into a single 'append' flag. If append is specified, and an output gallery already exists, and the gallery format supports read/write or has explicit append support, then enrollment will be restricted to those files in the input list not already present in the gallery, and the results will be appended to the existing gallery. append defaults to 'false', which is a deparature from previous behavior. The .gal format has explicit append support, for other cases, if the gallery supports both read and write (less common than you might think), we support append by reading the existing gallery, and writing back out to an overwriting file. It should be possible to add explicit append support to several other gallery types.
-
When setting up the stream in enroll, use the Stream interface's staging logic on the input algorithm. Making this happen is slightly complex when we don't particularly have header files, but on the plus side algorithms incorporating something like Show (e.g. ShowFaceDetection) will not be used as a single threaded stage in the process.
-
Since we no longer add distribute in AlgorithmCore::init, instead adding a task specific stream in AlgorithmCore::enroll, we also add a stream to the algorithm in train, so that the training data will be projected in parallel by PipeTransform::train.