diff --git a/docs/docs/api_docs/c_api/functions.md b/docs/docs/api_docs/c_api/functions.md index 95256c6..9d2b2e3 100644 --- a/docs/docs/api_docs/c_api/functions.md +++ b/docs/docs/api_docs/c_api/functions.md @@ -55,7 +55,7 @@ Removes duplicate [templates](../cpp_api/template/template.md) in a [gallery](.. ## br_cluster -Clusters one or more similarity matrices into a list of subjects. A [similarity matrix](../../technical.md#the-evaluation-harness) is a type of [Output](../cpp_api/output/output.md). The current clustering algorithm is a simplified implementation of \cite zhu11. +Clusters one or more similarity matrices into a list of subjects. A [similarity matrix](../../tutorials.md#the-evaluation-harness) is a type of [Output](../cpp_api/output/output.md). The current clustering algorithm is a simplified implementation of the algorithm proposed by Zhu et al[^1]. * **function definition:** @@ -66,7 +66,7 @@ Clusters one or more similarity matrices into a list of subjects. A [similarity Parameter | Type | Description --- | --- | --- num_simmats | int | Size of **simmats** - simmats[] | const char * | Array of [simmat](../../technical.md#the-evaluation-harness) composing one large self-similarity matrix arranged in row major order. + simmats[] | const char * | Array of [simmat](../../tutorials.md#the-evaluation-harness) composing one large self-similarity matrix arranged in row major order. aggressiveness | float | The higher the aggressiveness the larger the clusters. Suggested range is [0,10] csv | const char * | The cluster results file to generate. Results are stored one row per cluster and use gallery indices. @@ -87,8 +87,8 @@ Combines several equal-sized mask matrices. A comparison may not be simultaneous Parameter | Type | Description --- | --- | --- num_input_masks | int | Size of **input_masks** - input_masks[] | const char * | Array of [mask matrices](../../technical.md#the-evaluation-harness) to combine. All matrices must have the same dimensions. - output_mask | const char * | The file to contain the resulting [mask matrix](../../technical.md#the-evaluation-harness) + input_masks[] | const char * | Array of [mask matrices](../../tutorials.md#the-evaluation-harness) to combine. All matrices must have the same dimensions. + output_mask | const char * | The file to contain the resulting [mask matrix](../../tutorials.md#the-evaluation-harness) method | const char * | Possible values are: * **see:** [br_make_mask](#br_make_mask) @@ -251,8 +251,8 @@ Creates a **.csv** file containing performance metrics from evaluating the simil Parameter | Type | Description --- | --- | --- - simmat | const char * | The [simmat](../../technical.md#the-evaluation-harness) to use - mask | const char * | The [mask](../../technical.md#the-evaluation-harness) to use. + simmat | const char * | The [simmat](../../tutorials.md#the-evaluation-harness) to use + mask | const char * | The [mask](../../tutorials.md#the-evaluation-harness) to use. csv | const char * | (Optional) The **.csv** file to contain performance metrics. matches | int | (Optional) An integer number of matches to output around the EER. Default is 0. @@ -273,8 +273,8 @@ Evaluates the similarity matrix using the mask matrix. Function aborts if TAR @ Parameter | Type | Description --- | --- | --- - simmat | const char * | The [simmat](../../technical.md#the-evaluation-harness) to use - mask | const char * | The [mask](../../technical.md#the-evaluation-harness) + simmat | const char * | The [simmat](../../tutorials.md#the-evaluation-harness) to use + mask | const char * | The [mask](../../tutorials.md#the-evaluation-harness) accuracy | const float | Desired true accept rate at false accept rate of one in one thousand. * **output:** (void) @@ -293,7 +293,7 @@ Creates a **.csv** file containing performance metrics from evaluating the simil Parameter | Type | Description --- | --- | --- - simmat | const char * | The [simmat](../../technical.md#the-evaluation-harness) + simmat | const char * | The [simmat](../../tutorials.md#the-evaluation-harness) target | const char * | The name of a gallery containing metadata for the target set. query | const char * | The name of a gallery containing metadata for the query set. csv | const char * | (Optional) The **.csv** file to contain performance metrics. @@ -337,7 +337,7 @@ Evaluates and prints clustering accuracy to the terminal. Parameter | Type | Description --- | --- | --- csv | const char * | The cluster results file. -gallery | const char * | The [Gallery](../cpp_api/gallery/gallery.md) used to generate the [simmat](../../technical.md#the-evaluation-harness) that was clustered. +gallery | const char * | The [Gallery](../cpp_api/gallery/gallery.md) used to generate the [simmat](../../tutorials.md#the-evaluation-harness) that was clustered. truth_property | const char * | (Optional) which metadata key to use from **gallery**, defaults to Label * **output:** (void) @@ -425,10 +425,10 @@ Perform score level fusion on similarity matrices. Parameter | Type | Description --- | --- | --- num_input_simmats | int | Size of **input_simmats**. - input_simmats[] | const char * | Array of [simmats](../../technical.md#the-evaluation-harness). All simmats must have the same dimensions. + input_simmats[] | const char * | Array of [simmats](../../tutorials.md#the-evaluation-harness). All simmats must have the same dimensions. normalization | const char * | Valid options are: fusion | const char * | Valid options are: - output_simmat | const char * | [Simmat](../../technical.md#the-evaluation-harness) to contain the fused scores. + output_simmat | const char * | [Simmat](../../tutorials.md#the-evaluation-harness) to contain the fused scores. * **output:** (void) @@ -505,7 +505,7 @@ Checks if the provided algorithm is a classifier. Wrapper of [IsClassifier](../c ## br_make_mask -Constructs a [mask](../../technical.md#the-evaluation-harness) from target and query inputs. +Constructs a [mask](../../tutorials.md#the-evaluation-harness) from target and query inputs. * **function definition:** @@ -517,7 +517,7 @@ Constructs a [mask](../../technical.md#the-evaluation-harness) from target and q --- | --- | --- target_input | const char * | The target [Gallery](../cpp_api/gallery/gallery.md) query_input | const char * | The query [Gallery](../cpp_api/gallery/gallery.md) - mask | const char * | The file to contain the resulting [mask](../../technical.md#the-evaluation-harness). + mask | const char * | The file to contain the resulting [mask](../../tutorials.md#the-evaluation-harness). * **output:** (void) * **see:** [br_combine_masks](#br_combine_masks) @@ -526,7 +526,7 @@ Constructs a [mask](../../technical.md#the-evaluation-harness) from target and q ## br_make_pairwise_mask -Constructs a [mask](../../technical.md#the-evaluation-harness) from target and query inputs considering the target and input sets to be definite pairwise comparisons. +Constructs a [mask](../../tutorials.md#the-evaluation-harness) from target and query inputs considering the target and input sets to be definite pairwise comparisons. * **function definition:** @@ -538,7 +538,7 @@ Parameter | Type | Description --- | --- | --- target_input | const char * | The target [Gallery](../cpp_api/gallery/gallery.md) query_input | const char * | The query [Gallery](../cpp_api/gallery/gallery.md) -mask | const char * | The file to contain the resulting [mask](../../technical.md#the-evaluation-harness). +mask | const char * | The file to contain the resulting [mask](../../tutorials.md#the-evaluation-harness). * **output:** (void) * **see:** [br_combine_masks](#br_combine_masks) @@ -794,7 +794,7 @@ Returns the full path to the root of the SDK. ## br_get_header -Retrieve the target and query inputs in the [BEE matrix](../../technical.md#the-evaluation-harness) header. For information on managed return values see [here](../c_api.md#memory). +Retrieve the target and query inputs in the [BEE matrix](../../tutorials.md#the-evaluation-harness) header. For information on managed return values see [here](../c_api.md#memory). * **function definition:** @@ -804,7 +804,7 @@ Retrieve the target and query inputs in the [BEE matrix](../../technical.md#the- Parameter | Type | Description --- | --- | --- - matrix | const char * | The [BEE matrix](../../technical.md#the-evaluation-harness) file to modify + matrix | const char * | The [BEE matrix](../../tutorials.md#the-evaluation-harness) file to modify target_gallery | const char ** | The matrix target query_gallery | const char ** | The matrix query @@ -815,7 +815,7 @@ Retrieve the target and query inputs in the [BEE matrix](../../technical.md#the- ## br_set_header -Update the target and query inputs in the [BEE matrix](../../technical.md#the-evaluation-harness) header. +Update the target and query inputs in the [BEE matrix](../../tutorials.md#the-evaluation-harness) header. * **function definition:** @@ -825,7 +825,7 @@ Update the target and query inputs in the [BEE matrix](../../technical.md#the-ev Parameter | Type | Description --- | --- | --- - matrix | const char * | The [BEE matrix](../../technical.md#the-evaluation-harness) file to modify + matrix | const char * | The [BEE matrix](../../tutorials.md#the-evaluation-harness) file to modify target_gallery | const char ** | The matrix target query_gallery | const char ** | The matrix query @@ -1387,3 +1387,7 @@ Close a provided [Gallery](../cpp_api/gallery/gallery.md). gallery | [br_gallery](typedefs.md#br_gallery) | Pointer to a [Gallery](../cpp_api/gallery/gallery.md) * **output:** (void) + +[^1]: *Zhu et al.* + **A Rank-Order Distance based Clustering Algorithm for Face Tagging**, + CVPR 2011 diff --git a/docs/docs/api_docs/cl_api.md b/docs/docs/api_docs/cl_api.md index 3da9922..b07f81e 100644 --- a/docs/docs/api_docs/cl_api.md +++ b/docs/docs/api_docs/cl_api.md @@ -1,13 +1,13 @@ -The command line API is a tool to run OpenBR from the command line. The command line is the easiest and fastest way to run OpenBR! +The command line API is a tool to run OpenBR from the command line. The command line is the easiest and fastest way to run OpenBR! The following is a detailed description of the command line API. The command line API is really just a set of wrappers to call the [C API](c_api.md). All of the flags in this API have a corresponding C API call. To help display the examples the following shorthand definitions will be used: Shortcut | Definition --- | --- - | <> Represent an input argument -{arg} | {} Represent an output argument +<arg> | <> Represent an input argument +\{arg\} | \{\} Represent an output argument [arg] | [] Represent an optional argument -(arg0|...|argN) | (...|...) Represent a choice. +(arg0 | ... | argN) | (... | ...) Represent a choice. ## Algorithms @@ -17,7 +17,7 @@ Almost every command line process needs to specify an algorithm to work properly Make sure you use the quotes if your algorithm is longer than one plugin because special characters in OpenBR are also special characters (with very different meanings!) in Bash. -## Core Commands +## Core Commands ### -train {: #train } @@ -46,7 +46,7 @@ Compare query [Templates](cpp_api/template/template.md) against a target [Galler * **arguments:** -compare [{output}] - + * **wraps:** [br_compare](c_api/functions.md#br_compare) ### -pairwiseCompare {: #pairwisecompare } @@ -99,7 +99,7 @@ Perform score level fusion on similarity matrices. * **arguments:** -fuse ... (None|MinMax|ZScore|WScore) (Min|Max|Sum[W1:W2:...:Wn]|Replace|Difference|None) {simmat} - + * **wraps:** [br_fuse](c_api/functions.md#br_fuse) ### -cluster {: #cluster } @@ -129,7 +129,7 @@ Constructs a mask from target and query inputs considering the target and input * **arguments:** -makePairwiseMask {mask} - + * **wraps:** [br_make_pairwise_mask](c_api/functions.md#br_make_pairwise_mask) ### -combineMasks {: #combinemask } @@ -139,7 +139,7 @@ Combines several equal-sized mask matrices. A comparison may not be simultaneous * **arguments:** -combineMasks ... {mask} (And|Or) - + * **wraps:** [br_combine_masks](c_api/functions.md#br_combine_masks) ### -cat {: #cat } @@ -189,7 +189,7 @@ Evaluates and prints detection accuracy to terminal * **arguments:** -evalDetection [{csv}] [{normalize}] [{minSize}] [{maxSize}] - + * **wraps:** [br_eval_detection](c_api/functions.md#br_eval_detection) ### -evalLandmarking {: #evallandmarking } @@ -199,7 +199,7 @@ Evaluates and prints landmarking accuracy to terminal * **arguments:** -evalLandmarking [{csv} [ ] [sample_index] [total_examples]] - + * **wraps:** [br_eval_landmarking](c_api/functions.md#br_eval_landmarking) @@ -210,7 +210,7 @@ Evaluates regression accuracy to disk * **arguments:** -evalRegression - + * **wraps:** [br_eval_regression](c_api/functions.md#br_eval_regression) ### -assertEval {: #asserteval } @@ -220,7 +220,7 @@ Evaluates the similarity matrix using the mask matrix. Function aborts if TAR @ * **arguments:** -assertEval - + * **wraps:** [br_assert_eval](c_api/functions.md#br_assert_eval) ### -plotDetection {: #plotdetection } @@ -230,7 +230,7 @@ Renders detection performance figures for a set of .csv files created by [-evalD * **arguments:** -plotDetection ... {destination} - + * **wraps:** [br_plot_detection](c_api/functions.md#br_plot_detection) ### -plotLandmarking {: #plotlandmarking } @@ -265,7 +265,7 @@ A naive alternative to [-enroll](#enroll) ### -getHeader {: #getheader } -Retrieve the target and query inputs in the [BEE matrix](../technical.md#the-evaluation-harness) header +Retrieve the target and query inputs in the [BEE matrix](../tutorials.md#the-evaluation-harness) header * **arguments:** @@ -275,12 +275,12 @@ Retrieve the target and query inputs in the [BEE matrix](../technical.md#the-eva ### -setHeader {: #setheader } -Update the target and query inputs in the [BEE matrix](../technical.md#the-evaluation-harness) header +Update the target and query inputs in the [BEE matrix](../tutorials.md#the-evaluation-harness) header * **arguments:** -setHeader {} - + * **wraps:** [br_set_header](c_api/functions.md#br_set_header) ### -<key> <value> {: #setproperty } @@ -290,7 +290,7 @@ Appends a provided value to the [global metadata](cpp_api/context/context.md) us * **arguments:** - - + * **wraps:** [br_set_property](c_api/functions.md#br_set_property) @@ -303,7 +303,7 @@ Print command line API documentation to the terminal * **arguments:** -help - + * **wraps:** N/A ### -gui {: #gui } @@ -313,17 +313,17 @@ If this flag is set OpenBR will enable GUI windows to be launched. It must be th * **arguments:** br -gui - + * **wraps:** N/A ### -objects {: #objects } -Returns names and parameters for the requested objects. Each object is newline seperated. Arguments are seperated from the object name with a tab. This function uses [QRegExp][QRegExp] syntax +Returns names and parameters for the requested objects. Each object is newline separated. Arguments are separated from the object name with a tab. This function uses [QRegExp][QRegExp] syntax * **arguments:** -objects [abstraction [implementation]] - + * **wraps:** [br_objects](c_api/functions.md#br_objects) ### -about {: #about } @@ -333,7 +333,7 @@ Get a string with the name, version, and copyright of the project. This string i * **arguments:** -about - + * **wraps:** [br_about](c_api/functions.md#br_about) ### -version {: #version } @@ -348,7 +348,7 @@ Get the current OpenBR version ### -slave {: #slave } -For internal use via [ProcessWrapperTransform](../plugins/core.md#processwrappertransform) +For internal use via [ProcessWrapperTransform](plugins/core.md#processwrappertransform) * **arguments:** @@ -374,4 +374,4 @@ Exit the application -exit -* **wraps:** N/A \ No newline at end of file +* **wraps:** N/A diff --git a/docs/docs/api_docs/cpp_api/compositetransform/compositetransform.md b/docs/docs/api_docs/cpp_api/compositetransform/compositetransform.md index 387fbbe..1fac46a 100644 --- a/docs/docs/api_docs/cpp_api/compositetransform/compositetransform.md +++ b/docs/docs/api_docs/cpp_api/compositetransform/compositetransform.md @@ -12,4 +12,4 @@ See: * [Static Functions](statics.md) * [Functions](functions.md) -A CompositeTransform is a wrapper around a list of child transforms. It is used internally for plugins like pipes and forks. It offers \ No newline at end of file +CompositeTransforms are the base class for [Transforms](../transform/transform.md) that have a list of child transforms. It is used internally for plugins like pipes and forks. It inherits from [TimeVaryingTransform](../timevaryingtransform/timevaryingtransform.md) so that it can properly handle having children (one or many) that are time varying. \ No newline at end of file diff --git a/docs/docs/api_docs/cpp_api/compositetransform/constructors.md b/docs/docs/api_docs/cpp_api/compositetransform/constructors.md new file mode 100644 index 0000000..3d8d67a --- /dev/null +++ b/docs/docs/api_docs/cpp_api/compositetransform/constructors.md @@ -0,0 +1,3 @@ +Constructor | Description +--- | --- +CompositeTransform() | Default constructor. Calls the [TimeVaryingTransform](../timevaryingtransform/timevaryingtransform.md) [constructor](../timevaryingtransform/constructors.md) with [independent](../transform/members.md#independent) set to false. \ No newline at end of file diff --git a/docs/docs/api_docs/cpp_api/compositetransform/functions.md b/docs/docs/api_docs/cpp_api/compositetransform/functions.md new file mode 100644 index 0000000..9d1dbb4 --- /dev/null +++ b/docs/docs/api_docs/cpp_api/compositetransform/functions.md @@ -0,0 +1,121 @@ +## bool timeVarying() {: #timevarying } + +Check if the transform is time varying. The transform is time varying if any of its children are time varying. + +* **function definition:** + + bool timeVarying() const + +* **parameters:** NONE +* **output:** (bool) Returns [isTimeVarying](members.md#istimevarying) + +## void init() {: #init } + +Initialize the transform. If any of the child transform are time varying, [isTimeVarying](members.md#istimevarying) is set to true. Similarly if any of the child transforms are trainable, [trainable](../transform/members.md#trainable) is set to true. + +* **function definition:** + + void init() + +* **parameters:** NONE +* **output:** (void) + +## void project(const [Template](../template/template.md) &src, [Template](../template/template.md) &dst) {: #project-1 } + +If the transform is time varying call [timeInvariantAlias](../timevaryingtransform/members.md#timeinvariantalias) project, which ensures thread safety. If the transform is not time varying call [_project](#_project-1). + +* **function definition:** + + virtual void project(const Template &src, Template &dst) const + +* **parameters:** + + Parameter | Type | Description + --- | --- | --- + src | const [Template](../template/template.md) & | The input template + dst | [Template](../template/template.md) & | The output template + +* **output:** (void) + +## void project(const [TemplateList](../templatelist/templatelist.md) &src, [TemplateList](../templatelist/templatelist.md) &dst) {: #project-2 } + +If the transform is time varying call [timeInvariantAlias](../timevaryingtransform/members.md#timeinvariantalias) project, which ensures thread safety. If the transform is not time varying call [_project](#_project-2). + +* **function definition:** + + virtual void project(const TemplateList &src, TemplateList &dst) const + +* **parameters:** + + Parameter | Type | Description + --- | --- | --- + src | const [TemplateList](../templatelist/templatelist.md) & | The input template list + dst | [TemplateList ](../templatelist/templatelist.md) & | The output template list + +* **output:** (void) + +## void \_project(const [Template](../template/template.md) &src, [Template](../template/template.md) &dst) {: #\_project-1 } + +This is a pure virtual function. It should handle standard projection through the child transforms + +* **function definition:** + + virtual void _project(const Template &src, Template &dst) const = 0 + +* **parameters:** + + Parameter | Type | Description + --- | --- | --- + src | const [Template](../template/template.md) & | The input template + dst | [Template](../template/template.md) & | The output template + +* **output:** (void) + +## void \_project(const [TemplateList](../templatelist/templatelist.md) &src, [TemplateList](../templatelist/templatelist.md) &dst) {: #\_project-2 } + +This is a pure virtual function. It should handle standard projection through the child transforms + +* **function definition:** + + virtual void _project(const TemplateList &src, TemplateList &dst) const = 0 + +* **parameters:** + + Parameter | Type | Description + --- | --- | --- + src | const [TemplateList](../templatelist/templatelist.md) & | The input template list + dst | [TemplateList](../templatelist/templatelist.md) & | The output template list + +* **output:** (void) + +## [Transform](../transform/transform.md) \*simplify(bool &newTransform) {: #simplify } + +This is a virtual function. Calls [simplify](../transform/functions.md#simplify) on each child transform. + +* **function definition:** + + virtual Transform *simplify(bool &newTransform) + +* **parameters:** + + Parameter | Type | Description + --- | --- | --- + newTransform | bool & | True if a new, simplified, transform was allocated inside this call, false otherwise + +* **output:** ([Transform](../transform/transform.md) \*) Returns itself if none of the children can be simplified. newTransform is false in this case. If any child can be simplified a new [CompositeTransform](compositetransform.md) is allocated and its children are set as the result of calling simplify on each of the old children. newTransform is true in this case + +## [Transform](../transform/transform.md) \*smartCopy(bool &newTransform) {: #smartcopy } + +Get a smart copy, meaning a copy only if one is required, of this transform + +* **function definition:** + + Transform *smartCopy(bool &newTransform) + +* **parameters:** + + Parameter | Type | Description + --- | --- | --- + newTransform | bool & | True if a new, simplified, transform was allocated inside this call, false otherwise + +* **output:** ([Transform](../transform/transform.md) \*) Returns itself if [isTimeVarying](members.md#istimevarying) is false (no copy needed). newTransform is set to false in this case. If [isTimeVarying](members.md#istimevarying) is true, a new [CompositeTransform](compositetransform.md) is allocated and its children are set to the result of calling smartCopy on each of the old children. newTransform is true in this case. \ No newline at end of file diff --git a/docs/docs/api_docs/cpp_api/compositetransform/members.md b/docs/docs/api_docs/cpp_api/compositetransform/members.md new file mode 100644 index 0000000..7710dbc --- /dev/null +++ b/docs/docs/api_docs/cpp_api/compositetransform/members.md @@ -0,0 +1,3 @@ +Member | Type | Description +--- | --- | --- +isTimeVarying | bool | True if any child transform is time varying, false otherwise \ No newline at end of file diff --git a/docs/docs/api_docs/cpp_api/compositetransform/properties.md b/docs/docs/api_docs/cpp_api/compositetransform/properties.md new file mode 100644 index 0000000..ef4b0ca --- /dev/null +++ b/docs/docs/api_docs/cpp_api/compositetransform/properties.md @@ -0,0 +1,3 @@ +Property | Type | Description +--- | --- | --- +transforms | [QList][QList]<[Transform](../transform/transform.md)\*> | List of child transforms \ No newline at end of file diff --git a/docs/docs/api_docs/cpp_api/compositetransform/statics.md b/docs/docs/api_docs/cpp_api/compositetransform/statics.md new file mode 100644 index 0000000..db3df79 --- /dev/null +++ b/docs/docs/api_docs/cpp_api/compositetransform/statics.md @@ -0,0 +1 @@ +NONE \ No newline at end of file diff --git a/docs/docs/api_docs/cpp_api/filegallery/functions.md b/docs/docs/api_docs/cpp_api/filegallery/functions.md index bc4eae4..aa2d515 100644 --- a/docs/docs/api_docs/cpp_api/filegallery/functions.md +++ b/docs/docs/api_docs/cpp_api/filegallery/functions.md @@ -1,6 +1,6 @@ ## void init() {: #init } -Initialize the [FileGallery](filegallery.md). This sets [f](members.md#f) using the file name from [file](../object/members.md#file). It also calls [Gallery](../gallery/gallery.md)::[init](../gallery/functions.md#init). +Initialize the [FileGallery](filegallery.md). This sets [f](members.md#f) using the file name from [file](../object/members.md#file). It also calls [Gallery](../gallery/gallery.md)::[init](../object/functions.md#init). * **function definition:** diff --git a/docs/docs/api_docs/cpp_api/timevaryingtransform/constructors.md b/docs/docs/api_docs/cpp_api/timevaryingtransform/constructors.md new file mode 100644 index 0000000..e8a6e5c --- /dev/null +++ b/docs/docs/api_docs/cpp_api/timevaryingtransform/constructors.md @@ -0,0 +1,3 @@ +Constructor | Description +--- | --- +TimeVaryingTransform(bool independent = true, bool trainable = true) | Default constructor. Calls [Transform](../transform/transform.md) [constructor](../transform/constructors.md) with the given value of [independent](../transform/members.md#independent) and [trainable](../transform/members.md#trainable). It also initializes [timeInvariantAlias](members.md#timeinvariantalias). \ No newline at end of file diff --git a/docs/docs/api_docs/cpp_api/timevaryingtransform/functions.md b/docs/docs/api_docs/cpp_api/timevaryingtransform/functions.md new file mode 100644 index 0000000..1536119 --- /dev/null +++ b/docs/docs/api_docs/cpp_api/timevaryingtransform/functions.md @@ -0,0 +1,128 @@ +## bool timeVarying() {: #timevarying } + +This is a virtual function. Check if the transform is time varying. This always evaluates to true. + +* **function definition:** + + virtual bool timeVarying() const + +* **parameters:** NONE +* **output:** (bool) Returns true (the transform is always time varying) + +## void project(const [Template](../template/template.md) &src, [Template](../template/template.md) &dst) {: #project-1 } + +This is a virtual function. For [TimeVaryingTransforms](timevaryingtransform.md) normal enrollment calls [projectUpdate](#projectupdate-2). It is still possible to call this version of project instead but it must be done explicitly and is **strongly** discouraged. If this function is called [timeInvariantAlias](members.md#timeinvariantalias) is used to call [projectUpdate](#projectupdate-2) in a thread safe way. + +* **function definition:** + + virtual void project(const Template &src, Template &dst) const + + * **parameters:** + + Parameter | Type | Description + --- | --- | --- + src | const [Template](../template/template.md) & | The input template + dst | [Template](../template/template.md) & | The output template + +* **output:** (void) + +## void project(const [TemplateList](../templatelist/templatelist.md) &src, [TemplateList](../templatelist/templatelist.md) &dst) {: #project-2 } + +This is a virtual function. For [TimeVaryingTransforms](timevaryingtransform.md) normal enrollment calls [projectUpdate](#projectupdate-2). It is still possible to call this version of project instead but it must be done explicitly and is **strongly** discouraged. If this function is called [timeInvariantAlias](members.md#timeinvariantalias) is used to call [projectUpdate](#projectupdate-2) in a thread safe way. + +* **function definition:** + + virtual void project(const TemplateList &src, TemplateList &dst) const + + * **parameters:** + + Parameter | Type | Description + --- | --- | --- + src | const [TemplateList](../templatelist/templatelist.md) & | The input template list + dst | [TemplateList](../templatelist/templatelist.md) & | The output template list + +* **output:** (void) + +## void projectUpdate(const [Template](../template/template.md) &src, [Template](../template/template.md) &dst) {: #projectupdate-1 } + +This is a virtual function. This function should never be called because it is useless to implement a Template -> Template call using project update. An error is thrown if it is called. + +* **function definition:** + + virtual void projectUpdate(const Template &src, Template &dst) + +* **parameters:** + + Parameter | Type | Description + --- | --- | --- + src | const [Template](../template/template.md) & | The input template + dst | [Template](../template/template.md) & | The output template + +* **output:** (void) + +## void projectUpdate(const [Template](../template/template.md) &src, [Template](../template/template.md) &dst) {: #projectupdate-1 } + +This is a virtual function. This function should never be called because it is useless to implement a Template -> Template call using project update. An error is thrown if it is called. + +* **function definition:** + + virtual void projectUpdate(const Template &src, Template &dst) + +* **parameters:** + + Parameter | Type | Description + --- | --- | --- + src | const [Template](../template/template.md) & | The input template + dst | [Template](../template/template.md) & | The output template + +* **output:** (void) + +## void projectUpdate(const [Template](../template/template.md) &src, [Template](../template/template.md) &dst) {: #projectupdate-1 } + +This is a virtual function. This function should never be called because it is useless to implement a Template -> Template call using project update. An error is thrown if it is called. + +* **function definition:** + + virtual void projectUpdate(const Template &src, Template &dst) + +* **parameters:** + + Parameter | Type | Description + --- | --- | --- + src | const [Template](../template/template.md) & | The input template + dst | [Template](../template/template.md) & | The output template + +* **output:** (void) + +## void projectUpdate(const [TemplateList](../templatelist/templatelist.md) &src, [TemplateList](../templatelist/templatelist.md) &dst) {: #projectupdate-2 } + +This is a virtual function. This is the non-const alternative to [project](../transform/functions.md#project-1). It allows the internal state of the transform to be update at project time. + +* **function definition:** + + virtual void projectUpdate(const TemplateList &src, TemplateList &dst) + +* **parameters:** + + Parameter | Type | Description + --- | --- | --- + src | const [TemplateList](../templatelist/templatelist.md) & | The input template list + dst | [TemplateList](../templatelist/templatelist.md) & | The output template list + +* **output:** (void) + +## [Transform](../transform/transform.md) \*smartCopy(bool &newTransform) {: #smartcopy } + +This is a virtual function. Make a smart copy of the transform. + +* **function definition:** + + virtual Transform *smartCopy(bool &newTransform) + +* **parameters:** + + Parameter | Type | Description + --- | --- | --- + newTransform | bool & | True if a new, simplified, transform was allocated inside this call, false otherwise + +* **output:** ([Transform](../transform/transform.md) \*) Returns a copy of itself by default. newTransform is set to true in this case. \ No newline at end of file diff --git a/docs/docs/api_docs/cpp_api/timevaryingtransform/members.md b/docs/docs/api_docs/cpp_api/timevaryingtransform/members.md new file mode 100644 index 0000000..f20dc9f --- /dev/null +++ b/docs/docs/api_docs/cpp_api/timevaryingtransform/members.md @@ -0,0 +1,3 @@ +Members | Type | Description +--- | --- | --- + timeInvariantAlias | TimeInvariantWrapperTransform | A special case helper class that ensures usually thread safe calls like [project](../transform/functions.md#project-1) can be called non-thread safe objects. In a very simplified description, this calls [projectUpdate](functions.md#projectupdate-1) instead of [project](functions.md#project-1) \ No newline at end of file diff --git a/docs/docs/api_docs/cpp_api/timevaryingtransform/statics.md b/docs/docs/api_docs/cpp_api/timevaryingtransform/statics.md new file mode 100644 index 0000000..db3df79 --- /dev/null +++ b/docs/docs/api_docs/cpp_api/timevaryingtransform/statics.md @@ -0,0 +1 @@ +NONE \ No newline at end of file diff --git a/docs/docs/api_docs/cpp_api/timevaryingtransform/timevaryingtransform.md b/docs/docs/api_docs/cpp_api/timevaryingtransform/timevaryingtransform.md index 7b2d121..c6477ac 100644 --- a/docs/docs/api_docs/cpp_api/timevaryingtransform/timevaryingtransform.md +++ b/docs/docs/api_docs/cpp_api/timevaryingtransform/timevaryingtransform.md @@ -1,3 +1,14 @@ Inherits [Transform](../transform/transform.md) + +A [Transform](../transform/transform.md) that can change state at project time. + +See: + +* [Members](members.md) +* [Constructors](constructors.md) +* [Static Functions](statics.md) +* [Functions](functions.md) + +TimeVaryingTransforms are [Transforms](../transform/transform.md) that need to change their state at project time. This is opposed to [Transforms](../transform/transform.md) which can only change state at train time. Common examples include video processing across multiple frames, aggregate projection statistics etc. diff --git a/docs/docs/api_docs/cpp_api/wrappertransform/constructors.md b/docs/docs/api_docs/cpp_api/wrappertransform/constructors.md index e69de29..6b7a6a0 100644 --- a/docs/docs/api_docs/cpp_api/wrappertransform/constructors.md +++ b/docs/docs/api_docs/cpp_api/wrappertransform/constructors.md @@ -0,0 +1,3 @@ +Constructor | Description +--- | --- +WrapperTransform(bool independent = true) | Default constructor. Calls [TimeVaryingTransform](../timevaryingtransform/timevaryingtransform.md) [constructor](../timevaryingtransform/constructors.md) with the given value for independent. \ No newline at end of file diff --git a/docs/docs/api_docs/cpp_api/wrappertransform/functions.md b/docs/docs/api_docs/cpp_api/wrappertransform/functions.md index e69de29..16fd68f 100644 --- a/docs/docs/api_docs/cpp_api/wrappertransform/functions.md +++ b/docs/docs/api_docs/cpp_api/wrappertransform/functions.md @@ -0,0 +1,153 @@ +## bool timeVarying() {: #timevarying } + +Check whether the transform is timeVarying. + +* **function definition:** + + bool timeVarying() const + +* **parameters:** NONE +* **output:** (bool) Returns true if the [child transform](properties.md) is time varying, false otherwise + +## void train(const [QList][QList]<[TemplateList][../templatelist/templatelist.md)> &data) {: #train } + +Call train on the child transform + +* **function defintion:** + + void train(const QList &data) + +* **parameters:** + + Parameter | Type | Description + --- | --- | --- + data | const [QList][QList]<[TemplateList](../templatelist/templatelist.md)> & | The training data + +* **output:** (void) + +## void project(const [Template](../template/template.md) &src, [Template](../template/template.md) &dst) {: #project-1 } + +Call project on the child transform + +* **function definition:** + + void project(const Template &src, Template &dst) const + +* **parameters:** + + Parameter | Type | Description + --- | --- | --- + src | const [Template](../template/template.md) & | The input template + dst | [Template](../template/template.md) & | The output template + +* **output:** (void) + +## void project(const [TemplateList](../templatelist/templatelist.md) &src, [TemplateList](../templatelist/templatelist.md) &dst) {: #project-2 } + +Call project on the child transform + +* **function definition:** + + void project(const TemplateList &src, TemplateList &dst) const + +* **parameters:** + + Parameter | Type | Description + --- | --- | --- + src | const [TemplateList](../templatelist/templatelist.md) & | The input template list + dst | [TemplateList](../templatelist/templatelist.md) & | The output template list + +* **output:** (void) + +## void projectUpdate(const [Template](../template/template.md) &src, [Template](../template/template.md) &dst) {: #projectupdate-1 } + +Call projectUpdate on the child transform + +* **function definition:** + + void projectUpdate(const Template &src, Template &dst) + +* **parameters:** + + Parameter | Type | Description + --- | --- | --- + src | const [Template](../template/template.md) & | The input template + dst | [Template](../template/template.md) & | The output template + +* **output:** (void) + +## void projectUpdate(const [TemplateList](../templatelist/templatelist.md) &src, [TemplateList](../templatelist/templatelist.md) &dst) {: #projectupdate-2 } + +Call projectUpdate on the child transform + +* **function definition:** + + void projectUpdate(const TemplateList &src, TemplateList &dst) + +* **parameters:** + + Parameter | Type | Description + --- | --- | --- + src | const [TemplateList](../templatelist/templatelist.md) & | The input template list + dst | [TemplateList](../templatelist/templatelist.md) & | The output template list + +* **output:** (void) + +## void init() {: #init } + +Initialize the transform. Sets [trainable](../transform/members.md#trainable) to match the child transform (if the child is trainable so is the wrapper) + +* **function definition:** + + void init() + +* **parameters:** NONE +* **output:** (void) + +## void finalize([TemplateList](../templatelist/templatelist.md) &output) {: #finalize } + +This is a virtual function. Call finalize on the child transform + +* **function definition:** + + virtual void finalize(TemplateList &output) + +* **parameters:** + + Parameter | Type | Description + --- | --- | --- + output | const [TemplateList](../templatelist/templatelist.md) & | The output to finalize + +* **output:** (void) + +## [Transform](../transform/transform.md) \*simplify(bool &newTransform) {: #simplify } + +This is a virtual function. Calls simplify on the child transform. + +* **function definition:** + + virtual Transform *simplify(bool &newTransform) + +* **parameters:** + + Parameter | Type | Description + --- | --- | --- + newTransform | bool & | True if a new, simplified, transform was allocated inside this call, false otherwise + +* **output:** ([Transform](../transform/transform.md) \*) Returns itself if the child transform cannot be simplified. newTransform is set to false in this case. If the child can be simplified, a new WrapperTransform is allocated with the child transform set as the simplified version of the old child transform. newTransform is set to true in this case + +## [Transform](../transform/transform.md) \*smartCopy(bool &newTransform) {: #smartcopy } + +Get a smart copy, meaning a copy only if one is required, of this transform + +* **function definition:** + + Transform *smartCopy(bool &newTransform) + +* **parameters:** + + Parameter | Type | Description + --- | --- | --- + newTransform | bool & | True if a new, simplified, transform was allocated inside this call, false otherwise + +* **output:** ([Transform](../transform/transform.md) \*) Returns itself if the child transform is not time varying (no copy needed). newTransform is set to false in this case. If the child is time varying make a copy by calling [smartCopy](../timevaryingtransform/functions.md#smartcopy) on the child. newTransform is set to true in this case. \ No newline at end of file diff --git a/docs/docs/api_docs/cpp_api/wrappertransform/members.md b/docs/docs/api_docs/cpp_api/wrappertransform/members.md index e69de29..db3df79 100644 --- a/docs/docs/api_docs/cpp_api/wrappertransform/members.md +++ b/docs/docs/api_docs/cpp_api/wrappertransform/members.md @@ -0,0 +1 @@ +NONE \ No newline at end of file diff --git a/docs/docs/api_docs/cpp_api/wrappertransform/properties.md b/docs/docs/api_docs/cpp_api/wrappertransform/properties.md index e69de29..2bfeb56 100644 --- a/docs/docs/api_docs/cpp_api/wrappertransform/properties.md +++ b/docs/docs/api_docs/cpp_api/wrappertransform/properties.md @@ -0,0 +1,3 @@ +Property | Type | Description +--- | --- | --- +transform | [Transform](../transform/transform.md) * | The child transform \ No newline at end of file diff --git a/docs/docs/api_docs/cpp_api/wrappertransform/statics.md b/docs/docs/api_docs/cpp_api/wrappertransform/statics.md index e69de29..db3df79 100644 --- a/docs/docs/api_docs/cpp_api/wrappertransform/statics.md +++ b/docs/docs/api_docs/cpp_api/wrappertransform/statics.md @@ -0,0 +1 @@ +NONE \ No newline at end of file diff --git a/docs/docs/api_docs/cpp_api/wrappertransform/wrappertransform.md b/docs/docs/api_docs/cpp_api/wrappertransform/wrappertransform.md index 2377dc7..36b4ef9 100644 --- a/docs/docs/api_docs/cpp_api/wrappertransform/wrappertransform.md +++ b/docs/docs/api_docs/cpp_api/wrappertransform/wrappertransform.md @@ -1,8 +1,8 @@ -Inherits [Transform](../transform/transform.md) +Inherits [TimeVaryingTransform](../timevaryingtransform/timevaryingtransform.md) -Base class for [Transforms](../transform/transform.md) that act as decorators of another [Transform](../transform/transform.md) +Base class for [Transforms](../transform/transform.md) that have a single child transform See: @@ -10,4 +10,6 @@ See: * [Members](members.md) * [Constructors](constructors.md) * [Static Functions](statics.md) -* [Functions](functions.md) \ No newline at end of file +* [Functions](functions.md) + +WrapperTransforms are the base class for plugins that have a child transform. It inherits from [TimeVaryingTransform](../timevaryingtransform/timevaryingtransform.md) so that it can properly handle a child transform that is also time varying, WrapperTransform itself has no requirement to be time varying. The main purpose of WrapperTransform is to intelligently implement [simplify](functions.md#simplify) and [smartCopy](functions.md#smartcopy), all other calls are just passed to the child. \ No newline at end of file diff --git a/docs/docs/contribute.md b/docs/docs/contribute.md index 58800ee..6854a42 100644 --- a/docs/docs/contribute.md +++ b/docs/docs/contribute.md @@ -72,4 +72,57 @@ Finally, OpenBR supports automatic linking for abstractions found in comments. F ## Contributing to the API -You should contribute to the API if you want to add a new abstraction or extend an existing abstraction with new functionality. Please note, this occurs *very* *very* rarely. Our goal is to leave the core API as stable and consistent as possible and change only the surrounding plugins. If you believe your idea offers exciting new functionality or greatly increases efficiency please [open an issue](https://github.com/biometrics/openbr/issues) so that it can be discussed as a community. \ No newline at end of file +You should contribute to the API if you want to add a new abstraction or extend an existing abstraction with new functionality. Please note, this occurs *very* *very* rarely. Our goal is to leave the core API as stable and consistent as possible and change only the surrounding plugins. If you believe your idea offers exciting new functionality or greatly increases efficiency please [open an issue](https://github.com/biometrics/openbr/issues) so that it can be discussed as a community. + +--- + +## Style Guide + +The most important rule is that **new code should be consistent with the existing code around it**. The rules below illustrate the preferred style when cleaning up existing inconsistently-styled code. + +These rules are a work in progress and are subject to additions. Changes to the style can be made with a pull request implementing the change across the entire repository. + +### Structs & Classes + struct FooBar + { + + }; + +### Functions + int *fooBar(const int &x, int *y, int z) + { + *y = x + z; + return y; + } + +### Variables + int x = 2; + int *y = &x; + int &z = x; + +### Loops and Conditionals +#### Single-statement + for (int i=start; i
The two principal software artifacts are the shared library 'openbr' and command line application 'br'.
@@ -33,28 +39,28 @@ OpenBR is supported on multiple operating systems. Please select yours from the ## The Basics -We have created a few tutorials to help teach you the basic principles of the OpenBR system. If this is your first time using OpenBR you should look at these before moving on to the more technical descriptions below. +We have created a few tutorials to help teach you the basic principles of the OpenBR system. If this is your first time using OpenBR you should look at these before moving on to the more in-depth documentation below. * [Quick Start](tutorials.md#quick-start) -* [Training in OpenBR](tutorials.md#training-algorithms) +* [Algorithms in OpenBR](tutorials.md#algorithms-in-openbr) +* [Training Algorithms in OpenBR](tutorials.md#training-algorithms) +* [The Evaluation Harness](tutorials.md#the-evaluation-harness) * [Face Recognition](tutorials.md#face-recognition) * [Age Estimation](tutorials.md#age-estimation) * [Gender Estimation](tutorials.md#gender-estimation) --- -## Additional Materials +## The Documentation -OpenBR has several parts. Below are several more technical descriptions of important parts of the system. Enjoy! +Here is the complete documentation for OpenBR. Enjoy! -* [Algorithms in OpenBR](technical.md#algorithms-in-openbr) * [The C API](api_docs/c_api.md) -* [The Command Line Interface](api_docs/cl_api.md) * [The C++ Plugin API](api_docs/cpp_api.md) -* [The Evaluation Harness](technical.md#the-evaluation-harness) +* [The Command Line Interface](api_docs/cl_api.md) --- ## Help -Still have questions? Please reach out to us on our developer mailing list or our IRC channel +Still have questions? Please reach out to us on our [developer mailing list](https://groups.google.com/forum/?fromgroups#!forum/openbr-dev) or our [IRC channel](http://webchat.freenode.net/?channels=openbr). diff --git a/docs/docs/technical.md b/docs/docs/technical.md deleted file mode 100644 index 084d58c..0000000 --- a/docs/docs/technical.md +++ /dev/null @@ -1,92 +0,0 @@ -# Algorithms in OpenBR - -**So you've run `scripts/helloWorld.sh` and it generally makes sense, except you have no idea what `'Open+Cvt(Gray)+Cascade(FrontalFace)+ASEFEyes+Affine(128,128,0.33,0.45)+CvtFloat+PCA(0.95):Dist(L2)'` means or how it is executed.** - -Well if this is the case, you've found the right documentation. -Let's get started! - -In OpenBR an *algorithm* is a technique for enrolling templates associated with a technique for comparing them. -Recall that our ultimate goal is to be able to say how similar two face images are (or two fingerprints, irises, etc.). -Instead of storing the entire raw image for comparison, it is common practice to store an optimized representation, or *template*, of the image for the task at hand. -The process of generating this optimized representation is called *template enrollment* or *template generation*. -Given two templates, *template comparison* computes the similarity between them, where the higher values indicate more probable matches and the threshold for determining what constitutes an adequate match is determined operationally. -The goal of template generation is to design templates that are small, accurate, and fast to compare. -Ok, you probably knew all of this already, let's move on. - -The only way of creating an algorithm in OpenBR is from a text string that describes it. -We call this string the *algorithm description*. -The algorithm description is separated into two parts by a ':', with the left hand side indicating how to generate templates and the right hand side indicating how to compare them. -Some algorithms, like [age_estimation](tutorials.md#age-estimation) and [gender estimation](tutorials.md#gender-estimation) are *classifiers* that don't create templates. -In these cases, the colon and the template comparison technique can be omitted from the algorithm description. - -There are several motivations for mandating that algorithms are defined from these strings, here are the most important: - 1. It ensures good software development practices by forcibly decoupling the development of each step in an algorithm, facilitating the modification of algorithms and the re-use of individual steps. - 2. It spares the creation and maintenance of a lot of very similar header files that would otherwise be needed for each step in an algorithm, observe the absence of headers in `openbr/plugins`. - 3. It allows for algorithm parameter tuning without recompiling. - 4. It is completely unambiguous, both the OpenBR interpreter and anyone familiar with the project can understand exactly what your algorithm does just from this description. - -Let's look at some of the important parts of the code base that make this possible! - -In `AlgorithmCore::init()` in `openbr/core/core.cpp` you can see the code for splitting the algorithm description at the colon. -Shortly thereafter in this function we *make* the template generation and comparison methods. -These make calls are defined in the public [C++ plugin API](api_docs/cpp_api.md) and can also be called from end user code. - -Below we discuss some of the source code for `Transform::make` in `openbr/openbr_plugin.cpp`. -Note, the make functions for other plugin types are similar in spirit and will not be covered. - -One of the first steps when converting the template enrollment description into a [Transform](api_docs/cpp_api/transform/transform.md) is to replace the operators, like '+', with their full form: - - { // Check for use of '+' as shorthand for Pipe(...) - QStringList words = parse(str, '+'); - if (words.size() > 1) - return make("Pipe([" + words.join(",") + "])", parent); - } - -A pipe (see [PipeTransform](api_docs/plugins/core.md#pipetransform)) is the standard way of chaining together multiple steps in series to form more sophisticated algorithms. -PipeTransform takes a list of transforms, and *projects* templates through each transform in order. - -After operator expansion, the template enrollment description forms a tree, and the transform is constructed from this description starting recursively starting at the root of the tree: - - Transform *transform = Factory::make("." + str); - -At this point we reach arguably the most important code in the entire framework, the *object factory* in `openbr/openbr_plugin.h`. -The [Factory](api_docs/cpp_api/factory/factory.md) class is responsible for constructing an object from a string: - - static T *make(const File &file) - { - QString name = file.get("plugin", ""); - if (name.isEmpty()) name = file.suffix(); - if (!names().contains(name)) { - if (names().contains("Empty") && name.isEmpty()) name = "Empty"; - else if (names().contains("Default")) name = "Default"; - else qFatal("%s registry does not contain object named: %s", qPrintable(baseClassName()), qPrintable(name)); - } - T *object = registry->value(name)->_make(); - static_cast(object)->init(file); - return object; - } - -Going back to our original example, a [PipeTransform](api_docs/plugins/core.md#pipetransform) will be created with [OpenTransform](api_docs/plugins/io.md#opentransform), [CvtTransform](api_docs/plugins/imgproc.md#cvttransform), [CascadeTransform](api_docs/plugins/metadata.md#cascadetransform), [ASEFEyesTransform](api_docs/plugins/metadata.md#asefeyestransform), [AffineTransform](api_docs/plugins/imgproc.md#affinetransform), [CvtFloatTransform](api_docs/plugins/imgproc.md#cvtfloattransform), and [PCATransform](api_docs/plugins/classification.md#pcatransform) as its children. - -If you want all the tedious details about what exactly this algoritm does, then you should read the [project](api_docs/cpp_api/transform/functions.md#project-1) function implemented by each of these plugins. -The brief explanation is that it *reads the image from disk, converts it to grayscale, runs the face detector, runs the eye detector on any detected faces, uses the eye locations to normalize the face for rotation and scale, converts to floating point format, and then embeds it in a PCA subspaced trained on face images*. -If you are familiar with face recognition, you will likely recognize this as the Eigenfaces \cite turk1991eigenfaces algorithm. - -As a final note, the Eigenfaces algorithms uses the Euclidean distance (or L2-norm) to compare templates. -Since OpenBR expects *similarity* values when comparing templates, and not *distances*, the [DistDistance](api_docs/plugins/distance.md#distdistance) will return *-log(distance+1)* so that larger values indicate more similarity. - ---- - -# The Evaluation Harness - -The *Biometric Evaluation Environment* (BEE) is a [NIST](http://www.nist.gov/index.html) standard for evaluating biometric algorithms. - -OpenBR implements the following portions of the BEE specification: - -* Signature Set- A signature set (or *sigset*) is a [Gallery](api_docs/cpp_api/gallery/gallery.md) compliant **XML** file-list specified on page 9 of the [MBGC File Overview](misc/MBGC_file_overview.pdf) and implemented in [xmlGallery](api_docs/plugins/gallery.md#xmlgallery). Sigsets are identified with a **.xml** extension. - -* Similarity Matrix- A similarity matrix (or *simmat*) is an [Output](api_docs/cpp_api/output/output.md) compliant binary score matrix specified on page 12 of the [MBGC File Overview](misc/MBGC_file_overview.pdf) and implemented in [mtxOutput](api_docs/plugins/output.md#mtxoutput). Simmats are identified with a **.mtx** extension. See [br_eval](api_docs/c_api/functions.md#br_eval) for more information. - -* Mask Matrix- A mask matrix (or *mask*) is a binary matrix specified on page 14 of the [MBGC File Overview](misc/MBGC_file_overview.pdf) identifying the ground truth genuines and impostors of a corresponding *simmat*. Masks are identified with a **.mask** extension. See [br_make_mask](api_docs/c_api/functions.md#br_make_mask) and [br_combine_masks](api_docs/c_api/functions.md#br_combine_masks) for more information. - ---- diff --git a/docs/docs/tutorials.md b/docs/docs/tutorials.md index 96b45cb..bab9b09 100644 --- a/docs/docs/tutorials.md +++ b/docs/docs/tutorials.md @@ -34,24 +34,225 @@ You're webcam should be open again but this time a box should have appeared arou Pretty straightforward right? Each plugin completes one task and the passes the output on to the next plugin. You can pipe together as many plugins as you like as long as the output data from one can be the input data to the next. But wait! Output data? Input data? we haven't talked about data at all yet! How does OpenBR handle data? There are two objects that handle data is OpenBR; [Files](api_docs/cpp_api/file/file.md), which handle metadata, and [Templates](api_docs/cpp_api/template/template.md) which are containers for images and [Files](api_docs/cpp_api/file/file.md). Let's talk about [Files](api_docs/cpp_api/file/file.md) first. A file consists of file name, which is a path to a file on disk, and metadata which is a map of key-value pairs. The metadata can contain any textual information about the file. In the example above we use it to store the rectangles detected by [Cascade](api_docs/plugins/metadata.md#cascadetransform) and pass them along to [Draw](api_docs/plugins/gui.md#drawtransform) for drawing. [Templates](api_docs/cpp_api/template/template.md) are containers for images, given as OpenCV [Mats][Mat] and [Files](api_docs/cpp_api/file/file.md). They can contain one image or a list of images. Plugins are either [Template](api_docs/cpp_api/template/template.md) in, [Template](api_docs/cpp_api/template/template.md) out or [TemplateList](api_docs/cpp_api/templatelist/templatelist.md) in, [TemplateList](api_docs/cpp_api/templatelist/templatelist.md) out. [TemplateLists](api_docs/cpp_api/templatelist/templatelist.md) are, of course, just a list of [Templates](api_docs/cpp_api/template/template.md) which a few functions added for your convenience. -And there you go! You have gotten your quick start in OpenBR. We covered the [command line](api_docs/cl_api.md), [plugins, and the key data structures](api_docs/cpp_api.md) in OpenBR. If you want to learn more the next few tutorials will cover these fields with more depth. For even more information check out the [technical overviews](technical.md) and class documentation! +And there you go! You have gotten your quick start in OpenBR. We covered the [command line](api_docs/cl_api.md), [plugins, and the key data structures](api_docs/cpp_api.md) in OpenBR. If you want to learn more the next few tutorials will cover these fields with more depth. --- # Algorithms in OpenBR +The purpose of OpenBR is to express biometrics algorithms in a consistent and simple way. But what exactly does *algorithm* mean? In OpenBR an *algorithm* is a technique for enrolling templates associated with a technique for comparing them. Recall that our ultimate goal is to be able to say how similar two face images are (or two fingerprints, irises, etc.). + +Instead of storing the entire raw image for comparison, it is common practice to store an optimized representation, or *template*, of the image for the task at hand. The process of generating this optimized representation is called *template enrollment* or *template generation*. Given two templates, *template comparison* computes the similarity between them, where the higher values indicate more probable matches and the threshold for determining what constitutes an adequate match is determined operationally. The goal of template generation is to design templates that are small, accurate, and fast to compare. + +The only way of creating an algorithm in OpenBR is from a text string that describes it. We call this string the *algorithm description*. There are several motivations for mandating that algorithms are defined from these strings, here are the most important: + +1. It ensures good software development practices by forcibly decoupling the development of each step in an algorithm, facilitating the modification of algorithms and the re-use of individual steps. +2. It spares the creation and maintenance of a lot of very similar header files that would otherwise be needed for each step in an algorithm, observe the absence of headers in `openbr/plugins`. +3. It allows for algorithm parameter tuning without recompiling. +4. It is completely unambiguous, both the OpenBR interpreter and anyone familiar with the project can understand exactly what your algorithm does just from this description. + +OpenBR provides a syntax for creating more concise algorithm descriptions. The relevant symbols are: + +Symbol | Meaning +--- | --- +PluginName(property1=value1,...propertyN=valueN) | A plugin is described by its name (without the abstraction) and a list of properties and values. Properties of a plugin that are not specified are set to their default values. +: | Seperates *templatelate enrollment* from *template comparison*. Enrollment is on the left and comparison is on the right. This is optional, algorithms without comparisons are called *classifiers* +\+ | Pipe. Joins plugins together and projects input through them in series. The output of a plugin to the left of \+ become the input of the plugin to the right. + / | Fork. Joins plugins together and projects input through them in parallel. All plugins get the same input and the output is concattenated together. + \{\} | Cache. Cache the output of a plugin in memory for quick lookups later on + <\> | LoadStore. Plugins inside are stored on disk after training and / or loaded from disk before projection + () | Order of operations. Change the order of operations using parantheses. + +Let's look at some of the important parts of the code base that make this possible! + +In ```AlgorithmCore::init()``` in ```openbr/core/core.cpp``` you can see the code for splitting the algorithm description at the colon. +Shortly thereafter in this function we *make* the template generation and comparison methods. +These make calls are defined in the public [C++ plugin API](api_docs/cpp_api.md) and can also be called from end user code. + +Below we discuss some of the source code for `Transform::make` in `openbr/openbr_plugin.cpp`. +Note, the make functions for other plugin types are similar in spirit and will not be covered. + +One of the first steps when converting the template enrollment description into a [Transform](api_docs/cpp_api/transform/transform.md) is to replace the operators, like '\+', with their full form: + + { // Check for use of '+' as shorthand for Pipe(...) + QStringList words = parse(str, '+'); + if (words.size() > 1) + return make("Pipe([" + words.join(",") + "])", parent); + } + +After operator expansion, the template enrollment description forms a tree, and the transform is constructed from this description starting recursively starting at the root of the tree: + + Transform *transform = Factory::make("." + str); + +Let's use the algorithm in ```scripts/helloWorld.sh``` as an example. The algorithm is: + + Open+Cvt(Gray)+Cascade(FrontalFace)+ASEFEyes+Affine(128,128,0.33,0.45)+CvtFloat+PCA(0.95):Dist(L2) + +So what is happening here? Let's expand this using our new knowledge of OpenBR's algorithm syntax. First, the algorithm will be split into enrollment and comparison portions at the **:**. So enrollment becomes- + + Open+Cvt(Gray)+Cascade(FrontalFace)+ASEFEyes+Affine(128,128,0.33,0.45)+CvtFloat+PCA(0.95) + +and comparison is- + + Dist(L2) + +On the enrollment side the **+'s** are converted into a [PipeTransform](api_docs/plugins/core.md#pipetransform) with the other plugins as children. Enrollment is transformed to + + Pipe(transforms=[Open,Cvt(Gray),Cascade(FrontalFace),ASEFEyes,Affine(128,128,0.33,0.45,CvtFloat,PCA(0.95)]) + +If you want all the tedious details about what exactly this algoritm does, then you should read the [project](api_docs/cpp_api/transform/functions.md#project-1) function implemented by each of these plugins. +The brief explanation is that it *reads the image from disk, converts it to grayscale, runs the face detector, runs the eye detector on any detected faces, uses the eye locations to normalize the face for rotation and scale, converts to floating point format, and then embeds it in a PCA subspaced trained on face images*. +If you are familiar with face recognition, you will likely recognize this as the Eigenfaces[^1] algorithm. + +As a final note, the Eigenfaces algorithms uses the Euclidean distance (or L2-norm) to compare templates. +Since OpenBR expects *similarity* values when comparing templates, and not *distances*, the [DistDistance](api_docs/plugins/distance.md#distdistance) will return *-log(distance+1)* so that larger values indicate more similarity. + --- # Training Algorithms --- +# The Evaluation Harness + +OpenBR implements a complete, [NIST](http://www.nist.gov/index.html) compliant, evaluation harness for evaluating face recognition, face detection, and facial landmarking. The goal is to provide a consistent environment for the repeatable evaluation of algorithms to the academic and open source communities. To accompish this OpenBR defines the following portions of the biometrics evaluation evironment (BEE) standard- + +* Signature Set- A signature set (or *sigset*) is a [Gallery](api_docs/cpp_api/gallery/gallery.md) compliant **XML** file-list specified on page 9 of the [MBGC File Overview](misc/MBGC_file_overview.pdf) and implemented in [xmlGallery](api_docs/plugins/gallery.md#xmlgallery). Sigsets are identified with a **.xml** extension. + +* Similarity Matrix- A similarity matrix (or *simmat*) is an [Output](api_docs/cpp_api/output/output.md) compliant binary score matrix specified on page 12 of the [MBGC File Overview](misc/MBGC_file_overview.pdf) and implemented in [mtxOutput](api_docs/plugins/output.md#mtxoutput). Simmats are identified with a **.mtx** extension. See [br_eval](api_docs/c_api/functions.md#br_eval) for more information. + +* Mask Matrix- A mask matrix (or *mask*) is a binary matrix specified on page 14 of the [MBGC File Overview](misc/MBGC_file_overview.pdf) identifying the ground truth genuines and impostors of a corresponding *simmat*. Masks are identified with a **.mask** extension. See [br_make_mask](api_docs/c_api/functions.md#br_make_mask) and [br_combine_masks](api_docs/c_api/functions.md#br_combine_masks) for more information. + +The evaluation harness is also accessible from the command line! Please see [-eval](api_docs/cl_api.md#eval), [-evalDetection](api_docs/cl_api.md#evaldetection), [-evalLandmarking](api_docs/cl_api.md#evallandmarking), [-evalClassification](api_docs/cl_api.md#evalclassification), [-evalClustering](api_docs/cl_api.md#evalclustering), or [-evalRegression](api_docs/cl_api.md#evalregression) for relevant information. + +--- + # Face Recognition +This tutorial gives an example on how to perform face recognition in OpenBR. OpenBR implements the 4SF[^2] algorithm to perform face recognition. Please read the the paper for more specific implementation details. + +To start, lets run face recognition from the command line. Open the terminal and enter + + $ br -algorithm FaceRecognition \ + -compare ../data/MEDS/img/S354-01-t10_01.jpg ../data/MEDS/img/S354-02-t10_01.jpg \ + -compare ../data/MEDS/img/S354-01-t10_01.jpg ../data/MEDS/img/S386-04-t10_01.jpg + +Easy enough? You should see results printed to terminal that look like + + $ Set algorithm to FaceRecognition + $ Loading /usr/local/share/openbr/models/algorithms/FaceRecognition + $ Loading /usr/local/share/openbr/models/transforms//FaceRecognitionExtraction + $ Loading /usr/local/share/openbr/models/transforms//FaceRecognitionEmbedding + $ Loading /usr/local/share/openbr/models/transforms//FaceRecognitionQuantization + $Comparing ../data/MEDS/img/S354-01-t10_01.jpg and ../data/MEDS/img/S354-02-t10_01.jpg + $ Enrolling ../data/MEDS/img/S354-01-t10_01.jpg to S354-01-t10_01r7Rv4W.mem + $ 100.00% ELAPSED=00:00:00 REMAINING=00:00:00 COUNT=1 + $ 100.00% ELAPSED=00:00:00 REMAINING=00:00:00 COUNT=1 + $ 1.8812 + $ Comparing ../data/MEDS/img/S354-01-t10_01.jpg and ../data/MEDS/img/S386-04-t10_01.jpg + $ Enrolling ../data/MEDS/img/S354-01-t10_01.jpg to S354-01-t10_01r7Rv4W.mem + $ 100.00% ELAPSED=00:00:00 REMAINING=00:00:00 COUNT=1 + $ 100.00% ELAPSED=00:00:00 REMAINING=00:00:00 COUNT=1 + $ 0.571219 + +So what is 'FaceRecognition'? It's an abbrieviation to make running the algorithm easier. All of the algorithm abbreviations are located in ```openbr/plugins/core/algorithms.cpp```, please see the previous tutorial for an introduction to OpenBR's algorithm grammar. + +It also possible to perform face recognition evaluation (**note:** this requires [R][R] to be installed)- + + $ br -algorithm FaceRecognition -path ../data/MEDS/img/ \ + -enroll ../data/MEDS/sigset/MEDS_frontal_target.xml target.gal \ + -enroll ../data/MEDS/sigset/MEDS_frontal_query.xml query.gal \ + -compare target.gal query.gal scores.mtx \ + -makeMask ../data/MEDS/sigset/MEDS_frontal_target.xml ../data/MEDS/sigset/MEDS_frontal_query.xml MEDS.mask \ + -eval scores.mtx MEDS.mask Algorithm_Dataset/FaceRecognition_MEDS.csv \ + -plot Algorithm_Dataset/FaceRecognition_MEDS.csv MEDS + +face recognition search- + + $ br -algorithm FaceRecognition -enrollAll -enroll ../data/MEDS/img 'meds.gal;meds.csv[separator=;]' + $ br -algorithm FaceRecognition -compare meds.gal ../data/MEDS/img/S001-01-t10_01.jpg match_scores.csv + +and face recognition training- + + $ br -algorithm 'Open+Cvt(Gray)+Cascade(FrontalFace)+ASEFEyes+Affine(128,128,0.33,0.45)+(Grid(10,10)+SIFTDescriptor(12)+ByRow)/(Blur(1.1)+Gamma(0.2)+DoG(1,2)+ContrastEq(0.1,10)+LBP(1,2)+RectRegions(8,8,6,6)+Hist(59))+PCA(0.95)+Normalize(L2)+Dup(12)+RndSubspace(0.05,1)+LDA(0.98)+Cat+PCA(0.95)+Normalize(L1)+Quantize:NegativeLogPlusOne(ByteL1)' -train ../data/ATT/img FaceRecognitionATT + +all right from the command line! The entire command line API is documented [here](api_docs/cl_api.md). It is a powerful tool for creating and testing new algorithms. + +The command line isn't perfect for all situations however. So OpenBR exposes a [C++ API](api_docs/cpp_api.md) that can be embedded pretty much everywhere. Let's step through the example code at ```app/example/face_recognition.cpp``` and learn about using OpenBR as a library. + +Our main function starts with- + + br::Context::initialize(argc, argv) + +This is the first step in any OpenBR application, it initializes the global context. + + QSharedPointer transform = br::Transform::fromAlgorithm("FaceRecognition"); + QSharedPointer distance = br::Distance::fromAlgorithm("FaceRecognition"); + +Here, we split our algorithm into *enrollment* ([Transform](api_docs/cpp_api/transform/transform.md)::[fromAlgorithm](api_docs/cpp_api/transform/statics.md#fromalgorithm)) and *comparison* ([Distance](api_docs/cpp_api/distance/distance.md)::[fromAlgorithm](api_docs/cpp_api/distance/statics.md#fromalgorithm)) + + br::Template queryA("../data/MEDS/img/S354-01-t10_01.jpg"); + br::Template queryB("../data/MEDS/img/S382-08-t10_01.jpg"); + br::Template target("../data/MEDS/img/S354-02-t10_01.jpg"); + +These lines create our [Templates](api_docs/cpp_api/template/template.md) for enrollment. They are just images loaded from disk. For this example queryA is the same person (different picture) as target and queryB is a different person. + + queryA >> *transform; + queryB >> *transform; + target >> *transform; + +And now we enroll them! **>>** is a convienience operator for enrolling [Templates](api_docs/cpp_api/template/template.md) in [Transforms](api_docs/cpp_api/transform/transform.md). The enrollment is done in-place, which means the output overwrites the input. Our [Templates](api_docs/cpp_api/template/template.md) now store the results of enrollment. + + float comparisonA = distance->compare(target, queryA); + float comparisonB = distance->compare(target, queryB); + +Compare our query [Templates](api_docs/cpp_api/template/template.md) against the target [Template](api_docs/cpp_api/template/template.md). The result is a float. + + printf("Genuine match score: %.3f\n", comparisonA); + printf("Impostor match score: %.3f\n", comparisonB); + +Print out our results. You can see that comparisonA (between queryA and target) has a higher score then comparisonB, which is exactly what we expect! + + br::Context::finalize(); + +The last line in any OpenBR application has to be call to finalize. This shuts down OpenBR. + +And that's it! Now you can embed face recognition into all of your applications. + --- # Age Estimation +Age Estimation is very similar in spirit to [Face Recognition](#face-recognition) and will be covered in far less detail. + +To implement Age Estimation from the command line you can run + + $ br -algorithm AgeEstimation \ + -enroll ../data/MEDS/img/S354-01-t10_01.jpg ../data/MEDS/img/S001-01-t10_01.jpg metadata.csv + +The results will be stored in metadata.csv under the key 'Age'. Remember from the [Face Recognition](#face-recognition) 'AgeEstimation' is just an abbreviation for the full algorithm description. The expanded version is stored in ```openbr/plugins/core/algorithms.cpp```. + +The source code to run age estimation as an application is in ```app/examples/age_estimation.cpp``` + --- # Gender Estimation + +Gender Estimation is very similar in spirit to [Face Recognition](#face-recognition) and will be covered in far less detail. + +To implement Gender Estimation from the command line you can run + + $ br -algorithm GenderEstimation \ + -enroll ../data/MEDS/img/S354-01-t10_01.jpg ../data/MEDS/img/S001-01-t10_01.jpg metadata.csv + +The results will be stored in metadata.csv under the key 'Gender'. Remember from the [Face Recognition](#face-recognition) 'GenderEstimation' is just an abbreviation for the full algorithm description. The expanded version is stored in ```openbr/plugins/core/algorithms.cpp```. + +The source code to run gender estimation as an application is in ```app/examples/gender_estimation.cpp``` + + +[^1]: *Matthew Turk and Alex Pentland.* + **Eigenfaces for recognition.** + Journal of Cognitive Neuroscience, 71-86, 1991 +[^2]: *B. Klare.* + **Spectrally sampled structural subspace features (4SF).** + In Michigan State University Technical Report, MSUCSE-11-16, 2011 diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml index 2f7509a..ec57da9 100644 --- a/docs/mkdocs.yml +++ b/docs/mkdocs.yml @@ -13,12 +13,12 @@ include_next_prev: False markdown_extensions: - attr_list +- footnotes pages: - [index.md, Home] - [install.md, Install] - [tutorials.md, Tutorials] -- [technical.md, Technical Overviews] - [contribute.md, Contribute] - [papers.md, Publications] @@ -134,12 +134,26 @@ pages: # TimeVaryingTransform - [api_docs/cpp_api/timevaryingtransform/timevaryingtransform.md, C++ Plugin API, TimeVaryingTransform] +- [api_docs/cpp_api/timevaryingtransform/members.md, TimeVaryingTransform, Members] +- [api_docs/cpp_api/timevaryingtransform/constructors.md, TimeVaryingTransform, Constructors] +- [api_docs/cpp_api/timevaryingtransform/statics.md, TimeVaryingTransform, Static Functions] +- [api_docs/cpp_api/timevaryingtransform/functions.md, TimeVaryingTransform, Functions] # CompositeTransform - [api_docs/cpp_api/compositetransform/compositetransform.md, C++ Plugin API, CompositeTransform] +- [api_docs/cpp_api/compositetransform/properties.md, CompositeTransform, Properties] +- [api_docs/cpp_api/compositetransform/members.md, CompositeTransform, Members] +- [api_docs/cpp_api/compositetransform/constructors.md, CompositeTransform, Constructors] +- [api_docs/cpp_api/compositetransform/statics.md, CompositeTransform, Static Functions] +- [api_docs/cpp_api/compositetransform/functions.md, CompositeTransform, Functions] # WrapperTransform - [api_docs/cpp_api/wrappertransform/wrappertransform.md, C++ Plugin API, WrapperTransform] +- [api_docs/cpp_api/wrappertransform/properties.md, WrapperTransform, Properties] +- [api_docs/cpp_api/wrappertransform/members.md, WrapperTransform, Members] +- [api_docs/cpp_api/wrappertransform/constructors.md, WrapperTransform, Constructors] +- [api_docs/cpp_api/wrappertransform/statics.md, WrapperTransform, Static Functions] +- [api_docs/cpp_api/wrappertransform/functions.md, WrapperTransform, Functions] # Distance - [api_docs/cpp_api/distance/distance.md, C++ Plugin API, Distance] diff --git a/docs/scripts/check_links.py b/docs/scripts/check_links.py index 6e4b93e..89c065a 100644 --- a/docs/scripts/check_links.py +++ b/docs/scripts/check_links.py @@ -18,7 +18,7 @@ def walk(path, ext): class Link(): def __init__(self, path, raw_link): - if 'http' in raw_link: + if 'http' in raw_link or 'www' in raw_link: self.http = raw_link self.file = None self.anchor = None @@ -101,7 +101,7 @@ def main(): links = open('../docs/links.md', 'r').read() md_files = walk(docs_dir, ext) - md = markdown.Markdown( ['meta', 'toc', 'tables', 'fenced_code', 'attr_list'] ) + md = markdown.Markdown( ['meta', 'toc', 'tables', 'fenced_code', 'attr_list', 'footnotes'] ) html_files = [md.convert(open(f, 'r').read() + "\n\n" + links) for f in md_files] diff --git a/docs/site/api_docs/c_api/functions/index.html b/docs/site/api_docs/c_api/functions/index.html index e6b645c..945571f 100644 --- a/docs/site/api_docs/c_api/functions/index.html +++ b/docs/site/api_docs/c_api/functions/index.html @@ -9,19 +9,7 @@ - - - - - - - - - - - - - + @@ -34,7 +22,11 @@