ISO/IEC 5152:2024 情報技術 — 統計モデルを使用した生体認証パフォーマンス推定手法 | ページ 2

Foreword

ISO (the International Organization for Standardization) and IEC (the International Electrotechnical Commission) form the specialized system for worldwide standardization. National bodies that are members of ISO or IEC participate in the development of International Standards through technical committees established by the respective organization to deal with particular fields of technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the work.

The procedures used to develop this document and those intended for its further maintenance are described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the different types of document should be noted. This document was drafted in accordance with the editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives or www.iec.ch/members_experts/refdocs ).

ISO and IEC draw attention to the possibility that the implementation of this document may involve the use of (a) patent(s). ISO and IEC take no position concerning the evidence, validity or applicability of any claimed patent rights in respect thereof. As of the date of publication of this document, ISO and IEC had not received notice of (a) patent(s) which may be required to implement this document. However, implementers are cautioned that this may not represent the latest information, which may be obtained from the patent database available at www.iso.org/patents and https://patents.iec.ch . ISO and IEC shall not be held responsible for identifying any or all such patent rights.

Any trade name used in this document is information given for the convenience of users and does not constitute an endorsement.

For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and expressions related to conformity assessment, as well as information about ISO's adherence to the World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see www.iso.org/iso/foreword.html . In the IEC, see www.iec.ch/understanding-standards .

This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology, Subcommittee SC 37, Biometrics.

Introduction

This document provides a methodology for measuring the accuracy of biometric verification systems based on the statistics categorized as the extreme value theory.^[1] The methodology is particularly useful when estimating the false match rate with a relatively small sample set. The methodology is an alternative to empirical accuracy measurement.

In order to measure the false match rate of biometric verification systems, evaluators need to prepare a dataset with a sufficiently large number of non-mated attempts in order to observe a sufficient number of false match cases for a reliable estimation of the false match rate. For highly accurate systems the quantity of attempts required to test the false match rate is likely to be extremely large. As performance of biometric verification systems improves dramatically, acquiring representative data of non-mated attempts in sufficient quantity becomes increasingly difficult in terms of the time, cost and practicality of creating datasets. Policy considerations that apply to biometric data collection and use can pose further constraints.

If no false match case is found within the evaluation samples, metrics based on statistics known as “the rule of three” (as is defined in ISO/IEC 19795-1) are widely used in the biometric industry. However, the rule of 3 is only applicable when no false match case is observed within the tested sample set and do not give any indication of the accuracy and confidence levels expected if more than zero false matches were tested. Only if at least 30 false matches were observed, the “rule of thirty” applies, i.e. the true error rate is with 90 % confidence within ± 30 % of the observed error rate.

In this document, two major statistical methods are introduced to estimate the false match rate with a relatively small number of samples. Both methods are widely used in a variety of industries including civil engineering, meteorology, hydrology and financial engineering. Both methods are proven to be highly reliable techniques to estimate the probability of the occurrence of rare, extreme events such as maximum wind velocity or tsunami heights. These statistical methods are applied to similarly rare events of false match cases in biometrics and used to estimate the probability of occurrence of such cases if a larger non-mated sample set is not available. The estimated false match rate is available in the form of cumulative distribution function (CDF) and its interval of confidence.

This document defines procedures for extrapolating performance metrics in technology evaluations. These procedures can also be applied in scenario evaluations and operational evaluations if comparison scores are obtained. This document defines the methodology to be used by evaluators to reliably estimate the false match rate in case of a limited number of false match cases or even no false match case at all. This document does not address certification or conformance.

1 Scope

This document provides statistical methodologies to estimate false match rates (FMRs) from small biometric sample sets.

This document intends to:

• lay out a methodology for biometric performance estimation based on extrapolation using extreme value statistical models;
• provide statistical methodologies to estimate FMRs of biometric verification systems;
• be applicable to systems that include algorithms that produce likelihood dissimilarity or similarity scores;

  NOTE Throughout the document, if not otherwise specified, scores refer to similarity scores.

• specify the methodology for data recording and result reporting;
• introduce metrics for the estimated biometric performance.

The following are not within the scope of this document.

• Estimation of false positive identification rates for one-to-many implementations.
• Estimation of false accept rates for verification transactions.

2 Normative references

The following documents are referred to in the text in such a way that some or all of their content constitutes requirements of this document. For dated references, only the edition cited applies. For undated references, the latest edition of the referenced document (including any amendments) applies.

3 Terms and definitions

For the purposes of this document, the terms and definitions given in ISO/IEC 2382-37 and the following apply.

ISO and IEC maintain terminology databases for use in standardization at the following addresses:

Bibliography