Comparative File Review vs Matched-Pair Testing: What Examiners Actually Do
Published 18 August 2026 · Sources: the FFIEC Interagency Fair Lending Examination Procedures (August 2009) and the OCC Comptroller’s Handbook, Fair Lending, Version 1.0 (January 2023). Both quoted below and linked at the foot of the page.
The short answer. They are not two names for the same method, and one of them does not mean what the software industry uses it to mean. Comparative file review is the procedure named in the examination procedures for comparing prohibited basis and control group applicants using their actual loan files. Its counterpart in those procedures is statistical analysis — not “matched pair testing.”
The phrase “matched pair” does not appear in the Interagency Fair Lending Examination Procedures at all. In the current OCC Comptroller’s Handbook it appears once, and it means pre-application testing of potential applicants — sending testers, not reading files.
What the examination procedures actually say
Part IV of the Interagency Fair Lending Examination Procedures covers analysing potential disparities. On pricing and other terms and conditions it is explicit about the available approaches:
Two named approaches, which may also be combined. The procedures then set out the steps for the first of them, and the ordering is the part that gets lost in vendor marketing: Step 1: Determine Sample Selection, followed by Step 2: Conduct Comparative File Review. Selecting who to compare is not a rival method to comparative file review. It is the first step of it.
On how that sample is drawn, the procedures say the size “should be based on the number of prohibited basis group and control group originations for each focal point selected during the 12 months preceding the examination.” The OCC Comptroller’s Handbook puts the purpose plainly:
Where “matched pair” actually appears
We searched both documents. “Matched pair” as two words appears in neither. The hyphenated form appears exactly once across the two, in the OCC handbook, and this is the sentence:
Read the two qualifiers. Pre-application, and potential applicants. This is testing that happens before anyone has applied, to observe how prospective borrowers are treated. It is the practice usually described as mystery shopping, and it is conducted by enforcement agencies and fair housing organisations rather than by a lender analysing its own portfolio. It is not a file review, because at that point there are no files.
Software vendors, ours included until we checked, commonly use “match pair testing” to mean the file-based pairing analysis. That is close to the opposite of the regulator’s usage of the same phrase.
The three things being conflated
| Term | What it is in regulatory usage | Where it appears |
|---|---|---|
| Comparative file review | Comparing prohibited basis and control group applicants using their actual loan files, to test whether a legitimate factor explains a difference in outcome. Sample selection is Step 1; the review is Step 2. | Named procedure in the Interagency Procedures (Part IV); throughout the OCC handbook |
| Statistical analysis / statistical modeling | Modelling outcomes across the whole population to isolate the effect of a prohibited basis characteristic while controlling for legitimate factors. The companion approach to a file review, not a competing one. | Named alongside comparative file review in the Interagency Procedures; called “statistical modeling” throughout the OCC handbook |
| Matched-pair testing | Pre-application testing using testers posing as prospective applicants, to document treatment before an application exists. | Once, in the OCC handbook, in the pre-application context |
Note one terminology detail that trips up software comparisons: the current OCC handbook does not use the word “regression” anywhere. It says statistical modeling. Regression is the usual implementation of that, which is why vendors and consultants say “regression analysis” while the examination guidance says something broader. They are talking about the same territory.
Which approach an examiner uses is mostly about volume
The choice is not a philosophical preference. The OCC handbook ties it to how much lending there is to analyse:
And in the other direction, the handbook contemplates the case where “the examination does not involve enough lending volume to allow statistical modeling, and the examination involves a manual comparative file review (rather than statistical modeling).” For a community institution with modest volume in a given product, the manual review is not a lesser substitute. It is the appropriate method.
The two approaches also check each other
This is the part most useful to anyone running a self-assessment, and it is the reason treating them as rival methods is a mistake. A file review conducted after a model has run serves a specific purpose:
The handbook attaches two conditions worth knowing before you use a file review to argue against a model’s finding. First, a factor you want added to the model “must be legitimately derived from bank policies and occur sufficiently systematically to warrant changes to the model.” A one-off explanation for a single file does not qualify. Second, and more bluntly: “It is important not to dismiss statistical results based simply on findings from a manual file review.”
If your institution’s response to a statistical disparity is to pull a handful of files and explain them individually, the guidance anticipates that and does not accept it on its own.
What this means if you are self-testing
The practical question is not “which method should we use.” It is:
- Do we have the volume for a model in this product? If yes, the model is the primary analysis and the file review validates it. If no, the file review is the analysis.
- Are we comparing genuinely similarly situated applicants? Pairs are only informative if matched on the characteristics the institution actually used to decide — credit score, debt-to-income, loan-to-value, loan amount, collateral, product.
- Can we document why each difference is not a violation? That documentation is the output an examiner will ask to see, at pair, application and field level.
- If we plan to explain away a modelled disparity, is the explanation systematic and grounded in our own written policy? If it is neither, the handbook says it should not move the finding.
And if you are comparing software
Ask any vendor advertising “match pair testing” which of the three things above it means. Almost always it is automated pair selection plus side-by-side review of actual applications — which is the sample-selection-and-review procedure the examination procedures call a comparative file review. Then ask whether the same product also performs statistical modelling, because the procedures treat these as companions and which one you need depends on your volume in each product.
For what it is worth on our side: Comply Fair Lending builds the matched-pair pool on partitioning and matching criteria you set, then puts the paired files side by side and documents the review at file-review, application-pair, application and individual-field level. It also runs logistic regression for underwriting and linear regression for pricing, so the volume question above does not force a choice between products. Definitions for the terms used on this page, and about 190 others, are in our compliance glossary.
A note on the documents themselves
Two things worth knowing if you go to the sources, because both cost us time:
- The Interagency Fair Lending Examination Procedures currently in force are the August 2009 edition, which replaced the March 1994 procedures. There is no newer edition despite later reissues of the same text.
- The OCC Comptroller’s Handbook Fair Lending booklet exists online in two versions, and the earlier one is stamped RESCINDED on every page. The current booklet is Version 1.0, January 2023. Its cover page also records that references to disparate impact were removed from the booklet as of 14 July 2025, marked by strikethrough, per OCC Bulletin 2025-16. That is a change to the OCC’s examination booklet; ECOA, Regulation B and the Fair Housing Act are the underlying law and are unaffected by an edit to a supervisory handbook. Read the current booklet, not the first search result.
Sources
- Interagency Fair Lending Examination Procedures, Federal Financial Institutions Examination Council, August 2009. Quoted above from Part IV.D and the comparative file review steps. Copy hosted by NCUA (PDF).
- Comptroller’s Handbook, Consumer Compliance: Fair Lending, Version 1.0, Office of the Comptroller of the Currency, January 2023, as amended 14 July 2025. Current booklet at the OCC.
Frequently Asked Questions
Is comparative file review the same thing as matched-pair testing?
No, and in regulatory usage they are not even the same kind of activity. Comparative file review is the named procedure in the FFIEC Interagency Fair Lending Examination Procedures for comparing the treatment of prohibited basis group and control group applicants using their actual loan files. The phrase "matched pair" does not appear in those procedures at all. In the current OCC Comptroller's Handbook the single use of "matched-pair" refers to "pre-application, matched-pair testing to document the bank's treatment of potential applicants" — that is, sending testers to observe treatment before any application exists, which is a different activity from reviewing files. Software vendors commonly use "match pair testing" to mean the file-based pairing analysis, which is close to the opposite of the OCC's usage.
What is the actual alternative to a comparative file review?
Statistical modeling. The Interagency Fair Lending Examination Procedures state that the analysis of decisions on pricing and other terms and conditions "may involve a comparative file review, statistical analysis, a combination of the two, or other specialized technique used by an agency." Those are the two named approaches. Which one an examiner uses depends largely on lending volume: the OCC Comptroller's Handbook notes that in high-volume examinations files are compared primarily using statistical modeling, and that a manual comparative file review is used where volume is insufficient for a model.
How are the pairs in a comparative file review selected?
Sample selection is the first step of the procedure rather than a separate method. The Interagency Fair Lending Examination Procedures set out Step 1, Determine Sample Selection, before Step 2, Conduct Comparative File Review, and state that the sample size should generally be based on the number of prohibited basis group and control group originations for each focal point selected. The OCC Comptroller's Handbook adds that the purpose of selecting samples of prohibited basis group and control group applicants is to identify potential discriminatory outcomes, and that the sample "serves as the basis for a comparative file review." Applicants are compared where they are similarly situated on the characteristics the institution actually used to decide.
Does a comparative file review replace statistical analysis, or check it?
It can do either. Where lending volume is too low to support a model, a manual comparative file review is the analysis. Where a model has been run, a file review can test whether the model omitted a legitimate factor: the OCC Comptroller's Handbook directs that if a comparative file review identifies data errors or factors not accounted for in the statistical model, examiners should work to correct or update the model. The same passage cautions that it is important not to dismiss statistical results based simply on findings from a manual file review, and that any additional factor added to a model must be legitimately derived from the institution's own policies and occur systematically.
Why does the terminology matter when comparing compliance software?
Because two vendors can describe the same capability in different words, or different capabilities in the same words, and a feature-list comparison will not reveal which. If a vendor advertises "match pair testing," ask whether it means automated pair selection and side-by-side file review of actual applications, or pre-application testing with testers. Almost always it is the former, which is the sample-selection and review procedure the examination procedures call a comparative file review. Ask the same vendor whether the product also performs statistical modeling, because the procedures treat that as the companion approach and the choice between them depends on your lending volume.
Schedule a Fair Lending Demo How to evaluate compliance software
This page may be republished with attribution to RATA Associates and a link to rataassociates.com. If you believe a quotation or reading here is wrong, tell us — we would rather fix it than be cited incorrectly.
