automatedRecLin

Record Linkage Based on an Entropy-Maximizing Classifier

Adam Struzik
adastr5@st.amu.edu.pl

Faculty of Mathematics and Computer Science, Adam Mickiewicz University in Poznań, Poland

Centre for Urban Statistics, Statistical Office in Poznań, Poland

Maciej Beręsewicz
maciej.beresewicz@ue.poznan.pl

Department of Statistics, Poznań University of Economics and Business, Poland

Centre for the Methodology of Population Studies, Statistical Office in Poznań, Poland

useR! Conference, Warsaw, Poland
July 8, 2026

Problem

Record linkage — identifying records that refer to the same entity across two datasets.

Dataset A

name surname city
Emily Johnson Chicago
Michael Anderson Seattle
Sarah Thompson Boston

Dataset B

name surname city
Sara Tompson Boston
Emily Johnson Chicago
Michael Andersson Seatle
  • There are no shared unique identifiers.
  • Names, surnames, and locations may contain typos and discrepancies.

Method

  • Starting point: maximum entropy classification (MEC) for record linkage (Lee et al., 2022).
  • Original idea: compare records using agreement/disagreement (binary) indicators.
  • Our extension: use continuous comparison functions instead.
values binary comparison continuous comparison
“Johnson” vs “Johnson” agreement 0.000
“Anderson” vs “Andersson” disagreement 0.037
“Seattle” vs “Seatle” disagreement 0.048

Here, the continuous comparison function is 1 \(-\) Jaro-Winkler similarity (Winkler, 1990): 0 means identical values; larger values indicate stronger disagreement.

The automatedRecLin Package

Interface

mec(
  A = df1,
  B = df2,
  variables = c(
    "pername1", "pername2", "sex",
    "dob_day", "dob_mon", "dob_year"
  )
)

Unsupervised MEC with binary comparisons on all key variables.

Interface

mec(
  A = df1,
  B = df2,
  variables = c(
    "pername1", "pername2", "sex",
    "dob_day", "dob_mon", "dob_year"
  ),
  comparators = list(
    "pername1" = jarowinkler_complement(),
    "pername2" = jarowinkler_complement()
  ),
  methods = list(
    "pername1" = "continuous_parametric",
    "pername2" = "continuous_parametric"
  )
)

Unsupervised MEC with continuous comparisons (1 \(-\) Jaro-Winkler similarity) on name and surname. We use the parametric approach.

Example Results

Method No. of links False Linkage Rate Missing Match Rate
Binary 379 0.0053 0.0575
Continuous parametric 397 0.0025 0.0100

The true number of matches is 400. Note that False Linkage Rate is equal to 1 \(-\) Precision, and Missing Match Rate is equal to 1 \(-\) Recall.

Observations:

  • Continuous comparisons improve error rates.
  • They effectively capture small discrepancies between records.

automatedRecLin

References

Beręsewicz, M., & Struzik, A. (2026). blocking: Various blocking methods for entity resolution. https://cran.r-project.org/package=blocking
Lee, D., Zhang, L.-C., & Kim, J. K. (2022). Maximum entropy classification for record linkage. Survey Methodology, 48(1).
Sugiyama, M., Suzuki, T., Nakajima, S., Kashima, H., von Bünau, P., & Kawanabe, M. (2008). Direct importance estimation for covariate shift adaptation. Annals of the Institute of Statistical Mathematics, 60(4), 699–746. https://doi.org/10.1007/s10463-008-0197-x
Vo, T. H., Chauvet, G., Happe, A., Oger, E., Paquelet, S., & Garès, V. (2023). Extending the Fellegi-Sunter record linkage model for mixed-type data with application to the French national health data system. Computational Statistics & Data Analysis, 179(C).
Winkler, W. E. (1990). String comparator metrics and enhanced decision rules in the Fellegi-Sunter model of record linkage. Proceedings of the Section on Survey Research Methods, 354–359.