Balancing Variables in Clustering

…using the clustMixType R Package

Gero Szepannek, Stralsund University of Applied Sciences

02-07-2026

Clustering Mixed-Type Data in R

Popular R packages for clustering mixed-type data (Yu 2023).
Algorithm Authors Package Downloads
Gower + PAM Kaufman and Rousseeuw (1990) \(\texttt{cluster}\) 1253012
k-prototypes Szepannek (2018) \(\texttt{clustMixType}\) 67105
MixGHD Cristina Tortora et al. (2021) \(\texttt{MixGHD}\) 6370
kamila Foss and Markatou (2018) \(\texttt{kamila}\) 3243
Fuzzy clustering C. Tortora and Palumbo (2022) \(\texttt{FPDclustering}\) 4312
clustMD McParland and Gormley (2016) \(\texttt{clustMD}\) 4115
Mixed k-means Ahmad and Dey (2007) \(\texttt{DisimForMixed}\) 2260
Spectral clustering Mbuga and Tortora (2022) \(\texttt{SpectralClMixed}\) 2840

Original k-Prototypes (Huang 1998)

\[ d(x_i, \mu_j) = \underbrace{\sum_{m=1}^q (x_i^m - \mu_j^m)^2}_{numeric \, variables} + \lambda \underbrace{\sum_{m=q+1}^p \delta(x_i^m, \mu_j^m)}_{factor \, variables}\]

  1. Initialize prototypes \(\mu_j\) randomly.

  2. Assign observations \(x_i\) to their closest prototype w.r.t. \(d()\).

  3. Update prototypes with cluster specific means/modes.

  4. Go back to step 2.

The \(\texttt{kproto()}\) Function

library(clustMixType)
kpres <- kproto(x = x, 
                k = 4,
                lambda = NULL,
                nstart = 1,
                type = "huang"
                )
# NAs in variables:
x1 x2 x3 x4 
 0  0  0  0 
0 observation(s) with NAs.

Estimated lambda: 6.768405

\(\texttt{print.kproto}\)

kpres 
Distance type: huang 

Numeric predictors: 2 
Categorical predictors: 2 
Lambda: 6.768405 

Number of Clusters: 4 
Cluster sizes: 106 94 101 99 
Within cluster error: 332.0333 281.4438 374.4996 321.5224 

Cluster prototypes:
  x1 x2        x3        x4
1  B  B -1.666238 -1.796813
2  A  A -1.536931 -1.271467
3  A  A  1.646250  1.758957
4  B  B  1.188594  1.393680

Visualizing Cluster Profiles (I)…

clprofiles(kpres, x , vars = c("x3"))

…Visualizing Cluster Profiles (II)

clprofiles(kpres, x , vars = c("x1"))

Missing Values (Aschenbruck, Szepannek, and Wilhelm 2023)

  • Extension of k-POD (Chi, Chi, and Baraniuk 2016) to mixed-type data:
  • Missings are iteratively replaced by prototypes of the assigned cluster.
  • Stopping after one iteration corresponds to simply ignoring missings for computing \(d()\).
\(\texttt{na.rm}\)
\(\texttt{"yes"}\) cluster only complete cases
\(\texttt{"no"}\) ignore missings
\(\texttt{"imp.onestep"}\) impute only once after convergence
\(\texttt{"imp.internal"}\) impute after each iteration 
kpres <- kproto(x = x, k = 4, lambda = NULL, type = "huang", na.rm = "imp.onestep", keep.data = TRUE)

# imputed data
kpres$data

Variable specific \(\lambda\)

Original: \[ d(x_i, \mu_j) = \sum_{m=1}^q (x_i^m - \mu_j^m)^2 + \lambda \sum_{m=q+1}^p \delta(x_i^m, \mu_j^m)\]

New: \[ d(x_i, \mu_j) = \sum_{m=1}^q \lambda_m (x_i^m - \mu_j^m)^2 + \sum_{m=q+1}^p \lambda_m \delta(x_i^m, \mu_j^m)\]

Using other Distances

Original (Euklid/Hamming): \[ d(x_i, \mu_j) = \underbrace{\sum_{m=1}^q \lambda_m (x_i^m - \mu_j^m)^2}_{numeric \, variables} + \underbrace{\sum_{m=q+1}^p \lambda_m \delta(x_i^m, \mu_j^m)}_{categorical \, variables}\] Gower (1971), Podani (1999): \[\begin{align} d(x_i, \mu_j) &= \underbrace{\sum_{m=1}^q \lambda_m \frac{|x_i^m - \mu_j^m|}{\underset{i=1,\ldots,n}{range}\{x_i^m\}}}_{numeric \, variables} + \underbrace{\sum_{m=q+1}^p \lambda_m \delta(x_i^m,\mu_j^m)}_{categorical \, variables} \nonumber \\ &\phantom{=} + \underbrace{\sum_{m=p+1}^o \lambda_m \frac{|rank(x_i^m) - \mu_j^m|}{\underset{i=1,\ldots,n}{range}\{rank(x_i^m)\}}}_{ordinal \, variables} \end{align}\]

…Gower Extension (Szepannek, Aschenbruck, and Wilhelm 2025)

  1. Initialize prototypes \(\mu_j\) randomly.
  2. Assign observations \(x_i\) to their closest prototype w.r.t. \(d()\).
  3. Update prototypes with cluster specific medians/modes.
  4. Go back to step 2.
library(clustMixType)
kpres <- kproto(x = x, 
                k = 4,
                type = "gower"
                )

Simulation Study

Figure 1: Simulation Design

…Simulation Study (II)

Results for different numbers of ordinal levels.
Algorithm 2 3 4 8 16
k-proto (Huang) 0.873 0.424 0.404 0.313 0.302
k-proto (Gower) 0.871 0.346 0.366 0.726 0.810
Gower + PAM 0.871 0.346 0.319 0.740 0.811

Figure 2: Runtime in seconds

Cluster Validation (Aschenbruck and Szepannek 2020)

  • Finding the number of clusters \(k\).
  • Several internal validation indices transferred to mixed-type data.
Index Range
\(C\) index \([0,1]\) \(\rightarrow\) min
Dunn index \([0,\infty]\) \(\rightarrow\) max
\(\Gamma\) \([-1,1]\) \(\rightarrow\) max
Gplus \([0,N_W N_B / N_D]\) \(\rightarrow\) min
McClain \([0,\infty]\) \(\rightarrow\) min
Ptbserial \([-\infty,\infty]\) \(\rightarrow\) max
Silhouette \([-1,1]\) \(\rightarrow\) max
\(\tau\) \([-1,1]\) \(\rightarrow\) max

…Simulation study suggests SW, \(\Gamma\) or Gplus (SW fastest to compute).

…\(\texttt{validation_kproto()}\)

# calculate cindex-value for the given cluster partition
validation_kproto(method = "cindex", 
                  object = kpres)
[1] 0.04408281
# ...or find optimal number of clusters
val <- validation_kproto(method = "silhouette", 
                         data = x, 
                         k = 2:6, 
                         nstart = 5, verbose = FALSE)
val$indices
        2         3         4         5         6 
0.4321108 0.4404392 0.5485180 0.5131061 0.5064130 
val$k_opt
[1] 4

Silhouette Width

kpres <- kproto(x, 3, verbose = F)
sils  <- kproto2silhouette(kpres)
# apply functions from package cluster
library(cluster)
plot(sils)

Choosing \(\lambda\)

  • \(\lambda\) has to be specified by the user.
  • Huang (1998): ‘The average standard deviation of numeric attributes may be used as a guidance.’
  • As a default, data based heuristic \(\texttt{lambdaest()}\) to obtain balance of numeric and categorical variables (should be carefully used):
res <- kproto(x, 4)
# NAs in variables:
x1 x2 x3 x4 
 0  0  0  0 
0 observation(s) with NAs.

Estimated lambda: 6.442523 
# same:
res <- kproto(x, 4, lambda = lambdaest(x))

\(\texttt{lambdaest()}\) Heuristic

  • \(\hat{\lambda}_m^{num} = s^2_m\) for numeric variables and

  • \(\hat{\lambda}_m^{cat} = 1 - \sum_i p_i^2\) for factor variables.

  • \(\hat{\lambda} = \overline{\hat{\lambda}_m^{num}} / \overline{\hat{\lambda}_m^{cat}}\).

  • …more appropriate: argument \(\texttt{fac.method = 2}\): \[\begin{align} \hat{\lambda}_m^{cat} &=& 0 \cdot p_{mode} + 1 \cdot (1-p_{mode}) \\ &=& 1 - p_{mode} \end{align}\]

# return variable specific variations:
lambdaest(x, num.method = 1, fac.method = 2, outtype = "variation")
      x1       x2       x3       x4 
0.470000 0.455000 3.273097 3.131737
  • Note: Variation w.r.t. one unique prototype!

Variable Importance: Contribution to the Distances

  • Contribution of the variables to the target criterion \(T\). Decomposition:

\[ T = \sum_i d(x_i, \mu_{j(i)}) = \sum_i \left( \sum_{m=1}^q (x_i^m - \mu_{j(i)}^m)^2 + \lambda \sum_{m=q+1}^p \delta(x_i^m, \mu_{j(i)}^m) \right)\] \[ T =: \sum_{m=1}^p T^m = \sum_{m=1}^q \left( \sum_i (x_i^m - \mu_{j(i)}^m)^2 \right) + \sum_{m=q+1}^p \left( \lambda \sum_i \delta(x_i^m, \mu_{j(i)}^m) \right)\]

  • For \(\lambda = 100\) it is:
 Factor 1  Factor 2 Numeric 1 Numeric 2 
    0.000     0.000  1288.550  1376.301

Variable Importance

  • For the null model it is:

\[ T_0 =: \sum_{m=1}^p T_0^m = \sum_{m=1}^q \left( \sum_i (x_i^m - \mu^m)^2 \right) + \sum_{m=q+1}^p \left( \lambda \sum_i \delta(x_i^m, \mu^m) \right) \]

… and variable importance: \(VI^m = T^m - T_0^m\).

  • Alternatively the adjusted Rand index can be used to quantify how dissimilar two clustering results are if a variable is removed from clustering (Hennig and Murphy 2023).

Demo (I)

  • For \(\lambda = 100\) it is:
kpres <- kproto(x, 4, lambda = 100 , nstart = 20, verbose = F)
importance_kproto(kpres, rand = TRUE)
$vi_decomp
                      x1           x2           x3           x4
decomp         0.0000000    0.0000000 6.418022e+02 6.153018e+02
decomp0     9400.0000000 9100.0000000 6.513463e+02 6.232156e+02
decomp_redu 9400.0000000 9100.0000000 9.544130e+00 7.913863e+00
vi_rel         0.5076291    0.4914281 5.154125e-04 4.273731e-04

$vi_rand
[1] 0.6021046 0.6019655 0.2674440 0.0000000
  • For \(\lambda = 0.01\) it is:
kpres <- kproto(x, 4, lambda = 0.01 , nstart = 20, verbose = F)
importance_kproto(kpres)
$vi_decomp
              x1   x2          x3          x4
decomp      0.94 0.91 134.0010050  90.5807852
decomp0     0.94 0.91 651.3463133 623.2156261
decomp_redu 0.00 0.00 517.3453083 532.6348409
vi_rel      0.00 0.00   0.4927191   0.5072809

$vi_rand
[1] 0.0000000 0.0000000 0.3652064 0.6146235

Demo (II)

  • …without weighting variables (i.e. \(\lambda = 1\) for all variables):
kpres <- kproto(x, 4, lambda = 1, start = 20, verbose = F)
importance_kproto(kpres)
$vi_decomp
                     x1          x2         x3          x4
decomp      59.00000000 60.00000000 144.933419 151.2672169
decomp0     94.00000000 91.00000000 651.346313 623.2156261
decomp_redu 35.00000000 31.00000000 506.412895 471.9484092
vi_rel       0.03351331  0.02968321   0.484902   0.4519015

$vi_rand
[1] 0.5339776 0.5597048 0.6547060 0.6463334
  • …for a clustering result where two variables (x1 and x4) are irrelevant:
kpres <- kproto(x, 4, lambda = c(0.01, 6, 1, 0.01), nstart = 20, verbose = F)
importance_kproto(kpres)
$vi_decomp
                      x1          x2          x3         x4
decomp      0.4300000000   0.0000000 147.7798285 2.42448330
decomp0     0.9400000000 546.0000000 651.3463133 6.23215626
decomp_redu 0.5100000000 546.0000000 503.5664849 3.80767296
vi_rel      0.0004839242   0.5180835   0.4778196 0.00361299

$vi_rand
[1] 0.0000000 0.6704998 0.2972868 0.0000000

Demo (III)

  • …for the ‘default’ \(\lambda\) (here: 6.44):
kpres <- kproto(x, 4, nstart = 20, verbose = F)
importance_kproto(kpres)
$vi_decomp
                     x1          x2          x3          x4
decomp      154.6205430 122.4079299 156.2038547 189.7098929
decomp0     605.5971266 586.2695588 651.3463133 623.2156261
decomp_redu 450.9765837 463.8616289 495.1424586 433.5057332
vi_rel        0.2446324   0.2516219   0.2685902   0.2351554

$vi_rand
[1] 0.2327542 0.2853187 0.3813225 0.2911520

Variable importance can be used to assess the appropriateness of \(\lambda\).

Summary

  • Huang’s original algorithm
  • Implementation in the \(\texttt{clustMixType}\) package
  • \(\texttt{kproto()}\)
  • Missing values
  • Extension to Gower distance
  • Validation indices
  • \(\lambda\)
  • Variable Importance
  • VI can be used to post validate a choice of \(\lambda\)

Thank You!

Ahmad, A, and L Dey. 2007. “A k-Mean Clustering Algorithm for Mixed Numeric and Categorical Data.” Data Knowledge Engineering 63 (2): 503–27. https://doi.org/10.1016/j.datak.2007.03.016.
Aschenbruck, R, and G Szepannek. 2020. “Cluster Validation for Mixed-Type Data.” Archives of Data Science A 6 (1). https://doi.org/10.5445/KSP/1000098011/02.
Aschenbruck, R, G Szepannek, and A Wilhelm. 2023. “Imputation Strategies for Clustering Mixed-Type Data with Missing Values.” J Classif 40 (2). https://doi.org/110.1007/s00357-022-09422-y.
Chi, J, E Chi, and R Baraniuk. 2016. “K-POD: A Method for k-Means Clustering of Missing Data.” The American Statistician 70 (1): 91–99. https://doi.org/10.1080/00031305.2015.1086685.
Foss, Alexander H., and Marianthi Markatou. 2018. “kamila: Clustering Mixed-Type Data in R and Hadoop.” Journal of Statistical Software 83 (13): 1–45. https://doi.org/10.18637/jss.v083.i13.
Gower, J. 1971. “A General Coefficient of Similarity and Some of Its Properties.” Biometrics 27 (4). https://doi.org/10.2307/2528823.
Hennig, Christian, and Keefe Murphy. 2023. “Quantyfiying Variable Importance in Cluster Analysis.” In CLADAG 2023 Book of Abstracts and Short Papers, edited by Pietro Coretto, Giuseppe Giordano, Michele La Rocca, Maria Lucia Parrella, and Carla Rampichini, 515–18. Italia: Pearson Education Resources.
Huang, Z. 1998. “Extensions to the k-Means Algorithm for Clustering Large Data Sets with Categorical Values.” Data Mining and Knowledge Discovery, no. 2: 283–304. https://doi.org/10.1023/A:1009769707641.
Kaufman, L, and P Rousseeuw. 1990. Finding Groups in Data. Wiley.
Mbuga, Felix, and Cristina Tortora. 2022. “Spectral Clustering of Mixed-Type Data.” Stats 5 (1): 1–11. https://doi.org/10.3390/stats5010001.
McParland, D, and I Gormley. 2016. “Model Based Clustering for Mixed Data: clustMD.” Adv Data Anal Classif 10: 155–69. https://doi.org/10.1007/s11634-016-0238-x.
Podani, J. 1999. “Extending Gower’s General Coefficient of Similarity to Ordinal Characterss.” TAXON 48: 331–40. https://doi.org/10.2307/1224438.
Szepannek, G. 2018. “clustMixType: User-Friendly Clustering of Mixed-Type Data in R.” R Journal 10 (2): 200–208. https://doi.org/110.32614/RJ-2018-048.
Szepannek, G, R Aschenbruck, and A Wilhelm. 2025. “Clustering Large Mixed-Type Data with Ordinal Variables.” Adv Data Anal Classif 19: 749–67. https://doi.org/0.1007/s11634-024-00595-5.
Tortora, C, and F Palumbo. 2022. “Clustering Mixed-Type Data Using a Probabilistic Distance Algorithm.” Applied Soft Computing. https://doi.org/10.1016/j.asoc.2022.109704.
Tortora, Cristina, Ryan Browne, Aisha ElSherbiny, Brian Franczak, and Paul McNicholas. 2021. “Model-Based Clustering, Classification, and Discriminant Analysis Using the Generalized Hyperbolic Distribution: MixGHD R Package.” Journal of Statistical Software 98 (3): 1–24. https://doi.org/10.18637/jss.v098.i03.
Yu, Guangchuang. 2023. dlstats: Download Stats of R Packages. https://CRAN.R-project.org/package=dlstats.