# Balancing Variables in Clustering

…using the clustMixType R Package

Gero Szepannek, Stralsund University of Applied Sciences

02-07-2026

## Clustering Mixed-Type Data in R

Popular R packages for clustering mixed-type data (Yu 2023).
Algorithm | Authors | Package | Downloads
--- | --- | --- | ---
Gower + PAM | Kaufman and Rousseeuw (1990) | \(\texttt{cluster}\) | 1253012
k-prototypes | Szepannek (2018) | \(\texttt{clustMixType}\) | 67105
MixGHD | Cristina Tortora et al. (2021) | \(\texttt{MixGHD}\) | 6370
kamila | Foss and Markatou (2018) | \(\texttt{kamila}\) | 3243
Fuzzy clustering | C. Tortora and Palumbo (2022) | \(\texttt{FPDclustering}\) | 4312
clustMD | McParland and Gormley (2016) | \(\texttt{clustMD}\) | 4115
Mixed k-means | Ahmad and Dey (2007) | \(\texttt{DisimForMixed}\) | 2260
Spectral clustering | Mbuga and Tortora (2022) | \(\texttt{SpectralClMixed}\) | 2840

## Original k-Prototypes (Huang 1998)

\[ d(x_i, \mu_j) = \underbrace{\sum_{m=1}^q (x_i^m - \mu_j^m)^2}_{numeric \, variables} + \lambda \underbrace{\sum_{m=q+1}^p \delta(x_i^m, \mu_j^m)}_{factor \, variables}\]

1. Initialize prototypes \(\mu_j\) randomly.

2. Assign observations \(x_i\) to their closest prototype w.r.t. \(d()\).

3. Update prototypes with cluster specific means/modes.

4. Go back to step 2.

## The \(\texttt{kproto()}\) Function

    library(clustMixType)
    kpres <- kproto(x = x,
                    k = 4,
                    lambda = NULL,
                    nstart = 1,
                    type = "huang"
                    )

    # NAs in variables:
    x1 x2 x3 x4
     0  0  0  0
    0 observation(s) with NAs.

    Estimated lambda: 6.768405

## \(\texttt{print.kproto}\)

    kpres

    Distance type: huang

    Numeric predictors: 2
    Categorical predictors: 2
    Lambda: 6.768405

    Number of Clusters: 4
    Cluster sizes: 106 94 101 99
    Within cluster error: 332.0333 281.4438 374.4996 321.5224

    Cluster prototypes:
      x1 x2        x3        x4
    1  B  B -1.666238 -1.796813
    2  A  A -1.536931 -1.271467
    3  A  A  1.646250  1.758957
    4  B  B  1.188594  1.393680

## Visualizing Cluster Profiles (I)…

    clprofiles(kpres, x , vars = c("x3"))

## …Visualizing Cluster Profiles (II)

    clprofiles(kpres, x , vars = c("x1"))

## Missing Values (Aschenbruck, Szepannek, and Wilhelm 2023)

- Extension of k-POD (Chi, Chi, and Baraniuk 2016) to mixed-type data:
- Missings are iteratively replaced by prototypes of the assigned cluster.
- Stopping after one iteration corresponds to simply ignoring missings for computing \(d()\).

\(\texttt{na.rm}\) |
--- | ---
\(\texttt{"yes"}\) | cluster only complete cases
\(\texttt{"no"}\) | ignore missings
\(\texttt{"imp.onestep"}\) | impute only once after convergence
\(\texttt{"imp.internal"}\) | impute after each iteration

    kpres <- kproto(x = x, k = 4, lambda = NULL, type = "huang", na.rm = "imp.onestep", keep.data = TRUE)

    # imputed data
    kpres$data

## Variable specific \(\lambda\)

Original: \[ d(x_i, \mu_j) = \sum_{m=1}^q (x_i^m - \mu_j^m)^2 + \lambda \sum_{m=q+1}^p \delta(x_i^m, \mu_j^m)\]

New: \[ d(x_i, \mu_j) = \sum_{m=1}^q \lambda_m (x_i^m - \mu_j^m)^2 + \sum_{m=q+1}^p \lambda_m \delta(x_i^m, \mu_j^m)\]

## Using other Distances

Original (Euklid/Hamming): \[ d(x_i, \mu_j) = \underbrace{\sum_{m=1}^q \lambda_m (x_i^m - \mu_j^m)^2}_{numeric \, variables} + \underbrace{\sum_{m=q+1}^p \lambda_m \delta(x_i^m, \mu_j^m)}_{categorical \, variables}\] Gower (1971), Podani (1999): \[\begin{align} d(x_i, \mu_j) &= \underbrace{\sum_{m=1}^q \lambda_m \frac{|x_i^m - \mu_j^m|}{\underset{i=1,\ldots,n}{range}\{x_i^m\}}}_{numeric \, variables} + \underbrace{\sum_{m=q+1}^p \lambda_m \delta(x_i^m,\mu_j^m)}_{categorical \, variables} \nonumber \\ &\phantom{=} + \underbrace{\sum_{m=p+1}^o \lambda_m \frac{|rank(x_i^m) - \mu_j^m|}{\underset{i=1,\ldots,n}{range}\{rank(x_i^m)\}}}_{ordinal \, variables} \end{align}\]

## …Gower Extension (Szepannek, Aschenbruck, and Wilhelm 2025)

1. Initialize prototypes \(\mu_j\) randomly.
2. Assign observations \(x_i\) to their closest prototype w.r.t. \(d()\).
3. Update prototypes with cluster specific medians/modes.
4. Go back to step 2.

    library(clustMixType)
    kpres <- kproto(x = x,
                    k = 4,
                    type = "gower"
                    )

## Simulation Study

Figure 1: Simulation Design

## …Simulation Study (II)

Results for different numbers of ordinal levels.
Algorithm | 2 | 3 | 4 | 8 | 16
--- | --- | --- | --- | --- | ---
k-proto (Huang) | 0.873 | 0.424 | 0.404 | 0.313 | 0.302
k-proto (Gower) | 0.871 | 0.346 | 0.366 | 0.726 | 0.810
Gower + PAM | 0.871 | 0.346 | 0.319 | 0.740 | 0.811

Figure 2: Runtime in seconds

## Cluster Validation (Aschenbruck and Szepannek 2020)

- Finding the number of clusters \(k\).
- Several internal validation indices transferred to mixed-type data.

Index | Range |
--- | --- | ---
\(C\) index | \([0,1]\) | \(\rightarrow\) min
Dunn index | \([0,\infty]\) | \(\rightarrow\) max
\(\Gamma\) | \([-1,1]\) | \(\rightarrow\) max
Gplus | \([0,N_W N_B / N_D]\) | \(\rightarrow\) min
McClain | \([0,\infty]\) | \(\rightarrow\) min
Ptbserial | \([-\infty,\infty]\) | \(\rightarrow\) max
Silhouette | \([-1,1]\) | \(\rightarrow\) max
\(\tau\) | \([-1,1]\) | \(\rightarrow\) max

…Simulation study suggests SW, \(\Gamma\) or Gplus (SW fastest to compute).

## …\(\texttt{validation_kproto()}\)

    # calculate cindex-value for the given cluster partition
    validation_kproto(method = "cindex",
                      object = kpres)

    [1] 0.04408281

    # ...or find optimal number of clusters
    val <- validation_kproto(method = "silhouette",
                             data = x,
                             k = 2:6,
                             nstart = 5, verbose = FALSE)

    val$indices

            2         3         4         5         6
    0.4321108 0.4404392 0.5485180 0.5131061 0.5064130

    val$k_opt

    [1] 4

## Silhouette Width

    kpres <- kproto(x, 3, verbose = F)
    sils  <- kproto2silhouette(kpres)
    # apply functions from package cluster
    library(cluster)
    plot(sils)

## Choosing \(\lambda\)

- \(\lambda\) has to be specified by the user.
- Huang (1998): ‘The average standard deviation of numeric attributes may be used as a guidance.’
- As a default, data based heuristic \(\texttt{lambdaest()}\) to obtain balance of numeric and categorical variables (should be carefully used):

    res <- kproto(x, 4)

    # NAs in variables:
    x1 x2 x3 x4
     0  0  0  0
    0 observation(s) with NAs.

    Estimated lambda: 6.442523

    # same:
    res <- kproto(x, 4, lambda = lambdaest(x))

## \(\texttt{lambdaest()}\) Heuristic

- \(\hat{\lambda}_m^{num} = s^2_m\) for numeric variables and

- \(\hat{\lambda}_m^{cat} = 1 - \sum_i p_i^2\) for factor variables.

- \(\hat{\lambda} = \overline{\hat{\lambda}_m^{num}} / \overline{\hat{\lambda}_m^{cat}}\).

- …more appropriate: argument \(\texttt{fac.method = 2}\): \[\begin{align} \hat{\lambda}_m^{cat} &=& 0 \cdot p_{mode} + 1 \cdot (1-p_{mode}) \\ &=& 1 - p_{mode} \end{align}\]

    # return variable specific variations:
    lambdaest(x, num.method = 1, fac.method = 2, outtype = "variation")

          x1       x2       x3       x4
    0.470000 0.455000 3.273097 3.131737

- Note: Variation w.r.t. one unique prototype!

## Variable Importance: Contribution to the Distances

- Contribution of the variables to the target criterion \(T\). Decomposition:

\[ T = \sum_i d(x_i, \mu_{j(i)}) = \sum_i \left( \sum_{m=1}^q (x_i^m - \mu_{j(i)}^m)^2 + \lambda \sum_{m=q+1}^p \delta(x_i^m, \mu_{j(i)}^m) \right)\] \[ T =: \sum_{m=1}^p T^m = \sum_{m=1}^q \left( \sum_i (x_i^m - \mu_{j(i)}^m)^2 \right) + \sum_{m=q+1}^p \left( \lambda \sum_i \delta(x_i^m, \mu_{j(i)}^m) \right)\]

- For \(\lambda = 100\) it is:

     Factor 1  Factor 2 Numeric 1 Numeric 2
        0.000     0.000  1288.550  1376.301

## Variable Importance

- For the null model it is:

\[ T_0 =: \sum_{m=1}^p T_0^m = \sum_{m=1}^q \left( \sum_i (x_i^m - \mu^m)^2 \right) + \sum_{m=q+1}^p \left( \lambda \sum_i \delta(x_i^m, \mu^m) \right) \]

… and variable importance: \(VI^m = T^m - T_0^m\).

- Alternatively the adjusted Rand index can be used to quantify how dissimilar two clustering results are if a variable is removed from clustering (Hennig and Murphy 2023).

## Demo (I)

- For \(\lambda = 100\) it is:

    kpres <- kproto(x, 4, lambda = 100 , nstart = 20, verbose = F)
    importance_kproto(kpres, rand = TRUE)

    $vi_decomp
                          x1           x2           x3           x4
    decomp         0.0000000    0.0000000 6.418022e+02 6.153018e+02
    decomp0     9400.0000000 9100.0000000 6.513463e+02 6.232156e+02
    decomp_redu 9400.0000000 9100.0000000 9.544130e+00 7.913863e+00
    vi_rel         0.5076291    0.4914281 5.154125e-04 4.273731e-04

    $vi_rand
    [1] 0.6021046 0.6019655 0.2674440 0.0000000

- For \(\lambda = 0.01\) it is:

    kpres <- kproto(x, 4, lambda = 0.01 , nstart = 20, verbose = F)
    importance_kproto(kpres)

    $vi_decomp
                  x1   x2          x3          x4
    decomp      0.94 0.91 134.0010050  90.5807852
    decomp0     0.94 0.91 651.3463133 623.2156261
    decomp_redu 0.00 0.00 517.3453083 532.6348409
    vi_rel      0.00 0.00   0.4927191   0.5072809

    $vi_rand
    [1] 0.0000000 0.0000000 0.3652064 0.6146235

## Demo (II)

- …without weighting variables (i.e. \(\lambda = 1\) for all variables):

    kpres <- kproto(x, 4, lambda = 1, start = 20, verbose = F)
    importance_kproto(kpres)

    $vi_decomp
                         x1          x2         x3          x4
    decomp      59.00000000 60.00000000 144.933419 151.2672169
    decomp0     94.00000000 91.00000000 651.346313 623.2156261
    decomp_redu 35.00000000 31.00000000 506.412895 471.9484092
    vi_rel       0.03351331  0.02968321   0.484902   0.4519015

    $vi_rand
    [1] 0.5339776 0.5597048 0.6547060 0.6463334

- …for a clustering result where two variables (x1 and x4) are irrelevant:

    kpres <- kproto(x, 4, lambda = c(0.01, 6, 1, 0.01), nstart = 20, verbose = F)
    importance_kproto(kpres)

    $vi_decomp
                          x1          x2          x3         x4
    decomp      0.4300000000   0.0000000 147.7798285 2.42448330
    decomp0     0.9400000000 546.0000000 651.3463133 6.23215626
    decomp_redu 0.5100000000 546.0000000 503.5664849 3.80767296
    vi_rel      0.0004839242   0.5180835   0.4778196 0.00361299

    $vi_rand
    [1] 0.0000000 0.6704998 0.2972868 0.0000000

## Demo (III)

- …for the ‘default’ \(\lambda\) (here: 6.44):

    kpres <- kproto(x, 4, nstart = 20, verbose = F)
    importance_kproto(kpres)

    $vi_decomp
                         x1          x2          x3          x4
    decomp      154.6205430 122.4079299 156.2038547 189.7098929
    decomp0     605.5971266 586.2695588 651.3463133 623.2156261
    decomp_redu 450.9765837 463.8616289 495.1424586 433.5057332
    vi_rel        0.2446324   0.2516219   0.2685902   0.2351554

    $vi_rand
    [1] 0.2327542 0.2853187 0.3813225 0.2911520

Variable importance can be used to assess the appropriateness of \(\lambda\).

## Summary

- Huang’s original algorithm
- Implementation in the \(\texttt{clustMixType}\) package
- \(\texttt{kproto()}\)
- Missing values
- Extension to Gower distance
- Validation indices
- \(\lambda\)
- Variable Importance
- VI can be used to post validate a choice of \(\lambda\)

## Thank You!

Ahmad, A, and L Dey. 2007. “A k-Mean Clustering Algorithm for Mixed Numeric and Categorical Data.” Data Knowledge Engineering 63 (2): 503–27. https://doi.org/10.1016/j.datak.2007.03.016.
Aschenbruck, R, and G Szepannek. 2020. “Cluster Validation for Mixed-Type Data.” Archives of Data Science A 6 (1). https://doi.org/10.5445/KSP/1000098011/02.
Aschenbruck, R, G Szepannek, and A Wilhelm. 2023. “Imputation Strategies for Clustering Mixed-Type Data with Missing Values.” J Classif 40 (2). https://doi.org/110.1007/s00357-022-09422-y.
Chi, J, E Chi, and R Baraniuk. 2016. “K-POD: A Method for k-Means Clustering of Missing Data.” The American Statistician 70 (1): 91–99. https://doi.org/10.1080/00031305.2015.1086685.
Foss, Alexander H., and Marianthi Markatou. 2018. “kamila: Clustering Mixed-Type Data in R and Hadoop.” Journal of Statistical Software 83 (13): 1–45. https://doi.org/10.18637/jss.v083.i13.
Gower, J. 1971. “A General Coefficient of Similarity and Some of Its Properties.” Biometrics 27 (4). https://doi.org/10.2307/2528823.
Hennig, Christian, and Keefe Murphy. 2023. “Quantyfiying Variable Importance in Cluster Analysis.” In CLADAG 2023 Book of Abstracts and Short Papers, edited by Pietro Coretto, Giuseppe Giordano, Michele La Rocca, Maria Lucia Parrella, and Carla Rampichini, 515–18. Italia: Pearson Education Resources.
Huang, Z. 1998. “Extensions to the k-Means Algorithm for Clustering Large Data Sets with Categorical Values.” Data Mining and Knowledge Discovery, no. 2: 283–304. https://doi.org/10.1023/A:1009769707641.
Kaufman, L, and P Rousseeuw. 1990. Finding Groups in Data. Wiley.
Mbuga, Felix, and Cristina Tortora. 2022. “Spectral Clustering of Mixed-Type Data.” Stats 5 (1): 1–11. https://doi.org/10.3390/stats5010001.
McParland, D, and I Gormley. 2016. “Model Based Clustering for Mixed Data: clustMD.” Adv Data Anal Classif 10: 155–69. https://doi.org/10.1007/s11634-016-0238-x.
Podani, J. 1999. “Extending Gower’s General Coefficient of Similarity to Ordinal Characterss.” TAXON 48: 331–40. https://doi.org/10.2307/1224438.
Szepannek, G. 2018. “clustMixType: User-Friendly Clustering of Mixed-Type Data in R.” R Journal 10 (2): 200–208. https://doi.org/110.32614/RJ-2018-048.
Szepannek, G, R Aschenbruck, and A Wilhelm. 2025. “Clustering Large Mixed-Type Data with Ordinal Variables.” Adv Data Anal Classif 19: 749–67. https://doi.org/0.1007/s11634-024-00595-5.
Tortora, C, and F Palumbo. 2022. “Clustering Mixed-Type Data Using a Probabilistic Distance Algorithm.” Applied Soft Computing. https://doi.org/10.1016/j.asoc.2022.109704.
Tortora, Cristina, Ryan Browne, Aisha ElSherbiny, Brian Franczak, and Paul McNicholas. 2021. “Model-Based Clustering, Classification, and Discriminant Analysis Using the Generalized Hyperbolic Distribution: MixGHD R Package.” Journal of Statistical Software 98 (3): 1–24. https://doi.org/10.18637/jss.v098.i03.
Yu, Guangchuang. 2023. dlstats: Download Stats of R Packages. https://CRAN.R-project.org/package=dlstats.