Hybrid clustering algorithm

The K-Means algorithm, well-known for its simplicity, has limitations that impede its ability to produce flawless clustering results [4]. In response, literature [5] introduced the Fuzzy C-Means (FCM) algorithm, allowing points to belong to multiple clusters, a departure from K-Means’ strict partitioning. However, sensitivity to initial values persisted, leading to subsequent enhancements. The peak method proposed in literature [6] was employed to estimate cluster centers for initial partitioning, while literature [7] advocated adjusting proximity distances to fortify FCM against outliers. Further advancements aimed to enhance clustering algorithms’ efficacy. Literature [8] presented a probabilistic perspective on clustering, extracting data objects from specific probability distributions and assuming the overall data distribution as a blend of several distributions. Similarly, literature [9] showcased Density-Based Spatial Clustering of Applications with Noise (DBSCAN), a density-based algorithm capable of identifying arbitrarily shaped clusters without constraints, demonstrating robust performance on sizable datasets. Building upon DBSCAN, literature [10] introduced density peak clustering, derived from the DBSCAN algorithm, excelling in discerning non-spherical clusters and accurately delineating intricate data distributions.

These algorithms face challenges related to initial clustering center selection and the tendency to converge toward local optima, potentially hindering the attainment of globally optimal clustering outcomes. Recent research has focused on leveraging PSO-based data clustering to devise hybrid clustering algorithms, combining PSO with conventional methods to elevate clustering efficacy [11]. These PSO-based algorithms dynamically fine-tune parameters like inertia weights, using population fitness variance to ascertain the convergence timing of PSO and K-Means algorithms [12]. Additionally, they incorporate real-time monitoring of optimal values within individual particles and particle clusters, promptly executing mutation operations on prematurely converging particles to explore globally optimal initial clustering centers for K-Means algorithms.

Hybrid clustering algorithms provide a more comprehensive understanding of data, combining the global search and optimization capabilities of PSO with the expertise of traditional clustering algorithms tailored to specific data structures and features. Literature [13] introduces the PSO-K-means algorithm, which initializes a particle in the particle swarm using clustering results derived from K-Means. Alternatively, literature [14] presents a hybrid algorithm that combines PSO and K-Means, utilizing PSO for global search at the optimization’s outset and leveraging K-Means for accelerated convergence near the optimal solution. In addressing nonlinear division clustering challenges, literature [15] pioneers a hybrid clustering algorithm that integrates fuzzy adaptive PSO and K-Means methodologies.

PSO optimization model

To enhance the effectiveness of hybrid clustering algorithms, researchers have worked to refine both clustering and PSO methodologies. Literature [16] introduces the Evolutionary Particle Swarm Optimization (EPSO) algorithm, rooted in particle swarm evolution. This approach initiates particles uniformly across the input data space, with subsequent generations dynamically adjusting to pursue optimal positions. However, relying on inter-particle information exchange limits its ability to break free from local optima, curtailing performance on intricate problems and constraining global search capability.

In pursuit of reduced computational complexity, literature [17] proposes an enhanced mPSC (Particle Swarm Clustering) algorithm, aiming to streamline PSC for improved efficiency with large-scale datasets. Despite simplifying computational processes and reducing manual input parameters, mPSC faces challenges related to dataset characteristics and initial parameter choices. These challenges result in susceptibility to local optima, diminished convergence efficiency, and compromised clustering accuracy, highlighting the need for more comprehensive optimization. Literature [18] introduces a PSC-RCE algorithm, incorporating Rapid Centroid Estimation (RCE) [19] to simplify PSC’s update rules. This approach, termed PSC-RCE, aims to streamline the clustering process. While PSO-based clustering mitigates the risk of local optima, challenges persist. The update mechanism in PSO algorithms, reliant on historical optimal solutions, leads to diminished convergence efficiency during initial search stages and a subsequent decline in population diversity. This inefficiency poses a substantial challenge, particularly in scenarios involving large-scale datasets and stringent real-time requirements, due to prolonged convergence times and heightened computational resource consumption.

The PSO algorithm encounters limitations when handling high-dimensional, intricate shapes, and noisy data, often yielding clustering outcomes lacking accuracy and stability. These limitations directly impact the algorithm’s reliability and practical utility, constraining its applicability in real-world scenarios. Consequently, this paper aims to enhance the PSO methodology by integrating it with density algorithms, forging a higher-order hybrid clustering algorithm. The objective is to craft a health data management model tailored for the elderly within the framework of physical-medical integration.

Model design

The model diagram in Fig 1 illustrates the framework for elderly health information management based on the higher-order hybrid clustering algorithm developed in this paper. Our approach involves refining the PSO algorithm and integrating density values to construct an efficient higher-order hybrid clustering algorithm. This algorithm processes sensor-collected data, representing it in vector form, and concludes with the effective clustering management of elderly health data.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Model frame drawing.

https://doi.org/10.1371/journal.pone.0302741.g001

Optimization of PSO

Within the n-dimensional search space of the PSO algorithm, each particle is characterized by two vectors: the position vector (P) and the velocity vector (V). Initially, every particle possesses randomized initial velocity and position values. The particle’s position denotes a potential solution within the problem space. Throughout the search phase, the PSO algorithm iteratively updates particle information. Let the individual history optimal solution pbest_i for particle i and the global history optimal solution gbest_i for the whole population be represented as follows: (1) (2)

When the particle moves in the search space, the velocity v_id and position x_id of each particle i are updated according to the Formulas (3) and (4).

(3)

(4)

Where w denotes the inertia weights, c1 and c2 denote the acceleration constants, v_id(t) denotes the velocity of the ith particle in the dth dimension at the t-th iteration, and rand1 and rand2 are random numbers in the range [0,1]. It is established that the efficacy of PSO is heightened as the inertia weights progressively decrease in tandem with the number of iterations. The subsequent representation illustrates the linear descent of inertia weights: (5)

To further enhance the convergence efficacy of the PSO algorithm and mitigate local optima, our investigation reveals EDA [20] as a robust evolutionary algorithm. EDA estimates population evolution through probabilistic model sampling and learning, exhibiting exceptional performance in addressing intricate optimization problems. Within this framework, GEDM [21] assumes a pivotal role as a core constituent of EDA. Our decision was to integrate GEDM into the PSO algorithm, depicted in Fig 2. This integration involves constructing a GEDM model utilizing a weighted covariance matrix. This approach enables a more precise depiction of the population’s distribution through weighted covariance matrix calculations, thereby furnishing the algorithm with an enhanced estimation of the evolutionary trajectory. The computational methodology for the GEDM model based on the weighted covariance matrix is delineated below: (6) (7) (8)

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Weighted covariance matrix based GEDM model.

https://doi.org/10.1371/journal.pone.0302741.g002

We set m to denote the number of solutions selected as dominant, and {X₁,X₂,X₃,⋯,X_m} denotes the set of dominant solutions, where X₁ denotes the optimal solution. w_i denotes the weight of the ith dominant solution, the higher the ranking of the solution the greater the corresponding weight.

Cov(t) represents the weighted covariance matrix of the dominant solutions in the proposed model. When using GEDM to update the population information, we use the Formula (9) to update the position of particles in the population.

(9)

To effectively merge GEDM and PSO, a selection process determines whether GEDM or PSO updates the population information at various stages of the optimization process, guided by distinct probabilities. During the initial phase, GEDM receives a higher probability for updating population information. This prioritization leverages the strengths of advantageous particles to ascertain a superior evolutionary trajectory, thereby accelerating the convergence rate. Conversely, in the latter stages of optimization, PSO garners a higher probability for updating population information. This strategic shift capitalizes on PSO’s robust local search capabilities to enhance convergence accuracy. The probability model governing the selection between GEDM and PSO to update population information is depicted below: (10)

Where y_max and y_min denote the maximum and minimum probability of selecting GEDM to update population information, respectively.

Higher order hybrid clustering algorithm

Drawing from the density peak clustering method established in literature [9, 10], this approach adeptly manages clusters of varying shapes, identifies noise points within the data for exclusion from clustering, and offers an accurate depiction of data distribution. To enhance clustering precision, we introduce the density peak algorithm into the hybrid clustering framework based on the PSO algorithm in this section, thereby constructing a higher order hybrid clustering algorithm for performance enhancement. The PSO algorithm utilized within this higher-order framework is the refined PSO algorithm outlined in Section 3.1.

The final constructed higher-order hybrid clustering algorithm comprises three key components: initial data clustering, centroid acquisition, and class merging. The PSO clustering algorithm initially engages in data clustering, amalgamating multiple class clusters from datasets after a finite number of iterations. Candidate centroids are derived from the data attributes via the density peak search algorithm, as illustrated in Fig 3. Subsequently, the initial class clusters and candidate centroids facilitate centroid determination. Finally, a division-based approach allocates data points to the class corresponding to the nearest centroid. Given scenarios with multiple identified class centers, a class merging operation is employed at the algorithm’s conclusion to consolidate class clusters.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Flow chart of density peak clustering algorithm.

https://doi.org/10.1371/journal.pone.0302741.g003

After inputting the dataset to be clustered, the algorithm proceeds as follows:

Step 1: Calculate the distance between two data points d_ij(i≠j).

Step 2: Calculate the truncation distance d_c, where d_c = d_f(Mt), f(Mt) is obtained by rounding Mt.

Step 3: Calculate p_i and the distance φ_i. We use the truncated kernel calculation, which is the common way to calculate p_i. Thus , where χ is defined as follows: (11)

Distance φ_i is measured by calculating the minimum distance between point i and any other point with a higher density: (12)

For densely populated points: (13)

It can be noticed that only those points whose densities are locally or globally maximal are larger than the normal neighbour spacing, i.e., there is a large φ_i, so that points with abnormally large values of φ_i may also be the clustering centres. The clustering process of the algorithm is to take the points with large local density p_i and large φ_i as the class centre, and the points with small local density but large φ_i are considered as anomalous points. Once the class centres are identified, the other points are divided into the classes represented by the closest class centres.

Step 4: Calculate y_i and sort it in descending order according to the following formulae to get the set of candidate centroids CEN_sus = (cen₁,cen₂,⋯,cen_s), y_i is calculated as follows: (14)

Step 5: Run the improved PSO algorithm from Section 3.1 to get the initial clustering of the data Int = (c₁,c₂,⋯,c_n)

Step 6: Judge the candidate centroids of the class affiliation in the middle to get the exact set of clustered centroids CEN_sure.

Step 7: Divide the remaining data points to their nearest class centroid to get the final class cluster.

We are in step 5, the PSO algorithm stops iterating after a certain number of iterative runs, at this point the data has formed the beginnings of the class, but not all the data is well classified into a fixed class, at this point it is also necessary to appropriate processing of the existing results. If there is no data in the neighbourhood of an object or its neighbourhood is less than a certain constant, it is considered to be an isolated point and the point is not counted in the initial clustering. Otherwise the point and its neighbourhood are considered as a class, from which multiple classes are formed to get the initial clustering Int.

The formation of the candidate centroid set is carried out next. The candidate centroid set is defined in the algorithm in order to eliminate the manual selection process. Firstly, calculate y_i = p_i *φ_i to get y_i, then calculate the second-order difference value of y_i, find the index position where the very small value point is located, and intercept the first few data points of the index position, but in most cases, the number of points intercepted by this method is larger than the real class centre, so we form the candidate centroid set here for the next steps.

In determining the centroids from the set of candidate points, we mainly applied the initial clusters formed in step 5 Int, which determine the centroids as follows:

For each class, we first calculate the sum of the distances S from the data points in the class to the candidate centre cent_i, S is calculated as follows: (15)

The candidate centroid with the smallest S is subsequently used as the exact centroid.

Experimental indicators

The F-measure serves as a pivotal metric for evaluating clustering performance comprehensively, amalgamating precision and recall as vital indicators. Within clustering analysis, the F-measure value directly mirrors the efficacy of the clustering outcome.

Precision primarily quantifies the fraction of positive example samples correctly identified within the clustering results, indicating the ratio of true examples to the samples predicted as positive examples.

Conversely, Recall assesses the proportion of all actual positive example samples correctly identified, representing the ratio of true examples to the total number of actual positive example samples.

Formally, each benchmark classification C_i (given by the true labels of the input dataset) corresponds to a collection of n_i objects required for a query. Each cluster C_j' obtained by the clustering algorithm corresponds to the set of n_j objects retrieved from a query. n_ij denotes the number of objects in the base classification C_i in the cluster C_j'. For each benchmark classification C_j' and cluster C_j', F-measure(F), Precision and Recall are defined as follows: (16) (17) (18) (19) where b in Eq (18) is equal to 1, which allows both precision and recall to remain equally weighted.

Results

In order to comprehensively assess the performance and stability of the algorithms presented in this paper alongside the comparative algorithms, this subsection presents experimental comparison results across both datasets. Fig 4 illustrates the average F-measure and recall values for each scheme on datasets S1 and CMC. The algorithm introduced in this paper exhibits superior performance compared to the three comparative algorithms. Notably, on dataset S1, this paper’s algorithm attains an average F-measure and recall above 99%, surpassing all others. The best-performing PSC-RCE algorithm achieves only 88.57% and 88.92% for F-measure and recall, respectively, substantially trailing behind this paper’s scheme by 10.65% and 10.35%.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Experimental comparison of different clustering algorithms.

https://doi.org/10.1371/journal.pone.0302741.g004

Furthermore, the experimental results obtained by the proposed model display a highly concentrated overall distribution with minimal outliers. This observation indicates robust stability across both datasets, affirming the strength of this paper’s algorithm.

To scrutinize the stability factors behind the algorithms presented in this paper, we conducted ablation experiments, considering variations such as the original PSO algorithm, the improved PSO algorithm, and the inclusion or exclusion of density values for clustering. We denoted E1 as the clustering algorithm devoid of density values and lacking the improved PSO algorithm, E2 as the clustering algorithm devoid of density values but integrating the improved PSO algorithm, E3 as the clustering algorithm incorporating density values without the improved PSO algorithm, and E4 as the clustering algorithm integrating both density values and the improved PSO algorithm. Experimental evaluations of F-measure, recall, and precision for each model are depicted in Fig 5.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Results of ablation experiment.

https://doi.org/10.1371/journal.pone.0302741.g005

Comparing E2 with E1 highlights a substantial enhancement in F-measure, precision, and recall (73.91%, 71.91%, and 60.91%, respectively, on dataset CMC) upon employing the improved PSO algorithm. Relative to E3 and E1, it’s evident that introducing density values without a higher-order clustering algorithm doesn’t efficiently enhance performance. Moreover, on dataset S1, the recall values for E1, E2, E3, and E4 reach 60.92%, 80.62%, 73.23%, and 82.22%, respectively. These metrics underscore that the higher-order hybrid clustering algorithm devised in this paper augments traditional PSO clustering algorithm performance by approximately 34.96%.

Fig 6 illustrates the iterative training’s impact on the clustering algorithm’s ability to identify truly positive samples, as measured by recall and precision metrics. Initially, as the algorithm learns, it gradually identifies more truly positive samples, leading to a steady increase in both recall and precision. Notably, due to our mitigation of overfitting concerns, recall and precision remain steady without declining over the course of iterations.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Training process.

https://doi.org/10.1371/journal.pone.0302741.g006

Moreover, the relatively balanced distribution of positive and negative samples in both datasets, coupled with the algorithm’s high stability, results in moderate fluctuations in recall and precision. Ultimately, the recall stabilizes at 82.22% and 62.87%, while precision stabilizes at 98.89% and 73.87% across both datasets.

To further explore the efficacy of the algorithm presented in this study for text data clustering, we conducted experiments on synthetic datasets employing various text vectorization methods. This investigation aimed to ascertain the algorithm’s consistency across diverse text vector representations and identify the representations that optimize its performance. Fig 7 illustrates the impact of the algorithm on both synthetic and CMC datasets, depicting accuracy and recall concerning Doc2vec and LDA subject models. Given the flexibility to train vectors to varying dimensions, our experimentation with Doc2vec involved dimensions of 100 and 200, while LDA utilized dimensions of 30 and 50. The VSM dimension corresponds to the lexicon, encompassing 75,307 feature items.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Comparison of clustering performance under different text vector formation methods.

https://doi.org/10.1371/journal.pone.0302741.g007

Observing minimal differences in clustering outcomes across varying dimensions of the same representation, it’s evident that LDA, leveraging thematic vectors, crafts semantic-level textual representations, yielding the highest accuracy and recall post-clustering. Specifically, employing LDA thematic vectors with a dimensionality of 50, this paper’s approach achieves an accuracy rate of 79.2% and 83.5% on datasets S1 and CMC, respectively, alongside recall rates of 84.4% and 86.2%. These rates notably surpass other text data representations.

Contrarily, while LDA demonstrates superior performance, few studies adopt LDA topic vectors for text vectorization. To address this, we introduced Doc2vec, a document representation model necessitating iterative runs. Results indicate that with Doc2vec dimensions set at 100, this paper’s approach achieves accuracies of 73.1% and 75.1% on datasets S1 and CMC, respectively. With Doc2vec dimensions set at 200, accuracies reach 73.6% and 75.8% on the same datasets. Notably akin to the LDA model, Doc2vec generates text vectors at the semantic level via neural network-based formation, yet it lacks interpretability, warranting further investigation.

Furthermore, our findings underscore VSM’s notably inferior results. This can be attributed to VSM employing lexical items as vector dimensions and TFIDF features as word weights, resulting in excessively high-dimensional and sparse representations, thereby exhibiting poor algorithmic performance.

The LDA topic model’s dimensionality reduction yields a more concise and meaningful data representation, capturing semantic information within documents. This enhancement aids clustering algorithms in comprehending document content more effectively. Iterative experiments conducted on the higher-order hybrid clustering algorithm (Fig 8) reveal that employing LDA’s dimensionality reduction enables accurate clustering of similar data, enhancing clustering quality to achieve stability with fewer iterations.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Recall curve of higher-order hybrid clustering algorithm incorporating LDA model.

https://doi.org/10.1371/journal.pone.0302741.g008

Remarkably, on dataset S1, after 250 iterations, a rate of 93.41% is achieved, while on dataset CMC, 79.85% is reached at 130 iterations. This notably enhances cluster recall. Consequently, integrating the LDA Topic Vector Model into the design of health information management models for the elderly promises improved data input and representation for clustering algorithms, thereby enhancing clustering efficacy and efficiency.