1. Introduction

With the development of machine learning, support vector machine (SVM) is a new machine learning method in statistical learning theory, and has achieved remarkable results in face recognition, population prediction, image retrieval, data mining and other fields [1–4]. SVM has good generalization performance, and its classifier shows special advantages in solving the pattern recognition problems of small samples, nonlinearity and high dimension, especially in dealing with classification problems [5–7]. At present, the research of SVM mainly includes the following three aspects: First, the improvement of SVM model. Lopez et al. [8] proposed an extension of the non-parallel SVM method and experimentally demonstrated the advantages of achieving the best average performance compared to other SVM methods. Subsequently, Chen et al. [9] improved projective Twin Support Vector Machine (PTSVM) to a novel non-parallel classifier, and experimental evaluation on both synthetic and real-world datasets demonstrated the feasibility and effectiveness of the proposed approach. In literature [10], a new weighted quantum particle swarm optimization (WQPSO) hybrid model is proposed for sequence data clustering based on Smooth Support Vector Machine (SSVM) for classification. The limitation of the proposed work is an increase of computational complexity due to the usage of weighing optimization strategy. Some studies yet to be conducted to reduce the computational complexity. In order to solve the problem that the existing common classification algorithm has poor fault tolerance, identify single lithology and cannot effectively overcome the imbalance between classes, Su et al. [11] proposed an improved Synthetic Minority Over Sampling Technique (SMOTE)to process data sets and proposed a new fuzzy membership function to improve the fuzzy twin support vector machine. Sun et al. [12] proposed a partial binary tree twin support vector machine multi-classification algorithm based on optimal classification features (OCF-PBT-TWSVM) to achieve effective classification of non-stationary transient random signals with edge distortion of tooth profile images, and to meet the requirements of real-time gear vision measurement and distortion compensation accuracy claim. Fang et al. [13] proposed a new model: similarity feature transformation smooth support vector machine based on fuzzy C-means and the experimental results indicate that the proposed model has better performance compared with the conventional SVM model as well as some variants in terms of classification accuracy and training time. In order to solve the problem of low approximation accuracy of S-shaped smooth function adopted by smooth twin support vector machine (STWSVM), a polynomial smooth twin support vector machine model based on Newton Armijo optimization (PSTWSVM-NA) was proposed in [14].

The second is the application of SVM model. Indrawan et al. [15] focused on developing this prediction at the suicidal behavior of psychiatric patients by using smooth support vector machine method, which is very useful for the prevention of psychiatric hospitals. In order to solve the problem of vibration signal recognition of fiber perimeter, Ma et al. [16] proposed a vibration signal recognition method based on SVD and MPSO-SVM, and predicted the fiber signal, achieving good classification results. Wang et al. [17] proposed a new method combining computer numerical simulation method with machine learning to predict deep mineralization. Wang and Dong [18] combined support vector machine (SVM) and ant colony algorithm (ACA) to study the problem of power human accident, and proposed a statistical analysis model for data fitting and prediction of power human accident.

Third, the algorithm for solving the SVM model. In order to solve the excessive consumption of space and time in standard support vector machines, a new method of training support vector machines using boundary points is proposed in [19]. To improve the accuracy of group activity recognition in video, a group activity recognition algorithm based on tensor features and twin support vector machines is proposed in [20]. In order to reduce the error of communication information security situation prediction, Wang et al. [21] designed a support vector machine communication information security situation prediction model based on ant colony optimization algorithm, and compared it with the traditional support vector machine.

In general, support vector machine (SVM) is a new machine learning method based on statistical theory. It has outstanding performance in small sample, high-dimensional and nonlinear problems, and has achieved great success in many fields. Although the conventional support vector machine has many advantages, there are still many problems. In recent years, researchers have a large number of methods to reduce the impact of outliers and noise points on conventional support vector machines. At the same time, many researches have been carried out on how to filter support vectors effectively when dealing with large-scale datasets, and there are generally two aspects that need to be improved. Firstly, the conditional quadratic convex optimization problem restricts the use of some effective optimization algorithms in the field of unconditional optimization; Secondly, when large sample data sets need to be processed, the training time of the algorithm is relatively long and the efficiency is low.

Therefore, to overcome such problems, Lee and Mangasarian [22] first to introduce the Sigmoid integral function into the SVM to smooth the non-differentiable SVM model, and established a smooth support vector machine model (SSVM) based on the Sigmoid function. Thus, many fast optimization algorithms can be used to solve it, which greatly reduces the computational complexity. SVM has strict convexity and infinitely differentiable mathematical characteristics, and introduced efficient unconstrained optimization problems, which attracted many scholars to study SSVM models from different perspectives [23–29], Some proposed different smoothing functions [23–26], while others extended them to the prediction field [16,17]. Other applications of SVM and SSVM are introduced in [27–29].

In conclusion, because the objective function of unconstrained SVM model is not smooth, many fast optimization algorithms can’t solve it. Many scholars [22–26] proposed to solve such problems by approximating the objective function of support vector machine with smooth function. In the research, we found that the performance of SSVM will change with the change of smooth function, that is, the higher the approximation accuracy of smooth function, the better the convergence of the corresponding SSVM model, and thus the higher the classification accuracy. To further explore the smooth function with higher approximation accuracy, based on previous studies, this paper first constructs padé33 rational smooth function to approximate positive sign function through the calculation method of determinant formula and the definition of an orthogonal polynomial of padé type approximation. Secondly, the approximation accuracy of the smooth function and the convergence performance of the model are analyzed and proved, and compared with the existing smooth functions. Finally, through the numerical experiments on heart disease, it is proved that the smooth vector machine model based on the smooth function in this paper can obtain high classification accuracy when used to deal with classification problems.

This paper is organized as follows. First, Section 2 reviews some preliminary knowledge about smooth support vector machine model. Then, a new padé33 smooth function and two theorems are proposed in Section 3. After that, we present numerical examples to illustrate the potential applications of the new models and compare six methods of the model in Section 4 and Section 5. Finally, the concluding remarks and future research directions are given in Section 6.

2. Smooth support vector machine model

For binary classification problems [30], the training set includes m samples , where x_i∈Rⁿ is the row vector and y_i∈{1,−1}. These m samples such as x_i (i = 1,⋯,m) are represented by matrix A_m×n. x₁,x₂,…,x_m is divided into two categories A⁺ and A⁻. If A_i belongs to class A⁺, denoted as 1; otherwise, A_i belongs to class A⁻, denoted as -1, that is, the goal of the support vector machine is to divide A₁,A₂,…,A_m into two categories: A⁺ and A⁻. Therefore, the specific classification can be represented by a diagonal matrix D of type m×n, where the diagonal element of D is 1 or -1. Thus, a strongly convex support vector machine (SVM) model modified by this problem is established as follows: (1)

Where ξ is the relaxation variable, e = (1,…,1)^T is the column vector with all units of 1, C>0 is the misdivision penalty parameter, and ω is the normal vector of the following boundary surface: (2)

In Formula (2), b is the distance from the boundary surface to the origin.

For model (1), on the one hand, when A⁺ and A⁻ are strictly linearly separable, there is ξ = 0; x^Tω−b = 1 and x^Tω−b = −1 are the boundaries of A⁺ and A⁻ respectively. At this time, x^Tω = b is the classified hyperplane, that is, the plane is located between the two edge interfaces described in (2), as shown in S1 Fig. On the other hand, when A⁺ and A⁻ are quasilinear separable, then there is ξ>0, which will make: if x^T = A_i, D_ii = 1, then x^Tω−b+ξ_i≥1; Otherwise, x^T = A_i, D_ii = −1, and x^Tω−b+ξ_i≤−1.

Obviously, model (1) is a quadratic programming problem with constraints. In (1), if (3)

Here (.)₊ is a positive sign function, that is, , and i = 1,2,…,m, Replace (3) in Formula (1) to obtain an unconstrained optimization model: (4)

The objective function of the unconstrained optimization model (4) has the characteristics of strong convexity. Since the minimized objective function contains a non-smooth positive sign function ξ = (e−D(Aω−eb))₊, it cannot be solved by a smooth unconstrained optimization algorithm. Therefore, scholars have proposed different styles of smooth functions to approximate positive sign functions and established SSVM models with sigmoid integral functions, quadratic and quartic polynomials, cubic and quantic spline functions, and padé22 approximation rational functions as smooth functions [22–26]. Since different smooth functions can construct different SSVM models correspondingly, for convenience, the above smooth models are recorded as sigmoid-SSVM, P2 -SSVM, P4-SSVM, T3-SSVM, T5-SSVM, padé22-SSVM respectively. For the convenience of subsequent experiments, a unified model (5) is used to solve the linearly separable case, and the function f(x) in (5) can be replaced by different smooth functions.

(5)

3. Construction and analysis of a new padé33 smooth function

3.2 Approximation accuracy and model convergence analysis of padé33 rational smooth function

Theorem 1 Let x∈R, Padé(x,k) be a function defined by Eq (10), then:

1. (1) Padé33(x,k) has second-order smoothness and at the point satisfies the following conditions:

1. (2) Padé33(x,k)≥x₊;
2. (3)

Proof: (1) (2) easy to prove, which can also be seen from the image.

For (3), discuss according to the situation.

When , or , is true obviously.

When , let t = kx, then t∈[−1,1], so

Let , where t∈[−1,1], use MATLAB software to solve the maximum value of g(t) at t∈[−1,1], and get max g(t) = 0.0051, thus .

To sum up, no matter what value x takes, there is always .

Theorem 2 let A∈R^m×n and b∈R^n×1.The real function h(x):Rⁿ→R and g(x,k):Rⁿ×N→R are defined as follows: (11) (12) Where k>0, Padé33(x,k) is defined by Formula (10), then the following conclusion is true:

1. h(x) and g(x,k) are strongly convex functions;
2. The optimization problems and have unique solutions, which are recorded as and respectively;
3. For any k≥1, holds, where m is the number of training samples;
4. 。

Proof: the proof idea of this theorem is similar to the method involved in literature [7]

1. Because of the inherent strong convexity of , according to the formulas of h(x) and g(x,k), it can be concluded that these two functions also conform to strong convexity, so they are strong convex functions.
2. According to the formulas of h(x) and g(x,k), the corresponding level sets can be set separately as follows: . Because of , for any v≥0, they satisfy . Therefore, both min h(x) and min g(x,k) are compact sets in Rⁿ space, so the optimization problems min h(x) and min g(x,k) have optimal solutions. In addition, for any k∈Z⁺, h(x) and g(x,k) also satisfy strong convexity, so and have unique solutions, which can be recorded as and 。
3. Let the unique solutions of min h(x) a and min g(x,k) be and respectively, which is obvious: ,

Add the above two equations to get:

From (3) of Theorem 1:

1. (4) From (3): , so

Therefore .

From theorem 2, it can be found that when the smoothing coefficient k tends to infinity, the unique solution based on padé33 SSVM can be approximate to the solution of the original optimization model (4). Therefore, the solution of padé33 SSVM established in this paper meets the convergence. In addition, according to Table 1 and S2 Fig, it can be found that the Padé33 rational smooth function constructed in this paper is closer to the positive sign function than the previous smooth function, and its approximation accuracy is one order of magnitude higher than the original Sigmoid integral function. At the same time, in terms of convergence speed, the Padé33-SSVM established in this paper is also faster than the existing SSVM, and two orders of magnitude higher than the Sigmoid-SSVM.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Approximation accuracy of different smooth functions and the convergence rate of the model.

https://doi.org/10.1371/journal.pone.0280804.t001

4. A training algorithm for smooth support vector machines

In model (5), the nonlinear classification problem can be solved only by introducing a kernel function. Therefore, model (5) still maintains strong convexity and smoothness for any kernel function, and its convergence still holds in this nonlinear case. Therefore, for the smoothed model (5), the BFGS Armijo optimization algorithm [25] can be directly used to classify its solution. To make better use of this algorithm, this paper unifies the notation, denoting the objective function of the model (5) as φ(w,b), its first-order derivation as ∇φ(w,b), the solution accuracy of the algorithm itself is mainly ε = 10⁻⁵, and the number of iterative steps is denoted as i. Thus, the BFGS-Armijo optimization algorithm is used to give the calculation steps when the objective function is first-order smooth, as follows:

Step 1: Initialization set max = 1000, i = 0,and ε = 10⁻⁵.

Step 2: If i≤max, calculate v_i = ∇φ(u_i).

Step 3:If , then stop, the optimal solution of model (5) can be obtained as u_i = (ω_i,b_i). Otherwise, calculate d_i = −H_iv_i, then go to step 4;

Step 4:Performe a one-dimensional search along the direction d_i to get the step factor α_i>0:

Let

Then calculate

Step 5: Correct H_i to get H_i+1;

Step 6: Then let i = i+1, go to step 2; where φ(u_i) is defined according to model (4), and α_i can be obtained according to the following formula:

5. Numerical experiments

To verify the application effect of padé33-SSVM model constructed in this paper in practical classification problems, based on the training and testing data sets of the UCI machine learning database, the training and prediction experiments of the classifier were designed for the heart disease data set. The experiment in Section 5.1 uses different SSVM models to train on the heart disease data set with different scales, and compares them with classical SVM and LSSVM. The main purpose of the experiment in Section 5.2 is to verify whether the Padé33-SSVM can obtain more accurate classification accuracy in prediction under different numbers of training samples. In addition, simulation experiment 5.3 is mainly aimed at the classification machine training experiment of different SSVMS with kernel function under different datasets of different sizes, and mainly verifies the applicability of the proposed model in classification problems under large datasets.

5.1 Classifier training based on heart disease data

Experiment 1: The heart disease dataset was obtained from the Cleverland Clinic Foundation, with sample data available from the UCI Machine Learning Database. There are a total of 270 sample data, and each sample includes 13 attributes such as gender, age, diastolic blood pressure, cholestoral content per deciliter of plasma, chest pain category, and blood sugar. All the sample data on heart disease status are divided into two categories: presence and absence.

Due to the large individual differences of each patient and the different speed of heart rate, the value range is significantly different. At the same time, in view of the redundancy and noise of the real data provided by the hospital, it is necessary to normalize the data to make the data value between [–1,1]. By normalizing the data, the redundancy of sample data can be eliminated, and the low computational efficiency under large sample data can also be overcome.

To compare the classification effects of different smooth support vector machine models, the generalization ability of the classifier is generally used as an index. The generalization ability of the classifier is usually measured by the accuracy of the training samples, and the time CPU consumed by the algorithm is recorded as the training time (s). The experiment in this section adopts model (5), in which the function f(x) adopts Sigmoid integral function, quadratic and quartic polynomial, cubic and quintic spline functions, Padé22 and Padé33 rational smooth function constructed in this paper. By solving the smooth support vector machine (padé33-SSVM) based on padé33 rational smooth function, the decision function is g(x) = sgn(ω^Tx−b), where . Different SSVM models are used to train the data sets with inconsistent scales on the heart disease data sets, and BFGS Armijo algorithm is used to solve Eq (5). The performance indicators obtained from the solution are shown in Table 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Comparison of different SSVMS with SVM and LSSVM at different scales.

https://doi.org/10.1371/journal.pone.0280804.t002

Two conclusions can be drawn from the experimental results in Table 2. On the one hand, under different training scales, compared with classical SVM, the Padé33-SSVM constructed in this paper has an absolute advantage in training accuracy and time. Compared with LSSVM, the training accuracy in this paper is also relatively high. On the other hand, when different SSVM models are used for classification training under different data scales, the larger the scale, the higher the accuracy and the longer the time-consuming. In addition, under the same data scale, the smooth model established when the padé33 rational smooth function constructed in this paper approximates the positive function can obtain higher classification accuracy in the classification of heart disease data. Therefore, the smooth support vector machine (padé33-SSVM) based on the padé33 rational smooth function constructed in this paper can obtain higher accuracy when it is used for classification training of heart disease data. The numerical experiment shows that we can get the decision function through Padé33-SSVM, and can obtain more accurate judgment when we predict the data of heart diseases with 13 attributes.

5.2 Classifier prediction based on heart disease data

Experiment 2: According to 5.1, the total sample data about heart disease is 270, and each sample data has 13 attributes. This experiment mainly verifies whether the padé33-SSVM established in this paper can obtain more accurate classification accuracy in prediction. The experiment divides the sample data into two steps. The first step is to obtain the classification accuracy, training time, and decision function by randomly searching the data for training; The second step is to predict the remaining test data through the decision function of one stage, and the accuracy and time of prediction can be obtained. The experiment adopts model (5), in which the smooth function f(x) is the padé33 rational smooth function constructed in this paper. The classification effect is shown in Table 3 by solving padé33-SSVM through the optimization algorithm:

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Classification and test comparison of padé33-SSVM under different training sample functions.

https://doi.org/10.1371/journal.pone.0280804.t003

The experimental results are shown in Table 3. With the increase of the number of training samples, the more accurate the decision function is, and then when predicting the remaining sample data, the higher the accuracy is. This is also consistent with the reality, that is, the larger the sample data size, the more accurate the decision function can be trained. In addition, the decision function trained is also different due to the different number of training samples, and after the padé33-SSVM training constructed in this paper, the model can adjust the parameter C value to obtain the function with high training classification accuracy for the second stage of prediction experiment. Therefore, for the training samples of heart disease under the scale of large data, a more accurate decision function can be obtained after the Padé33-SSVM classification training in this paper. As a result, the data of heart disease can be diagnosed and predicted, the potential patients with heart disease can be predicted and treated early, and the early warning can be given to those who do not suffer from heart disease.

5.3 Classifier training based on large data sets

Experiment 3: The experiment in this section solves the classification problem of large data sets by solving model (5), and the kernel function selected is Gaussian kernel function [14] . The sample datasets used in this experiment include Banknote authentication, EEG Eye Status, QSAR biodegradation, Crowdsourced mapping and Diabetic Retinopathy Debrecen datasets, which are all from the UCI database (http://archive.ics.uci.edu/ml/datasets.php?format=&task=cla&att=&area=&numAtt=&numIns=&type=&sort=nameUp&view=table). The banknote authentication data is 1372 * 4, extracted from the image used to evaluate the banknote authentication program, and all data samples are divided into two categories. The original EEG state data is 14,980 * 20 dimensions, and the data are divided into two categories: ’1’ for closed eyes and ’0’ for open eyes; QSAR biodegradation data is 1055 * 41, and 1055 chemical samples are divided into 2 categories; The original mapping data of crowdsourcing is 10546 * 28, and the crowdsourcing data is divided into six categories. To facilitate calculation, two categories (farm and forest) data are randomly searched as training data. The Debrecen data of diabetes retinopathy is 1151 * 20. The data set contains features extracted from the Messidor image set to predict whether the image contains signs of diabetes retinopathy, and these 1151 chemicals are divided into two categories. In the experiment, 1372, 1500, 1055, 2000 and 1151 samples were randomly selected from the above dataset for classification training. The results are shown in Table 4. The data in each column is the training accuracy rate (%), and the training time (s) is in the parentheses in the second row.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Comparison of different SSVMS with SVM and LSSVM in different data sets.

https://doi.org/10.1371/journal.pone.0280804.t004

According to Table 4, under different data set sizes, compared with classical SVM and LSSVM, Pade33-SSVM constructed in this paper has a slightly lower training accuracy than SVM, but it takes a shorter time and speeds up the calculation speed. Compared with LSSVM, although the training time is relatively long, the training accuracy is significantly higher than that of LSSVM. Therefore, compared with SSVM and LSSVM, the Padé33-SSVM constructed in this paper is not only suitable for the training and prediction of heart disease data, but also more suitable for the classification of data sets of different sizes. In general, when different SSVMs with kernel function are used for classification training, the larger the scale is, the longer the time is. Under the same data scale, higher classification accuracy can be obtained when using Padé33 rational smooth function to approximate the positive sign function. Therefore, Padé33-SSVM is not only applicable to the classification of small-scale data, but also for the case of large data sets.

6. Conclusions

To overcome the non-differentiability of SVM, first, based on the Padé type approximation orthogonal polynomial definition and determinant algorithm, this paper constructs a Padé33 rational smooth function to approximate the positive sign function. Secondly, Theorem 1 and Theorem 2 are proposed to prove that the smoothness accuracy of the Padé33 function is significantly higher than that of the existing smooth functions, and the convergence performance of padé33-SSVM is better than that of other SSVMs. Finally, the new smooth model established in this paper is used to diagnose heart disease. The experimental results show that padé33-SSVM has better classification ability and higher classification accuracy than the other four SSVMs. Therefore, the padé33-SSVM constructed in this paper is of great significance in real life, not limited to the prediction of heart disease.

In the future, the padé33-SSVM model established in this paper can be used to predict the actual classification problems such as broader medical diagnosis, customer churn in company management, and whether the enterprise’s finance is in trouble.

Supporting information

Acknowledgments

This work was supported by the General Social Science Project of Beijing Municipal Education Commission [grant number SM202210037008] and the School level youth fund project of Beijing Wuzi University [grant number 2022XJQN35]. We would like to thank everyone who provided the materials included in this study. Additionally, we express our appreciation to Associate Professor Sun f or his valuable contribution in the English translation and editing of this work. We also thank anonymous reviewers for their constructive comments and suggestions.