Classification Performance Analysis in Medical Science: Using Kidney Disease Data R. A. Jeewantha, Malka N. Halgamuge*, Gullu Ekici, Azeem Mohammad, Ali Syed School of Computing and Mathematics, Charles Sturt University, Melbourne, Victoria 3000, Australia e-mail: {MHalgamuge, GEkici, AMohammad, ASyed}@studygroup.com Abstract. Health-care practices face storage problems in a growing world. Huge storage demands cause undeniably problems leaving health practitioners exclaimed. Without delay, aggregated data becomes too efficacious analyzed and handled by traditional approaches. As a solution to this problem, Data mining delivers the technology and procedure to convert these embankments of ordinary data into meaningful evidence for futuristic planning and decision-making. Data mining is a tool that not only solves the problem of piled up data; nonetheless it similarly turns it into meaningful data themes based on reoccurrences of trends in the data. The healthcare trade is mostly “information and document rich industry,” and manually handling is not feasible in practical life. These huge volumes of data have been key to the arena of data-mining to generate associations among the attributes and extract expedient information. Recent research shows that combating Kidney diseases is a complex assignment that involves considerable knowledge and experience for annual testing and screening. In developed nations, Kidney diseases have become a silent killer, that makes key factors of disease burden in third world nations. Various data mining procedures are available for forecasting diseases such as clustering, classification, association rules, regression, and summarizations. The key objective of this study is to analyze datasets collected from 400 patients grounded on 25 different attributes attended for treatment for Chronic Kidney Disease (CKD), after using classification methods to forecast class precisely. Our analysis illustrates that Multilayer Perceptron is the most suitable classification method that outperforms with highest classification accuracy by 99.75% (0.0085 error) with only 5% of fluctuation among algorithm measured. Introspectively, the computational time, Multilayer Perception can be time-consuming comparatively, when it comes to deal with billions of data. Nonetheless, for the field of bioinformatics and medical science accuracy, the key objective is to deal with sensitive data as a single error can lead to a disaster. Hence, our results show that Multilayer Perception classification method is the most accurate and suitable classification algorithm used in the field of bioinformatics and medical science, for further data analysis and predictions. This paper will be useful for many medical institutions and work related bioinformatics in pursuance to understand the prediction accuracies and data patterns in related work. Keywords: Big Data, Classification, Chronic Kidney Disease (CKD), Multilayer Perceptron 1. Introduction In modernistic ways, computers have contributed substantially to the storage and administration of gigantic volumes of data. Besides, the evolution of the database management systems (DBMS) constructs the path to a large numbers of medical/healthcare databases. Fashioning management and knowledge of significant amounts of miscellaneous data have turned into a key area of research. The process of recognizing potentially useful, original, accurate and conclusively comprehensible patterns in data, is namely Data Mining [1]. Data mining practices can be categorized into two groups, supervised and unsupervised learning techniques. Unsupervised learning methods do not build a hypothesis Proceedings of 2017 International Conference on Big Data Research (ICBDR 2017) Osaka, Japan, 22-24 October, 2017 prior to analysis directed by a variable. A model will be constructed, upon the results, and clustering is a common unsupervised technique [2]. Supervised learning method involves constructing a model, before the analysis is performed. Supervised learning methods such as Statistical Regression, Classification, and Association rules are used in both clinical and medical platforms [3]. This paper aims to use classification methods in the area of bioinformatics and medical science. Classification is frequently customary to a data mining method, and occupies a set of classified instances to create a prototype that can be acclimated to classify the group of hefty records. The primary objective of the classification is to forecast the target classes precisely and accurately for the given case. Chronic kidney disease (CKD) is a state that causes gradual failure of kidney function. The core utility of the Kidney is to filter the blood and eradicating wastes from the body. Kidney moves the filtered wastes to the bladder and removes it later through urination. When Kidney is unable to perform its regular work, then body loses its fluid balance, and it becomes weighed down with dispensable wastes. It is probable to say it drops as far as 90% of the kidney function without giving any problems or symptoms. That makes it an infamous silent killer. Kidney failures can be categorized into chronic or acute. Sudden failure of kidney functions take place when the extraordinary level of waste is produced from the metabolism of the body and added into the blood, as this is called acute kidney disease (AKD). Chronic kidney disease (CKD) is the permanent failure of the kidney with the gradual progress of failure. This is the furthermost common kidney disease type, as it takes place when the kidney is not functioning or damaged over some period (months or longer). Risk factors which proliferation the possibility of Kidney Disease are: Hypertension, Obesity, Heart Disease, Age, Diabetes, Drug Abuse, Family History of Kidney Disease, Race/Ethnicity. Indications of kidney diseases are: Difficulty or pain during voiding, swelling & pain in the back or side, changes in your urinary function, Blood in the urine, feeling cold all the time, extreme fatigue and generalized weakness, Skin rashes and itching, Dizziness & Inability to concentrate, Nausea and vomiting, Ammonia breath and metallic taste, Shortness of breath. Many scholars have used various data mining procedures and techniques for future predictions on bioinformatics and medical science. In 2014, Lakshmi et. al [4] analyzed the performance of Logical Regression, Artificial Neural Networks, and Decision Tree algorithms. These three algorithms measure survivability of kidney dialysis. The data mining methods is diagnosed by the measurements of specificity, sensitivity, and classification accuracy. They have used a data mining tool named Tanagra. The confusion matrix and 10fold cross-validation are used to evaluate each technique. Their final results show that ANN produces better results than other two methods. A forecasting system has been built [5] by using k-means and A-priori algorithm for kidney failure and heart disease forecasting. In their study k-means and A-priori algorithms were used to forecast kidney failure patient datasets that contain 42 attributes. They analyzed the dataset through machine learning mechanisms such as attributes and distribution statistics, trailed by k-means and A-priori algorithms. They appraised the data by using calibration plots and Receiver Operating Characteristic (ROC) plot. With the aim of forecast Long-Term Kidney Transplantation result; a group [6] construed discernment among the Logistic Regression and Artificial Neural Network. Evaluation has been built on the specificity and sensitivity of an Artificial Neural Network and Logistic Regression in the forecast of Kidney rejection in 10 validating and training datasets of kidney transplant receivers. Experimental results both show that algorithm methodologies were complementary and their combined algorithms were used to enhance the forecast of kidney transplantation and clinical decision-making process. The Artificial Neural Networks (ANN) has also been used [7] to categorize patients’ medical status as this is likely to lead to End Stage of Kidney Disease (ESKD).
The classifier inspires the results given by a group of 10 neural networks trained on the data gathered over 38year period at the University of Bari. The sophisticated application is available online and as well as on Android application. This is an important application worldwide based on its clinical usefulness. In 2012, Eyck et.al [8] investigated data mining procedures for forecasting acute kidney damages after elective cardiac surgery with machine learning and Gaussian process methods (regression task and classification task). Meanwhile, Kumar et.al [9] outlines the comparison of neural networks such as Radial basis function (RBF), multilayer perceptron (MLP) and Learning Vector Quantization (LVQ) based on computational time, accuracy and training dataset size. Later in 2014, Jain et.al [10] showed the effects that diabetics have on kidney. They used Tanagra mining tool and C4.5 algorithms and the performance was assessed based on recall, error rate, and precision. This study collected datasets of 400 patients grounded on 25 different attributes, who, attended for treatment for Chronic Kidney Disease (CKD), and analyzed it using classification methods to forecast the class precisely. 2. Material and Methods 2.1. Data Collection This study examines the accuracy of different classification algorithms on forecasting the survivability (class) of Chronic-Kidney Disease. The collected experiment data is based on Chronic Kidney Disease dataset downloaded from the University of California, Irvine, (UCI) School of Information and Computer Science [11]. Data include 400 patients grounded on 25 different attributes, comprehensive description of the data attributes which is presented in Table 1. Table 1: Data Attributes and Description of Attributes Attribute Description Age Age of the patient (Numerical value: years) Blood pressure Blood pressure of the patient (Numerical value: mm/Hg) Specific gravity The ratio of the density of substance (Nominal value: 1.005,1.010,1.015,1.020,1.025) Albumin Albumin level in the blood (Nominal value: 0,1,2,3,4,5) Sugar Blood Sugar of the patient (Nominal value: 0,1,2,3,4,5) Red blood cells Patients’ Red blood cells count (Nominal value: normal, abnormal) Plus cell Patients’ Plus cell count (Nominal value: normal, abnormal) Plus cell clumps Availability of Plus cell clumps in the blood (Nominal value: present, not present) Bacteria Availability of Bacteria in the blood (Nominal value: present, not present) Blood glucose random Patients’ Blood glucose random count (Numerical value: mgs/dl) Blood urea Patients’ Blood urea level (Numerical value: mgs/dl) Serum creatinine Patients’ Serum creatinine level in the blood (Numerical value: mgs/dl) Sodium Patients’ Sodium level in the blood (Numerical value: mEq/L) Potassium Patients’ Potassium level in the blood (Numerical value: mEq/L) Haemoglobin Patients’ Haemoglobin level in the blood (Numerical value: gms) Packed cell volume Patients’ Packed cell volume in the blood (Numerical value: Value) White blood cell count Patients’ White blood cell count (Numerical value: cells/cumm) Red blood cell count Patients’ Red blood cell count (Numerical value: millions/cmm) Hypertension Appearance of high blood pressure (Nominal value: yes, no) Diabetes mellitus Whether patient having diabetes (Nominal value: yes, no) Coronary artery disease Whether patient having Coronary artery disease (Nominal value: yes, no) Appetite Appetite of the patient (Nominal value: good, poor) Pedal Edema Whether patient having Pedal Edema (Nominal value: yes, no) Anaemia Whether patient having Anaemia (Nominal value: yes, no) Class Doctors’ conclusion, whether particular patient having kidney disease or not (Nominal value: ckd, not ckd) 2.2. Classifiers Used Classification is one of the extensively used supervised learning methods. It transmits data into pre-defined classes or groups. In this classification process, the groups or classes are determined prior to the data analysis. Therefore, it is stated as supervised learning. This is a process where the collections of data, ideas or objects are classified into groups or classes, with similar associate characteristic [12]. This inquiry used seven classification procedures to experiment classification performances and accuracies over the given data set. We have chosen a better combination of algorithms for this study that are mostly used in distributed data mining. Method used in the Study is the Multilayer Perceptron (MLP). This is an artificial neural network model. This algorithm can employee any number of inputs, and it can have any number of hidden layers attached with its units. In the input layers, MLP uses linear combination functions in the hidden layers; it mostly uses sigmoid activation functions. MLP algorithm generate a number of outputs along with its activation function. Since the reaction of the hidden layers can be shared over complete input space, MLPs are often called as distributed-processing networks. This study also compares the classification percentage with Support Vector Machine (SVM), logistic regression, Decision Table, Naïve Bayes Classifier, J48, Multilayer Perceptron and Conjunctive Rules. 2.3. Characteristics required for Classification Algorithm this study, used the following four measurements: (i) Correctly classified instance: The number of instances correctly classified by given classification algorithm, (ii) Incorrectly classified instances: The number of instances not correctly classified by given classification algorithm. Sometimes this can occur due to noisy, inconsistent or out of scope data, (iii) Accuracy: Percentage of correctly classified instances, and (iv) Computational time: time (sec) that a particular classifier has taken to build the model. 2.4. Performance Evaluation We have used the 10-fold cross-validation technique and percentage split techniques to examine the performance of classification methods. Performance evaluation was done based on the comparison of following measures; Root mean squared error (RMSE), Kappa statistics, mean absolute error (MAE), and Receiver Operating Characteristic (ROC) Area. 2.5. Data Analysis The main aim of the research is to apply the classification approaches to analyses the kidney disease data. Open source software is used for the application of algorithms which is Weka (Waikato Environment for Knowledge Analysis, version 3.8). This tool has an in-built machine that uses algorithms with visualization tools and easy-to-use interfaces. The computer processor is Intel Core i5-3337U, 1.8GHz, and RAM (Random Access Memory) was 4GB. 3. Results and Discussion 3.1. Classification 10-fold cross-validation The generated seven different classifiers using 10-fold cross-validation test method. Below Table 2 shows the computational time, Correctly Classified Instances, Kappa statistic, Incorrectly Classified Instances, Root mean squared error, mean absolute error (MAE), Receiver Operating Characteristic (ROC) and Accuracy of the model. Table 2: Results for 10-fold cross-validation test method Classification Method Computation Time (sec) Correctly classified Instances Incorrectly classified Instances Kappa Statistics Mean Absolute Error Root Mean Squared Error ROC Accuracy Conjunctive Rule 0.25 94.75% 5.25% 0.8869 0.0810 0.2237 0.942 94.75% J48 0.17 99% 1% 0.9786 0.0225 0.0807 0.999 99% Logistic Regression 0.44 97.50% 2.50% 0.9472 0.0252 0.1530 0.994 97.50% Support Vector Machine (SVM) 0.13 97.75% 2.25% 0.9526 0.0225 0.1500 0.982 97.75% Decision Table 0.63 99% 1% 0.9786 0.1815 0.2507 0.992 99% Naïve Bayes Classifier 0.03 95% 5% 0.8961 0.0479 0.2046 1 95% Multilayer Perception 4.89 97.75% 2.25% 0.9947 0.0085 0.0622 1 99.75% The Kappa statistics and error rates related to 10-fold cross-validation test method seem to provide the Kappa statistic closer to 1. Hence, all classifiers give good agreement, and when we looked at error rate, we can see the Multilayer Perception that gave us the lesser error rate compared to other classifiers. Figure 1: Accuracy comparison of all classifiers based on th
e average accuracy Figure 2: Classification accuracy of Multilayer Perceptron classifier with different percentage of training data Figure 1 shows that, the Multilayer Perceptron classifier has given a slightly better accuracy rate compared to other classifiers. Figure 2 shows the accuracies generated by Multilayer Perceptron classifier which we can see that it gave better accuracies around when we use 60-90% of data as the Training data. We also observed the accuracy of 99.75% given by Multilayer Perception and J48 and Decision Table give 99% respectively. The time taken by each classifier to build the models on 10-fold cross-validation test method, has been identified as a big difference in time taken by Multilayer Perception classifier when compared to all other classifiers. All other classifiers built their models under 1 sec time, nonetheless Multilayer Perception has taken around 4.89 sec CPU time, which means that when it comes to a big number of data, if we use Multilayer Perception it can take a higher computational time comparatively. 3.2. Classification: Percentage Split Table 3 shows results of the output on the data set generated relevant to seven different classifiers on different percentages of training data. In this we have generated 9 different models for each classifier based on different training data sections (90%, 80%, 70%, 60%, 50%, 40%, 30%, 20%, and 10%) by using percentage split test method. Table 3: Percentage Split test method for Multilayer Perceptron Percentage of Training Data (%) Percentage of Test Data (%) Correctly Classified Instances (%) Kappa statistic Mean absolute error Root mean squared error Accuracy 90 10 100.00% 1 0.0054 0.0198 100.00% 80 20 100.00% 1 0.0087 0.0497 100.00% 70 30 100.00% 1 0.0108 0.058 100.00% 60 40 98.13% 0.9596 0.0197 0.1062 98.13% 50 50 96.50% 0.9256 0.0279 0.1391 96.50% 40 60 97.50% 0.948 0.0276 0.1348 97.50% 30 70 99.29% 0.9849 0.0166 0.0962 99.29% 20 80 98.75% 0.974 0.0221 0.0944 98.75% 10 90 97.78% 0.9536 0.0373 0.1331 97.78% Next, we went through different accuracies generated by each classification method along with different percentages of training data. The accuracies generated by Conjunctive Rule classifier demonstrated better accuracies all around when we use 50% of data as Training data. The accuracies generated by Decision Table classifier shows that it gives better accuracies around when we use 90% of data as Training data. The accuracies generated by J48 classifier show more accurate results when we use 60-90% of the data as training data. The accuracies generated by Naïve Bayes Classifier gives better accuracies when we use 50% of data as training data. The accuracies generated by Multilayer Perceptron classifier show us better accuracies when we use 60-90% of data as training data. We can further see that other the Naïve Bayes Classifier and Conjunctive Rule classifiers, all the other classifiers produce better accuracy when the percentage of trained data is high, and we can get better accuracy on data analysis by using percentage split model, as this better to use. Table 4: Average Accuracies Produce by Each Classifier Classification Method Average Accuracy at all % Splits Conjunctive Rule 91 % J48 97.52 % logistic regression 94.04 % Support Vector Machine (SVM) 97.02 % Decision Table 95.6 % Naïve Bayes Classifier 94.16 % Multilayer Perceptron 98.66 % Table 4 shows the average accuracy produces by each classifier on Percentage Split test method. The average accuracies generated through the Percentage Split test model can identify the highest accuracy of 98.66% received by Multilayer Perceptron. J48 and Support Vector Machine (SVM) have provided the best average accuracies 97.52% and 97.02% respectively after Multilayer Perceptron. Storage problems of Medical Practices need a technological infrastructure that keeps up with high demand. without delay, the data accumulated in bulks is inclined to be difficult to handle with out-of-date methods. A resolution to this rising complicated dilemma, is Data mining. This software provides the machinery and technique to alter everyday data into significant decision-making means. Data mining is a device that simply deciphers the issue of stacked up data; nonetheless it correspondingly turns it into a substantially important data that forecast motifs and trends established on intermittence of the data. The healthcare profession is frequently “information and document rich,” which physical conduct is impracticable in a busy health care practice. The mass amounts of data stored or archived needs data-mining to generate a viable solution to this growing issue. Moreover, investigating various techniques for Big Data Databases [13, 14], security [15] and prediction and pattern analysis [16] could be an interesting path to explore in the future. 4. Conclusion The study has found that the best classification methods used to analyze data in areas of bioinformatics and medical science is big data. We have used more complex chronic kidney diseases data sets with 25 attributes to analyze the performance of classification algorithms. We have used seven different classifiers namely to Support Vector Machine (SVM), logistic regression, J48, Decision Table, Naïve Bayes Classifier, Multilayer Perceptron and Conjunctive Rule, and we have also tested them in two test methods (Percentage Split and 10- fold cross validation) to generate outputs on each classifier. Based on the findings received on Percentage Split test method; we could see that most classifiers tend to have a better accuracy when the percentage of training data is higher. Hence, we further concluded that, to get better accuracies in the analysis related to medical data we have to use a higher percentage of training data to build the models irrespectively to the classifier used. Additionally, the highest accuracy of 98.66% is given by Multilayer Perceptron. J48 and Support Vector Machine (SVM) have provided the best average accuracies 97.52% and 97.02% respectively after Multilayer Perceptron in Percentage Split test method. In 10-fold cross-validation test method, we can see that the highest accuracy of 99.75% is given by Multilayer Perception and J48 and Decision Table give 99% respectively. Multilayer Perception gives us the lesser error rate compared to other classifiers. Accounting for the computational time, Multilayer Perception can be time-consuming comparatively, when it comes to deal with billions of data. Nonetheless, for the field of bioinformatics and medical science accuracy, the key objective is to deal with sensitive data and a single error which can lead to a disaster. Hence, our results clearly show that Multilayer Perception classification method is the most accurate and most suitable classification algorithm used in the field of bioinformatics and medical science for further data analysis and predictions. 5. References [1] S. Dhamodharan. Liver Disease Prediction Using Bayesian Classification, 4th National Conference on Advance Computing, Application Technologies, May 2014. [2] A. V. Solanki. Data Mining Techniques using WEKA Classification for Sickle Cell Disease, International Journal of Computer Science and Information Technology, 5(4): 5857-5860,2014. [3] J. Joshi, D. Rinal, J. Patel. Diagnosis and Prognosis of Breast Cancer Using Classification Rules, International Journal of Engineering Research and General Science, 2(6):315-323, October 2014. [4] K. Lakshmi, Y. Nagesh and V. M. Krishna. Performance comparison of three data mining techniques for predicting kidney disease survivability. International Journal of Advances in Engineering & Technology, Mar. 2014. [5] C. Anu and G. Puneet. Detecting and Diagnosing a Disease by Patient Monitoring System. International Journal of Mechanical Engineering and Information Technology. 2014. 2(6): 493-499. [6] C. Giovanni, B. Roberto, L. Roberto, O. Sandro, C. Carlo and N. Giorgio. Comparison Between an Artificial Neural Network and Logistic Regression in Predicting Long Term Kidney Transplantation Outcome. Chapter 5, An open access article distributed under the terms of the Creative Commons Attribu
tion License. [7] D. N. Tommaso, C. Vito, P. Francesco, B. Giulio, N. David, P. Francesco, D. S. Eugenio. An end stage kidney disease predictor based on an artificial neural networks ensemble, Elsevier Publication, Expert Systems with Applications. 2013. 40: 4438–4445. [8] J. Van, J. Ramon, F. Guiza, G. Meyfroidt, M. Bruynooghe, M. Berghe and U. Leuven. Data mining techniques for predicting acute kidney injury after elective cardiac surgery. Springer, 2012 [9] K. Koushal and Abhishek. Artificial Neural Networks for Diagnosis of Kidney Stones Disease. I. J. Information Technology and Computer Science. 2012, 7: 20-25. [10] J. Divya and G. Sumanlata. Predicting the Effect of Diabetes on Kidney using Classification in Tanagra. International Journal of Computer Science and Mobile Computing. 2014. 3(4). [11] M. Lichman. UCI Machine Learning Repository [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California, School of Information and Computer Science. (Accessed: 04- April- 2016). [12] R. Cristóbal, S. Ventura, P. G. Espejo, and C. Martínez, Handbook of Educational Data Mining, Data Mining, Taylor & Francis, 2010. [13] V. Vargas, A. Syed, A. Mohammad, and M. N. Halgamuge. Pentaho and Jaspersoft: A Comparative Study of Business Intelligence Open Source Tools Processing Big Data to Evaluate Performances. International Journal of Advanced Computer Science and Applications (IJACSA). 2016. 7(10): 20-29, November. [14] S. Kalid, A. Syed, A. Mohammad and M. N. Halgamuge. Big-Data NoSQL Databases: Comparison and Analysis of “Big-Table”, “DynamoDB”, and “Cassandra”. IEEE 2nd International Conference on Big Data Analysis (ICBDA’17), Beijing, China, 10-12 March 2017. [15] K. Kaur, A. Syed, A. Mohammad and M. N. Halgamuge. Review: An Evaluation of Major Threats in Cloud Computing Associated with Big Data. IEEE 2nd International Conference on Big Data Analysis (ICBDA’17), Beijing, China, 10-12 March 2017. [16] A. Gupta, A. Mohammad, A. Syed, and M. N. Halgamuge. A Comparative Study of Classification Algorithms using Data Mining: Crime and Accidents in Denver City the USA. International Journal of Advanced Computer Science and Applications (IJACSA). 2016. 7(7): 374 – 381.