New Combined Clustering Method for Artificial Datasets

Zahra Rezaei^*

Department of Computer Engineering, Nourabad Mamasani Branch, Islamic Azad University, Nourabad, Iran

Abstract

In the past decade many new methods were proposed for creating diverse classifiers due to combination. In this paper a new method for constructing an ensemble is proposed which uses clustering technique to generate perturbation in training datasets. Main presumption of this method is that the clustering algorithm used can find the natural groups of data in feature space. During testing, the classifiers whose votes are considered as being reliable are combined using majority voting. This method of combination outperforms the ensemble of all classifiers considerably on several real and artificial datasets.

Keywords

Diversity, Classifier Fusion, Clustering, Classifier Ensembles

Received: August 12, 2015
Accepted: August 23, 2015
Published online: September 7, 2015

@ 2015 The Authors. Published by American Institute of Science. This Open Access article is under the CC BY-NC license. http://creativecommons.org/licenses/by-nc/4.0/

1. Introduction

Nowadays, usage of recognition systems has addressed many applications in almost all fields. However, Most of classification algorithms have obtained good performance for specific problems; they lack enough robustness for other problems. Therefore, recent researches are directed to the combinational methods which have more power, robustness, resistance, accuracy and generality. Combination of Multiple Classifiers (CMC) can be considered as a general solution method for pattern recognition problems. Inputs of CMC are result of separate classifiers and its output is combination of their predictions [1] and [2].

We may see CMC under numerous names like hybrid methods, decision combination, multiple experts, and mixture of experts, classifier ensembles, cooperative agents, opinion pool, decision forest, classifier fusion, and combinational systems and so on. Combinational methods usually result in the improvement of classification, because classifiers with different features and methodologies can complete each other [4]-[6]. Kuncheva in [7, 35, 36, 37, 38] using Condorcet Jury theorem [8], has shown that combination of classifiers can usually operate better than single classifier. It means if more diverse classifiers are used in the ensemble, then error of them can considerably be reduced because classifiers with different features and methodologies can complete each other [4]-[6]. Different categorizations of combinational classifier systems are represented in [9]-[11]-[39-44]. Valentini and Masouli divide methods of combining classifiers into two categories: generative methods, nongenerative methods. In generative methods, a set of base classifiers is created by a set of base algorithms or by manipulating dataset. This is done in order to reinforce diversity of base classifiers [9], [10]. For a good coverage on combinational methods the reader is referred to [1], [7], and [12]-[16].

Theoretical and empirical works showed that a good ensemble is one where the individual classifiers have both accuracy and diversity. In other words, the individual classifiers make their errors on difference parts of the input space [16] and [17]. Many approaches have been proposed to construct such ensembles. One group of these methods obtains diverse individuals by training accurate classifiers on different training set, such as bagging, boosting, cross validation and using artificial training examples [17]-[20]-[45-50]. Another group of these methods adopts different topologies, initial weight setting, parameter setting and training algorithm to obtain individuals. For example, Rosen in [21] adjusted the training algorithm of the network by introducing a penalty term to encourage individual networks to be decorrelated. Liu and Yao in [22] used negative correlation learning to generate negatively correlated individual neural network. The third group is named selective approach group where the diverse components are selected from a number of trained accurate networks. For example, Opitz and Shavlik in [23] proposed a generic algorithm to search for a highly diverse set of accurate networks. Lazarevic and Obradoric in [24] proposed a pruning algorithm to eliminate redundant classifiers; Navone et al. in [25] proposed another selective algorithm based on bias/variance decomposition; GASEN proposed by Zhou et al. in [26] and PSO based approach proposed by Fu et al. in [27] also were introduced to select the ensemble components.

In general, an ensemble is built in two steps, that is, generating multiple base classifiers and then combining their predictions. According to the styles of training the base classifiers, current ensemble learning algorithms can be roughly categorized into two classes, that is, algorithms where component learners must be trained sequentially, and algorithms where component learners could be trained in parallel. The representative of the first category is AdaBoost [28], which sequentially generates a series of base classifiers where the training instances wrongly predicted by a base classifier will play more important role in the training of its subsequent classifier. The representative of the second category is Bagging [18], which generates many samples from the original training set via bootstrap sampling [29,50,87] and then trains a base classifier from each of these samples, whose predictions are combined via majority voting.

Research on classification systems is an open problem in the pattern recognition yet. There are many ways to improve the performance of classifiers. The new classification systems try to investigate errors and propose a solution to compensate them [30]. One of these approaches is combination of classifiers. Dietterich in [31,88,99] has proved that a combination of classifiers is usually better than a single classifier, by three kinds of reasoning: Statistical, computational and pictorial reasoning. However, there are many ways to combine classifiers; there is no proof to determine the best one [32]. One of the most important characteristics of combination of classifiers is diversity. We try to preserve the differences between classifiers. In this way, we can investigate more aspects of data.

In section 2 we will briefly overview combining classifier levels. We will try in section 3 to obtain diverse classifiers using manipulation of dataset labels. And finally section 4 is paper’s conclusion.

2. Combining Classifiers

In general, creation of combinational classifiers may be in four levels. It means combining of classifiers may happen in four levels. Figure 1 depicts these four levels. In level four, we try to create different subset of data in order to make independent classifiers. Bagging and boosting are examples of this method [18], [33]. In these examples, we use different subset of data instead of all data for training. In level three, we use subset of features for obtaining diversity in ensemble. In this method, each classifier is trained on different subset of features [32], [34]-[35]. In level two, we can use different kind of classifiers for creating the ensemble [32,100,129]. Finally, in the level one, method of combining (fusion) is considered.

Figure 1. Different levels of creation of classifier ensemble.

In the combining of classifiers, we aim to increase the performance of classification. There are several ways for combining classifiers. The simplest way is to find best classifier and use it as main classifier. This method is offline CMC. Another method that is named online CMC uses all classifier in ensemble, for example, by voting. We will show that combining method can improve the result of classification.

3. Proposed Method

3.1. Background

In this article, classification problem for a particular kind of dataset is argued. The goal is to break each class data into smaller subclasses such that error rate in each subclass is less than a threshold. It has been assumed that a class of data can include more than one cluster. For example in Farsi handwritten optical character recognition problem, digit 5 is written at least in two kinds of shape (2 clusters). This problem is shown in Figure 2.

In [36], it is shown that changing labels of classes can improve classification performance. So initial digit ‘5’ class is divided into two subclasses, digit ‘5’ type 1 and digit ‘5’ type 2, in order to ease classification goal of learning digit ‘5’ initial class complicated boundaries.

According to [7,13,152], if we have some really independent classifiers better than random classifiers, the simple ensemble (majority vote) of them can outperform their average performance in accuracy. Generally even if we increase the number of those independent classifiers, we can reach to any arbitrary accuracy, even 100%. But the problem restricting us for this goal is our incapability in obtaining those really independent classifiers.

Figure 2. Data of class ‘5’ and ‘0’; 5 is in left and 0 is in right.

It implies that making an ensemble of classifiers cannot surely always lead to generating diverse outputs by those classifiers; indeed their mistakes usually coincide with each other as well as their correct results. We are looking to find these really independent and approximately accurate (at least more accurate than random) classifiers with a method that will be examined in following section.

3.2. Proposed Algorithm

In proposed solution, according to error rate of each class, the class is divided into some subclasses in order to ease learning of decision boundaries by classifier. For a better understanding have a look at Figure 3.

This problem in dimension more than 2 will be probably more crucial. In this article the presumption is that a class is composed of more than one cluster which means that in a classification process with c classes, the number of real classes may be different from c.

Figure 3. A dataset with 3 class in wich class 1 contain 2 subclass.

In following, you can see the pseudo code of the proposed algorithm.

Algorithm1(original data set);

m(1: number_of_classes)=1;

         relabel training set using clusters;

         save_classifiers(c)=classifier;

end for

for i=1 to max_iteration

         out(i)=test(save_classifiers(i),test data);

end for

ensemble=majority_vote(out(1.. max_iteration));

accuracy=compute_accuracy(ensemble);

return accuracy,save_classifiers;

As you can see at the Figure, this method get dataset as input, and put it into three partitions: training set, test set and validation set. Here, the training set, test set and validation set contain 60%, 15% and 25% of entire dataset respectively. Then the data of each class is extracted from the original training dataset. Firstly we initial the number of cluster in each class to one. After that we repeat the following process as many as the predetermined number. This predetermined number is considered 10 here:

1. At first a classifier is trained on training data.

2. Using validation data, error rate of each class is approximated.

3. We increase the number of clusters in each class with error rate greater than a threshold, by one, and also then data of that class is clustered using fuzzy K-means [153-171]. If this clustering causes to creation of a sparse cluster, we will rollback the entire process of this section for that class. We decrease the number of clusters in that class, and then recluster those data with decreased number.

4. After that according to clustering in the previous section, the data are relabeled.

5. Finally the current classifier is added to the ensemble and this iteration is concluded.

After above loop, the outputs of all classifiers in the ensemble on test set are fused using majority vote mechanism, and the algorithm returns accuracy of ensemble and ensemble itself. All classifiers existing in the ensemble are support vector machine [172-191].

It can be said about time order of this algorithm that the method just multiplies a constant multiplicand in the time order of simple algorithm (training a simple classifier). Suppose that the time order of training a simple classifier on a dataset with n datapoints and c classes is O (f(n,c)), also assume that the time order of clustering on that dataset is O(g(n,c)) and also m to be the number of max_iteration. Then the time order of this method is Ω (m*f(n,c)+c*g(n/c,q))) where q is a number that in average and experimentally is less than c; provided that clustering is performed in each iteration. For simplicity assume that time order of clustering and training a classifier on a dataset are approximately the same. It is obvious that g(n,c) is not a linear function and g(n/c,q)<<g(n,c) where q<c. We also assumed that g(n,c)~f(n,c), then g(n/c,q)<<f(n,c). So we come to the conclusion that factor c*g(n/c,q) is negligible in compare to factor f(n,c). Consequently the time order of the method will be O(m*f(n,c)) which is worse than initial classifier time order just as little as a constant multiplicand. Of course this waste of time is completely tolerable against important achieved accuracy.

4. Conclusion

It was shown that the necessary diversity of an ensemble can be achieved by clustering data points of each multipart class. The method was explained above in detail and the result over real and non-real dataset prove the correctness of our claim. As it was mentioned before, this method is sensitive to shape of dataset. It cannot work well on those of datasets with very singular dense classes.

References

1. B. Minaei-Bidgoli, G. Kortemeyer and W.F. Punch, Optimizing Classification Ensembles via a Genetic Algorithm for a Web-based Educational System, (SSPR /SPR 2004), Lecture Notes in Computer Science (LNCS), Volume 3138, Springer-Verlag, ISBN: 3-540-22570-6, pp. 397-406, 2004.
2. Saberi., M. Vahidi , B. Minaei-Bidgoli, Learn to Detect Phishing Scams Using Learning and Ensemble Methods, IEEE/WIC/ACM International Conference on Intelligent Agent Technology, Workshops (IAT 07), pp. 311-314, Silicon Valley, USA, November 2-5, 2007.
3. T.G. Dietterich, Ensemble learning, in The Handbook of Brain Theory and Neural Networks, 2nd edition, M.A. Arbib, Ed. Cambridge, MA: MIT Press, 2002.
4. S. Gunter and H. Bunke, Creation of classifier ensembles for handwritten word recognition using feature selection algorithms, IWFHR 2002 on January 15, 2002.
5. B. Minaei-Bidgoli, G. Kortemeyer, W. F. Punch, Mining Feature Importance: Applying Evolutionary Algorithms within a Web-Based Educational System, Proc. of the Int. Conf. on Cybernetics and Information Technologies, Systems and Applications, CITSA 2004.
6. L. I. Kuncheva, Combining Pattern Classifiers, Methods and Algorithms, New York: Wiley, 2005.
7. L. Shapley and B. Grofman, Optimizing group judgmental accuracy in the presence of interdependencies, Public Choice, 43:329-343, 1984.
8. F. Roli and J. Kittler, editors. Proc. 2nd Int. Workshop on Multiple Classifier Systems (MCS 2001), Vol. 2096 of Lecture Notes in Computer Science LNCS Springer-Verlag, Cambridge, UK, 2001.
9. F. Roli and J. Kittler, editors. Proc. 3rd Int. Workshop on Multiple Classifier Systems (MCS 2002), Vol. 2364 of Lecture Notes in Computer Science LNCS Springer-Verlag, Cagliari, Italy, 2002.
10. L. Lam. Classifier combinations: implementations and theoretical issues. In J. Kittler and F. Roli, editors, Multiple Classifier Systems, Vol. 1857 of Lecture Notes in Computer Science, Cagliari, Italy, 2000, Springer, pp. 78-86.
11. T.G. Dietrich, Machine-learning research: four current direction, AI Magazine, 18, 4, winter 1997, 97-135.
12. A.K. Jain, R.P.W. Duin, J. Mao, Satanical pattern recognition: a review, IEEE Transaction on Pattern Analysis and Machine Intelligence, PAMI-22, 1, January 2000, 4-37.
13. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification, 2nd ed. John Wiley & Sons, NY, 2001.
14. J. C. Sharkey, editor, Combining Artificial Neural Nets. Ensemble and Modular Multi-Net Systems, Springer-Verlag, London, 1999.
15. L. K. Hansen, P. Salamon, Neural network ensembles. IEEE Transaction on Pattern Analysis and Machine Intelligence, 12(10):993-1001, 1990.
16. L. Breiman, Bagging predictors. Machine Learning, 24(2):123-140, 1996.
17. Krogh, J. Vedelsdy, Neural Network Ensembles Cross Validation, and Active Learning, In: G. Tesauro, D. Touretzky, T. Leen (Eds.), Advances in Neural Information Processing Systems, Volume 7. MIT Press, Cambridge, MA, p.231-238, 1995.
18. R.E. Schapire, The strength of weak learn ability, Machine Learning, 5(2):1971-227, 1990.
19. P. Melville, R. Mooney, Constructing Diverse Classifier Ensembles Using Artificial Training Examples, Proc. of the IJCAI-2003, Acapulco, Mexico, p.505-510, 2003.
20. B. E. Rosen, Ensemble learning using decorrelated neural network. Connection Science, 8(3-4):373-384, 1996.
21. RobertW. Floyd. Assigning meanings to programs. In Symposium on Applied Mathematics, pages 19–32. American Mathematical Society, 1967.
22. M. D. Ernst, J. Cockrell, W. G. Griswold, D. Notkin, Dynamically discovering likely program invariants to support program evolution, IEEE TSE 27 (2) (2007) 99–123
23. B. Weiß. Inferring invariants by static analysis in KeY. Diplomarbeit, University of Karlsruhe, March 2007
24. Neil D. Jones and Flemming Nielson. Abstract interpretation: A semanticsbased tool for program analysis. In S. Abramsky, D. M. Gabbay, and T. S. E. Maibaum, editors, Handbook of Logic in computer Science, volume 4, pages 527–636. Oxford University Press, 1995.
25. M. Boshernitsan, R. Doong, A. Savoia, From Daikon to Agitator: Lessons and challenges in building a commercial tool for developer testing, ISSTA (2006) 169–179.
26. S. Hangal, M. S. Lam, Tracking down software bugs using automatic anomaly detection, in: ICSE, 2002, pp. 291–301.
27. C. Csallner et al. DySy: Dynamic symbolic execution for invariant inference. In Proc. of ICSE, 2008.
28. Michael D. Ernst, Adam Czeisler, William G. Griswold, and David Notkin. Quickly detecting relevant program invariants. In ICSE, Limerick, Ireland, June 7-9, 2000.
29. Michael D. Ernst, William G. Griswold, Yoshio Kataoka, and David Notkin. "Dynamically Discovering Program Invariants Involving Collections", Technical Report, University of Washington, 2000.
30. S. Nadeem and S. Saleem, Theoretical Investigation of MHD Nanofluid Flow Over a Rotating Cone: An Optimal Solutions, Information Sciences Letters, 3(2), 55-62 (2014).
31. M. Zamoum, M. Kessal,Analysis of cavitating flow through a venture, Scientific Research and Essays, 10(11), 383-391 (2015).
32. H. Morad, GPS Talking For Blind People, Journal of Emerging Technologies in Web Intelligence, 2(3), 239-243 (2010).
33. D. Rawtani and Y. K. Agrawal, Study the Interaction of DNA with Halloysite Nanotube-Gold Nanoparticle Based Composite, Journal of Bionanoscience, 6, 95-98 (2012).
34. V. Karthick and K. Ramanathan, Investigation of Peramivir-Resistant R292K Mutation in A (H1N9) Influenza Virus by Molecular Dynamics Simulation Approach, Journal of Bioinformatics and Intelligent Control, 2, 29-33 (2013).
35. R. Uthayakumar and A. Gowrisankar, Generalized Fractal Dimensions in Image Thresholding Technique, Information Sciences Letters, 3(3), 125-134 (2014).
36. B. Ould Bilal, D. Nourou, C. M. F Kébé, V. Sambou, P. A. Ndiaye and M. Ndongo, Multi-objective optimization of hybrid PV/wind/diesel/battery systems for decentralized application by minimizing the levelized cost of energy and the CO2 emissions, International Journal of Physical Sciences, 10(5), 192-203 (2015).
37. A. Maqbool, H. U. Dar, M. Ahmad, G. N. Malik, G. Zaffar, S. A. Mir and M. A. Mir, Comparative performance of some bivoltine silkworm (Bombyx mori L.) genotypes during different seasons, Scientific Research and Essays, 10(12), 407-410 (2015).
38. R. B. Little, A theory of the relativistic fermionic spinrevorbital, International Journal of Physical Sciences, 10(1), 1-37 (2015).
39. Z. Chen, F. Wang and Li Zhu, The Effects of Hypoxia on Uptake of Positively Charged Nanoparticles by Tumor Cells, Journal of Bionanoscience, 7, 601-605 (2013).
40. A.Kaur and V. Gupta, A Survey on Sentiment Analysis and Opinion Mining Techniques, Journal of Emerging Technologies in Web Intelligence, 5(4), 367-371 (2013).
41. P. Saxena and M. Agarwal, Finite Element Analysis of Convective Flow through Porous Medium with Variable Suction, Information Sciences Letters, 3(3), 97-101 (2014).
42. J. G. Bruno, Electrophoretic Characterization of DNA Oligonucleotide-PAMAM Dendrimer Covalent and Noncovalent Conjugates, Journal of Bionanoscience, 9, 203-208 (2015).
43. K. K. Tanaeva, Yu. V. Dobryakova, and V. A. Dubynin, Maternal Behavior: A Novel Experimental Approach and Detailed Statistical Analysis, Journal of Neuroscience and Neuroengineering, 3, 52-61 (2014).
44. E. Zaitseva and M. Rusin, Healthcare System Representation and Estimation Based on Viewpoint of Reliability Analysis, Journal of Medical Imaging and Health Informatics, 2, 80-86 (2012).
45. R. Ahirwar, P. Devi and R. Gupta, Seasonal incidence of major insect-pests and their biocontrol agents of soybean crop (Glycine max L. Merrill), Scientific Research and Essays, 10(12), 402-406 (2015).
46. H. Boussak, H. Chemani and A. Serier, Characterization of porcelain tableware formulation containing bentonite clay, International Journal of Physical Sciences, 10(1), 38-45 (2015).
47. Q. Xiaohong, and Q. Xiaohui, an Evolutionary Particle Swarm Optimizer Based on Fractal Brownian Motion, Journal of Computational Intelligence and Electronic Systems, 1, 138 (2012).
48. G. Minhas and M. Kumar, LSI Based Relevance Computation for Topical Web Crawler, Journal of Emerging Technologies in Web Intelligence, 5(4), 401-406 (2013).
49. Y. Shang, Efficient strategies for attack via partial information in scale-free networks, Information Sciences Letters, 1(1), 1-5 (2012).
50. I. Rathore and J. C. Tarafdar, Perspectives of Biosynthesized Magnesium Nanoparticles in Foliar Application of Wheat Plant,Journal of Bionanoscience, 9, 209-214 (2015).
51. H. Yan and H. Hu, Research and Realization of ISIC-CDIO Teaching Experimental System Based on RFID Technology of Web of Things, Journal of Bionanoscience, 7, 696-702 (2013).
52. R. Teles, B. Barroso, A. Guimaraes and H. Macedo, Automatic Generation of Human-like Route Descriptions: A Corpus-driven Approach, Journal of Emerging Technologies in Web Intelligence, 5(4), 413-423 (2013).
53. E. S. Hui, Diffusion Magnetic Resonance Imaging of Ischemic Stroke, Journal of Neuroscience and Neuroengineering, 1, 48-53 (2012).
54. O. E. Emam, M. El-Araby and M. A. Belal, On Rough Multi-Level Linear Programming Problem, Information Sciences Letters, 4(1), 41-49 (2015).
55. B. Prasad, D.C. Dimri and L. Bora,Effect of pre-harvest foliar spray of calcium and potassium on fruit quality of Pear cv. Pathernakh, Scientific Research and Essays, 10(11), 392-396 (2015).
56. H. Parvin, H. Alinejad-Rokny and M. Asadi, An Ensemble Based Approach for Feature Selection, Australian Journal of Basic and Applied Sciences, 7(9), 33-43 (2011).
57. Fouladgar M.H., Minaei-Bidgoli B., Parvin H.: Enriching Dynamically Detected Invariants in the Case of Arrays. International Conference on Computational Science and Its Applications (ICCSA 2011), LNCS, ISSN: 0302-9743. LNCS. Springer, Heidelberg, pp. 622–632, 2011.
58. H. Parvin, H. Alinejad-Rokny, S. Parvin,Divide and Conquer Classification,Australian Journal of Basic & Applied Sciences, 5(12), 2446-2452 (2011).
59. H. Parvin, B. Minaei-Bidgoli, H. Alinejad-Rokny, A New Imbalanced Learning and Dictions Tree Method for Breast Cancer Diagnosis, Journal of Bionanoscience, 7(6), 673-678 (2013).
60. H. Parvin H., H. Alinejad-Rokny, M. Asadi, An Ensemble Based Approach for Feature Selection, Journal of Applied Sciences Research, 7(9), 33-43 (2011).
61. H. Parvin, H. Helmi, B. Minaie-Bidgoli, H. Alinejad-Rokny, H. Shirgahi, Linkage learning based on differences in local optimums of building blocks with one optima, International Journal of Physical Sciences, 6(14), 3419-3425 (2011).
62. H. Parvin, B. Minaei-Bidgoli, H. Alinejad-Rokny, S. Ghatei, An innovative combination of particle swarm optimization, learning automaton and great deluge algorithms for dynamic environments, International Journal of Physical Sciences, 6(22), 5121-5127 (2011).
63. H. Parvin, H. Alinejad-Rokny, S. Parvin,A Classifier Ensemble of Binary Classifier Ensembles, International Journal of Learning Management Systems, 1(2), 37-47 (2013).
64. H. Parvin, B. Minaei-Bidgoli, H. Alinejad-Rokny, W.F. Punch,Data weighing mechanisms for clustering ensembles, Computers & Electrical Engineering, 39(5): 1433-1450 (2013).
65. H. Parvin, H. Alinejad-Rokny, B. Minaei-Bidgoli, S. Parvin, A new classifier ensemble methodology based on subspace learning, Journal of Experimental & Theoretical Artificial Intelligence, 25(2), 227-250 (2013).
66. R. Agrawal, J. Gehrke, D. Gunopulos, P. Raghavan, Automatic Subspace Clustering of High Dimensional Data for Data Mining Applications, In Proceedings of the 1998 ACM SIGMOD international conference on Management of data, (1998) 94-105.
67. A. Blum, R. Rivest, Training a 3-node neural network is NP-complete, Neural Networks, 5 (1992) 117-127.
68. J.W. Chang, D.S. Jin, A new cell-based clustering method for large-high dimensional data in data mining applications, In Proceedings of the ACM symposium on Applied computing, (2002) 503-507.
69. S. Dudoit, J. Fridlyand, Bagging to improve the accuracy of a clustering procedure, Bioinformatics, 19(9) (2003) 1090-1099.
70. K. Faceli, C.P. Marcilio, D. Souto, Multi-objective Clustering Ensemble, Proceedings of the Sixth International Conference on Hybrid Intelligent Systems (HIS'06), (2006).
71. A.K. Jain, R.C. Dubes R.C, Algorithms for Clustering Data, Prentice Hall, (1988).
72. R. Kohavi R G. John, Wrappers for feature subset selection, Artificial Intelligence, 97(1-2) (1997) 273-324.
73. B. Liu, Y. Xia, P.S. Yu, Clustering through decision tree construction, In Proceedings of the ninth international conference on Information and knowledge management, (2000), 20-29.
74. R. Miller, Y. Yang, Association rules over interval data, In Proc. ACM SIGMOD International Conf. on Management of Data, (1997) 452-461.
75. A. Mirzaei, M. Rahmati, M. Ahmadi, A new method for hierarchical clustering combination, Intelligent Data Analysis, 12(6), (2008) 549-571.
76. C.B.D.J Newman, S. Hettich S, C. Merz, UCI repository of machine learning databases, http://www.ics.uci.edu/˜mlearn/MLSummary.html, (1998).
77. L. Parsons, E. Haque, H. Liu, Subspace clustering for high dimensional data: a review, ACM SIGKDD Explorations Newsletter, 6(1) (2004) 90-105.
78. C.M. Procopiuc, M. Jones, P.K. Agarwal P.K, T.M. Murali T.M, A Monte Carlo algorithm for fast projective clustering, In: Proceedings of the ACM SIGMOD conference on management of data, (2002) 418-427.
79. R. Srikant, R. Agrawal, Mining Quantitative Association Rules in Large Relational Tables, In Proc. of the ACM SIGMOD Conference on Management of Data, Montreal, Canada, (1996).
80. C.H. Cheng, A.W. Fu, Y. Zhang, Entropy-based subspace clustering for mining numerical data, In Proceedings of the fifth ACM SIGKDD international conference on Knowledge discovery and data mining, (1999) 84-93.
81. C. Domeniconi, M. Al-Razgan, Weighted cluster ensembles: Methods and analysis, TKDD, 2(4) (2009).
82. C. Domeniconi, D. Gunopulos, S. Ma, B. Yan, M. Al-Razgan, D. Papadopoulos, Locally adaptive metrics for clustering high dimensional data, Data Mining & Knowledge Discovery, 14(1) (2007) 63-97.
83. A. Strehl, J. Ghosh J, Cluster ensembles-a knowledge reuse framework for combining multiple partitions, Journal of Machine Learning Research, 3 (2002) 583-617.
84. J. Munkres, Algorithms for the Assignment and Transportation Problems, Journal of the Society for Industrial and Applied Mathematics, 5(1) (1957) 32-38.
85. Fred, A. and Jain, A. K. (2002). "Data Clustering Using Evidence Accumulation", Proc. of the 16th Intl. Conf. on Pattern Recognition, ICPR02, Quebec City, pp. 276 – 280.
86. A. Fred, "Finding Consistent Clusters in Data Partitions," Proc. Second Int’l Workshop Multiple Classifier Systems, J. Kittler and F. Roli, eds., pp. 309-318, 2001.
87. A. Fred and A.K. Jain, "Evidence Accumulation Clustering Based on the k-means Algorithm," Proc. Structural, Syntactic, and Statistical Pattern Recognition, Joint IAPR Int’l Workshops SSPR 2002 and SPR 2002, T. Caelli, et al., eds., pp. 442-451, 2002.
88. Fred A. and Jain A.K. (2005). Combining Multiple Clusterings Using Evidence Accumulation. IEEE Trans. on Pattern Analysis and Machine Intelligence, 27(6):835–850.
89. Fern X.Z. and Lin W. (2008), Cluster Ensemble Selection, SIAM International Conference on Data Mining, pp. 787-797.
90. M.H. Fouladgar, B. Minaei-Bidgoli, H. Parvin, H. Alinejad-Rokny, Extension in The Case of Arrays in Daikon like Tools, Advanced Engineering Technology and Application, 2(1), 5-10 (2013).
91. I. Jamnejad, H. Heidarzadegan, H. Parvin, H. Alinejad-Rokny, Localizing Program Bugs Based on Program Invariant, International Journal of Computing and Digital Systems, 3(2), 141-150 (2014).
92. H. Parvin, H. Alinejad-Rokny, S. Parvin, H. Shirgahi, A new Conditional Invariant Detection Dramework (CIDF), Scientific Research and Essays, 8(6), 265-273 (2013).
93. Chang, Y.H. and C.H. Yeh, 2001. Evaluating airline competitiveness using multi attribute decision making. Omega 29: 405-415.
94. Hu, Y.C. and J.F. Tsai, 2006. Backpropagation multi-layer perceptron for incomplete pairwise comparison matrices in analytic hierarchy process. Applied mathematics and computation 181(1): 53-62.
95. Hwang, C.L. and K. Yoon, 1981. Multiple attribute decision making.Springer-Verlag, Berlin Heidelberg New York.
96. Agalgaonkar, A.P., S.V. Kulkarni. and S.A. Khaparde, 2005. Multi-attribute decision making approach for strategic planning of DGs. Power Engineering Society General Meeting 3: 2985-2990.
97. Byun, H.S. and K.H. Lee, 2006. Determination of the optimal build direction for different rapid prototyping processes using multi criterion decision making. Robotics and Computer-Integrated Manufacturing 22: 69-80.
98. Kabassi, K. and M. Virvou, 2004. Personalised adult e-training on computer use based on multiple attribute decision making. Interacting with Computers 16: 115-132.
99. Yang, T., M.C. Chen. and C.C. Hung, 2007. Multiple attribute decision-making methods for the dynamic operator allocation problem. Mathematics and Computers in Simulation 73(5): 285-299.
100. Geoffrion, M., J.S. Dyer, A. Feinberg, 1972. An interactive approach for multi-criterion optimization, with an application to the operation of an academic department. Management Science 19(4): 357-368.
101. Stewart, T.J. 1992. A critical survey on the status of multiple criteria decision making theory and practice. Omega 20(5): 569-586.
102. Malakooti, B. and Y. Zhou, 1994. Feed-forward artificial neural networks for solving discrete multiple criteria decision making problems. Management Science 40(11): 1542-1561.
103. Sun, A., A. Stam and R.E. Steuer, 1996. Solving multiple objective programming problems using feed-forward artificial neural networks: the interactive FFANN procedure. Management Science 42(6): 835-849.
104. Wang, J. and B. A. Malakooti, 1992. Feed forward neural network for multiple criteria decision making. Computers & Operations Research 19(2): 151-167.
105. Li, Q. A. 2008. Fuzzy neural network based Multi-criteria decision making approach for outsourcing supplier evaluation. The 3^rd IEEE Conference on Industrial Electronics and Applications 1: 192-196.
106. Kong, F. and H. Liu, 2006. Fuzzy RBF neural network model for multiple attribute decision making. The 13th International Conference on Neural Information Processing Part III: 1046-1054.
107. N. Lalithamani and M. Sabrigiriraj , Dual Encryption Algorithm to Improve Security in Hand Vein and Palm Vein-Based Biometric Recognition, Journal of Medical Imaging and Health Informatics, 5, 545-551 (2015).
108. M. Khan and R. Jehangir,Fuzzy resolvability modulo fuzzy ideals, International Journal of Physical Sciences, 7(6), 953- 956 (2012).
109. M. Ravichandran and A.Shanmugam, Amalgamation of Opportunistic Subspace & Estimated Clustering on High Dimensional Data, Australian Journal of Basic and Applied Sciences, 8(3), 88-97, (2014).
110. M. Zhang, Optimization of Inter-network Bandwidth Resources for Large-Scale Data Transmission, Journal of Networks, 9(3), 689-694 (2014).
111. O. O. E. Ajibola, O. Ibidapo-Obe, and V. O. S. OIunloyo, A Model for the Management of Gait Syndrome in Huntington's Disease Patient, Journal of Bioinformatics and Intelligent Control, 3, 15-22 (2014).
112. L. Z. Pei, T. Wei, N. Lin and Z. Y. Cai, Electrochemical Sensing of Histidine Based on the Copper Germanate Nanowires Modified Electrode, Journal of Bionanoscience, 9, 161-165 (2015).
113. M. K. Elboree, Explicit Analytic Solution for the Nonlinear Evolution Equations using the Simplest Equation Method, Mathematical Sciences Letters, 3(1), 59-63 (2014).
114. R. Yousef and T. Almarabeh, An enhanced requirements elicitation framework based on business process models, Scientific Research and Essays, 10(7), 279-286 (2015).
115. K. Manimekalai and M.S. Vijaya, Taxonomic Classification of Plant Species Using Support Vector Machine, Journal of Bioinformatics and Intelligent Control, 3, 65-71 (2014).
116. S. Rajalaxmi and S. Nirmala, Automated Endo Fitting Curve for Initialization of Segmentation Based on Chan Vese Model, Journal of Medical Imaging and Health Informatics, 5, 572-580 (2015).
117. T. Mahmood and K. Hayat, Characterizations of Hemi-Rings by their Bipolar-Valued Fuzzy h-Ideals, Information Sciences Letters, 4(2), 51-59 (2015).
118. Agarwal and N. Mittal, Semantic Feature Clustering for Sentiment Analysis of English Reviews, IETE Journal of Research, 60(6), 414-422 (2014).
119. S. Radharani and M. L.Valarmathi, Content Based Watermarking Techniques using HSV and Fractal Dimension in Transform Domain, Australian Journal of Basic and Applied Sciences, 8(3), 112-119 (2014).
120. H. W. and W. Wang, an Improved Artificial Bee Colony Algorithm and Its Application on Production Scheduling, Journal of Bioinformatics and Intelligent Control, 3, 153-159 (2014).
121. L. Gupta,Effect of orientation of lunar apse on earthquakes, International Journal of Physical Sciences, 7(6), 974-981 (2012).
122. S. Iftikhar, F. Ahmad and K. Fatima, A Semantic Methodology for Customized Healthcare Information Provision, Information Sciences Letters, 1(1), 49-59 (2012).
123. P. D. Sia, Analytical Nano-Modelling for Neuroscience and Cognitive Science, Journal of Bioinformatics and Intelligent Control, 3, 268-272 (2014).
124. C. Guler, Production of particleboards from licorice (Glycyrrhiza glabra) and European black pine (Pinus Nigra Arnold) wood particles, Scientific Research and Essays, 10(7), 273-278 (2015).
125. Z. Chen and J. Hu, Learning Algorithm of Neural Networks on Spherical Cap, Journal of Networks, 10(3), 152-158 (2015).
126. W. Lu, Parameters of Network Traffic Prediction Model Jointly Optimized by Genetic Algorithm, Journal of Networks, 9(3), 695-702 (2014).
127. K. Boubaker,An attempt to solve neutron transport equation inside supercritical water nuclear reactors using the Boubaker Polynomials Expansion Scheme, International Journal of Physical Sciences, 7(19), 2730-2734 (2012).
128. K. Abd-Rabou, Fixed Point Results in G-Metric Space, Mathematical Sciences Letters, 3(3), 141-146 (2014).
129. Binu and M. Selvi, BFC: Bat Algorithm Based Fuzzy Classifier for Medical Data Classification, Journal of Medical Imaging and Health Informatics, 5, 599-606 (2015).
130. C. Kamath, Analysis of Electroencephalogram Background Activity in Epileptic Patients and Healthy Subjects Using Dispersion Entropy, Journal of Neuroscience and Neuroengineering, 3, 101-110 (2014).
131. G. Kaur and E. M. Bharti, Securing Multimedia on Hybrid Architecture with Extended Role-Based Access Control, Journal of Bioinformatics and Intelligent Control, 3, 229-233 (2014).
132. M. Ramalingam and D. Rana, Impact of Nanotechnology in Induced Pluripotent Stem Cells-driven Tissue Engineering and Regenerative Medicine, Journal of Bionanoscience, 9, 13-21 (2015).
133. S. Downes, New Technology Supporting Informal Learning, Journal of Emerging Technologies in Web Intelligence, 2(1), 27-33 (2010).
134. R. Periyasamy, T. K. Gandhi, S. R. Das, A. C. Ammini and S. Anand, A Screening Computational Tool for Detection of Diabetic Neuropathy and Non-Neuropathy in Type-2 Diabetes Subjects, Journal of Medical Imaging and Health Informatics, 2, 222-229 (2012).
135. Y. Qin, F. Wang and C. Zhou, A Distributed UWB-based Localization System in Underground Mines, Journal of Networks, 10(3), 134-140 (2015).
136. P. Saxena and C. Ghosh, A review of assessment of benzene, toluene, ethylbenzene and xylene (BTEX) concentration in urban atmosphere of Delhi, International Journal of Physical Sciences, 7(6), 850-860 (2012).
137. J. Hu, Z. Zhou and M. Teng, The Spatiotemporal Variation of Ecological Risk in the Lijiang River Basin Based on Land Use Change, Journal of Bionanoscience, 9, 153-160 (2015).
138. N. Saleem, M. Ahmad, S. A. Wani, R. Vashnavi and Z. A. Dar, Genotype-environment interaction and stability analysis in Wheat (Triticum aestivum L.) for protein and gluten contents, Scientific Research and Essays, 10(7), 260-265 (2015).
139. R. A. Rashwan and S. M. Saleh, A Coupled Fixed Point Theorem for Three Pairs of w-Compatible Mappings in G-metric spaces, Mathematical Sciences Letters, 3(1), 17-20 (2014).
140. S. P. Singh and B. K. Konwar, Carbon Nanotube Assisted Drug Delivery of the Anti-Malarial Drug Artemesinin and Its Derivatives-A Theoretical Nanotechnology Approach, Journal of Bionanoscience, 7, 630-636 (2013).
141. R. Dinasarapu and S. Gupta, Biological Data Integration and Dissemination on Semantic Web-A Perspective, Journal of Bioinformatics and Intelligent Control, 3, 273-277 (2014).
142. W. Qiaonong, X. Shuang, and W. Suiren, Sparse Regularized Biomedical Image Deconvolution Method Based on Dictionary Learning, Journal of Bionanoscience, 9, 145-152 (2015).
143. C. Prema and D. Manimegalai, Adaptive Color Image Steganography Using Intra Color Pixel Value Differencing, Australian Journal of Basic and Applied Sciences, 8(3), 161-167 (2014).
144. R. Adollah, M. Y. Mashor, H. Rosline, and N. H. Harun, Multilevel Thresholding as a Simple Segmentation Technique in Acute Leukemia Images, Journal of Medical Imaging and Health Informatics, 2, 285-288 (2012).
145. H. Uppili, Proton-Induced Synaptic Transistors: Simulation and Experimental Study, Journal of Neuroscience and Neuroengineering, 3, 117-129 (2014).
146. Chen, J. and S. Lin, 2003. An interactive neural network-based approach for solving multiple criteria decision-making problems. Decision Support Systems 36: 137-146.
147. Chen, J. and S. A. Lin, 2004. Neural network approach-decision neural network (DNN) for preference assessment. IEEE Transactions on systems, Man, and Cybernetics-Part C: Applications and reviews 34: 219-225.
148. H. B. Kekre and T. K. Sarode, Vector Quantized Codebook Optimization Using Modified Genetic Algorithm, IETE Journal of Research, 56(5), 257-264 (2010).
149. M. Gera, R. Kumar, V. K. Jain, Fabrication of a Pocket Friendly, Reusable Water Purifier Using Silver Nano Embedded Porous Concrete Pebbles Based on Green Technology, Journal of Bionanoscience, 8, 10-15 (2014).
150. M. S. Kumar and S. N. Devi, Sparse Code Shrinkage Based ECG De-Noising in Empirical Mode Decomposition Domain, Journal of Medical Imaging and Health Informatics, 5, 1053-1058 (2015).
151. C. Zhou, Y. Li, Q. Zhang and B. Wang, An Improved Genetic Algorithm for DNA Motif Discovery with Gibbs Sampling Algorithm, Journal of Bionanoscience, 8, 219-225 (2014).
152. R. Bhadada and K. L. Sharma, Evaluation and Analysis of Buffer Requirements for Streamed Video Data in Video on Demand Applications, IETE Journal of Research, 56(5), 242-248 (2010).
153. M. Kurhekar and U. Deshpande, Deterministic Modeling of Biological Systems with Geometry with an Application to Modeling of Intestinal Crypts, Journal of Medical Imaging and Health Informatics, 5, 1116-1120 (2015).
154. S. Prabhadevi and Dr. A.M. Natarajan, A Comparative Study on Digital Signatures Based on Elliptic Curves in High Speed Ad Hoc Networks, Australian Journal of Basic and Applied Sciences, 8(2), 1-6 (2014).
155. X. Jin and Y. Wang, Research on Social Network Structure and Public Opinions Dissemination of Micro-blog Based on Complex Network Analysis, Journal of Networks, 8(7), 1543-1550 (2013).
156. O. G. Avrunin, M. Alkhorayef, H. F. I. Saied, and M. Y. Tymkovych, The Surgical Navigation System with Optical Position Determination Technology and Sources of Errors, Journal of Medical Imaging and Health Informatics, 5, 689-696 (2015).
157. R. Zhang, Y. Bai, C. Wang and W. Ma, Surfactant-Dispersed Multi-Walled Carbon Nanotubes: Interaction and Antibacterial Activity, Journal of Bionanoscience, 8, 176-182 (2014).
158. B. K. Singh, Generalized Semi-bent and Partially Bent Boolean Functions, Mathematical Sciences Letters, 3(1), 21-29 (2014).
159. S. K. Singla and V. Singh, Design of a Microcontroller Based Temperature and Humidity Controller for Infant Incubator, Journal of Medical Imaging and Health Informatics, 5, 704-708 (2015).
160. N. Barnthip and A. Muakngam, Preparation of Cellulose Acetate Nanofibers Containing Centella Asiatica Extract by Electrospinning Process as the Prototype of Wound-Healing Materials,Journal of Bionanoscience, 8, 313-318 (2014).
161. R. Jac Fredo, G. Kavitha and S. Ramakrishnan, Segmentation and Analysis of Corpus Callosum in Autistic MR Brain Images Using Reaction Diffusion Level Sets, Journal of Medical Imaging and Health Informatics, 5, 737-741 (2015).
162. Wang, B. Zhu, An Improved Algorithm of the Node Localization in Ad Hoc Network, Journal of Networks, 9(3), 549-557 (2014).
163. T. Buvaneswari and A. A. Iruthayaraj, Secure Discovery Scheme and Minimum Span Verification of Neighbor Locations in Mobile Ad-hoc Networks, Australian Journal of Basic and Applied Sciences, 8(2), 30-36 (2014).
164. H. Parvin, H. Alinejad-Rokny, N. Seyedaghaee, S. Parvin,A Heuristic Scalable Classifier Ensemble of Binary Classifier Ensembles, Journal of Bioinformatics and Intelligent Control, 1(2), 163-170 (2013).
165. M.H. Fouladgar, B. Minaei-Bidgoli, H. Parvin, H. Alinejad-Rokny, Extension in The Case of Arrays in Daikon like Tools, Advanced Engineering Technology and Application, 2(1), 5-10 (2013).
166. H. Parvin, M. MirnabiBaboli, H. Alinejad-Rokny, Proposing a Classifier Ensemble Framework Based on Classifier Selection and Decision Tree, Engineering Applications of Artificial Intelligence, 37, 34-42 (2015).
167. Y. Zhang, Z. Wang and Z. Hu, Nonlinear Electroencephalogram Analysis of Neural Mass Model, Journal of Medical Imaging and Health Informatics, 5, 783-788 (2015).
168. S. Panwar and N. Nain, A Novel Segmentation Methodology for Cursive Handwritten Documents, IETE Journal of Research, 60(6), 432-439 (2014).
169. H. Mao, On Applications of Matroids in Class-oriented Concept Lattices, Mathematical Sciences Letters, 3(1), 35-41 (2014).
170. D. Kumar, K. Singh, V. Verma and H. S. Bhatti, Synthesis and Characterization of Carbon Quantum Dots from Orange Juice, Journal of Bionanoscience, 8, 274-279 (2014).
171. V. Kumutha and S. Palaniammal, Enhanced Validity for Fuzzy Clustering Using Microarray data, Australian Journal of Basic and Applied Sciences, 8(3), 7-15 (2014).
172. Y. Wang, C. Yang and J. Yu, Visualization Study on Cardiac Mapping: Analysis of Isopotential Map and Isochron Map, Journal of Medical Imaging and Health Informatics, 5, 814-818 (2015).
173. R. Su, Identification Method of Sports Throwing Force Based on Fuzzy Neural Network, Journal of Networks, 8(7), 1574-1581 (2013).
174. Matsuda, S. A. 2005. Neural network model for the decision-making process based on AHP. Proceedings of International Joint Conference on Neural Networks, Montreal, Canada.
175. Kohonen, T. 1987. Self-Organizing and associative memory, 2end edition. Berlin:Springer-Verlag.
176. Haykin S. 1999. Neural networks: a comprehensive foundation. prentice hall.
177. Milan, J. and R. Aura, 2002. An application of the multiple criteria decision making analysis to the selection of a new hub airport. EJTIR 2(2): 113-141.
178. Y. Liu, X. Yao, Evolutionary ensembles with negative correlation learning, IEEE Trans. Evolutionary Computation, 4(4):380-387, 2000.
179. D. Opitz, J. Shavlik, Actively searching for an effective neural network ensemble, Connection Science, 8(3-4):337-353, 1996.
180. Lazarevic, Z. Obradovic, Effective pruning of neural network classifier ensembles. Proc. International Joint Conference on Neural Networks, 2:796-801, 2001.
181. H. D. Navone, P. F. Verdes, P. M. Granitto, H. A. Ceccatto, Selecting Diverse Members of Neural Network Ensembles, Proc. 16th Brazilian Symposium on Neural Networks, p.255-260, 2000.
182. Z. H. Zhou, J. X. Wu, Y. Jiang, S. F. Chen, Genetic algorithm based selective neural network ensemble, Proc. 17th International Joint Conference on Artificial Intelligence, 2:797-802, 2001.
183. Q. Fu, S. X. Hu, S. Y. Zhao, A PSO-based approach for neural network ensemble, Journal of Zhejiang University (Engineering Science), 38(12):1596-1600, 2004, (in Chinese).
184. Y. Freund, R.E. Schapire, A decision-theoretic generalization of online learning and an application to boosting, in Proceedings of the 2nd European Conference on Computational Learning Theory, Barcelona, Spain, pp.23–37, 1995.
185. B. Efron, R. Tibshirani, An Introduction to the Bootstrap, New York: Chapman & Hall, 1993.
186. V. Dobra, Scalable Classification And Regression Tree Construction, Ph.D. Dissertation, Cornell University, Ithaca, NY, 2003.
187. T. G. Dietterich, Ensemble methods in machine learning. In J. Kittler and F. Roli, editors, Multiple Classifier Systems, volume 1857 of Lecture Notes in Computer Science, Springer, pp. 1–15, Cagliari, Italy, 2000.
188. F. Roli, G. Giacinto, G. Vernazza. Methods for designing multiple classifier systems. In J. Kittler and F. Roli, editors, Proc. 2nd International Workshop on Multiple Classifier Systems, Vol. 2096 of Lecture Notes in Computer Science, Springer- Verlag, pp. 78–87, Cambridge, UK, 2001.
189. S. Dudoit, J. Fridlyand, Bagging to improve the accuracy of a clustering procedure. Bioinformatics, 19 (9), pp. 1090-1099, 2003.
190. L.I. Kuncheva, L.C. Jain, Designing Classifier Fusion Systems by Genetic Algorithms. IEEE Transaction on Evolutionary Computation, Vol. 33, 351-373, 2000.
191. Strehl, J. Ghosh, Cluster ensembles a knowledge reuse framework for combining multiple partitions. Journal on Machine Learning Research, pp. 583-617, 2002.