A New Method for Software Testing

Hani Fouladgar^*

Department of Computer Engineering, Iran University of Science and Technology, Tehran, Iran

Abstract

Software engineering is the work of designing, implementing and modifying of software to build software fast and have a high quality, efficient and maintainable software. Invariant helps programmer and tester to perform some aspect of software engineering. Since arrays and pointers are more probable to be faulty, invariants which report these properties are more useful. By presenting first and last elements of arrays we can detect the carelessness in using index which mostly happens in loops. The idea of employing number of mutual elements between two same type arrays can help us to catch faults in the cases that an array is gained from changes in another array. These two properties help us to capture lots of prevalent faults and therefore try to remove them. This paper proposes an interesting sort of extension over Daikon like tools. It can generate more invariant in the case of array.

Keywords

Dynamic Invariant Detection, Software Testing, Array Property, Array’s First and Last Elements, Mutual Element Between Arrays

Received: June 16, 2015
Accepted: June 28, 2015
Published online: August 17, 2015

@ 2015 The Authors. Published by American Institute of Science. This Open Access article is under the CC BY-NC license. http://creativecommons.org/licenses/by-nc/4.0/

1. Introduction

Invariants are some properties of different program points which are true in all executions of the program. These properties can be seen in formal specification or assert statement. For example in a sort program that its job is to sort array of integers, invariant (array a sorted >=) is reported. This idea is introduced in [1,10,11,13,15]. Invariants develop data structures and algorithms and are employed in all aspects of software engineering from design to maintenance [2,12,14,16,17,19,34]. Invariants represent program properties; hence after an update to the code, invariants can show properties which remain unchanged or those that are violated through code revision. Invariants, somehow, are treated such as documentation and specification of a program. Specification is used in all software engineering steps like design, coding, verification, testing, optimization and maintenance. Invariants might be detected by static and dynamic approaches.

Static analysis checks syntactic structure and runtime behavior of program without actually running [3,21,22]. Static analysis is an entirely automated method. Data-flow analysis is a traditional technique which is employed in the compilers in order to collect necessary information for optimization. Data-flow analysis can determine the properties of program points. In practice the collected information is considered invariant. Abstract interpretation is a theoretical framework for static analysis [4,18,19,35,68]. The most precise imaginable abstract interpretation is called the static semantics or accumulating semantics. Another related static analysis technique is predicate abstraction. In this technique, an absolute and finite set of predicates is used to induce abstract domains.

Dynamic invariant extraction from program context is brought to software engineering realm during recent ten years. In contrast to static analysis, dynamic invariant detection extracts program properties and information by executing of the program with different inputs. Dynamic invariant detection, first time, was quoted by Daikon [2,20,21,69,87] - a full-featured and robust implementation of dynamic invariant detection. With the help of test suits and different executions of a program as well as using an invariant inference system, properties of specific point of program (usually function entries and exits) are revealed. These points, lexically, are named program points. The properties are not certainly true, but indeed, are determined through several executions on test cases with specific confidence. An important attitude of process of dynamic invariant extraction is that what invariant presents is not only program behavior but also indicates assumption of test cases. This nature of invariant causes double usage of it, so dynamically-produced invariant determines program specification more properly.

This paper concentrates on the dynamic extraction of invariants. We introduce some ideas which improve the effect of invariants that could be useful in bug detection and testing. We propose two extensions to the Daikon like tool for invariant detection. The extensions are concerned with the treatment of arrays. We employ more properties of arrays to get better results. The rest of this paper starts with related work (section 2) and continues with paper terminology (section 3) and contributions (section 4). Then we bring some simple examples to clarify the ideas (section 5). In section 6 some actual examples and adjustment for our ideas are presented. We evaluate our idea in section 7. Finally we conclude the paper and talk about future work (section 8).

2. Related Work

In this section, we discuss some implementations of dynamic invariant detection. Many valuable efforts have been done in this field but we mention here only these ones which are more relevant.

Dynamic invariant detection, as mentioned, is quoted by Daikon [2]. Daikon is the most prosperous software in dynamic invariant detection developed until now, comparing with other dynamic invariant detection methods [2,125,143]. However this software has some problems out of which the most serious one is being time-consuming.

DySy proposes a dynamic symbolic execution technique to improve the quality of inferred invariant [7,144,157]. It executes test cases like other dynamic invariant inference tools but, as well, coincidentally performs a symbolic execution. For each test unit, DySy results in program's path conditions. At the end, all path conditions are combined and build the result.

Software Agitator is a commercial testing tool which is represented by Agitar and is inspired by Daikon [5]. Software agitation is a testing technique that joins the results of research in test-input generation and dynamic invariant detection. The results are called observations. Agitar won the Wall street Journal's 2005 Software Technology Innovation Award.

The DIDUCE tool [6,87,112] helps the programmer by detecting errors and determining the root causes. Besides detecting dynamic invariant, DIDUCE checks the program behavior against extracted invariants up to each program points and reports all detected violations. DIDUCE checks simple invariants and does not need up-front instrument.

3. Terminology

In this section we bring some terms which are used frequently in this paper. The aim of the section is to help readers obtain a better perception of the paper.

Definition 1. Invariants could be defined as prominent relation among program variables. Invariants in programs are formulas or rules that are emerged from the source code of a program and remain unique and unchanged with respect to the running phase of a program with different parameters.

Definition 2. Program points are specific points in a program, such as the Enter or Exit point of a function, which serve as report points for variable relations and invariants. Most frequent program points in use are the Enter and Exit points of sub-programs and functions.

Definition 3. Pre-conditions of a program point are the conditions, relations and invariants that hold immediately before approaching to that program point. In the case of sub-programs or a function Enter point of a sub-program or a function acts as its pre-condition.

Definition 4. Post-conditions of a program point are the conditions, relations and invariants that hold immediately after leaving from that program point. In the case of sub-programs, a function Exit point of a sub-program or a function is considered as its post-condition of it.  Typically, post-condition also contains relations between the original value of a variable and its modified one (before and after that program point). In other words, invariants in post-conditions contain relations between variables in pre-condition and post-condition.

4. Paper Contributions

Software testing is one of the most time consuming parts of software engineering because regarding inputs, different executing paths happen and unchecked paths can be defective. Even if a software tester checks all paths, the code can be faulty. In this situation, because of their structure, arrays and pointers are more probable to be faulty. In the C program language, an array is a kind of pointer. Therefore if any improvement is achieved for the handling of arrays, it can be simultaneously considered as an improvement for the pointers. Another source of fault points can be the exceptions or the try-catch blocks in our case. Exceptions are handled by the catch block and this prevents the programmer or the tester form detecting the faults. This means that a program might operate as expected but it actually has some faults.

The first and the last elements of an array possess very crucial properties because these elements are impacted by the carelessness in using the indexes. By involving some array elements in invariant detection, a dramatic improvement in fault detection might happen. The number of these elements can be the least size of an array or they can be optional. This contribution exposes inattention in using index which mostly happens with the first and the last indexes and corresponding to the first and last elements of an array.

Besides employing array elements, enlisting the number of mutual elements of same type arrays for each program point is useful in detecting faults. In other words, for each program point, the number of elements' values which are shared in two different same type arrays is employed in invariants detection. It helps the programmer to evaluate his program in the cases that an array is gained from changes in another array. The mutual elements show the correct elements which should be unchanged through the process. We discuss more about this contribution in the next sections and clarify the number of mutual elements of same type arrays for each program point.

Overall our contributions comprise the following:

Ÿ Adding the number of first and last elements of an array as new variables for invariant detection.

Ÿ Enlisting the number of mutual elements of same type arrays for each program point

5. Illustration of Contributions

To clarify the contributions we discussed earlier, we present some pieces of program code and their post-condition invariant. The programs are in the form of pseudo-code and do not assume the use of any specific programming. For presenting our ideas, the Exit program point invariants which represent post-condition properties for a program point are used because post-condition properties can show both the pre-condition and post-condition values of variables.

For invariant detection, we might propose the use of the number of first and last elements of an array. The number of these elements can be the least size of the array or can be optional. This contribution exposes carelessness in using index which mostly happens to first and last indexes and corresponding to the first and last elements of array. To illustrate the effeteness of this view, consider Fig.1.

Fig. 1. Program A: Inattention in using index.

Fig. 1 presents the code of bubbleSort () function. It accepts 2 as input one of which is the array and another is the length of the array. The output is the sorted array. In this example, the index j starts at 1 instead of 0 so the first element of array is not considered in the sorting. With the contribution of the produced invariants from the first and last elements of the array the fault is detected. Adding the first and last elements of each array, as part of the related invariants in the Exit point of the bubbleSort () function is illustrated in Fig.1.

The presented invariants in Fig.1 are in the form of Daikon output. For array x, x[1] is the last element of x, x[2] the element before the last one and so forth. In Fig.1, line 17 shows that the first element of the input array always equals to the first element of the return value. Lines 18 to 23 show that the rest of the elements are sorted. Therefore obviously only the first element is never involved in sorting. This helps the programmer to detect the fault.

6. Evaluation

As previously quoted we propose two extensions to Daikon like tools. Beside there are arrays invariant, we consider these two ideas as crucial properties which can be raised as invariants. Therefore, this section discusses the matter in order to clarify and evaluate our proposed algorithm. To perform this job, we present two analyses. Fist we compare the running-time and the time-order of the original Daikon with the modified one. Then we use relevance [8] to measure the quality of the produced invariants.

The running times of the proposed modified Daikon and the original one in terms of millisecond is shown in the Fig.11. As it is understood from Fig.1 the time-order of both modified and original versions of Daikon are linear. It is also inferred that the original Daikon, as expected, does not excel at running-time compared to the modified Daikon. The higher number of variables, the higher slope in terms of time order. However, an increase in the number of variables does not change the time order in terms of the numbers of data trace-files [2].

From another perspective, the output of the modified Daikon over some typical programs is summarized in Table.1. As we discussed in section 6 we involve basic and small subprograms. We believe that every program, either big or small, has small parts and might be raised in small subprograms. These subprograms include arrays as their variables and effectively present the effect of the ideas.

Table 1. Efficacy of modified Daikon in some case studies

Rows in table 1 are representative of different sub-programs we discussed in previous sections and columns are number of different sorts of invariants. All the inferred invariants are not proper. In table 1 we proposed the number of implied and irrelevant invariant. For example if two invariants x != 0 and x in [7..13] are determined to be true, there is no sense to report both because the latter implies the former.

7. Conclusion

As mentioned before, invariants attracted significant attention in software engineering in recent years. In this paper, different properties of a program code are checked at different program points. More useful properties lead to receive more valuable invariants and thereby lead to infer more prevalent faults. Hence we propose two properties which dramatically change positively the effect of invariants in the case of arrays. Since arrays and pointers are objects which are more probable to be faulty we add two useful properties to other invariants. As most of fault operating happens in the first and last elements of arrays we enhance the effect of fault detecting by employing these elements as some properties of the array. Another property which prepares a good condition to gain more useful invariants is the mutual element for same type arrays. As mentioned earlier, this property is helpful when in a program point an array is returned after changing elements in another array.

Although some ideas about arrays are valid in the case of pointers, some others inherently differ. For future work, the pointers can be dealt with in more details.

References

1. RobertW. Floyd. Assigning meanings to programs. In Symposium on Applied Mathematics, pages 19–32. American Mathematical Society, 1967.
2. M. D. Ernst, J. Cockrell, W. G. Griswold, D. Notkin, Dynamically discovering likely program invariants to support program evolution, IEEE TSE 27 (2) (2007) 99–123
3. B. Weiß. Inferring invariants by static analysis in KeY. Diplomarbeit, University of Karlsruhe, March 2007
4. Neil D. Jones and Flemming Nielson. Abstract interpretation: A semanticsbased tool for program analysis. In S. Abramsky, D. M. Gabbay, and T. S. E. Maibaum, editors, Handbook of Logic in computer Science, volume 4, pages 527–636. Oxford University Press, 1995.
5. M. Boshernitsan, R. Doong, A. Savoia, From Daikon to Agitator: Lessons and challenges in building a commercial tool for developer testing, ISSTA (2006) 169–179.
6. S. Hangal, M. S. Lam, Tracking down software bugs using automatic anomaly detection, in: ICSE, 2002, pp. 291–301.
7. C. Csallner et al. DySy: Dynamic symbolic execution for invariant inference. In Proc. of ICSE, 2008.
8. Michael D. Ernst, Adam Czeisler, William G. Griswold, and David Notkin. Quickly detecting relevant program invariants. In ICSE, Limerick, Ireland, June 7-9, 2000.
9. Michael D. Ernst, William G. Griswold, Yoshio Kataoka, and David Notkin. "Dynamically Discovering Program Invariants Involving Collections", Technical Report, University of Washington, 2000.
10. S. Nadeem and S. Saleem, Theoretical Investigation of MHD Nanofluid Flow Over a Rotating Cone: An Optimal Solutions, Information Sciences Letters, 3(2), 55-62 (2014).
11. M. Zamoum, M. Kessal, Analysis of cavitating flow through a venture,Scientific Research and Essays,10(11), 383-391 (2015).
12. H. Morad, GPS Talking For Blind People, Journal of Emerging Technologies in Web Intelligence, 2(3), 239-243 (2010).
13. D. Rawtani and Y. K. Agrawal, Study the Interaction of DNA with Halloysite Nanotube-Gold Nanoparticle Based Composite, Journal of Bionanoscience, 6, 95-98 (2012).
14. V. Karthick and K. Ramanathan, Investigation of Peramivir-Resistant R292K Mutation in A (H1N9) Influenza Virus by Molecular DynamicsSimulation Approach, Journal of Bioinformatics and Intelligent Control, 2, 29-33 (2013).
15. R. Uthayakumar and A. Gowrisankar, Generalized Fractal Dimensions in Image Thresholding Technique, Information Sciences Letters, 3(3), 125-134 (2014).
16. B. Ould Bilal, D. Nourou, C. M. F Kébé, V. Sambou, P. A. Ndiaye and M. Ndongo, Multi-objective optimization of hybrid PV/wind/diesel/battery systems for decentralized application by minimizing the levelized cost of energy and the CO2 emissions,International Journal of Physical Sciences,10(5), 192-203 (2015).
17. A. Maqbool, H. U. Dar, M. Ahmad, G. N. Malik, G. Zaffar, S. A. Mir and M. A. Mir, Comparative performance of some bivoltine silkworm (Bombyx mori L.) genotypes during different seasons, Scientific Research and Essays,10(12), 407-410 (2015).
18. R. B. Little, A theory of the relativistic fermionic spinrevorbital, International Journal of Physical Sciences,10(1), 1-37 (2015).
19. Z. Chen, F. Wang and Li Zhu, The Effects of Hypoxia on Uptake of Positively Charged Nanoparticles by Tumor Cells, Journal of Bionanoscience, 7, 601-605 (2013).
20. A.Kaur and V. Gupta, A Survey on Sentiment Analysis and Opinion Mining Techniques, Journal of Emerging Technologies in Web Intelligence, 5(4), 367-371 (2013).
21. P. Saxena and M. Agarwal, Finite Element Analysis of Convective Flow through Porous Medium with Variable Suction, Information Sciences Letters, 3(3), 97-101 (2014).
22. J. G. Bruno, Electrophoretic Characterization of DNA Oligonucleotide-PAMAM Dendrimer Covalent and Noncovalent Conjugates, Journal of Bionanoscience, 9, 203-208 (2015).
23. K. K. Tanaeva, Yu. V. Dobryakova, and V. A. Dubynin, Maternal Behavior: A Novel Experimental Approach and Detailed Statistical Analysis, Journal of Neuroscience and Neuroengineering, 3, 52-61 (2014).
24. E. Zaitseva and M. Rusin, Healthcare System Representation and Estimation Based on Viewpoint of Reliability Analysis, Journal of Medical Imaging and Health Informatics, 2, 80-86 (2012).
25. R. Ahirwar, P. Devi and R. Gupta, Seasonal incidence of major insect-pests and their biocontrol agents of soybean crop (Glycine max L. Merrill), Scientific Research and Essays, 10(12), 402-406 (2015).
26. H. Boussak, H. Chemani and A. Serier, Characterization of porcelain tableware formulation containing bentonite clay,International Journal of Physical Sciences,10(1), 38-45 (2015).
27. Q. Xiaohong, and Q. Xiaohui, an Evolutionary Particle Swarm Optimizer Based on Fractal Brownian Motion, Journal of Computational Intelligence and Electronic Systems, 1, 138 (2012).
28. G. Minhas and M. Kumar, LSI Based Relevance Computation for Topical Web Crawler, Journal of Emerging Technologies in Web Intelligence, 5(4), 401-406 (2013).
29. Y. Shang, Efficient strategies for attack via partial information in scale-free networks, Information Sciences Letters, 1(1), 1-5 (2012).
30. I. Rathore and J. C. Tarafdar, Perspectives of Biosynthesized Magnesium Nanoparticles in Foliar Application of Wheat Plant,Journal of Bionanoscience, 9, 209-214 (2015).
31. H. Yan and H. Hu, Research and Realization of ISIC-CDIO Teaching Experimental System Based on RFID Technology ofWeb of Things,Journal of Bionanoscience, 7, 696-702 (2013).
32. R. Teles, B. Barroso, A. Guimaraes and H. Macedo, Automatic Generation of Human-like Route Descriptions: A Corpus-driven Approach, Journal of Emerging Technologies in Web Intelligence, 5(4), 413-423 (2013).
33. E. S. Hui, Diffusion Magnetic Resonance Imaging of Ischemic Stroke, Journal of Neuroscience and Neuroengineering, 1, 48-53 (2012).
34. O. E. Emam, M. El-Araby and M. A. Belal, On Rough Multi-Level Linear Programming Problem, Information Sciences Letters, 4(1), 41-49 (2015).
35. B. Prasad, D.C. Dimri and L. Bora,Effect of pre-harvest foliar spray of calcium and potassium on fruit quality of Pear cv. Pathernakh, Scientific Research and Essays,10(11), 392-396 (2015).
36. H. Parvin, H. Alinejad-Rokny and M. Asadi, An Ensemble Based Approach for Feature Selection, Australian Journal of Basic and Applied Sciences, 7(9), 33-43 (2011).
37. Fouladgar M.H., Minaei-Bidgoli B., Parvin H.: Enriching Dynamically Detected Invariants in the Case of Arrays. International Conference on Computational Science and Its Applications (ICCSA 2011), LNCS, ISSN: 0302-9743. LNCS. Springer, Heidelberg, pp. 622–632, 2011.
38. H. Parvin, H. Alinejad-Rokny, S. Parvin, Divide and Conquer Classification, Australian Journal of Basic & Applied Sciences, 5(12), 2446-2452 (2011).
39. H. Parvin, B. Minaei-Bidgoli, H. Alinejad-Rokny, A New Imbalanced Learning and Dictions Tree Method for Breast Cancer Diagnosis, Journal of Bionanoscience, 7(6), 673-678 (2013).
40. H. Parvin H., H. Alinejad-Rokny, M. Asadi, An Ensemble Based Approach for Feature Selection, Journal of Applied Sciences Research, 7(9), 33-43 (2011).
41. H. Parvin, H. Helmi, B. Minaie-Bidgoli, H. Alinejad-Rokny, H. Shirgahi, Linkage learning based on differences in local optimums of building blocks with one optima, International Journal of Physical Sciences, 6(14), 3419-3425 (2011).
42. H. Parvin, B. Minaei-Bidgoli, H. Alinejad-Rokny, S. Ghatei, An innovative combination of particle swarm optimization, learning automaton and great deluge algorithms for dynamic environments, International Journal of Physical Sciences, 6(22), 5121-5127 (2011).
43. H. Parvin, H. Alinejad-Rokny, S. Parvin, A Classifier Ensemble of Binary Classifier Ensembles, International Journal of Learning Management Systems, 1(2), 37-47 (2013).
44. H. Parvin, B. Minaei-Bidgoli, H. Alinejad-Rokny, W.F. Punch, Data weighing mechanisms for clustering ensembles, Computers & Electrical Engineering, 39(5): 1433-1450 (2013).
45. H. Parvin, H. Alinejad-Rokny, B. Minaei-Bidgoli, S. Parvin, A new classifier ensemble methodology based on subspace learning, Journal of Experimental & Theoretical Artificial Intelligence, 25(2), 227-250 (2013).
46. R. Agrawal, J. Gehrke, D. Gunopulos, P. Raghavan, Automatic Subspace Clustering of High Dimensional Data for Data Mining Applications, In Proceedings of the 1998 ACM SIGMOD international conference on Management of data, (1998) 94-105.
47. A. Blum, R. Rivest, Training a 3-node neural network is NP-complete, Neural Networks, 5 (1992) 117-127.
48. J.W. Chang,  D.S. Jin, A new cell-based clustering method for large-high dimensional data in data mining applications, In Proceedings of the ACM symposium on Applied computing, (2002) 503-507.
49. S. Dudoit, J. Fridlyand, Bagging to improve the accuracy of a clustering procedure, Bioinformatics, 19(9) (2003) 1090-1099.
50. K. Faceli, C.P. Marcilio, D. Souto, Multi-objective Clustering Ensemble, Proceedings of the Sixth International Conference on Hybrid Intelligent Systems (HIS'06), (2006).
51. A.K. Jain, R.C. Dubes R.C, Algorithms for Clustering Data, Prentice Hall, (1988).
52. R. Kohavi R G. John, Wrappers for feature subset selection, Artificial Intelligence, 97(1-2) (1997) 273-324.
53. B. Liu, Y. Xia, P.S.  Yu, Clustering through decision tree construction, In Proceedings of the ninth international conference on Information and knowledge management, (2000), 20-29.
54. R. Miller, Y. Yang, Association rules over interval data, In Proc. ACM SIGMOD International Conf. on Management of Data, (1997) 452-461.
55. A. Mirzaei, M. Rahmati, M. Ahmadi, A new method for hierarchical clustering combination, Intelligent Data Analysis, 12(6), (2008) 549-571.
56. C.B.D.J Newman, S. Hettich S, C. Merz, UCI repository of machine learning databases, http://www.ics.uci.edu/˜mlearn/MLSummary.html, (1998).
57. L. Parsons, E. Haque, H. Liu, Subspace clustering for high dimensional data: a review, ACM SIGKDD Explorations Newsletter, 6(1) (2004) 90-105.
58. C.M. Procopiuc, M. Jones, P.K. Agarwal P.K, T.M. Murali T.M, A Monte Carlo algorithm for fast projective clustering, In: Proceedings of the ACM SIGMOD conference on management of data, (2002) 418-427.
59. R. Srikant, R. Agrawal, Mining Quantitative Association Rules in Large Relational Tables, In Proc. of the ACM SIGMOD Conference on Management of Data, Montreal, Canada, (1996).
60. C.H. Cheng, A.W. Fu, Y. Zhang, Entropy-based subspace clustering for mining numerical data, In Proceedings of the fifth ACM SIGKDD international conference on Knowledge discovery and data mining, (1999) 84-93.
61. C. Domeniconi, M. Al-Razgan, Weighted cluster ensembles: Methods and analysis, TKDD, 2(4) (2009).
62. C. Domeniconi, D. Gunopulos, S. Ma, B. Yan, M. Al-Razgan, D. Papadopoulos, Locally adaptive metrics for clustering high dimensional data, Data Mining & Knowledge Discovery, 14(1) (2007) 63-97.
63. A. Strehl, J. Ghosh J, Cluster ensembles-a knowledge reuse framework for combining multiple partitions, Journal of Machine Learning Research, 3 (2002) 583-617.
64. J. Munkres, Algorithms for the Assignment and Transportation Problems, Journal of the Society for Industrial and Applied Mathematics, 5(1) (1957) 32-38.
65. Fred, A. and Jain, A. K. (2002). "Data Clustering Using Evidence Accumulation", Proc. of the 16th Intl. Conf. on Pattern Recognition, ICPR02, Quebec City, pp. 276 – 280.
66. A. Fred, "Finding Consistent Clusters in Data Partitions," Proc. Second Int’l Workshop Multiple Classifier Systems, J. Kittler and F. Roli, eds., pp. 309-318, 2001.
67. A. Fred and A.K. Jain, "Evidence Accumulation Clustering Based on the k-means Algorithm," Proc. Structural, Syntactic, and Statistical Pattern Recognition, Joint IAPR Int’l Workshops SSPR 2002 and SPR 2002, T. Caelli, et al., eds., pp. 442-451, 2002.
68. Fred A. and Jain A.K. (2005). Combining Multiple Clusterings Using Evidence Accumulation. IEEE Trans. on Pattern Analysis and Machine Intelligence, 27(6):835–850.
69. Fern X.Z. and Lin W. (2008), Cluster Ensemble Selection, SIAM International Conference on Data Mining, pp. 787-797.
70. M.H. Fouladgar, B. Minaei-Bidgoli, H. Parvin, H. Alinejad-Rokny, Extension in The Case of Arrays in Daikon like Tools, Advanced Engineering Technology and Application, 2(1), 5-10 (2013).
71. I. Jamnejad, H. Heidarzadegan, H. Parvin, H. Alinejad-Rokny, Localizing Program Bugs Based on Program Invariant, International Journal of Computing and Digital Systems, 3(2), 141-150 (2014).
72. H. Parvin, H. Alinejad-Rokny, S. Parvin, H. Shirgahi, A new Conditional Invariant Detection Dramework (CIDF), Scientific Research and Essays, 8(6), 265-273 (2013).
73. Chang, Y.H. and C.H. Yeh, 2001. Evaluating airline competitiveness using multi attribute decision making. Omega 29: 405-415.
74. Hu, Y.C. and J.F. Tsai, 2006. Backpropagation multi-layer perceptron for incomplete pairwise comparison matrices in analytic hierarchy process. Applied mathematics and computation 181(1): 53-62.
75. Hwang, C.L. and K. Yoon, 1981. Multiple attribute decision making.Springer-Verlag, Berlin Heidelberg New York.
76. Agalgaonkar, A.P., S.V. Kulkarni. and S.A. Khaparde, 2005. Multi-attribute decision making approach for strategic planning of DGs. Power Engineering Society General Meeting 3: 2985-2990.
77. Byun, H.S. and K.H. Lee, 2006. Determination of the optimal build direction for different rapid prototyping processes using multi criterion decision making. Robotics and Computer-Integrated Manufacturing 22: 69-80.
78. Kabassi, K. and M. Virvou, 2004. Personalised adult e-training on computer use based on multiple attribute decision making. Interacting with Computers 16: 115-132.
79. Yang, T., M.C. Chen. and C.C. Hung, 2007. Multiple attribute decision-making methods for the dynamic operator allocation problem. Mathematics and Computers in Simulation 73(5): 285-299.
80. Geoffrion, M., J.S. Dyer, A. Feinberg, 1972. An interactive approach for multi-criterion optimization, with an application to the operation of an academic department. Management Science 19(4): 357-368.
81. Stewart, T.J. 1992. A critical survey on the status of multiple criteria decision making theory and practice. Omega 20(5): 569-586.
82. Malakooti, B. and Y. Zhou, 1994. Feed-forward artificial neural networks for solving discrete multiple criteria decision making problems. Management Science 40(11): 1542-1561.
83. Sun, A., A. Stam and R.E. Steuer, 1996. Solving multiple objective programming problems using feed-forward artificial neural networks: the interactive FFANN procedure. Management Science 42(6): 835-849.
84. Wang, J. and B. A. Malakooti, 1992. Feed forward neural network for multiple criteria decision making. Computers & Operations Research 19(2): 151-167.
85. Li, Q. A. 2008. Fuzzy neural network based Multi-criteria decision making approach for outsourcing supplier evaluation. The 3^rd IEEE Conference on Industrial Electronics and Applications 1: 192-196.
86. Kong, F. and H.  Liu, 2006. Fuzzy RBF neural network model for multiple attribute decision making. The 13th International Conference on Neural Information Processing Part III: 1046-1054.
87. N. Lalithamani and M. Sabrigiriraj,Dual Encryption Algorithm to Improve Security in Hand Vein and Palm Vein-Based Biometric Recognition, Journal of Medical Imaging and Health Informatics, 5, 545-551 (2015).
88. M. Khan and R. Jehangir,Fuzzy resolvability modulo fuzzy ideals, International Journal of Physical Sciences, 7(6), 953- 956 (2012).
89. M. Ravichandran and A.Shanmugam, Amalgamation of Opportunistic Subspace & Estimated Clustering on High Dimensional Data, Australian Journal of Basic and Applied Sciences, 8(3), 88-97, (2014).
90. M. Zhang, Optimization of Inter-network Bandwidth Resources for Large-Scale Data Transmission, Journal of Networks, 9(3), 689-694 (2014).
91. O. O. E. Ajibola, O. Ibidapo-Obe, and V. O. S. OIunloyo, A Model for the Management of Gait Syndrome in Huntington's Disease Patient, Journal of Bioinformatics and Intelligent Control, 3, 15-22 (2014).
92. L. Z. Pei, T. Wei, N. Lin and Z. Y. Cai, Electrochemical Sensing of Histidine Based on the Copper Germanate Nanowires Modified Electrode, Journal of Bionanoscience, 9, 161-165 (2015).
93. M. K. Elboree, Explicit Analytic Solution for the Nonlinear Evolution Equations using the Simplest Equation Method, Mathematical Sciences Letters, 3(1), 59-63 (2014).
94. R. Yousef and T. Almarabeh, An enhanced requirements elicitation framework based on business process models, Scientific Research and Essays,10(7), 279-286 (2015).
95. K. Manimekalai and M.S. Vijaya, Taxonomic Classification of Plant Species Using Support Vector Machine, Journal of Bioinformatics and Intelligent Control, 3, 65-71 (2014).
96. S. Rajalaxmi and S. Nirmala, Automated Endo Fitting Curve for Initialization of Segmentation Based on Chan Vese Model, Journal of Medical Imaging and Health Informatics, 5, 572-580 (2015).
97. T. Mahmood and K. Hayat, Characterizations of Hemi-Rings by their Bipolar-Valued Fuzzy h-Ideals, Information Sciences Letters, 4(2), 51-59 (2015).
98. Agarwal and N. Mittal, Semantic Feature Clustering for Sentiment Analysis of English Reviews, IETE Journal of Research, 60(6), 414-422 (2014).
99. S. Radharani and M. L.Valarmathi, Content Based Watermarking Techniques using HSV and Fractal Dimension in Transform Domain, Australian Journal of Basic and Applied Sciences, 8(3), 112-119 (2014).
100. H. W. and W. Wang, an Improved Artificial Bee Colony Algorithm and Its Application on Production Scheduling, Journal of Bioinformatics and Intelligent Control, 3, 153-159 (2014).
101. L. Gupta,Effect of orientation of lunar apse on earthquakes, International Journal of Physical Sciences, 7(6), 974-981 (2012).
102. S. Iftikhar, F. Ahmad and K. Fatima, A Semantic Methodology for Customized Healthcare Information Provision, Information Sciences Letters, 1(1), 49-59 (2012).
103. P. D. Sia, Analytical Nano-Modelling for Neuroscience and Cognitive Science, Journal of Bioinformatics and Intelligent Control, 3, 268-272 (2014).
104. C. Guler, Production of particleboards from licorice (Glycyrrhiza glabra) and European black pine (Pinus Nigra Arnold) wood particles,Scientific Research and Essays,10(7), 273-278 (2015).
105. Z. Chen and J. Hu, Learning Algorithm of Neural Networks on Spherical Cap, Journal of Networks, 10(3), 152-158 (2015).
106. W. Lu, Parameters of Network Traffic Prediction Model Jointly Optimized by Genetic Algorithm, Journal of Networks, 9(3), 695-702 (2014).
107. K. Boubaker,An attempt to solve neutron transport equation inside supercritical water nuclear reactors using the Boubaker Polynomials Expansion Scheme, International Journal of Physical Sciences, 7(19), 2730-2734 (2012).
108. K. Abd-Rabou, Fixed Point Results in G-Metric Space, Mathematical Sciences Letters, 3(3), 141-146 (2014).
109. Binu and M. Selvi, BFC: Bat Algorithm Based Fuzzy Classifier for Medical Data Classification, Journal of Medical Imaging and Health Informatics, 5, 599-606 (2015).
110. C. Kamath, Analysis of Electroencephalogram Background Activity in Epileptic Patients and Healthy Subjects Using Dispersion Entropy, Journal of Neuroscience and Neuroengineering, 3, 101-110 (2014).
111. G. Kaur and E. M. Bharti, Securing Multimedia on Hybrid Architecture with Extended Role-Based Access Control, Journal of Bioinformatics and Intelligent Control, 3, 229-233 (2014).
112. M. Ramalingam and D. Rana, Impact of Nanotechnology in Induced Pluripotent Stem Cells-driven Tissue Engineering and Regenerative Medicine, Journal of Bionanoscience, 9, 13-21 (2015).
113. S. Downes, New Technology Supporting Informal Learning, Journal of Emerging Technologies in Web Intelligence, 2(1), 27-33 (2010).
114. R. Periyasamy, T. K. Gandhi, S. R. Das, A. C. Ammini and S. Anand, A Screening Computational Tool for Detection of Diabetic Neuropathy and Non-Neuropathy inType-2 Diabetes Subjects,Journal of Medical Imaging and Health Informatics, 2, 222-229 (2012).
115. Y. Qin, F. Wang and C. Zhou, A Distributed UWB-based Localization System in Underground Mines, Journal of Networks, 10(3), 134-140 (2015).
116. P. Saxena and C. Ghosh,A review of assessment of benzene, toluene, ethylbenzene and xylene (BTEX) concentration in urban atmosphere of Delhi, International Journal of Physical Sciences,7(6), 850-860 (2012).
117. J. Hu, Z. Zhou and M. Teng, The Spatiotemporal Variation of Ecological Risk in the Lijiang River Basin Based on Land Use Change, Journal of Bionanoscience, 9, 153-160 (2015).
118. N. Saleem, M. Ahmad, S. A. Wani, R. Vashnavi and Z. A. Dar,Genotype-environment interaction and stability analysis in Wheat (Triticum aestivum L.) for protein and gluten contents, Scientific Research and Essays,10(7), 260-265 (2015).
119. R. A. Rashwan and S. M. Saleh, A Coupled Fixed Point Theorem for Three Pairs of w-Compatible Mappings in G-metric spaces, Mathematical Sciences Letters, 3(1), 17-20 (2014).
120. S. P. Singh and B. K. Konwar, Carbon Nanotube Assisted Drug Delivery of the Anti-Malarial Drug Artemesinin and Its Derivatives-A TheoreticalNanotechnology Approach, Journal of Bionanoscience, 7, 630-636 (2013).
121. R. Dinasarapu and S. Gupta, Biological Data Integration and Dissemination on Semantic Web-A Perspective,Journal of Bioinformatics and Intelligent Control, 3, 273-277 (2014).
122. W. Qiaonong, X. Shuang, and W. Suiren, Sparse Regularized Biomedical Image Deconvolution Method Based on Dictionary Learning, Journal of Bionanoscience, 9, 145-152 (2015).
123. C. Prema and D. Manimegalai, Adaptive Color Image Steganography Using Intra Color Pixel Value Differencing, Australian Journal of Basic and Applied Sciences, 8(3), 161-167 (2014).
124. R. Adollah, M. Y. Mashor, H. Rosline, and N. H. Harun, Multilevel Thresholding as a Simple Segmentation Technique in Acute Leukemia Images, Journal of Medical Imaging and Health Informatics, 2, 285-288 (2012).
125. H. Uppili, Proton-Induced Synaptic Transistors: Simulation and Experimental Study, Journal of Neuroscience and Neuroengineering, 3, 117-129 (2014).
126. Chen, J. and S. Lin, 2003. An interactive neural network-based approach for solving multiple criteria decision-making problems. Decision Support Systems 36: 137-146.
127. Chen, J. and S. A. Lin, 2004. Neural network approach-decision neural network (DNN) for preference assessment. IEEE Transactions on systems, Man, and Cybernetics-Part C: Applications and reviews 34: 219-225.
128. H. B. Kekre and T. K. Sarode, Vector Quantized Codebook Optimization Using Modified Genetic Algorithm, IETE Journal of Research, 56(5), 257-264 (2010).
129. M. Gera, R. Kumar, V. K. Jain, Fabrication of a Pocket Friendly, Reusable Water Purifier Using Silver Nano Embedded Porous Concrete PebblesBased on Green Technology, Journal of Bionanoscience, 8, 10-15 (2014).
130. M. S. Kumar and S. N. Devi, Sparse Code Shrinkage Based ECG De-Noising in Empirical Mode Decomposition Domain, Journal of Medical Imaging and Health Informatics, 5, 1053-1058 (2015).
131. C. Zhou, Y. Li, Q. Zhang and B. Wang, An Improved Genetic Algorithm for DNA Motif Discovery with Gibbs Sampling Algorithm, Journal of Bionanoscience, 8, 219-225 (2014).
132. R. Bhadada and K. L. Sharma, Evaluation and Analysis of Buffer Requirements for Streamed Video Data in Video on Demand Applications, IETE Journal of Research, 56(5), 242-248 (2010).
133. M. Kurhekar and U. Deshpande, Deterministic Modeling of Biological Systems with Geometry with an Application to Modeling of Intestinal Crypts, Journal of Medical Imaging and Health Informatics, 5, 1116-1120 (2015).
134. S. Prabhadevi and Dr. A.M. Natarajan, A Comparative Study on Digital Signatures Based on Elliptic Curves in High Speed Ad Hoc Networks, Australian Journal of Basic and Applied Sciences, 8(2), 1-6 (2014).
135. X. Jin and Y. Wang, Research on Social Network Structure and Public Opinions Dissemination of Micro-blog Based on Complex Network Analysis, Journal of Networks, 8(7), 1543-1550 (2013).
136. O. G. Avrunin, M. Alkhorayef, H. F. I. Saied, and M. Y. Tymkovych, The Surgical Navigation System with Optical Position Determination Technology and Sources of Errors,Journal of Medical Imaging and Health Informatics, 5, 689-696 (2015).
137. R. Zhang, Y. Bai, C. Wang and W. Ma, Surfactant-Dispersed Multi-Walled Carbon Nanotubes: Interaction and Antibacterial Activity, Journal of Bionanoscience, 8, 176-182 (2014).
138. B. K. Singh, Generalized Semi-bent and Partially Bent Boolean Functions, Mathematical Sciences Letters, 3(1), 21-29 (2014).
139. S. K. Singla and V. Singh, Design of a Microcontroller Based Temperature and Humidity Controller for Infant Incubator, Journal of Medical Imaging and Health Informatics, 5, 704-708 (2015).
140. N. Barnthip and A. Muakngam, Preparation of Cellulose Acetate Nanofibers Containing Centella Asiatica Extract by Electrospinning Process as the Prototype of Wound-Healing Materials,Journal of Bionanoscience, 8, 313-318 (2014).
141. R. Jac Fredo, G. Kavitha and S. Ramakrishnan, Segmentation and Analysis of Corpus Callosum in Autistic MR Brain Images Using Reaction Diffusion Level Sets, Journal of Medical Imaging and Health Informatics, 5, 737-741 (2015).
142. Wang, B. Zhu, An Improved Algorithm of the Node Localization in Ad Hoc Network, Journal of Networks, 9(3), 549-557 (2014).
143. T. Buvaneswari and A. A. Iruthayaraj, Secure DiscoveryScheme and Minimum Span Verification of Neighbor Locations in Mobile Ad-hoc Networks, Australian Journal of Basic and Applied Sciences, 8(2), 30-36 (2014).
144. H. Parvin, H. Alinejad-Rokny, N. Seyedaghaee, S. Parvin, A Heuristic Scalable Classifier Ensemble of Binary Classifier Ensembles, Journal of Bioinformatics and Intelligent Control, 1(2), 163-170 (2013).
145. M.H. Fouladgar, B. Minaei-Bidgoli, H. Parvin, H. Alinejad-Rokny, Extension in The Case of Arrays in Daikon like Tools, Advanced Engineering Technology and Application, 2(1), 5-10 (2013).
146. H. Parvin, M. MirnabiBaboli, H. Alinejad-Rokny, Proposing a Classifier Ensemble Framework Based on Classifier Selection and Decision Tree, Engineering Applications of Artificial Intelligence, 37, 34-42 (2015).
147. Y. Zhang, Z. Wang and Z. Hu, Nonlinear Electroencephalogram Analysis of Neural Mass Model, Journal of Medical Imaging and Health Informatics, 5, 783-788 (2015).
148. S. Panwar and N. Nain, A Novel Segmentation Methodology for Cursive Handwritten Documents, IETE Journal of Research, 60(6), 432-439 (2014).
149. H. Mao, On Applications of Matroids in Class-oriented Concept Lattices, Mathematical Sciences Letters, 3(1), 35-41 (2014).
150. D. Kumar, K. Singh, V. Verma and H. S. Bhatti, Synthesis and Characterization of Carbon Quantum Dots from Orange Juice, Journal of Bionanoscience, 8, 274-279 (2014).
151. V. Kumutha and S. Palaniammal, Enhanced Validity for Fuzzy Clustering Using Microarray data, Australian Journal of Basic and Applied Sciences, 8(3), 7-15 (2014).
152. Y. Wang, C. Yang and J. Yu, Visualization Study on Cardiac Mapping: Analysis of Isopotential Map and Isochron Map, Journal of Medical Imaging and Health Informatics, 5, 814-818 (2015).
153. R. Su, Identification Method of Sports Throwing Force Based on Fuzzy Neural Network, Journal of Networks, 8(7), 1574-1581 (2013).
154. Matsuda, S. A. 2005. Neural network model for the decision-making process based on AHP. Proceedings of International Joint Conference on Neural Networks, Montreal, Canada.
155. Kohonen, T. 1987. Self-Organizing and associative memory, 2end edition. Berlin:Springer-Verlag.
156. Haykin S. 1999. Neural networks: a comprehensive foundation. prentice hall.
157. Milan, J. and R. Aura, 2002. An application of the multiple criteria decision making analysis to the selection of a new hub airport. EJTIR 2(2): 113-141.