A New Simplified Approach for Learning Outcomes Assessment in Engineering Education Programs

Ashraf Balabel^{1, 2, *}

¹Mechanical Engineering Dept., Faculty of Engineering, Taif University, Al-Haweiah, Taif, Saudi Arabia

²Mechanical Engineering Dept., Faculty of Engineering, Menoufia University, Shebin El-Kom, Menoufia, Egypt

Abstract

Engineering education programs are considered to provide future engineers with fundamental knowledge, professional skills and engineering ethics as well as the capability to deal with different and important society’s issues. The achievement of such goals is based on the students learning outcomes of the courses taught in a certain engineering program. Therefore, the assessment of learning outcomes is an important issue in engineering education programs. Moreover, most of accrediting agencies for academic engineering programs have been turned to increasing emphasis on the assessment strategic plans for programs to demonstrate its achievement of program objectives. Consequently, different assessment plans for educational objectives have been recently developed. The present paper introduces a new simplified approach for learning outcomes assessment in engineering programs. The proposed approach is based firstly on the preliminary determination of the course learning outcomes weight through the contact hours provided by the course director through different teaching methods. Secondly, a direct measure for the learning outcomes through different assessment methods is provided. The proposed approach determines the achievement of the program objectives in a simplified way and enables us to monitor and improve simply the courses contents in response to students’ performance or changes in modern engineering technology.

Keywords

Engineering Education Programs, Learning Outcomes, Assessment Methods

Received: March 28, 2015
Accepted: April 9, 2015
Published online: April 10, 2015

@ 2015 The Authors. Published by American Institute of Science. This Open Access article is under the CC BY-NC license. http://creativecommons.org/licenses/by-nc/4.0/

1. Introduction

Engineering education programs are designed to serve new future engineers by providing them with specific skills and abilities according to the rapid development in the industrial engineering sectors and the need of the society. In order to proof that the designed engineering program has achieved specified standards necessary to produce qualified graduates, the engineering program should be accredited. The accreditation is a process in which certification of competence in a specified area of expertise awarded by a recognized and respected accrediting organization. The Accreditation Board for Engineering and Technology, abbreviated as ABET, is a non-governmental and non-profit organization that internationally accredits engineering education programs. More information about ABET organization can be found in the website of ABET [1]. All the engineering programs seeking accreditation from ABET must prove that they satisfy general criteria for Baccalaureate level. These accreditation criteria have been discussed extensively since their introduction in 1996. However, Criterion 3’s program outcomes are a different matter as a large number of important questions are raised by the courses’ directors regarding the teaching and assessment methods [2]. The program outcomes are the abilities that students are supposed to have on graduation from the designed program. These outcomes are rather different even from engineering program to another [3]. The engineering program learning outcomes are adopted from the well-known engineering accreditation body ABET a-to-k student outcomes [4]. Table 1 shows the ABET a-to-k student outcomes (SOs) that ABET suggests for an engineering program. The last column in Table 1 is added by the author in order to classify the description of the outcome into subtitles used for the courses learning outcomes (CLOi).

Table 1. ABET a-to-k student outcomes for an engineering program.

The ABET a-to-k student outcomes for an engineering program can be related to the so-called Behavioral Objectives which are used for measuring the students achievements in different educational programs. Table 2 shows the relationship between the ABET a-to-k student outcomes and the corresponding Behavioral Objectives.

Table 2. shows the relationship between the ABET a-to-k student outcomes and the corresponding Behavioral Objectives.

As it can be seen from the above table, that ABET a-to-k student outcomes include all the Behavioural Objectives, consequently ABET a-to-k student outcomes are considered to be suitable for the assessment requirements.

2. Designing Engineering Program and Courses to Satisfy Criterion

The design of an engineering program is essentially based on identifying the program’s mission and its educational objectives related to the university and college missions. The program educational objectives should be formulated following the specifications of Criterion 2, and the program outcomes, that encompass ABET a-to-k are assigned to address the educational objectives. The important next step is to identify the program core which consists of a number of different courses in the program curriculum chosen to satisfy the attitude defined in the ABET a-to-k students outcomes in a consistent manner. Elective courses should not be included in the program core as they changed from one schedule to another.

2.1. Courses Learning Objectives

The course learning objectives, some times referred to as intended learning outcomes or course specific goals, are defined as the statements that state what the students should know and able to demonstrate as well as skills to possess upon the completion of the course. The integration of the all courses learning objectives is related to the program educational objectives. For writing effective course learning objectives, one should follow some known rules [5]. Moreover, an assessment for course learning objectives before and after teaching the course from the course director is required.

2.2. Courses Learning Outcomes (CLOi)

The course learning outcomes are statements that can describe the essential and the significant learning that students have achieved.  By the end of the course, the students should acquire enduring knowledge, skills and attitudes that will be integrated from a specified course or program. The approach of learning outcomes to education is mainly based on the program and curriculum design, course content and assessment on an analysis of the integrated knowledge, skills and values needed by both students and society.

2.3. Comparison between Objectives and Outcomes

The learning objectives are defined as the intended results of the course or program, while the outcomes are the achieved results or consequences of what was learned, that means the outcomes are evidences that learning took place. The objectives are concentrated on specific types of performance that students are required to demonstrate by the end of the instruction. However, learning outcomes describe what the students should learn. Therefore, the method of assessment applied for learning outcomes is the key point of education program.

2.4. The New Proposed Assessment Approach

According to Bloom’s Taxonomy, shown in Fig. 1, where stages for course design are not sequential arranged, the information gathered in each stage is fed back to each of the others in a cycle which leads to the continuous improvement process. The Bloom’s Taxonomy can provide course directors with the important framework to make focus on higher order thinking, to design performance tasks and questions, and provide an efficient feedback on students achievement.

Following the basic rules in Bloom’s Taxonomy, a new flow chart is designed for obtaining the course learning outcomes in the next section. This flow chart can be applied for any educational program.

Figure 1. Elements of course design

3. Developed Flow Chart for Program/Course Design

Figure 2 shows the flow chart of the program/course design applied in the present paper. In the following section we will assume a case study for a course in the mechanical engineering program, Faculty of engineering, Taif University.

Figure 2. Flow chart of Program/Course outcomes design.

4. Implementation of the Proposed Approach (Case Study)

The mechanical engineering program, Faculty of Engineering, Taif University aims to integrate ABET expectations, particularly those focused on graduating distinct engineers in the various disciplines of engineering, equipped with knowledge and skills. The graduate student from the mechanical engineering program must complete a total of 160 credit-hours taken from the various courses in the curriculum. One of the important courses in the mechanical program is the Hydraulic Machines and Hydraulic Power Stations. In the following, the proposed approach for learning outcomes assessment is applied to that course as a Case Study.

4.1. Course Description

According to the curriculum of the Mechanical Engineering Program, the contents of the course can be written as: "Classification of hydraulic machines, Study and analysis of the flow within centrifugal pumps, Study and analysis of the flow within axial pumps, The characteristics of positive displacement pumps, Performance laws and similarity of hydraulic machines, Basic construction and operation of hydraulic system components, Examples of hydraulic systems, Introduction to hydraulic turbines, Impulse and Reaction turbines."

4.2. Course Basic Information

The course basic information as stated in the curriculum of Mechanical Engineering Program, Faculty of Engineering, Taif University is stated below, as shown in Table 3, and it includes the most important information of the course.

Table 3. The selected course basic information is described as follows:

4.3. Course Map

In the present section, the so called Course Map is designed in order to link the course topics and the associated learning outcomes. Table 4 shows the course map designed for the course under consideration. The basic idea of the designed course map is, firstly, converting the course description to a number of general topics and subtopics, e.g. [(T1: TI, TII, TIII, TIV)]. The associated course learning intended, known as SOI, will be also formulated, e.g. [SOI1: (SOI1-I, SOI1-II, SOI1-III, SOI1-IV)]. The program outcomes can be classified to course learning outcomes CLOi (a₁, a₂, a₃, etc.) as explained previously in Table 1. The elements of the program outcomes can be weighted according to the teaching hours corresponding to the sum of CLOi in the different subtopics of the course. At the end of the course map, a new table, (Table 5), is given which indicates the percentage of each outcome of the program.

The graph in Fig. 3 shows the plot of each outcome according to its weight in hours of course teaching hours and in percentage. The advantage of this approach are summarized in the easily definition of the number of hours of each outcome in a specified course and consequently in all program. This enables us to compare the program outcomes with a specified academic standard.  Moreover, each outcome can be increased or decreased by adjusting the number of teaching hours and consequently the course contents associated. At the end, the nature of the course can be classified according to the highest weigh outcome.

Table 4. The Designed Course Map.

Table 5. The weight of program learning outcomes for the specified course.

Figure 3. The weight of Student outcomes related to the teaching hours of the course.

5. Teaching Methods for Learning Outcomes

As considered previously, the learning outcomes are an essential issue of any program/course outline because they set out how the aim of the program/course will be realized. The success in formulating the learning outcomes for any program or course should be accompanied by an effective teaching method for such learning outcomes.

The review of such subject in the literature indicates that is an important issue in the higher education system a large number of papers have devoted to review, develop and assess the learning and teaching methods for learning outcomes, see for more details the original work of [7]. The conclusion drawn from the literature review of such subject is the existence of different common teaching methods for each of the learning outcome. The different teaching methods are listed below: Interactive Lecture, Case-Based Learning, Problem-Based Learning, Simulation, Role Play and Fish-Bowl Observation, Tutorial, Self-Directed Learning, Experiential Learning, Laboratory Work, Field work, Peer Tutoring, and e-Learning. More teaching methods appropriated to engineering education can be added to the above list, e.g. Presentation and movies, Brain storming, Site visit, and Discussion The importance of selecting the teaching method appropriated to the corresponding outcomes is represented in the success of teaching and learning of each outcome and the further selection of the assessment method applied.

6. Assessment Methods for Learning Outcomes

The specified learning outcomes are related to the teaching and assessment method through the diagram of constructive alignments [8]. Figure 4 shows the diagram of constructive alignment according to Ref [8].

Figure 4. The Diagram of constructive alignment [8].

It should be pointed out that, there is a difference between assessment and evaluation. The assessment is the systematic collection of data to monitor the achievement in a course or program learning outcomes. However, evaluation is the judgment of the director to what extent the course or the program has achieved the learning outcomes. That means, the assessment focuses on learning, teaching and outcomes. It provides information for continuous improving the learning and teaching methods. Assessment is an interactive process between students and faculty that informs faculty how well their students are learning what they are teaching. However, evaluation focuses on grades and could include discussion, cooperation, attendance, and verbal ability.

Considering assessment methods, it is particularly useful to think first about what qualities or abilities you are seeking to create in the students. Different broad categories of learning outcomes are summarized in Ref. [9]. A group of assessment method appropriated to engineering education can be suggested, such as: Written exam, Oral exam, Tutorial assessment, Project assessment, Report, Discussion, Laboratory test, Home exam, and Monitoring.

6.1. Constructive Alignment of the Selected Course

In the following section, the diagram of the constructive learning for the course selected is designed. Table 6 shows the required diagram.

Table 6. The diagram of constructive learning for the selected course.

6.2. Evaluation of the Program Learning Outcomes

As it is explained previously, the educational program consists of a number of courses have different outcomes. If each outcome has been assessed and evaluated using the above algorithm, then the all program can be assessed and evaluated. This is the key to another important issue; the continuous improvement of the program curriculum, which is considered as one the most important issues in program accreditation [10].  This will be discussed in our future research article.

7. Conclusions

A new simplified approach for deciding, assessing, and evaluating the learning outcomes of an engineering educational program has been presented. This approach has many advantages over the existed ones; e.g. its simplicity, rationality, its power to classify the course taught in the program and its capability to adjust the weight of each outcome in the course/program by adjusting the course contents and its teaching hours. This approach can be applied not only to the engineering programs but also to any educational program in higher education.

References

1. ABET’s website: www.abet.org.Felder, R. M. And Brent, R. Designing and teaching courses to satisfy the ABET engineering criteria, Journal of Engineering Education, 92 (1), 7-25, 2003.
2. Wilczynski, D. and Ragusa, G. A Practical and comprehensive approach of assessing ABET outcome achievement in computer science and computer engineering, American Society for Engineering Education, 2012.
3. ABET, Criteria for accrediting engineering programs. Engineering Accreditation Commission, 2013, (http://www.abet.org/).Soulsby, E., How to write program objectives/outcomes accessed online at: www.assessment.uconn.edu, Feb. 9, 2011.
4. Krathwohl, D., A revision of Bloom’s Taxonomy: An overview, Theory into practice, Vol. 41, No. 4, pp. 1-54, 2002.
5. Bourner, T., Teaching methods for learning outcomes, Education + Training, Volume 39, Number 9, pp. 344–348, 1997.
6. Biggs, J. and Tang, C., Teaching for quality learning at university, Third Edition, Mcgrawhill, 2007.
7. Nightingale, P., Te Wiata, I.T., Toohey, S., Ryan, G., Hughes, C., and Magin, D., Assessing Learning in Universities Professional Development Centre, University of New South Wales, Australia, 1996.
8. Biggs, J. and Tang, C., Teaching for quality learning at university, Third Edition, Mcgrawhill, 2007.
9. Nightingale, P., Te Wiata, I.T., Toohey, S., Ryan, G., Hughes, C., and Magin, D., Assessing Learning in Universities Professional Development Centre, University of New South Wales, Australia, 1996.
10. Chandra, R. S., Omar F. and Essaid B., Continuous Improvement Process Based on Outcomes Based Education, Proceeding of the 2008 LAJC-IJME International Conference ISBN 978-1-60643-379-9, 2008.