A model assessment plan

Introduction

The Assessing Student Learning project sought an example of assessment best practice that recognised the importance of evolving assessment practices across the year levels: from first year assessment, when students have their entire undergraduate studies before them, to final year, the brink of professional practice. Such an example is provided below by Stuart Palmer of Deakin University. Stuart Palmer offers an excellent example of a carefully designed, strategic assessment regime that is thoroughly integrated with his teaching and learning goals. Its features include:

In all, the approach to assessment described below:

This is a model of good practice in assessment in higher education.

Authenticity in assessment: Reflecting professional practice

Stuart Palmer
School of Engineering and Technology, Deakin University

The context

Engineering at Deakin University

Engineering Management studies

The modern disciplines of engineering and management are inextricably linked and it is well known that most engineers make the transition from technical to management responsibilities some time in their careers (Babcock, 1996). The career advancement of engineers depends principally on their ability to become effective managers of the engineering function in particular, and of technology in general (Kinsky, 1994).

There is an international consensus that management skills and studies are an important component of the professional formation and on-going development of professional engineers. The accrediting body for undergraduate engineering courses, the Institution of Engineers, Australia, has a mandatory requirement for management studies in undergraduate courses, “...integrated exposure to professional engineering practice (including management and professional ethics). This element should be 10% of the total course content…” (Institution of Engineers Australia, 1997).

Engineering Management subjects:

1st year: SEB121 Fundamentals of Technology Management
4th year: SEB421 Strategic Issues in Engineering

At Deakin University, the undergraduate programs contain at least one ‘engineering management’ unit per year, that is one out of each eight units studied. The author has academic responsibility for first year/first semester unit SEB121 Fundamentals of Technology Management, and the fourth year/second semester unit SEB421 Strategic Issues in Engineering. The two units have well-defined stated aims.

SEB121

This unit aims both to develop the basic skills you will need as an independent, adult learner, and to build up knowledge in several areas of engineering, technology and society. It is also an introduction to modern practices in the engineering workplace. The unit has four modules:

SEB421

This unit consists of three modules:

The Technological Forecasting and Assessment module discusses methods for long-term forecasting, factors in technological innovations, and the impact of technological changes on business and society. The topics in the Policy Design in Engineering Organisations module are policy structure, designing organisational structure to support policy, and modelling and analysis of policy alternatives. The Issues in Productivity Improvement module focuses on labour productivity, productivity improvement techniques, benchmarking and the changing nature of work practices.

The underpinning assessment principles

The approach to assessment in these subjects is driven by a number of beliefs about effective assessment. These beliefs are as follows:

1. Clear aims and objectives

Learning aims and objectives can come from many sources, including institutional policy, course accrediting bodies, industry groups, professional bodies and academic staff. Aims and objectives inform the syllabus for a unit, and, by testing the student mastery of the syllabus, the assessment indirectly tests achievement of the unit aims and objectives. Certainly, students are likely to be confused if they are told certain aims are important, and are then assessed on something else. Within strands or major streams in a course there should be vertical integration of assessment, and across an entire course there should be horizontal integration of assessment. The author has some scope to address integration within a stream, as he has academic responsibility for the ‘top and tail’ of the management studies strand in the undergraduate course.

2. Authenticity and value

Assessment tasks should reflect and develop the skills that students will need in their university studies and in their professional practice. If assessment is perceived by students to be authentic, it is more likely to be valued. If not, it is likely to be confusing to students and perceived as irrelevant.

3. Fairness and objectivity

Many theories of human motivation stress the importance of perceived equity and fairness. The author finds that marking schemas are a valuable aide in the speedy and consistent marking of student work. They also form a permanent record that can be retained in case of student queries regarding assessment. As student work is to be assessed against objective criteria, it is important that these requirements be clearly spelled out in the assessment details given to students.

4. Efficiency and practicality

The author prefers a series of smaller, appropriately valued assessment tasks spread across the semester to keep students engaged with the course material, rather than a small number of highly valued tasks that can promote ‘crisis-mode’ study as the submission due dates approach. However, there is a need to balance the amount of assessment with the very real issue of student and staff assessment exhaustion. Practical options for increasing the efficiency of assessment include:

Exams versus assignments?

In science and engineering it is not uncommon to find the assessment for a unit consists of two assignments plus an exam, where the exam counts for 70 per cent of the final unit mark. While an exam provides a measure of quality assurance that students have attained a basic familiarity with the topic, such a skewed weighting on an end-of-semester exam is not representative of the skills required in engineering practice. Where an examination is required, it might be given a 50 per cent or less weighting in the total assessment, making more room for a range of semester assignments. If the examination phase of assessment is considered an essential quality requirement, then it can still be given a less than traditional weighting, possibly with a hurdle requirement added that students must pass the exam to pass the unit overall.

Meeting the needs of off-campus versus on-campus students?

Approximately half of our student body is enrolled in off-campus mode. These students are primarily mature age students. All undergraduate units are offered in off-campus mode, and it is possible (with appropriate advanced standing and/or special arrangements for some assessment) to complete the entire undergraduate course without on-campus attendance. Hence, the off-campus student group is important to the School and deserves consideration in the design of assessment. Many off-campus engineering students come to study with significant work experience in the engineering workforce, often with supervisory or management experience, and are generally highly motivated to succeed: promotion and/or continuation of employment and/or fee subsidies are often contingent on the student attaining minimum grade levels.

Some assessment activities may cause difficulties for off-campus students, such as arranging an oral presentation or group work. Group work can be made optional for off-campus students, as many will already be experienced in group work from their employment. Where on-campus students are asked to make a number of oral presentations in a unit, it can be satisfactory to require off-campus students to make only one, where they organise a small audience, video tape their presentation for submission and ask the audience to provide an evaluation of their presentation skills. Of course, there are instances where off-campus students may be at an advantage to their conventional counterparts – where organisational case studies are required, many mature age, off-campus students use their own workplace as the case, whereas on-campus students may have to locate a suitable case study documented in the literature. As much as possible, the author treats on- and off-campus students the same, making accommodations only where there actually are real differences that matter.

The assessment plan: Methods and rationale

Assessment for 1st year: SEB121 Fundamentals of Technology Management

The assessment portfolio for this unit consists of:

This is a first year, first semester unit and the students have their entire undergraduate studies in front of them. Emphasis is placed on skills and knowledge that will be useful for university study, with less emphasis on discipline and practice skills and knowledge. There is a portfolio of assessment types including developing basic competencies with university systems, foundation skills such as information literacy and investigation, some tests of discipline knowledge, and some generic practice skills such as written and oral communication. There is an exam with a ‘must pass’ hurdle requirement, but it does not dominate the assessment. There are a relatively large number of smaller assessment tasks, the aim being student engagement with the course content across the semester, starting small and simple, and building up in size. Continuous assessment starting early in the semester has the benefit of quickly identifying those student falling behind and perhaps at risk, so remedial action can be taken.

The majority of assessable items in this unit are submitted on-line via the TopClass course management system. To build student familiarity with the system and to ensure that any problems are flushed out early in the semester, assignment 1 is a minor exercise, requiring students to get on-line in the first two weeks of semester, access the TopClass system and to introduce themselves in an on-line discussion forum.

Assignment 3 involves students attending a Library familiarisation session where information resources related to engineering and technology are presented. To encourage student attendance the Library has developed an exercise to test students on the resources and search strategies presented, and the completed exercise is submitted and marked as Assignment 3. Attaching marks to this exercise means that most students will complete this important foundation skill building activity.

Assignment 4 looks to build upon and extend the skills developed in Assignment 3, and tie in with related class material addressing written communication, plagiarism, etc. Students use the Library to locate and produce formatted references for a number of each of the following information sources – textbooks, journal papers, conference papers and web sites. After this assignment students should be familiar with a range of information sources and be able to cite and reference them in their work appropriately.

Normally, Assignment 2 would appear next in the assessment sequence, to exercise the skills developed in assessment tasks completed so far and to tie in with class work dealing with professional practice and ethics. Students have to locate a published case study relating to the failure of technology and assess the ethics of the parties involved. The work submitted is a small written report professionally presented with appropriate graphics and referencing. Students may work in groups of up to three, with the word limit increasing in proportion to the number of team members. This year, the Library was unable to take the students for their familiarisation exercise until later in the semester than normal, hence a pragmatic re-arrangement of the class and assessment sequence was required. The ethics report had to come before Assignment 3 and 4, necessitating some early presentation of information literacy and referencing material prior to Assignment 2, and a consequent allowance in the assessment of Assignment 2 for the lesser level of preparation of students in these areas.

Assignments 5 and 6 are similar, being multiple-choice tests completed on-line in the TopClass environment. The questions relate to the course material being studied at the time, and form a mini-bracket of continuous assessment for a two-week period, one test being completed each week. The multiple-choice format is used to vary the assessment style in the unit, and to reduce the overall marking effort required, as both tests are automatically marked by the TopClass system.

The final major assignment for the semester seeks to integrate and further exercise the skills and knowledge gained throughout the semester, in the context of a real-world case study. Students, working in groups of up to three, identify and investigate a real organisation via published literature and/or visit to interview a manager. In a written report they document and analyse the organisation’s approach to a number of technology management issues studied across the semester. Based on this work the group then prepares and delivers a 10 minute oral presentation to the class that describes their findings.

Assessment for 4th year, SEB421 Strategic Issues in Engineering

The assessment portfolio for this unit consists of:

This is a final year, final semester unit and the next stop for many students is professional practice. At this stage, the students have developed maturity and an intrinsic interest in and motivation for the course material. Emphasis is now placed on discipline and practice skills and knowledge, and advanced conceptual topics, with less emphasis on other types of skills. There is less diversity in assessment types and there are a smaller number of more significant assessment tasks, with a focus on practising professional skills in the context of discipline area case studies. There is an exam with a ‘must pass’ hurdle requirement.

Reflective thinking based on experiential learning is a key skill required for the lifelong learner and the socially mature engineering professional (Schön, 1995). Assignment 1 aims to develop skills in critical reflection on action, and is completed across the semester by asking students to reflect weekly in short written form on what they learned and of what value it might be in the future. At the end of the semester students are asked to prepare a reflective report that identifies: the important things learned in the unit, insights they have gained into the way(s) they learn, and suggestions for improving the unit (this final point not only requires students to reflect critically, but also provides additional feedback on the unit and its assessment).

Assignment 2 requires students, working in groups of three, to locate a published case study relating to the issues currently being studied in class. In a written report they document and analyse the organisations approaches to the issues. Based on this work the group then prepares and delivers a 15 minute oral presentation to the class that describes their findings. The topic for Assignment 2 is technological forecasting. Assignment 3 has identical requirements to Assignment 2, except that the topic is policy design.

The final major assignment for the semester seeks to integrate and further exercise the skills and knowledge gained throughout the semester (and across the entire management stream of study), in the context of a real-world case study. Students, working in groups of up to three, identify and investigate a real organisation via a visit to interview an engineering manager. In a written report they document and analyse the organisations approach to the range of the technology management issues studied across the semester, comparing and contrasting the approaches of the organisation to those studied in class. Based on this work the group then prepares and delivers a 20 minute oral presentation to the class that describes their findings.

In actual engineering practice, graduates will be assessed on their performance as an individual, but also on their effectiveness in working in teams. Assignments 2, 3 and 4 involve group work, and while individual contributions to the class presentations can be assessed, it is difficult to assess individual contributions to a group written report. The groups are given the opportunity to indicate the relative contribution of each group member to each assignment by completing a ‘group work declaration’ with every group submission. Here the groups may state the percentage of the work contributed by each member. The group must agree on the proportions stated and all group members must sign the declaration. In practice, very few groups elect to report other than equal contributions.

References

Babcock, D. (1996). Managing Engineering and Technology (2nd ed.). Upper Saddle River, New Jersey: Prentice-Hall Inc.
Institution of Engineers Australia. (1997). Manual for the Accreditation of Professional Engineering Courses. Canberra, Australia: The Institution of Engineers, Australia.
Kinsky, R. (1994). Engineering Management. South Melbourne, Victoria: Thomas Nelson Australia.
Lloyd, B., Baker, L., & Briggs, H. (1996). Off-campus Articulated Education in Engineering at Deakin University for Mature Students. Paper presented at the 8th Annual Convention and Conference of the Australasian Association for Engineering Education, Sydney.
Schön, D. A. (1995). The Reflective Practitioner: How Professionals Think in Action. Aldershot, England: Arena.