University Quality Activities 2005 - Report to Council

1.  Introduction

1.1  The Monash approach to quality is identified in Quality at Monash: Values and Principles (www.adm.monash.edu.au/cheq) and is based on the principles of ‘fitness for purpose’ and the quality cycle. Fitness for purpose suggests that the university as a whole and each of its areas identifies and defines what is to be achieved and is then judged on the extent to which these objectives are achieved. This is also the approach taken by the Australian Universities Quality Agency (AUQA). 

1.2  The quality cycle outlines a continuous process of improvement to achieve objectives. The elements of the quality cycle are plan, act, evaluate (monitor and review) and improve.

1.3  This report outlines activities in the area of quality over the past year and in terms of each aspect of the quality cycle. It reports quality initiatives of university-wide importance rather than achievements in particular faculties or service divisions.

2.  Plan

2.1  The Vice-Chancellor’s Group (Quality) continued its major role in conceiving and planning quality across all areas and aspects of the university. As the peak body for quality, VCG(Q) paid particular attention to planning for the 2006 quality audits in Australia (by AUQA) and in South Africa by the Higher Education Quality Committee (HEQC). 

2.2  Reports on planning and preparations for the audits have been made to many groups including VCG(Q), Council, Academic Board, Senior Management Committee and Monash Quality Network. A summary report on planning and preparation for the audits is attached (Attachment 1).

2.3  An important aspect of planning for audit was the commissioning by VCG(Q) of a ‘mid-term’ report on the university’s progress in implementing recommendations of the institutional self-review “Still Learning” 2002 (Attachment2). “Still Learning” made 33 recommendations, some of which were multi-part. The implementation of “Still Learning” recommendations has been monitored by VCG(Q) (itself a consequence of one the recommendations) with regular reports having been received over the last 3 years. In preparation for the AUQA audit, however, a ‘stock-take’ was thought necessary and this was undertaken by Ms Robyn Harris, newly appointed to CHEQ from her previous role as an AUQA Audit Director.

2.4  In brief, most of the recommendations made in “Still Learning” have been actioned; progress has been made on a number of others which are of a longer-term nature (eg matrix management, developing a sense of ‘belonging’, mobility); and, three recommendations were identified as not having been addressed. These were discussed by in detail by VCG(Q) and it was agreed that each remained valid and should be pursued. Action plans for addressing these recommendations were formed and will be progressed over coming months. The highest priority recommendation is that Monash “ensure effective use of data on academic standards for monitoring and improvement purposes” (Recommendation 27).

2.5  AUQA has been operating substantively since 2002; it is coming to the end of its first cycle of audits and is thus planning for its own development. An external review of AUQA has been commissioned by its Board for 2006. To feed into this, universities have been requested to provide comment on the possible form of Audit Cycle 2 and the Monash submission is attached (Attachment 3).

2.3  Act

3.1  As outlined earlier, the Vice-Chancellors Group (Quality) has major institution-wide responsibility for acting to embed the Monash approach to quality assurance and improvement in all areas of the university’s operation. http://www.adm.monash.edu.au/execserv/committees/vcg-quality/

3.2  The Monash Quality Network (MQN) came into being in 2005 as the merger of two former quality networks that had been in existence for a number of years, one academic and the other concerning support services. MQN was designated and has been acting as the reference group for audit preparations. http://www.adm.monash.edu.au/cheq/mqn/index.html

3.3  Within faculties, planning for quality takes place either in quality management groups with input from CHEQ (eg Business and Economics, Medicine, Nursing and Health Sciences, Information Technology, Engineering) or within other faculty structures.

3.4  A discussion paper applying the Monash Quality Cycle to committees of all kinds and at all levels was developed by CHEQ during 2005 (Attachment 4). Originating in the need to demonstrate committee effectiveness for audit purposes, the paper is of interest in that Council has led the way in attempting to evaluate and monitor its own effectiveness over a period of 7 or so years and this formed the basis for some of the suggestions contained in the discussion paper. It should be emphasised that the discussion paper represents an attempt to stimulate thought rather than suggest that each issue raised needs to be addressed. http://www.adm.monash.edu.au/cheq/quality/committees-quality-cycle.html

3.5  In October, CHEQ concluded a project with the Presidents of Monash University Student Associations to produce a Student Association Quality Guide. In-coming elected office bearers of Student Associations are required to learn rapidly about their roles and responsibilities and the Guide is aimed at assisting with this process. http://www.adm.monash.edu.au/cheq/resources/docs/student-assn-quality-guide.html

4.  Monitor

4.1  A significant advance in monitoring at the institutional level was the development of KPIs with associated targets and ‘traffic-light’ signals on performance. This represents a considerable step forward in the clear identification of institutional KPIs, their ability to be presented to internal constituents and their suitability to chart progress with regard to both internal and external exigency. Further information on KPIs for organisational monitoring is provided to Council under a separate agenda item.

4.2 The monitoring aspect of ‘evaluate’ is defined by shorter term measures to provide formative feedback undertaken by internal personnel and for internal purposes. Monitoring is often effected by surveys and a summary of core surveys is provided (Attachment 5).

4.3  The most important development in monitoring systems during the year was the implementation of unit (subject) evaluation across the whole university. This project, undertaken by a cross functional team of more than 60 people, involved the introduction of new hardware, software and the development of new business processes. The result is that every unit is evaluated by students at least once a year, with the results being published to the world wide web and thus being publicly visible. As approved by Academic Board, there are 8 ‘common items’ in each unit evaluation allowing for comparisons to be made of each unit with: the whole university, the faculty, the campus, the department etc. There are also 2 open-ended questions offering students the opportunity to make comments on their experience. The unit evaluation system affords the possibility for a more systematic approach to unit management; the identification of units that need to improve and units that in demonstrating ‘good practice’, will help inform improvement (see Attachment 6).

4.4  The Monash Experience Questionnaire (MEQ) which was first administered in 2003 when 17,260 current students completed the survey representing a response rate of 40%. MEQ05 has just been completed with 18,463 current students completing the survey to represent a 41% response rate. Reports for MEQ05 will follow the pattern for MEQ03 (ie there will be comparisons by faculty, campus, mode, student group etc) with the major difference being that the new reports will include time-series 03-05 comparisons.

4.5  Other monitoring surveys for graduated students include the Course Experience Questionnaire (CEQ), the Graduate Destination Survey (GDS), the Postgraduate Research Experience Questionnaire (PREQ), all of which are required by DEST. The Learning and Teaching Performance Fund will distribute a considerable amount of funding according to CEQ, GDS and other measures. In order to improve performance in the delivery of quality education, targets have therefore been set for performance on these measures. The internal Postgraduate Research Supervision Survey, which is normally undertaken every four years, was conducted in 2005. Reports on DEST survey data are presented at the senior planning conference early each year.

4.6  In the support services, a Service Level Agreement has been in operation since 2002. This was negotiated between the 10 faculties and 14 major central service providers (divisions) as an agreement for 66 service statements and 366 sub service statements, based on the activity based costing model. It represents a crucial way in which not only the costs of central services are determined and monitored, but also the quality of services. Support services publish their performance according to agreed KPIs on the www. During 2005 work has progressed towards a special SLA with the Gippsland campus, in support of the development of greater autonomy for the campus. There has also been initiation of differentiated campus service statements with respect to the Caulfield and Peninsula campuses (see: http://www.adm.monash.edu.au/cheq/sla/)

5. Review

5.1  Following on from the major institutional self-review “Still Learning” in 2002 and the Research and Research Management Review in 2005, it is anticipated that the International Self-Review” will report late in 2005 or early 2006.

5.2 As part of the International Self-Review, an independent view of Monash University’s approach to transnational quality assurance and improvement was commissioned to be undertaken by an independent expert. Monash has had a review process for transnational programmes for a decade through the Off-Shore Quality Assurance Committee (OQAC), the Global Alliance for Transnational Education (GATE) and recently, the Transnational Quality Assurance and Improvement Committee (TQAIC). Taking into consideration the independent external consultant’s report, a proposal for improving the roles and functions undertaken by TQAIC will be brought forward in the report of the International Self-Review.

5.3  In preparing for the HEQC audit of Monash South Africa, MSA is undertaking a campus self-review for completion in 2005. Monash University Malaysia undertook a similar campus self-review in 2004, which has proven to be extremely useful in preparing for the AUQA audit and in raising important issues which the campus is systematically addressing.

5.4  Apart from institutional, thematic and campus reviews, there is a requirement that each ‘element’ (eg as appropriate, each department, school, centre, faculty, division) undertake an Academic or Support Service Review at least every 5 years. Training workshops concerning the undertaking of reviews are provided by CHEQ, which also maintains a reviews database that is used by VCG(Q) to monitor reviews. There has been a growth in reviews over the last few years as illustrated below, which indicates that the annual level of review activity is now starting to approach that necessary for compliance with university policy (see: http://www.adm.monash.edu.au/cheq/reviews/ and  http://www.adm.monash.edu.au/cheq/reviews/database/)

Increase in Review Activity to Comply with University Policy

6.  Improve

6.1 Much of the improvement activity undertaken is at operational faculty and divisional level. However, a central ‘improvements’ website was established in 2004 to allow students and staff to see improvements made as a result of surveys. At this site, the actual survey can be accessed, together with analyses and reports, and improvements that have been recorded as a consequence. http://www.adm.monash.edu.au/cheq/reports/improvements/index.html

6.2  The improvements website also contains reports relating to two symposia organised by CHEQ during 2005 and relating to the major problem surfaced by the MEQ03 survey: the provision of useful and timely feedback. Statistical analysis of responses from approximately 17,000 students on 80 quantitative items (and confirmed in 46,000 open-ended comments), demonstrated that four of the ‘worst’ six items across all areas of the university were as follows (where 1 = the ‘worst’ or most negative feedback).

1. Feedback I receive on my submitted work is provided in time to help me improve.

2. The teaching staff put a lot of time into commenting on my work.

5. The teaching staff normally give me helpful feedback on how I am going.

6. Feedback I receive on my submitted work is useful.

6.3  The first symposium concerned good practice in providing timely and appropriate feedback to students and the second the establishment of systematic management processes to provide appropriate and timely feedback. Six faculties and a number of support areas contributed to the symposia; reports on the symposia and materials presented are available from the improvements website.

Attachments

• Attachment 1: Planning and Preparation for Audit (Word) 
• Attachment 2: Mid-Term Report on Implementation of Still Learning (Word)
• Attachment 3: Response to AUQA ‘Audit Round 2’ Discussion Paper  (Word)
• Attachment 4: Committees and the Quality Cycle (Word)
• Attachment 5: Monash University Core Evaluation Instruments (Word)
• Attachment 6: Unit Evaluation Project Update (Word)

Download this document in Word format