|
|
|
The CHEQ Survey Standards
The Centre for Higher Education Quality (CHEQ) provides
a number of services in the area of evaluation. These include designing,
analysing, and reporting on surveys in the areas of teaching, research and
research training and support services. Surveys may be for individuals
(e.g., MonQueST), for Departments/Schools/Centres/Units (eg Learning and
Growth Survey, Head of School/Department Feedback Questionnaire), for the
whole university (eg Monash Experience Questionnaire) and for national
reporting (eg, Course Experience Questionnaire, Graduate Destination
Survey).
In order to assure the quality of questionnaires and
surveys, a process has been developed where only those questionnaires and
surveys designed in consultation and with the endorsement of CHEQ will be
analysed and reported by CHEQ. This assures that all aspects of
questionnaire development are considered and all safeguards are in place,
prior to the administration of a survey. It represents a real benefit to
CHEQ clients in establishing the credibility of their questionnaire or
survey, and safeguards the reputation of CHEQ.
The steps in gaining the CHEQ Survey Standard are as
follows.
Steps
|
Overview
|
|
Survey
purpose |
Client
(eg, individual or group) contacts CHEQ to discuss the survey and its
requirements. Contact Dr. Sid Nair at Sid.Nair@adm.monash.edu.au
or see the CHEQ organisational chart.
CHEQ
staff will discuss whether an existing survey may be appropriate. |
|
Ethical
considerations including privacy legislation |
Discussion
with the client considering the ethical treatment of respondents in
the survey/questionnaire (eg, rights of participants). Where
necessary, clients will be directed towards the university ethical
approval process. |
|
Design
of Question/Item/Scale |
If
no current instrument is suitable, the client and CHEQ staff will
develop questions, items and scales for the new questionnaire. This
includes technical aspects including validity and reliability,
scanning of responses, inclusion of privacy provisions, reporting and
storage of data. |
|
Draft
Questionnaire |
A
draft questionnaire is produced by CHEQ and the client for
presentation to the sponsoring or endorsing body. Recommended
improvements are then incorporated. |
|
Trial |
A
focus group may be organised to test understanding of the
questionnaire and to obtain further feedback from potential
participants. Following the incorporation of comments, the survey
instrument is ready for administration. |
|
Administration |
Timelines
and process are agreed for administration and reporting. The
survey is administered as either paper-based (scannable instrument) or
web-based or both. There may be cost implications depending on the
type of administration. |
|
Analysis
and Reporting |
Data
is collated and analysed. A report is produced which may either show
simple data trends, or a more comprehensive analysis of the data.
Privacy
of participants in reporting of the survey is properly protected. |
|
Data
storage |
All
data analysed by CHEQ are stored on the CHEQ database. Raw data are
usually discarded after data capture and acceptance by the client of
the final report. Surveys utilising scannable questionnaires are
discarded immediately after scanning. There may be cost implications
associated with the capture of qualitative data. |
|
Refinement |
Post-hoc
refinement or fine-tuning of a questionnaire may be undertaken based
on reliability and validity data. |
|
Evaluation |
CHEQ
undertakes a brief evaluation of the client’s experience. |
|
Completion |
Both
CHEQ and the client complete the survey process by finalising the CHEQ
Survey Standard checklist below. (pdf) |
The CHEQ Survey Standard Checklist
|
No
|
Checklist
|
Tick
|
| 1 |
Survey purpose confirmed |
|
| 2 |
Existing surveys appropriate/inappropriate established |
|
| 3 |
Cost implications agreed |
|
| 4 |
Ethical considerations 9eg University, CHEQ) confirmed |
|
| 5 |
Questionnaires scales, items, questions developed |
|
| 6 |
instructions developed |
|
| 7 |
Privacy laws/statements in survey agreed |
|
| 8 |
Sponsors feedback incorporated |
|
| 9 |
Trial completed |
|
| 10 |
Administration process agreed |
| |
Paper based
|
|
| |
Paper based (scannable)
|
|
| |
Web-based
|
|
| |
Timeline for administration
|
|
| 11 |
Analysis and Reporting |
| |
Type of analysis
|
|
| |
Timelines for analysis and reporting
|
|
| |
Type of reporting (eg simple, comprehensive)
|
|
| |
Production of draft report for client
|
|
| |
Data storage
|
|
| |
Qualitative data
|
|
| |
Privacy of participants in report
|
|
| |
Placing of reports on CHEQ web (if required/recommended)
|
|
| 12 |
Refinement |
|
| 13 |
Evaluation |
|
| 14 |
Completion |
|
|
