Research Project Evaluation
Format MLA Volume of 1200 – 1400 pages (5 pages) The evaluation must include: Some notes from SACE on the moderation/grading process and general points: Stage 2 Research Project performance standards – Synthesis Evaluation Writing the Written Summary An example: Written Summary Comments from last year’s examiners report highlights some key issues and provides details about how to do your best in this external assessment: Evaluation of decisions made in response to challenges and/or opportunities (E2)
Assignment type : Research Paper
Description
Evaluation Instructions:
This portion of the research project is externally assessed (moderated)! It is worth 30% of the research project mark and covers 4 performance standards: S3, E1, E2 and E3.
Students:
• evaluate the research processes used
• reflect on the chosen capability and its relevance to themselves and their research project
• reflect on the research outcome and its value to themselves and, where applicable, to others.
• a 150 word written summary of the research project, research processes used, and research outcome (the summary provides background information for the assessors and forms part of the evaluation)
• an evaluation in written form; it can include visual material such as photographs and diagrams integrated into the written text
• a written assessment of a maximum of 1500 words (excluding the written summary).
• The evaluation is against four performance standards and is given a mark out of 30, SACE assessors examine your teacher’s grading of your work and confirm (leave it alone)/change this based on the level of work produced and the grade assigned to it (to ensure adherence to standards)
• E1 is given a mark out of 15, and E2,E3 and S3 are combined to give a mark out of 15 for a total of 30
• First and Third person can be used throughout both the outcome and the evaluation
• This piece of work should be a critical evaluation of the research and its processes, not a narrative/list of what you did – these responses can only achieve at the C grade level.
• It is best to structure your work into subheadings (provided A+ exemplar is a good reference for this)
• They are looking for analysis of sources – they want to know about reliability, credibility and bias and expect some evidence of thought about how and why you made decisions about your research and its processes
A S3 Clear and coherent expression of ideas. E1 Insightful evaluation of the research processes used, specific to the research question.
E2 Critical evaluation of decisions made in response to challenges and/or opportunities specific to the research processes used.
E3 Insightful evaluation of the quality of the research outcome.
B S3 Mostly clear and coherent expression of ideas. E1 Considered evaluation of the research processes used, specific to the research question.
E2 Some complexity in evaluation of decisions made in response to challenges and/or opportunities specific to the research processes used.
E3 Considered evaluation of the quality of the research outcome.
C S3 Generally clear expression of ideas. E1 Recount with some evaluation of the research processes used.
E2 Some evaluation, with mostly description of decisions made in response to challenges and/or opportunities specific to the research processes used.
E3 Satisfactory evaluation of the quality of the research outcome.
D S3 Basic expression of ideas. E1 Superficial description of the research processes used.
E2 Basic description of decisions made in response to challenges and/or opportunities specific to the research processes used.
E3 Superficial evaluation of the quality of the research outcome.
E S3 Attempted expression of ideas. E1 Attempted description of the research process used.
E2 Attempted description of decisions made in response to a challenge and/or opportunity specific to the research processes used.
E3 Attempted evaluation of the quality of the research outcome.
The purpose of the written summary (150 words) is to provide background information about the topic and nature of the Research Project for the external assessors.
Suggested contents:
• Research topic – title and reasons for choice
• Research processes – identification of main activities
• Chosen capability – brief description/explanation
• Research Outcome – brief description/explanation
Research Topic: Produce a design and work plan suitable for a mural outside the local swimming pool.
I really like murals and am interested in how they are both designed and commissioned. I wanted to put a proposal to the council about putting a mural at the front entrance of the new swimming pool.
I researched murals and kept records about their similarities and differences; location, themes, size, materials, colour, artist’s name etc. I also read a number of magazines and articles and talked to my art teacher about design techniques and materials. I interviewed three people who had completed commissions to produce murals in Australia, talked to quite a few councils who had supplied the money for public art and talked to the manager of the local swimming pool.
My chosen capability was personal development and I demonstrated my initiative, creative abilities and understanding of notion/construction of identity. All of these things, combined with a growth in self-confidence and planning/problem-solving skills, have emerged as relevant aspects of my capability during my research project.
My Research Outcome consists of a work plan which includes a design, together with suggested materials, construction details, and costs.
Markers drew attention to issues regarding word count. Teachers are reminded that in addition to the 150 word written summary, there is a 1500 word maximum for the evaluation. The majority of students wrote between 1400 and 1500 words and included an appropriate summary. However, at times word count was difficult to judge, especially if the work was over the word limit and a word count for the summary was not provided. It would be helpful for a word count to be included after the written summary, or if students provided two separate word counts — one for the written summary and one for the evaluation. Some students did not provide a
150 word summary, or it was indistinguishable from the body of the evaluation. For others, it was unclear whether the word count on the cover sheet included the summary. Markers commented that a great number of students spent an overly large proportion (up to two-thirds) of the evaluation addressing specific feature E1, one-third addressing E2, and very little or nothing on addressing E3. This reduced their opportunities to achieve at the highest level.
Evaluation of the research processes used, specific to the research question (E1)
In 2014 there was a minor change to this specific feature, which required students to evaluate their research processes in the context of their specific research question. Students then needed to tailor their evaluation of a particular process to the specific question they had formulated, rather than resorting to generic judgments about the research processes.
Markers observed that in the most successful responses, students:
• gave a general overview of a process and then talked specifically about a source, providing concrete examples of reliability, credibility, and bias
• used a range of qualifiers to differentiate between the levels of usefulness
• used and named specific research sources and provided balanced judgments comprising both strengths and limitations of the usefulness, value, and reliability of the process, particularly in relation to their research question. In doing so, they showed understanding of how the usefulness of a process may vary, according to what question it is being employed to help answer
• provided reasons as to why a process was valid, reliable, or credible
• made clear links between the research process and its value to the research (valuable and credible information) and how it contributed to the increase in the student’s knowledge and/or the quality of the outcome
• clearly distinguished between the terms credible and reliable and did not use them together as though they were one word.
Less effective responses:
• demonstrated a misunderstanding of what is meant by research processes. Many students still used examples such as mind maps, visits to the library, journals, how they referenced, and how they used pens to highlight important information. As this has been a recurring issue in this subject, teachers are encouraged to provide explicit definitions of research processes that are more helpful; for example, that research processes are the activities undertaken that provide a student with evidence or data for their research
• provided very little about the validity of site or author, and made simplistic judgments about how findings were valid because they came from an expert, rather than linking this process to answering their research question
• made brief references to how useful the research processes were for them without explaining how or why
• named the research process, but focussed on what the student did, rather than how useful the process was to the research or what evidence or data the process provided
• relied on recount and diary-like discussion, often reflecting on how enjoyable it was for them, and how it helped them with their other subjects, rather than actual evaluation
• discussed why they did not choose a certain process — for example, a survey — rather than discussing the processes they had used.
This specific feature was new in 2014. It invites students to engage in a complex level of thought, forming judgments about the decisions they made in response to challenges and/or opportunities they faced. This proved to be very challenging for most students.
Markers observed that successful responses:
• identified a problem that arose, but mostly focused on the decision they made regarding the problem/challenge and based their judgment around the consequences of the decision in light of how well they progressed in their research as a result
• focused their evaluation primarily on the decisions made and their consequences, rather than just describing the problem or challenge
• critically weighed up the consequences of the decisions made and how these impacted on their research outcome; for example, whether the decision had a negative or positive impact on their research
• evaluated rather than recounted their challenges, and recognised the fact that when something goes wrong it can open up other (often better) avenues
• did not consider ‘time management’ as a challenge, unless time limitations particularly impacted on the research process
• focused on only a few challenges and decisions in detail
• used specific evidence to support assertions.
On the other hand, markers noted that the least effective responses:
• described or listed problems but ignored the decisions they made
• identified challenges and described what they did as a result, without identifying the ‘decision’ they made or weighing up whether it was good or bad
• discussed what went wrong or not to plan, often relating to time management or lack of motivation with no link to the decisions made
• contained lots of ‘could have, should have, and would have’ statements
• made judgments but did not provide reasons as to why these decisions/opportunities/challenges were useful/limiting to the research
• restricted their ‘challenges’ to issues such as poor time management, lack of motivation, wasting time, not doing homework outside of class, lack of responses to emails requesting information, losing USBs, catching up on missed lessons
• named the problem and then stated what they could have done, rather than showing the link between the decision and how this impacted on the outcome or led to another challenge
• for the most part did not even mention the words ‘decisions’, ‘challenges’, or ‘opportunities’. At best, they suggested other things they could have done, without examining consequences or implications
• stated that ‘this went wrong, this went right’, but made no explicit judgments about why and how this affected their research and outcome
• provided explanatory lists of what could not be done, usually in terms of lack of time or personal organisation
• shifted the blame, by complaining that requests for information or interviews were ignored or tardy.
Evaluation of the quality of the research outcome (E3)
This specific feature was changed to an evaluation of the quality of the research outcome, rather than a reflection on its value to the student and/or others, as in previous years. This required a shift in focus to the quality of the research outcome. When the research outcome was defined as the resolution to the student’s research question, this led to far greater insight and recognition of the student’s research in the context of the field of knowledge. However, when students focused on the quality of the mode — for example, the PowerPoint presentation or their spelling and grammar if written — their evaluation tended to be superficial and generic rather than providing an opportunity to demonstrate knowledge and understanding about research as an intellectual activity.
In general, markers noted that in successful responses, students:
• provided a realistic rather than an exaggerated evaluation of the quality of the outcome
• made it clear how the question was answered and how well, discussing strengths/weaknesses/limitations etc.
• provided several reasons for their judgments about the outcome’s quality, including its value to self and others. They then made balanced comments on the value of outcome to self but did not overestimate its importance to others, showing awareness that their research is not the greatest work on that subject ever undertaken
• provided specific discussion of how quality could be improved, linking this to processes rather than time management
• discussed how a range of sources backed up their key findings or discussed how some credible sources conflicted, therefore showing different views on the question
• based their judgment about the overall quality of the outcome after assessing the success in meeting the set objective/goal and critically evaluated the usefulness of the outcome
• gave a balanced view as to the quality of the outcome, including evidence produced, conclusions reached, and if they were able to produce new evidence.
Conversely, markers noted that the less effective responses tended to:
• exaggerate the work’s quality and impact, making claims such as ‘the world will benefit from my research’
• describe what form their outcome had taken
• make reference to the Research Project as a whole rather than the particular outcome, generically discussing how they benefited from completing the whole Research Project
• talk about how the outcome was written, what it looked like, and give general terms such as ‘it was great’ rather than a balanced assessment of strengths and limitations
• make little or no mention of the word quality; rather, many talked about how enjoyable it all was
• focus on the format rather than the findings; some research questions did not indicate the chosen format but then decided to evaluate the format — for example, a brochure
• make general statements about what they learned or repeat some of the issues already covered in specific feature E1.
Expression of ideas (S3)
This specific feature assesses students’ ability to express their ideas.
Generally this has been very well done over the years. Markers noted that students were most successful in this area when they:
• provided a generally sophisticated, coherently written document that tried to target the criteria holistically rather than in specific chunks; however, the use of subheadings worked well for some as well as key words
• were set out in a clear logical manner and clearly structured according to the three specific features
• made good use of vocabulary specific to the question, research processes, and outcome; also appropriately used terminology such as credibility, reliability, validity, and accuracy
• were logical, and had an easy-to-follow paragraph and sentence structure
• used headings that related to the current subject outline.
Features of the least effective evidence included:
• failing to separate the summary from the rest of the evaluation
• failing to use section headings, correct terminology, or new paragraphs to discuss a new point
• repeating information to reach the word count
• exhibiting poor organisation, repetition of material, lack of paragraphs, poor proof-reading, spelling errors, and excessively long and poorly constructed sentences
• using informal language with no clear structure or logical order
• revealing poor flow which made the meaning difficult to ascertain
• using incorrect words and expressions, suggesting poor understanding of some concepts such as validity
• applying incorrect headings or not using them at all
• utilising inconsistent syntax and sentence structure
• missing paragraphs
• repeating work — either retelling or explaining what was done rather than evaluating
• providing a very low word count.