MDLS | ELECTENG291

MDLS Home

ELECTENG 291 Banner image

OVERVIEW

 Contact

Dr William Lee

Project Goal

Students increasingly find themselves in a risk-averse culture where the fear of damaging grades overshadows the need to develop resilience to failure. Consequently, learning often becomes superficial with the goal of avoiding, rather than supplanting, mistakes, and students, therefore, do not develop the ‘emotional capital’ (Cousin, 2006) necessary to navigate negative learning experiences that are becoming inevitable in an increasingly uncertain academic and professional landscape.

To help students develop the resilience to deal with their preconceptions, we wanted to create an environment where learning occurs through conversations about one’s mistakes. An environment where there are perhaps minimal repercussions for making technical errors, a mode of assessment where getting something ‘wrong’ is not necessarily penalised, and therefore a learning space with the affordance for students to examine their practices leading to those mistakes.

If we can provide the psychological safety for students to confront their preconceptions, we allow them to challenge, rather than accept, knowledge, to learn through discovery rather than by instructions, and to transition from a dependent learner to an independent thinker. Ultimately, our goal was to design coursework that minimises superficial learning driven by a fear of getting things wrong and maximises meaningful learning driven by a reflection of 'what am I doing wrong'.

OVERVIEW OF LEARNING OUTCOMES AND ARTEFACTS

Learning Outcomes and Intentions

The main learning outcome is to help students develop good reflective practices when examining technical ideas, no matter their validity, especially when those ideas are contentious. To do so, we wanted to

  • interweave these transferrable skills into a technical context by assessing for high-level (reflective) thinking using low-level (technical) problems, and
  • develop a framework that de-couples the assessment from the technical findings but emphasises the scientific practice towards justifying those findings.

The goal is to develop students to articulate their ideas and document their work transparently, honestly, and reflectively in a way that invites critique, discussions, and ongoing improvements.

We argue that any improvements towards these transferrable skills will inevitably lead to the 'correct' findings as a by-product. We, therefore, shift the assessment away from the ‘product of learning’ and towards the ‘process of learning’, so that the assessments assist the students in developing these good scientific practices.

Artefacts

The ‘artefact’ is a framework for the laboratory coursework with exemplars (from ELECTENG 291) and documentations that can be adopted and incorporated into other disciplinary contexts. It comprises:

  • A lab series consisting of five (fortnightly) labs to allow students to learn from their mistakes, provide ongoing opportunities to demonstrate the learning of those mistakes, and facilitate consistent exposure to good reflective practice.
  • A set of rubrics that focuses on proper scientific practices to facilitate conversations between the students and the teaching assistants during the labs and for the deliverables.
  • A collaborative practical tutorial (‘practorial’) that walks the students through the lab process to establish a safe space for learning for the remainder of the semester.

PROCESS

Co-Design Approach

Since the goal for this course was to build a community of learners where students and GTAs feel safe to test different ideas and learn from mistakes, we followed a co-design approach to collaborate and empathise with students better.

  • The teaching academic, Dr William Lee and the GTA, Dave Langdon, would meet with the learning design team on a weekly basis to analyse, brainstorm, and ideate how to improve the course further based on students' needs. 
  • The teaching team would hold regular weekly check-ins with all of the GTAs to receive feedback and empathise with them.

You can see our online collaboration work on Miro below. To open the board in a new browser tab, click here on this link Links to an external site.. You don't need to have a Miro account to view the below board, you have "View" access to navigate and zoom on different parts of the board.

ARTEFACT 1

Laboratory Series Design

Each lab considers the students’ existing understanding by examining a problematic idea from a relevant topic to prompt reflection. It is designed using a predict-observe-reflect cycle and comprises four categories of tasks: analyse, set-up, measurement, and reflection. Students are asked to analyse a concept, set up the experiment to observe and make measurements about that concept, and reflect on the observations to reconcile with, or extend, their analyses.

The tasks are purposed to draw out common novice practices pertaining to each category of tasks. The deliverable is a documentation (lab journal) of their work to which feedback is provided by the teaching assistants. The feedback emphasises the coherence, comprehension, and cohesion of the work with the aim of detailing ’how to do better’ rather than justifying ‘what you have done wrong’. This is framed in accordance with the supporting rubric, which establishes the ‘dimensions’ that constitute good practice for each category of tasks.

Here is an example of a lab in a Canvas quiz format with commentary outlining the design thinking and rationale for each stage of the lab.


Example lab design with commentary


Lab series design diagram

EE291 Labs Combined.pdf Download EE291 Labs Combined.pdf

ARTEFACT 2

Formative Rubrics

For each category of tasks (analysis, set-up, measurement, and reflection), we have identified three ’dimensions’ of ‘good practice’ which form the rubric. The criteria are deliberate and prompt conversations between students and instructors (teaching assistants) to think more deeply about the work (feedback) they present (provide). The dimensions focus on transferrable skills but are embedded within a general scientific context, it can therefore be cherry-picked and as appropriate for the tasks. It is important to note that the rubric is not designed as a checkbox exercise but serves as a conversation starter for students (teaching assistants) to engage in the process of learning.

Formative rubric structure for labs

 

Analysis Tasks

Every solution is an argument in miniature. An argument to convince the reader that the answer (i.e., the conclusion of the argument) to the problem is correct. The strength (hence, the validity) of an argument relies not only on its technical merit but also on its comprehensibility. A final answer is of little value if the mounting reasons are not communicated to, and understood by, the reader.

Students often mistaken "show your working" to mean provide a list of calculations towards some answer, but this omits the idea(s) behind those calculations. Submissions that require the reader to infer the intentions and conclusions behind the calculations, therefore, lack purpose and offload the critical thinking to the reader. This attitude goes against the spirit of good technical writing as it encourages a habit of 'brain-dump’ and undermines the importance of effective communication.

Dimensions: Preamble, Premise, Conclusion.

Reflective Novice-reflective Non-reflective
Preamble The opening clearly informs or justifies the approach or method used to investigate for a solution, and how the findings relate to the problem investigation. Some attempt made at conveying the approach for a solution to the investigation, but with gaps that requires the reader to interpret what is to be demonstrated. The work begins abruptly with minimal or no context or explanation. No identifiable intent is present that informs the reader what is to be demonstrated.
Premise The work follows the stated (or implied) approach from the preamble, with appropriate descriptions or explanations to aid the comprehension of the answer. The work follows the intended approach but may have missing connecting details between steps, or excessive details that disrupt the flow of the main argument. The work is presented with minimal or no explanations or descriptions, and may be illogical, abrupt, or incomprehensible.
Conclusion The final finding is clearly identifiable from work, and a clear effort has been made to reflect on its validity. Where appropriate, the answer is summarised with a statement that extends from the work. The final finding may be present but is not apparent and/or may not follow directly from the preceding work. Little or no effort is made to audit the validity of the findings. No attempt has been made to conclude the work. The work ends abruptly at some calculations and/or after some irrelevant descriptions. There is no way to reflect on the validity of the findings.
Setup Tasks

In order to test the hypothesis of the experiment, we require a system that captures the relevant phenomena from which we can measure via the system’s behaviour. The setup of the system (in this case, an electrical circuit) hence needs to accurately represent the intended experiment to be performed (it is faithful to the experiment), while being repeatable to allow for confirmation of results. The resulting documentation should be examinable; that is, detailed, but presented in a way that is relevant to the intention of the performed steps, which supports further discussion of the documentation.

Students often fail to account for many of the considerations when assembling electrical circuitry for the first time. The following rubric and prompts are an assessment of the student’s documentation of their experimental setup, and a guide toward getting students to think about the necessary steps and considerations of their setup. The prompts are general enough to be applicable to laboratories in other fields, which further encourages the student to consider how the prompt is relevant to their current task.

Dimensions: Repeatable, Faithful, Examinable.

Reflective Novice-reflective Non-reflective
Repeatable The experimental procedure is documented with appropriate detail such that the setup can be accurately reproduced. Some attempt is made at documenting the experimental procedure, but there is a lack of relevant details preventing an accurate replication of the setup. The experiment is unrepeatable from the documentation either due to missing important details about the setup or a lack of indication of what was set up.
Faithful There is clear evidence of awareness in the purpose of the experiment and appropriate steps have been taken to ensure the experiment is set up to align with that purpose There are some indications of actions taken to ensure the setup aligns with the purpose of the experiment, but with insufficient details to provide confidence in the data that would result from the setup compared to the experimental purpose. There is insufficient or no information to suggest that the setup is faithful to the purpose of the experiment. The documentation provides little to no confidence in the data that would result from the setup compared to the experimental purpose.
Examinable The documentation is a transparent, detailed account of the experiment. The included information has purpose and clarity, and invites critique and discussion based on the presented setup. The documentation is honest, being representative of the tasks and changes performed, but simplistic in its presentation and detail. The documentation is not transparent due to absent or hidden detail of the performed setup. Included information is presented unclearly and/or the setup shows minimal consideration toward its future use.
Measurement Tasks

Work in progress, to be completed (TBC). More content will be added soon to this section.

Dimensions: Context, Awareness, Relevance.

Reflective Novice-reflective Non-reflective
Context TBC TBC TBC
Awareness TBC TBC TBC
Relevance TBC TBC TBC
Reflection Tasks

Impartial: Being able to interpret measurements against the analysis in an unbiased way. Removing/detaching the performer from the expected outcome of the experiment, therefore being an impartial observer.

Reflect: Being able to make sense of the measurements against one’s own understanding. This demonstrates trying to reflect, rather than blindly accepting the results. This relates to the impartial dimension, in that it shows consideration of all aspects of the results (being unbiased).

Generalise: Generalising the experimental behaviour, possibly from mathematical calculations. Demonstrating understanding and application of relevant theoretical concepts in the context of the experiment, rather than relying on mathematical analysis alone.

Dimensions: Impartial, Reflect, Generalise.

Reflective Novice-reflective Non-reflective
Impartial There is clear evidence that the presented results have been evaluated critically, with consideration given toward the validity of the measurements in accepting the experimental hypothesis. TBC The results have been accepted faithfully, and hence there is no evidence of any evaluation of the results. No consideration has been given toward the validity of the results, and the hypothesis has been accepted/rejected with minimal reasoning.
Reflect The documentation shows critical analysis of the results and demonstration of relevant connections to learned concepts. TBC There is no analysis of the results present, and minimal progress made toward connecting the results to learned concepts.
Generalise There is clear understanding of the relationship between the observed behaviour from the experiment, and the related attribute(s) explored. Relevant existing concepts are utilised to expand upon initial observations to provide a generalised explanation of the observed relationship. There may be supporting mathematical detail, if required. TBC The relationship between the observed behaviour from the experiment and the related attribute(s) is accepted with no deeper insight present. There may be results present from mathematical analysis but no attempt has been made to relate these to the underlying relationship being examined.

ARTEFACT 3

Guided Practorial

The rubric on its own is a conversation starter, and so it requires initial unpacking, and the practorial serves that purpose. The first lab should be run as a guided tutorial, where instructors (teaching assistants) establish the lab process using an experiential cycle: getting the students to provide initial documentation of a category of tasks in light of the rubric (draw out novice habits), a developed set of question prompts specific to each dimension of the task category is then used to engage the students in discussion in light of their attempts (reflect on their habits), and asking them to re-attempt to document the tasks in light of the discussion of their initial attempt (conceptualise their new understanding). The practorial also sets the expectation for all future interactions (in the labs or deliverables), and the question prompts provide a guide to think more clearly about their practice for each of the categories of tasks.

 

Below you can see a few examples of different tools we tested to engage students with the prompts for a lab setup stage.

Setup Prompts - Padlet Task

In this task, we will walk you through how a lab session will run while asking you to answer and reflect on the prompts that we have designed for each stage of the lab activity. You are asked to work in groups and share your collective response to each prompt with the rest of the class using the following Padlet board:

Made with Padlet

Setup Prompts - MS Forms Task

 

QRCode for ELECT ENG 291- Setup Prompts (1).png

 


Setup Prompts - H5P Task

 

QR code for ELECTENG

You can scan the QR code to see an example of the H5P set-up prompts. You can also click here Links to an external site. to open the H5P activity in a new browser tab.

 


STUDENTS COMMENTS

Unsolicited Response Towards the Labs

Shown below are some of the unsolicited student feedbacks from the SET evaluation. Only comments addressing specifically the laboratory coursework are shown. The comments on various aspects of the labs highlight the iterative nature of the co-design process – this development is not an end in itself, but it, too, is an invitation for new opportunities around, on-going conversations about, and incremental refinements to, the lab framework.

ELECTENG 291 Labs - Students comments.pdf Download ELECTENG 291 Labs - Students comments.pdf