Thoughts from a recent GW4 meeting at University of Bath

 

On Friday 23rd March, Mike, Naomi, Robyn, Han and I headed over to Bath for the latest GW4 meeting of minds. As decided in the previous meeting, the main topics for discussion were e-assessment and portfolios, but we also discussed MOOC development and learning analytics. Unfortunately, no one from Exeter could make it up this time, so it was us from Bristol, along with colleagues from Bath and Cardiff. As before, we used Padlets to pool ideas and discussion points as we discussed in smaller groups.  

Portfolios 

Portfolios seem to be a common focus (dare I even say, headache). Bath and Cardiff have been using Mahara, and have been trying to overcome some of its limitations in-house. There was a strong feeling that none of us have found a portfolio which delivers what we need, and that if we ganged up on the providers they might be able to find a solution. The next step is to try to define what it is we do need from a portfolio, which tools we use (or have already investigated), and what we can do to find a common solution. Some immediate themes were e-portfolios as assessment tools (and how they integrate with current systems), GDPR implications, students being able to share parts of portfolios externally and internally, and how long students can have access to their portfolio.

MOOCS   

As something we all have experience of, to a greater or lesser degree, there was inevitably quite a bit of discussion around MOOCs. We talked about the processes we follow to develop MOOCs, and the different support we provide to academics. For example, Gavin from Bath showed us how he uses Camtasia to produce videos in house; in fact, he was able to knock up an example of such a video in 20 minutes during the session, with mini interviews and shots from the day. We also discussed the data we get from FutureLearn, and how we all find it difficult to do anything with that data. With so much information, and not much time, it tends to become something we’d all like to do more with but never quite find the time for. 

The discussion also retuned to an idea we’ve been kicking around GQ4 for a while, that of a collaborative MOOC. We discussed the idea of perhaps making courses for pre-entry undergrads, or students embarking on PhDs, or perhaps staff development and CPD courses for new academics (which Cardiff are already building a bank of in FutureLearn). The idea of creating small modular courses or MOOCS, where each of us could provide a section which is based on our own expertise and interests, was also popular…let’s see how this develops!

E-assessment 

Tools and systems around e-assessment was also a common theme. As well as thinking about Blackboard assignments, use of Turnitin and QMP, there was also talk about peer assessment tools and practice and adopting a BYOD approach. It seemed that we all had experiences of e-assessment being very mixed, with huge disparity in adoption and approach within our institutions. We’re all working on e-assessment, it seems, for example our EMA project, which is quite similar to that of Bath. However, other trials are also going ahead, such as Cardiff’s trial of ‘Inspera‘. I think we’re all keen to see what their experiences of that project are, as the Scandinavian approach to e-exams has often been heralded as the future! 

What next? 

For the future, we discussed more of a ‘show and tell’ approach, where we could get a closer look at some of the things we’re up to. There was also talk of upping our use of communication channels in between meeting in person, particularly using the Yammer group more frequently, and perhaps having smaller virtual meetings for specific topics.   

It wasn’t decided who would host the next session, particularly as Exeter weren’t represented, although we did tentatively offer to host here at Bristol. But, seeing as Bath really did set the bar high for lunch expectations – with special mention to the excellent pies and homemade cake – if we do host I think we’d better start planning the food already…!  

 

 

Reflections on the ABC mini-conference from Suzanne

Heading to London for the ABC mini event on Friday 9 February at UCL, I was a tiny bit apprehensive. This curriculum development tool was something I have used, in various forms, but without ever actually seeing how it should be ‘properly’ done, or ever receiving any training from Clive and Natasha, who came up with it. What I soon found was that our renegade use of the tool wasn’t in fact that renegade.

The morning session, where I got to actually try to develop a course using the tool, was pretty reassuring. It turns out I had actually been running the sessions ‘properly’ after all, which I would say is testament to how straightforward and logical the tools are to use.

After being on the other side of the table during a session, I learned how enjoyable it is to make such visible progress in such a short time. I also realised how much you have to remember if you end up talking through a whole sequence of learning without noting down the detail (ie, before you ‘flip the cards’). By the time we came to adding detail, we all had to try and remember what we’d had in mind. This is definitely something I’ll bear in mind the next time I run a session.

 

 

As well as the hands on session, hearing about what others have been using the method for, and what they had learned from it, was inspiring. The main things that stuck in my mind were:

  • How useful the method is as a review tool (as I had previously used it to design new courses). It helps people visualise and recognise all the great things they already do, before thinking about how they might want to develop their course for the future. The act of discussing it with others surfaces long held beliefs and assumptions which might no longer apply. When redesigning a course, unit or programme, I can see how helpful this might be.
  • Secondly, this tool is really effective at a programme level. The evaluation of individual courses or units seems to take on a new dimension when done in a room with all the units and courses in the programme being evaluated at the same time. Without asking people to do this explicitly, connections between units can be spotted and developed, duplication can be discussed, and people involved across the whole programme can start to get a real sense of what the students’ experience of the whole programme actually is. A ‘ground-up’ programme development seems to happen, which is more holistic and sustainable than a ‘top-down’ directive.

For our purposes, this certainly seems like a useful tool for two big projects that the University of Bristol is tackling: programme level assessment, and embedding the Bristol Futures themes into the core curriculum. Being able to quickly map where things already happen, and then talk about it in an open and positive environment, could be a really engaging way to get these conversations started. Let’s see where learning our ABCs can get us…

ABC mini conference – talk from Bristol

Notes from Suzanne Collins and Suzi Wells on using the ABC cards in Bristol. This talk was given at the ABC mini conference, UCL, London, 9 March 2018. See the ABC Learning Design web pages for further resources.

Suzi: Trialling ABC as a tool in workshops

I first came across the ABC curriculum design method while browsing UCL’s digital education pages looking for ideas. It immediately appealed. My background is in structuring and building websites, and I had used paper-based storyboarding in that context.

First trial: a single unit

Colleagues were enthusiastic and we started looking for contexts to trial it. An academic approached us with a view to involving us in significantly redesign a unit and we suggested the ABC approach.

As a tool for discussion, and for engaging a more diverse group of people – two academics, two learning technologists, one librarian, and someone else – it worked very well. They were very engaged and all could contribute. Although they couldn’t agree on a single tweet.

But we didn’t complete all the activities in the time. We also didn’t talk to them about how it should fit in to the overall development cycle and didn’t have much opportunity to follow up on what next. To me it felt like there was less value in talking about a single unit in isolation, that there would have been more benefit if we’d been working on a programme.

It was a useful tool and an enjoyable session but it wasn’t right yet.

Second trial: developing online courses

Not long after that we were asked to get involved in developing three online courses which would be promoted to our own students, as well as to the public more widely. Each course would be developed by a group of academics from a variety of different disciplines, many of whom had not worked together before.

The timescales were extremely short (by university standards). The academics involved were extremely busy with their existing work. These courses had to be innovative, transformative, cross-disciplinary, interlinked, approachable by anyone, essentially self-sustaining … and should encourage the development of transferable skills. No small ask.

Having pitched their ideas and been selected to lead or participate, the teams were assembled for an initial one day event. As part of this we ran several short sessions. We asked them to do an elevator pitch (they resolutely failed to follow the instructions on this). We also did a pre-mortem (imagine it’s a year down the line and these projects have been an absolute disaster, tell us what went wrong – very popular and a great way of surfacing problems and clearing the air).

We then ran an ABC workshop, with three tables myself and my colleagues Roger Gardner and Mike Cameron running a table each.

We modified the cards slightly to make them more platform-focused. We also added a time wheel to each week. Students would be expected to spend three hours a week in total on these courses and from conversations we’d had with the academics we knew that they were veering towards providing three hours of video a week (plus readings and activities). We wanted to focus attention on how students would spend their time.

We attempted to fit all this within an hour, because that was all the time available in the schedule.

For stimulating discussion, getting everyone to contribute, and shifting focus towards the student experience it worked well. The teams understood it and could work with it quickly. We were definitely over-ambitions about how much we could get through in an hour. Added to this, it was too early in the process and teams still had divergent or vague ideas about content (even on a big-picture scale) which couldn’t be resolved in a short time available.

One interesting finding was about the value of pushing people through the process. The other two facilitators used the framework and cards but took a more freeform approach, allowing discussions to run on. I was much stricter, pushing people through the activities. At the end of the day my group were the only one who asked to take the cards away and declared that they would use it themselves. Working through all the activities seemed to help people see the value of the process (though of course that may not mean that the discussion was more valuable).

Suzanne: Using ABC throughout online course design

My experiences of using the ABC method came later in the process of developing these online courses. My colleague Hannah O’Brien and I worked intensively with the three course teams, and we turned to ABC to help us do that. When we started, there were a lot of ideas, too many in fact(!), and we tried to find ways to get those ideas somehow on to paper, so that we could all evaluate them, and work them into a course design.

We ran a series of shorter, small group ABC sessions, using the modified cards from Suzi and Roger’s previous session. The courses were going to end up in the FutureLearn platform, so the course design by nature needed to work in a linear sequence of weeks of learning. In each week, we needed a series of ‘activities’, which were made up of different ‘steps’. Anyone familiar with FutureLearn can tell you that there isn’t a great deal of choice for what these steps are: a text article, a video, a discussion, a quiz, or a limited selection of external ‘exercises’.

What the ABC sessions highlighted early on for our teams was that having lots of video and articles explaining ideas might look jazzy, but is all very similar (and not very active) in terms of learning types. We all noticed there was far too much of the turquoise ‘acquisition’ happening in courses which were designed to develop skills such as communication and self-efficacy.

To help our academics come up with alternatives ideas for how students could, within the limits of FutureLearn, have a more interactive and challenging learning experience, we also created a bank of good examples, which we called our ‘Activity Bank’. As we worked to try and think of ideas for collaboration, or inquiry, for example, we could direct them to explore these examples, and adapt the ideas for their own purposes.

Overall, the ABC ended up being a useful tool to get everyone talking about the pedagogical choices they were making in a similar way. We could map the learning experience quickly and visually, so that we could prototype, evaluate and  iterate course designs. It also kept us all clearly focused on what the learners were doing during the course, rather than how amazingly we were presenting the materials.

Since then, I’ve found myself returning to the ABC tools and ideas regularly. The learning type ‘colours’ got quite embedded in our way of thinking and documenting learning designs. They cropped up in a graphic course design map created to demonstrate the pedagogical choices for the online courses (see below), and are now doing so again in a different context.

This new context, and the next big project for me is the Bristol Futures Optional Units. These are blended, scalable, credit bearing, multidisciplinary, investigative units, open to all students, around the Bristol Futures themes of Global Citizenship, Innovation and Enterprise and Sustainable Futures. So, no small ask, once again.

For this, the ABC cards have been tweaked again, this time to generate ideas for both online and face-to-face ideas for course elements, to allow for a flexible and student-choice driven learning experience. How can we provide a similar learning experience for students who might end up taking the unit in very different ways? We’re in the early days of course design, but I imagine that we’ll end up using the ABC workshops in various forms during the coming year!

In all, the ABC has become a bit of an ace up our sleeves. When we need temas to work more collaboratively, when we need the focus shifted back to the student, when we need to make progress rapidly and efficiently, even when we come to evaluate learning design – the ABC tools seem to provide us with a way to talk, act, design, and iterate.

Reflections on the ABC mini-conference from Suzi

On Friday 9 March myself and my colleague Suzanne Collins made our way to UCLs London Knowledge Lab, round the back of Lambs Conduit Street, to attend a mini-conference on the ABC curriculum design methodology developed by Clive Young and Nataša Perović.

It’s something we’ve been using an adapted version of at Bristol for just over a year, so it was great to see Clive and Nataša in action at the masterclass, and to hear about the great work being done at Glasgow, Canterbury Christ Church and Reading.

Some useful points from the day:

  • Glasgow have been using an online tool to make an electronic version (and have templates available)
  • Canterbury Christ Church have used PowerPoint to create an electronic copy while the workshop runs
  • Other coloured stars have been added to make visible: places where they engage with the education strategy; developing employability skills; other priorities (identified by the course teams)
  • Who is in the workshop is critical. Do you have students? Library staff? A critical friend?
  • It’s not just us – everybody adapts the cards (sometimes they even change the colours).

During the morning session people talked about using the cards with students, to allow them to design the course. One speaker suggested using them with evidence of BAME / gender engagement (in different types of activity), to address the way the course works for different learners. It was great to see how quickly people picked up the idea and started taking it on as their own.

Lots of potential and positivity. I look forward to seeing how the network grows.

Notes from the recent GW4 meeting at Cardiff University

Last Friday, Han, Mike and I attended a GW4 event in Cardiff, where the main topics on the agenda were students as collaborators, and shared projects that we can embark on together.

The day started with brief updates from each team:

Cardiff…

  • Have a new space for learning and teaching experimentation
  • Are working on a Curriculum Design Toolkit, as part of which they are looking at unbundling content to work in different ways for different markets
  • Have a Learning Hub Showcase (http://www.cardiff.ac.uk/learning-hub)
  • Have funding that students can bid for, for teaching projects
  • Ran a Summer Intern Project – one of which focused on advice in how to use lecture capture
  • Had a blank course rollover with a new minimum standard

Bath…

  • Have a major curriculum design project upcoming
  • Are moving towards programme-level assessment rather than modular
  • Have new funding for staff and students to work together
  • Are championing a flipped learning approach
  • Have a placement student
  • Are working on a ‘Litebox project’ (http://blogs.bath.ac.uk/litebox/) where students create an environment where the whole University can learn about new and existing technologies for use in learning and teaching, and share their experiences of them
  • Are expanding their distance learning postgraduate numbers


During the second part of the day, we talked about students as producers. Splitting into small groups, we shared our experiences, challenges and tips for working with students on both accredited and unaccredited courses. It was widely accepted by the group that collaboration with students is mutually beneficial. Students are able to move from being passive consumers of knowledge to genuine partners in their education, and we as professionals have a lot to gain from the expertise, connections with other students, and knowledge of life at the university that students can offer us.

The experience of working with students a Bristol, Bath and Cardiff has been positive but limited. All three universities have hired student interns in the past, but would like to do more in terms of making ‘students as producers’ a key underpinning concept in accredited courses. The expectations of ‘what university learning will be like’ puts a dent in the willingness of students to engage with accredited collaborative projects. We discussed how students may see universities as institutions of teaching rather than of learning, particularly as  tuition fees have risen, and expect more teacher-to-student time for their money. Our group talked about introducing the idea of innovative learning techniques earlier in students’ degree programmes, and even on pre-university open days, in order to change the expectations of students from traditional lecture-based learning to problem-based modules and more.

For the last part of the day, we talked about projects that the GW4 could collaborate on, and contributed to this padlet board. We all shared ideas, then each of us cast three votes for the projects we’d like to see most. A common theme was the sharing of knowledge and expertise in areas like FutureLearn, ePortfolios and case studies. We also talked about working together to put pressure on companies or to bid for shared funding in order to improve practice in ways that wouldn’t be possible for a single institution.

Bath have volunteered to host the next meeting in February or March, in which we’ll talk about ePortfolios and assessment.

Games and Simulation enhanced Learning (GSeL) Conference

GSeL Logo - 8bit style graphics.Last Thursday I caught the 8.44am cross country to Plymouth to attend the first Games and Simulation enhanced Learning (GSeL) Conference. GSeL is a newly formed interdisciplinary research theme group, part of Plymouth University’s Pedagogic Research Institute and Observatory (PedRIO).

VR Hackathon

Plymouth University 2nd November 2017

The main event was on Friday the 3rd, but a session billed for the previous day – ‘Hackathon: VR for Non-programmers’ sounded promising. So, I ventured down a day early to channel my inner geek. I’ve got a basic (but rusty) understanding of coding so hoped that the ‘non-programmers’ tagline was true. Turns out the session was well designed for those with little to no experience. Michael Straeubig expertly guided around 15 attendees with differing skills through the process of creating a simple VR equivalent of ‘Hello World’ over the course of 2 hours.

The Hackathon was a hands on workshop running through downloading a-frame framework template project files from Michael’s github, installing open source software atom.io for editing/coding.

Sounds complicated? Yeah, sort of – but Michael’s laid-back-whilst-enthusiastic delivery helped fill in the gaps and moved at a steady pace we could all keep up with. He guided us through creating our first scene, adding in various 3-dimensional objects, altering their size and colour. Setting up a local server on our laptops via atom.io, we were able to move beyond viewing the 3D space we’d programmed and view it on a pair of budget VR goggles (Google Cardboard) on our smartphones.

It was a great primer for dipping toes/feet/legs into creating simple VR spaces from scratch using free tools. The a-frame project files supplied had additional examples of how to extend and develop. I don’t mind admitting I spent a large part of the rest of the day tinkering. A Michael drily observed during the session, we’d become ‘cool coders’.

Games and Simulation enhanced Learning (GSeL) Conference

Plymouth University 3rd November 2017

A day of talks and workshops based around the use of Games and Simulations, both real-life and digital. Some personal highlights for me included:

Professor Nicola Whitton’s keynote ‘Play matters: exploring the pedagogic value of games and simulation’ which tapped eloquently into themes like Failing without Consequence and motivation/engagement through playing games.

Matthew Barr (University of Glasgow) ‘Playing games at University: the role of video games in higher education and beyond’ – a great talk about his work with ‘gaming groups’ and the benefits cooperative video game playing brought students. “If I ruled the world, every student would play Portal 2”.

James Moss (Imperial College) ‘Gamification: leveraging elements of game design in medical education’ – some brilliant examples of using scenario based games in medical education. ‘Stabed to Stable’ involved scenario/persona based learning, a horizontal whiteboard, post-it notes and pens, with students clustered around trying to map out processes (checks/actions) they needed to go through, whilst James periodically helped guide or threw related spanners into the works. An overhead time-lapse video showed a dynamic session in action. A second game involved teams becoming the ‘medical officer’ helping a team of characters climb Everest. This simulation included mountain noise recordings (incrementally getting louder), random wildcards presenting challenges, lighting changes and James squirting participants in the face with water.

Michael Parsons (University of South Wales) ‘Keeping it Real: Integrating Practitioners in a Public Relations Crisis Simulation’ – shared his experience running a real-time simulation for PR students. Students attempted to handle a recreation of the infamous Carnival Triumph ‘Poop Cruise’ in the University’s Hydra Minerva Suite. The simulation used news report recordings, archived social media posts and live interaction with actors via telephones over several ‘acts’ to simulate a PR teams attempts to handle a particularly disastrous voyage. It all went well till the passengers were close enough to land to get mobile phone reception (and access to social networks).

The conference presented a feast of examples of using games and simulations in teaching and learning. From creating crosswords to utilising digital badges to recognise achievements to data visualisation in Virtual Reality, the place was abuzz with ideas. The focus on the potential of play and gaming to engage students meant the event had something for everyone, whether die-hard techy or strictly analogue.

Designing and Evaluating Accessible Learning Experiences 

I attended an Accessibility in Education workshop in London last Wednesday (24th May).  Lisa Taylor-Sayles and Dr Eric Jensen presented two strands – design and evaluation.

Lisa’s presentations covered designing for accessibility and inclusivity. Her focus was on Universal Design for Learning (UDL).  Designing learning experiences fit for everyone, regardless of their needs.

She began with a potted history, WW2 veterans returning with disabling injuries. This led to changes in approach to town planning, infrastructure, and assistive technology. Later the principles of Universal Design developed into the Three Principles for UDL.

Read more about the Seven Principles for Universal Design here.

Read more about the Three Principles for Universal Design for Learning

Lisa gave practical advice on small changes that build to improve learning experiences. Simple steps such as using a tool to check colours benefit everyone. Rather than assuming you’ll retrofit for accessibility if needed, you increase inclusivity.

This approach dovetailed well with Eric’s strand which focussed on effective evaluation. He covered Formative evaluation, surveys and qualitative methods such as Empowerment Evaluation. Eric gave real life examples of where he’s used these approaches in his work (and what works and doesn’t).

Sessions and workshops alternated between the two strands, keeping it fresh. For me this also helped cement the need for evaluation needs to be core to any learning design. Would definitely recommend the event if it’s repeated.

https://www.methodsforchange.org/designing-evaluating-accessible-learning-experiences/

Learning Design Cross Institutional Network #5

On Friday 24th February, my colleague Hannah and I ventured up to Northampton University to attend the fifth Learning Design Cross Institutional Network (LDCIN) event. The LDCIN was formed in 2015, with colleagues from a number of institutions across the world taking place in discussions about learning design in education.

The event began with an introduction from Simon Walker, who heads up the Educational Development team at the University of Greenwich. He discussed the future of learning design; the increased interest with the introduction of the TEF, and the impact big data will have on how we design our courses, briefly touching on the report the Open University have recently published on data analytics and learning design (see below for more information).

Participants who had offered to give a ten-minute overview of their work were then invited to deliver. This session started with Natasa Petrovic, from UCL, who discussed her ABC model for learning design – a process my colleague Hannah had successfully used the day before for her Bristol Futures enrichment course! This model is becoming widely adopted as a method to develop course design, with participants only having to attend a 90-minute to 2-hour session for a complete overhaul of their module. More on this method can be found here.

We then had three further presentations from colleagues across the country. Fiona Hale from the University of Edinburgh presented their new model for learning design, which (she admits) very closely resembles the CAIeRO model, created at Northampton. Adele Gordon from Falmouth discussed their development as a learning design team, and how their focus was on employability above anything else – a method that will hopefully be increasingly adopted across the sector.

Finally, Tom Olney and Jitse van Ameijde from the Open University talked about their work on data analytics and retention-satisfaction. They have created a model for designing activities that ensure high retention and success (the ‘ICEBERG’ model) and have discovered some interesting trends. For example, students have higher satisfaction on courses where there are fewer collaborative activities, yet their ‘success’ (in terms of retention, meeting learning outcomes and grades) is lower. Similarly, more collaborative activities meant lower student satisfaction, yet much higher success rates. The report on designing for retention can be found here. This report will have the biggest impact where universities offer online-only courses, where retention is higher than on traditional courses.

These sessions were followed by a tea break (no biscuits provided!) and then a session from Jisc’s Ruth Drysdale, who posed the ‘wicked’ question of how to evidence the impact of TEL – a question that was best answered by Jitse van Ameijide, who simply said ‘You can’t – and shouldn’t.’ The impact of TEL should only be measured by learning success as a whole, rather than how technology has impacted on learning. This focus on successful learning rather than the impact of various technologies was a key theme throughout the morning, and potentially the focus of the next LDCIN meeting.

Next on the agenda was a session from our hosts, which asked us to answer the question ‘How do you solve a problem like Waterside?’ Waterside is a new university campus being built in the heart of Northampton but, unlike a traditional campus, Waterside will have no lecture theatres – teaching will take place online and via small-group or one-to-one tutorials. All course programmes (over 2,000 of them!) have to be redesigned to fit the new teaching style, which also means that the minds of all academics will have to be won over to face this new and radical change. We were tasked with deciding on the best way to motivate staff to engage with this strategy, thinking about five key areas: grassroots campaigns, community, strategic, faculty-level and research-based.

Our group created three models to engage staff with the new teaching strategy. The first, and least desirable, was a ‘top-down’ model, where senior management forced staff to engage with workshops to redesign their courses. However, this would not be a positive change, and would leave staff feeling demotivated and uninspired. The second was a ‘hand-holding’ approach, where a great deal of resource was added to the learning design team to ensure each academic had a bespoke session to redesign their course, with a number of community groups set up to support staff and provide on-hand advice whenever it was needed. The final approach was champion-led, where each faculty had a self-elected ‘champion’ of learning design, who could create a buzz inside their faculty and be available to support staff at short notice. Technology Enhanced Learning has been engaged with most in departments that have appointed learning technologists, and we believe this model is key to success, especially when it comes to changing culture and mindset.

Unfortunately, we had to leave after this session, but judging from the Twitter feed the afternoon was also a success, with a workshop from Edinburgh’s Fiona Hale on mapping learning activities and anther session from the Learning Design team at Northampton on evaluating learning design support.

In an ever-changing sector, it is essential colleagues working towards similar goals come together to share their experiences, methods and thoughts. I was especially inspired by colleagues at Northampton, who are leading the way in terms of a blended approach to education – I can’t wait to see how the challenge of Waterside works out. The LDCIN will be meeting again in the summer to discuss this and other projects taking place across the network. To keep up to date with the latest LDCIN updates, click here.

Assessment and Feedback: Transforming Curricula and Assessment in HE

On Thursday 2nd February I attended an event at the University of Bath entitled “Assessment and Feedback: Transforming Curricula and Assessment in HE.” There were many interesting sessions , of which the following were some of my personal highlights.

Dr Alex Buckley from the University of Strathclyde spoke about their use of the TESTA  approach to reviewing assessment at a Programme level. Funded by the HEA, the project, standing for Transforming the Experience of Students Through Assessment,  originally involved four partner institutions, Bath Spa, Chichester, Winchester and Worcester.  It is now used in over 50 universities in an attempt to address deep challenges of assessment which Alex stressed need to be considered at Programme level. 

The TESTA website contains further information and resources on this approach, including a manual on how to implement it. Alex explained that it involves triangulating data from a programme audit, an assessment experience questionnaire for students and focus groups.  At Strathclyde those programmes who have engaged with TESTA have found it an extremely useful diagnostic tool as well as helping colleagues to think differently about assessment. After the process, programme teams have a workshop with educational developers where they consider practical changes that can be made which address the TESTA findings.   The TESTA website contains case studies and best practice guides with concrete suggestions. An example is reducing reliance on formal documentation to communicate standards, and putting greater effort into providing exemplars in order make explicit, and open to discussion, the meaning of assessment criteria and enable to students internalise these through marking exercises and self and peer assessment in relation to the exemplars.

Kay Sambell from Edinburgh Napier University expanded in the afternoon on Alex’s point that we need to facilitate student engagement with feedback rather than simply flagging up when feedback is being provided. However both Kay and Jane Rand recognised that this can be easier said than done. Literature provided evidence of the effectiveness of this a decade ago, Jane said, but much practice hasn’t changed.  

Kay went on to show some practical strategies that can be used. Her work is based on the Assessment Standards Knowledge Exchange at Oxford Brookes,  which provides a range of useful resources. She also referred to the work of Winstone, Nash, Parker & Rowntree (2016) entitled Supporting Learners’ Agentic Engagement With Feedback: A Systematic Review and a Taxonomy of Recipience Processes, which emphasises the importance of “proactive recipience” of feedback.

Kay talked about dialogic use of exemplars, which can take different forms; complete or part of an assignment, authentic or re-created, annotated with feedback or not. She went on to give an example of a peer review workshop from her own practice, the process for which is outlined in the photo on the right. Students really valued this opportunity for extended dialogue around assessment criteria.

Kay also referred to the work of Nicol, Thomas and Breslin on feedback production being recognised as just as valuable for learning as receipt of feedback. She recognised that students are sometimes reluctant to engage with engagement activities (such as peer review)! However, when they do engage they find them extremely useful, and she has found that exemplar assignments are highly effective as “vicarious peer assessment”. Kay mentioned the work of Carless and Chan on managing dialogic use of exemplars. This contains analysis of how teachers can orchestrate dialogue around exemplars. They suggest in the paper that ” the dialogic use of exemplars should be a core aspect of teachers’ repertoire of assessment for learning strategies, in that the development of student skills in making academic judgements is fundamental to the university experience.”  This is a point often made by D Royce Sadler, well known for his work on conditions necessary for students to benefit from feedback (Sadler, 1989) . In his own teaching Sadler makes use of a version of exemplars in the peer review of formative writing his students do. He puts his own attempt at the writing task in with his students’ which are distributed and peer reviewed. Sadler describes this in more detail in “‘Opening up feedback: Teaching learners to see“.

References

David Carless & Kennedy Kam Ho Chan (2016): Managing dialogic use of exemplars, Assessment & Evaluation in Higher Education

David Nicol, Avril Thomson & Caroline Breslin (2014) Rethinking feedback practices in higher education: a peer review perspective, Assessment & Evaluation in Higher Education, 39:1, 102-122

Sadler, D. R. (2013) Opening up feedback: Teaching learners to see” ( In Merry, S., Price, M., Carless, D., & Taras, M. (Eds.) Reconceptualising Feedback in Higher Education: developing dialogue with students. (Ch. 5, 54-63). London: Routledge.

Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18, 119-144

Naomi E. Winstone, Robert A. Nash, Michael Parker & James Rowntree (2016): Supporting Learners’ Agentic Engagement With Feedback: A Systematic Review and a Taxonomy of Recipience Processes, Educational Psychologist

 

How can we use ‘unconferencing’ to enrich the Bristol Futures experience?

I went to ‘a conference on unconferencing’ (which actually turned out to be an unconference on unconferencing) in Birmingham on 20th January. Having no preconceptions as to what an unconference was, I went with the aim of gathering some ideas around enrichment activities for Bristol Futures Workstream 5.

The day started with a presentation from one of the organisers, Daniel King, on why conferences matter. He identified three main reasons:

  • Career development; networking; status
  • Field configuring; knowledge exchange; gaining a common perspective
  • Cultural management; learning how to act within a certain field

Conferences often work around a set agenda and hierarchy; the few talking to the many. The audiences take a passive role and the conference is a ‘man in suit’ affair, reinforcing existing power relations.

The idea of an unconference is to challenge visible hierarchies in conventional conferences, encouraging participation and inclusivity. The participants set the agenda, and little involvement or facilitation is needed from the organisers.

So how did it work?

We sat around in a circle, with post it notes and pens in the centre. There was an empty timetable on the wall, with locations on one side (‘middle bit’ ‘by the plant pot’ etc). Everyone was given the opportunity to ‘pitch’ an idea for a session, which involved writing a short description, question or discussion point on the paper, reading it out to the group and posting it onto the timetable (there was a bit of negotiation involved here, particularly where ideas crossed over and became one session rather than two). The person who suggested the session ‘owns’ it, and is responsible for kickstarting the discussion as well as typing up and sharing the notes afterwards. I pitched a session on how we can tackle the issue of invisible hierarchies within unconferences and how, despite the focus on inclusion, a lack of structure will invite certain forms of elite, such as those with social confidence taking over the discussion.

The sessions began, and everything went really smoothly! Before starting, the organisers had let us know that unconferences operate on a ‘rule of two feet’, which means that if you feel that you are no longer contributing or getting anything from a conversation, you’re entitled to leave it and join another session whenever you like.

How can this work for Bristol Futures?

The unconference format could work really well as a Bristol Futures Enrichment activity for all three of the online courses. The easy going, non hierarchical structure made for really interesting, balanced debates and conversations. Unconferences are designed to facilitate peer-to-peer learning, encourage less separation between different points of the hierarchy (from undergraduate through to academics), and have a focus on experience and views rather than status.

We could hold an unconference (maybe calling them something other than ‘unconference’) for each of the three pathways, encouraging students to pitch topics for discussion. This could be anything related to the content of a course (eg. ‘how can an individual make an impact on a global level?’), the course design (eg. ‘why I didn’t think week two worked well’), or something related to the overall theme that the course has not covered. The students would experience the unconference as an enrichment activity and opportunity to connect with each other and collaborate in a meaningful way. For us as lead educators and learning designers,  the unconference format could be used not only as an enrichment activity, but also as a way of using student insight to inform future iterations of the courses and make changes where needed.

An unconference would require little organisation outside booking a room and providing stationary and simple guidance, as well as little resource in terms of facilitation, as students become facilitators through the unconference format. These events could be held to kick off the course run, as a ‘touch base’ point during the middle of the course run, to round off the courses in a meaningful and useful way, or all three.

The aim of the Bristol Futures enrichment courses is to equip students with the skills they need to be happy, well rounded, resourceful adults. Through participating in an unconference, students will develop many of the Bristol Attributes including:

  • Intellectual risk, through participating in discussions potentially outside one’s comfort zone
  • Active and self aware learning, through pitching suggestions and taking ownership of a session
  • Inquisitiveness and initiative, through discussing a topic and trying to find a solution
  • Collaboration, through working with others in the group to define and run the unconference
  • Influence and leadership, through motivating and directing others to invite effective contributions
  • Responsibility, through managing the sessions without too much input from the organisers

Unconferencing encourages students to interact in a respectful, innovative and democratic way and could make a really effective enrichment activity across all three pathways.