Assessing Learning

In my last post I talked about Analysis and some data pertaining to unconscious competence and the impact that that can have upon a business (a risk to health and safety, compliance and reputation). In this post I want to look at how we know learning has taken place.

Bloom’s Taxonomy has been the distinct tool of choice for the setting of measurable learning objectives in both training and teaching circles for many years. The below graphic shows the Taxonomy;

Blooms_Taxonomy_pyramid_cake-style-use-with-permission

(University of Arkansas, 2017)

We can use the taxonomy to measure and assess the level of employees/learners when they learn information or skills within the workplace and as such how well they are able to recall and apply the knowledge. In my experience of workplace eLearning, this generally involves reaching the ‘Apply’ level of the taxonomy. We want to see our employees applying the skills that they have learnt to their working environment.

The main issue with workplace training and Bloom is the need to train in behaviours to address unconscious incompetence within workforces. This, for me, falls outside the remit of what Bloom’s Taxonomy can provide for us. Whilst we can use Bloom to some extent, it does not address the need to assess that the learner is actually applying the learnt behaviours in a work environment.

It appears that Bloom may not be the best approach for workplace learning. Sugrue (2002) argues that there should be no taxonomy, instead “..learners will practice or be assessed on the particular performance in representative task situations.” (Allen, 2016) Allen’s analysis of this style of assessment divides knowledge into 5 areas, defined below with the information presented to the learner;

  • Fact – The fact is needed.
  • Concept – The definition, critical attributes, examples, non examples.
  • Principle/Rule – The principle/rule, examples, analogies, stories.
  • Procedure – List of steps, demonstration.
  • Process – Description of stages, inputs, outputs, diagram, examples, stories.

(Allen, 2016)

The key to this process is the assessment process. This is split into two sections, remember and use. For example, where the fact is concerned, the learner may be asked to recognise or recall the fact in order to show remembering. To show use they will be asked to recognise or recall the fact whilst performing a task.

For me Sugrue’s principles, as applied by Allen, make for better learning experiences that are less theory based than a typical Bloom-driven experience. The focus is clearly on the use of the knowledge gained in order to perform a clear, work-based task. I also believe that this is more measurable in the workplace as we can assess learning by either simply asking for recall of facts and processes or by observing the use of the process, be this in the workplace or in a role-play or scenario-driven environment.

References

Allen M, (2016), “Michael Allen’s Guide to E-learning”, Second Edition, Wiley.

Sugrue B, (2002), “Problems With Blooms Taxonomy. Performance Express, December. (Sourced from Allen, 2016)

Sugrue B, (2013), “A Learning Science Alternative to Bloom’s Taxonomy”, Learning Solutions Magazine, March (Sourced from Allen, 2016)

University of Arkansas, (2017), “Using Bloom’s Taxonomy to Write Effective Learning Objectives”,  https://tips.uark.edu/using-blooms-taxonomy/, last accessed 8/4/2017.

 

 

All About Analysis

Analysis can often be a boring thing that’s uninspiring to teachers, trainers and other learning professionals. It’s often seen as a box-ticking exercise that produces information that cannot possibly be useful (this was genuinely my belief as a sixth-form teacher!).

My move into eLearning development in a corporate environment has seen me looking at data in a totally different way. Owen Ashby in his talk on Business Aligned learning identified the following statistics;

In the workplace it’s identified that;

  • 30% of workers have misplaced confidence in their abilities.
  • 50% of workers have an immediate learning need.
  • 20% of workers are confident in their abilities and competent in their work.

(Ashby, 2017)

Ashby argued that, potentially, 30% of all workers in an organisation are a risk to Health and Safety, compliance to regulations and, ultimately, the reputation of a company. So how can this be addressed?

Many answers can be found in the field of Higher Education. Similar to the 30 and 50% of workers, JISC identify that;

“Learners themselves, particularly when beginning higher education, often have little idea of how they are performing in comparison with others, have gaps in prerequisite knowledge, and lack key study skills” (JISC, 2017)

This shows that Universities, too, have an issue with misplaced confidence and immediate training/learning needs. JISC also identify that;

“Adaptive learning systems are emerging to help students develop skills and knowledge in a more personalised way.” (JISC, 2017)

Adaptive learning systems could prove very useful for businesses. Learning needs could be ascertained during induction, this could then be used to register employees on an individual learning track that addresses any learning needs they may have.

Ashby (2017) encourages this, he identified that learning gaps need to be assessed and employees guided on how to access learning that addresses the gaps in their knowledge. Regular assessment can help to support the 50% of people who have a training need and increase the safety and compliance of the 30%, thereby reducing the risks to the company as a whole.

It appears that, in the world of analytics, Universities and Business could learn much from each other.

References

Ashby, O.; 2017; “Targeting Training to Tackle Unconcious Incompetence”; Learning Technologies 2017, Available at https://www.youtube.com/watch?v=wW9_nvUhfkg&index=25&list=PLouQOQiIxgr_pNAEFVO-jhmI1tVgshvdL

Jisc, (2016). Learning Analytics in Higher Education – A review of UK and international practice. [online] Bristol: Jisc. Available at: https://www.jisc.ac.uk/reports/learning-analytics-in-higher-education [Accessed 02 April 2017].

Theory into Practice

There’s a big dilemma in Learning and Development within business and that is, how do we ensure that our learners do what they have learnt?

Working within eLearning it seems that compliance is the main driver. Demonstrable outcomes are secondary to ensuring that staff have looked at a training package and passed a test or assessment to show that they are competent. But how do we ensure that our staff apply their learning in a meaningful and useful way?

Before I examine the solution, I will first define the problem.

Certainly within our learning structure (with some exceptions) learning events are seen as one-offs, within eLearning, for example, we release training, expect the staff to take it once and then apply the principles independently.

The issue with this model is that people forget, Will Thalheimer (2006) provides an example of what happens after training has finished.

forgettingCurve

As can be seen, once training finishes staff generally begin to forget what they have learnt. Thalheimer identifies that learning takes place very quickly, but then again, so does forgetting. This can provide real barriers to applying things that have been learnt, particularly if they have been forgotten.

So how can we address the forgetting curve?

Thalheimer identifies spacing learning events out in order to ensure that learners have their memory jogged at certain intervals to help them remember.

Julie Dirksen (2016) identifies that incorporating spaced learning with practice can allow learners to develop new practical skills. She argues that learning tends to be very information led.

“Most learning experiences are structured around lots of new information” (Dirksen, 2016)

She then puts forward an idea as outlined below;

“… allow the learner to acclimate and assimilate the information before moving to the next level” (Dirksen, 2016)

This approach is also echoed by others, Stella Collins refers to psychologist Herman Ebbinghaus when she states;

“When you read you forget 80 per cent of what you’ve learned 24 hours later. When you learn without meaning and without repeating the memories later you are very likely to forget most of what you learned.” (Collins, 2016)

The idea of repetition and spaced repetition is a common theme with all of these writers. So how can this be applied to help staff to apply what they have learnt to their day-to-day practice?

The answer seems to be in structured learning experiences that are incorporated into the working environment. Towards Maturity identified that, of companies identified as being within the “Top Deck” (i.e. identified as providing some of the best L and D provision in the country) 85% of them “.. agree that their approach is shaped by models that support learning directly in the flow of work.”(Towards Maturity, 2016).

In reference to the earlier posts I have made, I believe that the 70:20:10 model, integrated with Electronic Performance Support Systems and solid mentoring should allow learners to apply their theory to practice. We need to ensure that staff have access to on-demand learning that supports their working practice. I have found this area of L and D fascinating and am looking forward to researching further into this practice.

References

Collins, S; (2016); “Neuroscience for Learning and Development”; Kogan Page

Dirksen, J; (2016); “Design for How People Learn”; Second Edition; New Riders

Thalheimer, W; (2006); “Spacing Learning Over Time”; Retrieved March 24th, 2017, from http://www.work-learning.com/catalog/.

Towards Maturity; (2016);  “Unlocking Potential, releasing the potential of the business and its people through learning”; 2016-17 Learning Benchmark Report.

70:20:10

Further reflection this week, particularly pertaining to business-aligned learning. There is a great model for learning used worldwide by many businesses. It is known as the 70:20:10 model.

It states that;

  • 70% of learning is experiential, on-the-job learning.
  • 20% of learning is social and on-demand.
  • 10% of learning is formal and delivered through structured, taught sessions.

Interestingly this model is rather old (it was identified in the 1980’s) but, according to trainingindustry.com;

“One frequent observation is that while the model’s specific ratios do not reflect current learning opportunities, it remains generally consistent with the developmental experiences of many individuals. Thus, the model continues to serve as a valuable guideline on how to employ various developmental experiences.” (trainingindustry.com, 2017)

So how can 70:20:10 be enhanced for blended and online programmes?

Possible through clever application of technology to address the 20% and 10% with applications to provide evidence of the 70%.

Social learning (20%) can be extremely useful for learners. In fact Mcabe and Gonzalez-Flores state;

“Student-to-student interaction is one of the greatest advantages of online learning.” (2017)

For remote students social learning can really help to encourage reading and further enhance their understanding through communication with peers and tutors. This is highly evidenced through the course that I am currently studying on. I have felt encouraged and supported by peers and found readings that I would never have looked at unless there was a social element to the course.

10% of learning can be done through webinars, digital lectures and other formal means. The Learning and Performance Institute offer great webinars about conducting webinars and using online classrooms. They state that just uploading slides isn’t enough. Resources should be adapted and interactive.This is to address the fact that the 1080 group identified that 92% of people multi-task during webinars (LPI, 2017).

Finally the 70%. Really I only have academic experience in this area from a technical support point of view. Registering CPD and the like can be used to evidence the 70% but I have found that applying the theory gained in the more formal 10 and 20% sections has been the mainstay of my 70% experience. In fact I would classify this blog as part of my 70% as I am reflecting and applying theory in a less formal way.

From what I can see 70:20:10 is a model that will continue to be used. There was a heavy endorsement of its continued use at this years learning technologies conference, so I guess it’s here to stay.

References

McCabe, M. F.; Gonzalez-Flores, P; (2017); “Essentials of Online Teaching, A Standards-Based Guide”, Routledge

trainingindustry.com; (last accessed 20/03/17); “70:20:10 Model for Learning and Development”; https://www.trainingindustry.com/wiki/entries/the-702010-model-for-learning-and-development.aspx

 

Motivation and Behaviour

I am intrigued by the adoption of technology for learning. In my reading I came across the Technology Adoption Model (TAM) as proposed by Davis in 1989. Park explains Davis’ model as below;

“Two cognitive beliefs are posited by TAM: perceived usefulness and perceived ease of use. According to TAM, one’s actual use of a technology system is influenced directly or indirectly by the user’s behavioural intentions, attitude, perceived usefulness of the system, and perceived ease of the system.” (Park, 2009)

In motivating learners it is vital that we appease at least two of these areas of TAM with system design, namely perceived usefulness of the system and perceived ease of the system. If we can make a course site easy to use, relevant and useful to our learners we can then motivate them to adopt the system, from a technical standpoint at least. This can be done through user experience design and ensuring that items are on-hand for learners.

More importantly we need to appeal to the behavioural intentions and attitudes of our learners to encourage their adoption of blended and online learning.  Simpson (2013)  argues that “distance learners are particularly vulnerable to loss of motivation.” . Simpson also proposes that we can motivate learners by ensuring that they can say “I’m convinced this particular learning is exactly right for me” (Simpson, 2013, pp.78).

In my work I deal with compliance training for a business. The big question comes in “How can we make learners want to take compliance training?” How can we motivate them to engage with elearning that they have to do from a legal and ethical point of view? How can we get them to say that the training is right for them?

I have explained this before in other forums, but I believe that the use of “Authentic activities” is key to motivating learners to engage with elearning. Oliver et al’s (2006, pp.504) definition that “Authentic activities have real-world relevance” provides an excellent basis for motivating learners to see the relevance of their learning thus helping to shape their behavioural intentions and attitude.

Our adapted framework allows us to address some of the questions that unmotivated learners may have. I have adapted this from a JISC framework (2012);

  1. We answer the question “Why do I need to learn this?” – the most common question asked by our learners.
  2. We answer the question “What do I need to know?” – this is where we deliver any essential theory within the pharmacy context.
  3. We answer the question “How do I use this in my job?” – here we focus on the patients that may come into the pharmacy and how to offer advice.
  4. Practice – we provide a pharmacy-based scenario to allow the staff to try different responses on a patient and gauge reaction, this is usually done through an articulate storyline simulation. We have also performed face-to-face role-play activities with newly qualified pharmacists.
  5. Closure and Evaluation – The practice is reviewed and fed-back upon. This can then be written up as CPD or applied in the pharmacy.

This five-stage system allows us to address the burning questions that learners may have early on in a training context. We can then focus on embedding and applying the skills required for the job.

Whilst not entirely perfect (we still need to do some work to increase participation and compliance) I believe that this is a good launching point for discussions about motivating our learners to address the two more difficult aspects of TAM within an elearning context.

References

JISC, (2016), “Constructing the Learning Framework”, JISC Netskills

Oliver, R., Herrington, J., Thomas, R., (2006) “Chapter 36: Creating authentic learning environments through blended learning approaches” from Bonk, C., Graham, C, The handbook of blended learning: global perspectives, local designs pp.502-515, San Francisco: Pfeiffer.

Park, S. Y. (2009). An Analysis of the Technology Acceptance Model in Understanding University Students’ Behavioral Intention to Use e-Learning. Educational Technology & Society, 12 (3), 150–162.

Simpson, Ormond. 2013., Supporting Students for Success in Online and Distance Education. [online]. Routledge. Available from:<http://www.myilibrary.com?ID=428548> 23 February 2017

 

BOE Blog for MSc

Welcome!

This is a space for me to blog about my experiences during Module 2 of the Edinburgh Napier University MSc in Blended and Online Education (BOE). In addition to discussing this I will share some of my experiences of design and delivery of Blended and “eLearning” solutions.

Anything to do with my Masters Degree will be under the “BOE” category of this site. I will try to keep the content as organised as possible for ease of access!

The picture that I have tagged in this post? The University of Liverpool, a place for which I am grateful for inspiring me on this route in my education.