Skip to content

5 techniques to improve assessment entropy

December 14, 2009

The purpose of a work place assessment is to evaluate the student’s competency in a particular area in real working conditions. During the assessment, we have two primary concerns:

  • The assessment conditions are as real as possible, and
  • The student is behaving in a fashion that they would normally if this was as standard working environment.

Now these can be difficult to achieve, even if the environment is identical to those conditions the student would be subjected to post training. We can usually identify the events or environmental conditions that directly affect the realism and even though they are often beyond our control, we account for them in our evaluation.

It’s the subtle influences that we tend to forget about and ironically, these can have the biggest impact on the integrity of the evaluation. Why? Because we usually discount them altogether. What is most painful to learn is that we, the assessor, are usually the greatest contributor to these chaotic forces, often even before we start.

Here are five ways that you obstruct an assessment, before you begin.

  1. Be late. Be on time. Nothing says ‘my time is far more important than yours’ than tardiness. The impact can be devastating. While you delay, the student sits there pensive and uncertain. Is the assessment today? Did I get the timings wrong? Am I in the wrong place? Now even after you arrive, your rushing to get things back on schedule will influence the student; hey, they want to please you right, so they should pick up the pace as well.
  2. Neglect your personal care. This is a catch all area for all things about you that may be offensive to the senses of your coworkers. Sure, body odour can be off putting, but so can the smell of coffee on the breath or dirt embedded under the finger nails. Chew a mint, fire off an extra burst of deodorant and save the gym sessions until after work.
  3. Bring a conversation with you. Chatting can help the student to forget that they are being assessed, but excess unrelated conversation or continual witty retorts to other workers can alienate the student. First, they may feel like they are the only ones working (everybody else is chatting) and second, they may not feel that that have been fully initiated into the team (once they pass, then they can join in). Regardless of why, it will hinder their ability to concentrate or work with the rest of the team, even if it just to stop interrupting them.
  4. Bring the rest of your life with you. The assessment is now your primary focus. Unless you are needed to perform CPR on someone, don’t get involved with the machinations or politics of the workplace outside the realms of the assessment. That means no email, no phone calls, no quick words with the boss to keep the wheels turning, no marking papers, checking your diary or planning your weekend. ‘Multi tasking’ is another way of saying ‘diluting the time you are giving to the student’. You will miss something.
  5. Don’t prepare. Bring the appropriate documentation, writing implements and other necessary tools. Make sure that you are mentally ready as well. What happens if the student fails, is ill or finishes early? Are you ready to take over if necessary (this is a workplace assessment)? Is the rest of the process up to date (documentation, reports, assignments, written examinations etc). Have you reviewed previous training or assessment reports.

I’m pretty sure that we are all guilty of at least one of these at some time during our assessing career. It can be easy to be come so accustomed to the assessment process that it can be a little ‘ho hum’, but trust me on this, this is an enormously, huge deal for the student. Carelessly considered remarks, gestures and practices can, and will, be interpreted by the student and affect their performance. They have a lot more riding on this than you do.

Look at how you conduct yourself before, during and after an assessment and ask yourself two things; what impact do you have on the assessment process (including the student) and how do you reduce that footprint?

Next post, I’ll look at the assessment itself and propose some assessor technique (etiquette).


Download my free e-book THE WORKPLACE TRAINERS TOOL KIT Eight models for effective on the job training.

New e-book coming soon: “IMPROVE YOUR TRAINING FROM WITHIN” Using the ‘hot wash’ to refine your training program.

Leave no stone unturned

December 4, 2009

The general purpose of training is to achieve the best results in the least amount of time. This can mean leveraging the student’s strengths and previous experience to maximise the rate of development. This ‘leveraging’ can include classifying some of their previous successes as a foundation for the new skills that they will need to acquire. So does’ that mean that we can simply ‘sign off’ the skills and knowledge that they may have utilised some time in their past?

It’s not that I’m completely against the concept of recognising current competencies (RCC) or previous learning (RPL). Rather, that as a trainer, you must be aware of the limitations and the implications of using previous skills when assessing overall competency in the new role or task.

Here are some of the complications that we can face when using a student’s previous experience in lieu of training.

  • We forget: Over time we lose expertise in those areas that we haven’t recently practiced. Not only that, we tend to blur practices together and can end up with an alternative or blended view of what we should be doing in any given circumstance.
  • We lack practice / familiarity: Even if we retain the knowledge, our skills will degrade if not exercised  with regular use. We may not be able to achieve the same level of result or perform at the same rate that we had previously. The effect of this will vary given the complexity of the skill, the time since it had been used and how well practiced it was.
  • Testing can be inaccurate / representative: When we look at the actual competency itself, we may even be able to see flaws on how the student was initially designated competent. Was the skill used in isolation? In a sterile environment? On a limited sub-set of problems? Was it a single demonstration, not representative of the overall level of accuracy/precision over time? Examinations themselves only look at a student during a relatively short period.
  • Conditions vary: The actual conditions that the student worked under have changed, so the skills that they may have used with relative ease can call on completely different areas of expertise when coupled with the challenges of this new role. The environment itself is also a factor. Are the lighting conditions different? Is it now an outdoor role that was previously conducted indoors? Is there significant PPE or equipment variations? What about the level of autonomy the student has in the role? Some people can’t work with a supervisor/trainer breathing down the back of their neck, whilst others prefer a little guidance every now and then.
  • Ramifications can change: Unsurprisingly, the impact of the results on our environment can affect the effort and attention that we place on the tasks that we do. At the very least, moving into a safety or time critical environment can change the student’s perspective on the competency if not completely alter how they usually apply their efforts. This could also be the case when moving into a team environment or one with public outcomes.
  • Standards may change: It is not unusual in any field to find than an accepted practice, implemented successfully for years, is suddenly classified as ‘risky’ and altered. When a student changes their industry altogether, they may be questioning whether their existing skills actually fit in with the new position that they are undertaking. This can even occur within the same organisation, just by changing locations (interpretations of the same document can vary significantly between the assessors and trainers, especially in isolated locations or positions).
  • Techniques can evolve: Over time, we tend to find the best and most efficient means to complete our work tasks (this is what experience is all about of course). This is of referred to simply as ‘technique’ and although not necessarily laid down as a specific work method or standard, it is the best practice that has been honed to near perfection over trial and error. Thus a student who has a break, even a small one or is moving into a different area, may find the tried and true methods that they had applied earlier, not so welcome in this particual neck of the woods.

Now don’t ignore the student’s precious background  and by all means used their experience as a foundation for the new skill being taught, What I am suggesting is that care is taken when conducting a ‘needs assessment’ and that every competency is realistically considered, not just taken for granted. Trust me, the cost of going over something twice (I call that practice) is often far less than the consequence of overlooking it all together.


Download my free e-book THE WORKPLACE TRAINERS TOOL KIT Eight models for effective on the job training.

New e-book coming soon: “IMPROVE YOUR TRAINING FROM WITHIN” Using the ‘hot wash’ to refine your training program.

Broken links and training of my own

November 4, 2009

As you may have gathered from the silence, I have been absent for the past six months. This is the result of a career swing back into my original field of expertise – Air Traffic Control.  Unfortunately, this has incurred a six month or so stint of  ‘humility recalibration’ (or ‘refresher training’ as it is also known) and greedily eaten into my spare team.  After twelve years on the giving end, it is a invigorating reminder of how stressful life can be at the other end of the stick when your career is on the line.

For the trainers out there, please remember this.

I’ve started repairing the broken links that occurred through my failure to maintain the sister domain of  Training Tools. My apologies for any inconvenience this may have caused.

I hope to start hearing from you all again soon.

5 ways to reinforce negative behaviours

April 20, 2009

I was discussing with a colleague how procedures at our current job were less well known to the staff than the ones at my former job. Now both jobs involved an element of safety (one was actioning requests for emergency services, the other controlling aircraft). Failing to apply the correct procedure in either role could have dire consequences.

So why are the workplaces so different?

We came to the conclusion that it was a result of the ‘culture’ that had developed within the organisation and the perceived consequences of of failing to act appropriately. In particular, we identified five main mistakes that the organisation was propagating down the chain of management (remember, as far as an employee is concerned, the next level up is ‘the management’ and representative of the entire organisation’s leadership). These were:

  • Ignoring the product, unless a complaint was made
  • Displaying indifference unless their own performance was at stake
  • Condoning incorrect practices by not fixing them
  • Encouraging behaviours inconsistent with documentation
  • Enforcing procedures based on consensus, rather than performance indicators.

This is also relevant to workplace training, particularly in post-mentoring development where the student has achieved basic competency, but needs to develop further and hone their expertise. Hence, the same five mistakes occurring within this particular organisation can be applied to supervising any employee during their post-training development.

The five mistakes

Ignorance: Ultimately, behaviours are established through feedback. We will correct ineffective behaviours based on feedback from the experts and likewise, repeat those behaviours that have a positive impact on our performance objectives. When senior staff don’t assess our performance, or only investigate it when a complaint is bought to bear, there is no opportunity for feedback or a to establish whether our actions are the most appropriate for the given situation.

Don’t get me wrong, most of us can readily apply some self-assessment techniques, but often these are limited by our experience and our awareness of how our actions impact on the rest of the organisation and its objectives.

indifference: A worse problem occurs when we are being assessed, but are not advised of the results. An employee who continually makes mistakes that go unacknowledged, or worse, unnoticed, will begin to question the value of their efforts. Why place extra effort to achieve better results when mediocre performance is accepted. Typically, this will be applied equally to high performers who will then become discontented with the lack of acknowledgement for their extra effort.

I am regularly countered with the “But aren’t we all adults” argument and that we should all be trusted to work autonomously. Of course we do mean well and usually don’t ‘slack off’ just because we can, but we all have different standards and interpretations of what is expected of us

Condoning incorrect practices: When we or someone else makes a mistake and it’s taken with a ‘grain of salt’ or ‘laughed off’ without fixing it, the consequences of the actions leading up to it can be ignored. Additionally, the mistake mat not be understood by the person committing it. How would it be fixed? How can it be avoided or its impact mitigated? What else does it affect? Once again, The value is in the feedback – what behaviours do we reinforce, adjust or avoid?

Encouraging bad behaviours: Through your own actions or behaviours, you could be encouraging others to do something wrong. Do you cut corners, adjust procedures or use alternative strategies to those prescribed? Do you actually know the procedure or just ‘wing it’ using the benefit of experience or a wider range of knowledge? This may be an acceptable practice, but how will it be viewed (an implemented) by those with lessor understanding or skill.

What if everybody is doing it wrong and by doing things the right way causes problems with the the rest of the team’s performance?

Enforcing incorrect procedures: By failing to standardise practices, you can force someone to do the wrong thing. This commonly occurs when two supervisors like things done their particular way and this forces staff to adopt different practices, depending on who is currently running the show. No big deal right – what if one of them isn’t following the procedure to the letter of the law through ignorance, misinterpretation or ‘knowing better’? What does the employee do then?

What if everybody does it wrong (I know the book says this, but that’s not the way it is done around here)? This could mean that the documented procedure needs amending, but then who gets to decide which procedures are treated similarly and more importantly, how do I know which ones are affected?

Conclusion

These insidious behaviours may seem innocent enough in context, but often are an indicator of bigger problems. It is similar to the “Fixing Broken Windows” theory. Simple things like feedback, post training performance evaluation and recognising the efforts of employees, especially those still trying to find their place in the organisation, can go a long way to ensuring that team members are actually given the opportunity to perform.


Download my free e-book THE WORKPLACE TRAINERS TOOL KIT Eight models for effective on the job training.

New e-book coming soon: “IMPROVE YOUR TRAINING FROM WITHIN” Using the ‘hot wash’ to refine your training program.

10 tips for a more realistic ‘practice assessment’

April 8, 2009

This is part two to a post I made earlier in the week. In part one I discussed the reasoning behind preparing your student for evaluation by having them complete a practice the assessment before conducting the real thing. I also made a post on assessment preparation around 12 months ago, but only glanced over the concept of running a practice session.

Remember, the purpose is to have the student experience assessment conditions. At this stage of the training they should have demonstrated that they can do the job competently and all we are trying to do is avoid under-performance or ‘choking’ during the actual assessment.

Here are ten tips to make your practice assessments more valuable

  • Lead up to it: Stimulate the build up and anticipation for the student by giving them a few days notice. Use any training time to prepare for the assessment and refer to it regularly. Test theory regularly, focusing particularly on any weak areas.
  • Meet with the assessor: What are their pet hates, favourite question to ask, what style do they use and what constitutes an instant fail? Do they have any ‘patter’ that they use during the pre-brief? Do the sit back or scrutinise every action with an audible “hmmmm”.
  • Practice all components: Don’t just start at in the middles of the action scene – simulate the entire event. Start at the pre-brief, continue through to the debrief, with a question and answer session (for those areas not observed), and finally, give a report, using the actual assessment form or template. If the assessment contains a theory exam, give them one.
  • Make it as real as possible: The effectiveness of this technique will hinge on how much the student believes that these are the assessment conditions. Warn other staff to treat this like one. Invariably they will act a little different (more forgiving, less critical etc.) when one of their team mates is being assessed.
  • Be early or be late: Don’t just turn up on time. This will add to the apprehension, making their feelings closer to what they can expect on the day.
  • Consider an actor: If possible, have someone else conduct the assessment. Even if that person is only a trainer and not a workplace assessor, a new face can help simulate what it is like have someone different evaluate the student.
  • Don’t trainer, assess: Use the actual assessment form, stop the assessment if something goes wrong and don’t explain the best way to do things unless it constitutes a breach. Unless it is something that could cause harm to any participants or the organisation, let them dig themselves out of any trouble they encounter.
  • Give them a result: I know I said this in the third point, but give them a list of their shortcomings if unsuccessful. Brutal honesty is the key, but tactfully delivered of course because we don’t want to reinforce any unfounded fears.
  • Throw in an exam: If they have to do one, disregard this point (see point three), but if there is no associated theory exam, throw one in anyway, Keep them thinking about theory and make them continue to ‘hit the books’.
  • Induce pressure: If all is going smoothly, make something go wrong to increase the workload, just to show them what might happen during the assessment. This is an exercise in preparing for the assessment conditions. Once again, judge your timing and severity, we don’t want to be scaring them into a state of despair.

Conclusion

The purpose of practising for the assessment is to simulate the conditions that the student will be facing on the assessment day. You may feel that some of these techniques are deliberately making it tough for the student or sabotaging their chance of success.

The point is, this isn’t actually about performance (although as a secondary objective, this can be a useful tool for identifying student shortcomings). It is about having them practice what they have learned, under assessment conditions, so that it won’t come as a shock to them when they have to do it when it really counts.


Download my free e-book THE WORKPLACE TRAINERS TOOL KIT Eight models for effective on the job training.

New e-book coming soon: “IMPROVE YOUR TRAINING FROM WITHIN” Using the ‘hot wash’ to refine your training program.

Practising for assessment – Part one

April 6, 2009

The culmination of many training programs comes is the formal assessment. It’s not meant to be the climax of the event, instead just a regular day at the office to see how well you now fit into the daily routines of the workplace. All of the hard word should be over with; basic skills have been honed, advanced techniques, if not perfected are at least understood and can be applied as the circumstances dictate. You know what to expect and how to deal with it, otherwise your trainer wouldn’t be putting you through this right?

However, in reality, this is ‘the test’ that makes or breaks you (or so it seems at the time). This is where you must prove your worth to the organisation and show them that the efforts employed are about to give them a return on their investment. These are common thoughts for anybody that is about to face an assessment, particularly if it is the final competency evaluation that decides whether or not this is really the profession for you.

Apprehension is unavoidable; what if you are found wanting, what if the pressure is too great, what if circumstances conspire to turn the best that you have to offer into failure? What if you just choke? In an article by the Scientific American on Choking Under Pressure, the author explains that the problem of assessment under-performance may be contributed to the fact that we are to busy concentrating on our efforts to just get on with the job.

Why does this happen?

Choking, apprehension and our fear of failure usually stem from our lack of familiarity with assessment conditions. After a few years, these feelings subside as we become accustomed to the routine, especially in those roles where regular or constant evaluation occurs (e.g. aviation, emergency services etc). Of course, as soon as it comes to something new, we tend to return to ‘non-productive’ worrying over unfamiliar ground.

So if familiarity is the solution, preparation for assessment can improve the student’s knowledge, and hence understanding of the assessment process. It can also help by stimulating the emotions associated with the assessment conditions to give the student a sample of what they can expect to be feeling on assessment day.

Preparing for Assessment

A solution offered in the article above explains how to prepare for assessment condition:

“The best way to make a performance situation feel like rehearsal, says Raôul R. D. Oudejans, a psychologist at Free University Amsterdam, is to subject yourself to the same anxiety-packed conditions during practice that you expect to encounter during your moment in the spotlight.”

It then goes on with a interesting story about about a study done on the marksmanship assessment of Dutch police officers, which is worth reading.

Conclusion

The solution is to practice for assessment during the on-the-job training, which means taking the time to create ‘assessment periods’ for your student so that they can build up to and experience assessment conditions. This is a particular effective tool for newer students or those unfamiliar with this type of training and can immediately highlight some critical flaws in their preparation.

In the next post, I’ll offer some tips to make your assessment practice more effective.


Download my free e-book THE WORKPLACE TRAINERS TOOL KIT Eight models for effective on the job training.

New e-book coming soon: “IMPROVE YOUR TRAINING FROM WITHIN” Using the ‘hot wash’ to refine your training program.

Remember who you are talking to

April 1, 2009

Remember who you are talking to

In the military, it is important that you style your communication appropriately for the target audience. Now I’m sure that this is important in every job, but there are some things you just have take a bit more seriously in a military environment. For example, you can go to jail for insubordination (rarely would you, but it is possible), they use equipment designed to extinguish life and you must accept that in theory, every staff member can be sacrificed for the greater good of the nation. The three key components are style, tone and level.

Style is the language you use. Are you talking to a fifth grader or the company director, are they a novice or an expert returning after a hiatus or lengthy period? The difference can mean jumping straight to the point, laying down some background knowledge first or deciding to garnish up the bad news with a bowl of great, but potentially irrelevant news.

Tone is the direction and emotion that put into the meaning. Are you asking or telling, directing or berating, rewarding or condescending? What is your intent; fear, intimidation, guidance, assistance or just information? What is your mood? Our words are influenced by our emotion and although you are saying one thing, the underlying message or what is written between the lines may contradict your intent (unless that is your intent anyway).

Level is targeting the actual words that you use at the audience. Do they know the jargon? If they don’t, you won’t impress them with your vocational verbiage and if they do, you can come off sounding inexperienced. Do you know the technical terms? Bluffing will make it far worse and credibility in communication is essential if the listener is going to be swayed by your argument.

How does this work in training?

I overheard some training yesterday in a role that I’m familiar with The student fell into one of those ‘traps for young players’ and the trainer handled it rather ineffectively, requiring them to get things ‘back on track’ later on in the training.

The trainer gave the student feedback with one of the following statements;

  1. “You did it wrong again”
  2. “Don’t do that again”
  3. “If you do that again we are finished here”
  4. “When you do that, it sounds like we are finished with the customer – they may hang up prematurely and we don’t want that to happen”

Each statement suggests that they alter their behaviour. Each uses a different style and tone (level is about the same). Now your next question may be; well how far are they through the training and what do they respond better to?

Very valid questions because communication is about the audience. In this case, the student was still early in the training, day two of sixteen and customer control is what you would consider one of the advance topics because the student was still learning the system i.e. how to serve the customer.

My point is that when we are in a training environment, our responsibilities differ in such that:

  • We aren’t the boss, even though we may be in charge.
  • We do not know everything, even though we know more about this than the student.
  • They do not know nothing, even though they do not know enough about this.
  • They are human, even though we may feel like we are ‘deities’ in this role.

I’ve seen student’s treated like idiots, enemies or with complete indifference. Remember, these are our future co-workers and team members. Your performance may end up in their hands further down the track and if you choose use your position to bolster your confidence, authority or agenda, you may just find yourself working for them one day.

You can probably guess from my angst that the trainer had chosen number three.


Download my free e-book “THE WORKPLACE TRAINER’S TOOLKIT” Eight models for effective on the job training.

New e-book coming soon: “IMPROVE YOUR TRAINING FROM WITHIN”Using the ‘hot wash’ to refine your training program.