Here are ten tips for writing assessment questions. Thanks to our authors for their feedback, in particular Nigel Davies and Paul Aubin.
1. Aim to assess skills not knowledge. Anyone can look up which tool to use in a help file, or on the internet.
2. Plan your assessment out properly first. Before you even start thinking about questions, be sure you know exactly what skills you are trying to assess and the structure of the modules. It’s no good just uploading a whole series of questions – you need to think about whether anyone could sit the assessment and miss out on a critical area altogether.
3. Write the whole assessment first in Word or Excel, or some other package where you can organise the questions effectively. Make sure you include a reference number or some other way of identifying each question later (and any associated files) the full question wording, all options and answers, and the type of question. This will help you when preparing the online content. (However, also see tip 10 below..)
4. Name your files the same as the question reference to which they relate. It will help track them later.
5. Look for a balance of question types, and vary the length of question to keep the candidates from growing restless or bored. Also use plenty of images to make the test experience interesting from a user perspective.
6. Try to avoid lengthy text-based answers as candidates will need to provide exact responses to obtain full marks. Where text fields are used, try to keep the answers to numbers or single words. When asking for numerical input, be clear to define whether you expect the units to be entered, and how many decimal places the answer should show.
7. Questions where the candidate has to put a list of instructions in the correct order are an effective way of testing their ability to use a tool properly. If they don’t know the answer when sitting the test, the question can be very helpful in increasing their abilities. Do not overuse this question type as they can become very lengthy.
8. One of the main strengths of KnowledgeSmart is the ability to assign files to a question. Avoid using multiple-choice, knowledge only questions wherever possible and instead consider how to assess the same skill by editing or interrogating a file. For example, instead of asking “Which variable controls the linetype scaling in AutoCAD?” consider asking “To what value is the linetype scaling factor set in the file?”.
9. Make sure your questions are completely unambiguous. It is a good idea to get a colleague to sit an initial draft of your questions before you upload them to test your wording. You’ll be surprised how many different ways a question you thought was clear can be misinterpreted!
10. Copy and paste all text into a text editor before copying into your online questions. It avoids any messy formatting being passed over from Word or Excel.
As we get up and running, and doubtless receive many more good ideas from our customers on helpful aspects of question authoring, we'll continue to post the best ones on the blog.
T-minus 8 days and counting..
Rory
Monday, 25 January 2010
BIM Report from McGraw Hill
As more and more AEC firms embrace BIM in earnest, it helps to keep abreast of the latest news on the subject. One of the best annual updates is the SmartMarket Report from McGraw Hill Construction.
This year's report focuses on the Business case for BIM. Click here for a PDF copy:
http://www.nibs.org/client/assets/files/bsa/mhc_bim_smartmarket09.pdf
Rory
This year's report focuses on the Business case for BIM. Click here for a PDF copy:
http://www.nibs.org/client/assets/files/bsa/mhc_bim_smartmarket09.pdf
Rory
Recession - a good time for listening!
So, the powers that be have decided that we're more or less out of recession now, have they? Doesn't really feel much different yet, truth be told, but it's fair to say that things are starting to gain momentum again. I was reading a discussion thread on the IoD LinkedIn forum last week about when things get 'back to normal'. Trouble is, we haven't defined 'normal' yet! I posted on the blog at the end of 2008 that we were planning for recovery to arrive around Q3 2010 - and nothing I've seen since has caused me to believe that that wasn't a reasonably accurate read on events.
Whilst it's still too early to start looking over our shoulder at the past 18 months of economic turmoil and draw hard conclusions, the simple psychological impact of seeing the new year swing around has definitely helped in many quarters. 2009 can be consigned as another 'Annus Horribilis' in the history books!
However, whilst certainly the financial repercussions of the recession were - and remain - a challenge, there was also a different side to the coin, at least as far as customer communication goes.
We found that many AEC firms in 2009, for the first time in a long time, had a bit of breathing space, at least from an IT admin point of view. Some much needed housekeeping was taken care of, coupled with some forward planning for the post-recession rebuilding phase.
During the course of many, many conversations last year, we were able to identify a number of service improvement ideas, by asking the question; 'What don't you like about our service?'. We frankly know what we do rather well, but it's the things at which we don't excel which are of the most interest. The niggles in our service. The bits our customers would like us to tweak, change, or even tear up and start again.
As a result, we went back to the (proverbial) drawing board (we are in the CAD & BIM world, after all! :) ) and started with a fresh look at how we could deliver future test modules for a host of design and engineering software apps (108 at the last count!).
And so KnowledgeSmart was born, as part of this process. Next month, we roll out the first phase of our new web-based tools, which will give AEC businesses even more insight into the skills base and competency of their teams. All existing 'Premier' customers will be upgraded to our new 'Enterprise' subscription, which offers a wide range of choices for knowledge capture and dissemination.
The months to follow will see a phased release of some new tools to help with the analysis and sharing of the newly captured performance data. Testing is already underway on a skills matrix tool, which will give HR and Training leaders a valuable insight into more focused learning, not to mention smoother resourcing of projects.
So, whilst the recession was (and still is) a hurdle on the overall journey to running an enjoyable and profitable enterprise, we are also, in a funny way, grateful that it came along when it did. It taught us how to listen again.
Rory
Whilst it's still too early to start looking over our shoulder at the past 18 months of economic turmoil and draw hard conclusions, the simple psychological impact of seeing the new year swing around has definitely helped in many quarters. 2009 can be consigned as another 'Annus Horribilis' in the history books!
However, whilst certainly the financial repercussions of the recession were - and remain - a challenge, there was also a different side to the coin, at least as far as customer communication goes.
We found that many AEC firms in 2009, for the first time in a long time, had a bit of breathing space, at least from an IT admin point of view. Some much needed housekeeping was taken care of, coupled with some forward planning for the post-recession rebuilding phase.
During the course of many, many conversations last year, we were able to identify a number of service improvement ideas, by asking the question; 'What don't you like about our service?'. We frankly know what we do rather well, but it's the things at which we don't excel which are of the most interest. The niggles in our service. The bits our customers would like us to tweak, change, or even tear up and start again.
As a result, we went back to the (proverbial) drawing board (we are in the CAD & BIM world, after all! :) ) and started with a fresh look at how we could deliver future test modules for a host of design and engineering software apps (108 at the last count!).
And so KnowledgeSmart was born, as part of this process. Next month, we roll out the first phase of our new web-based tools, which will give AEC businesses even more insight into the skills base and competency of their teams. All existing 'Premier' customers will be upgraded to our new 'Enterprise' subscription, which offers a wide range of choices for knowledge capture and dissemination.
The months to follow will see a phased release of some new tools to help with the analysis and sharing of the newly captured performance data. Testing is already underway on a skills matrix tool, which will give HR and Training leaders a valuable insight into more focused learning, not to mention smoother resourcing of projects.
So, whilst the recession was (and still is) a hurdle on the overall journey to running an enjoyable and profitable enterprise, we are also, in a funny way, grateful that it came along when it did. It taught us how to listen again.
Rory
Wednesday, 20 January 2010
CAD & BIM - State of the Nation(s) article
Happy New Year. Towards the end of last year we held a series of (taped) discussions with the guys at Evolve (http://www.evolve-consultancy.com) and Robert Green (http://www.cad-manager.com) , which culminated in a published article looking at the big picture of CAD training, CAD management, BIM, and a whole lot more.
Credit to Karen for her amazing editing skills and to Nigel for making a fair dent in his red wine collection, as the evenings wore on! :)
The results were published in a variety of places, but most recently, here: http://www.itshowcase.co.uk/News/2009-05-12/Is-your-business-smart-enough-An-interview-on-CAD-usage/.
So, we're finally out of 2009 (hooray!). 2010 is looking much more encouraging, with the economy starting to drag itself off the floor. 'Bout time!
February sees the rollout of the new KnowledgeSmart web-based testing tools. If you'd like a demo, drop a line to info@knowledgesmart.net, with 'DEMO' in the subject line, and we'll pencil you in.
Plenty more to follow over the coming weeks..
Rory
Credit to Karen for her amazing editing skills and to Nigel for making a fair dent in his red wine collection, as the evenings wore on! :)
The results were published in a variety of places, but most recently, here: http://www.itshowcase.co.uk/News/2009-05-12/Is-your-business-smart-enough-An-interview-on-CAD-usage/.
So, we're finally out of 2009 (hooray!). 2010 is looking much more encouraging, with the economy starting to drag itself off the floor. 'Bout time!
February sees the rollout of the new KnowledgeSmart web-based testing tools. If you'd like a demo, drop a line to info@knowledgesmart.net, with 'DEMO' in the subject line, and we'll pencil you in.
Plenty more to follow over the coming weeks..
Rory
Subscribe to:
Posts (Atom)
Posts
