Creating and editing KS questions

The KnowledgeSmart online skills assessment system offers a high degree of flexibility to administrators, in terms of the way they can customize their test library.

We work closely with specialist authors around the world, to create quality assessments for a range of design and engineering software titles.  These assessments can reliably be used 'out of the box', or, as we prefer to say, 'off the shelf'. (We don't actually have a box!).

But many firms prefer to edit the OTS content, to better match their own particular standards, workflows or project environment.  They also have tools to create their own test library material from scratch.

Let's take a brief look at the question creation journey.  We have built an intuitive, 5-step process, for writing and editing test questions.

To edit an existing KS question, first hover your mouse over the 'Edit question' button, in the 'Your Questions' page of your admin dashboard.

Note that a small pop-up menu appears to the left of the 'Edit question' icon.  This menu allows you to skip straight to the relevant step of the edit process, depending on the change you want to make.

Step one is where you enter the question name, a brief summary of the particular features being covered by the question and the question wording.

Step two provides you with the option of adding a file (or multiple files) to your task-based questions.  There are a number of file types which can be assigned to questions, for example, .dwg, .dgn, .rvt, PDF's, image files, movie files, and so on.  If you move questions around your library, from one module to another, the files move with the question.

Step three is where you input the answer to a question.  The fields vary, depending on the type of question, for example, free text, multiple choice, pick list, order list, or true/false question styles.

Step four is where you assign your meta tags, or keywords, to each question.  Category tags are used to help you search and group your questions, back in your library.  Training tags are used to help determine skills gaps amongst your users.  It is important to think carefully about which tags are assigned to your questions, as these will play an important part in your results analysis later on.

Step five is where you can view and edit the coaching notes for each question.  Coaching notes provide detailed feedback for users, on how to answer each question accurately and efficiently.  Coaching notes can be provided in a number of formats, for example; text only, text with accompanying screen shots, PDF’s with words & images, short movies, etc.

When you have finished creating or editing your questions, you can view a complete summary of all the key elements for each question, by clicking on the question name, back in 'Your Questions'.  This displays a green popup summary box - what we refer to as the question 'footprint'.

Here are some practical hints & tips for KS authors:

- A full question set is defined as follows; usually 18-25 questions, format to be determined by author, typically covering a range of core skills in a specified software application.

- A question set comprises live skills (task based) and theory (knowledge based) content. (Author to determine appropriate mix of questions).

- A task based question requires the candidate taking the test to have access to and use the software in order to elicit the answer.

- A knowledge based question can be answered without direct use of the software.

- You have 5 types of questions to choose from; Free text, Multiple choice, Order list, Pick list and True or False. Try to get a balance of different styles in your question set.

- When writing multiple-choice or pick-list questions, provide at least 5 answer options.  Try to avoid general answers, such as ‘All of the above’, or ‘None of the above’.

- Don’t write too many ‘True or False’ type questions – it’s too easy to guess the answers!

- Use screen shots and image files, or short videos, to enhance the effectiveness of some questions (i.e. PNG, JPEG, Camtasia, etc.).

- Think about the meta tags to be assigned to your qestions.  You will be making practical use of these tags when you analyze your test results and create groups and charts with your results data.

- Coaching notes should accompany each question. They are the author’s way of telling a user how they should answer a particular question accurately and efficiently.


Next, we'll look at the process for creating and editing your test modules.

R