Here is a short guide for KS administrators, which shares some tips from other KS customers, on how to prepare for a skills assessment rollout.
Communicate your plans
It is important to explain to users precisely why you are asking them to undertake a skills assessment. You might do this by including the assessment in a wider discussion about training needs and more efficient use of the overall CAD & BIM infrastructure within your company. This could include training needs, recruitment, induction and annual appraisals processes.
Improving technical software skills is a continuously improving process – the KnowledgeSmart tools simply highlight a likely starting point. As with most things, good communication goes a long way. An effective strategy is to hold a team meeting and explain your overall plans for improving CAD & BIM performance within the firm. This can include a Q & A session, where people have an opportunity to clarify any aspects of the assessment process – and understand how it fits into the bigger learning picture.
It is important that people don’t feel the results of their skills evaluation may be used in a negative context, particularly if they don’t perform well first time around. Ensure they understand this is a designed to be a learning process – and everyone has to start somewhere. If you have a CAD or BIM software upgrade scheduled, this can also feed into the process. By taking a snapshot of each individual’s current skills at a basic level, you can feed this data into your training plans moving forwards.
Evidence shows that better overall understanding of basic commands can also benefit the implementation of in-house Standards. Once you have explained to your teams the business case for measuring and improving CAD & BIM productivity, you need to set a timeframe for rolling out your assessments.
Prepare a timetable
You need to decide over what period of time you want to complete your assessments. This will depend on 4 variables; 1) the number of people to be assessed 2) the number of PC’s you can make available (with a licensed copy of CAD or BIM software and a network connection) 3) the number of offices in your organisation and 4) general day-to-day work pressures.
You need to decide what your preferred assessment environment will be. Some companies prefer their teams to use their own work stations, whilst others favour a more formal setting. In truth, there is no right or wrong answer to this question; you want people to feel comfortable and remove any potential for apprehension or pre-assessment nerves.
Experience shows that a formal assessment environment makes for a smoother rollout, where people simply take an hour or so at a specified time to sit their assessment. This way, they aren’t distracted by project work, email, ringing phones or interruptions from colleagues.
Use the invite tools in the KS dashboard to manage your assessment programme. For multi-office firms, it is best to start with one office and, when this office is done, gradually ramp up the level of activity until the assessment programme is complete. Principal admins will need the support of regional project technology co-ordinators if this is to be a success.
Prepare your team
We recommend you brief your teams prior to their assessments. You might like to share with them a simple Q & A document or present your plans at a team meeting. Here's an example of what other KS customers have done: http://the-knowledgesmart-blog.blogspot.com/2011/04/sample-ks-candidate-q.html.
Here's a brief user guide to taking a KS assessment: http://the-knowledgesmart-blog.blogspot.com/2011/04/taking-ks-assessment.html.
It is best to circulate the information at least a few days before people are due to take their assessments.
Send your invites
Use the invite tools in the KS dashboard to send your test invites. You can send invites individually, or add a list of users via csv import and invite them as a group.
Here's a brief blog article about sending invites: http://the-knowledgesmart-blog.blogspot.com/2011/04/inviting-users-to-take-ks-test.html.
Next, you may customise your welcome message in the invite mail and on the test landing page. You might wish to present users with specific instructions on where to locate their sample test files (if you have chosen to store them on your network, for example).
Here's a brief blog about editing your invite text: http://the-knowledgesmart-blog.blogspot.com/2011/03/customising-your-test-invite-text.html.
Complete your assessments
You can monitor your team's progress, as they complete their assessments. Admins receive an email with a link to test results, as and when they are completed.
You can use the invite history page of your dashboard to track invites and completed assessments. You can also re-send invites from here.
After the assessments
At the end of their assessment, users may receive an instant result and report with a breakdown of their score, training keywords and coaching notes. In addition, they can be sent an email containing a link to their summary report.
Account admins can access all assessment results, feedback, training keywords and coaching notes – for each candidate – in the KS dashboard.
Via the dashboard, it is possible to search results, view performance charts and export data into Excel for in-house reports. It is important to provide the opportunity for individuals to access a structured training programme, after their skills have been evaluated. The results will highlight areas where future training workshops can be focused for maximum improvement.
Here's an 8-step guide to better training needs analysis: http://the-knowledgesmart-blog.blogspot.com/2011/04/using-ks-for-training-needs-analysis.html.
R
Thursday, 28 April 2011
Wednesday, 27 April 2011
Sample KS Candidate Q & A
It is important for AEC firms to explain to users precisely why they are being asked to take a skills assessment. The key message is that the KS tools are an aid to identifying skills gaps and highlighting training needs.
Here are some sample notes that KS admins can copy - and adapt for their own use - to help give their teams a bit more background to the KS assessment process.
What is KnowledgeSmart?
KnowledgeSmart is a leading developer of skills assessment tools for CAD, BIM and engineering software applications – designed to help AEC firms measure and improve productivity. Since 2003, KnowledgeSmart has facilitated in excess of 20,000 individual assessments – across a range of architectural and engineering practices globally – helping to create a benchmark for basic CAD & BIM proficiency.
The KnowledgeSmart tools perform a comprehensive assessment of a user’s ability focusing on fundamental skills groups associated with design and engineering software applications, including AutoCAD, Revit, 3ds Max, Civil 3D, Rhino, SketchUp, InDesign, Photoshop, MicroStation and Bentley Architecture. The process aims to assess current performance in order to identify areas where individuals can benefit from additional training.
What is the point of KnowledgeSmart - and why should I do it?
This process is not about a big-brother style test to check up on unsuspecting staff! Nor is it a matter of pointing out who is the more knowledgeable software user. Rather, the point of the exercise is to highlight how much each individual currently knows – in order to identify those areas where focused training can improve basic knowledge.
Everyone uses CAD, BIM and engineering software in a different way. Some people use it full time in their role; others may only use the tools from time to time. The objective is not to teach a veteran CAD user how to use a polyline or write text. But the fact remains, a more detailed knowledge of basic commands and operations can alleviate apprehension in using technical software applications and assist in speeding up routine tasks.
Is it the same for full-time and casual users?
There are a number of different test modules to choose from, covering a range of skill levels and created by a team of specialist authors. Firms can also write their own test material in-house. For basic CAD skills, as an example, practices can choose from a 2D fundamentals module or an ‘occasional users' module.
The fact is, there are many differing levels of ability and many different requirements placed upon the way CAD & BIM software is used within each practice.
Summary reports take into consideration whether an individual is a core user, or someone who only uses the software occasionally as part of a wider role. Follow-up comparisons and training recommendations should be reflected accordingly.
What topics does the assessment cover?
KnowledgeSmart assessments typically break down technical software applications into a series of modular topics.
For example, the 2D fundamentals assessments (for AutoCAD & MicroStation) cover a range of basic CAD commands, including; basic object/element creation, layers/levels, blocks/cells, annotation, referencing & printing, UCS/ACS and preferences.
Assessments typically comprise 20-25 questions and present scores as a percentage, based on the accuracy of the answers presented. There are 2 question types and 5 question styles. Question types are either task-based (i.e. you need to perform a task using the software in order to work out the answer) or knowledge-based (i.e. you are being tested on your understanding of certain software functions, without having to actually use the tools). Question styles include; free text, multiple choice, pick list, order list and true/false.
What if I do not score very well?
This exercise is not about who is the best. You will identify areas where you can improve your own performance with the help of more targeted training workshops.
Following the assessment everyone receives an overall result and summary report with feedback on dropped marks, training keywords and coaching material.
When and where will the assessments take place?
Your technology administrator will have details of your assessment schedule. If you are unavailable to attend on the date or time scheduled please contact your administrator to rearrange. You will need a PC or laptop, with a web connection and a copy of the software you are being tested on.
How long will the assessment take?
There is no limit to how long the exercise will take. The average time is approximately an hour, but this is not a strict rule.
Try to work at your usual speed. Don’t rush, as many simple mistakes are made by people not reading the instructions carefully. Conversely, don’t take too long to complete each task, as your score plus time taken will be looked at in the overall results analysis. Your score will not be affected by the time you take.
How long before I know my results?
Depending on the level of feedback assigned by your administrator, you will typically receive your score straight away when you finish the assessment. You will also be emailed with a link to your results report and feedback on any marks which have been dropped.
Additional Info
The amount of time you take will not affect the accuracy of your score.
Before you start, you may be presented with a zip folder containing files which you’ll need to refer to during your test. Save the zip folder locally on your machine – and remember where you put it – as you will need to navigate back to this folder during your assessment.
Please read the written instructions carefully. This is also an assessment of your ability to follow instructions. Often, marks are dropped through candidates simply not doing what they have been asked to do.
You can answer the questions in any order, using the KnowledgeSmart 'question navigator', or simply read through the test instructions before attempting to answer each question. You can also edit your answers at any time during your assessment.
The time taken to complete an assessment varies depending on the individual, but an average is about an hour. You don’t have to complete all the questions in one sitting; you can exit and return later if you prefer. When you resume, your previous answers will have been saved.
Here’s a link to a KS blog article which tells you more about what to expect during a KS assessment: http://the-knowledgesmart-blog.blogspot.com/2011/04/taking-ks-assessment.html.
R
Here are some sample notes that KS admins can copy - and adapt for their own use - to help give their teams a bit more background to the KS assessment process.
What is KnowledgeSmart?
KnowledgeSmart is a leading developer of skills assessment tools for CAD, BIM and engineering software applications – designed to help AEC firms measure and improve productivity. Since 2003, KnowledgeSmart has facilitated in excess of 20,000 individual assessments – across a range of architectural and engineering practices globally – helping to create a benchmark for basic CAD & BIM proficiency.
The KnowledgeSmart tools perform a comprehensive assessment of a user’s ability focusing on fundamental skills groups associated with design and engineering software applications, including AutoCAD, Revit, 3ds Max, Civil 3D, Rhino, SketchUp, InDesign, Photoshop, MicroStation and Bentley Architecture. The process aims to assess current performance in order to identify areas where individuals can benefit from additional training.
What is the point of KnowledgeSmart - and why should I do it?
This process is not about a big-brother style test to check up on unsuspecting staff! Nor is it a matter of pointing out who is the more knowledgeable software user. Rather, the point of the exercise is to highlight how much each individual currently knows – in order to identify those areas where focused training can improve basic knowledge.
Everyone uses CAD, BIM and engineering software in a different way. Some people use it full time in their role; others may only use the tools from time to time. The objective is not to teach a veteran CAD user how to use a polyline or write text. But the fact remains, a more detailed knowledge of basic commands and operations can alleviate apprehension in using technical software applications and assist in speeding up routine tasks.
Is it the same for full-time and casual users?
There are a number of different test modules to choose from, covering a range of skill levels and created by a team of specialist authors. Firms can also write their own test material in-house. For basic CAD skills, as an example, practices can choose from a 2D fundamentals module or an ‘occasional users' module.
The fact is, there are many differing levels of ability and many different requirements placed upon the way CAD & BIM software is used within each practice.
Summary reports take into consideration whether an individual is a core user, or someone who only uses the software occasionally as part of a wider role. Follow-up comparisons and training recommendations should be reflected accordingly.
What topics does the assessment cover?
KnowledgeSmart assessments typically break down technical software applications into a series of modular topics.
For example, the 2D fundamentals assessments (for AutoCAD & MicroStation) cover a range of basic CAD commands, including; basic object/element creation, layers/levels, blocks/cells, annotation, referencing & printing, UCS/ACS and preferences.
Assessments typically comprise 20-25 questions and present scores as a percentage, based on the accuracy of the answers presented. There are 2 question types and 5 question styles. Question types are either task-based (i.e. you need to perform a task using the software in order to work out the answer) or knowledge-based (i.e. you are being tested on your understanding of certain software functions, without having to actually use the tools). Question styles include; free text, multiple choice, pick list, order list and true/false.
What if I do not score very well?
This exercise is not about who is the best. You will identify areas where you can improve your own performance with the help of more targeted training workshops.
Following the assessment everyone receives an overall result and summary report with feedback on dropped marks, training keywords and coaching material.
When and where will the assessments take place?
Your technology administrator will have details of your assessment schedule. If you are unavailable to attend on the date or time scheduled please contact your administrator to rearrange. You will need a PC or laptop, with a web connection and a copy of the software you are being tested on.
How long will the assessment take?
There is no limit to how long the exercise will take. The average time is approximately an hour, but this is not a strict rule.
Try to work at your usual speed. Don’t rush, as many simple mistakes are made by people not reading the instructions carefully. Conversely, don’t take too long to complete each task, as your score plus time taken will be looked at in the overall results analysis. Your score will not be affected by the time you take.
How long before I know my results?
Depending on the level of feedback assigned by your administrator, you will typically receive your score straight away when you finish the assessment. You will also be emailed with a link to your results report and feedback on any marks which have been dropped.
Additional Info
The amount of time you take will not affect the accuracy of your score.
Before you start, you may be presented with a zip folder containing files which you’ll need to refer to during your test. Save the zip folder locally on your machine – and remember where you put it – as you will need to navigate back to this folder during your assessment.
Please read the written instructions carefully. This is also an assessment of your ability to follow instructions. Often, marks are dropped through candidates simply not doing what they have been asked to do.
You can answer the questions in any order, using the KnowledgeSmart 'question navigator', or simply read through the test instructions before attempting to answer each question. You can also edit your answers at any time during your assessment.
The time taken to complete an assessment varies depending on the individual, but an average is about an hour. You don’t have to complete all the questions in one sitting; you can exit and return later if you prefer. When you resume, your previous answers will have been saved.
Here’s a link to a KS blog article which tells you more about what to expect during a KS assessment: http://the-knowledgesmart-blog.blogspot.com/2011/04/taking-ks-assessment.html.
R
Revit - 3 levels of Proficiency
Based on a number of similar discussions in recent months, we're introducing a subtle change to the labeling of some of the KS assessment modules.
I attended a web meeting this week with the Head of Model Development for a major UK construction client. Like many AEC firms, they are wrestling with the challenge of making BIM work for their business, whilst finding the cost savings, efficiency gains, and so on, that all big businesses have to make these days.
Looking back to the KS user group in London, a couple of months ago, a similar topic arose as part of the general discussion. That is, how do we identify someone who has basic skills in, say, Revit and distinguish them from an advanced user?
Further, do AEC firms actually need all of their users to be 'specialists' in Revit? The simple answer is no, in my opinion. In many instances, users just need to receive a file, open it up and perform some straightforward analysis of the model (without breaking anything!). For example, construction managers, PM's and the like. Then, for the general modeling team, AEC firms require a broader level of understanding for their wider user base. The people creating the detailed production information on a project. And last, firms need to have a handful of Revit 'specialists'. People with a deeper understanding of the way in which Revit is deployed on a project, an appreciation of the collaborative process and the impact of using the technology across a broader environment. And maybe, if content creation is a key part of their business, a handful of Revit Family specialists, who understand the importance of creating quality content, that can be re-used on subsequent projects, not simply re-modelled from scratch every time.
So it makes sense to establish 3 recognised levels of competence;
Level 1 – basic skills
Level 2 – intermediate skills
Level 3 – advanced skills
With respect to Revit, this will work as follows;
L1 – Basics of Revit (i.e. a test aimed at Construction Managers, PM’s, etc.)
L2 – Revit fundamentals
L3 – More advanced concepts of Revit, i.e. Revit project process, collaborative working, Family creation, etc.
AEC firms can use the assessments, as a means of demonstrating to clients (and prospective clients) that they have the skills required, to truly deliver on a BIM project. For example, all CM’s can be trained and encouraged to take the Revit L1 assessment (scoring X% or higher in Y mins or less). All Revit modelers can add the Revit L2 assessment to their CV (scoring X% or higher in Y mins or less). All Revit ‘specialists’ need to ace the L3 assessment(s). Anyone working on content creation, must tackle the families assessment. You get the picture.
Firms might also consider writing their own modules, based on their BIM Standards. All users can demonstrate an understanding of the correct standards, by passing this module.
Now, we've made extremely good progress in building our Revit assessment library. We already have the following modules in place:
- RAC 2010 fundamentals (Metric & Imperial) – L2
- RAC 2011 fundamentals (M & I) – L2
- RAC Xpress (M & I) – L2
- RST 2010 fundamentals (M & I) – L2
- RST 2011 fundamentals (I) – L2
- RST Xpress (M & I) – L2
- RMEP 2011 fundamentals (Mechanical) (I) – L2
- RMEP 2011 fundamentals (Plumbing) (I) – L2
We are currently working on the following new modules:
- Revit Project Process (goes live early May) – L3
- Revit Families (goes live late May) – L3
- RMEP 2011 fundamentals (Electrical) (I) (c.June) – L2
- RAC 2012 fundamentals – 13 modules (M) (c.June) – L2
- RMEP fundamentals 2012 (M, E & P) (I) (c.July) – L2
- RST 2012 fundamentals – 13 modules (M) (c.August) – L2
- RMEP 2012 fundamentals – 13 modules (M) (c.September) – L2
- Revit for occasional users (i.e. general content for PM’s, CM’s. Not discipline specific). (M&I) (c.September) – L1
In the Fall, we'll be adding these modules:
- RAC 2012 fundamentals – 6 additional modules (M) – L2
- RST 2012 fundamentals – 6 additional modules (M) – L2
- RMEP 2012 fundamentals – 6 additional modules (M) – L2
- Imperial data sets for all Revit 2012 content – L2
- RAC 2012 advanced (M & I) (i.e. harder test than fundamentals (L2) for core users) – L3
- RST 2012 advanced (M & I) (i.e. harder test for core users) – L3
- RMEP 2012 advanced (M & I) (i.e. harder test for core users) – L3
I think this will help AEC firms, Construction clients and contractors alike, to get a much better handle on who knows what, with respect to deploying Revit on a BIM project. It will also help firms to demonstrate their BIM credentials and give users a logical (and more appropriate) path for skills development.
R
Wednesday, 20 April 2011
Taking a KS Assessment
It's extremely important for AEC firms to explain to their users precisely why they are being asked to take a technical software skills assessment. The best firms always handle this well. As with all things in life, a little bit of good communication goes a long way.
This is not all about who is the smartest, fastest, or best Revit user. Or the most experienced MicroStation wizard. Although these are useful things to know. The real value in the KS tools is trying to help users, at an individual level, identify what they know and also what they need to brush up on next, in order to improve their skills, with a particular piece of software.
Over the past 8 years, we have facilitated over 20,000 individual test sessions, across hundreds of AEC firms. Experience has shown that, in general, people don't like the idea of taking a test - at least that's usually how they feel at first. However, experience has also shown that, for most people, when they complete their assessment, they realise that it wasn't half as bad as they thought it would be! Further, they often learn new skills as they go through the process of answering the test questions. And lastly, they find the test feedback and coaching notes extremely helpful, in giving them an unbiased appraisal of their skills using a particular software application - relative to their peers in industry. It's important that users have the opportunity to review their test scores and feedback with their team leader or BIM manager. And it's also important that firms explain to their users that all test scores are confidential. Best practice would recommend that only senior administrators should have access to firm-wide results data.
Provided firms explain to their teams that the main point of rolling out an assessment program, is to gather valuable training needs analysis data, rather than a big-brother style exercise in peeking over shoulders, then most users feel OK about the idea. And when firms follow through with more focused, modular training workshops, which target recognised skills gaps, then it's a win-win for users and firms alike.
So let's take a look at the user journey, when asked to complete a KnowledgeSmart skills assessment.
We'll assume the firm has already done the pre-assessment prep - and explained to their teams what is happening and why they are being asked to take an assessment.
The next step is for a user to receive their test invite. (Here is a separate article on how a KS admin sends out test invites: http://the-knowledgesmart-blog.blogspot.com/2011/04/inviting-users-to-take-ks-test.html).
Admins can set up test sessions straight from a browser, but most invites are sent by email. Either way, the user ends up in the same place. Here is a typical KS test invite:
The system assigns a username and password to each user. They access their assessment by clicking the unique URL at the bottom of their invite mail. This takes them to the assessment login page, where they will be prompted to enter their username and password. (If a user loses or forgets their password, they can hit the 'Forgot details?' link to receive a new one).
When they log in, they arrive at the main assessment landing page. Here they will see a summary of the test they are scheduled to take and any special instructions from their administrator.
There are 5 types of questions, which typically comprise a KS assessment. Let's take a brief look at each one..
Free Text
This type of question requires a specific answer, which needs to match the answer stored in the KS database. Look out for precise instructions on how to format your answer, for example, number of decimal places, case sensitive answers, accurate spelling, and so on.
Multiple Choice
Here, you will be offered a selection of possible answers - and you need to select the best one. Watch out for 'distractor' entries, which are designed to make you think before selecting your answer.
Pick List
This question type offers you a range of options, which could represent a correct answer. Choose the best answers from the list provided, using the check boxes, to the left of the answers. Partial marks are often awarded for entering some of the correct answers on pick list questions.
Order List
This question type requires you to place a number of items in the correct sequence. Drag and drop the answer options, until they are in the right order.
True or False
This question type is pretty straightforward! Read the statement or question and decide if you think it is true or false. Enter your answer using the radio buttons.
OK, so when the user arrives at the landing page, before they begin, they need to download the sample data files that they will be using during their assessment. There are a couple of ways they might have to do this. The usual way is to select the orange, 'Download files' button.
This prompts the user to save the files locally on their work station. They will need to navigate back to this folder during their assessment, so most users choose their desktop, or My Documents, etc. The data sets do not need to be saved during a test session, so they can simply be deleted after the assessment has been completed.
Sometimes firms choose to zip up all sample test files and host them locally, on their own servers. If this is the case, then the orange button will not display. Instead, the admin will provide special instructions on the assessment landing page, explaining which file path to follow, in order to find the sample data sets.
Either method is fine - and the user will achieve the same result! They simply need to know which folder to navigate to, in order to access the files for their randomly generated set of test questions.
When they have their sample test files and they have read any other instructions from their administrator, they can hit the green 'Start' button and their assessment will begin.
Each assessment is timed, although the time taken to finish does not impact the overall score. The test user interface (UI) is easy to navigate. All test sessions start on question 1. There is also a small timer, which allows users to keep track of how long they are taking during their session.
Each question has a name and brief summary, which explains to users the main features of the software being addressed.
If a question is task-based, i.e. the user has to use the software in order to answer the question, then the corresponding sample data set can be found in the zip folder they downloaded at the start. The orange instruction box explains where to find the files for each question.
Users can answer the test questions in any order. The 'Question Navigator' allows users to move around the questions and generally familiarise themselves with the exercises. The simple colour code helps users to keep track of their progress. Green squares indicate questions for which an answer has been submitted. Orange represents the current question. Red squares are questions that have yet to be attempted.
If a user makes a mistake, they can re-visit the green squares at any time, to edit their answers. A test session finishes when all the questions have been answered. Users have one last opportunity to review and edit their answers, before hitting the green 'Finish' button, which triggers their assessment to be graded.
(Sometimes, an admin will set a time limit for an assessment, in which case the session will automatically close after that time has elapsed. Answers submitted will be marked and any questions which have not been attempted will be scored as zero).
When the session is finished, the user may receive a test report, with feedback on any dropped marks. The level of feedback received can vary, depending on the settings prescribed by the account admin.
A helpful summary of the test questions, feedback on incorrect answers, coaching notes and training tags may be viewed by the user, by selecting the 'Show question detail' link at the top of their summary report.
The training tags for questions where score = < 100%, form a personal curriculum of suggested training workshop topics.
R
This is not all about who is the smartest, fastest, or best Revit user. Or the most experienced MicroStation wizard. Although these are useful things to know. The real value in the KS tools is trying to help users, at an individual level, identify what they know and also what they need to brush up on next, in order to improve their skills, with a particular piece of software.
Over the past 8 years, we have facilitated over 20,000 individual test sessions, across hundreds of AEC firms. Experience has shown that, in general, people don't like the idea of taking a test - at least that's usually how they feel at first. However, experience has also shown that, for most people, when they complete their assessment, they realise that it wasn't half as bad as they thought it would be! Further, they often learn new skills as they go through the process of answering the test questions. And lastly, they find the test feedback and coaching notes extremely helpful, in giving them an unbiased appraisal of their skills using a particular software application - relative to their peers in industry. It's important that users have the opportunity to review their test scores and feedback with their team leader or BIM manager. And it's also important that firms explain to their users that all test scores are confidential. Best practice would recommend that only senior administrators should have access to firm-wide results data.
Provided firms explain to their teams that the main point of rolling out an assessment program, is to gather valuable training needs analysis data, rather than a big-brother style exercise in peeking over shoulders, then most users feel OK about the idea. And when firms follow through with more focused, modular training workshops, which target recognised skills gaps, then it's a win-win for users and firms alike.
So let's take a look at the user journey, when asked to complete a KnowledgeSmart skills assessment.
We'll assume the firm has already done the pre-assessment prep - and explained to their teams what is happening and why they are being asked to take an assessment.
The next step is for a user to receive their test invite. (Here is a separate article on how a KS admin sends out test invites: http://the-knowledgesmart-blog.blogspot.com/2011/04/inviting-users-to-take-ks-test.html).
Admins can set up test sessions straight from a browser, but most invites are sent by email. Either way, the user ends up in the same place. Here is a typical KS test invite:
The system assigns a username and password to each user. They access their assessment by clicking the unique URL at the bottom of their invite mail. This takes them to the assessment login page, where they will be prompted to enter their username and password. (If a user loses or forgets their password, they can hit the 'Forgot details?' link to receive a new one).
When they log in, they arrive at the main assessment landing page. Here they will see a summary of the test they are scheduled to take and any special instructions from their administrator.
There are 5 types of questions, which typically comprise a KS assessment. Let's take a brief look at each one..
Free Text
This type of question requires a specific answer, which needs to match the answer stored in the KS database. Look out for precise instructions on how to format your answer, for example, number of decimal places, case sensitive answers, accurate spelling, and so on.
Multiple Choice
Here, you will be offered a selection of possible answers - and you need to select the best one. Watch out for 'distractor' entries, which are designed to make you think before selecting your answer.
Pick List
This question type offers you a range of options, which could represent a correct answer. Choose the best answers from the list provided, using the check boxes, to the left of the answers. Partial marks are often awarded for entering some of the correct answers on pick list questions.
Order List
This question type requires you to place a number of items in the correct sequence. Drag and drop the answer options, until they are in the right order.
True or False
This question type is pretty straightforward! Read the statement or question and decide if you think it is true or false. Enter your answer using the radio buttons.
OK, so when the user arrives at the landing page, before they begin, they need to download the sample data files that they will be using during their assessment. There are a couple of ways they might have to do this. The usual way is to select the orange, 'Download files' button.
This prompts the user to save the files locally on their work station. They will need to navigate back to this folder during their assessment, so most users choose their desktop, or My Documents, etc. The data sets do not need to be saved during a test session, so they can simply be deleted after the assessment has been completed.
Sometimes firms choose to zip up all sample test files and host them locally, on their own servers. If this is the case, then the orange button will not display. Instead, the admin will provide special instructions on the assessment landing page, explaining which file path to follow, in order to find the sample data sets.
Either method is fine - and the user will achieve the same result! They simply need to know which folder to navigate to, in order to access the files for their randomly generated set of test questions.
When they have their sample test files and they have read any other instructions from their administrator, they can hit the green 'Start' button and their assessment will begin.
Each assessment is timed, although the time taken to finish does not impact the overall score. The test user interface (UI) is easy to navigate. All test sessions start on question 1. There is also a small timer, which allows users to keep track of how long they are taking during their session.
Each question has a name and brief summary, which explains to users the main features of the software being addressed.
If a question is task-based, i.e. the user has to use the software in order to answer the question, then the corresponding sample data set can be found in the zip folder they downloaded at the start. The orange instruction box explains where to find the files for each question.
Users can answer the test questions in any order. The 'Question Navigator' allows users to move around the questions and generally familiarise themselves with the exercises. The simple colour code helps users to keep track of their progress. Green squares indicate questions for which an answer has been submitted. Orange represents the current question. Red squares are questions that have yet to be attempted.
If a user makes a mistake, they can re-visit the green squares at any time, to edit their answers. A test session finishes when all the questions have been answered. Users have one last opportunity to review and edit their answers, before hitting the green 'Finish' button, which triggers their assessment to be graded.
(Sometimes, an admin will set a time limit for an assessment, in which case the session will automatically close after that time has elapsed. Answers submitted will be marked and any questions which have not been attempted will be scored as zero).
When the session is finished, the user may receive a test report, with feedback on any dropped marks. The level of feedback received can vary, depending on the settings prescribed by the account admin.
A helpful summary of the test questions, feedback on incorrect answers, coaching notes and training tags may be viewed by the user, by selecting the 'Show question detail' link at the top of their summary report.
The training tags for questions where score = < 100%, form a personal curriculum of suggested training workshop topics.
R
Friday, 8 April 2011
Organising your KS Library
After a while, you will probably find that your KS test library starts to grow. This is particularly relevant for our 'Enterprise' customers, who have access to a wide range of 'off the shelf' KS test topics. Further, many firms are writing their own test questions, modules and tests, which adds even more content to your library. So how do you query your questions, manage your modules and tame your tangled test titles? In short, how do you organise your KS content library?
All tests in the KS 'off the shelf' library are locked down, i.e. you can't edit them or modify them in any way - whilst they remain in your OTS library page. (This is the page in your dashboard called, 'KnowledgeSmart tests').
Managing your Tests
When you want to edit a KS test, the first step is to create a copy of the original. To do this, go to the 'KnowledgeSmart tests' page of your dashboard, select the test you want to copy and click the 'Import to your account' tool.
This creates an exact copy of the test, which appears in the 'Your Tests' page of your dashboard. You'll notice that your copy has a different ID number from the original. Every test in the KS system has a unique ID number, so you can distinguish between various copies and easily manage your library material.
Your tests display in A-Z order on this page. When an OTS test is 'imported' into your account, this unlocks the test and you can now modify and customize many features of the test.
The quickest way to sort your tests is to use the ordering tools at the top of the list; these quickly arrange your tests in ascending (or descending) order by ID number, or A-Z (or Z-A) order by test name.
To find out more about creating and editing KS tests, take a look at this separate blog article:
http://the-knowledgesmart-blog.blogspot.com/2011/03/creating-and-editing-ks-tests.html.
Managing your Modules
When you create a copy of a KS OTS test, you also create a separate library of the individual component modules, which comprise the overall test. These will appear in the 'Your Modules' page of your dashboard.
When you import several tests into your account, you will find that your KS modules library pretty quickly grows to several pages of content. You have a number of content management tools available, to help you filter, organize and manage your modules.
Unlike your tests, which list in A-Z order, your modules will be listed in the order that they were imported into your account. The quickest way to sort your modules is to use the ordering tools at the top of the list; these quickly arrange your modules in ascending (or descending) order by ID number, or A-Z (or Z-A) order by module name.
If you want to filter your KS module library, you can use the searching and grouping tools. You'll find these tucked away in the orange bar, at the top of the page.
Hit the 'Show search' link to expand the searching tools box.
You can search on module name, module ID number and also by category tag. When a module is created, we assign a number of keywords, which describe the main subject areas being covered by the module. You can use these keywords to search and filter your modules.
When you have run a search and located the module(s) that you need, you can use the grouping tools, to create sub-sets of your module library.
Highlight the module(s), using the check boxes to the right of the module names. Create a new group, i.e. Revit MEP, in the groups table.
Then select 'Create' and your modules will be copied across to the new group.
In this way, you can create a number of sub-groups for your overall module library. When you want to filter your library, simply use the 'View group' dropdown to navigate to the required module(s).
You can also use the grouping tools to delete groups, edit group names, add new modules to existing groups and remove modules from existing groups.
To find out more about creating and editing KS modules, take a look at this separate blog article:
http://the-knowledgesmart-blog.blogspot.com/2011/03/creating-and-editing-ks-modules.html.
Managing your Questions
When you create a copy of a KS OTS test, you also create a separate library of the individual questions, which in turn comprise the modules, which combine to make the overall test. These will appear in the 'Your Questions' page of your dashboard.
When you import several tests into your account, you will find that your KS question library pretty quickly grows to several pages of content. You have a number of content management tools available, to help you filter, organize and manage your questions.
Like your modules library, your questions will be listed in the order that they were imported into your account. The quickest way to sort your questions is to use the ordering tools at the top of the list; these quickly arrange your questions in ascending (or descending) order by ID number, or A-Z (or Z-A) order by question name.
If you want to further filter your KS question library, you can also use the searching and grouping tools. You'll find these at the top of the page.
You can search on question name, question ID number and also by category tag. When a question is created, we assign a number of keywords, which describe the main subject areas being covered by the question. You can use these keywords to search and filter your questions.
When you have run a search and located the question(s) that you need, you can use the grouping tools, to create sub-sets of your question library.
Highlight the question(s), using the check boxes to the right of the question names. Create a new group, i.e. AutoCAD 2D, in the groups table. You can use the 'select all' check box at the top to highlight all questions on the page. Each page displays 20 questions, so might need to navigate several pages, in order to capture all the questions you need for your new group. You can select check boxes across multiple pages.
Then select 'Create' and your questions will be copied across to the new group.
In this way, you can create a number of sub-groups for your overall question library. When you want to filter your library, simply use the 'View group' dropdown to navigate to the required question(s).
As an additional aid to help you manage your question library, when you import a KS test into your account, the system automatically creates a sub-group of the related questions for each test. You'll find these sub-groups by using the 'View group' dropdown.
To find out more about creating and editing KS questions, take a look at this separate blog article:
http://the-knowledgesmart-blog.blogspot.com/2011/03/creating-and-editing-ks-questions.html.
One final point; any benchmark data published by KS will only relate to the original OTS tests. When you start making changes to your test library, we lose the ability to make a reliable comparison, from a benchmark point of view.
R
All tests in the KS 'off the shelf' library are locked down, i.e. you can't edit them or modify them in any way - whilst they remain in your OTS library page. (This is the page in your dashboard called, 'KnowledgeSmart tests').
Managing your Tests
When you want to edit a KS test, the first step is to create a copy of the original. To do this, go to the 'KnowledgeSmart tests' page of your dashboard, select the test you want to copy and click the 'Import to your account' tool.
This creates an exact copy of the test, which appears in the 'Your Tests' page of your dashboard. You'll notice that your copy has a different ID number from the original. Every test in the KS system has a unique ID number, so you can distinguish between various copies and easily manage your library material.
Your tests display in A-Z order on this page. When an OTS test is 'imported' into your account, this unlocks the test and you can now modify and customize many features of the test.
The quickest way to sort your tests is to use the ordering tools at the top of the list; these quickly arrange your tests in ascending (or descending) order by ID number, or A-Z (or Z-A) order by test name.
To find out more about creating and editing KS tests, take a look at this separate blog article:
http://the-knowledgesmart-blog.blogspot.com/2011/03/creating-and-editing-ks-tests.html.
Managing your Modules
When you create a copy of a KS OTS test, you also create a separate library of the individual component modules, which comprise the overall test. These will appear in the 'Your Modules' page of your dashboard.
When you import several tests into your account, you will find that your KS modules library pretty quickly grows to several pages of content. You have a number of content management tools available, to help you filter, organize and manage your modules.
Unlike your tests, which list in A-Z order, your modules will be listed in the order that they were imported into your account. The quickest way to sort your modules is to use the ordering tools at the top of the list; these quickly arrange your modules in ascending (or descending) order by ID number, or A-Z (or Z-A) order by module name.
If you want to filter your KS module library, you can use the searching and grouping tools. You'll find these tucked away in the orange bar, at the top of the page.
Hit the 'Show search' link to expand the searching tools box.
You can search on module name, module ID number and also by category tag. When a module is created, we assign a number of keywords, which describe the main subject areas being covered by the module. You can use these keywords to search and filter your modules.
When you have run a search and located the module(s) that you need, you can use the grouping tools, to create sub-sets of your module library.
Highlight the module(s), using the check boxes to the right of the module names. Create a new group, i.e. Revit MEP, in the groups table.
Then select 'Create' and your modules will be copied across to the new group.
In this way, you can create a number of sub-groups for your overall module library. When you want to filter your library, simply use the 'View group' dropdown to navigate to the required module(s).
You can also use the grouping tools to delete groups, edit group names, add new modules to existing groups and remove modules from existing groups.
To find out more about creating and editing KS modules, take a look at this separate blog article:
http://the-knowledgesmart-blog.blogspot.com/2011/03/creating-and-editing-ks-modules.html.
Managing your Questions
When you create a copy of a KS OTS test, you also create a separate library of the individual questions, which in turn comprise the modules, which combine to make the overall test. These will appear in the 'Your Questions' page of your dashboard.
When you import several tests into your account, you will find that your KS question library pretty quickly grows to several pages of content. You have a number of content management tools available, to help you filter, organize and manage your questions.
Like your modules library, your questions will be listed in the order that they were imported into your account. The quickest way to sort your questions is to use the ordering tools at the top of the list; these quickly arrange your questions in ascending (or descending) order by ID number, or A-Z (or Z-A) order by question name.
If you want to further filter your KS question library, you can also use the searching and grouping tools. You'll find these at the top of the page.
You can search on question name, question ID number and also by category tag. When a question is created, we assign a number of keywords, which describe the main subject areas being covered by the question. You can use these keywords to search and filter your questions.
When you have run a search and located the question(s) that you need, you can use the grouping tools, to create sub-sets of your question library.
Highlight the question(s), using the check boxes to the right of the question names. Create a new group, i.e. AutoCAD 2D, in the groups table. You can use the 'select all' check box at the top to highlight all questions on the page. Each page displays 20 questions, so might need to navigate several pages, in order to capture all the questions you need for your new group. You can select check boxes across multiple pages.
Then select 'Create' and your questions will be copied across to the new group.
In this way, you can create a number of sub-groups for your overall question library. When you want to filter your library, simply use the 'View group' dropdown to navigate to the required question(s).
You can also use the grouping tools to delete groups, edit group names, add new questions to existing groups and remove questions from existing groups.
As an additional aid to help you manage your question library, when you import a KS test into your account, the system automatically creates a sub-group of the related questions for each test. You'll find these sub-groups by using the 'View group' dropdown.
To find out more about creating and editing KS questions, take a look at this separate blog article:
http://the-knowledgesmart-blog.blogspot.com/2011/03/creating-and-editing-ks-questions.html.
One final point; any benchmark data published by KS will only relate to the original OTS tests. When you start making changes to your test library, we lose the ability to make a reliable comparison, from a benchmark point of view.
R
Thursday, 7 April 2011
Inviting Users to take a KS Test
Administering the KS tools is a pretty straightforward process. The system is web-based, so there is no software to install, maintain and upgrade. In general, it's a low-impact system, from an IT administration point of view.
Let's take a look at the process for inviting users to take a test. There are three different options, depending on your requirements. Let's take a look at each one..
Option one - inviting a small number of users to a test
Step one
Select the test you want your user(s) to take, from your library. Each test has a small 'envelope' icon, next to the test name.
Click on the envelope icon, which takes you to the test invite page. Your selected test will automatically appear in the test list dropdown.
Alternatively, you can select the INVITES link, from the top navigation bar of your dashboard. (You'll end up in the same place). However, bear in mind that if you do use the top navigation route (rather than the envelope icon next to a specific test title) then your required test will not automatically get selected in your dropdown. (It presents your test library in A-Z order, starting with 3ds Max). If you don't select the test you want from the dropdown, before you send out your invites, then your users will receive an invite to a 3ds Max test! :)
Step two
Enter the email, first name and last name of your test taker(s) in the fields provided. You can also capture user status, i.e. employee, interviewee, student, etc. Click Add to confirm your candidate's details.
Step three
Decide on your test invite text. You have a couple of different options for customising your invites. Here is a separate blog entry on this topic: http://the-knowledgesmart-blog.blogspot.com/2011/03/customising-your-test-invite-text.html.
Step four (optional)
You can set an expiry date on your test invites. After this date has passed, your users will no longer be able to access this test session. If they haven't completed their test before the expiry date, you will have to issue a new test invite.
Step five
OK, so you have completed the following steps; selected the correct test title from your library; entered all your user data; edited your invite text; maybe set an expiry date. The final step is to hit the Send Email button. This step prompts the following routines to take place, in the background; it generates your users' profile(s) on the KS system, adds them to your user list (in the 'Users' page of your dashboard), creates a username/password for each user and sends out their individual test invite emails.
Option two - inviting a large number of users to a test
You have the option to batch invite larger groups of users to take a test. Rather than entering their user details one at a time, as highlighted in the process above, you can invite them all at once, from the 'Users' section of your dashboard. Here is a separate blog entry about building your user list and batch inviting users to the same test: http://the-knowledgesmart-blog.blogspot.com/2011/03/building-your-list-of-users.html.
Option three - setting up a test session from a browser
Sometimes, you will need to set up a test session, without sending an email invite to the user. It is possible to set up a KS test session, straight from a web browser. Here is a separate blog entry on this topic: http://the-knowledgesmart-blog.blogspot.com/2011/03/setting-up-test-sessions-from-browser.html.
R
Let's take a look at the process for inviting users to take a test. There are three different options, depending on your requirements. Let's take a look at each one..
Option one - inviting a small number of users to a test
Step one
Select the test you want your user(s) to take, from your library. Each test has a small 'envelope' icon, next to the test name.
Click on the envelope icon, which takes you to the test invite page. Your selected test will automatically appear in the test list dropdown.
Alternatively, you can select the INVITES link, from the top navigation bar of your dashboard. (You'll end up in the same place). However, bear in mind that if you do use the top navigation route (rather than the envelope icon next to a specific test title) then your required test will not automatically get selected in your dropdown. (It presents your test library in A-Z order, starting with 3ds Max). If you don't select the test you want from the dropdown, before you send out your invites, then your users will receive an invite to a 3ds Max test! :)
Step two
Enter the email, first name and last name of your test taker(s) in the fields provided. You can also capture user status, i.e. employee, interviewee, student, etc. Click Add to confirm your candidate's details.
Step three
Decide on your test invite text. You have a couple of different options for customising your invites. Here is a separate blog entry on this topic: http://the-knowledgesmart-blog.blogspot.com/2011/03/customising-your-test-invite-text.html.
Step four (optional)
You can set an expiry date on your test invites. After this date has passed, your users will no longer be able to access this test session. If they haven't completed their test before the expiry date, you will have to issue a new test invite.
Step five
OK, so you have completed the following steps; selected the correct test title from your library; entered all your user data; edited your invite text; maybe set an expiry date. The final step is to hit the Send Email button. This step prompts the following routines to take place, in the background; it generates your users' profile(s) on the KS system, adds them to your user list (in the 'Users' page of your dashboard), creates a username/password for each user and sends out their individual test invite emails.
Option two - inviting a large number of users to a test
You have the option to batch invite larger groups of users to take a test. Rather than entering their user details one at a time, as highlighted in the process above, you can invite them all at once, from the 'Users' section of your dashboard. Here is a separate blog entry about building your user list and batch inviting users to the same test: http://the-knowledgesmart-blog.blogspot.com/2011/03/building-your-list-of-users.html.
Option three - setting up a test session from a browser
Sometimes, you will need to set up a test session, without sending an email invite to the user. It is possible to set up a KS test session, straight from a web browser. Here is a separate blog entry on this topic: http://the-knowledgesmart-blog.blogspot.com/2011/03/setting-up-test-sessions-from-browser.html.
R
Logging in to the KS system
OK, this might seem like a pretty basic task, but we thought it was worth documenting for new users of the KS system. How do admins access and log in to the KS system?
Here's how..
Step one
Go to the KS website homepage and click on the CUSTOMER LOGIN link, top right.
Step two
This takes you to the KS admin dashboard landing page. Here is the direct link: https://online.knowledgesmart.net.
Step three
To access your KS admin dashboard, click on the ADMIN DASHBOARD LOGIN link, just under the KnowledgeSmart logo.
Enter your administrator username and password, in the grey box and hit the Log In button.
And that's it! The first page you will see is the KS test list, which lists all of the 'off the shelf' tests on your account.
Here are a couple of additional things to be mindful of. First, when you are assigned your initial password by the KS system, it is generated by a software program. Consequently, it is a random series of characters and numbers. When you first log in to your dashboard, we suggest you change your system generated password, to one of your own choosing. Here's how..
Hit the Change password link, top right of your dashboard homepage. Copy & paste your system password in the first box. Enter your own password in box 2 and repeat in box 3. Then hit the Change password button - and you're done!
If you forget or lose your KS password, you have the option to re-set the password at any time. Go to the login page, hit the ADMIN DASHBOARD LOGIN link and click the Forgot details? link at the footer of the grey box.
This takes you to the password re-set page. Type in your username and hit Submit. You will be sent a new password to your registered email address.
Lastly, a word about the difference between the 'CADsmart' database and the 'KnowledgeSmart' system. You'll notice that there is a separate link called 'CADsmart Login', at the footer of the KS dashboard landing page. This link takes you to the login page for legacy CADsmart users. It is an entirely separate system and should not be confused with the login routine for the KnowledgeSmart tools. The CADsmart system has a separate account number and password combination. If you attempt to login to the CADsmart system using your KS username and password, you won't get very far. So don't try! :)
Conversely, you can't access a CADsmart dashboard with your KS credentials!
R
Here's how..
Step one
Go to the KS website homepage and click on the CUSTOMER LOGIN link, top right.
Step two
This takes you to the KS admin dashboard landing page. Here is the direct link: https://online.knowledgesmart.net.
Step three
To access your KS admin dashboard, click on the ADMIN DASHBOARD LOGIN link, just under the KnowledgeSmart logo.
Enter your administrator username and password, in the grey box and hit the Log In button.
And that's it! The first page you will see is the KS test list, which lists all of the 'off the shelf' tests on your account.
Here are a couple of additional things to be mindful of. First, when you are assigned your initial password by the KS system, it is generated by a software program. Consequently, it is a random series of characters and numbers. When you first log in to your dashboard, we suggest you change your system generated password, to one of your own choosing. Here's how..
Hit the Change password link, top right of your dashboard homepage. Copy & paste your system password in the first box. Enter your own password in box 2 and repeat in box 3. Then hit the Change password button - and you're done!
If you forget or lose your KS password, you have the option to re-set the password at any time. Go to the login page, hit the ADMIN DASHBOARD LOGIN link and click the Forgot details? link at the footer of the grey box.
This takes you to the password re-set page. Type in your username and hit Submit. You will be sent a new password to your registered email address.
Lastly, a word about the difference between the 'CADsmart' database and the 'KnowledgeSmart' system. You'll notice that there is a separate link called 'CADsmart Login', at the footer of the KS dashboard landing page. This link takes you to the login page for legacy CADsmart users. It is an entirely separate system and should not be confused with the login routine for the KnowledgeSmart tools. The CADsmart system has a separate account number and password combination. If you attempt to login to the CADsmart system using your KS username and password, you won't get very far. So don't try! :)
Conversely, you can't access a CADsmart dashboard with your KS credentials!
R
Subscribe to:
Posts (Atom)