Tuesday, 13 March 2018

10 Best Practice Tips for Writing Custom Assessments

KnowledgeSmart offers companies a flexible solution for assessing their teams.

Choose from a wide range of 'off the shelf' test titles, for design and engineering software or write your own custom assessments from scratch.

We have put together a handy guide with 10 best practice tips for writing custom assessments.

10 Best Practice Tips for Writing Custom Assessments - KnowledgeSmart

Friday, 9 March 2018


The most critical element of any skills assessment activity is to draw together support from management to drive the process and ensure a successful deployment and adoption.

Experience shows that it is helpful to gain support from the following key business areas:
  • Human Resources and Learning & Development
  • Technology and Building Information Modelling Management (to ensure alignment with company CAD / BIM standards and protocols e.g. BIM Level 2)
  • Senior Management
Without collective management support, your skills assessment activity might turn into a very frustrating, futile exercise.

Clarify the concept of skills assessment

People’s perception of skills assessment may be very different. Make sure that you define your interpretation of skill assessments, keeping in mind the priorities for each business division and ensuring alignment.

Once the definition is clarified it is substantially easier to explain the resulting benefits. Since most people have an adverse reaction when the word ‘test’ is mentioned, it is often a good idea to start with a quick comparison between ‘test’ and ‘assessment’. In most cases, the purpose of most tests is to assign a score. They offer limited diagnostic information to identify areas for improvement. Users often report feeling judged when tested and fear results might be used for other purposes than skills development.

To achieve the ultimate purpose of assessment - to improve learning and development - we need to answer not only questions on how our teams have learned but also questions on how they learned and why specific results occurred. Thus, an assessment is a far-reaching concept, focalising its efforts on learning, and serving the purpose of enhancement and development through a variety of ways.

Demonstrate the benefits

Firstly, your influencing skills will be tried and tested. There is no doubt that you will have to demonstrate how a skills assessment process can help your organisation with quantifiable benefits.

Linking the skills assessment benefits to a current strategic goal will undoubtedly go a long way towards helping to spark and keep management attention.

The most important benefits typically focus on the company’s recruitment process, efficient use of training budgets, productive skills development plans and optimised project resourcing. The challenges that each business division face might differ, but you will often find they are symbiotically interlinked.

In some cases, it is best practice to identify a ‘pilot’ group within the company first and develop a compelling internal case study before rolling out an assessment plan across your whole organisation. Stay aware of the tangible benefits of each business division. It is not productive for managers to continually fight fires. A successful assessment process shines a light on the root causes of some of the challenges faced. A successful skills assessment plan allows for the necessary course correction to be identified and implemented.

Catalyse improved management with improved data-driven decisions

Make sure you understand how your management team is measured and what their current strategic deliverables are. Connect the outcomes of the assessment programme to the overall metrics that management needs for reporting and to the overall strategic priorities and you will have a better chance of getting approval and buy-in. 

At RTKL – a global architecture, engineering, planning and creative services organisation – HR leadership challenged its BIM management team to source a BIM skills analysis tool to help the firm make smarter, more well-informed data-driven decisions about people’s CAD and BIM skills when resourcing project teams. The team reviewed the market and selected KnowledgeSmart based on reasons described by Andrew Victory, an Associate at RTKL, London, UK.


HR now understands ‘why’ people were hired and provides a high-level view of the firm’s talent base, but historically it has not fully appreciated some of the finer details about CAD and BIM skills. The nature of project-centric resourcing determines that individuals are moved from one project team to another as the workflow rises and falls across a project’s lifecycle, and their BIM skill-levels will critically influence their ability to contribute to their new team. With KnowledgeSmart results and analysis in hand, HR now has perhaps the most comprehensive oversight of all staff and their skills.”

Remember the financial benefits

Remember that a big priority for any successful management team is to show a good return on investment or prove financial benefits. Make sure you illustrate how the outcomes of your proposed assessment plan can lead to those management priorities. 

A quick cost-benefit analysis focusing on one of the following factors could be helpful:
  • the cost to employ inappropriate personnel over a 3-month period the cost of losing top performers (due to lack of recognition, support or training)
  • retaining an under-performing user (due to poor performance management)
  • project productivity gains due to identified performance improvements (being able to deliver on projects faster or with fewer resources

KnowledgeSmart is an invaluable tool for professionals that helps businesses capture knowledge on a wide range of software tools, workflows and, standards. More information is available at www.knowledgesmart.net

Sunday, 11 February 2018

KS Value Proposition - an Explanation

A KS software subscription price is determined by two factors: number of users to be tested (in a year) and number of required library topics to be included in the assessment mix.

There are 4 KS license types:
KS Pro1 - 1 primary library topic, i.e. Revit Architecture
KS Pro2 - 2 primary topics, i.e. Revit Architecture, Revit MEP
KS Pro3 - 3 primary topics, i.e. Revit Architecture, Revit MEP, Revit Structure
KS Enterprise - 4 or more primary topics

We also have different user number bands, to help identify firm size, i.e. 1-25, 26-50, 51-100, 101-250, 251-500, 501-1,000, etc.

For firms who only need to perform a small number of assessments each year, the 'pay-as-you-go' cost is $50/£50/E50 per test.

Before deciding on the best type of KS license, an AEC business should give due consideration to both of these factors.

It is important to establish precisely how many primary test topics they plan to use. If it is 4 (or greater), then KS Enterprise is the best fit. If they want to focus on fewer titles, then perhaps a KS Pro1, Pro2 or Pro3 would be more appropriate. 

Also, we do not monetise 'secondary' library titles, if a customer has invested in one or more 'primary' titles.

Here are some examples:

Revit Architecture, Revit MEP, Revit Structure, AutoCAD, Civil 3D, InfraWorks, Plant 3D, Inventor, AECOsim, MicroStation, GEOPAK, InRoads, ProjectWise, ARCHICAD, MS Office, BIM L2, Primavera P6, MS Project, Newforma.

Bluebeam, Design Review, Photoshop, InDesign, Rhino, SketchUp, Navisworks, Glue, Solibri, Vault, Revit Process, BIM Fundamentals, BIM Software Management, BIM Coordination, BIM Authoring, USACE BIM.

To further clarify, a 'test topic' refers to all related assessment titles under an 'umbrella' heading. For example, to assess Revit Architecture  skills for a company would require a KS Pro1 license, but might include the following titles:

Revit Architecture for occasional users
Revit Architecture fundamentals
Revit Architecture – Xpress 
Revit Architecture advanced
Revit Architecture - Extra questions
Revit Content Creation
Revit for Interiors
Revit Project Process
BIM Authoring with Revit
BIM Software Management (Revit)
BIM Coordination (Revit)
Practical BIM fundamentals

If a customer exceeds their user count by a small margin, we tend not to worry about it, and will review their future needs at the license anniversary. If they find their requirements go up by a bigger number (part-way through the license), then we typically have a conversation and sometimes upgrade to the next user band before the anniversary swings around. In this circumstance, they pay the difference between price bands, pro-rated to the license anniversary date. Customers are not blocked from using the tools at any time.

Here are links to the KS product and price pages.


Sunday, 4 February 2018

Developing Group Learning Strategies

We talk a lot about collaboration in the AEC industry... particularly in the context of delivering BIM projects.

Group learning, also known as “collaborative learning”, refers to the instructional strategy of having learners work together in groups in order address a task or goal.

The clever folks at efront Learning wrote an articulate and informed blog about developing a better approach to group learning strategies.

By taking a new approach to group learning, AEC businesses can develop an effective strategy to increase employees’ team spirit, collaboration skills, and workplace productivity.

Group learning which includes a mix of mentoring, lunch and learns, classroom training and focused E-learning - backed up with effective skills gap analysis - allows businesses to implement a blended, collaborative learning program that allows employees to study and work together to achieve common goals for themselves and the wider organisation.


Tuesday, 23 January 2018

Assessing BIM Skills

I recently read an articulate and informed blog post by Carillion's Technical BIM Lead, John Ford, about the business of measuring BIM skills.

Read the full post here: 'Are we Underestimating the Value of BIM Assessments?'

John makes a number of perceptive points about the organisational and process challenges facing AEC businesses, on their journey to BIM excellence.

There is no 'right or wrong' way to capture useful business intelligence, if it feeds into a wider BIM improvement plan. As Ford points out in his article, there are many effective methods for analysing data and metrics around the topic of BIM experience and capability, both within an organisation and across the wider industry supply chain.

A mix of formal assessment, self-assessment, survey responses, RAG tables and industry certification offer AEC firms all the tools they need, to create a clearer picture of BIM experience and credentials.

KnowledgeSmart has been working consistently in this space for the past 5 years, creating software and library content designed to help AEC businesses measure BIM maturity, skills gaps, training needs and technical expertise.

Here is a link to a recent KS webcast, which provides all the necessary tools and benchmark data for Global AEC businesses to create their own tailored BIM excellence plan.

To slightly misquote management guru, Peter Drucker, 'If you can't measure it, you can't manage it'. Wise words.


Wednesday, 25 October 2017

KS Webinar - Demonstrating Individual and Team BIM Level 2 Capability

Earlier today, we recorded our first Webinar to support the upcoming launch of our new KS library material: The presentation is titled, "Demonstrating Individual and Team BIM Level 2 Capability".

The session was co-presented by KnowledgeSmart CEO, Rory Vance, and Evolve Consultancy Principal, Nigel Davies.

The session runs for 45 mins, plus 15 mins delegate Q&A at the end.  Here is the agenda:

The content is aimed at AEC business leadership, BIM professionals and technical management. Here is a link to a recording of the session.


Monday, 9 October 2017

New Feature - Bulk Question Upload

KS customers have a number of options for editing existing library content and also creating new assessment questions from scratch.

It is now possible to upload multiple questions in one go, via a new tool called, 'Bulk Add'.

From this page, you can bulk upload questions and answers from an Excel spread sheet, for 5 question types: True or False, Multiple Choice, Pick List, Order List and Free Text.

To use this feature, click on the link in the help notes box to download the template Excel file.

Replace the contents of this spreadsheet with your questions and use the controls on this page to upload your file.

Sanity check your content on the preview page.

Click 'Add Questions', to upload your new material.  You will see an advisory note which tells you that the upload was successful.

Click 'View questions', to see your new question displayed in your draft library area.

You can make further edits from here, using the normal editing controls.

Here are some more detailed instructions, which will help you when you are completing your Excel template:

The column names of your spreadsheet must match the column names in the template file. The first column denotes the question type. This can be one of 5 values: FT, MC, PL, OL, or TF. These represent the question types: Free text, Multiple choice, Pick list, Order list and True or false respectively.

For all questions, the QuestionName, QuestionSummary and QuestionText values are mandatory.

Enter your category tags and training tags in the fields provided. Separate each entry with a comma. These are extremely important for later gap analysis and charting, so be sure to include them!

Free Text
For free text questions, enter your answer into the Answer1 column. With free text questions, you can also provide the following details:
FTNumeric - enter True if the answer is numeric.
FTCaseWhitespaceVariance - enter True to enable case and white space variance. When true, the marking will disregard incorrect case and white spacing.
FTAllowVariance - for numeric questions, enter True to enable variance.
FTPercent - when variance is true, your can enter a percentage value.
FTLower - when variance is true, enter the lower threshold.
FTUpper - when variance is true, enter the upper threshold.
FTPrecisionHint - for numeric questions, enter True to enable precision hint.

True or False
For true or false questions, enter the correct answer into the Answer1 column. Valid values are True or False.

Multi Choice
For multi choice questions, enter the available options in the columns Answer1 through to Answer10. Prefix the correct answer with an * symbol. If more than one column contains a * symbol, the import process records the first answer with an * symbol as the correct option.

Pick List
For pick list questions, enter the available options in the columns Answer1 through to Answer10. You can prefix multiple correct answers with the * symbol.
PLPartialScoring - for pick list questions, enter True to enable partial scoring.

Order List
For order list questions, enter the answers in the columns Answer1 through to Answer10. You should enter the answers in the correct sequence.