Global Test Scores Posted by CAD & BIM Managers

CAD and BIM Managers from across the world have submitted a wide range of KS test scores, in recent years. Let’s take a more detailed look at the people behind the headline stats.

We capture some interesting user demographic data at the start of each test session. This includes the following questions:

- Primary Industry/Discipline
- Primary Role
- Country & State
- Self rating
- How many years have you used this tool?
- Where did you learn to use this tool?
- How often do you use this tool? (Daily, weekly, occasional)
- How did you learn to use this tool? (On the job, Formal training, College)

The top 6 tests completed by CAD & BIM Managers are, in order (by volume):

Revit Architecture
BIM Management
Revit MEP
Revit Structure
AutoCAD
MicroStation

The results data comes from a wide range of countries and territories, primarily, US, Canada, UK, Australia, New Zealand, Singapore, HK, South Africa and Middle East regions.

Here are the overall results:

Revit Architecture

Avg score/time for:

Regular users = 81.7% in 2551 secs

Part-time users = 78.3% in 3945 secs

1 to 5 years' experience (all types) = 76.3% in 2940 secs

6 to 10 years' experience (all types) = 82.0% in 2912 secs

11 to 15 years' experience (all types) = 81.1% in 2832 secs

On the job learning (all types) = 82.3% in 3725 secs

Formal learning (all types) = 76.7% in 3317 secs

The average Revit Architecture self-rating score for CAD & BIM Managers is: 4.2.

BIM Management

Avg score/time for:

Regular users = 79.1% in 1767 secs

Part-time users = 71.2% in 2252 secs

1 to 5 years' experience (all types) = 72.3% in 2771 secs

6 to 10 years' experience (all types) = 79.3% in 1682 secs

11 to 15 years' experience (all types) = 78.7% in 1711 secs

On the job learning (all types) = 76.4% in 2248 secs

Formal learning (all types) = 72.5% in 1843 secs

The average BIM self-rating score for CAD & BIM Managers is: 4.3.

Revit MEP

Avg score/time for:

Regular users = 77.6% in 3136 secs

Part-time users = 72.8% in 3316 secs

1 to 5 years' experience (all types) = 70.8% in 2996 secs

6 to 10 years' experience (all types) = 78.7% in 2552 secs

11 to 15 years' experience (all types) = 75.1% in 2732 secs

On the job learning (all types) = 77.6% in 3470 secs

Formal learning (all types) = 74.8% in 3311 secs

The average Revit MEP self-rating score for CAD & BIM Managers is: 4.1.

Revit Structure

Avg score/time for:

Regular users = 78.6% in 3214 secs

Part-time users = 73.3% in 3745 secs

1 to 5 years' experience (all types) = 69.7% in 3640 secs

6 to 10 years' experience (all types) = 81.0% in 2819 secs

11 to 15 years' experience (all types) = 80.2% in 2832 secs

On the job learning (all types) = 80.1% in 2902 secs

Formal learning (all types) = 75.7% in 2493 secs

The average Revit Structure self-rating score for CAD & BIM Managers is: 4.0.

AutoCAD

Avg score/time for:

Regular users = 64.0% in 4230 secs

Part-time users = 62.8% in 4535 secs

1 to 5 years' experience (all types) = 53.2% in 3827 secs

6 to 10 years' experience (all types) = 57.6% in 3621 secs

11 to 15 years' experience (all types) = 59.7% in 4038 secs

On the job learning (all types) = 66.9% in 3782 secs

Formal learning (all types) = 62.2% in 3814 secs

The average AutoCAD self-rating score for CAD & BIM Managers is: 3.5.

 MicroStation

Avg score/time for:

Regular users = 63.6% in 4154 secs

Part-time users = 57.2% in 4766 secs

1 to 5 years' experience (all types) = 55.6% in 4901 secs

6 to 10 years' experience (all types) = 63.3% in 4546 secs

11 to 15 years' experience (all types) = 64.4% in 4527 secs

On the job learning (all types) = 61.8% in 4273 secs

Formal learning (all types) = 58.5% in 4139 secs

The average MicroStation self-rating score for CAD & BIM Managers is: 3.6.

Trends:

Overall, regular users are more accurate and faster than part-time users.

CAD & BIM Managers with 6 to 10 years' of experience with these tools are the most efficient. They reach a plateau in performance after c. 7 years and make no further productivity gains.

CAD & BIM Managers who learn to apply these tools on projects are more accurate than those who have primarily formal training, but less practical experience. Formally trained users are faster overall than self-taught users.

CAD & BIM Managers are generally very confident in their ability to use technical software tools on Construction projects.

CAD & BIM Managers are more confident and more accurate using modeling software than drafting software.

R

Global Test Scores Posted by Architects

Architects from across the world have submitted several thousand KS test scores, in recent years. Let’s take a more detailed look at the people behind the headline stats.

We capture some interesting user demographic data at the start of each test session. This includes the following questions:

- Primary Industry/Discipline
- Primary Role
- Country & State
- Self rating (out of 5)
- How many years have you used this tool?
- Where did you learn to use this tool?
- How often do you use this tool? (Daily, weekly, occasional)
- How did you learn to use this tool? (On the job, Formal training, College)

The top 4 skills assessments completed by Architects are, in order (by volume):

Revit Architecture
AutoCAD
MicroStation
BIM Management

The results data comes from a wide range of countries and territories, primarily, US, Canada, UK, Australia, New Zealand, Singapore, South Africa and Middle East regions.

Here are the overall results:

Revit Architecture

Avg score/time for:

Regular users = 73.5% in 4894 secs

Part-time users = 67.3% in 5542 secs

1 to 5 years' experience (all types) = 66.2% in 5333 secs

6 to 10 years' experience (all types) = 74.5% in 5215 secs

11 to 15 years' experience (all types) = 73.0% in 5199 secs

On the job learning (all types) = 73.3% in 5637 secs

Formal learning (all types) = 70.3% in 4834 secs

The average Revit Architecture self-rating score for Architects is: 3.7.

  

AutoCAD

Avg score/time for:

Regular users = 55.5% in 4919 secs

Part-time users = 52.8% in 6019 secs

1 to 5 years' experience (all types) = 45.9% in 4241 secs

6 to 10 years' experience (all types) = 55.1% in 5772 secs

11 to 15 years' experience (all types) = 53.7% in 5430 secs

On the job learning (all types) = 57.5% in 5140 secs

Formal learning (all types) = 51.8% in 4932 secs

The average AutoCAD self-rating score for Architects is: 3.4.

MicroStation

Avg score/time for:

Regular users = 61.2% in 4652 secs

Part-time users = 56.2% in 5165 secs

1 to 5 years' experience (all types) = 58.2% in 4901 secs

6 to 10 years' experience (all types) = 62.3% in 4639 secs

11 to 15 years' experience (all types) = 63.4% in 4519 secs

On the job learning (all types) = 62.6% in 4673 secs

Formal learning (all types) = 65.8% in 5037 secs

The average MicroStation self-rating score for Architects is: 3.5.

BIM Management

Avg score/time for:

Regular users = 71.1% in 2029 secs

Part-time users = 67.8% in 2152 secs

1 to 5 years' experience (all types) = 68.3% in 2037 secs

6 to 10 years' experience (all types) = 69.1% in 1947 secs

11 to 15 years' experience (all types) = 67.7% in 1923 secs

On the job learning (all types) = 70.4% in 2035 secs

Formal learning (all types) = 70.5% in 1947 secs

The average BIM self-rating score for Architects is: 3.6.

Trends:

Overall, regular users are more accurate and faster than part-time users.

Architects with 6 to 10 years' of experience with these tools are the most efficient. They reach a plateau in performance after c. 7 years and make no further productivity gains.

Architects who learn to apply these tools on projects are more accurate than those who have primarily formal training, but less practical experience. Formally trained users are faster overall than self-taught users.

R

Navisworks Test Scores Around The World

Navisworks is a 3D design review package, used primarily in the AEC sector to complement 3D design packages (such as Revit, AutoCAD and MicroStation). 

Navisworks allows users to open and combine 3D models, navigate around them in real-time and review the model using a set of tools including comments, redlining, viewpoint and measurements.

Over the years, KnowledgeSmart has captured test scores for a wide range of of Navisworks users, in multiple countries across the globe.

Here are the variances in our overall benchmark data, from 2013 to 2015:

2013
Average Score: 54.0%
Average Time: 49 mins

2014
Average Score: 57.0%
Average Time: 52 mins

2015
Average Score: 58.4%
Average Time: 43 mins


Let's take a look at some overall aggregate score and time data on a country vs country basis. Numbers of test scores vary, so this list does not imply that one country is 'better' than another. It's interesting to see what kind of data is generated in different parts of the globe.

Navisworks Fundamentals

Country	Score (%)	Time (secs)
Australia	67.7	2172.2
Bahrain	53.3	3123.3
Canada	63.4	2275.5
Hong Kong	68.4	2643.3
India	49.7	2789.8
Kuwait	52.4	3320.2
New Zealand	68.1	2346.7
Oman	51.2	3630.4
Pakistan	53.2	2478.8
Philippines	60.0	2489.8
Qatar	52.6	2880.6
Saudi Arabia	59.8	2590.0
Singapore	58.8	2467.7
UAE	63.1	3543.5
UK	66.5	2525.9
USA	57.7	2491.4

We also capture data on a per State basis, for our US customers. The list below represents those States where the volume of data makes for meaningful data sharing.

State	Score (%)	Time (secs)
USA all	57.7	2491.4
Arizona	56.5	2676.1
California	63.4	2374.2
Colorado	58.8	2298.8
Florida	62.9	2387.6
Georgia	59.6	2478.5
Illinois	54.6	2664.4
Louisiana	55.7	2599.4
Michigan	58.8	2490.7
Missouri	51.2	2974.1
New York	58.9	2185.5
North Carolina	53.3	2776.1
Oregon	52.7	2687.8
Pennsylvania	57.9	2489.9
Tennessee	54.3	2567.7
Texas	63.2	2234.4
Utah	57.3	2512.2
Virginia	56.1	2775.5

Navisworks is a Top 10 KS test topic, across our global customers. Overall results data illustrates that there is a clear need for training across the wider industry, with respect to model navigation and clash detection.

R

Evolve Consultancy - BIM Workshops

Longtime friends of KS, Evolve Consultancy, are hosting a series of BIM workshops in the coming weeks, to help UK Construction prepare for the BIM L2 target date of early April.

Here are the details:

2016 is here! But are you ready for the biggest single change to UK construction in recent history?

You will, of course, be aware of Building Information Modelling by now, and the fact that all UK publicly procured projects (not forgetting the many private projects following suit) need to be delivered as Level 2 BIM projects from April.

But what does that mean? And what effect will it have on you and your business?

You are invited to join us and find out all the answers at our popular interactive BIM Level 2 Implementation Workshops being held across the country in 2016.

LONDON, MARYLEBONE STATION
24 Feb 2016, 10am-4pm

BRISTOL, HIGH STREET
22 March 2016, 10am-4pm

MANCHESTER, DUCIE STREET
22 April 2016, 10am-4pm


This is not your normal training course. We’ll lead you through a series of discussions and exercises designed to help you apply the implications of BIM to your own office practices. You will go away with an action plan totally and utterly tailored to your needs.

All levels of staff are welcome from large and small practices.

What will we cover? Everything essential to understand what BIM means to you:

·                The UK Government BIM initiative

·                The Common Data Environment

·                Understanding BIM Level 2

·                Collaborative working

·                Strategic priorities & project delivery


All for only £100 (+VAT) per person. Just fill in the attached booking form and send it back to admin@evolve-consultancy.com. Spaces are limited.

Where else could you get a full day’s BIM Level 2 workshop for only £100?

(KnowledgeSmart customers: please contact Evolve for a special discount code to save 20% off the booking fee!)

R

Revit MEP Test Scores Around The World

Autodesk Revit is building information modeling software for architects, structural engineers, MEP engineers, designers and contractors. It allows users to design a building and structure and its components in 3D, annotate the model with 2D drafting elements, and access building information from the building model's database. Revit is 4D BIM capable with tools to plan and track various stages in the building's lifecycle, from concept to construction and later demolition.

Over the years, KnowledgeSmart has captured test scores for thousands of Revit MEP users, in multiple countries across the globe.

Here are the variances in our overall benchmark data, from 2013 to 2015. (For the first couple of years, the data was combined into an overall value):

2013
Average Score: 65.0%
Average Time: 1 hr 16 mins

2014
Average Score: 66.0%
Average Time: 1 hr 12 mins

2015
Electrical
Average Score: 63.7%
Average Time: 1 hr 6 mins

Mechanical
Average Score: 62.2%
Average Time: 1 hr 7 mins

Plumbing
Average Score: 70.1%
Average Time: 1 hr 11 mins


Let's take a look at some overall aggregate score and time data on a country vs country basis. Numbers of test scores vary, so this list does not imply that one country is 'better' than another. It's interesting to see what kind of data is generated in different parts of the globe.

Revit MEP Fundamentals - Electrical

Country	Score (%)	Time (secs)
Australia	67.7	3683.0
Bahrain	58.7	5031.1
Canada	66.8	4008.6
Hong Kong	65.9	4071.9
India	57.1	3770.5
Kuwait	53.7	4280.8
New Zealand	71.8	3487.1
Oman	56.5	4855.6
Pakistan	53.1	3872.2
Philippines	59.9	3136.8
Qatar	55.0	3505.2
Saudi Arabia	58.7	3716.6
Singapore	66.1	4765.1
UAE	60.3	3715.4
UK	67.7	3525.3
USA	65.7	4143.9

Revit MEP Fundamentals - Mechanical

Country	Score (%)	Time (secs)
Australia	65.9	4176.4
Bahrain	57.6	4365.4
Canada	64.2	4787.8
Hong Kong	62.7	4034.4
India	58.0	4467.4
Kuwait	56.6	4576.7
New Zealand	71.2	3413.3
Oman	54.4	5123.5
Pakistan	51.1	4176.6
Philippines	56.0	3663.3
Qatar	58.1	3687.7
Saudi Arabia	60.1	4245.6
Singapore	65.7	4838.3
UAE	58.0	3822.4
UK	68.3	3516.3
USA	66.3	4076.4

Revit MEP Fundamentals - Plumbing

Country	Score (%)	Time (secs)
Australia	69.7	4378.4
Bahrain	64.3	4897.1
Canada	73.4	3597.5
Hong Kong	73.1	4346.1
India	63.8	3951.8
Kuwait	58.8	4020.0
New Zealand	73.2	4132.9
Oman	59.8	4676.4
Pakistan	60.5	4876.3
Philippines	67.6	3176.7
Qatar	58.7	4899.4
Saudi Arabia	67.8	4346.7
Singapore	74.1	4343.3
UAE	67.8	3984.0
UK	72.8	3474.5
USA	69.9	3919.5

We also capture data on a per State basis, for our US customers. The list below represents those States where the volume of data makes for meaningful data sharing.

Revit MEP Fundamentals - Electrical

State	Score (%)	Time (secs)
USA all	65.7	4143.9
Arizona	63.7	3956.7
California	67.7	4697.8
Colorado	71.5	4234.5
Florida	73.2	4212.5
Georgia	68.5	3790.7
Illinois	68.1	5212.2
Louisiana	62.2	3987.5
Michigan	65.8	3887.4
North Carolina	62.3	4960.0
Oregon	58.7	4345.9
Pennsylvania	65.0	3803.8
Texas	68.8	3806.8
Utah	64.8	4234.8
Virginia	61.3	4340.0

Revit MEP Fundamentals - Mechanical

State	Score (%)	Time (secs)
USA all	66.3	4076.4
Alabama	64.2	3815.4
Arizona	61.1	3595.0
California	72.5	5092.5
Colorado	72.4	5051.2
Florida	69.1	3531.4
Georgia	70.7	4470.8
Illinois	64.4	3424.2
Kentucky	65.0	3765.9
Louisiana	65.5	3815.3
Maryland	63.1	3402.5
Michigan	69.4	3580.1
Missouri	72.0	3920.4
North Carolina	59.7	4268.9
Ohio	73.2	3880.0
Oregon	63.0	4022.0
Pennsylvania	63.3	3769.6
Tennessee	65.9	5410.0
Texas	67.1	3522.5
Utah	66.7	4237.7
Virginia	61.0	3950.1
Washington	71.1	4326.6
Wisconsin	66.5	4345.3

Revit MEP Fundamentals - Plumbing

State	Score (%)	Time (secs)
USA all	69.9	3919.5
Arizona	68.0	3315.0
California	74.7	4693.4
Colorado	71.5	4234.5
Florida	73.7	4252.7
Georgia	73.0	4132.2
Illinois	67.9	3822.2
Louisiana	67.6	3987.5
Michigan	71.1	3619.9
North Carolina	65.7	4268.9
Oregon	62.2	4205.5
Pennsylvania	69.8	3769.6
Texas	72.0	4374.4
Utah	69.7	4167.8
Virginia	63.3	4123.3

Revit MEP Advanced

2015
Mechanical
Average Score: 56.0%
Average Time: 1 hr 3 mins

Plumbing
Average Score: 57.0%
Average Time: 1 hr 6 mins


Revit MEP is a Top 5 KS test topic, across our global customers. The impact of BIM on the AEC industry is significant and Revit is the dominant design tool, in terms of overall market share and breadth of supply chain resources.

R

KS New Release - Data Query Tool

The KS admin dashboard has a new element this month... a new query builder and reporting tool. The release is part one of three, for this tool.

When you log in to your dashboard as a KS admin, you will now see a new landing page, called 'HOME'.

The top half of the page displays 4 sections, which summarize some useful at-a-glance data, as follows:

Number of Invites sent in last 30 days
Number of Tests in last 30 days (Not Started, In Progress and Complete)
Top 8 Skills Gaps (based on results from last 6 months' results)
Results (last 5 completed tests)

At the bottom of the page, you will find the new query tool. This includes a range of 'pre-defined' queries, as follows:

50% or less test scores
70% or greater test scores
All invites from last 30 days
All results from last 30 days
Incomplete invites from last 30 days
All employee scores
All interviewee scores
All expired invites

You can edit pre-defined copies by first taking a copy, then editing the copy and saving with a different name.

Create new queries by clicking on the 'Create New Query' button.

Edit queries by clicking on the 'Edit Query' button.

Copy existing (or pre-defined) queries by clicking on the 'Copy Query' button.

Delete queries by clicking on the 'Delete Query' button.

Run queries across all linked accounts, or selected child accounts, by checking the relevant boxes in the 'Run Against Accounts' area:

When you create a new query, you will see this box appear:

There are 4 main sections which can be presented in your report: 

Users - this captures data about the user profile and account where the user record is stored

Background - this captures data about user demographics info

Invites - this captures data about invite (and related test) status

Results - this captures data about results (and related test) status

Click 'Display these columns' to expand the top section. Use the check boxes to build up the columns of your data query. The columns selected here will also be presented in your csv export. Click on the '+' icon, if you want to add this element to your filter options in the 'match ALL conditions' section below.  

In the query conditions section below, use the filter on the left of the screen to toggle between the following search topics: Users, Background, Invites, Results.

Queries can be a combination of 'ALL of these conditions' only, or 'ANY of these conditions' only, or both.

Use the dropdowns to build up the logic of your queries. There are lots of different options available, so you can filter your user, invites and results data in lots of new ways.

Export queries to csv for further analysis, by clicking on the 'Download' button.

This release is part one of three. Here's what we're working on next.

Part Two

Allow KS admins to perform a series of bulk actions, with the results of a query.  

For example:

Delete users

Move users from one account to another

Invite users to a test

Delete results

Move results from one account to another

Delete invites

Change invite expiry date

Re-send invites

We'll also add individual training tags to the search options, allowing you to create workshop groups for users who have common skills gaps.

Part Three

Graphics & charts: we plan to display a range of visual summary fields on the dashboard landing page.

For example:

No. of invites sent in last x days

No. of completed tests in last x days

No. of incomplete tests in last x days

Users who have 'incomplete tests' 

Most popular test topics by number taken

Average score and time for top 5 test topics

Average score for Employees vs Interviewees

Global benchmark comparison for top 5 test topics

Last x results: user names, test names, scores & times

Top 20 skills gaps (i.e. training tags)

Top 20 hardest test questions (i.e. lowest scoring average)

We will also include the ability to send auto-email reports to KS admins, which contain custom values and timescales.

As always, we welcome your feedback and ideas, so if you have any additional feature requests, in the context of searching, filtering, grouping and reporting data, we'd love to hear them!

R