Wednesday, 6 January 2016

AutoCAD Civil 3D Test Scores Around The World

AutoCAD Civil 3D is a popular civil engineering design and documentation software solution, owned and developed by Autodesk.
 
Over the years, KnowledgeSmart has captured test scores for a significant number of Civil 3D users, in several countries worldwide.
 
Here are the variances in our overall benchmark data, from 2013 to 2015:
 
2013
Average Score: 56.0%
Average Time: 1 hr 16 mins

 
2014
Average Score: 58.0%
Average Time: 1 hr 13 mins

 
2015
Average Score: 67.2%
Average Time: 1 hr 3 mins


 
Let's take a look at some overall aggregate score and time data on a country vs country basis. Numbers of test scores vary, so this list does not imply that one country is 'better' than another. It's interesting to see what kind of data is generated in different parts of the globe.
 
CountryScore (%)Time (secs)
Australia69.13425.0
Canada70.23323.3
Hong Kong63.03500.2
Philippines62.84131.7
South Africa53.62458.4
UAE51.83947.1
UK67.82305.9
USA 69.63316.2

 
We also capture data on a per State basis, for our US customers. The list below represents those States where the volume of data makes for meaningful data sharing.

StateScore (%)Time (secs)
USA (all)69.63316.2
California70.24172.2
Colorado54.05201.9
Delaware73.52568.1
Florida78.82273.2
Hawaii56.84061.9
Kansas66.04927.5
Maryland61.24472.4
Massachusetts70.43548.9
Missouri53.35014.2
Nevada74.82707.5
New Jersey74.83487.2
Pennsylvania69.62467.9
Texas68.02452.5
Vermont72.02995.3
Virginia77.13555.1
Washington64.02812.5
Wyoming71.03248.3
 
 
Much of our Civil 3D data originates in the USA, where the tool is widely used on Infrastructure projects. As with the majority of our assessment data, Civil 3D capability varies from country to country and between US States. North American users have a 20%+ variance across different territories.
 
R

Wednesday, 30 December 2015

AutoCAD Test Scores Around The World

According to Wikipedia, AutoCAD is defined as 'a commercial software application for 2D and 3D computer-aided design (CAD) and drafting, available since 1982 as a desktop application and since 2010 as a mobile web and cloud-based app'.

The 2016 release marked the 30th major release of AutoCAD for Windows. The 2014 release marked the fourth consecutive year of AutoCAD for Mac.

Over the years, KnowledgeSmart has captured test scores for thousands of AutoCAD users, in multiple countries across the globe.

Here are the variances in our overall benchmark data, from 2009 to 2015:

2009
Average Score: 59.5%
Average Time: 1 hr 16 mins

2010-12
Average Score: 61.0%
Average Time: 1 hr 18 mins

2013
Average Score: 62.0%
Average Time: 1 hr 13 mins

2014
Average Score: 62.0%
Average Time: 1 hr 10 mins

2015
Average Score: 58.5%
Average Time: 1 hr 13 mins


Let's take a look at some overall aggregate score and time data on a country vs country basis. Numbers of test scores vary, so this list does not imply that one country is 'better' than another. It's interesting to see what kind of data is generated in different parts of the globe.

Country Score (%) Time (secs)
Australia 63.3 4777.9
Bahrain 61.8 4330.0
Canada 47.1 3395.0
China 54.6 7542.8
Hong Kong 37.7 3610.0
India 49.1 4203.8
Ireland 49.5 5573.8
Kuwait 45.7 3741.4
New Zealand 69.7 4399.8
Oman 39.3 5649.2
Philippines 50.3 4881.8
Poland 62.0 3150.0
Qatar 50.9 5003.4
Saudi Arabia 60.5 3361.3
Singapore 49.4 4752.0
South Africa 49.2 4468.0
Spain 62.7 3481.0
UAE 62.6 5112.7
UK 53.4 4572.3
USA  61.3 5511.8


We also capture data on a per State basis, for our US customers. The list below represents those States where the volume of data makes for meaningful data sharing.

State Score (%) Time (secs)
USA (all) 61.3 5511.8
California 63.4 5580.1
Florida 70.6 3140.9
Georgia 48.4 5096.4
Louisiana 67.3 3263.3
Nevada 75.3 3282.8
New York 52.6 5444.5
Washington 45.5 7336.9


Score and time data varies quite considerably, from country to country and between US States.

Even though the software has been commercially available for over 3 decades, there are still notable skills gaps across the overall user base. Our data shows quite clearly that AutoCAD users reach a performance 'plateau' after about 5 years and then remain at a similar skill level in subsequent years. 

When we started capturing skills data in the AEC industry, over 10 years ago, AutoCAD was the principal tool of choice for design and engineering firms. Whilst still widely used, the growth of BIM in the Construction sector has seen a rise in the number of different software tools deployed on projects  in the built environment.  Users must now learn a wider variety of software tools to do their jobs, than in years past.

R

Wednesday, 2 December 2015

Benchmark Stats for Popular KS Test Titles

Benchmarking software capability at individual, team and office level is a key part of the KnowledgeSmart experience.

Our library of skills assessments for the AEC industry covers a variety of  popular software tools, used on projects on Construction projects worldwide.

Our results data comes from a wide variety of countries and territories, including: US, UK, Australia, Canada, New Zealand, South Africa, Singapore, Vietnam, India and the Middle East.

Here is the overall average results data for our most popular topics (% score and time to complete):

Test Title % Score Time
3ds Max fundamentals 54.5 57m
Adobe InDesign occasional 64 45m
Adobe Photoshop occasional 62 48m
AutoCAD 2D fundamentals 58.5 1h 13m
AutoCAD 2D occasional 73.1 36.5m
AutoCAD Civil 3D fundamentals 67.2 1h 3m
AutoCAD Civil 3D advanced 52.2 45.5m
AutoCAD Plant 3D fundamentals 63 53m
Bentley ABD fundamentals 51.5 1h 28m
Bentley ProjectWise fundamentals 59 1h 3m
BIM Management 68.5 46m 
InfraWorks 360 fundamentals 58.5 1h 2m
Inventor fundamentals 64 1h 15m
MicroStation 2D fundamentals 59.3 1h 24m
MicroStation 3D fundamentals 52.5 58m
Navisworks fundamentals 58.4 43m
Revit Architecture fundamentals 68.5 1h 8m
Revit Architecture advanced 57.1 1h 24m
Revit Content Creation 57.8 1h 3m
Revit MEP fundamentals - Electrical 63.7 1h 6m
Revit MEP fundamentals - Mechanical 62.2 1h 7m
Revit MEP fundamentals - Plumbing 70.1 1h 11m
Revit MEP advanced 53.3 1h 2m
Revit Project Process 56.4 46.5m 
Revit Structure fundamentals 67.1 1h 12m
Revit Structure advanced 57.2 1h 4m
Rhino 61.2 44.5m
SketchUp 53.4 50m


KnowledgeSmart independent skills testing enables businesses to compare performance and capability against industry standards and benchmark their organisation against peers, competitors and partners. Measuring and demonstrating technical software experience and BIM capability is a key differentiator for AEC firms when bidding for new projects.

R

Monday, 9 November 2015

KS Country Stats

In the course of carrying out multiple thousands of skills assessments in recent years, we have gathered a wide range of fascinating and useful user demographic data. In the months ahead, we will be presenting a range of information, which can be used for measuring, comparing and benchmarking performance.

Let's take a look at some overall aggregate score data, across a number of different KS library test topics, on a country vs country basis. Numbers of test scores vary, so this list does not imply that one country is 'better' than another, per se. It's just interesting to see what kind of data is generated in different parts of the globe.


CountryScore %
Argentina 75.6
Vietnam 72.0
Nigeria 71.6
Poland 69.8
USA 69.6
New Zealand 69.0
Singapore 68.1
Sweden 67.3
Hong Kong 66.4
Saudi Arabia 65.9
Bahrain 65.8
Australia  65.5
Egypt 65.4
France 65.2
Canada  64.7
UK 63.0
Spain 62.2
UAE 61.7
Portugal 60.8
South Korea 58.8
Jordan 58.3
China 57.1
Ireland 55.7
Germany 55.3
South Africa 53.5
Philippines 52.6
India 52.4
Qatar 52.4
Italy 51.9
Kuwait 51.2
Brazil 46.0
Oman 45.9
Pakistan 45.2
Denmark 45.1





We also capture data on a per State basis, for our US customers. The list below represents those States where the volume of data makes for meaningful data sharing (i.e. if any States are missing, the overall number of tests taken is too low to publish).


US StateScore %
Illinois 76.9
Nevada 75.1
Virginia 74.9
Missouri 74.4
Louisiana 74.0
Michigan 73.6
Ohio 73.5
Delaware 73.4
District of Columbia 73.1
Utah 72.9
Texas 72.1
Minnesota 71.0
Wyoming 71.0
Oregon 70.5
Mississippi 70.1
USA (overall) 69.6
Wisconsin 69.0
Pennsylvania 68.9
California 68.8
Kentucky 68.0
New York 68.0
Georgia 67.6
Massachusetts 67.6
Maryland 67.5
Alabama 67.4
Florida 66.4
Kansas 65.9
New Jersey 65.7
Washington 64.1
South Carolina 63.1
North Carolina 60.7
Hawaii 59.7
Colorado 58.6
Arizona 57.5
Nebraska 48.8




R

Wednesday, 16 September 2015

KS Devs Update

So, the children are back to school and summer in the UK is fast becoming a memory. The autumn rains are upon us, which means it must be KS Devs update time!

We've had a busy year in 2015, keeping the KS library titles up to date with the latest releases. We've also been working steadily through the customer wishlist, improving existing tools and adding some new ones into the mix.

Here are the highlights..

New Query Tool

We've been working on new and improved ways to filter and search KS results, users and invites data.  This will be a 3-phase dev, as follows:

Phase 1

Create new query tool. We have built a new tool for searching, filtering and capturing KS data.



Query outputs can be captured in a table and exported to csv.



We are capturing a list of popular 'pre-defined' queries, which can be copied and edited.

For example:

Most popular test topics by number taken 
Average score and time for top 5 test topics
Global benchmark comparison for top 5 test topics
Users who have 'incomplete tests' 
Last 20 results: user names, test names, scores & times
Top 20 skills gaps (i.e. training tags)
Top 20 hardest test questions (i.e. lowest scoring average)

And so on.


Phase 2

Allow KS admins to perform a series of bulk actions, with the results of a query.  

For example:

Delete users
Move users from one account to another
Invite users to a test
Delete results
Move results from one account to another
Delete invites
Change invite expiry date
Re-send invites

And so on.


Phase 3

Graphics & charts: we aim to display summary fields on  a new dashboard landing page.

For example:

No. of invites sent in last 30 days
No. of completed tests in last 30 days
No. of incomplete tests in last 30 days
Most popular test topics by number taken
Average score and time for top 5 test topics
Average score for Employees vs Interviewees
Global benchmark comparison for top 5 test topics
Last 20 results: user names, test names, scores & times
Top 20 skills gaps (i.e. training tags)
Top 20 hardest test questions (i.e. lowest scoring average)

Plus offer an auto email report to KS admins, which can contain custom values and timescales.


Survey Tool

We have built a survey tool, which sits inside the KS admin dashboard and allows you to create a hybrid assessment/survey. For those times when you want to include questions in the mix, which don't have a 'right or wrong' answer. The UI is mobile responsive and captures survey results and charts in the same KS admin dashboard, for further analysis.


Other Items and a look ahead to 2016

Add Ecommerce page to KS site 

Helpdesk part 1: Add new 'Support' section in admin dashboard. Simple form to complete for tech support which is captured in KS CRM system. Future updates may follow, if more functionality and issues log is required.

Design refresh on main dashboard login page/password reset page: make page mobile responsive. 

Go live with new KS test report design.

Design refresh on main test taking UI: make mobile responsive;  add 'confidence meter' and associated admin dashboard links.

API calls: we have an ongoing dev with a third party vendor, to add more integration to training material hosted in their LMS.

Admin UI design refresh.

System mails: Revisit logic for KS system mails (invites, results, reminders) and how admins/others are copied on system mails. 

Allow more than one user group/dynamic group to be selected at a time when sending invites.  

Allow selection of default landing account for admins of multiple accounts.

Allow bulk copy & paste of emails on invite page.

Allow bulk upload/import of KS questions.

Key skills search tool.


As always, we'll keep you  posted as these devs progress to go live status.

R