Saturday, 11 June 2016

KS Dev Update - New Survey Tool

Over the past 6-12 months, the number one most requested new feature from KS customers has been to include the ability to create surveys, as well as assessments, direct from the KS admin dashboard.

So, we have been working on creating a 'survey mode' area, which enables KS admins to deploy a standalone survey, or a hybrid combined test/survey, with a mix of standard test questions and questions which do not require a 'right or wrong' answer.

Here's a walk through of the new tools:

Log in to the KS admin dashboard, in the same way as usual, but note the addition of a new 'Survey' button on the main login page:


When you log in to the admin dashboard, you'll see a new feature top right of the page. Hover over the 'Current mode' text and you'll see this popup menu:


Click the 'Switch Mode' button and you'll be taken to a new area of the admin dashboard, which hosts the editing, inviting and reporting tools for the new survey tools.

The Library > Draft Content area has a similar look & feel to the current draft tests menu layout.

There are Draft Questions:



Draft Modules:



And Draft Surveys:



Creating a survey follows the same logic and authoring process as the existing KS Assessments.

Questions

There are 4 types of question to choose from. You have a 3,000 character limit on question text:



Free Comment

This is a free text question, where the user can enter any text to an open-ended question.


There are no additional settings or inputs required, after the main question text has been entered.


The usual meta tags can be added at step 3, to help content management later on.


Yes or No

This is a straightforward yes or no question, where the user can enter a direct response to a closed question.


There are no additional settings or inputs required, after the main question text has been entered.


Multiple Choice (Select one)

This is a multiple choice question, where the user can enter one valid answer option.



Enter your answer options at edit step 2, after the main question text has been entered.


Multiple Choice (Select many)

This is a multiple choice question, where the user can enter one or more valid answer options.


Enter your answer options at edit step 2, after the main question text has been entered.



Modules

Once created, you can group questions of a similar theme into a survey module.


Select questions from your library and use the green '+' tool to add them to your module.


The usual meta tags can be added at step 3, to help content management later on.


In the survey UI, questions from each module will display on separate pages. i.e. if you have 3 modules assigned to a survey, then it will be a 3-page survey.


Surveys

Once created, you can assign one or more modules to your new survey.


Select modules from your library and use the green '+' tool to add them to your survey.


Add an introduction or welcome message and survey instructions at edit step 3, 'Survey settings'.


The usual meta tags can be added at step 3, to help content management later on.

When you have created your survey, you can capture a unique URL or a unique survey code, which can be used for inviting users to complete your survey later on.


Inviting users to take a survey

You have 3 options for inviting your users to complete a KS survey.

The first option is when you want to capture a list of users who have participated in your survey. Options 2 and 3 are for anonymous surveys, where you do not need to capture a specific record of users who have completed your survey.

Invite Users Page

Similar to the KS assessments invite process, go to the Invites > Invite Users page of your dashboard and add your users at the top of the page.


You can see a history of survey invites and statuses on the Invites > History page of your dashboard.



Unique URL

Capture the unique URL for your survey (from the section at the bottom of the survey summary page) and email your team with instructions about your survey and include the link to the survey in the body text of your mail.



Unique Survey Code

Capture the unique survey code for your survey (from the section at the bottom of the survey summary page) and email your team with this link:
https://online.knowledgesmart.net/Survey/SurveySelect.aspx.



They just need to enter the survey code to begin the survey.



Survey UI

The KS survey user interface (UI) is mobile responsive and works equally well on desktop, tablet or mobile screens.

To begin, go to the survey start page, read the instructions and hit the Continue button.


Answer the questions on the screen, by selecting the relevant answer boxes, or by typing answers to free text questions.


At the end of the survey, a simple answer summary page displays.


Click 'Finish survey' to exit the survey.


Hybrid Assessment & Survey

KS admins have the option to create a hybrid test & survey, by appending a survey at the end of a standard KS test.

Go to test edit step 4, 'Test settings' and look for the survey dropdown at the bottom of the screen. Select the survey to be added and click 'Finish' to save your changes.


Now, when a user completes their assessment, they will see a transition page, which takes them to their survey. When the survey has been completed, their test report will then display, as usual.


Survey Results

To review survey responses and charts, go to the Results > Data page. You will see a list of completed surveys in the 'Survey Results' area.


Click on a survey name to view a detailed summary of responses and charts for that particular survey.









Click the 'Export to csv' tool, to create an export of your survey responses, for additional analysis.





R

Tuesday, 26 April 2016

How Good Are Engineers at Using Revit Structure?

With the global adoption of Building Information Modelling (BIM) well underway, it is a huge challenge for Engineering businesses to manage the change in their project delivery processes from more conventional drawing production environments, to that of virtual design, clash detection and 'intelligent' modelling.

Looking at the global AEC picture, it is clear that the commercial software application adopted by a wide majority of the world's Structural Engineering firms, as their modelling tool of choice, is Autodesk's Revit Structure platform.

In recent years, KnowledgeSmart's online skills assessment software platform has captured test scores from thousands of Engineers, representing hundreds of firms, spread across six continents and a couple of dozen countries.

We have analysed the results data generated from these live assessments, at a basic skills level, and noted the following trends. For this particular study, the majority of our data is generated in the USA, UK, Canada and Australia (in that order).

A simple key to interpret the data below:

Str Eng = Job Title
RST = Revit Structure
1 to 5 = 1 to 5 years' experience using the tool
6 to 10 = 6 to 10 years' experience using the tool
Regular = use the tool daily (or almost daily)
Occasional = use the tool weekly (or less than weekly)
Self taught = learned to use the tool primarily 'on the job'
Formal = learned to the tool primarily via formal classroom or online training
All = overall KS benchmark data
AUS = Australia
CAN = Canada
UK = UK
USA = USA


Here is a summary of the results presented by different types of Engineers.

Overall

RST Fundamentals score overall: 67.1% in 4320 s

Str Eng/RST/1 to 5/regular/self taught/All = 59.3% in 4854 s
Str Eng/RST/1 to 5/regular/formal/All = 58.2% in 4738 s
Str Eng/RST/6 to 10/regular/self taught/All = 68.3% in 4141 s
Str Eng/RST/6 to 10/regular/formal/All = 67.4% in 4372 s

Str Eng/RST/1 to 5/occasional/self taught/All = 55.2% in 5353 s
Str Eng/RST/1 to 5/occasional/formal/All = 54.5% in 5244 s
Str Eng/RST/6 to 10/occasional/self taught/All = 63.3% in 4967 s
Str Eng/RST/6 to 10/occasional/formal/All = 62.5% in 4777 s

Australia

RST Fundamentals score Australia: 68.7% in 4191 s

Str Eng/RST/1 to 5/regular/self taught/US = 59.4% in 4786 s
Str Eng/RST/1 to 5/regular/formal/US = 58.0% in 4553 s
Str Eng/RST/6 to 10/regular/self taught/US = 68.6% in 4003 s
Str Eng/RST/6 to 10/regular/formal/US = 67.2% in 3967 s
 
Str Eng/RST/1 to 5/occasional/self taught/US = 53.4% in 4822 s
Str Eng/RST/1 to 5/occasional/formal/US = 52.0% in 4687 s
Str Eng/RST/6 to 10/occasional/self taught/US = 59.6% in 4566 s
Str Eng/RST/6 to 10/occasional/formal/US = 58.7 in 4330 s

Canada

RST Fundamentals score Canada: 72.4% in 3886 s

Str Eng/RST/1 to 5/regular/self taught/US = 63.4% in 4378 s
Str Eng/RST/1 to 5/regular/formal/US = 61.6% in 4233 s
Str Eng/RST/6 to 10/regular/self taught/US = 72.6% in 3717 s
Str Eng/RST/6 to 10/regular/formal/US = 71.4% in 3578 s
 
Str Eng/RST/1 to 5/occasional/self taught/US = 58.1% in 4516 s
Str Eng/RST/1 to 5/occasional/formal/US = 56.7% in 4439 s
Str Eng/RST/6 to 10/occasional/self taught/US = 63.6% in 4248 s
Str Eng/RST/6 to 10/occasional/formal/US = 62.5% in 4036 s

UK

RST Fundamentals score UK: 71.1% in 4887 s

Str Eng/RST/1 to 5/regular/self taught/US = 62.2% in 5343 s
Str Eng/RST/1 to 5/regular/formal/US = 60.6% in 5154 s
Str Eng/RST/6 to 10/regular/self taught/US = 71.3% in 4673 s
Str Eng/RST/6 to 10/regular/formal/US = 70.1% in 4491 s
 
Str Eng/RST/1 to 5/occasional/self taught/US = 58.4% in 5616 s
Str Eng/RST/1 to 5/occasional/formal/US = 56.9% in 5552 s
Str Eng/RST/6 to 10/occasional/self taught/US = 62.6% in 5254 s
Str Eng/RST/6 to 10/occasional/formal/US = 61.7% in 4987 s

USA

RST Fundamentals score USA: 73.7% in 4218 s

Str Eng/RST/1 to 5/regular/self taught/US = 64.6% in 4843 s
Str Eng/RST/1 to 5/regular/formal/US = 62.9% in 4661 s
Str Eng/RST/6 to 10/regular/self taught/US = 73.8% in 4073 s
Str Eng/RST/6 to 10/regular/formal/US = 72.1% in 3891 s
 
Str Eng/RST/1 to 5/occasional/self taught/US = 59.3% in 4916 s
Str Eng/RST/1 to 5/occasional/formal/US = 57.9% in 4856 s
Str Eng/RST/6 to 10/occasional/self taught/US = 64.8% in 4668 s
Str Eng/RST/6 to 10/occasional/formal/US = 63.7% in 4448 s


Observations

All four territories in this study are mature markets, in terms of Revit adoption. 

Each country has an established Autodesk Training Centre market, offering high quality learning and development opportunities and consultancy. 

Each country has a pool of further education colleges, community colleges and universities, many of whom teach technical software skills as part of courses in the built environment. 

Each country is well supported by professional bodies and information-rich conference series', such as BIM Forum, RTC, AU, BIM Day Out and BIM Show Live.

Overall, regular users are more accurate and faster than part-time users.

Engineers with 6 to 10 years' of experience with these tools are the most efficient. They reach a plateau in performance after c. 7 years and make no further productivity gains.

Engineers who learn to apply these tools on projects are more accurate than those who have primarily formal training, but less practical experience. Formally trained users are faster overall than self-taught users.

R

Friday, 11 March 2016

How Good Are Engineers at Using Revit MEP?

With the global adoption of Building Information Modelling (BIM) well underway, it is a huge challenge for Engineering businesses to manage the change in their project delivery processes from more conventional drawing production environments, to that of virtual design, clash detection and 'intelligent' modelling.

Looking at the global AEC picture, it is clear that the commercial software application adopted by a wide majority of the world's MEP Engineering firms, as their modelling tool of choice, is Autodesk's Revit MEP platform.

In recent years, KnowledgeSmart's online skills assessment software platform has captured test scores from thousands of Engineers, representing hundreds of firms, spread across six continents and a couple of dozen countries.

We have analysed the results data generated from these live assessments, at a basic skills level, and noted the following trends. For this particular study, the majority of our data is generated in the USA, UK, Canada and Australia (in that order).

A simple key to interpret the data below:

MEP Eng = Job Title
RMEP = Revit MEP
1 to 5 = 1 to 5 years' experience using the tool
6 to 10 = 6 to 10 years' experience using the tool
Regular = use the tool daily (or almost daily)
Occasional = use the tool weekly (or less than weekly)
Self taught = learned to use the tool primarily 'on the job'
Formal = learned to the tool primarily via formal classroom or online training
All = overall KS benchmark data
AUS = Australia
CAN = Canada
UK = UK
USA = USA


Here is a summary of the results presented by different types of Engineers.

Overall

RMEP Fundamentals score overall: 66.3% in 4084 s

MEP Eng/RMEP/1 to 5/regular/self taught/All = 58.6% in 4771 s
MEP Eng/RMEP/1 to 5/regular/formal/All = 57.7% in 4626 s
MEP Eng/RMEP/6 to 10/regular/self taught/All = 67.1% in 4188 s
MEP Eng/RMEP/6 to 10/regular/formal/All = 66.4% in 4111 s
MEP Eng/RMEP/1 to 5/occasional/self taught/All = 53.3% in 5146 s
MEP Eng/RMEP/1 to 5/occasional/formal/All = 52.8% in 5025 s
MEP Eng/RMEP/6 to 10/occasional/self taught/All = 61.3% in 4644 s
MEP Eng/RMEP/6 to 10/occasional/formal/All = 60.4% in 4561 s

Australia

RMEP Fundamentals score Australia: 66.7% in 4079 s

MEP Eng/RMEP/1 to 5/regular/self taught/AUS = 57.8% in 4971 s
MEP Eng/RMEP/1 to 5/regular/formal/AUS = 57.0% in 4786 s
MEP Eng/RMEP/6 to 10/regular/self taught/AUS = 66.8% in 4006 s
MEP Eng/RMEP/6 to 10/regular/formal/AUS = 65.9% in 3768 s
 
MEP Eng/RMEP/1 to 5/occasional/self taught/AUS = 53.7% in 5116 s
MEP Eng/RMEP/1 to 5/occasional/formal/AUS = 52.6% in 4910 s
MEP Eng/RMEP/6 to 10/occasional/self taught/AUS = 59.8% in 4557 s
MEP Eng/RMEP/6 to 10/occasional/formal/AUS = 58.7% in 4388 s

Canada

RMEP Fundamentals score Canada: 68.1% in 4391 s

MEP Eng/RMEP/1 to 5/regular/self taught/CAN = 60.5% in 4835 s
MEP Eng/RMEP/1 to 5/regular/formal/CAN = 58.8% in 4754 s
MEP Eng/RMEP/6 to 10/regular/self taught/CAN = 68.6% in 4272 s
MEP Eng/RMEP/6 to 10/regular/formal/CAN = 67.3% in 4113 s
MEP Eng/RMEP/1 to 5/occasional/self taught/CAN = 55.3% in 5002 s
MEP Eng/RMEP/1 to 5/occasional/formal/CAN = 54.2% in 4849 s
MEP Eng/RMEP/6 to 10/occasional/self taught/CAN = 62.2% in 4458 s
MEP Eng/RMEP/6 to 10/occasional/formal/CAN = 61.4% in 4367 s

UK

RMEP Fundamentals score UK: 69.4% in 3705 s

MEP Eng/RMEP/1 to 5/regular/self taught/UK = 61.2% in 4216 s
MEP Eng/RMEP/1 to 5/regular/formal/UK = 60.6% in 4011 s
MEP Eng/RMEP/6 to 10/regular/self taught/UK = 69.5% in 3665 s
MEP Eng/RMEP/6 to 10/regular/formal/UK = 68.5% in 3487 s
 
MEP Eng/RMEP/1 to 5/occasional/self taught/UK = 56.2% in 4558 s
MEP Eng/RMEP/1 to 5/occasional/formal/UK = 54.3% in 4543 s
MEP Eng/RMEP/6 to 10/occasional/self taught/UK = 62.4% in 4313 s
MEP Eng/RMEP/6 to 10/occasional/formal/UK = 61.6% in 4282 s

USA

RMEP Fundamentals score USA: 67.3% in 4046 s

MEP Eng/RMEP/1 to 5/regular/self taught/US = 59.6% in 4632 s
MEP Eng/RMEP/1 to 5/regular/formal/US = 57.8% in 4411 s
MEP Eng/RMEP/6 to 10/regular/self taught/US = 67.6% in 3933 s
MEP Eng/RMEP/6 to 10/regular/formal/US = 66.7% in 3731 s
 
MEP Eng/RMEP/1 to 5/occasional/self taught/US = 55.1% in 4806 s
MEP Eng/RMEP/1 to 5/occasional/formal/US = 53.2% in 4723 s
MEP Eng/RMEP/6 to 10/occasional/self taught/US = 60.8% in 4543 s
MEP Eng/RMEP/6 to 10/occasional/formal/US = 59.7% in 4368 s


Observations

All four territories in this study are mature markets, in terms of Revit adoption. 

Each country has an established Autodesk Training Centre market, offering high quality learning and development opportunities and consultancy. 

Each country has a pool of further education colleges, community colleges and universities, many of whom teach technical software skills as part of courses in the built environment. 

Each country is well supported by professional bodies and information-rich conference series', such as BIM Forum, RTC, AU, BIM Day Out and BIM Show Live.

Overall, regular users are more accurate and faster than part-time users.

Engineers with 6 to 10 years' of experience with these tools are the most efficient. They reach a plateau in performance after c. 7 years and make no further productivity gains.

Engineers who learn to apply these tools on projects are more accurate than those who have primarily formal training, but less practical experience. Formally trained users are faster overall than self-taught users.

R

Saturday, 27 February 2016

How Good Are Architects at Using MicroStation?

Architects around the world have been using 2D draughting tools on live projects for over 20 years.  One of the most popular applications for design firms is Bentley Systems' MicroStation platform.

In recent years, KnowledgeSmart's online skills assessment software platform has captured test scores from thousands of Architects, representing hundreds of design firms, spread across six continents and a couple of dozen countries.

We have analysed the results data generated from these live assessments, at a basic skills level, and noted the following trends. For this particular study, the majority of our data is generated in the UK, USA, Australia and Canada (in that order).

A simple key to interpret the data below:

Architect = Job Title
MicroStation = MicroStation 2D skills
1 to 5 = 1 to 5 years' experience using the tool
6 to 10 = 6 to 10 years' experience using the tool
11 to 15 = 11 to 15 years' experience using the tool
Regular = use the tool daily (or almost daily)
Occasional = use the tool weekly (or less than weekly)
Self taught = learned to use the tool primarily 'on the job'
Formal = learned to the tool primarily via formal classroom or online training
All = overall KS benchmark data
AUS = Australia
CAN = Canada
UK = UK
USA = USA

Here is a summary of the results presented by different types of Architects.

Overall

MicroStation score overall: 59.3% in 5040 secs

Architect/MicroStation/1 to 5/regular/self taught/all = 53.8% in 5249 secs
Architect/MicroStation/1 to 5/regular/formal/all = 53.1% in 5131 secs
Architect/MicroStation/6 to 10/regular/self taught/all = 60.4% in 4974 secs
Architect/MicroStation/6 to 10/regular/formal/all = 59.9% in 4854 secs
Architect/MicroStation/11 to 15/regular/self taught/all = 59.7% in 4824 secs
Architect/MicroStation/11 to 15/regular/formal/all = 59.3% in 4754 secs

Architect/MicroStation/1 to 5/occasional/self taught/all = 49.5% in 5657 secs
Architect/MicroStation/1 to 5/occasional/formal/all = 48.8% in 5520 secs
Architect/MicroStation/6 to 10/occasional/self taught/all = 55.3% in 5309 secs
Architect/MicroStation/6 to 10/occasional/formal/all = 55.2% in 5267 secs
Architect/MicroStation/11 to 15/occasional/self taught/all = 55.8% in 5278 secs
Architect/MicroStation/11 to 15/occasional/formal/all = 55.1% in 5222 secs

Australia

MicroStation score Australia: 58.7% in 4873 secs

Architect/MicroStation/1 to 5/regular/self taught/AUS = 53.6% in 5201 secs
Architect/MicroStation/1 to 5/regular/formal/AUS = 53.0% in 5160 secs
Architect/MicroStation/6 to 10/regular/self taught/AUS = 59.2% in 4954 secs
Architect/MicroStation/6 to 10/regular/formal/AUS = 58.8% in 4889 secs
Architect/MicroStation/11 to 15/regular/self taught/AUS = 58.8% in 4876 secs
Architect/MicroStation/11 to 15/regular/formal/AUS = 58.2% in 4565 secs

Architect/MicroStation/1 to 5/occasional/self taught/AUS = 48.1% in 5595 secs
Architect/MicroStation/1 to 5/occasional/formal/AUS = 47.3% in 5476 secs
Architect/MicroStation/6 to 10/occasional/self taught/AUS = 54.9% in 5212 secs
Architect/MicroStation/6 to 10/occasional/formal/AUS = 54.2% in 5131 secs
Architect/MicroStation/11 to 15/occasional/self taught/AUS = 55.2% in 5152 secs
Architect/MicroStation/11 to 15/occasional/formal/AUS = 54.4% in 5059 secs

Canada

MicroStation score Canada: 57.7% in 5172 secs

Architect/MicroStation/1 to 5/regular/self taught/CAN = 52.7% in 5421 secs
Architect/MicroStation/1 to 5/regular/formal/CAN = 52.6% in 5387 secs
Architect/MicroStation/6 to 10/regular/self taught/CAN = 58.2% in 5178 secs 
Architect/MicroStation/6 to 10/regular/formal/CAN = 57.8% in 5102 secs
Architect/MicroStation/11 to 15/regular/self taught/CAN = 58.3% in 5046 secs
Architect/MicroStation/11 to 15/regular/formal/CAN = 57.6% in 5012 secs

Architect/MicroStation/1 to 5/occasional/self taught/CAN = 47.4% in 5754 secs
Architect/MicroStation/1 to 5/occasional/formal/CAN = 46.8% in 5632 secs
Architect/MicroStation/6 to 10/occasional/self taught/CAN = 54.6% in 5512 secs
Architect/MicroStation/6 to 10/occasional/formal/CAN = 53.8% in 5466 secs
Architect/MicroStation/11 to 15/occasional/self taught/CAN = 55.1% in 5432 secs

Architect/MicroStation/11 to 15/occasional/formal/CAN = 54.7% in 5313 secs


UK

MicroStation score UK: 61.3% in 4859 secs

Architect/MicroStation/1 to 5/regular/self taught/UK = 55.5% in 5285 secs
Architect/MicroStation/1 to 5/regular/formal/UK = 55.1% in 5222 secs
Architect/MicroStation/6 to 10/regular/self taught/UK = 61.5% in 4862 secs
Architect/MicroStation/6 to 10/regular/formal/UK = 61.1% in 4828 secs
Architect/MicroStation/11 to 15/regular/self taught/UK = 61.3% in 4901 secs
Architect/MicroStation/11 to 15/regular/formal/UK = 60.8% in 4788 secs

Architect/MicroStation/1 to 5/occasional/self taught/UK = 49.7% in 5456 secs
Architect/MicroStation/1 to 5/occasional/formal/UK = 49.1% in 5402 secs
Architect/MicroStation/6 to 10/occasional/self taught/UK = 56.3% in 5232 secs
Architect/MicroStation/6 to 10/occasional/formal/UK = 55.6% in 5179 secs
Architect/MicroStation/11 to 15/occasional/self taught/UK = 56.7% in 5143 secs
Architect/MicroStation/11 to 15/occasional/formal/UK = 55.8% in 5056 secs

USA

MicroStation score USA: 59.2% in 5374 secs

Architect/MicroStation/1 to 5/regular/self taught/US = 54.2% in 5767 secs
Architect/MicroStation/1 to 5/regular/formal/US = 53.6% in 5647 secs
Architect/MicroStation/6 to 10/regular/self taught/US = 59.6% in 5387 secs
Architect/MicroStation/6 to 10/regular/formal/US = 59.2 in 5322 secs
Architect/MicroStation/11 to 15/regular/self taught/US = 59.5% in 5278 secs
Architect/MicroStation/11 to 15/regular/formal/US = 59.2% in 5166 secs

Architect/MicroStation/1 to 5/occasional/self taught/US = 48.4% in 5767 secs
Architect/MicroStation/1 to 5/occasional/formal/US = 47.7% in 5665 secs
Architect/MicroStation/6 to 10/occasional/self taught/US = 55.1% in 5489 secs
Architect/MicroStation/6 to 10/occasional/formal/US = 54.8% in 5447 secs 
Architect/MicroStation/11 to 15/occasional/self taught/US = 55.6% in 5420 secs
Architect/MicroStation/11 to 15/occasional/formal/US = 55.1% in 5391 secs


Observations

All four territories in this study are mature markets, in terms of MicroStation adoption. 

Each country is an established Bentley Systems' market, with a high quality learning and development environment and vendor consultancy support. 

Each country is well supported by professional bodies, AEC industry conferences and established vendor user community groups and online resources.

Overall, regular users are more accurate and faster than part-time users.

Architects with 6 to 10 years' of experience with these tools are the most efficient. They reach a plateau in performance after c. 7 years and make no further productivity gains.

Architects who learn to apply these tools on projects are more accurate than those who have primarily formal training, but less practical experience. Formally trained users are faster overall than self-taught users.


R

Sunday, 21 February 2016

How Good Are Architects at Using Revit Architecture?

With the global adoption of Building Information Modelling (BIM) well underway, it is a huge challenge for Architectural design businesses to manage the change in their project delivery processes from more conventional draughting and sketching production environments, to that of virtual design and 'intelligent' modelling.

Looking at the global AEC picture, it is clear that the commercial software application adopted by over half of the world's design firms, as their modelling tool of choice, is Autodesk's Revit Architecture platform.

In recent years, KnowledgeSmart's online skills assessment software platform has captured test scores from thousands of Architects, representing hundreds of design firms, spread across six continents and a couple of dozen countries.

We have analysed the results data generated from these live assessments, at a basic skills level, and noted the following trends. For this particular study, the majority of our data is generated in the USA, UK, Australia and Canada (in that order).

A simple key to interpret the data below:

Architect = Job Title
RAC = Revit Architecture
1 to 5 = 1 to 5 years' experience using the tool
6 to 10 = 6 to 10 years' experience using the tool
11 to 15 = 11 to 15 years' experience using the tool
Regular = use the tool daily (or almost daily)
Occasional = use the tool weekly (or less than weekly)
Self taught = learned to use the tool primarily 'on the job'
Formal = learned to the tool primarily via formal classroom or online training
All = overall KS benchmark data
AUS = Australia
CAN = Canada
UK = UK
USA = USA


Here is a summary of the results presented by different types of Architects.

Overall

RAC Fundamentals score overall: 68.8% in 4764 secs

Architect/RAC/1 to 5/regular/self taught/All = 63.3% in 5121 secs
Architect/RAC/1 to 5/regular/formal/All = 62.8% in 4976 secs
Architect/RAC/6 to 10/regular/self taught/All = 69.4% in 4588 secs
Architect/RAC/6 to 10/regular/formal/All = 68.7% in 4461 secs
Architect/RAC/11 to 15/regular/self taught/All = 69.7% in 4701 secs
Architect/RAC/11 to 15/regular/formal/All = 68.4% in 4529 secs
Architect/RAC/1 to 5/occasional/self taught/All = 58.4% in 5446 secs
Architect/RAC/1 to 5/occasional/formal/All = 57.8% in 5228 secs
Architect/RAC/6 to 10/occasional/self taught/All = 66.7% in 4843 secs
Architect/RAC/6 to 10/occasional/formal/All = 65.9% in 4677 secs
Architect/RAC/11 to 15/occasional/self taught/All = 67.4% in 4802 secs
Architect/RAC/11 to 15/occasional/formal/All = 66.5% in 4646 secs

Australia

RAC Fundamentals score Australia: 69.4% in 4737 secs

Architect/RAC/1 to 5/regular/self taught/AUS = 63.8% in 5181 secs
Architect/RAC/1 to 5/regular/formal/AUS = 62.3% in 4986 secs
Architect/RAC/6 to 10/regular/self taught/AUS = 69.8% in 4836 secs
Architect/RAC/6 to 10/regular/formal/AUS = 69.1% in 4768 secs
Architect/RAC/11 to 15/regular/self taught/AUS = 69.4% in 4787 secs
Architect/RAC/11 to 15/regular/formal/AUS = 68.8% in 4679 secs
Architect/RAC/1 to 5/occasional/self taught/AUS = 57.4% in 5336 secs
Architect/RAC/1 to 5/occasional/formal/AUS = 56.5% in 5210 secs
Architect/RAC/6 to 10/occasional/self taught/AUS = 65.8% in 4757 secs
Architect/RAC/6 to 10/occasional/formal/AUS = 65.2% in 4588 secs
Architect/RAC/11 to 15/occasional/self taught/AUS = 66.4% in 4818 secs
Architect/RAC/11 to 15/occasional/formal/AUS = 66.2% in 4701 secs

Canada

RAC Fundamentals score Canada: 70.1% in 4341 secs

Architect/RAC/1 to 5/regular/self taught/CAN = 64.1% in 5227 secs
Architect/RAC/1 to 5/regular/formal/CAN = 63.6% in 5026 secs
Architect/RAC/6 to 10/regular/self taught/CAN = 70.6% in 4532 secs
Architect/RAC/6 to 10/regular/formal/CAN = 70.3% in 4311 secs
Architect/RAC/11 to 15/regular/self taught/CAN = 70.8% in 4488 secs
Architect/RAC/11 to 15/regular/formal/CAN = 70.4% in 4501 secs
Architect/RAC/1 to 5/occasional/self taught/CAN = 58.3% in 5411 secs
Architect/RAC/1 to 5/occasional/formal/CAN = 57.2% in 5317 secs
Architect/RAC/6 to 10/occasional/self taught/CAN = 65.7% in 4858 secs
Architect/RAC/6 to 10/occasional/formal/CAN = 64.8% in 4767 secs
Architect/RAC/11 to 15/occasional/self taught/CAN = 66.1% in 4777 secs
Architect/RAC/11 to 15/occasional/formal/CAN = 65.8% in 4689 secs

UK

RAC Fundamentals score UK: 73.6% in 4783 secs

Architect/RAC/1 to 5/regular/self taught/UK = 65.2% in 5075 secs
Architect/RAC/1 to 5/regular/formal/UK = 64.5% in 5033 secs
Architect/RAC/6 to 10/regular/self taught/UK = 73.1% in 4767 secs
Architect/RAC/6 to 10/regular/formal/UK = 72.4% in 4554 secs
Architect/RAC/11 to 15/regular/self taught/UK = 73.5% in 4748 secs
Architect/RAC/11 to 15/regular/formal/UK = 72.6% in 4579 secs
Architect/RAC/1 to 5/occasional/self taught/UK = 60.3% in 5186 secs
Architect/RAC/1 to 5/occasional/formal/UK = 59.3% in 5032 secs
Architect/RAC/6 to 10/occasional/self taught/UK = 67.4% in 4711 secs
Architect/RAC/6 to 10/occasional/formal/UK = 66.9% in 4665 secs
Architect/RAC/11 to 15/occasional/self taught/UK = 67.4% in 4788 secs
Architect/RAC/11 to 15/occasional/formal/UK = 67.1% in 4667 secs

USA

RAC Fundamentals score USA: 72.4% in 4672 secs

Architect/RAC/1 to 5/regular/self taught/US = 64.7% in 5039 secs
Architect/RAC/1 to 5/regular/formal/US = 63.8% in 4911 secs
Architect/RAC/6 to 10/regular/self taught/US = 72.6% in 4577 secs
Architect/RAC/6 to 10/regular/formal/US = 71.7% in 4491 secs
Architect/RAC/11 to 15/regular/self taught/US = 72.3% in 4632 secs
Architect/RAC/11 to 15/regular/formal/US = 71.6% in 4478 secs
Architect/RAC/1 to 5/occasional/self taught/US = 59.1% in 5206 secs
Architect/RAC/1 to 5/occasional/formal/US = 58.2% in 5123 secs
Architect/RAC/6 to 10/occasional/self taught/US = 66.9% in 4743 secs
Architect/RAC/6 to 10/occasional/formal/US = 66.1% in 4668 secs
Architect/RAC/11 to 15/occasional/self taught/US = 67.3% in 4725 secs
Architect/RAC/11 to 15/occasional/formal/US = 66.4% in 4512 secs


Observations

All four territories in this study are mature markets, in terms of Revit adoption. 

Each country has an established Autodesk Training Centre market, offering high quality learning and development opportunities and consultancy. 

Each country has a pool of further education colleges, community colleges and universities, many of whom teach technical software skills as part of courses in the built environment. 

Each country is well supported by professional bodies and information-rich conference series', such as BIM Forum, RTC, AU, BIM Day Out and BIM Show Live.

Overall, regular users are more accurate and faster than part-time users.

Architects with 6 to 10 years' of experience with these tools are the most efficient. They reach a plateau in performance after c. 7 years and make no further productivity gains.

Architects who learn to apply these tools on projects are more accurate than those who have primarily formal training, but less practical experience. Formally trained users are faster overall than self-taught users.

R

Saturday, 30 January 2016

Global Test Scores Posted by Structural Engineers

Structural Engineers from across the world have submitted a wide variety of KS test scores, in recent years. Let’s take a more detailed look at the people behind the headline stats.

We capture some interesting user demographic data at the start of each test session. This includes the following questions:

- Primary Industry/Discipline
- Primary Role
- Country & State
- Self rating
- How many years have you used this tool?
- Where did you learn to use this tool?
- How often do you use this tool? (Daily, weekly, occasional)
- How did you learn to use this tool? (On the job, Formal training, College)


The top 2 tests completed by Structural Engineers are, in order (by volume):

Revit Structure
AutoCAD


The results data comes from a wide range of countries and territories, primarily, US, Canada, UK, Australia, South Africa, Singapore, HK and Middle East regions.

Here are the overall results:

Revit Structure

Avg score/time for:

Regular users = 71.1% in 4201 secs
Part-time users = 67.2% in 3871 secs
1 to 5 years = 66.7% in 4110 secs
6 to 10 years = 69.9% in 3489 secs
11 to 15 years = 67.1% in 3892 secs
On the job learning = 69.1% in 4287 secs
Formal learning = 64.3% in 3956 secs

The average Revit Structure self-rating score for Structural Engineers is: 3.7.


AutoCAD

Avg score/time for:

Regular users = 56.7% in 4756 secs
Part-time users = 54.9% in 5064 secs
1 to 5 years = 51.3% in 4536 secs
6 to 10 years = 56.3% in 4201 secs
11 to 15 years = 56.7% in 5337 secs
On the job learning = 54.8% in 4466 secs
Formal learning = 52.1% in 4215 secs

The average AutoCAD self-rating score for Structural Engineers is: 3.3.


Trends:

Overall, regular users are more accurate and faster than part-time users.

Structural Engineers with 6 to 10 years' of experience with these tools are the most efficient. They reach a plateau in performance after c. 7 years and make no further productivity gains.

Structural Engineers who learn to apply these tools on projects are more accurate than those who have primarily formal training, but less practical experience. Formally trained users are faster overall than self-taught users.

Structural Engineers are generally confident in their ability to use technical software tools on Construction projects, although overall, there is still room for further improvement in their core skills.

Structural Engineers are more confident and more accurate using modeling software than drafting software.

R