'BIG 5' Benchmark Stats

If your firm increases productivity by 1% per person annually, you gain 2 extra working days per team member.

How do you measure your firm's productivity against your peers and the wider AEC industry?

KnowledgeSmart has been capturing test scores across a wide range of technical software products, used by Architects & Engineers around the world.

We have data-mined thousands of results for the 'BIG 5' Autodesk software titles, as follows:

Revit Architecture
Revit MEP
Revit Structure
AutoCAD
AutoCAD Civil 3D

The results form a fascinating insight into the way these tools are deployed and utilised in different territories around the world. This is the most detailed study of software proficiency ever carried out with live test data, across thousands of users and hundreds of AEC businesses.

A detailed PDF report can be downloaded for each topic from this microsite.

Now, for the first time, management teams can compare their in-house performance and technical software capability against the wider Global industry benchmark.

This data can be included in the hiring process, new team selection, annual appraisals and when bidding for new project work.

R

KS exhibiting at Autodesk University

A key theme for us this year is that of measuring technical software skills and performance, against a wider body of results data. We are featuring our analysis of the 'BIG 5' Autodesk skills assessments at this year's Autodesk University conference, which takes place from 15-17 November, in Las Vegas, USA.  Here is the conference link: http://au.autodesk.com.

The BIG 5 tests are:
Revit Architecture, Revit Structure, Revit MEP, AutoCAD and AutoCAD Civil 3D.

KnowledgeSmart will be exhibiting in the main hall, at booth number 2810.

We look forward to seeing you there!

R

How Good Are Engineers at Using Revit MEP?

With the global adoption of Building Information Modelling (BIM) well underway, it is a huge challenge for Engineering businesses to manage the change in their project delivery processes from more conventional drawing production environments, to that of virtual design, clash detection and 'intelligent' modelling.

Looking at the global AEC picture, it is clear that the commercial software application adopted by a wide majority of the world's MEP Engineering firms, as their modelling tool of choice, is Autodesk's Revit MEP platform.

In recent years, KnowledgeSmart's online skills assessment software platform has captured test scores from thousands of Engineers, representing hundreds of firms, spread across six continents and a couple of dozen countries.

We have analysed the results data generated from these live assessments, at a basic skills level, and noted the following trends. For this particular study, the majority of our data is generated in the USA, UK, Canada and Australia (in that order).

A simple key to interpret the data below:

MEP Eng = Job Title

RMEP = Revit MEP

1 to 5 = 1 to 5 years' experience using the tool

6 to 10 = 6 to 10 years' experience using the tool

Regular = use the tool daily (or almost daily)

Occasional = use the tool weekly (or less than weekly)

Self taught = learned to use the tool primarily 'on the job'

Formal = learned to the tool primarily via formal classroom or online training

All = overall KS benchmark data

AUS = Australia

CAN = Canada

UK = UK

USA = USA

Here is a summary of the results presented by different types of Engineers.

Overall

RMEP Fundamentals score overall: 66.3% in 4084 s

MEP Eng/RMEP/1 to 5/regular/self taught/All = 58.6% in 4771 s

MEP Eng/RMEP/1 to 5/regular/formal/All = 57.7% in 4626 s

MEP Eng/RMEP/6 to 10/regular/self taught/All = 67.1% in 4188 s

MEP Eng/RMEP/6 to 10/regular/formal/All = 66.4% in 4111 s

MEP Eng/RMEP/1 to 5/occasional/self taught/All = 53.3% in 5146 s

MEP Eng/RMEP/1 to 5/occasional/formal/All = 52.8% in 5025 s

MEP Eng/RMEP/6 to 10/occasional/self taught/All = 61.3% in 4644 s

MEP Eng/RMEP/6 to 10/occasional/formal/All = 60.4% in 4561 s

Australia

RMEP Fundamentals score Australia: 66.7% in 4079 s

MEP Eng/RMEP/1 to 5/regular/self taught/AUS = 57.8% in 4971 s

MEP Eng/RMEP/1 to 5/regular/formal/AUS = 57.0% in 4786 s

MEP Eng/RMEP/6 to 10/regular/self taught/AUS = 66.8% in 4006 s

MEP Eng/RMEP/6 to 10/regular/formal/AUS = 65.9% in 3768 s

MEP Eng/RMEP/1 to 5/occasional/self taught/AUS = 53.7% in 5116 s

MEP Eng/RMEP/1 to 5/occasional/formal/AUS = 52.6% in 4910 s

MEP Eng/RMEP/6 to 10/occasional/self taught/AUS = 59.8% in 4557 s

MEP Eng/RMEP/6 to 10/occasional/formal/AUS = 58.7% in 4388 s

Canada

RMEP Fundamentals score Canada: 68.1% in 4391 s

MEP Eng/RMEP/1 to 5/regular/self taught/CAN = 60.5% in 4835 s

MEP Eng/RMEP/1 to 5/regular/formal/CAN = 58.8% in 4754 s

MEP Eng/RMEP/6 to 10/regular/self taught/CAN = 68.6% in 4272 s

MEP Eng/RMEP/6 to 10/regular/formal/CAN = 67.3% in 4113 s

MEP Eng/RMEP/1 to 5/occasional/self taught/CAN = 55.3% in 5002 s

MEP Eng/RMEP/1 to 5/occasional/formal/CAN = 54.2% in 4849 s

MEP Eng/RMEP/6 to 10/occasional/self taught/CAN = 62.2% in 4458 s

MEP Eng/RMEP/6 to 10/occasional/formal/CAN = 61.4% in 4367 s

UK

RMEP Fundamentals score UK: 69.4% in 3705 s

MEP Eng/RMEP/1 to 5/regular/self taught/UK = 61.2% in 4216 s

MEP Eng/RMEP/1 to 5/regular/formal/UK = 60.6% in 4011 s

MEP Eng/RMEP/6 to 10/regular/self taught/UK = 69.5% in 3665 s

MEP Eng/RMEP/6 to 10/regular/formal/UK = 68.5% in 3487 s

MEP Eng/RMEP/1 to 5/occasional/self taught/UK = 56.2% in 4558 s

MEP Eng/RMEP/1 to 5/occasional/formal/UK = 54.3% in 4543 s

MEP Eng/RMEP/6 to 10/occasional/self taught/UK = 62.4% in 4313 s

MEP Eng/RMEP/6 to 10/occasional/formal/UK = 61.6% in 4282 s

USA

RMEP Fundamentals score USA: 67.3% in 4046 s

MEP Eng/RMEP/1 to 5/regular/self taught/US = 59.6% in 4632 s

MEP Eng/RMEP/1 to 5/regular/formal/US = 57.8% in 4411 s

MEP Eng/RMEP/6 to 10/regular/self taught/US = 67.6% in 3933 s

MEP Eng/RMEP/6 to 10/regular/formal/US = 66.7% in 3731 s

MEP Eng/RMEP/1 to 5/occasional/self taught/US = 55.1% in 4806 s

MEP Eng/RMEP/1 to 5/occasional/formal/US = 53.2% in 4723 s

MEP Eng/RMEP/6 to 10/occasional/self taught/US = 60.8% in 4543 s

MEP Eng/RMEP/6 to 10/occasional/formal/US = 59.7% in 4368 s

Observations

All four territories in this study are mature markets, in terms of Revit adoption. 

Each country has an established Autodesk Training Centre market, offering high quality learning and development opportunities and consultancy. 

Each country has a pool of further education colleges, community colleges and universities, many of whom teach technical software skills as part of courses in the built environment. 

Each country is well supported by professional bodies and information-rich conference series', such as BIM Forum, RTC, AU, BIM Day Out and BIM Show Live.

Overall, regular users are more accurate and faster than part-time users.

Engineers with 6 to 10 years' of experience with these tools are the most efficient. They reach a plateau in performance after c. 7 years and make no further productivity gains.

Engineers who learn to apply these tools on projects are more accurate than those who have primarily formal training, but less practical experience. Formally trained users are faster overall than self-taught users.

R

Global Test Scores Posted by MEP Engineers

MEP Engineers from across the world have submitted a wide variety of KS test scores, in recent years. Let’s take a more detailed look at the people behind the headline stats.

We capture some interesting user demographic data at the start of each test session. This includes the following questions:

- Primary Industry/Discipline
- Primary Role
- Country & State
- Self rating
- How many years have you used this tool?
- Where did you learn to use this tool?
- How often do you use this tool? (Daily, weekly, occasional)
- How did you learn to use this tool? (On the job, Formal training, College)

The top 2 tests completed by MEP Engineers are, in order (by volume):

Revit MEP
AutoCAD

The results data comes from a wide range of countries and territories, primarily, US, Canada, UK, Australia, South Africa, Singapore, HK and Middle East regions.

Here are the overall results:

Revit MEP

Avg score/time for:

Regular users = 68.5% in 3110 secs

Part-time users = 65.7% in 3871 secs

1 to 5 years' experience (all types) = 56.6% in 3901 secs

6 to 10 years' experience (all types) = 68.7% in 3401 secs

On the job learning (all types) = 70.2% in 3781 secs

Formal learning (all types) = 64.8% in 3911 secs

The average Revit MEP self-rating score for MEP Engineers is: 3.8.

AutoCAD

Avg score/time for:

Regular users = 59.8% in 5356 secs

Part-time users = 54.9% in 4864 secs

1 to 5 years' experience (all types) = 52.7% in 5752 secs

6 to 10 years' experience (all types) = 57.5% in 5240 secs

11 to 15 years' experience (all types) = 54.7% in 5449 secs

On the job learning (all types) = 57.8% in 4516 secs

Formal learning (all types) = 55.1% in 4344 secs

The average AutoCAD self-rating score for MEP Engineers is: 3.4.

Trends:

Overall, regular users are more accurate and faster than part-time users.

MEP Engineers with 6 to 10 years' of experience with these tools are the most efficient. They reach a plateau in performance after c. 7 years and make no further productivity gains.

MEP Engineers who learn to apply these tools on projects are more accurate than those who have primarily formal training, but less practical experience. Formally trained users are faster overall than self-taught users.

MEP Engineers are generally confident in their ability to use technical software tools on Construction projects, although overall, there is still room for further improvement in their core skills.

MEP Engineers are more confident and more accurate using modeling software than drafting software.

R