Benchmark Stats for Popular KS Test Titles

Benchmarking software capability at individual, team and office level is a key part of the KnowledgeSmart experience.

Our library of skills assessments for the AEC industry covers a variety of  popular software tools, used on projects on Construction projects worldwide.

Our results data comes from a wide variety of countries and territories, including: US, UK, Australia, Canada, New Zealand, South Africa, Singapore, Vietnam, India and the Middle East.

Here is the overall average results data for our most popular topics (% score and time to complete):

Test Title	% Score	Time
3ds Max fundamentals	54.5	57m
Adobe InDesign occasional	64	45m
Adobe Photoshop occasional	62	48m
AutoCAD 2D fundamentals	58.5	1h 13m
AutoCAD 2D occasional	73.1	36.5m
AutoCAD Civil 3D fundamentals	67.2	1h 3m
AutoCAD Civil 3D advanced	52.2	45.5m
AutoCAD Plant 3D fundamentals	63	53m
Bentley ABD fundamentals	51.5	1h 28m
Bentley ProjectWise fundamentals	59	1h 3m
BIM Management	68.5	46m
InfraWorks 360 fundamentals	58.5	1h 2m
Inventor fundamentals	64	1h 15m
MicroStation 2D fundamentals	59.3	1h 24m
MicroStation 3D fundamentals	52.5	58m
Navisworks fundamentals	58.4	43m
Revit Architecture fundamentals	68.5	1h 8m
Revit Architecture advanced	57.1	1h 24m
Revit Content Creation	57.8	1h 3m
Revit MEP fundamentals - Electrical	63.7	1h 6m
Revit MEP fundamentals - Mechanical	62.2	1h 7m
Revit MEP fundamentals - Plumbing	70.1	1h 11m
Revit MEP advanced	53.3	1h 2m
Revit Project Process	56.4	46.5m
Revit Structure fundamentals	67.1	1h 12m
Revit Structure advanced	57.2	1h 4m
Rhino	61.2	44.5m
SketchUp	53.4	50m

KnowledgeSmart independent skills testing enables businesses to compare performance and capability against industry standards and benchmark their organisation against peers, competitors and partners. Measuring and demonstrating technical software experience and BIM capability is a key differentiator for AEC firms when bidding for new projects.

R

The Smartest Human Resources Officers are 'Knowledge Smart'

According to www.Prospects.ac.uk (the UK's leading student and graduate careers advice service) Human Resources (HR) Officers must have a clear understanding of their employer's business objectives and be able to devise and implement policies which select, develop and retain the right staff needed to meet these objectives.

The smartest HR Officers are therefore subscribing to KnowledgeSmart to assess the CAD and BIM skills of new hires and to guide skills improvement over time.

We use KnowledgeSmart to enable our clients to recruit smarter candidates and to prove their ability; we find that it ensures better outcomes. By replacing guesswork with the KnowledgeSmart skills assessment, our clients know precisely how and where their candidates will add value to their organisation before they hire them.

Mike Johnson Managing Director, JohnsonBIM, London, UK

In addition, HR Officers use KnowledgeSmart skills assessments to:

- Liaise with team leaders to identify skills gaps and improve staff performance;
- Interview and select job candidates;
- Develop and implement policies for performance management;
- Advise on pay and other remuneration issues, including promotion and benefits;
- Plan, and sometimes deliver, training - including inductions for new staff;
- Analyse training needs in conjunction with departmental managers.

With subscribers spread across 6 continents KnowledgeSmart online skills assessments are used by architects, engineers, construction firms and contractors, to great effect.


Using KnowledgeSmart Skills Assessments to Inform Annual Appraisals

At HOK – a global design, architecture, engineering and planning firm with a network of 24 offices on 3 continents – the HR team actively supports staff to be the best they can be. This in turn delivers significant value for individuals and the organisation at large. With a focus on continuous improvement, HOK's skills assessment results are reviewed annually as part of the appraisal process and continuous improvement is rewarded through the company bonus scheme and through improved promotion opportunities.

We have learned lots of back-end lessons using KnowledgeSmart. Most notably, we have realised how and where we can improve productivity and profitability for the whole company.

David Ivey Building Smart Manager, Chicago, USA

HR teams, operating at the highest level within HOK, welcome this approach to integrated assessment and support the Building Smart Managers in their work. With similar motivations for improved service and performance, Stantec – a 15,000-person strong professional services firm – is embracing KnowledgeSmart to proactively make skills development a corporate initiative.

We have made the employee and his supervisor responsible for identifying goals and for measuring progress as part of the annual review process. This empowers and supports teams and enables individuals to reach their full potential and KnowledgeSmart is an invaluable ingredient.

Susan Clark-Serdiuk HR lead at Stantec, Toronto, Canada


HR Officers are Increasingly Knowledge Smart when Hiring New Staff

Smart recruitment firms and HR Officers are turning to KnowledgeSmart for insight when hiring. Gone are the days of simply sifting through CVs and arranging interviews, firms now demand more from their recruitment teams.

Hiring new staff without first assessing their BIM and CAD skills is an extraordinarily expensive way to do business as many firms may not learn how bad the new recruits have performed until much damage is done to the project's data quality. With KnowledgeSmart BIM Skills Assessments we gain the insight we need to make the right decisions.

Senior Director of Professional Services, US CAD, California, USA

To avoid such unnecessary expenditure and to reduce their risk, California-based US CAD invites all job candidates to complete a KnowledgeSmart BIM skills assessment and makes it clear from the outset that there is no intention to fail anyone. US CAD openly uses the BIM skills assessment results to better plan for training needs and to benchmark a candidate's assessed-ability against his own claimed-ability to better understand the individual.


HR Officers are Bringing KnowledgeSmart Results to the Project Resourcing Table

At RTKL – a worldwide architecture, engineering, planning and creative services organization – HR leadership challenged its BIM management team to source a BIM skills analysis tool to help the firm make smarter, more well-informed data-driven decisions about people's CAD and BIM skills when resourcing project teams. The team reviewed the market and selected KnowledgeSmart.

HR generally understands 'why' people were hired and provides a high-level view of the firm's talent base, but historically it has not fully appreciated some of the finer details about CAD and BIM skills. The nature of project-centric resourcing determines that individuals are moved from one project team to another as the workflow rises and falls across a project's lifecycle, and their BIM skill-levels will critically influence their ability to contribute to their new team. With KnowledgeSmart results and analysis in hand, HR now has perhaps the most comprehensive oversight of all staff and their skills.

Andrew Victory Associate, RTKL, London, UK


Identifying Skills Gaps for Training Needs Analysis by HR Teams

Australian-based A2K Technologies – one of the largest Autodesk channel partners globally – finds that HR Officers are very often involved in purchase decisions for training. Furthermore, A2K finds that HR Officers are very interested in benchmarking for greater insight into team performance; enabling them to extend their role beyond welfare and Health & Safety.

As the recession fades and growth again is the driver for change, firms are increasingly motivated to retain their best staff. KnowledgeSmart helps HR Officers measure their teams' software skills as part of their culture to support, reward and retain value-adding staff.

Sean Twomey ANZ Consulting Manager, A2K Technologies, Sydney, Australia

A2K also believes that KnowledgeSmart provides younger staff with targets to aim for; finding that KnowledgeSmart results can contribute to the building of a framework for ambition and career growth.

If you are a HR Professional, looking for a better way to measure staff skills and interviewee experience, why not get in touch and request a free trial of the KS tools?

R

KS Devs Update

Here is a brief update of what's being discussed at our next tech team workshop. With thanks to our customer feedback forums for the majority of these suggestions.

Our current dev is progressing well. More detailed notes to follow shortly for each, but our top 2 current wishlist items are:

User dashboard pages
Survey questions

One those devs hit the live site, we'll take a look at the following:

CSV Export Consistency

Review CSV export across the KS dashboard to ensure it is behaving in a consistent manner.

Account Name in CSV Export

Many companies have multiple child accounts but would like to perform one csv export of all users.  However there is no clear identifier in the csv export as to which account each user belongs.  Include ‘Account Name’ in CSV exports.

Improved data selection in CSV Exports

Consider implementing ‘Field Chooser’ functionality for CSV exports.  When exporting a csv, the Admin User can select which fields to appear and save this selection for future exports.  We would also look to add further options such as ‘Full Name’ (concatenating first name and surname), ‘Training Tags – All’ and ‘Training Tags – Individual’ etc.  By making this functionality available in KS we can dramatically reduce the amount of time Admin Users need to spend outside of KS manipulating the data.

Allow admins to set up regular auto-reports (i.e. weekly/monthly/quarterly) which summarise key activities. For example:  Invites sent: Not started, In progress, Pending, Completed : Results, Scores within a certain range: User type: Test name(s): Date range: Select data for a particular account or accounts. Choose recipients. Full list of selection criteria to be workshopped.

Search /Grouping Criteria – Date Range

Extend search and group criteria across the system to allow the inputting of a date range.  This provides Admin Users with more powerful options when reviewing invites, results etc. Ensure consistency across all dynamic grouping filters. (Invite History/Results/Users).

Add Test Not Completed to Status Dropdown

Currently test invites do not have a single ‘Not Completed’ status.  If an Admin User wishes to identify all outstanding tests they would need to join together ‘Not Started’ and ‘In progress’. Amend grouping options to bring together these two into a single ‘Not Completed’.

Auto-Email when extending a test

When an Admin User extends the expiry date of a test invite, admins would like KS to send an email to confirm along with the new expiry date.  This could be managed in much the same way as the standard invites are with Admin Users having the option of amending the default text. Include option to add additional email addresses.

Increase Character Limit on Multiple Choice and Matching List Answer Options

Multiple choice questions currently have a limit of 50 characters per answer.  KS to review this to identify why the limit is in place (display purposes, database restriction etc) and if it can be extended. Also look at Matching List fields.

Default for Admins of Multiple Accounts

Where an Admin is managing multiple accounts, KS currently defaults to the first account in the list (alphabetically).  Admins have requested a change in this behaviour.  KS to consider alternative default selection options.

Add column in Invites – Date of last reminder/extension

When viewing invites KS currently displays when the invite was first sent and the date and time of the Users last login.  Admins would like an additional column which advises when the User was last reminded (either by reminder or date extension).

Remove Invites from system when a User is deleted

When a User is marked as Ex-Employee there is a clear requirement to retain their interaction with KS however when a User is deleted from the system, admins would like to have any outstanding invites (Not Completed) deleted from the system.


Additional Points for Discussion

More detailed benchmarking & analysis of user demographic & results data.

New API for LMS linking (esp. Olive/Absorb/Pinnacle).

Add number of results to Results > Data page.

Add date stamp to Global Comparisons view and export.

Add user count to training requirements chart.

Review display of Global question benchmark data.

Add question ID to training request csv export.

Improve page speed: Users > Users and Invites > History.

Investigate single sign-on and Active Directory links for user data import.

Investigate Content Delivery Network - adding zip files for test start to CDN environment.


R

KS Tips for Getting Started with Your Assessments

Making a positive start with your KS assessment program is a key part of generating momentum in your wider learning and development plans.  Here is a brief guide which walks you through the right steps to make sure you get off to a flying start!

1. Discuss your goals. What do you want to achieve?  How will you define success?  Do you have management buy-in to support your plan?

2. Select your KS system administrators.  Decide who will have global access and who will have regional access.  Do you want to include sub-admins?

3. Book your KS Getting Started web meeting with the KS team and/or training partner.  Review the information on the Settings > Resources page of your KS dashboard.

4. Set up your KS accounts.  Do you need one main account or multiple linked accounts?  Add your company branding to your dashboard, test UI & reports.

5. Agree your message and decide how to communicate your plans to your team.

6. Identify a list of users to be tested.  Do you have any positive-minded ‘super users’ or champions who can be relied on to assist your plans?

7. What topics will you include in your test program?  Will you use the KS library ‘off the shelf’, or will you make some changes?  Do you want to add any custom questions of your own?

8. Build your user list.  Filter your user list into groups.

9. How much user feedback do you want to provide in your reports?  Include coaching?  Include links to learning?  Include additional user data capture?

10. Create a schedule of invites.  Who, where, when, what?  Do you want to send a custom invite message?  Will you test new hires at interview?  Who will help to administer this?

11. Send your test invites.  Monitor progress on the Invites > History page.  Set up reminders.

12. Monitor overall test progress and results.

13. Schedule results analysis workshop/web meeting with the KS team and chosen training partner.

14. Create reports and feed KS results data into your wider training and learning program. Include your preferred training partner in this phase of activity.

15. Re-test after training to monitor effectiveness of your material and measure performance improvement.


R

KS Admin FAQ's

Here are the most common questions and points of discussion for KS admins.

Q - What's the difference between a Sub-admin and a Full-admin?

A - Sub-admins can set up test sessions from a browser, using the 'Assessment' link, but not access the main KS dashboard.  Full-admins can log in to the KS dashboard and use the browser test set-up tools.

Q - How do I re-set or change my KS password?

A - All users can re-set their KS password at any time, by clicking the 'Reset password' link on the KS login page:

This takes you to the following page: https://online.knowledgesmart.net/PassRecover.aspx.  Enter your KS username and the system will email a new password.

KS admins can set their own password, by logging in to their KS dashboard and selecting the 'Change password' link.

Copy the original password into the first field, then confirm your own password and save the changes.

Q - How do I add a new KS administrator to my account?

A - Go to the Accounts > Your Accounts page of the KS dashboard and click on the 'Administrators' tool (small people icon).

From here you can add new Sub-admins or Full-admins, using the 'Add administrator' fields:

Q - I get an error message when I try to import my user list.  Why is this?

A - First, make sure that you are using a .csv file format, not .xls or .xlsx.  (See help notes on the Users > Upload User Data page for a link to a formatted .csv template).  Also, make sure that you have 10 data columns in your .csv file.


Q - My invite mails are not getting through to users every time.  Why not?

A - Sometimes KS invite mails are blocked by spam prevention software.  If the KS mails are not in junk mail, then your corporate security might not be allowing the mails through.  This happens from time to time.  A combination of the following steps usually resolves the issue:

‘White flag’ the domain @knowledgesmart.net.
‘White flag’ the domain @ks-server.net.
‘White flag’ mails from system@ks-server.net.

That should allow the mails to work OK.  However, if they are still getting blocked, there is one more thing to try.  Our hosting company (1&1) sometimes uses a relay server in Germany, called kundenserver.de.

‘White flag’ the domain @kundenserver.de.

That should sort everything out, from a corporate mails point of view.


Q - Some invite mails contain a username and password.  Others do not.  Why is this?

A - The KS system appends a username and password on invite mails for regular users (i.e. non-admins), but not for system administrators. KS admins can manage their own passwords, so the KS system does not include admin usernames and passwords on invite mails.


Q - When  a user tries to log into a test session, their password does not work.  Why is this?

A - If a user receives two or more test invites in close succession, the latter invite(s) will overwrite the password(s) from earlier mails.  Instruct them to use the password from the last email invite received to access ALL open KS test sessions.

All users can re-set their KS password at any time, by clicking the 'Reset password'' link on the KS dashboard login page, which takes you here: https://online.knowledgesmart.net/PassRecover.aspx. Enter your KS username and the system will email a new password.

Q - If  a user logs out of a test session, will all answers be saved?

A - Yes.  All your answers will be saved if you resume a test session, at a later date.


Q - If a user logs out of a test session, does the clock stop?

A - Yes.  The clock stops recording elapsed time when you log out of a test session and starts again when you log back in.


Q - A test result did not upload correctly at the end of a test session.  Can it be rescued?

A - Yes.  If a user loses their web connection at the end of a test session, occasionally a result will not upload to the dashboard successfully.  Just notify us and the KS team will manually retrieve the score and add it to your dashboard.  Submitted answers will not be lost.


Q - Can I mix & match questions from different test modules?

A - Yes. KS admins have full editorial control over all KS test material, including authoring tools for creating new questions from scratch and sorting and moving existing questions between modules and tests.


Q - Can I extend the expiry date on an invite?

A - Yes. Go to Invites > History, select your invite(s) and use the 'Extend invite' tool to change the expiry date of your invites.

Q - Can I set a time limit on a KS test?

A - Yes. Import your KS test and go to Library > Draft Content > Draft Tests, edit step 4 (Test Settings). Enter your chosen time in the 'Time limit' box.

Q - Can I hide the clock which appears on the KS test UI?

A - Yes. Import your KS test and go to Library > Draft Content > Draft Tests, edit step 4 (Test Settings). Uncheck the 'Show timer' box (see above).


Q - How do I copy admins on KS invite, reminder & results mails?

A - Go to Accounts > Administrators and toggle the account invites tool (small envelope icon) to 'on'.

Go to Settings > Invites and check the box below:

Go to Settings > Reminders and check the appropriate boxes:

Q - Is it possible to edit score and time data for KS results?

A - Yes. If you have a KS result that needs to be edited, email the details across to us and we can make the edits via the main KS db.


Q - Can I copy dynamic groups from the Users page to the Results page?

A - Yes. Go to Settings > User Datafields and click on 'Share Dynamic Groups', to share existing groups across the Users, Results and Invite History pages.

Q - Can I filter results by date of test taken?

A - Yes.  You can filter KS results data across a variety of search criteria. Go to Results > Data page and click on the 'Show search' tool to display the search options:

Q - Can I export test results?

A - Yes.  Go to the Results > Data page and click on the 'Show groups' tool.  Look for the 'Export results to csv' tool.  You have a number of export options to choose from:

Q - Can I edit the user rating labels which appear at the start of a test?

A - Yes. Go to Settings > Test UI Options and apply your changes in the fields provided.

Q - Can I choose whether or not to display the user survey pages at the start and end of a test session?

A - Yes. Go to Settings > Test UI Options and apply your change in the relevant check box.

Q - Can I stop users from logging out of a test session?

A - Yes. Go to Settings > Test UI Options (see above) and apply your change using the relevant check box.


Q - How can I compare our results to the overall average KS benchmark stats?

A - Go to Results > Global Comparisons and select the title(s) you want to compare your results data against.

R

Knowledge Trax - A New Approach

This is one of the best approaches to skills assessment and custom training that we have seen.  It's called Knowledge Trax and it combines best practice for assessment, online & class based training and service delivery.

Knowledge Trax breaks this process down into 6 simple steps:

1. Assess
2. Analyze
3. Prescribe
4. Educate
5. Re-assess
6. Refine

Here's a detailed summary of the Knowledge Trax process, to help customers measure and improve performance and productivity.

Discovery
Discovery begins with a kick-off meeting where we help you identify technology, workflow, and organizational structure, along with staff roles and responsibilities.

Next, we’ll conduct a skills assessment, which includes assessment management, building role-based assessment or creating custom assessments based on your needs, and develop custom questions based on your company’s workflow. Our management of this process ensures that all required assessments are conducted and you are able to track who within your organization has and has not taken the assessments.

Planning
In this step, we will analyze your assessment results and build an education plan that combines instructor-led classroom training, instructor-led online training, and pre-recorded self-paced online training. We will then deliver and review this plan with you.

Execution
In the Execution phase, the Knowledge Trax team delivers the training based on the approved education plan. And because these plans are based on the unique needs of your company’s staff and roles, your team may be involved in instructor-led classroom training at one of our facilities or online training.

Maintain
In this phase, we conduct an initial post assessment which will identify areas for ongoing training needs, ongoing refresher training, or gap training. Based on your staff’s individual needs, we’ll deliver the training in the method that best fits them.

Take a look at this video which tells you more about the Knowledge Trax process.

Knowledge Trax partners with KnowledgeSmart and Global eTraining to deliver a complete customer experience.

R

KS Library Update

It's been a busy few months of library updates, but we've made excellent progress this year, keeping everything up to date and moving forwards.

Advanced material, BIM and company standards appear to be the main themes for content this year.

All of the main Autodesk titles are available in 2012, 2013 and 2014 formats. Here's a summary of live titles:

Our newest titles include: Inventor (parts 1-4), AutoCAD Plant 3D (parts 1-4), Revit Structure advanced and Graphisoft ArchiCAD (parts 1-4).  We've also created a French version of our popular Revit Architecture Fundamentals  test.

Here's what we're working on at the moment:

AutoCAD Civil 3D advanced
Revit MEP advanced
Revit MEP fundamentals (metric data sets)
Autodesk Vault
Bentley AECOsim Building Designer
Synchro
Tekla Structures

If there are any topics you would like to see, in addition to those topics listed here, do please let us have your feedback and we'll add it to our rolling list of library updates.

R

KS summer update

As usual, it has been a busy summer period for the KS team. Our 2014 library updates are in full swing, due to go live later this month. And the next release of the KS software platform is also due to go live shortly.

Here is a brief summary of what's included in this release:

Content management

We made a number of updates to the KS admin dashboard, to make searching & grouping library material more consistent.

You can now search on a range of new fields to manage your question library, including: question name, training tags, question type, skill level, parent module, parent test and task (T) or knowledge (K).

Draft and Published modules can now be searched and grouped:

Draft and Published tests can also now be searched and grouped:

Due to popular request, we added a delete option for child accounts. Be careful when selecting this option, however, as all related user and results data will also be removed!

Another popular request, we added the option to toggle off tests in your library, so they do not appear in your invite list dropdown (both in your dashboard Invites page and in the browser login dropdown menu).

We also added the option to hide the 'Logout' link on the test UI:

We added an extra check box to cc admins on test invite mails.

Results

You can now create a chart view of the questions which cause the 'Request training' button to be selected during live test sessions. This chart represents the most commonly requested training issues from your team.

You can now create a chart view of the questions which cause the 'Skip question' button to be selected during live test sessions. This chart represents the most commonly skipped questions issues from your team.

You can now include results from all linked accounts, when viewing data on the ‘Group Comparisons’ page. This makes results analysis across multi-office firms more meaningful.

You can now include results from all linked accounts, when viewing data on the ‘Global Comparisons’ page:

We added a new csv export for question results data, on the Results > Question Performance page:

This report displays: test name, test date, test score, test time, result ID, score & time values per question, average question score & time per question and training tags for each question.

We added a new tool for viewing 'required' and 'submitted' answer data for each question, on a per test basis.

Test UI

We have made a number of improvements to the code which drives the test UI and user experience during live test sessions.  This update addresses the issue of occasional error messages which display when submitting answers and removes the phantom page 'flicker'.

Plus a number of smaller updates, including:

Increased word count on the test settings welcome message/user instructions field.
Change advisory message on pick list questions.
Increase system timeout from 20 to 40 mins (if no onscreen activity occurs).
Main website refresh, including customer testimonials, product table and promo movie.
Updates to our own KSadmin interface, for analysing results and user data.

Later this month, we will also be adding the following updates:
New benchmark charts dropdown on Results > Global Comparisons page and include 2013 data, plus new test titles.
Module Performance chart page and csv export with per module results data.
Skipped questions search filter & csv export on Results > Data page.

Work is already under way on the technical specification for the Fall release, which will include; new question types, survey options, user data capture reporting, plus the latest customer wish list requests.

R

Learning Plateaus

At KnowledgeSmart, we spend a lot of time speaking to firms about performance improvement. So what happens when, in the pursuit of advanced learning, we experience a slow-down in our development? We hit what is sometimes referred to as a 'learning plateau'.

A learning plateau occurs when forward progress seems to have stopped while engaged in learning a new skill. These plateaus are normal and commonly experienced periodically when learning to play a musical instrument, speak a new language, or learn some other complex discipline.

Dutch author, Lodewijk van den Broek, describes hitting the plateau as, ‘the experience where you feel that no matter how hard you try, there is no progress in learning. And even though this is not entirely true, the feeling is very real’.

According to van den Broek, there are two major things happening here. The first is that you have progressed past the bend in the 'learning curve'. There are different stages in the learning curve:

•  Beginner stage (0% to 60%)

The curve is very steep and you learn very fast.

•  Intermediate stage (60% to 80%)

This is past the bend in the total curve. Learning speed declines and this phase takes longer than the beginner stage.

•  Advanced stage (80% to 95%)

Learning speed declines even further and the length of the stage increases as again.

•  Expert or Master stage (95% to 100%)

Learning speed drops to slow progress and the length of the stage stretches into eternity.

So once you hit Intermediate level, your progress slows down. And at the same time it gets increasingly difficult to measure that progress. These effects combined result in the feeling that you have hit a plateau in learning.

US authors, Brett & Kate McKay, describe a road map for overcoming Performance Plateaus. Back in the 1960’s, two psychologists, Paul Fitts and Michael Posner, set out to uncover why we plateau. They discovered that when we acquire a skill, we go through three stages.

The first stage of skill acquisition is called the cognitive phase. In this phase, we must concentrate intently on what we’re doing as we figure out strategies on how to accomplish the skill more efficiently and effectively. The cognitive phase is riddled with mistakes as we learn the ins and outs of our new pursuit.

The second phase of skill acquisition is the associative phase. During the associative phase, we make fewer mistakes. Consequently, we feel more comfortable with the skill and begin to concentrate less on what we’re doing.

The final stage is the autonomous phase, or what Joshua Foer, author of ‘Moonwalking with Einstein’, calls the “OK plateau.” We reach a skill level where we’re able to capably do the task without having to really think about it at all. Remember about how much you thought about what you were doing when you first got your driver’s license? Now driving is fairly automatic, like brushing your teeth.

People used to think that you couldn't break past these plateaus because a plateau represented the limit of your genetic ability. No amount of exertion or education would help you overcome this wall. But Fitts, Posner, and other psychologists discovered that with the right approach and a few attitude adjustments, all of us can bust through our plateaus and reach even higher.

How to Overcome Plateaus

Take risks. Growth comes when we stretch past our comfort zone. The big reason many people (especially high-achievers) plateau is because they don’t like to fail. Instead of taking on challenges that will help us grow, we stick with routines that we know we can successfully do. To protect our ego, we’d rather do the wrong things correctly, than do the right things wrongly. This aversion to risk is a recipe for plateauing.

Embrace your failure. To overcome your aversion to risk, you have to give yourself permission to fail and be mediocre. Instead of avoiding the things that are hardest for them, the greats of the world specifically focus on those things; they purposefully concentrate on the areas in which they make the most mistakes. This keeps them from getting stuck in the autonomous phase and propels their progress. So instead of seeing failure as a negative thing, think of your failures as steps to success. If you choose to learn from your failures, they can bring you closer to your goal.

Another reason we plateau is because everyone around us is telling us everything is fine. We often confide in people who tell us what we want to hear, not what we need to hear. For example, we finish a project and take it to somebody for some “constructive criticism,” when really we just want some positive affirmation on what we did. If you feel stuck in an area of your life, seek out mentors who won’t pull any punches and will give you the honest criticism you need to improve. Yes, your ego will get bruised, but that’s the price one must pay for personal and professional growth.

Practice deliberately. Fitts and Posner discovered three keys to breaking through your plateau: 

1) focus on technique, 2) stay goal oriented, and 3) and get immediate feedback on the performance. In other words, you need to practice deliberately to break through plateaus.

Get back to basics. Even when you’re advanced at something, delving back into the basics can actually give you fresh insights that help you progress even further.

Think long term. When we think short-term, we have a tendency to feel that plateaus are permanent. But when we take the big picture view of things, we start to see plateaus as temporary way-stations that we’ll eventually get past with a bit of hard work. Moreover, by thinking long term, we give ourselves more latitude to take risks and fail because we see that missteps are just momentary setbacks in the long journey of life.

To cultivate this attitude, reflect on a time where you felt you had reached the end of your development in some area, only to later bust through the plateau. If it was possible then, it’s possible now.

Psychology author, Russell A. Dewey, PhD, describes what happens with a typical learning curve.

With repetition of almost any motor task, learning occurs, and a person becomes more efficient or effective at carrying out a task. Progress in skill learning commonly follows an S-shaped curve, with some measure of skill on the Y axis and number of trials on the X-axis. Progress is slow at first, then a subject may experience a burst of learning that produces a rapid rise on the graph.

The S-shaped "learning curve" typical of complex learning

Why does an S-curve of growth eventually level off?

What people call a plateau may be a period of stability after a skill is learned as well as it can be learned. Most growth processes follow the same S-shaped curve as motor learning. In general, an S-shaped curve of growth levels off because stability is attained, a resource needed for growth is limited, or a ceiling of performance is reached. 

What phases of learning a complex skill cause the "S-curve" pattern?

The S-shaped learning curve is most obvious when someone learns a highly complex task. The initial part of the curve rises slowly as a person becomes familiar with basic components of a skill. The steep ascending phase occurs when there is enough experience with rudiments or simple components to start "putting it all together." Rapid progress follows until the skill "hits a ceiling" or stabilizes at a high level.

To put this in the context of AEC-related skills monitoring, KnowledgeSmart analyzed the results of several thousand test results, covering basic 2D CAD skills (using AutoCAD and MicroStation software).

The results were as follows:

0-1 years' experience:  53% in 87 mins

4-5 year's experience:  60% in 77 mins

8-10 years' experience:  61% in 79 mins

Many times, we hear people say, I don't need skills testing or training, because I've been using (XYZ software) for many years. Could it be that, even in the case of experienced users, a learning plateau might have been reached, and a fresh approach to learning could prove beneficial in helping to further improve performance?

R

KS user group - April 11

It's that time of year again. We have scheduled the date for the next KS user group.

Here are the details:

Location:  Smeaton Room, Buro Happold, 17 Newman Street, London, W1T 1PD

Date:  Thursday 11 April

Start:  09.00

End:  14.00

Cost:  Free (Breakfast & lunch included)

We will be reviewing the latest KS tools in some detail, including a look at what’s coming up in the next 3-6 months. Plus a guest presenter or two.  Full agenda to follow, as the date gets closer. 

Places will be limited to 25, on a first come, first served basis.

Hope to see you there!

R

Testing Revit Architecture Skills - A 3-Step Approach

Revit Architecture is the most popular test title in the KS library. No big surprise. So let's take a look at how an AEC firm might create an appropriate testing environment for Revit Architecture, catering for users of all levels of experience and abilities.

Firstly, it's important to establish that not all users need to become Revit 'experts'. Many people have a primary role which means they come into contact with Revit on an occasional basis, but no more than that. So, do they need to become adept in the finer points of massing, curtain walls and Family editing? Certainly not! For example, PM's, Project Architects, Practice Principals, and so on.  However, in many instances, these people can benefit from a degree of knowledge and familiarity with the technology, not least from a project planning or resourcing perspective.  So some basic knowledge of the tools can be beneficial.

For these individuals, we have created a 'Level 1' assessment, called 'Revit Architecture for occasional users'. As the name suggests, it is a fairly gentle, introductory level test, which looks at some basic concepts, including:  Files & File Formats, Navigation, Views & Sheets, Measuring, Exporting Data, Families and Element Selection.

The fact is, casual users of Revit software sometimes make silly mistakes, mainly through lack of understanding, thus creating work for more experienced users tasked with tidying up their mess. This is a common complaint from design firms and one which is easily avoided.  Simply put, no-one gets access to a license of Revit, without first achieving a minimum 'pass' mark on the Revit occasional test (level to be determined by individual firms). Don't give someone the keys to the car, without first establishing they have a basic understanding of how to drive.

For the primary modeling team, we have the 'Level 2', or 'Revit fundamentals' level material. This is a general level test, covering a wider range of topics, including: Basic Element Creation, Views & Sheets, Detailing, Keynoting & Annotation, Worksharing, Dimensions & Rules, Interoperability, Families & Parts, Scheduling, Coordinates & Orientation and Outputs.

This level of test material can be used to create reliable benchmark data for the firm, compared to industry average statistics. Performance 'quartiles' (see image below) are an effective way of targeting users with incremental productivity improvements over time, with the appropriate modular training workshops addressing highlighted skills gaps.

For users placing in upper quartile 3 or quartile 4, we have a variety of modules which are designed to address more advanced Revit concepts. The 'Level 3' material covers more process based scenarios and looks at the impact of using the software in a project environment. Topics such as Revit Families, Work flows, Project process and Worksharing are covered in greater detail. 

In addition to test modules about Revit software, we have written some general questions about BIM and BIM management. So firms can create assessments which cover technology and also the wider Building Information Modeling environment, within which the software is deployed.

By targeting users with material appropriate for their job function, and also their current level of ability, AEC firms can now create more meaningful benchmark data, plan a more focused training strategy and adopt a stepped approach to measuring Revit and BIM knowledge across their teams.

R

2013 Library Updates

Happy New Year!

We've been busy over the holiday period, putting the finishing touches to a selection of new KS library updates.  These include:

Revit Architecture 2013 - Advanced
Part 1 of 2 sets of challenging questions, looking at more advanced aspects of applying RAC on projects. Written by Paul Aubin.

Revit Architecture 2013 - Occasional Users
An introductory set of questions, written by Chris Senior, with part-time users of the software in mind.

Inventor 2013 - Part 3 (Drawings)
Part 3 of 4 sets, written by John Evans. Part 1 covers Assemblies. Part 2 covers Part Modeling. Part 4 to follow in Feb/Mar covering Welding & Sheet Metal.

BIM Management
A general set of questions, edited by Evolve Consultancy, covering a variety of different aspects of BIM Management and Coordination, including IFC and COBie.

Plant 3D - Part 1 (Project Setup and Specifications)
A general level set of questions, written by Joel Harris, covering project setup, template creation and customization. Parts 2-5 to follow in the coming months, covering the following topics: Piping specs & catalogs, 3D modeling, Document generation and P&ID.

USACE BIM Process
A general set of questions about the official BIM process, as created by the US Army Corps of Engineers. This assessment comprises questions looking at both the Policy and Content of the USACE BIM protocols. Edited by a sub-committee of the USACE BIM team.

We are working on some new titles in the Spring months.  These include:
Inventor Part 4 - Welding & Sheet Metal
Plant 3D Part 2 - Piping Specifications & Catalogs
Revit Structure 2013 Advanced
Revit MEP 2013 Advanced
Civil 3D 2013 Advanced
ArchiCAD
Bentley ProjectWise
Synchro

R

KS server update

Quick technical heads up for this Sunday. We're moving the KS system from the existing Rackspace local server to a cloud based server. Nothing especially earth shattering. But an important update to our back-end hosting, which sets us up well for next year and beyond. The nice thing about cloud based hosting is the scalability. If we need to ramp up our RAM or increase overall capacity in future, then we'll have greater capability to do so.

So for 2-3 hours on Sunday 09 December, around 12pm EST / 5pm GMT, you might experience a little down-time in the live system.  We'll do everything we can to minimise the impact of this work.

R

KS Devs Update

As Q4 rolls around, we thought it would be useful to jot down a brief summary of what we're working on at the moment.

KS Library

All of the existing (Autodesk) library titles have been updated to 2013 format.  Our favourite task of the year. (Not really).  This includes: Revit (x3), AutoCAD, Civil 3D, Navisworks, Design Review and 3ds Max.

We're over half way to finishing an expanded Inventor 2013 question set.  Working with ace US author, John Evans, we are covering 4 key areas: Assemblies, Part Modeling, Drawings (and presentations) and Welding and sheet metal.  Our goal is to complete the set in time for AU2012.

We're compiling a larger set of general BIM related questions, for the KS Community area of the library.  These will be a FREE addition to the KS library.  Should be ready in time for AU, or thereabouts.

We have started work on a set of ArchiCAD fundamentals questions - our first foray into the Graphisoft world.  Covering 4 levels of difficulty, this set will be available in the New Year.

Advanced RAC, advanced RST and Revit for occasional users are all works in progress. Our deadline for these sets has slipped a bit, due to author commitments, but we're aiming for something on all 3 fronts by AU2012.

USACE has put together a working group to check the Attachment F assessment material.  We're awaiting the green light from the Corps, to release this free question set into the general KS library.

On the drawing board (can we still use that phrase?) for the next round of updates, include the following topics: Advanced RMEP, advanced Civil 3D, Bentley ProjectWise, AutoPlant and Plant 3D.

A variety of vendors are currently discussing writing test material for their own software products.  These include: CADS (CADS RC), Tekla (Tekla Structures), Synchro (Synchro Professional), Savoy (AutoTrack) and Codebook (Codebook).


KS System

The latest release of the KS tools hit the live site in early October.  This completed our program of works for the summer, within two weeks of our original goal, which is excellent.

Next up, we have two point-releases in the works, which address the remaining items on the KS user group wish list.  These include: capturing additional user data for more detailed statistical & benchmarking analysis; update to the test report format; additional library searching & grouping; tidy up of the users data page (including option to edit and delete master records); prep for supporting different languages in the test library;  update to account hierarchy options (including option to delete and move accounts); new charting option for flagged training request data, plus a few other smaller tidy-up tasks.

We'll also be moving to a new cloud server next month, which gives us a greater capacity to grow.  We anticipate a small amount of down-time, during the move, so we'll keep you updated.

Looking ahead, we'll be discussing the following additional features with our user group, in the coming months: User dashboard pages, new question types, survey questions & charting, links to third party systems, more detailed regional benchmark stats, additional charting & reporting options and a web based skills matrix.


Out and About

We'll be attending a few events in the next couple of months.  Autodesk host their annual BIM conference, in London, on 19 November.  The same week sees a free BIM learning and networking conference, hosted by Ramboll, in London, on 22 November.

Plus, KS is once again co-hosting the AUGI Top DAUG skills contest at Autodesk University.

So hopefully we'll have a chance to cross paths at some point this quarter.  Once again, thanks to our ever-enthusiastic user group, for a steady stream of ideas for new system features and library additions.

R