Thursday, 19 May 2011

RTC - Revit Top Cat



The seventh Revit Technology Conference takes place next week, on the Gold Coast, Australia. (I know, there could be worse places to spend 3 days learning about Revit & BIM, right?).

In the lead up to the show, KS has been working with the RTC Committee to arrange a fun contest, for users to demonstrate their Revit credentials, flex their modelling muscles - and prove beyond doubt who is the top player in town.

In fact (using a play on words from AUGI's well established 'Top-Daug' AutoCAD quiz format) we're calling this year's contest - Revit Top Cat (RTC, geddit? :) ).


Here's how it will all go down..

On Thursday and Friday, there will be 6 PC's set up with copies of RAC, RST and RMEP 2011.  Users will have 10 minutes (12 mins for MEP) to answer 14 questions about their chosen Revit discipline.

The winners from each stream, on both days, will qualify for Saturday's Grand Final.  (Here's where we really separate the Revit men/women from the boys/girls!).  In short, best score combined with fastest time wins each heat.

On Saturday, we'll have our 6 finalists lined up to take the new Revit Project Process test.  (NB This is the first of our new 'level 3' modules - and it's not for the faint hearted!  In beta testing, we had more than one Revit veteran literally run screaming from the room whilst taking this test.  Well, OK, not really, but it is a challenge!).

So, the Grand Final.  10 questions in 12 minutes - and the new Revit Top Cat will be crowned.

As well as some cash prizes for the finalists - the overall winner will have the kudos of being recognised as the most renowned Reviteer on the planet! (Or at least the Gold Coast!).

So, do you have what it takes to be the RTC AU - Revit Top Cat??

See you at the show..

R

Tuesday, 10 May 2011

Nightingale & Evolve case study

When I speak to firms about measuring and improving CAD & BIM performance, there are a number of factors which come into play.  The fact is that it takes a lot of hard work, management focus, commitment, the right tools, strong technical leadership and reliable consulting advice - to bring it all together.

One such example is the work being undertaken by the team at Evolve (www.evolve-consultancy.com) - a leading UK BIM consultancy - and healthcare specialists Nightingale Associates, under the technical leadership of Chris Seymour-Smith.

Take a look at the full story here:  http://www.evolve-consultancy.com/article.php?incat_id=161&category_id=38.

Over a two year period, the firm improved overall accuracy by 21%, with nearly half of their Architects posting above average or excellent scores.  Further, over 40% of their users raised their game individually, by 25% or greater.  That's a remarkable achievement!

R

Thursday, 5 May 2011

KS sponsors RTC AU

KnowledgeSmart has signed as a sponsor of the seventh Revit Technology Conference, which takes place later this month, on Australia's Gold Coast.

This will be our second trip down under, to support this event.  It's a great conference, covering all things Revit and BIM, run independently of, but with full endorsement from, Autodesk.

The conference has been growing steadily in size and stature, over the past six years, to the point that this year also sees the inaugural RTC USA later in the summer.

Many of the industry's top Revit trainers and authors make the trip, to present classes and workshops, including, Paul Aubin, Robert Manna, David Harrington, Chris Price, Simon Whitbread and Phil Read.

KnowledgeSmart will be showing the latest web based skills assessment tools, for Revit and a range of other CAD, BIM and Engineering software titles.

Full details can be found here: http://www.revitconference.com.au.

R

Thursday, 28 April 2011

KS Admin Help Notes

Here is a short guide for KS administrators, which shares some tips from other KS customers, on how to prepare for a skills assessment rollout.

Communicate your plans

It is important to explain to users precisely why you are asking them to undertake a skills assessment.  You might do this by including the assessment in a wider discussion about training needs and more efficient use of the overall CAD & BIM infrastructure within your company.  This could include training needs, recruitment, induction and annual appraisals processes.

Improving technical software skills is a continuously improving process – the KnowledgeSmart tools simply highlight a likely starting point.  As with most things, good communication goes a long way.  An effective strategy is to hold a team meeting and explain your overall plans for improving CAD & BIM performance within the firm.  This can include a Q & A session, where people have an opportunity to clarify any aspects of the assessment process – and understand how it fits into the bigger learning picture.

It is important that people don’t feel the results of their skills evaluation may be used in a negative context, particularly if they don’t perform well first time around.  Ensure they understand this is a designed to be a learning process – and everyone has to start somewhere.  If you have a CAD or BIM software upgrade scheduled, this can also feed into the process.  By taking a snapshot of each individual’s current skills at a basic level, you can feed this data into your training plans moving forwards.

Evidence shows that better overall understanding of basic commands can also benefit the implementation of in-house Standards. Once you have explained to your teams the business case for measuring and improving CAD & BIM productivity, you need to set a timeframe for rolling out your assessments.

Prepare a timetable

You need to decide over what period of time you want to complete your assessments.  This will depend on 4 variables; 1) the number of people to be assessed 2) the number of PC’s you can make available (with a licensed copy of CAD or BIM software and a network connection) 3) the number of offices in your organisation and 4) general day-to-day work pressures.

You need to decide what your preferred assessment environment will be.  Some companies prefer their teams to use their own work stations, whilst others favour a more formal setting.  In truth, there is no right or wrong answer to this question; you want people to feel comfortable and remove any potential for apprehension or pre-assessment nerves.

Experience shows that a formal assessment environment makes for a smoother rollout, where people simply take an hour or so at a specified time to sit their assessment.  This way, they aren’t distracted by project work, email, ringing phones or interruptions from colleagues.

Use the invite tools in the KS dashboard to manage your assessment programme.  For multi-office firms, it is best to start with one office and, when this office is done, gradually ramp up the level of activity until the assessment programme is complete.  Principal admins will need the support of regional project technology co-ordinators if this is to be a success.

Prepare your team

We recommend you brief your teams prior to their assessments.  You might like to share with them a simple Q & A document or present your plans at a team meeting.  Here's an example of what other KS customers have done: http://the-knowledgesmart-blog.blogspot.com/2011/04/sample-ks-candidate-q.html.

Here's a brief user guide to taking a KS assessment: http://the-knowledgesmart-blog.blogspot.com/2011/04/taking-ks-assessment.html.

It is best to circulate the information at least a few days before people are due to take their assessments.

Send your invites

Use the invite tools in the KS dashboard to send your test invites.  You can send invites individually, or add a list of users via csv import and invite them as a group.

Here's a brief blog article about sending invites: http://the-knowledgesmart-blog.blogspot.com/2011/04/inviting-users-to-take-ks-test.html.

Next, you may customise your welcome message in the invite mail and on the test landing page.  You might wish to present users with specific instructions on where to locate their sample test files (if you have chosen to store them on your network, for example).

Here's a brief blog about editing your invite text: http://the-knowledgesmart-blog.blogspot.com/2011/03/customising-your-test-invite-text.html.

Complete your assessments

You can monitor your team's progress, as they complete their assessments.  Admins receive an email with a link to test results, as and when they are completed.

You can use the invite history page of your dashboard to track invites and completed assessments.  You can also re-send invites from here.

After the assessments

At the end of their assessment, users may receive an instant result and report with a breakdown of their score, training keywords and coaching notes.  In addition, they can be sent an email containing a link to their summary report.

Account admins can access all assessment results, feedback, training keywords and coaching notes – for each candidate – in the KS dashboard.

Via the dashboard, it is possible to search results, view performance charts and export data into Excel for in-house reports.  It is important to provide the opportunity for individuals to access a structured training programme, after their skills have been evaluated.  The results will highlight areas where future training workshops can be focused for maximum improvement.

Here's an 8-step guide to better training needs analysis: http://the-knowledgesmart-blog.blogspot.com/2011/04/using-ks-for-training-needs-analysis.html.

R

Wednesday, 27 April 2011

Sample KS Candidate Q & A

It is important for AEC firms to explain to users precisely why they are being asked to take a skills assessment.  The key message is that the KS tools are an aid to identifying skills gaps and highlighting training needs.

Here are some sample notes that KS admins can copy - and adapt for their own use - to help give their teams a bit more background to the KS assessment process.

What is KnowledgeSmart?

KnowledgeSmart is a leading developer of skills assessment tools for CAD, BIM and engineering software applications – designed to help AEC firms measure and improve productivity.  Since 2003, KnowledgeSmart has facilitated in excess of 20,000 individual assessments – across a range of architectural and engineering practices globally – helping to create a benchmark for basic CAD & BIM proficiency.

The KnowledgeSmart tools perform a comprehensive assessment of a user’s ability focusing on fundamental skills groups associated with design and engineering software applications, including AutoCAD, Revit, 3ds Max, Civil 3D, Rhino, SketchUp, InDesign, Photoshop, MicroStation and Bentley Architecture. The process aims to assess current performance in order to identify areas where individuals can benefit from additional training.

What is the point of KnowledgeSmart - and why should I do it?


This process is not about a big-brother style test to check up on unsuspecting staff!  Nor is it a matter of pointing out who is the more knowledgeable software user.  Rather, the point of the exercise is to highlight how much each individual currently knows – in order to identify those areas where focused training can improve basic knowledge.

Everyone uses CAD, BIM and engineering software in a different way.  Some people use it full time in their role; others may only use the tools from time to time.  The objective is not to teach a veteran CAD user how to use a polyline or write text.  But the fact remains, a more detailed knowledge of basic commands and operations can alleviate apprehension in using technical software applications and assist in speeding up routine tasks.


Is it the same for full-time and casual users?

There are a number of different test modules to choose from, covering a range of skill levels and created by a team of specialist authors.  Firms can also write their own test material in-house.  For basic CAD skills, as an example, practices can choose from a 2D fundamentals module or an ‘occasional users' module.

The fact is, there are many differing levels of ability and many different requirements placed upon the way CAD & BIM software is used within each practice.

Summary reports take into consideration whether an individual is a core user, or someone who only uses the software occasionally as part of a wider role.  Follow-up comparisons and training recommendations should be reflected accordingly.


What topics does the assessment cover?

KnowledgeSmart assessments typically break down technical software applications into a series of modular topics.

For example, the 2D fundamentals assessments (for AutoCAD & MicroStation) cover a range of basic CAD commands, including; basic object/element creation, layers/levels, blocks/cells, annotation, referencing & printing, UCS/ACS and preferences.

Assessments typically comprise 20-25 questions and present scores as a percentage, based on the accuracy of the answers presented.  There are 2 question types and 5 question styles.  Question types are either task-based (i.e. you need to perform a task using the software in order to work out the answer) or knowledge-based (i.e. you are being tested on your understanding of certain software functions, without having to actually use the tools).  Question styles include; free text, multiple choice, pick list, order list and true/false.

What if I do not score very well?

This exercise is not about who is the best.  You will identify areas where you can improve your own performance with the help of more targeted training workshops.

Following the assessment everyone receives an overall result and summary report with feedback on dropped marks, training keywords and coaching material.

When and where will the assessments take place?

Your technology administrator will have details of your assessment schedule.  If you are unavailable to attend on the date or time scheduled please contact your administrator to rearrange.  You will need a PC or laptop, with a web connection and a copy of the software you are being tested on.

How long will the assessment take?

There is no limit to how long the exercise will take.  The average time is approximately an hour, but this is not a strict rule.

Try to work at your usual speed.  Don’t rush, as many simple mistakes are made by people not reading the instructions carefully.  Conversely, don’t take too long to complete each task, as your score plus time taken will be looked at in the overall results analysis.  Your score will not be affected by the time you take.

How long before I know my results?

Depending on the level of feedback assigned by your administrator, you will typically receive your score straight away when you finish the assessment.  You will also be emailed with a link to your results report and feedback on any marks which have been dropped.

Additional Info

The amount of time you take will not affect the accuracy of your score.

Before you start, you may be presented with a zip folder containing files which you’ll need to refer to during your test.  Save the zip folder locally on your machine – and remember where you put it – as you will need to navigate back to this folder during your assessment.

Please read the written instructions carefully.  This is also an assessment of your ability to follow instructions. Often, marks are dropped through candidates simply not doing what they have been asked to do.

You can answer the questions in any order, using the KnowledgeSmart 'question navigator', or simply read through the test instructions before attempting to answer each question.  You can also edit your answers at any time during your assessment.

The time taken to complete an assessment varies depending on the individual, but an average is about an hour.  You don’t have to complete all the questions in one sitting; you can exit and return later if you prefer.  When you resume, your previous answers will have been saved.

Here’s a link to a KS blog article which tells you more about what to expect during a KS assessment: http://the-knowledgesmart-blog.blogspot.com/2011/04/taking-ks-assessment.html.


R

Revit - 3 levels of Proficiency



Based on a number of similar discussions in recent months, we're introducing a subtle change to the labeling of some of the KS assessment modules.

I attended a web meeting this week with the Head of Model Development for a major UK construction client.  Like many AEC firms, they are wrestling with the challenge of making BIM work for their business, whilst finding the cost savings, efficiency gains, and so on, that all big businesses have to make these days.

Looking back to the KS user group in London, a couple of months ago, a similar topic arose as part of the general discussion.  That is, how do we identify someone who has basic skills in, say, Revit and distinguish them from an advanced user?

Further, do AEC firms actually need all of their users to be 'specialists' in Revit?  The simple answer is no, in my opinion. In many instances, users just need to receive a file, open it up and perform some straightforward analysis of the model (without breaking anything!).  For example, construction managers, PM's and the like.  Then, for the general modeling team, AEC firms require a broader level of understanding for their wider user base.  The people creating the detailed production information on a project.  And last, firms need to have a handful of Revit 'specialists'.  People with a deeper understanding of the way in which Revit is deployed on a project, an appreciation of the collaborative process and the impact of using the technology across a broader environment.  And maybe, if content creation is a key part of their business, a handful of Revit Family specialists, who understand the importance of creating quality content, that can be re-used on subsequent projects, not simply re-modelled from scratch every time.

So it makes sense to establish 3 recognised levels of competence;

Level 1 – basic skills
Level 2 – intermediate skills
Level 3 – advanced skills

With respect to Revit, this will work as follows;

L1 – Basics of Revit (i.e. a test aimed at Construction Managers, PM’s, etc.)
L2 – Revit fundamentals
L3 – More advanced concepts of Revit, i.e. Revit project process, collaborative working, Family creation, etc.

AEC firms can use the assessments, as a means of demonstrating to clients (and prospective clients) that they have the skills required, to truly deliver on a BIM project.  For example, all CM’s can be trained and encouraged to take the Revit L1 assessment (scoring X% or higher in Y mins or less).  All Revit modelers can add the Revit L2 assessment to their CV (scoring X% or higher in Y mins or less).  All Revit ‘specialists’ need to ace the L3 assessment(s).  Anyone working on content creation, must tackle the families assessment.  You get the picture.

Firms might also consider writing their own modules, based on their BIM Standards.  All users can demonstrate an understanding of the correct standards, by passing this module.

Now, we've made extremely good progress in building our Revit assessment library.  We already have the following modules in place:

-  RAC 2010 fundamentals (Metric & Imperial) – L2
-  RAC 2011 fundamentals (M & I) – L2
-  RAC Xpress (M & I) – L2
-  RST 2010 fundamentals (M & I) – L2
-  RST 2011 fundamentals (I) – L2
-  RST Xpress (M & I) – L2
-  RMEP 2011 fundamentals (Mechanical) (I) – L2
-  RMEP 2011 fundamentals (Plumbing) (I) – L2

We are currently working on the following new modules:

-  Revit Project Process (goes live early May) – L3
-  Revit Families (goes live late May) – L3
-  RMEP 2011 fundamentals (Electrical) (I) (c.June) – L2
-  RAC 2012 fundamentals – 13 modules (M) (c.June) – L2
-  RMEP fundamentals 2012 (M, E & P) (I) (c.July) – L2
-  RST 2012 fundamentals – 13 modules (M) (c.August) – L2
-  RMEP 2012 fundamentals – 13 modules (M) (c.September) – L2
-  Revit for occasional users (i.e. general content for PM’s, CM’s.  Not discipline specific). (M&I) (c.September) – L1

In the Fall, we'll be adding these modules:

-  RAC 2012 fundamentals – 6 additional modules (M) – L2
-  RST 2012 fundamentals – 6 additional modules (M) – L2
-  RMEP 2012 fundamentals – 6 additional modules (M) – L2
-  Imperial data sets for all Revit 2012 content – L2

-  RAC 2012 advanced (M & I) (i.e. harder test than fundamentals (L2) for core users) – L3
-  RST 2012 advanced (M & I) (i.e. harder test for core users) – L3
-  RMEP 2012 advanced (M & I) (i.e. harder test for core users) – L3

I think this will help AEC firms, Construction clients and contractors alike, to get a much better handle on who knows what, with respect to deploying Revit on a BIM project.  It will also help firms to demonstrate their BIM credentials and give users a logical (and more appropriate) path for skills development.

R

Wednesday, 20 April 2011

Taking a KS Assessment

It's extremely important for AEC firms to explain to their users precisely why they are being asked to take a technical software skills assessment.  The best firms always handle this well.  As with all things in life, a little bit of good communication goes a long way.

This is not all about who is the smartest, fastest, or best Revit user.  Or the most experienced MicroStation wizard.  Although these are useful things to know.  The real value in the KS tools is trying to help users, at an individual level, identify what they know and also what they need to brush up on next, in order to improve their skills, with a particular piece of software.

Over the past 8 years, we have facilitated over 20,000 individual test sessions, across hundreds of AEC firms.  Experience has shown that, in general, people don't like the idea of taking a test - at least that's usually how they feel at first.  However, experience has also shown that, for most people, when they complete their assessment, they realise that it wasn't half as bad as they thought it would be! Further, they often learn new skills as they go through the process of answering the test questions.  And lastly, they find the test feedback and coaching notes extremely helpful, in giving them an unbiased appraisal of their skills using a particular software application - relative to their peers in industry.  It's important that users have the opportunity to review their test scores and feedback with their team leader or BIM manager.  And it's also important that firms explain to their users that all test scores are confidential.  Best practice would recommend that only senior administrators should have access to firm-wide results data.

Provided firms explain to their teams that the main point of rolling out an assessment program, is to gather valuable training needs analysis data, rather than a big-brother style exercise in peeking over shoulders, then most users feel OK about the idea.  And when firms follow through with more focused, modular training workshops, which target recognised skills gaps, then it's a win-win for users and firms alike.

So let's take a look at the user journey, when asked to complete a KnowledgeSmart skills assessment.

We'll assume the firm has already done the pre-assessment prep - and explained to their teams what is happening and why they are being asked to take an assessment.

The next step is for a user to receive their test invite.  (Here is a separate article on how a KS admin sends out test invites: http://the-knowledgesmart-blog.blogspot.com/2011/04/inviting-users-to-take-ks-test.html).

Admins can set up test sessions straight from a browser, but most invites are sent by email.  Either way, the user ends up in the same place.  Here is a typical KS test invite:


The system assigns a username and password to each user.  They access their assessment by clicking the unique URL at the bottom of their invite mail.  This takes them to the assessment login page, where they will be prompted to enter their username and password.  (If a user loses or forgets their password, they can hit the 'Forgot details?' link to receive a new one).


When they log in, they arrive at the main assessment landing page.  Here they will see a summary of the test they are scheduled to take and any special instructions from their administrator.


There are 5 types of questions, which typically comprise a KS assessment.  Let's take a brief look at each one..

Free Text 
This type of question requires a specific answer, which needs to match the answer stored in the KS database.  Look out for precise instructions on how to format your answer, for example, number of decimal places, case sensitive answers, accurate spelling, and so on.

Multiple Choice

Here, you will be offered a selection of possible answers - and you need to select the best one.  Watch out for 'distractor' entries, which are designed to make you think before selecting your answer.


Pick List
This question type offers you a range of options, which could represent a correct answer.  Choose the best answers from the list provided, using the check boxes, to the left of the answers.  Partial marks are often awarded for entering some of the correct answers on pick list questions.


Order List
This question type requires you to place a number of items in the correct sequence.  Drag and drop the answer options, until they are in the right order.


True or False
This question type is pretty straightforward!  Read the statement or question and decide if you think it is true or false.  Enter your answer using the radio buttons.


OK, so when the user arrives at the landing page, before they begin, they need to download the sample data files that they will be using during their assessment.  There are a couple of ways they might have to do this.  The usual way is to select the orange, 'Download files' button.


This prompts the user to save the files locally on their work station.  They will need to navigate back to this folder during their assessment, so most users choose their desktop, or My Documents, etc.  The data sets do not need to be saved during a test session, so they can simply be deleted after the assessment has been completed.


Sometimes firms choose to zip up all sample test files and host them locally, on their own servers.  If this is the case, then the orange button will not display.  Instead, the admin will provide special instructions on the assessment landing page, explaining which file path to follow, in order to find the sample data sets.

Either method is fine - and the user will achieve the same result!  They simply need to know which folder to navigate to, in order to access the files for their randomly generated set of test questions.

When they have their sample test files and they have read any other instructions from their administrator, they can hit the green 'Start' button and their assessment will begin.


Each assessment is timed, although the time taken to finish does not impact the overall score. The test user interface (UI) is easy to navigate.  All test sessions start on question 1.  There is also a small timer, which allows users to keep track of how long they are taking during their session.


Each question has a name and brief summary, which explains to users the main features of the software being addressed.


If a question is task-based, i.e. the user has to use the software in order to answer the question, then the corresponding sample data set can be found in the zip folder they downloaded at the start.  The orange instruction box explains where to find the files for each question.


Users can answer the test questions in any order.  The 'Question Navigator' allows users to move around the questions and generally familiarise themselves with the exercises.  The simple colour code helps users to keep track of their progress.  Green squares indicate questions for which an answer has been submitted.  Orange represents the current question.  Red squares are questions that have yet to be attempted.


If a user makes a mistake, they can re-visit the green squares at any time, to edit their answers.  A test session finishes when all the questions have been answered.  Users have one last opportunity to review and edit their answers, before hitting the green 'Finish' button, which triggers their assessment to be graded.


(Sometimes, an admin will set a time limit for an assessment, in which case the session will automatically close after that time has elapsed.  Answers submitted will be marked and any questions which have not been attempted will be scored as zero).

When the session is finished, the user may receive a test report, with feedback on any dropped marks.  The level of feedback received can vary, depending on the settings prescribed by the account admin.


A helpful summary of the test questions, feedback on incorrect answers, coaching notes and training tags may be viewed by the user, by selecting the 'Show question detail' link at the top of their summary report.


The training tags for questions where score = < 100%,  form a personal curriculum of suggested training workshop topics.

R