Top DAUG 2014 - The Results

This year, KnowledgeSmart and AUGI again teamed up to provide an interactive skills assessment, across 10 popular Autodesk software tracks: 3ds Max, AutoCAD, AutoCAD Civil 3D, AutoCAD Plant 3D, Inventor, Navisworks, Revit Architecture, Revit MEP, Revit Structure and Vault.

Once again, we had some great prizes up for grabs.  Track winners walked away with a $100 USD Amazon voucher (a combination of high scores and fastest times determined the winner of each track), a glass trophy and the overall winner won a HP laptop and free pass to AU2015.

We spent several hours on Tuesday, setting up 16 networked PC's in the exhibition hall, at the AUGI stand.

 Next, 144 copies of Autodesk software needed checking, then all the KS sample data sets had to be uploaded to each machine and checked again.  Big thanks to Tony Brown for his assistance in accomplishing this task.

Here's how we looked when everything was finished:

The main competition ran over 3 days (1 x 3 hour slot on day one, 2 x 3 hour slots on day two and a final 1 hour slot on day three). Contestants had to answer 8-10 questions, using the 2015 version of each software title. Each session was limited to just 12 minutes, so people had to work fast! Special thanks to awesome AUGI team members, Kristin, Bob, Michael, Donnia and Richard, for briefing users and making sure everything ran smoothly.

Here are the Top DAUG's in action:

By the end of the contest, we posted 215 results.

Throughout the competition, we displayed a rolling list of the top 10 contestants for each track, on the AUGI big screen.

The Results

Congratulations to the following contestants, who won their respective tracks:

And a special mention to the overall winner of AUGI Top DAUG 2014:

Tracy Chadwick

Analysis

So, let's take a detailed look at the results of this year's Top DAUG competition.

Overall

No. of Tests Completed:  215

Overall Average:  59% in 9 mins 14 secs

(NB the average score for 2013 was 48%).

Track 1 - 3ds Max

Track Winner: Herb Wagner

Winning Score: 55% in 11 mins 35 secs

Top 2 Contestants:

No. Completed: 2

Group Average: 35% in 11 mins 45 secs

Track 2 - AutoCAD

Track Winner: Tracy Chadwick

Winning Score: 100% in 3 mins 50 secs

Top 10 Contestants:

No. Completed: 68

Group Average: 63% in 8 mins 2 secs

Track 3 - AutoCAD Civil 3D

Track Winner: Christopher Fugitt

Winning Score: 88% in 5 mins 55 secs

Top 10 Contestants:

No. Completed: 21

Group Average: 66% in 9 mins 12 secs

Track 4 - AutoCAD Plant 3D

Track Winner: Patrick Alonzo

Winning Score: 72% in 11 mins 30 secs

Top 4 Contestants:

No. Completed: 4

Group Average: 52% in 10 mins 38 secs

Track 5 - Inventor

Track Winner: JD Mather

Winning Score: 81% in 11 mins 25 secs

Top 10 Contestants:

No. Completed: 21

Group Average: 41% in 11 mins 9 secs

Track 6 - Navisworks

Track Winner: Michael Doty

Winning Score: 60% in 6 mins 0 secs

Top 10 Contestants:

No. Completed: 13

Group Average: 49% in 9 mins 45 secs

Track 7 - Revit Architecture

Track Winner: Erik Eriksson

Winning Score: 99% in 10 mins 40 secs

Top 10 Contestants:

No. Completed: 47

Group Average: 57% in 10 mins 51 secs

Track 8 - Revit MEP

Track Winner: Ken Fields

Winning Score: 100% in 9 mins 15 secs

Top 10 Contestants:

No. Completed: 24

Group Average: 64% in 7 mins 32 secs

Track 9 - Revit Structure

Track Winner: Rebecca Frangipane

Winning Score: 85% in 8 mins 30 secs

Top 9 Contestants:

No. Completed: 9

Group Average: 70% in 9 mins 51 secs

Track 10 - Vault

Track Winner: Bryson Anderson

Winning Score: 60% in 6 mins 50 secs

Top 6 Contestants:

No. Completed: 6

Group Average: 53% in 6 mins 17 secs

Most Popular Tracks

The most popular tracks, in order of completed tests, were as follows:

AutoCAD - 68 results

Revit Architecture - 47 results

Revit MEP - 24 results

Inventor - 21 results

AutoCAD Civil 3D - 21 results

Navisworks - 13 results

Revit Structure - 9 results

Vault - 6 results

AutoCAD Plant 3D - 4 results

3ds Max - 2 results

Total = 215 results

Honourable Mentions

100% Club

The following people achieved a maximum score of 100% in their track:

Tracy Chadwick - AutoCAD

Glen Sullivan - AutoCAD

Christine Noble - AutoCAD

Ken Fields - Revit MEP

(Plus Erik Eriksson scored a tantalisingly close 99% in the Revit Architecture track!)

Top Performers

The following people demonstrated impressive versatility by achieving above average scores in multiple tracks:

Chris Brown - AutoCAD, Inventor and Navisworks

Tracy Chadwick - AutoCAD, Revit Architecture, Inventor and Vault

Seth Cohen - AutoCAD and Civil 3D

Jason Dupree - AutoCAD and Inventor

Ken Fields - AutoCAD and Revit MEP

John Fout - Navisworks, Revit Architecture and Revit Structure

Rebecca Frangipane - Navisworks, Revit Architecture and Revit Structure

Brent McAnney - AutoCAD, Civil 3D and Vault

Kate Morrical - AutoCAD and Revit Structure

Mohaimie Mosmin - AutoCAD and Inventor

David Rushforth - Navisworks, Revit Architecture and Revit MEP

Maxime Sanschagrin - Revit Architecture, Revit MEP and Revit Structure

Jim Swain - AutoCAD and Inventor

Nick Tanner - AutoCAD, Navisworks and Revit Architecture

Boys vs Girls

Finally, let's take a look at the demographic breakdown of the competition. Out of 215 contestants, 185 were male and 30 female. The average overall performance for each group breaks down like this:

Girls: 64% in 9 mins 25 secs

Boys: 58% in 9 mins 11 secs

So, that's Top DAUG finished for another year. A thoroughly enjoyable 3 days at AU2014. 215 completed tests, across 10 popular tracks. Once again, the overall standard was extremely high, with some outstanding individual performances from our track winners and top 10 contestants.

Congratulations to all our winners. Thanks to the AUGI team for their support. Lastly, a big thank you to everyone who took part in this year's contest.

See you all in 2015 at the Venetian for more fun & games!

R

AUGI Top DAUG 2014

KnowledgeSmart is partnering with the team at AUGI once again this year, for the annual Top DAUG skills contest at AU2014.

This year's contest is shaping up well, with 10 tracks planned for AU delegates.

Here's what we hope to include, using 2015 versions of the software:

3ds Max
AutoCAD 2D
AutoCAD Civil 3D
AutoCAD Plant 3D
Inventor
Navisworks
Revit Architecture
Revit MEP
Revit Structure

Vault

As in year's past, contestants will have just 10 minutes to answer a selection of knowledge-based and practical tasks, covering the main features of each application.

The person who posts the highest score, combined with the fastest time, in each track, will take the prize for their chosen topic. Then we analyse the data from all streams, to crown the overall winner of the Top DAUG contest.

Hope to see you at the Manadalay Bay!  #therecanbeonlyone

R

KS Devs Update

Here is a brief update of what's being discussed at our next tech team workshop. With thanks to our customer feedback forums for the majority of these suggestions.

Our current dev is progressing well. More detailed notes to follow shortly for each, but our top 2 current wishlist items are:

User dashboard pages
Survey questions

One those devs hit the live site, we'll take a look at the following:

CSV Export Consistency

Review CSV export across the KS dashboard to ensure it is behaving in a consistent manner.

Account Name in CSV Export

Many companies have multiple child accounts but would like to perform one csv export of all users.  However there is no clear identifier in the csv export as to which account each user belongs.  Include ‘Account Name’ in CSV exports.

Improved data selection in CSV Exports

Consider implementing ‘Field Chooser’ functionality for CSV exports.  When exporting a csv, the Admin User can select which fields to appear and save this selection for future exports.  We would also look to add further options such as ‘Full Name’ (concatenating first name and surname), ‘Training Tags – All’ and ‘Training Tags – Individual’ etc.  By making this functionality available in KS we can dramatically reduce the amount of time Admin Users need to spend outside of KS manipulating the data.

Allow admins to set up regular auto-reports (i.e. weekly/monthly/quarterly) which summarise key activities. For example:  Invites sent: Not started, In progress, Pending, Completed : Results, Scores within a certain range: User type: Test name(s): Date range: Select data for a particular account or accounts. Choose recipients. Full list of selection criteria to be workshopped.

Search /Grouping Criteria – Date Range

Extend search and group criteria across the system to allow the inputting of a date range.  This provides Admin Users with more powerful options when reviewing invites, results etc. Ensure consistency across all dynamic grouping filters. (Invite History/Results/Users).

Add Test Not Completed to Status Dropdown

Currently test invites do not have a single ‘Not Completed’ status.  If an Admin User wishes to identify all outstanding tests they would need to join together ‘Not Started’ and ‘In progress’. Amend grouping options to bring together these two into a single ‘Not Completed’.

Auto-Email when extending a test

When an Admin User extends the expiry date of a test invite, admins would like KS to send an email to confirm along with the new expiry date.  This could be managed in much the same way as the standard invites are with Admin Users having the option of amending the default text. Include option to add additional email addresses.

Increase Character Limit on Multiple Choice and Matching List Answer Options

Multiple choice questions currently have a limit of 50 characters per answer.  KS to review this to identify why the limit is in place (display purposes, database restriction etc) and if it can be extended. Also look at Matching List fields.

Default for Admins of Multiple Accounts

Where an Admin is managing multiple accounts, KS currently defaults to the first account in the list (alphabetically).  Admins have requested a change in this behaviour.  KS to consider alternative default selection options.

Add column in Invites – Date of last reminder/extension

When viewing invites KS currently displays when the invite was first sent and the date and time of the Users last login.  Admins would like an additional column which advises when the User was last reminded (either by reminder or date extension).

Remove Invites from system when a User is deleted

When a User is marked as Ex-Employee there is a clear requirement to retain their interaction with KS however when a User is deleted from the system, admins would like to have any outstanding invites (Not Completed) deleted from the system.


Additional Points for Discussion

More detailed benchmarking & analysis of user demographic & results data.

New API for LMS linking (esp. Olive/Absorb/Pinnacle).

Add number of results to Results > Data page.

Add date stamp to Global Comparisons view and export.

Add user count to training requirements chart.

Review display of Global question benchmark data.

Add question ID to training request csv export.

Improve page speed: Users > Users and Invites > History.

Investigate single sign-on and Active Directory links for user data import.

Investigate Content Delivery Network - adding zip files for test start to CDN environment.


R

New KS Question Types

We have added some new test question types to the KS system this month.  Here's a brief summary of the new choices.

Matching list

This question type asks the user to correctly identify, or match the relationship between 2 lists of data.  Here's an example:

Here's a look at how to build up the answer options, for Matching list questions:

Complete the blank

This question type asks the user to fill in one (or more) blank fields, or select from a list of available options in a sentence or paragraph.  Here's an example:

Here's a look at how to build up the answer options, for Complete the blank questions:

Matrix

This question type asks the user to review a word or short phrase and select a radio button option that relates to the ‘correct’ description.

Here's a look at how to build up the answer options, for Matrix questions:

Essay

This question type asks the user to write a longer answer to a question. This will not be automatically marked and will require manual scoring.

To assign marks to an Essay question, go to Invites > History and click on the 'Pending' link.

Click the 'Edit Score' link and assign your marks. You can also add comments, which will appear in the test report. Save your changes and the final score will appear on the Results > Data page.

The 'Create Question' page now lists all 9 KS test question types:

You'll also see  a brief description for each question type on the test landing page:

R

KS Tips for Getting Started with Your Assessments

Making a positive start with your KS assessment program is a key part of generating momentum in your wider learning and development plans.  Here is a brief guide which walks you through the right steps to make sure you get off to a flying start!

1. Discuss your goals. What do you want to achieve?  How will you define success?  Do you have management buy-in to support your plan?

2. Select your KS system administrators.  Decide who will have global access and who will have regional access.  Do you want to include sub-admins?

3. Book your KS Getting Started web meeting with the KS team and/or training partner.  Review the information on the Settings > Resources page of your KS dashboard.

4. Set up your KS accounts.  Do you need one main account or multiple linked accounts?  Add your company branding to your dashboard, test UI & reports.

5. Agree your message and decide how to communicate your plans to your team.

6. Identify a list of users to be tested.  Do you have any positive-minded ‘super users’ or champions who can be relied on to assist your plans?

7. What topics will you include in your test program?  Will you use the KS library ‘off the shelf’, or will you make some changes?  Do you want to add any custom questions of your own?

8. Build your user list.  Filter your user list into groups.

9. How much user feedback do you want to provide in your reports?  Include coaching?  Include links to learning?  Include additional user data capture?

10. Create a schedule of invites.  Who, where, when, what?  Do you want to send a custom invite message?  Will you test new hires at interview?  Who will help to administer this?

11. Send your test invites.  Monitor progress on the Invites > History page.  Set up reminders.

12. Monitor overall test progress and results.

13. Schedule results analysis workshop/web meeting with the KS team and chosen training partner.

14. Create reports and feed KS results data into your wider training and learning program. Include your preferred training partner in this phase of activity.

15. Re-test after training to monitor effectiveness of your material and measure performance improvement.


R

KS Admin FAQ's

Here are the most common questions and points of discussion for KS admins.

Q - What's the difference between a Sub-admin and a Full-admin?

A - Sub-admins can set up test sessions from a browser, using the 'Assessment' link, but not access the main KS dashboard.  Full-admins can log in to the KS dashboard and use the browser test set-up tools.

Q - How do I re-set or change my KS password?

A - All users can re-set their KS password at any time, by clicking the 'Reset password' link on the KS login page:

This takes you to the following page: https://online.knowledgesmart.net/PassRecover.aspx.  Enter your KS username and the system will email a new password.

KS admins can set their own password, by logging in to their KS dashboard and selecting the 'Change password' link.

Copy the original password into the first field, then confirm your own password and save the changes.

Q - How do I add a new KS administrator to my account?

A - Go to the Accounts > Your Accounts page of the KS dashboard and click on the 'Administrators' tool (small people icon).

From here you can add new Sub-admins or Full-admins, using the 'Add administrator' fields:

Q - I get an error message when I try to import my user list.  Why is this?

A - First, make sure that you are using a .csv file format, not .xls or .xlsx.  (See help notes on the Users > Upload User Data page for a link to a formatted .csv template).  Also, make sure that you have 10 data columns in your .csv file.


Q - My invite mails are not getting through to users every time.  Why not?

A - Sometimes KS invite mails are blocked by spam prevention software.  If the KS mails are not in junk mail, then your corporate security might not be allowing the mails through.  This happens from time to time.  A combination of the following steps usually resolves the issue:

‘White flag’ the domain @knowledgesmart.net.
‘White flag’ the domain @ks-server.net.
‘White flag’ mails from system@ks-server.net.

That should allow the mails to work OK.  However, if they are still getting blocked, there is one more thing to try.  Our hosting company (1&1) sometimes uses a relay server in Germany, called kundenserver.de.

‘White flag’ the domain @kundenserver.de.

That should sort everything out, from a corporate mails point of view.


Q - Some invite mails contain a username and password.  Others do not.  Why is this?

A - The KS system appends a username and password on invite mails for regular users (i.e. non-admins), but not for system administrators. KS admins can manage their own passwords, so the KS system does not include admin usernames and passwords on invite mails.


Q - When  a user tries to log into a test session, their password does not work.  Why is this?

A - If a user receives two or more test invites in close succession, the latter invite(s) will overwrite the password(s) from earlier mails.  Instruct them to use the password from the last email invite received to access ALL open KS test sessions.

All users can re-set their KS password at any time, by clicking the 'Reset password'' link on the KS dashboard login page, which takes you here: https://online.knowledgesmart.net/PassRecover.aspx. Enter your KS username and the system will email a new password.

Q - If  a user logs out of a test session, will all answers be saved?

A - Yes.  All your answers will be saved if you resume a test session, at a later date.


Q - If a user logs out of a test session, does the clock stop?

A - Yes.  The clock stops recording elapsed time when you log out of a test session and starts again when you log back in.


Q - A test result did not upload correctly at the end of a test session.  Can it be rescued?

A - Yes.  If a user loses their web connection at the end of a test session, occasionally a result will not upload to the dashboard successfully.  Just notify us and the KS team will manually retrieve the score and add it to your dashboard.  Submitted answers will not be lost.


Q - Can I mix & match questions from different test modules?

A - Yes. KS admins have full editorial control over all KS test material, including authoring tools for creating new questions from scratch and sorting and moving existing questions between modules and tests.


Q - Can I extend the expiry date on an invite?

A - Yes. Go to Invites > History, select your invite(s) and use the 'Extend invite' tool to change the expiry date of your invites.

Q - Can I set a time limit on a KS test?

A - Yes. Import your KS test and go to Library > Draft Content > Draft Tests, edit step 4 (Test Settings). Enter your chosen time in the 'Time limit' box.

Q - Can I hide the clock which appears on the KS test UI?

A - Yes. Import your KS test and go to Library > Draft Content > Draft Tests, edit step 4 (Test Settings). Uncheck the 'Show timer' box (see above).


Q - How do I copy admins on KS invite, reminder & results mails?

A - Go to Accounts > Administrators and toggle the account invites tool (small envelope icon) to 'on'.

Go to Settings > Invites and check the box below:

Go to Settings > Reminders and check the appropriate boxes:

Q - Is it possible to edit score and time data for KS results?

A - Yes. If you have a KS result that needs to be edited, email the details across to us and we can make the edits via the main KS db.


Q - Can I copy dynamic groups from the Users page to the Results page?

A - Yes. Go to Settings > User Datafields and click on 'Share Dynamic Groups', to share existing groups across the Users, Results and Invite History pages.

Q - Can I filter results by date of test taken?

A - Yes.  You can filter KS results data across a variety of search criteria. Go to Results > Data page and click on the 'Show search' tool to display the search options:

Q - Can I export test results?

A - Yes.  Go to the Results > Data page and click on the 'Show groups' tool.  Look for the 'Export results to csv' tool.  You have a number of export options to choose from:

Q - Can I edit the user rating labels which appear at the start of a test?

A - Yes. Go to Settings > Test UI Options and apply your changes in the fields provided.

Q - Can I choose whether or not to display the user survey pages at the start and end of a test session?

A - Yes. Go to Settings > Test UI Options and apply your change in the relevant check box.

Q - Can I stop users from logging out of a test session?

A - Yes. Go to Settings > Test UI Options (see above) and apply your change using the relevant check box.


Q - How can I compare our results to the overall average KS benchmark stats?

A - Go to Results > Global Comparisons and select the title(s) you want to compare your results data against.

R