Thursday 8 December 2011

AUGI Top DAUG 2011 - The Results



The AUGI Top DAUG competition was introduced at Autodesk University, way back in 2000. The contest originally consisted of two parts, based on AutoCAD 2D knowledge.  This year,  AUGI teamed up with KnowledgeSmart to expand the range of topics to include 7 tracks; 3ds Max, AutoCAD 2D, AutoCAD Civil 3D, Inventor, Revit Architecture, Revit MEP and Revit Structure.

Here is an overview of the contest from AUGI Hot News:
http://augi.typepad.com/files/augi-hotnews-daily---wednesday-nov-30-2011.pdf

There were some great prizes up for grabs, too.  Track winners walked away with a $100 USD Amazon voucher (a combination of high scores and fastest times determined the winner of each track) and the overall winner also won a HP notebook PC and a free pass to AU 2012!

We spent several hours on the Monday, setting up 18 networked PC's in the exhibition hall, at the AUGI stand. All 126 copies of the Autodesk software needed checking, then all the KS sample data sets had to be uploaded to each machine and checked again.  Big thanks to Daniel Heselwood, from Evolve, for answering our call for help.  (We actually called Daniel to ask for his help with a missing xref - and 4 hours later he was still helping us finish up!).

Here's a snapshot of the stand, when we were all set up:


The competition ran over 3 days (2 x 3 hour slots on the first 2 days, with a final hour session on the last day).   Contestants had to answer a range of questions, using the 2012 version of each software title. Each session was limited to just 14 minutes, so people had to work fast!  The new format proved popular, with eager users queuing up to grab a spare seat.  Special thanks to AUGI team members, Bob Diaz and Michael Patrick, for briefing users and making sure everything ran smoothly.

Here are the Top DAUG's in action:


By the end of the contest, we posted 305 results, across 7 tracks.  On a conference venue web connection, with the added ingredients of free beer and wine, we had a completion rate of 99%, which was way ahead of our anticipated 95% target. (Our support team promptly rescued the missing two scores, so all completed tests were captured and logged on the AUGI dashboard).

Throughout the competition, we posted a rolling list of the top 10 contestants for each track, on the AUGI big screen.



The Results

Congratulations to the following contestants, who won their respective Tracks:

3ds Max Track Winner:  James Clarke
AutoCAD 2D Track Winner:  Brent McAnney
AutoCAD Civil 3D Track Winner:  Brian Hailey
Inventor Track Winner:  Joe Bartels
Revit Architecture Track Winner:  Aaron Maller
Revit MEP Track Winner:  David Raynor
Revit Structure Track Winner:  Rebecca Frangipane

And a special mention to the overall winner of AUGI Top DAUG 2011:

Brian Hailey



Analysis

So, let's take a detailed look at the results of this year's Top DAUG competition.

Overall

No. of Tests Completed:  305
Overall Average:  53% in 12 mins 56 secs



Track 1 - 3ds Max

Track Winner: James Clarke
Winning Score: 64% in 13 mins 55 secs

Top 10 Contestants:

James Clarke
Jesse Sandifer
Daniel Heselwood
Paul Mazzoni
Fernando Oliveira
Jens Tange
Matti Oopik
Yosun Chang
Douglas Bowers
Charlie Forsythe

No. Completed: 13
Group Average: 18% in 12 mins 15 secs
Top 10 Average:  20% in 12 mins 2 secs


Track 2 - AutoCAD 2D

Track Winner: Brent McAnney
Winning Score: 100% in 11 mins 25 secs

Top 10 Contestants:

Brent McAnney
Alex Lepeska
Scott Wilcox
Heather Shrieves
Timothy Vaughan
Ben Rand
Decio Ferreira
Jim LaPier
Richard Lawrence
Youssri Salman

No. Completed: 110
Group Average: 42% in 13 mins 43 secs
Top 10 Average:  82% in 12 mins 42 secs


Track 3 - AutoCAD Civil 3D

Track Winner: Brian Hailey
Winning Score: 100% in 6 mins 10 secs

Top 10 Contestants:

Brian Hailey
Richard Lawrence
Travis Winter
Bryan Thomasy
Kirk Noonan
Jeff Nichols
Bruce Klug
Brent McAnney
Charles D'Errico
Bill Neuhauser

No. Completed: 26
Group Average: 61% in 12 mins 33 secs
Top 10 Average:  90% in 11 mins 24 secs


Track 4 - Inventor

Track Winner: Joe Bartels
Winning Score: 80% in 8 mins 20 secs

Top 10 Contestants:

Joe Bartels
Gerrard Hickson
Andrew Warren
Bill Graham
Alex Karan
Marius Minnen
Anders Tokerud
Chris Brown
Scott Wayand
Curtiss Cooke

No. Completed: 27
Group Average: 59% in 13 mins 6 secs
Top 10 Average:  75% in 12 mins 9 secs


Track 5 - Revit Architecture

Track Winner: Aaron Maller
Winning Score: 100% in 6 mins 55 secs

Top 10 Contestants:

Aaron Maller
Brian Mackey
Anthony Tiefenbach
Steve Faust
Sean Darnell
Eric Bernier
Adam Ward
Douglas Bowers
Andrew Fisher
David Ivey

No. Completed: 84
Group Average: 79% in 11 mins 35 secs
Top 10 Average:  99% in 10 mins 7 secs


Track 6 - Revit MEP

Track Winner: David Raynor
Winning Score: 80% in 13 mins 25 secs

Top 10 Contestants:

David Raynor
John Karben
Fernando Oliveira
Jason Vaia
Philip Charlson
David Rushforth
Clifford Baker
Douglas Bowers
Maxime Sanschagrin
Paul Beseman

No. Completed: 23
Group Average: 54% in 13 mins 50 secs
Top 10 Average:  67% in 13 mins 49 secs


Track 7 - Revit Structure

Track Winner: Rebecca Frangipane
Winning Score: 98% in 13 mins 45 secs

Top 10 Contestants:

Rebecca Frangipane
Andrew Lawrence
Dezi Ratley
Brian Mackey
Eric Bernier
Matthew Hill
Tina Bos
Michael Patrick
Jason Rhodes
Fernando Oliveira

No. Completed: 22
Group Average: 56% in 13 mins 32 secs
Top 10 Average:  77% in 13 mins 5 secs




Further Analysis & Observations

Popular Tracks

The most popular tracks, in order of completed tests, were as follows:

AutoCAD 2D - 110 results
Revit Architecture - 84 results
Inventor - 27 results
AutoCAD Civil 3D - 26 results
Revit MEP - 23 results
Revit Structure - 22 results
3ds Max - 13 results


Training Needs Analysis

So what does all this mean, in terms of performance and training?

For 3ds Max, the top 10 training items were, in priority order:
Creation Parameters, Cross Sections, Map Channels, Map Coordinates, Object Data Flow, Object Modifiers, Object Properties, Pivots, Rotate and Space Warps.



For AutoCAD 2D, the top 10 training items were, in priority order:
Attaching References, Modifying References, Dimensioning Objects, Measuring, System Variables, Object Snaps, Arcs, Drafting Tools, Lines and Tangent.



For AutoCAD Civil 3D, the top 10 training items were, in priority order:
Changing a Slope, Pipe Networks, Sections, Corridor Surfaces, Corridors, Breaklines, Surfaces, Survey, Alignments and Parcels.



For Inventor, the top 10 training items were, in priority order:
Parameters, Tolerances, Features, Revolve, Sketch, Hole, Project Settings, Styles, Components and Parts.



For Revit Architecture, the top 10 training items were, in priority order:
Clipboard, Data Integrity, Modeling, Family Editor, Families, File Management, Worksharing, Room Tags, Walls and Annotation.



For Revit MEP, the top 10 training items were, in priority order:
System Inspector, Apparent Load Values, Creating Power Systems, Editing Properties, Modifying Light Fixtures, Panel Circuits, Panel Schedules, Photometric Web Files, Selecting Equipment and Electrical Settings.



For Revit Structure, the top 10 training items were, in priority order:
Families, Scope Boxes, Types, Visibility, Roofs, Floors, Slabs, Structural Deck, Annotation and Span Direction Symbol.



Revit Data

It's interesting to compare the Revit results, across the 3 disciplines, which arguably reflects where Revit currently sits, in terms of wider industry adoption.  Revit Architecture proved to be the most popular track and the overall experience of the users taking part in Top DAUG, suggests that this software is the most mature, in terms of individual expertise.  Revit MEP and Revit Structure were very close, in terms of overall performance, but considerably behind Revit Architecture, on a like-for-like productivity comparison.

RAC overall:  79% in 11 mins 35 secs
RMEP overall:  54% in 13 mins 50 secs
RST overall:  56% in 13 mins 32 secs


It's also worth noting that the average Revit Architecture score, for the top 10 users, was an impressive 99% in 10 mins 7 secs.  In fact, 31 users out of 84 posted scores of 90% or higher, which is outstanding.

As an additional comparison, we posted 28 results for Revit Architecture at RTC USA, earlier in the year.  The overall average for Revit Architecture, at the RTC 'Top Cat' contest, was 79% in 9 mins 7 secs. (NB the questions were different at RTC, but the level of difficulty was on par with Top DAUG).


Range of scores

Interestingly, across the 7 tracks, we saw scores ranging from 0% to 100%.  Here is a summary of both ends of the performance scale:

7 x 100% scores (1 x AutoCAD 2D, 1 x AutoCAD Civil 3D, 5 x Revit Architecture).
13 x 0% scores (1 x 3ds Max, 11 x AutoCAD 2D, 1 x Revit Architecture).


Honourable mentions

Along with our track winners, the following contestants deserve a special mention, for their performance in the competition:

Brian Mackey - winner of the RTC USA Top Cat contest in the summer, Brian placed second in the Top DAUG RAC track (scoring 100%), and also placed in the top 4 for RST.  An exceptional effort!

Brent McAnney - won the AutoCAD 2D track and placed in the top 8 for AutoCAD Civil 3D.

Eric Bernier - placed inside the top 6 for both the RAC and RST tracks.

Douglas Bowers - placed inside the top 8 for both the RAC and RMEP tracks.

Fernando Oliveira - placed inside the top 10 for the 3ds Max, RMEP and RST tracks.

Richard Lawrence -  placed inside the top 10 for both the AutoCAD 2D and AutoCAD Civil 3D tracks.

Heather Shrieves - placed inside the top 5 for AutoCAD 2D and top 12 for Inventor.

Jason Vaia - placed inside the top 5 for RMEP and top 18 for RAC.


Boys vs Girls

Finally, let's take a look at the demographic breakdown of the competition. Out of 305 contestants, 271 were male and 34 female.  The average overall performance for each group breaks down like this:

Female:  61% in 12 mins 37 secs
Male:  53% in 12 mins 57 secs



So, there we have it! A thoroughly enjoyable 3 days at AU2011. 305 completed tests, across 7 popular Autodesk software applications.  It's fair to say that the overall standard was extremely high, with some truly outstanding individual performances from our track winners and top 10 contestants.  

Congratulations to all our winners.  Thanks again to the AUGI team for all their support, in helping us put together the new format for Top DAUG.  Particular thanks to AUGI President, David Harrington, past-President, Mark Kiker and past-Board member, Bob Diaz.  Lastly, a big thank you to everyone who took part in this year's contest.  See you all at the Mandalay Bay in 2012? 

R