Tuesday, 14 December 2010

SECI Model for Organizational Learning

How does an organization learn?

In 1995, Professors Nonaka & Takeuchi (at Hitotsubashi University, Tokyo) developed a four stage spiral model of organizational learning.

  • Socialization
  • Externalization
  • Combination
  • Internalization

Tacit knowledge is personal, context specific, subjective knowledge, whereas explicit knowledge is codified, systematic, formal, and easy to communicate. The tacit knowledge of key personnel within an organization can be made explicit, codified in manuals, and incorporated into new systems and processes. This process is called "externalization".

The reverse process (from explicit to implicit) is called "internalization" because it involves employees internalizing an organization's formal rules, procedures, standards and other forms of explicit knowledge.

“Socialization" denotes the sharing of tacit knowledge and the term "combination" denotes the dissemination of codified knowledge.

According to Nonaka & Takeuchi, knowledge creation and organizational learning take a path of socialization, externalization, combination, internalization, socialization, externalization, combination . . . and so on, in an infinite spiral.


So what exactly is an 'expert'?

The term 'expert' is defined as 'somebody with a great deal of knowledge about, or skill, training, or experience in, a particular field or activity'.

In CAD & BIM circles, the label AutoCAD 'expert', Revit 'expert', MicroStation 'guru', and so on, are used pretty liberally. In some cases, the monicker is perfectly justified. In others, however...

It's often interesting to capture a user's perception of how good they think they are at a particular software app, prior to them taking a skills assessment. We like to ask the following question:

Please rate yourself on your ability to use the software, for which you are about to take a test, on a scale of 1 to 5:

1 = Very basic knowledge; not enough to work confidently on a project
2 = Basic knowledge; can get by working on a project but could do better
3 = Good knowledge; can produce a good standard of work on projects
4 = Advanced knowledge; can teach the basics to others
5 = Expert knowledge; can perform and teach others at an advanced level

Interestingly, we are mirroring a recognised format for gauging skills measurement. In the fields of education and operations research, the Dreyfus model of skill acquisition is a model of how students acquire skills through formal instruction. Brothers Stuart and Hubert Dreyfus proposed the model in 1980 in an influential report on their research at the University of California, Berkeley, Operations Research Center for the United States Air Force Office of Scientific Research.

The model proposes that a student passes through five distinct stages; novice, advanced beginner, competent, proficient and expert.

1. Novice
"rigid adherence to taught rules or plans"
no exercise of "discretionary judgment"

2. Advanced beginner
limited "situational perception"
all aspects of work treated separately with equal importance

3. Competent
"coping with crowdedness" (multiple activities, accumulation of information)
some perception of actions in relation to goals
deliberate planning
formulates routines

4. Proficient
holistic view of situation
prioritizes importance of aspects
"perceives deviations from the normal pattern"
employs maxims for guidance, with meanings that adapt to the situation at hand

5. Expert
transcends reliance on rules, guidelines, and maxims
"intuitive grasp of situations based on deep, tacit understanding"
has "vision of what is possible"
uses "analytical approaches" in new situations or in case of problems

In the worlds of music and dance, becoming an expert requires an investment of approximately 10,000 hours in practice and execution!

Consider how this breaks down:

Hours per week Hours per year Years to achieve 'expert'

4 200 50
8 400 25
12 600 16.7
16 800 12.5
20 1,000 10
24 1,200 8.3

So the next time someone professes to be an expert, ask yourself, is that what they really mean?


KnowledgeSmart wish list

We've been holding a series of web review meetings with customers in recent weeks, discussing how their assessment programs are coming along, and what tools they would like to see next in the KS admin dashboard.

Here are the most requested items, the majority of which will feature in the next point release (scheduled for Feb 2011);
  • More charting; would like an easier way to tell (at a glance) what the main problem areas/training topics are
  • Would like to see industry comparisons for most popular test scores
  • Want to display more results per page
  • Would like ability to invite interview candidates more easily
  • Would like to control level of test report feedback for interview candidates
  • Ability to delete test scores
  • View results for linked accounts
  • Notify admin if test expiry date passes (but test not taken)
  • Change status of users (i.e. ex-employee, interview candidate to employee, etc.)
We'll also be adding basic branding options for firms, new chart styles in the dashboard and more content management options for test modules.

In addition, personal dashboards for users, basic survey tools, a community area for sharing test content and a new coaching mode are all coming up in the next few months.

Our rolling program of new test content authoring is ongoing. This week sees the release of our first Adobe InDesign module. More RMEP content will be out shortly, plus the first in a series of Civil 3D modules goes live in January. Intermediate RAC content, a Revit families module and basic Photoshop content will follow in early 2011.

If you have additional suggestions for new features or tools, we'd welcome your feedback.


Unskilled and unaware; why self-assessment is fundamentally flawed (part two)

This is part two of my case against the use of self-assessment, as a reliable means of measuring staff performance.

The Latin maxim “ignoramus et ignorabimus”, meaning "we do not know and will not know", stood for a position on the limits of scientific knowledge, in the thought of the nineteenth century.

In September 1930, mathematician David Hilbert pronounced his disagreement in a celebrated address to the Society of German Scientists and Physicians, in Königsberg;
“We must not believe those, who today, with philosophical bearing and deliberative tone, prophesy the fall of culture and accept the ignorabimus. For us there is no ignorabimus, and in my opinion none whatever in natural science. In opposition to the foolish ignorabimus our slogan shall be: We must know — we will know!”

In the late twentieth century, Ex-United States Secretary of Defense, Donald Rumsfeld, whilst defending his country's position on the Iraq war, made the following (now infamous) statement; “There are ‘known knowns’. These are things we know that we know. There are ‘known unknowns’. That is to say, there are things that we now know we don’t know. But there are also ‘unknown unknowns’. These are things we do not know we don’t know”.
There are four recognised stages of competence.

1) Unconscious Incompetence
The individual neither understands nor knows how to do something, nor recognizes the deficit, nor has a desire to address it. The person must become conscious of their incompetence before development of the new skill or learning can begin.

2) Conscious Incompetence
Though the individual does not understand or know how to do something, he or she does recognize the deficit, without yet addressing it.

3) Conscious Competence
The individual understands or knows how to do something. Demonstrating the skill or knowledge requires a great deal of consciousness or concentration.

4) Unconscious Competence
The individual has had so much practice with a skill that it becomes “second nature” and can be performed easily. He or she may or may not be able to teach it to others, depending upon how and when it was learned.

"The Invisible Gorilla" experiment is one of the most famous psychological demo's in modern history. Subjects are shown a video, about a minute long, of two teams, one in white shirts, the other in black shirts, moving around and passing basketballs to one another. They are asked to count the number of aerial and bounce passes made by the team wearing white, a seemingly simple task. Halfway through the video, a woman wearing a full-body gorilla suit walks slowly to the middle of the screen, pounds her chest, and then walks out of the frame. If you are just watching the video, it’s the most obvious thing in the world. But when asked to count the passes, about half the people miss it. It is as though the gorilla is completely invisible.

In his popular KM blog, Nick Milton (http://www.knoco.co.uk/Nick-Milton.htm) writes in detail about the impact of this experiment and picks up on a number of key trends discussed in the book of the same name, authored by Christopher Chabris and Daniel Simons (the guys behind the original experiment).

The subtitle of the book is "ways our intuition deceives us", and the authors talk about a number of human traits (they call them illusions) which we need to be aware of in Knowledge Management, as each of them can affect the reliability and effectiveness of Knowledge Transfer.

To paraphrase Milton, the illusions which have most impact on Knowledge Management are;

The illusion of memory
The illusion of confidence
The illusion of knowledge

Our memory of events fades over time, to the point that even firm documentary evidence to the contrary doesn't change what we remember. The implication is that if you will need to re-use tacit knowledge in the future, then you can't rely on people to remember it. Even after a month, the memory will be unreliable. Details will have been added, details will have been forgotten, the facts will have been rewritten to be closer to "what feels right".

Tacit knowledge is fine for sharing knowledge on what's happening now, but for sharing knowledge with people in the future then it needs to be written down quickly while memory is still reliable.

Without a written or photographic record, the tacit memory fades quickly, often retaining enough knowledge to be dangerous, but not enough to be successful. And as the authors say, the illusion of memory can be so strong that the written on photographic record can come as a shock, and can feel wrong, even if it’s right.

Any approach that relies solely on tacit knowledge held in the human memory can therefore be very risky, thanks to the illusion of memory.

The illusion of confidence represents the way that people value knowledge from a confident person. This would be fine if confidence and knowledge go hand in hand, but in fact there is almost an inverse relationship. A lack of knowledge is, instead, allied to overconfidence. Lack of knowledge leads to confidence, which leads to you being seen as knowledgeable.

Each chess player is given a points rating based on their competition results, which is in fact a very effective and reliable measure of their ability. Yet 75% of chess players believe they are underrated, despite the evidence to the contrary. They are overconfident in their own ability.

In studies of groups of people coming together to solve a maths problem, you would expect the group to defer to the person with the greatest maths knowledge, wouldn't you? In fact, the group deferred to the most confident person, regardless of their knowledge. In trials, in 94% of the cases, the final answer given by the group is the first answer suggested, by the most confident person present, regardless if whether it is right or wrong.

In a Harvard study of confidence vs knowledge in a trivia test, they certainly saw overconfidence in individuals - people were confident of their answer 70% of the time, while being correct only 54% of the time! When people were put together in pairs, the counterintuitive outcome was that the pairs were no more successful than the individuals, but they were a lot more confident! When two low-confidence people were put together, their overall confidence increased by 11%, even though their success rate was no higher than before.

The Illusion of Knowledge is behind the way we overestimate how much we know. The authors refer to how people think they know how long a project will take, and how much it will cost, despite the fact that projects almost always overrun in both cost and time. "We all experience this sort of illusory knowledge, even for the simplest projects" they write. "We underestimate how long they will take or how much they will cost, because what seems simple and straightforward in our mind typically turns out to be more complex when our plans encounter reality. The problem is that we never take this limitation into account. Over and over, the illusion of knowledge convinces us that we have a deep understanding of what a project will entail, when all we really have is a rough and optimistic guess based on shallow familiarity"

"To avoid this illusion of knowledge, start by admitting that your personal views of how expensive and time-consuming your own seemingly unique project will be are probably wrong. If instead, you seek out similar projects that others have completed, you can use the actual time and cost of these projects to understand how long yours will take. Taking such an outside view of what we normally keep in our own minds dramatically changes how we see our plans"

If we are unaware of these 3 illusions, we can feel confident in our knowledge, based on our memories of the past, without realising that the confidence is false, the knowledge is poor, and the memories are unreliable and partially fictitious. Awareness of these illusions allows us also to challenge the individual who confidently declares "I know how to do this. I remember how we did it 5 years ago", because we recognise the shaky nature of confidence, knowledge and memory.

A natural human tendency is that we tend to think that we know more than we do and that
we are better than we are. We suffer from what psychologists call the “Lake Wobegon effect”.
Based on Garrison Keillor’s fictional town where “all the women are strong, all the men are good-looking and all the children are above average.” According to the author’s own survey, 63% of Americans consider themselves more intelligent than the average American.

In contrast, 70% of Canadians said they considered themselves smarter than the average Canadian. In a survey of engineers 42% thought their work ranked in the top 5% among their peers. A survey of college professors revealed that 94% thought they do ‘‘above average’’ work – a figure that defies mathematical plausibility! A survey of sales people found that the average self-assessment score (for sales demos) was 76%. The average % of demos that achieved
objectives (for the same group) was 57%. The list goes on..

So, in summary, any strategy for capturing user skills data, which relies solely on an individual's ability to self-rate themselves on a given subject, is simply doomed to fail. I leave the last word to David Dunning; “In essence, our incompetence masks our ability to recognize our incompetence”.


Unskilled and unaware; why self-assessment is fundamentally flawed (part one)

In October, I travelled to Boston, to present a paper to the AIA HR Large Firm Roundtable. I decided to make the case against the use of self-assessment, as a reliable means of capturing management metrics for staff performance.

Over the years, many AEC firms have confidently stated to me, 'We don't need independent skills testing, we already know how good our users are'. When one enquires further, what they actually mean, is that they sent out a user survey, asking staff to rate themselves (usually out of 5) on a range of different skills topics, including AutoCAD, Revit, InDesign, et al. What they end up with is a spreadsheet (why is it always a spreadsheet??) with a bunch of names down one side, a list of skills categories across the top - and a sheet filled with 3's and 4's. Why 3's and 4's, I hear you ask? Simply because people don't want to risk the personal penalties that might go along with admitting they're a 1 or a 2. And conversely, they don't want to stick their head above the parapet by admitting to a 5 (even if they are a 5) because this can cause all sorts of new issues (more work, more people pestering them for answers to the same questions, you get the picture). So it's 3's and 4's all the way. Congratulations XYZ Architects, you have your completed spreadsheet, so you are now totally self-aware, as an organization. (Not). And the real rub here is that, more often than not, people have no clue how good they are, relative to the rest of the team, or wider industry!

So I decided to put this argument to bed, once and for all. Here's my evidence..

Let me begin with a story. Earlier this year, NY Times Online posted a series of articles by filmmaker Errrol Morris. He tells the tale of Bank robbery suspect McArthur Wheeler, who was recognized by informants who tipped detectives to his whereabouts after his picture was telecast one Wednesday night, during the Pittsburgh Crime Stoppers segment of the 11 o’clock news. At 12:10 am, less than an hour after the broadcast, he was arrested. Wheeler had walked into two Pittsburgh banks and attempted to rob them in broad daylight. What made the case peculiar is that he made no visible attempt at disguise. The surveillance tapes were key to his arrest. There he is with a gun, standing in front of a teller demanding money. Yet, when arrested, Wheeler was completely disbelieving. “But I wore the juice,” he said. Apparently, he was under the deeply misguided impression that rubbing one’s face with lemon juice rendered it invisible to video cameras.

Pittsburgh police detectives who had been involved in Wheeler’s arrest explained that Wheeler had not gone into “this thing” blindly but had performed a variety of tests prior to the robbery. Although Wheeler reported the lemon juice was burning his face and his eyes, and he was having trouble (seeing) and had to squint, he had tested the theory, and it seemed to work. He had snapped a Polaroid picture of himself and wasn’t anywhere to be found in the image. There are three possibilities:

(a) the film was bad;
(b) Wheeler hadn’t adjusted the camera correctly; or
(c) Wheeler had pointed the camera away from his face at the critical moment when he snapped the photo

Pittsburgh Police concluded that, 'If Wheeler was too stupid to be a bank robber, perhaps he was also too stupid to know that he was too stupid to be a bank robber — that is, his stupidity protected him from an awareness of his own stupidity.'

Now, this sorry tale might have been just another footnote in history, were it not for the fact that it came to the attention of David Dunning, a Cornell professor of social psychology. After reading this story in 1996, Dunning wondered whether it was possible to measure one’s self-assessed level of competence against something a little more objective – say, actual competence.

Over the next 3 years, Dunning (assisted by colleague Justin Kruger) undertook a major academic study and, in 1999, published the paper, “Unskilled and Unaware of It: How Difficulties of Recognizing One’s Own Incompetence Lead to Inflated Self-assessments”.

Dunning’s epiphany was; “When people are incompetent in the strategies they adopt to achieve success and satisfaction, they suffer a dual burden; not only do they reach erroneous conclusions and make unfortunate choices, but their incompetence robs them of the ability to realize it. Instead, like Mr. Wheeler, they are left with the erroneous impression they are doing just fine. In essence, our incompetence masks our ability to recognize our incompetence”.

Dunning & Kruger also quote the “above-average effect”, or the tendency of the average person to believe he or she is above average, a result that defies the logic of statistics. Participants scoring in the bottom quartile on tests grossly overestimated their performance and ability. Although test scores put them in the 12th percentile they estimated themselves to be in the 62nd.

Conversely, Because top performers find the tests they confront to be easy, they mistakenly assume that their peers find the tests to be equally easy. As such, their own performances seem unexceptional. In studies, the top 25% tended to think that their skills lay in the 70th–75th percentile, although their performances fell roughly in the 87th percentile.

Dunning and Kruger proposed that, for a given skill, incompetent people will:

tend to overestimate their own level of skill;
fail to recognize genuine skill in others;
fail to recognize the extremity of their inadequacy;
recognize and acknowledge their own previous lack of skill, if they can be trained to substantially improve.

As a follow up study, “Why the unskilled are unaware: Further explorations of (absent) self-insight among the incompetent”, was published in 2006. (David Dunning, Justin Kruger, Joyce Ehrlingera, Kerri Johnson, Matthew Banner).

In part two, I'll take a look at the 4 stages of competence - and how the combined illusions of memory, confidence and knowledge can impact on a firms' knowledge management strategy.


Friday, 10 December 2010

UK BIM Roundtable - meeting notes

Thanks again to those of you who attended and contributed to the UK BIM Roundtable meeting, last month.

The meeting notes have been posted online, here: http://www.bimroundtable.com/.

Please feel free to provide any comments or feedback, as appropriate.

We anticipate running a follow up session in the Spring, focusing on one or two of the key discussion points from meeting one. More details to follow in the NY..


Testing Adobe InDesign skills

Another day, another milestone! We’ve had quite a few requests this year, mostly from our Architectural customers, to create some assessments for Adobe’s Creative Suite of tools.

To that end, the first module – Adobe InDesign for occasional users (CS5) – goes live this month.

In the same style as our assessment content for Autodesk and Bentley Systems' technical software apps, our system provides Adobe users with a live online skills test experience, presenting a mix of theory and task-based questions. (Theory questions can be answered without opening InDesign; task-based questions require a copy of the software). A test takes on average 30-60 mins, depending on the experience of the user. At the end, candidates can view a detailed test report, complete with feedback, coaching notes and suggestions for further training workshops.

Having spent the past 7 years or so, involved solely in the Autodesk and Bentley space, it's quite a treat to be adding a third major vendor to our portfolio of assessment offerings!

Over the coming months, we have the following additional assessments planned:

Adobe Photoshop for occasional users (CS5) – early 2011
Adobe InDesign fundamentals (CS5) – Spring 2011
Adobe Illustrator for occasional users (CS5) – Spring 2011
Adobe Photoshop fundamentals (CS5) – Spring/Summer 2011
Adobe Acrobat 9 fundamentals – Summer 2011

If you'd like to trial the new InDesign test, drop us a line or use the contact form on the main KS website. Enterprise customers will see the new modules appear in their dashboards, as soon as we go live.


Thursday, 25 November 2010

Testing software skills of new hires (in the US)

I've blogged on this subject in the past, but it just keeps coming up in conversation with HR professionals, Stateside; is it OK to test the basic CAD or BIM skills of prospective employees in the USA? A common conundrum surrounding skills assessments is whether you are legally permitted to screen the technical software ability of prospective employees and contract staff, before they join your team.

The short answer is yes, but only under the right circumstances. Let’s examine some of the evidence.

The most important issue underlying the use of pre-employment assessments is validity. The question we need to ask is this; ‘Is the test valid for this intended purpose; does it support the decisions that are going to be made?’

So what is 'validity'? Validity measures how appropriate a test is for a specific purpose. Simply put, a test may be considered valid for one use and invalid for another. Why do pre-employment tests need to be validated? In 1978, the Equal Employment Opportunities Commission (EEOC) created guidelines to ensure that the knowledge gained from testing is applied with impartiality to protect minority applicants from discriminatory employment procedures. What is the best method of validation? The EEOC guidelines do not state that one method is better than another; the method used must fit the needs of the business or organization.

There are three methods of validation set forth by the EEOC:

1) Criterion Validity: If data demonstrates that a test is significantly correlated with a vital measure of job performance, the test is said to demonstrate criterion validity. For example, if all the users that scored highly on a selected test to measure CAD skills completed their projects accurately and on time, the test would demonstrate criterion validity.

2) Construct Validity: The term construct is a technical term for personality traits like intelligence and creativity. Construct validity is demonstrated if a test measures traits that have been found to influence successful performance of a job. A test that measures the interpersonal communication skills of a potential customer service representative would demonstrate construct validity.

3) Content Validity: This is demonstrated if the questions that make up an assessment are representative of content that is required to perform a particular activity or task. A test made up of algebra questions given to an applicant for a maths teacher's position would demonstrate content validity.

Many employers use employment tests and other selection procedures in making employment decisions. Examples of these tools, many of which can be administered online, include the following:

- Cognitive tests assess reasoning, memory, perceptual speed and accuracy, and skills in arithmetic and reading comprehension, as well as knowledge of a particular function or job;
- Physical ability tests measure the physical ability to perform a particular task or the strength of specific muscle groups, as well as strength and stamina in general;
- Sample job tasks (e.g. performance tests, simulations, work samples, and realistic job previews) assess performance and aptitude on particular tasks. NB CAD & BIM skills assessments fall into this category;
- Medical inquiries and physical examinations, including psychological tests, assess physical or mental health;
- Personality tests and integrity tests assess the degree to which a person has certain traits or dispositions (e.g. dependability, cooperativeness, safety) or aim to predict the likelihood that a person will engage in certain conduct (e.g. theft, absenteeism);
- Criminal background checks provide information on arrest and conviction history;
- Credit checks provide information on credit and financial history;
- Performance appraisals reflect a supervisor’s assessment of an individual’s performance; and
- English proficiency tests determine English fluency.

An important item to remember when interpreting assessment scores is to put the results in context, and to compare them to an external performance benchmark. For example, a 58% score does not reflect ‘failure’. If this score is presented by a CAD user against an office average of 60%, it is likely the candidate will comfortably fit in to the team.

Test questions should also have a recognizable skill level; basic, intermediate or advanced. In this way, the benchmark data can be reliably used to compare performance and make sound hiring decisions.

In conclusion, for a CAD or BIM skills test to be valid, it must contain content that reflects a representative sample of the target skill. A reliable skills test should comprise sample job tasks (e.g. performance tests, simulations, work samples, and realistic job previews) and assess performance and aptitude on particular tasks.

Skills evaluations should not discriminate according to; race, colour, national origin, sex, religion, age (40 or older), or disability.

Lastly, employers should ensure that employment tests and other selection procedures are properly validated for the positions and purposes for which they are used. The test or selection procedure must be job-related and its results appropriate for the employer's purpose.


Business Benefits of Benchmarking

Benchmarking is the process of comparing the cost, time or quality of what one organization does against what another organization does. The result is often a business case for making changes in order to make improvements. Also referred to as "best practice benchmarking" or "process benchmarking", it is a process used in management where organizations evaluate various aspects of their processes in relation to best practice, usually within their own sector. This then allows organizations to develop plans on how to make improvements or adopt best practice, with the aim of increasing performance.

The most prominent methodology is the 12 stage approach by Robert Camp (who literally wrote the book on benchmarking in 1989). It consists of;

1. Select subject 2. Define the process 3. Identify potential partners 4. Identify data sources 5. Collect data and select partners 6. Determine the gap 7. Establish process differences 8. Target future performance 9. Communicate 10. Adjust goal 11. Implement 12. Review / recalibrate.

Benchmarking can take various guises:
Process benchmarking - a firm focuses its observation and investigation of business processes with a goal of identifying and observing the best practices from one or more benchmark firms. Activity analysis will be required where the objective is to benchmark cost and efficiency.

Financial benchmarking - a company performs a financial analysis and compares the results in an effort to assess overall competitiveness.

Performance benchmarking - allows a firm to assess their competitive position by comparing products and services with those of target firms.

Product benchmarking - the process of designing new products or upgrades to current ones. This process can sometimes involve reverse engineering competitors’ products to find strengths and weaknesses.

Strategic benchmarking - involves observing how others compete. This type is usually not industry specific, meaning it is best to look at other industries.

Functional benchmarking - a company will focus its benchmarking on a single function in order to improve the operation of that particular function, i.e. Human Resources, Finance and ICT.

Internal benchmarking - involves benchmarking businesses or operations from within the same organisation (e.g. business units in different countries).

External benchmarking - analysing outside organisations that are known to be best in class provides opportunities of learning from those who are at the ‘leading edge’.

International benchmarking - best practitioners are identified and analysed elsewhere in the world; globalisation and advances in information technology are increasing opportunities for international projects.

Benchmarking involves four key steps:
1) Understand in detail existing business processes
2) Analyse the business processes of others
3) Compare own business performance with that of others
4) Implement the steps necessary to close the performance gap

Benchmarking should not be considered a one-off exercise. To be effective, it must become an ongoing, integral part of an ongoing improvement process with the goal of keeping abreast of ever-improving best practice.

Why Bother?
There are many benefits of benchmarking; the following list summarises the main ones:

• provides realistic and achievable targets
• prevents companies from being industry led
• challenges operational complacency
• encourages continuous improvement
• allows employees to visualise improvement which can be a strong motivator for change
• creates a sense of urgency for improvement
• confirms the belief that there is a need for change
• helps to identify weak areas and indicates what needs to be done to improve.

So how does all this apply to us - what are the benefits of benchmarking CAD & BIM performance? In summary:

• Gain visibility of core CAD & BIM skills
• Identify individuals’ strengths & weaknesses
• Implement better CAD & BIM training plans for staff
• Improve recruitment processes
• Share performance data across an organisation
• Promote collaborative working between teams & offices
• Measure the performance of outsourcing or off-shoring partners
• Provide ‘best practice’ for CAD & BIM development
• Offer clearer staff inductions and more meaningful staff appraisals
• Enjoy better skills resourcing for projects
• Develop a continuous improvement process for CAD & BIM
• Save time and money on construction projects and offer best value for clients

In the end, it all comes down to better visibility of your teams and their real ability to use often complex technology and tools for maximum effect. If you can't measure it, you can't manage it.


Monday, 15 November 2010

UK Construction - The Impact of Recession (so far)

During the summer, I was invited to present to a group of AEC professionals about the impact of the recession on the UK Construction Industry. Whilst still a wee bit premature to be looking back with the benefit of hindsight - let's face it, we're not out of the woods just yet! - it is possible to gather some useful and interesting data on what has been happening during the past 3 years of economic downturn.

First, the basics; a recession is a significant decline in economic activity spread across the economy, lasting more than a few months, normally visible in production, employment, real income, and other indicators. A recession begins when the economy reaches a peak of activity and ends when the economy reaches its trough.

The first of 6 successive quarterly falls in UK GDP started in Q2 2008. The commonly accepted indicator that a recession has begun used to be, “two consecutive quarters of negative growth in GDP”. However, it's not quite so simple!

According to the National Bureau of Economic Research – UK (the official arbiter of recessions) the current recession began as early as December 2007.

5 key measurements NBER UK points to, with respect to a recession:

- GDP (a measure of a country's economic activity, namely of all the services and goods produced in a year)
- Retail/wholesale sales figures
- Overall employment
- Personal income
- Industrial production

Meanwhile, the Business Cycle Dating Committee of the National Bureau of Economic Research (NBER) – USA, maintains a chronology of the beginning and ending dates (months and quarters) of U.S. recessions. The committee determined that a peak in economic activity occurred in the U.S. economy in December 2007.

So whilst many firms have reported that the UK economy really went into a nose-dive in Q4 2008, it was actually nearly a year earlier that the problems began!

Some interesting data was quoted in a Building Magazine editorial, back in June 2010;

Experian predicts a slight upturn for 2011 and 2012 – depending on recovery in the private sector and no more than £5bn cuts in the public sector. (RV - hmmm not sure that they could foresee the current round of cuts when that was written!)

FT review of Office of National Statistics data found the following unemployment figures:

  • 720% increase for architects
  • 655% increase for QS’s
  • Similar figures for Planners, Construction Managers & Engineers (500%+)

Earlier in the year, the Chartered Institute of Building (CIOB) published the results of their third annual survey, where they polled opinion from 1182 UK construction firms.

  • Results show that the industry is still suffering a skills shortage despite the recession and downturn in construction demand.
  • 77% of respondents believe there is a skills shortage in construction and 78% of those feel that the loss of skills will hinder the industry’s recovery when the economy improves.
  • 54% state that their company has had to make redundancies and 14% expect further redundancies to occur.

Similar comments echoed by Construction Skills, who observed; 'Forecasts demonstrate the severe effect that the recession is having on the UK construction industry, with rising job losses risking huge skills deficits in the long term.

The Construction Skills Network (CSN) shows the prognosis for the UK construction industry in the short term (2009 to 2010) is poor. The industry is expected to contract in construction output terms by at least 12% in 2009, followed by a more marginal decline of around 2-4% in 2010.

Consistent recovery is not forecasted until 2011 and even then, it is likely to be a slow and steady return to moderate levels of growth as confidence returns to the market.'

Construction News published the following figures;

  • UK Construction employment began to fall in 2008 with a decline of 1% over the year, however, CSN forecasts a much larger drop of 15% across 2009 to 2010, with the largest losses (around 13%) expected in 2009. The total number of construction job losses from 2009 to 2010 could be up to 450,000 if output contracts by the suggested higher rate of 13% in 2009. The slow recovery forecast from 2011 to 2014 is set to create around 125,000 jobs.

  • Coupled with the large number of workers who are set to leave the industry in the next 10 years through retirement, the need to increase the flow of apprentices and graduates into the industry and retain as many existing skilled workers as possible is vital if the UK is to sustain a skilled construction workforce.

Data from the CIOB survey, noted the following points about Apprentices & Graduates;

  • Only 37% are sure their companies are still employing apprentices.
  • 11% stated that their companies usually employ apprentices, but cannot afford to in the current economic climate.
  • 12% of respondents are aware of their companies recruiting more graduates.
  • Only 1% are recruiting the same number of graduates as before the recession.

So what are the lessons we can take from all of this? What mistakes have firms made and what are the opportunities for growth when things pick up?


  • Panic firing/firing the wrong people, i.e. firing the best software users or even the CAD/BIM Manager!
  • Hoping things will get better sooner; waiting too long to make key strategic decisions
  • Freeze on technology investment
  • Freeze on all hiring
  • Freeze on all training & learning budgets
  • Lack of cash flow management


  • Removal of ineffective staff (who perhaps shouldn’t have been hired in the first place?)
  • Better users remaining
  • More time to spend on IT strategy & ‘housekeeping’
  • Better hiring practices in future
  • Better training & learning plans in future
  • Better cash flow procedures
  • Better, more realistic business plans
  • More focused roadmaps for technology, marketing and business strategy
  • Better relationships with remaining clients
  • Leaner working practices
  • More enthusiastic adoption of BIM

A final word from an article which appeared in the FT in the summer; “Those businesses that survive a recession of this magnitude, are the ones who are best placed to ride the wave back up again!”.


UK BIM Roundtable

Recent months have seen a plethora of forums and conferences about this year's numero uno hot potato, namely, BIM. I've attended a number of these sessions and, whilst the networking is always useful, the content has varied tremendously. So many meetings still focus on the big picture of BIM, almost selling the idea that BIM is the future and we all need to get on board. I think, for most firms, they've already made that leap. They kind of get it already, that we can all do better in terms of delivering more efficient projects and harnessing the latest technology to make it happen.

The question now is, how exactly do we, as individual firms and collectively an industry, sally forth and turn all the rhetoric into BIM-reality? If you speak to the marketing teams of many UK design & engineering firms, the story is that BIM is altogether sorted, that they are 100% BIM proficient and can deliver projects in any technology, at any time, with maximum efficiency, no fuss and zero downtime! :) Talk to the operations teams, who are tasked with actually making this all happen, and the story isn't quite the same. And this is in no way a criticism of these hard-working people. Quite the opposite. But, ironically, when firms speak openly about this stuff, the reality is that they are all wrestling with the same challenges!

To this end, we've been working with a group of connected people this year, to put together a programme of round table meetings, whose sole aim is to provide detailed answers to some key questions about BIM - at a practical level. The nice thing about this initiative is that no-one is forcing an agenda onto the proceedings. Whilst we all have a vested interest in the UK Construction industry picking itself up off the floor, the rest is all about using whatever connections or influence we may have individually, and pooling our resources to bring together the right people, at the same time, to figure out some new answers to these questions.

This Thursday in London, we have the first UK BIM Roundtable. All being well, we anticipate that this could develop into a regular series of half-yearly, or even quarterly, forums, each addressing a different area of BIM.

The first meeting focuses on the dynamics of the relationship between Architects and Engineers and General Contractors. Discussion will be based around the following topics:

What are the needs and requirements of consultants and what are the needs and requirements of contractors, when undertaking a BIM project?

How can we negate conflicts occurring at the interface of a design-based BIM as it undergoes the transition to the construction phase?

Getting BIM deliverables in sync throughout the project; exploring commonly accepted workflows and identifying the requirements of each discipline.

The meeting is by invitation and all 30 available places were filled very quickly. The following firms will have representation at the session;

Consultants: AECOM, WSP, Buro Happold, Halcrow, Hoare Lea, Capita Architecture, Zaha Hadid, Atkins, Levitt Bernstein, Foster + Partners, Davis Langdon, Waterman, IBI Nightingale, HOK, Tribal, Jacobs, Ramboll UK, Scott Wilson, BDP.

Contractors: Skanska, TPS Carillion, Carillion, Bovis Lend Lease, Mace, Laing O'Rourke, Costain, BAM, Vinci, Robert McAlpine, plus UKCG.

We also anticipate having people from TfL, Crossrail and BIS in attendance.

This is designed to be an interactive discussion forum. There will be an agenda and specific objectives for the session, steered by co-chairs representing both groups, but the rest is up to the delegates. We have no commercial sponsors, nor formal presentations from pre-arranged speakers. The minutes of the roundtable will be documented and issued in the form of a short report following the meeting. Thanks to Buro Happold for providing the venue for this meeting.


Wednesday, 27 October 2010

Sketchup is dead! (Long live Sketchup!)

I've noticed an interesting trend on my travels this year, regarding what was, up until recently, the darling of the design office (at least from a drafting software pov) - yes, I mean our old friend Google Sketchup.

We have had a ton of requests for a Sketchup test this year. And I should make this clear, we have manfully resisted this for quite some time, feeling that a Sketchup test was somehow an exercise in overkill. Many times I have asked the question, 'Why on earth would you want a Sketchup test?!'.

Nevertheless, it still keeps coming up in feedback forums, so it has been added to our list of short term devs.


...I don't know if something is in the water, but over the past few months, I have been in meetings with heads of design and technology (mostly in Architectural firms, to be fair) where Sketchup is no longer the poster-child of design software. Oh no, in fact, quite the opposite! 'We're trying to kill Sketchup', was the anguished cry from one leading design firm in New York. And back in Blighty, the same trend is emerging; 'Sketchup has become my biggest pain in the ****!', bemoans another technology leader.

But what has prompted such a collective venting of creative spleens, I hear you cry? Well, it seems that, whilst Sketchup is great for knocking out quick visualizations, too many people are trying to use it for much more than that.

In some ways, Sketchup is becoming a victim of its own simplicity. Because Architects are nowhere near as scared of learning Sketchup as they are of, say, Revit or MicroStation, they're now trying to use it for all sorts of inappropriate outputs. Let's make this clear - Sketchup is not Revit. You can't create a BIM model in Sketchup - so stop trying!

And it doesn't stop there. Not only is misguided use of the drafting tool causing the collective blood-pressure of BIM managers everywhere to reach Vesuvian levels, it is also now being cited in a couple of legal cases where, I kid you not, firms of Architects are being sued by clients for mishandled project information leading to mistakes during a build. Turns out that shadow diagrams quickly crunched out in Sketchup have been used as hard data during a build, which, in turn, has led to some pretty upset clients!

Suddenly, the ramifications of using technical software inappropriately on projects isn't funny at all. It can cost AEC firms thousands of dollars in lost income - not to mention the management time that goes into yet another round of unnecessary litigation. And those design firms who poo-poo the importance of technical software as a highly sought after skill? You have been warned. No longer can you get away with the familiar refrain, 'We only employ good designers. Software skills don't matter - we can always get them up to speed in Revit later'. (For the record, my blood freezes whenever I hear those 5 words - get... them... up... to... speed).

Oh, and don't blame Sketchup for this mess. It's not the one at fault here! The clue is in the name, stupid! ;)


Monday, 27 September 2010

Herbert Hoover's Engineer

I saw an engaging post on a LinkedIn discussion forum this week, about why people became engineers and what their favourite quotes are about the profession.

Here's the one I liked best, a quote from engineer and past US President, Herbert Hoover:

"It is a great profession. There is the fascination of watching a figment of the imagination emerge through the aid of science to plan on paper. Then it moves to realization in stone or metal or energy. Then it brings jobs and homes to men. Then it elevates the standards of living and adds to the comfort of life. That is the engineer’s high privilege.

The great liability of the engineer compared to men of other professions is that his works are out in the open where all can see them. His acts, step by step, are in hard substance. He cannot, like the architect, cover his failures with trees and vines. He cannot like the politician screen his shortcomings by blaming his opponents and hope the people will forget. The engineer simply cannot deny he did it. If his works do not work he is damned.

On the other hand, unlike the doctor, his is not a life among the weak. Unlike the soldier, destruction is not his purpose. Unlike the lawyer, quarrels are not his daily bread. To the engineer falls the job of clothing the bare bones of science with life, comfort and hope. No doubt as years go by the people forget which engineer did it, even if they ever knew. Or some politician put his name on it. Or they credit it to some promoter who used other people’s money…

But the engineer himself looks back at the unending stream of goodness which flows from his success with satisfaction that few professions may know. And the verdict of his fellow professionals is all the accolade he wants."


Thursday, 19 August 2010

540,000 hours

I was chatting to one of our large AEC customers last week. They have about 1,000 CAD, BIM & Engineering software users around the group. During our chat, we discussed the fact that they are putting together a business case for management, aimed at securing additional budgets for investing in new training & learning tools. As part of this process, they worked out that they spend approx. 540,000 man (or woman) hours EACH YEAR, producing CAD drawings. And that's just in the UK, which is only part of the wider group!

I don't know what their hourly charge-out rates are, but let's say, for argument's sake, that the figure is £100/hr. That's a cool £54M per year on CAD production alone.

Now let's picture the impact of a 2% improvement in productivity across the year - that's over £1M in savings, or half a million quid for every 1% increase!

Suddenly, spending a few grand on testing and training tools starts to make a lot of business sense!

Let's look at some more numbers;

Take an average salary of £25k p.a. and a 225 day working year.

For every 1% improvement in productivity, a practice will save £250 or 2.25 days per person per year. Targeting a 5% annual improvement in performance, this is worth £1,250 or 11.25 days per person per year. (11.25 days = 90 working hours (average 8 hour work day)).

For a firm with 20 staff, this is worth the equivalent of a new member of staff! (20 x £1,250 = £25,000).

Now let's look at hiring; how much does it cost to employ inappropriate personnel for 3 months?

Assuming a £25k annual salary:

15% agency fee = £3,750
3 months’ salary = £6,250

Assuming £18/hr:

7.5 hrs x 5 days = £675 x 12 weeks = £8,100

These examples don’t include advertising costs for staff employed direct, or the cost of re-working for drawings which weren’t done right first time.


Thursday, 5 August 2010


This month, we are introducing a change to the way assessments are authored, based on the idea of linking separate modules in a 'daisy-chain', to create a longer test.

In essence, the KS system breaks down like this;

- A test is one or more modules linked together.
- A module is comprised of typically 4-6 questions, covering the same topic, i.e. Families in Revit or UCS in AutoCAD, etc.
- A question tests an individual's understanding of a particular software feature or command.

Questions and modules have training tag keywords (meta tags) associated with them, which makes training needs analysis, post-assessment, much more relevant.

Questions can be "task" based or "knowledge" based.

A task based question has files associated with it. The user has to do something to the file, in order to work out the answer, as follows;

- Task (interrogate) - the user has to open the file and find the answer, but they don't have to modify or change the file in any way.
- Task (modify) - the user has to open and modify the file, to work out the correct answer.
- Knowledge - this type of question does not require a file to be associated with it.

Test administrators can present a single module as a test, provided they create a new test from the module first.

A 'daisy-chain' happens when an administrator links a series of modules together, to create a longer test.

This concept allows us to create a wide range of test material, in bite-sized blocks, which map closely to corresponding training and learning material.

Daisy-chains go live next month.


MCMC 2010

It's that time of year again! One of the year's most enjoyable events takes place on August 10-11; the Mid Continent MicroStation Community User Conference. The venue is once again the excellent Overland Park Convention Center, in Kansas City.

The organisers have pulled out the stops, with an agenda packed full of high-quality speakers, over the course of the two days.

See this link for full details:
http://www.tmcmcmc.org/meetings/2010/Onsite_Agenda_2010.pdf .

Tuesday night's entertainment includes dinner and a repeat trip to Power Play, for another high energy Whirlyball contest. (For the un-initiated, Whirlyball is a mix of lacrosse, basketball - and bumber cars - not for the faint hearted!).

Tickets are just $95 per day, including lunch and snacks throughout the day.

The theme for this conference is Versatility. Covering multiple tracks, including, MicroStation, Civils, Plotting, CAD Management, Collaboration, Building, Structural Analysis and US Army Corps projects.

Wednesday, 9 June 2010

What happened in Chicago...

A brief postscript to my recent visit to Chicago, where I attended the inaugural KA-Connect Knowledge management conference. The team at Knowledge Architecture have been posting some of the presenter videos recorded over the two days, onto their website.

Here is mine: http://ka-connect.com/talks.php?vdx=48.

The funny thing about my presentation was that I had the very last slot on day 2. It's fair to say that of my Pecha Kucha session - 20 slides, 20 seconds per slide - about 75% of what I had to speak about, had already been covered at some point during the previous 35 talks! So much of what you see here is delivered on the fly. :)

There were some terrific presenters during the course of the 2 days. Here are some of my favourites:

Laurie Dreyer, head of HR at US design firm Anshen + Allen (34 mins long, but frankly unmissable viewing!): http://ka-connect.com/talks.php?vdx=40.

John Moebes, Director of Construction at Crate+Barrel, with a fascinating insight into what owners expect from their design, engineering and construction partners:

Michael Kilkelly, Associate at Gehry Partners, gives us a look behind the scenes of one of the world's leading design firms: http://ka-connect.com/talks.php?vdx=57.

KA-Connect 2011 takes place in San Francisco, in April 2011.


Friday, 4 June 2010

EYC CAD Managers Forum - summer 2010

I'll be joining the EYC team at this summer's CAD Managers Forum in London, later this month.

See link for full details: http://www.eatyourcad.com/resource.php?resource_id=1676

The meeting takes place on June 30, at Buro Happold's Newman Street offices, from 10am to 4.30pm.

There are two main topics on the agenda; I'll be talking about the effects of the recession (isn't it over yet??) and what lessons we may have learned. The afternoon session looks at BIM contracts, technology and deliverables.

All in all, it promises to be an excellent forum. Tickets are just £50, to cover EYC's costs, so be sure to book your slot - and I look forward to seeing you in London.


Coming up in KS..

Just back from a very enjoyable few days in Philadelphia, attending an excellent Bentley conference, Be Together. The keynotes were thought-provoking, the seminars plentiful, the after-hours activities stimulating and the live zone format worked a treat. Slightly earlier promotion of the event next time, coupled with a bit more time between sessions, and it's a winner.

Anyway, I digress. The point of this post is to shed some light on what's coming up on the KS dev side over the summer period. Our technical roadmap has always been, I think it's fair to say, a somewhat fluid affair. We regularly receive tremendously bright feedback from our user-base, which has a direct impact on our delivery schedules. (Especially the really good ideas, when I ask the team, 'Why didn't we think of that!?' ;) ).

So, here's what we're working on over the next 2-3 months, in rough priority order;

System architecture

- Options for admins to deploy test files locally.
- More searching fields in the results look-up area of the dashboard.
- Additional charting options and export of results feature.
- Individual dashboard login for users to track their own test history and results, training recommendations and coaching notes.
- Improved content management tools, for accounts with large question libraries.
Additional benchmark data and statistics for OTS (off-the-shelf) tests.

We're currently working on the following test modules, which will be going live between June and September;

Test Libraries

- MicroStation v8i 2D for occasional users
- AutoCAD 2D for occasional users
- MicroStation v8i 3D fundamentals
- Design Review
- Bentley View
- 3ds Max 2011 (4 modules; Fundamentals, Rendering, Modeling and Maps & Materials)
- Revit MEP (1st of 3 modules)
- Bentley GEOPAK (1st of 4 modules)
- Bentley InRoads (1st of 4 modules)
- Rhino
- Adobe Photoshop
- Google Sketchup
- Bentley ProjectWise
- Bentley Navigator (1st of 2 modules)
- Bentley Structural Modeler
- Civil 3D (1st of 4 modules)

Towards the end of the year, we'll be turning our attention to analysis tools, including; Navisworks Manage, Ecotect, GBS, Hevacomp and IES VE. Bentley RAM and Bentley Map are also coming up more in conversation, so we'll endeavour to squeeze those two in somewhere as well!

As with all technical wish-lists, it's a constant juggle of time and money. With respect to our content libraries, I fear we might have a job for life in attempting to deliver quality test material across such a broad product base! Our challenge is principally that of, 'do we go wide or deep?' i.e. do we attempt to add a wide number of test topics with relatively small question libraries, or do we offer less topics, with greater numbers of questions for each? In truth, there is no right or wrong answer, hence the perpetual juggling act. As in all things we do, the answer ultimately lies with our customers; 'What do they want us to do?', is the question which shapes our ultimate roadmap.

As anyone who develops software or learning material will attest, the job is never truly finished. There will always be a new product update, or a change in technology, or a change in industry direction, or a change in vendor strategy, or yet another acquisition, the list goes on.

Whilst we're pleased with the progress we've made in the arena of CAD, BIM & engineering software skills testing, over the past 7 years or so, I sometimes feel that we're just getting started! The industry has changed rapidly over the past couple of years, impacted in considerable fashion by the twin effects of BIM adoption and economic hardship. And so, inevitably, the technology roadmap for AEC firms has been impacted.

As always, if you have any suggestions, ideas, requests for test content or general feedback, we'd love to hear them. Our own roadmap is, after all, in your hands! :)


Wednesday, 5 May 2010

KnowledgeSmart sponsors Be Together

KnowledgeSmart have signed as Associate sponsors at the Bentley Systems' user conference, Be Together, which takes place in Philadelphia next month.

This is a new format for Bentley, with multiple session tracks, covering a variety of AEC and Infrastructure topics; www.bentley.com/en-US/Community/BE+conference/Agenda.htm.

Sponsors find themselves in new territory this year, in the form of a series of exhibitor 'Zones', each focusing on a different area of technology. These are; Modeling & analysis, Visualization, Intelligent Deliverables, CAD Management, Technology Futures and Knowledge Resources. All zones are linked by the Collaboration zone.

KnowledgeSmart will be featuring our new online skills testing tools for CAD, BIM and Engineering software, situated in the Knowledge Resources zone.

In particular, we'll be presenting tools for skills testing across the following Bentley Systems' products; MicroStation 2D fundamentals, MicroStation for occasional users, MicroStation 3D fundamentals and Bentley Architecture v8i fundamentals, plus a preview of our new Bentley InRoads and Bentley GEOPAK assessments, which go live in the Fall.

Come and say hello to the team, if you're planning to attend. We might even treat you to a Philly Cheesesteak!


Thursday, 15 April 2010

KA Connect - a new kind of Tribe

Just back from an enjoyable trip to Chicago, where we sponsored and presented at the inaugural KA Connect, knowledge management conference.

Thanks to Chris P and the team at Knowledge Architecture for hosting what was the most enjoyable and useful conference I've been to in a long time.

What set KA Connect apart from most events was the format. Taking Seth Godin's theme of creating a Tribe (http://www.sethgodin.com/) and combining it with Derek Sivers' 'Shirtless Dancing Guy' discussion on TED.com about how to start a movement;(http://www.ted.com/talks/derek_sivers_how_to_start_a_movement.html?awesm=on.ted.com_8EPn).

The event, over 2 days, zipped along at a fair old pace. With 36 presenters, covering a range of KM angles, there was a smorgasbord of good ideas.

Particular highlights for me were;

- John Moebes from Crate & Barrel, discussing a new process for design-bid-build. This is the future of construction clients.

- Justin Quimby from Quimby Heavy Industries, with insights on how the AEC industry can learn lessons from the gaming sector.

- Michael Kilkelly from Gehry Partners, showcasing the way model-making impacts on their high-end designs (plus a bit of AutoCAD :) )

- Darren Rizza from Satellier, on why it's OK to whack a sacred cow with a stick once in a while!

- Pete Walsham from Axomic, comparing most architects to peasants (from a digital asset management pov).

- John Soter from ZweigWhite, with some parables about monkeys.

- Laurie Dreyer from Anshen + Allen, who brought the house down with her 'dinosaur lessons' and hilarious take on life from a HR & Learning professional's perspective.

Chicago was a perfect venue for this event. The Architecture is breathtaking, the weather was surprisingly temperate and the evenings were a blast (thanks in part to some rather stunning Russion Imperial stout!).

Next year's KA Connect event is already being lined up to take place in San Francisco. Keep an eye out on http://www.ka-connect.com/ for news of what's happening next. This is the beginning of a Tribe which is going to bring tremendous value to the AEC sector..


Wednesday, 14 April 2010

And the winner is...

I'd like to thank my parents, my wife, my kids, my agent, my fans...

KnowledgeSmart has been nominated for an award at this year's SWBE Awards, organised in partnership with Constructing Excellence. Here's a bit more background: http://www.buildsw.org.uk/awards.

There are 9 categories. We've been shortlisted down to the final 3 in the 'Leadership and People Development' category. They're looking for a company that 'clearly defines improvements resulting from leadership and training interventions... entries should show respect and win respect from the workforce and the community through workforce development, skills and training, benchmarking, equality and diversity... and improvements to the working environment.'

The awards dinner is in Bristol in June. We're shortlisted alongside two much larger organisations, in Bath & North East Somerset Council and Wilmott Dixon Construction. However, it's not the winning, darlings, it's being recognised for ones art, right? :)

Will keep you posted of the outcome...now where did I put my dress shoes...?


Thursday, 25 February 2010

KnowledgeSmart sponsors KA Connect Conference

Our friends at Knowledge Architecture are organising and hosting the first knowledge management conference for the AEC industry - KA Connect - and KnowledgeSmart are proud to be sponsoring the event.

This is a fascinating conference, with a superb lineup of leading AEC figures presenting on a range of topics. With a general theme of, 'Short Talks, Big Ideas', this promises to be one of the key events of our calendar year. With influences like TED (http://www.ted.com/) and marketing guru Seth Godin (http://www.sethgodin.com/), KA-Connect promises to be breath of fresh air.

A format tailor-made to stimulate fresh ideas and original thinking, boasting a blend of blue sky talks with Pecha Kucha (20 slides, 20 seconds per slide), expert panel discussions, video and breakout groups.

The speaker lineup is diverse, sharing emerging trends, new ideas, best practice and lessons learned for building knowledge-driven firms. The goal is to empower leaders in AEC firms to engage in, 'the process of systematically creating, capturing and sharing knowledge assets to leverage (yourself and) your organization'.

The conference organiser is Knowledge Architecture (http://www.knowledge-architecture.com/), a knowledge management and information systems consultancy, based in San Francisco.

The show takes place in Chicago on April 8 & 9, at theWit Hotel. (http://www.ka-connect.com/). Maybe we'll see you there.


Wednesday, 24 February 2010

KnowledgeSmart sponsors Bentley Winter Conference

KnowledgeSmart are pleased to once again be sponsoring the Bentley Community UKI Winter Workshop, showcasing our new range of online skills assessment tools.

A full agenda, with an emphasis on the impact of BIM, should make for an engaging and informative conference.

Here's the event itinerary: www.bentleyuser.org/events/BCUKI_W2010Agenda.pdf.

Come and visit our table if you're planning to attend.


KnowledgeSmart goes live

Well, we rolled out the new tools last week and so far, so good! It's a new direction for us in terms of the core approach to skills testing. The new web based platform allows greater flexibility in the way firms can deploy test modules and capture knowledge across a wide range of topics.

We're leaving a beta label on the system for a few more weeks; it's fully operational, but we welcome user feedback on any issues, bugs, missing data sets, or indeed anything that looks like it doesn't belong! Our next priorities are to add some more functionality to the admin dashboard; this includes some content management tools (for when question libraries get to a certain size), some file management tools (for marrying data sets to test questions in a variety of ways) and some general usability items, which have been suggested during pre-launch testing.

In addition, we have a rolling programme of new test material creation. Currently under development include the following libraries; Rhino for Architecture, AutoCAD & MicroStation for occasional users, Bentley Structural Modeller, MicroStation 3D fundamentals, Vectorworks, AEC Revit Standards, Ecotect fundamentals, Bentley InRoads and Civil 3D. These will all be making an appearance over the next few months.

KnowledgeSmart Enterprise customers will automatically see new library modules appear in their dashboard, as soon as they go live. The next challenge is to pull the live test results data into an online skills matrix tool, which administrators can use to display a range of useful metrics. Our intention is for this tool to dovetail into existing intranet environments, feeding into training and project resourcing activity, fully searchable, helping firms to ID the right people, with the right skills, at the right time.

Part of the challenge the new tools present is the variety of options available to admins when deciding just what test types to present to users. In the past, we have provided tests in any colour the customer could choose (so long as they chose black!). Now they really do have choice - and lots of it! The new 'Build Your Own' (BYO) feature enables firms to create their own test material from scratch, using an intuitive online authoring wizard to guide them through the process. AEC firms can capture knowledge across multiple topics, whether that be technical software ability, understanding and adherence to in-house standards, discipline specific knowledge, project-specific information, or indeed anything at all!

On the subject of Standards, we have partnered with the team at Evolve Consultancy to create test content for AEC (UK) CAD Standards for Layers/Levels. Next week, we'll be meeting the team at the US National CAD Standards committee to discuss a skills test for US CAD Standards.
We're running a series of web meetings over the next few weeks, to demonstrate the new system and tools on offer. To book a session, email info@knowledgesmart.net with 'Webinar' in the subject line and we'll schedule you in.

We have an ambitious programme of marketing and promotional activities planned between now and the Summer, both at home and in the US. We're sponsoring the Bentley Community UKI Winter Workshop in Oxford this month (http://www.bentleyuser.org/). We're presenting at the first WAN HR Forum in London on March 18th. And soon after that, we're heading off to Chicago, where we are sponsors at the inaugural KA-Connect knowledge management conference (http://www.ka-connect.com/).

Hope to see you online or in person in the coming weeks!


Monday, 25 January 2010

10 tips for writing your own test content

Here are ten tips for writing assessment questions. Thanks to our authors for their feedback, in particular Nigel Davies and Paul Aubin.

1. Aim to assess skills not knowledge. Anyone can look up which tool to use in a help file, or on the internet.

2. Plan your assessment out properly first. Before you even start thinking about questions, be sure you know exactly what skills you are trying to assess and the structure of the modules. It’s no good just uploading a whole series of questions – you need to think about whether anyone could sit the assessment and miss out on a critical area altogether.

3. Write the whole assessment first in Word or Excel, or some other package where you can organise the questions effectively. Make sure you include a reference number or some other way of identifying each question later (and any associated files) the full question wording, all options and answers, and the type of question. This will help you when preparing the online content. (However, also see tip 10 below..)

4. Name your files the same as the question reference to which they relate. It will help track them later.

5. Look for a balance of question types, and vary the length of question to keep the candidates from growing restless or bored. Also use plenty of images to make the test experience interesting from a user perspective.

6. Try to avoid lengthy text-based answers as candidates will need to provide exact responses to obtain full marks. Where text fields are used, try to keep the answers to numbers or single words. When asking for numerical input, be clear to define whether you expect the units to be entered, and how many decimal places the answer should show.

7. Questions where the candidate has to put a list of instructions in the correct order are an effective way of testing their ability to use a tool properly. If they don’t know the answer when sitting the test, the question can be very helpful in increasing their abilities. Do not overuse this question type as they can become very lengthy.

8. One of the main strengths of KnowledgeSmart is the ability to assign files to a question. Avoid using multiple-choice, knowledge only questions wherever possible and instead consider how to assess the same skill by editing or interrogating a file. For example, instead of asking “Which variable controls the linetype scaling in AutoCAD?” consider asking “To what value is the linetype scaling factor set in the file?”.

9. Make sure your questions are completely unambiguous. It is a good idea to get a colleague to sit an initial draft of your questions before you upload them to test your wording. You’ll be surprised how many different ways a question you thought was clear can be misinterpreted!

10. Copy and paste all text into a text editor before copying into your online questions. It avoids any messy formatting being passed over from Word or Excel.

As we get up and running, and doubtless receive many more good ideas from our customers on helpful aspects of question authoring, we'll continue to post the best ones on the blog.

T-minus 8 days and counting..


BIM Report from McGraw Hill

As more and more AEC firms embrace BIM in earnest, it helps to keep abreast of the latest news on the subject. One of the best annual updates is the SmartMarket Report from McGraw Hill Construction.

This year's report focuses on the Business case for BIM. Click here for a PDF copy:



Recession - a good time for listening!

So, the powers that be have decided that we're more or less out of recession now, have they? Doesn't really feel much different yet, truth be told, but it's fair to say that things are starting to gain momentum again. I was reading a discussion thread on the IoD LinkedIn forum last week about when things get 'back to normal'. Trouble is, we haven't defined 'normal' yet! I posted on the blog at the end of 2008 that we were planning for recovery to arrive around Q3 2010 - and nothing I've seen since has caused me to believe that that wasn't a reasonably accurate read on events.

Whilst it's still too early to start looking over our shoulder at the past 18 months of economic turmoil and draw hard conclusions, the simple psychological impact of seeing the new year swing around has definitely helped in many quarters. 2009 can be consigned as another 'Annus Horribilis' in the history books!

However, whilst certainly the financial repercussions of the recession were - and remain - a challenge, there was also a different side to the coin, at least as far as customer communication goes.

We found that many AEC firms in 2009, for the first time in a long time, had a bit of breathing space, at least from an IT admin point of view. Some much needed housekeeping was taken care of, coupled with some forward planning for the post-recession rebuilding phase.

During the course of many, many conversations last year, we were able to identify a number of service improvement ideas, by asking the question; 'What don't you like about our service?'. We frankly know what we do rather well, but it's the things at which we don't excel which are of the most interest. The niggles in our service. The bits our customers would like us to tweak, change, or even tear up and start again.

As a result, we went back to the (proverbial) drawing board (we are in the CAD & BIM world, after all! :) ) and started with a fresh look at how we could deliver future test modules for a host of design and engineering software apps (108 at the last count!).

And so KnowledgeSmart was born, as part of this process. Next month, we roll out the first phase of our new web-based tools, which will give AEC businesses even more insight into the skills base and competency of their teams. All existing 'Premier' customers will be upgraded to our new 'Enterprise' subscription, which offers a wide range of choices for knowledge capture and dissemination.

The months to follow will see a phased release of some new tools to help with the analysis and sharing of the newly captured performance data. Testing is already underway on a skills matrix tool, which will give HR and Training leaders a valuable insight into more focused learning, not to mention smoother resourcing of projects.

So, whilst the recession was (and still is) a hurdle on the overall journey to running an enjoyable and profitable enterprise, we are also, in a funny way, grateful that it came along when it did. It taught us how to listen again.


Wednesday, 20 January 2010

CAD & BIM - State of the Nation(s) article

Happy New Year. Towards the end of last year we held a series of (taped) discussions with the guys at Evolve (http://www.evolve-consultancy.com) and Robert Green (http://www.cad-manager.com) , which culminated in a published article looking at the big picture of CAD training, CAD management, BIM, and a whole lot more.

Credit to Karen for her amazing editing skills and to Nigel for making a fair dent in his red wine collection, as the evenings wore on! :)

The results were published in a variety of places, but most recently, here: http://www.itshowcase.co.uk/News/2009-05-12/Is-your-business-smart-enough-An-interview-on-CAD-usage/.

So, we're finally out of 2009 (hooray!). 2010 is looking much more encouraging, with the economy starting to drag itself off the floor. 'Bout time!

February sees the rollout of the new KnowledgeSmart web-based testing tools. If you'd like a demo, drop a line to info@knowledgesmart.net, with 'DEMO' in the subject line, and we'll pencil you in.

Plenty more to follow over the coming weeks..