christian hyland's blog

Having heard the maxim "what gets measured, gets done" promoted heavily on projects that failed it was interesting to read this piece on the Performance Improvement blog.

It's also been been my experience that metrics alone are ineffective. Organisations can certainly benefit from a process of gathering and reporting information. However, it requires good leadership to translate metrics into improved operational performance. This type of leadership has the ability to assess the impact of metrics on group behaviour as well as the skill to draw conclusions from the information and act.

Members of a group will often respond to the stimulus of metrics through conforming behaviours. Sales targets are a good example. If an organisation measure sales by individuals each month against a target and there is a clear understanding that the individuals are accountable for the results, then most individuals will seek to achieve the target. In this simple scenario the maxim holds.

However, what happens if an individual consistently does not meet their target. The group will question the norms of behaviour. They will watch for a response. If the response is slow, or not forthcoming members of the group are likely to question the value of the measure as well the effectiveness of the leader(s). The efficacy of the process of collecting information is undermined as is the credibility of the leadership.

This scenario can be extended to the process of measuring the performance of product lines, manufacturing plants, marketing programs and the overall business. 

The optimal use of metrics is in a structure of re-enforcing blocks that combine to form a pyramid. In this structure, each function is measured against its contribution to the organisation's purpose. For most enterprises the purpose is profitability - this being the ultimate measure of the surplus created through the effective allocation of resources. This measure is therefore at the top of the pyramid whereas functional activities can be arrayed in any sequence as long as they support the organisation's process for creating value.

The range, scope and aggregation of metrics in the building blocks depends on the needs of the organisation. Below are some of the questions that ought to be considered in the design process:

• What are the criteria for the success of this function | program | business?
• How does this contribute to the organisation's purpose?
• How will the attributes of success be measured?
• What information will be shared? How? When?
• What effect will this reporting process have on behavioural norms? How will we know?
• Who will be accountable?

For this metrics structure to work, an organisation's leadership must demonstrate a commitment to both valid data and taking action on the results. This can be achieved through a combination of peer reviews, staff selection, performance evaluation and staff development. Many modern human capital management programs effectively link metrics to domains of expertise and spans of authority. However, for any of these steps work, leaders themselves need to be measured by their ability to act.

IT development of any kind, and in particular “business intelligence” systems, need to consider how humans interact with systems. The attitude that good design is 'cosmetics' and does not rank with speed and accuracy is precambrian thinking.

For those who say it is only the engineering that counts. Reflect on the Model T Ford. Henry Ford built a fast (for its time), reliable and affordable car for the masses – the Model T. It was a great success until General Motors decided to offer similar cars in colours other than black.

The same goes for customers who invest hundreds of thousand dollars on Business Intelligence systems. Most do not see the artful data transformations, the clever code or the multitude of man hours that go into making 1 + 1 = 2 in a server-based analytics and reporting system. They see a screen with information and judge the system on how good it is based on whether it is intuitive, enjoyable and useful.

Publishing spreadsheets to the Web and 'prettying up' the data is not a recipe for success in information system design. It certainly will not lead to the kind of appeal that is required to deliver Business Intelligence to the masses.

This is because poorly presented content lacks credibility. If users did not care about how they interact with technology DOS would be the system of choice, Wang computers would be on everyone's desk and Bing would be the #1 search engine.

The stand out successes in the Web age combine good engineering with great design. I give you Apple, Google and Facebook. I can't think of one global brand that succeeded with “ours works better and we're cheaper”.

Apple's emphasis on industrial design and Google's focus on the user experience are part of their organisational DNA. Some have even accused Steven Jobs of putting design ahead of technology. Tut tut say the techies – look at the share price says I. For its part Google studies every move users make with their search engine. User interaction is a science at Google.

Both Apple and Google get that it is not the tool but how appealing it is to use. This is a far sighted approach within information technology. Their leadership embedded and nurtured great design in their organisational DNA. More of this thinking is needed in BI.

I am often asked how commercial open source software works. A lot of folks, myself included until a few months ago, hold a somewhat outdated view of open source software as 'cool' but a bit user unfriendly and a little risky. Other people that I've met can't get their heads around the idea of having to pay for a version of something that they can get for free. This blurb is what I've come up with to explain how the new generation of commercial open source software products work.

An open standards platform accelerates the development of software applications and reduces the cost of software ownership for consumers. It is an example of a virtuous circle business model made possible by the World Wide Web. With an open source software program, a community of tech-savvy users contributes improvements to a freely available version of the application. Because the number of user-developers on open source projects is often in the tens of thousands, they form a global research and development group that can dwarf the programming teams of even the largest traditional software companies like Microsoft.

The result is an acceleration in the rate of development without the high R & D costs incurred by traditional software companies who closely guard their source code.

With the source code available to all (hence 'open'), open source software development takes place under the scrutiny of a peer review process. It is a matter of pride for passionate aficionados to contribute good code and correct the mistakes of others. This naturally maintains the quality of the underlying code. Furthermore, these open source development projects are often led by senior programmers who operate like orchestra conductors vetting and testing user contributions from the developer/community to maintain the overall integrity of the source code.

In commercial open source projects, these orchestra conductors are employed by a software company. They also augment 'community' product development by employing in-house developers to refine the end-user experience for commercial customers. Commercial open source companies create two versions of the software: a “freely available but unsupported” version for individual users; and a fully supported “enterprise grade” version designed for commercial customers.

Enterprise versions makes sense for commercial customers who want the cost benefits of open source software development without the business risk. By offering a warranty, customer support and a range of enhancements commercial open source vendors make it easier for large organisations to switch from traditional software companies to commercial open source applications. Even with the enhanced services, commercial open source costs a fraction of the comparable traditional products.

Commercial open source companies have figured out that if they can match the traditional software companies like Microsoft, Oracle and SAP in warranties and customer service they can offer a compelling value proposition to enterprise users. Customers can reduce wasteful overheads and increase performance by improving how information is used in the organisation.

Milton Friedman said, “there is no such thing as a free lunch” but he did not say that you should have to pay for someone else's buffet.

Christian Hyland is an Information Management Consultant with BizCubed in Sydney, Australia. BizCubed is a Gold Partner of Pentaho (a commercial open source vendor).s

The brain isn’t a flawless piece of machinery. Although it is powerful and comes in an easy to carry container, it has it’s weaknesses. A field in psychology which studies these errors, known as biases. Although you can’t upgrade your mental hardware, noticing these biases can clue you into possible mistakes.

How Bias Hurts You

If you were in a canoe, you’d probably want to know about any holes in the boat before you start paddling. Biases can be holes in your reasoning abilities and they can impair your decision making.

Simply noticing these holes isn’t enough; a canoe will fill with water whether you are aware of a hole or not. But by being aware of the holes you can devise methods to patch them up. The entire domain of the scientific method has largely been an effort to overcome the natural inclination towards bias in reasoning.

Biases hurt you in a number of areas:

• Decision making. A number of biases can distort decision making. The confirmation bias can lead you to discount information that opposes existing theories. Anchoring can throw off negotiations by forcing you to sit around an arbitrary value.
• Problem solving. Biases can impede your creativity when solving problems. A framing bias can cause you to look at a problem too narrowly. And the illusion of control can cause you to overestimate the amount your actions influence results.
• Learning. Thinking errors also impact how you learn. The Von Restorff effect can cause you to overemphasize some information compared to the whole. Clustering illusions can also trick you into thinking you’ve learned more than you actually have.

Here are some common thinking errors:

1) Confirmation Bias

The confirmation bias is a tendency to seek information to prove, rather than disprove our theories. The problem arises because often, one piece of false evidence can completely invalidate the otherwise supporting factors.

Consider a study conducted by Peter Cathcart Wason. In the study, Wason showed participants a triplet of numbers (2, 4, 6) and asked them to guess the rule for which the pattern followed. From that, participants could offer test triplets to see if their rule held.

From this starting point, most participants picked specific rules such as “goes up by 2“ or “1x, 2x, 3x.” By only guessing triplets that fit their rule, they didn’t realize the actual rule was “any three ascending numbers.” A simple test triplet of “3, 15, 317“ would have invalidated their theories.

2) Hindsight Bias

Known more commonly under “hindsight is 20/20“ this bias causes people to see past results as appearing more probable than they did initially. This was demonstrated in a study by Paul Lazarsfeld in which he gave participants statements that seemed like common sense. In reality, the opposite of the statements was true.

3) Clustering Illusion

This is the tendency to see patterns where none actually exist. A study conducted by Thomas Gilovich, showed people were easily misled to think patterns existed in random sequences. Although this may be a necessary by product of our ability to detect patterns, it can create problems.

The clustering illusion can result in superstitions and falling for pseudoscience when patterns seem to emerge from entirely random events.

4) Recency Effect

The recency effect is the tendency to give more weight to recent data. Studies have shown participants can more easily remember information at the end of a list than from the middle. The existence of this bias makes it important to gather enough long-term data, so daily up’s and down’s don’t lead to bad decisions.

5) Anchoring Bias

Anchoring is a well-known problem with negotiations. The first person to state a number will usually force the other person to give a new number based on the first. Anchoring happens even when the number is completely random. In one study, participants spun a wheel that either pointed to 15 or 65. They were then asked the number of countries in Africa that belonged to the UN. Even though the number was arbitrary, answers tended to cluster around either 15 or 65.

6) Overconfidence Effect

And you were worried about having too little confidence? Studies have shown that people tend to grossly overestimate their abilities and characteristics from where they should. More than 80% of drivers place themselves in the top 30%.

One study asked participants to answer a difficult question with a range of values to which they were 95% certain the actual answer lay. Despite the fact there was no penalty for extreme uncertainty, less than half of the answers lay within the original margin.

7) Fundamental Attribution Error

Mistaking personality and character traits for differences caused by situations. A classic study demonstrating this had participants rate speakers who were speaking for or against Fidel Castro. Even if the participants were told the position of the speaker was determined by a coin toss, they rated the attitudes of the speaker as being closer to the side they were forced to speak on.

Studies have shown that it is difficult to out-think these cognitive biases. Even when participants in different studies were warned about bias beforehand, this had little impact on their ability to see past them.

What an understanding of biases can do is allow you to design decision making methods and procedures so that biases can be circumvented. Researchers use double-blind studies to prevent bias from contaminating results. Making adjustments to your decision making, problem solving and learning patterns you can try to reduce their effects.

Most successful innovations, according to Drucker, come from a conscious, purposeful search for innovation opportunities. He saw four areas of opportunity inside organizations: unexpected occurrences, incongruities, process needs, and industry and market changes. Opportunities also exist outside organizations in its social and intellectual environments: demographic changes, changes in perception, and new knowledge. Together these seven account for almost all opportunities for innovation.

Effective innovation is simple and focused. If it tries to do too many things, people will get confused and implementation will be compromised. He also states that innovation is hard, focused, purposeful work. It requires knowledge and focus, and often, requires ingenuity. The foundation of entrepreneurship is the practice of systematic innovation.

Peter Drucker was a university professor, writer, and business guru (Wikipedia). This article was adapted from Drucker’s book Innovation and Entrepreneurship: Practice and Principles, Harper and Row, 1985.

Citation

Drucker, P. (2002). The discipline of innovation. Harvard Business Review: 95-102