One of the first assignments of my Masters in Business Intelligence was writing a 500 word essay on Data v Information v Knowledge (an exercise in concise writing). With the kind permission of the course coordinator, I’m happy to share with a broader audience.  The course runs as a part-time (2yr) or full-time(1yr) course, and I’d highly recommend it to anyone in the BI field wanting to get a technology agnostic view of all elements of BI.

Data v Information v Knowledge - 19th February 2010

.-- .... . .-. .   .. ...   - .... .   .-- .. ... -.. --- --   .-- .   .... .- ...- .   .-.. --- ... -   .. -.   -.- -. --- .-- .-.. . -.. --….--..

.-- .... . .-. .   .. ...   - .... .   -.- -. --- .-- .-.. . -.. --. .   .-- .   .... .- ...- .   .-.. --- ... -   .. -.   .. -. ..-. --- .-. -- .- - .. --- -. ..--..

Does this mean anything to you?  Some of you may identify these series of dots and dashes as a representation of Morse code.

Some of you may even understand it:

“Where is the wisdom we have lost in knowledge?

Where is the knowledge we have lost in information?”

And some of you may be aware that it is from T.S.Eliot’s ‘Choruses from “The Rock”’.[[1]]

This demonstrates that, as individuals, we can each form a different view of the same material.  This material may be referred to as data, information, or knowledge.  It all depends on the context.

Dots and dashes are most certainly data, if we assume “Data is recorded, captured or stored, symbols and signal readings”[2].  The dots and dashes represent the length of beep to be sounded.  They are captured in a written format so that we can recreate the sounds in sequence.

Observing the dots and dashes, or hearing the sounds, is the point at which the data becomes information.  Whether it is useful information is dependent on the observer.  Someone who understands Morse code can apply their prior knowledge, to get the meaning from the data.

We see that prior knowledge can be applied to identify the source as being T.S.Eliot’s poem.  “Conclusions spring from data when this information is combined with prior knowledge.”[3]

The extract from Eliot’s poem, is believed to be the earliest reference to, what today is known as, the DIKW (Data, Information, Knowledge, Wisdom) hierarchy[[4]][[5]].

There are a variety of flavours to the hierarchy[[6]][[7]][[8]], each trying to define the different levels, and how we transition from one level to the next.

When making the transition from data to knowledge, we should be concerned about the quality of our data.  Data quality affects the information we retrieve.  Misinterpretation of information proliferates into our knowledge.  Our knowledge is then tainted by inaccurate information.  Ultimately this will affect the decisions we make.

One form of misinformation is taking a small sampling of information, observing certain behaviours, and treating these as fact.  This is similar to how some forms of superstition are created.[9]

In the marketing, advertising and corporate report[10] world, authors and designers tend to present the information they want us to focus on in such a way that it is more prominent or positive[11].  This is a form of information sampling.  Tufte demonstrates how newspapers reinforce the information that they want you to take from the article.[12]

In these cases, someone who is knowledgeable is aware of the nuances of how this information is presented, and can use their experience to gain an accurate, unbiased, representation of the world by looking at all the available information, and applying any prior knowledge they have.

In a BI context we need to factor all these things into our system.  Populating our data warehouse with poor data means the information is not accurate.  Even if we have accurate data, misrepresenting it can lead to people drawing the wrong conclusions.

Bibliography

Burns, R “The Naked Trader: How anyone can make money trading shares”, Harriman House, Cambridge 2005

Cleveland, WS, “Visualizing Data” Hobart Press, Summit, New Jersey, 1993

Levitt, SD & Dubner, SJ “Freakonomics” Penguin Books 2006 (first published by William Morrow Ltd 2005)

Huff, D “How to Lie with Statistics” Penguin Books, 1991 (first published 1954 by Victor Gollancz)

Sachs, J “Plato’s Theaetetus : Translated with Introduction and Notes”, Focus Publishing, R.Pullins Co., Newburyport, MA 2004

Tufte, ER “Envisioning Information”, Graphics Press, Cheshire, Connecticut, 1990

[1] Eliot, TS “Choruses from ‘The Rock’”, Faber & Faber, 1934

[2] Liew, A, “Understanding Data, Information, Knowledge And Their Inter-Relationships”, Journal of Knowledge Managment Practice Vol. 8 No. 2, June 2007

[3] Cleveland, WS, “Visualizing Data” Hobart Press, Summit, New Jersey, 1993

[4] Eliot’s poem did not refer to the DIKW hierarchy by that name, and in fact, did not include data as part of the hierarchy.  This is later attributed to Yi-Fu Tuan and Daniel Bell’s version of the hierarchy, referenced by H.Cleveland.

[5] Cleveland, H, “Information as Resource”, The Futurist, December 1982 p34-39

[6] Ackoff, RL, “From Data to Wisdom”, Journal of Applied Systems Analysis, Volume 16, 1989 p3-9 – Ackoff’s model introduces the level of Understanding.

[7] Zeleny, M “Management Support Systems: Towards Integrated Knowledge Management”, Human Systems Management, 7, (1987) 1 pp. 59-70 – Zeleny’s model introduces enlightenment at the highest level.

[8] Rowley, J “The wisdom hierarchy: representations of the DIKW hierarchy”, Journal of Information Science 2007 33; pp 163 -180 – Rowley ably defines the components and relationships between the levels in the data, information, knowledge hierarchies, and compares the two earlier models of Arkoff and Zeleny.

[9] Skinner, BF (1948) “’Superstition’ in the pigeon”, Journal of Experimental Psychology, 38(2), pp 168-172

[10] Burns talks about a traffic light system, his way of visualising all the jargon that they use in corporate reports, to provide visual indicators of the true nature of the report - Burns, Robbie “The Naked Trader: How anyone can make money trading shares”, Harriman House, Cambridge 2005 p102

[11] “Experts” - from criminologists to real-estate agents – use their informational advantage to serve their own agenda.  Levitt, SD & Dubner, SJ, “Freakonomics”, Pengiun Books, 2006 p13

[12] Tufte, ER “Envisioning Information”, Graphics Press, Cheshire, Connecticut, 1990 p30