|
T |
his document is written for individuals who are involved in writing examination questions. The techniques suggested here can be applied to the creation of questions at any level (from access to post-graduate) and any subject – although the examples relate to Computing and Information Systems. The document has two objectives:
v update setters on the revision to Bloom’s taxonomy
v suggest way of applying the revised taxonomy to the production of exam questions.
It is primarily intended for individuals who set questions (‘setters’) for the Scottish Qualifications Authority, but may also be of interest to individuals who simply require a brief introduction to the revised taxonomy for personal purposes.
The document borrows from the textbook on which it is based – A Taxonomy for Learning, Teaching and Assessment (ISBN 0-8013-1903-X) – which was written by Lorin W Anderson and David R Krathwohl. It is available from most book stores and online. Readers of this document are highly recommended to purchase the book for a much fuller treatment of each topic. The book focuses on teaching and learning rather than assessment, but this document concentrates on assessment – and the use of the revised taxonomy to produce exam questions.
The
original textbook – Taxonomy of
Educational Objectives: Book 1
Cognitive Domain by Benjamin Bloom – is still in print (in the
Scottish Qualification
Authority
Version 1.1, February 2002
1. Background
3. Constructing question papers
4. Summary
The original Bloom’s Taxonomy was published in 1956. It was designed to provide a common framework for individuals who were responsible for constructing assessments – particularly written examinations within universities. The original framework provided a means of classifying academic ability into one of six categories:
|
Figure
1 -
Bloom's original Taxonomy |
Bloom further sub-divided these categories. For example, knowledge was broken down into: knowledge of specific facts, knowledge of ways and means of dealing with specifics, and knowledge of universals and abstractions. Even some of the sub-categories were sub-divided. For example, knowledge of specifics was broken down into: knowledge of terminology, and knowledge of specific facts. In total, Bloom proposed 21 categories of cognitive ability, ranging from ‘knowledge of terminology’ to ‘judgements in terms of external criteria’.
The taxonomy is hierarchical in that each category builds on the one underneath. So, for example, analysis depends on comprehension which in turn depends on underpinning knowledge (see figure above). In other words, you can’t analyse something until you comprehend it, and you can’t comprehend something until you know about it.
Bloom’s Taxonomy has been the basis of test paper construction for 50 years. A full treatment of his Taxonomy can be found here.
Bloom’s Taxonomy has remained unchanged since 1956. However, a revision of his taxonomy was published in 2001 entitled A Taxonomy For Learning, Teaching & Assessment written by Lorin Anderson and David Krathwohl. The revised taxonomy focuses on teaching and learning, rather than assessment, but can be applied to the construction of test items. The core of the revised taxonomy is a taxonomy table.
The taxonomy table is a two-dimensional system for classifying knowledge and cognitive skills (see Table 1). One dimension classifies knowledge and the other classifies cognition.
|
Knowledge dimension |
Cognitive dimension | |||||
|
1. Remember |
2. Understand |
3. Apply |
4. Analyse |
5. Evaluate |
6. Create | |
|
A. Factual knowledge |
|
|
|
|
|
|
|
B. Conceptual knowledge |
|
|
|
|
|
|
|
C. Procedural knowledge |
|
|
|
|
|
|
|
D. |
|
|
|
|
|
|
Table
1 -
Taxonomy table
Taken together, the two dimensions (or classifications) allow
you to categorise learning – or assessment. For example, a specific question
might assess the recall of factual knowledge – this test item would be placed in
cell 1A; another question might assess the evaluation of procedural knowledge –
which would be placed in cell 5C.
The table has a sense of complexity and hierarchy. Complexity can be measured by how far the test item is placed from the top left-hand corner. In the above example, the recall of factual knowledge (cell 1A) is less complex than the evaluation of procedural knowledge (cell 5C). Of course, this is a crude generalisation and assumes that both questions relate to the same knowledge domain – the recall of factual knowledge relating to nuclear fusion is probably more complex than the application of procedural knowledge of multiplication tables.
The table also has a sense of hierarchy. Factual knowledge underpins conceptual knowledge which underpins procedural knowledge which underpins meta knowledge. This is illustrated in Figure 2.

Figure
2
- Knowledge dimension as a hierarchy
The above diagram provides an example of the hierarchy of knowledge in the context of computer programming. Knowledge of a programming language’s syntax and semantics (factual knowledge) is required for a student to understand program constructs (conceptual knowledge) which is needed before a student can write a complete program (procedural knowledge) and thereby know his/her limitations as a programmer (meta knowledge).
Each of the dimensions in the table will now be explained.
The knowledge dimension has four categories:
Each category can be further broken down. For example, factual knowledge has two sub-categories – knowledge of terminology and knowledge of specific details (see Table 2).
|
Category |
Examples |
|
Factual
knowledge: The basic elements
candidates must know to be acquainted with a discipline. | |
|
Knowledge of
terminology. |
Technical vocabulary,
knowledge of symbols, knowledge of measures, knowledge of acronyms and
abbreviations. |
|
Knowledge of specific
details. |
History of the
Internet, descriptions of features of specific WP program, sources of
information, knowledge of a programming language. |
|
Conceptual
knowledge: The relationships
between components or systems. | |
|
Knowledge of
classifications. |
Types of programming
language, types of computer system. |
|
Knowledge of
systems. |
Basic structure of a
computer, ISO reference model, knowledge of a specific operating
system. |
|
Knowledge of
principles and generalisations. |
Stored program
concept, programming techniques, |
|
Knowledge of theories,
models and structures. |
Program testing
strategies, SSADM, program design, JSP. |
|
Procedural
knowledge: How to do something,
methods of research, criteria for using methods and
techniques. | |
|
Knowledge of
subject-specific skills and algorithms. |
Knowledge of how to
use an application package, knowledge of how to write a computer program,
sorting and searching algorithms. |
|
Knowledge of
subject-specific techniques and methods. |
Top-down program
design, normalisation, structured programming, systematic
fault-finding. |
|
Knowledge of criteria
for using procedures. |
Knowledge of when to
use a specific algorithm, knowledge of criteria for selecting a type of
applications package. |
|
| |
|
Strategic
knowledge. |
Knowledge of learning
strategies, knowledge of the use of heuristics, knowledge of mind
mapping. |
|
Knowledge about
cognitive tasks. |
Knowledge about the
relative complexity of different procedures, exam technique. |
|
Self
knowledge. |
Awareness of personal
strengths and weaknesses, awareness of extent of own knowledge about a
particular topic. |
Table
2
- Knowledge dimension
Most types of knowledge can be classified under one of these
categories (or sub-categories). We will look at how these classifications can be
used to assist with the production of exam questions in the next
section.
The cognitive dimension has six categories:
|
Keywords |
Definitions and
examples | |
|
Remember:
Retrieve relevant
knowledge from memory. | ||
|
Recognising |
Identify Match |
Matching descriptions
with visual representations. For example, identifying the components of a
microcomputer system. |
|
Recalling |
State Define Describe |
Retrieving knowledge
from long-term memory. For example, stating four characteristics of
information or defining the meaning of an acronym. |
|
Understand: Construct meaning
from instructions. | ||
|
Interpreting |
Estimate Convert Translate |
Changing from one form
of representation to another. For example, interpreting an advert for
computer hardware or converting one unit or measurement to another (e.g.
bytes to megabytes). |
|
Exemplifying |
Give
examples Illustrate Demonstrate Show |
Finding a specific
example of a concept or principle. For example, relating a specific
package’s features to the generic features of a type of
package. |
|
Classifying |
Arrange Classify Categorise Sort |
Assigning something to
a specific class or category or re-ordering a list. For example,
classifying specific software products by software type (freeware,
shareware, commercial etc.). |
|
Summarising |
Summarise Review |
Abstracting a general
theme or major points. For example, writing a short review of a specific
software product. |
|
Inferring |
Predict Deduce Extrapolate |
Drawing a conclusion
from presented information. For example, given a number of specific cases,
produce rules using an expert system. |
|
Comparing |
Compare Contrast Evaluate Map |
Detecting
correspondences between ideas and/or objects. For example, contrast two
programming languages in terms of their data structure
facilities. |
|
Explaining |
Give reasons
Explain Justify |
Constructing a
cause-and-effect model of a system. For example, give reasons for the
emergence of the Internet. |
|
Apply: Carry out or use a
procedure in a given situation. | ||
|
Executing |
Carry out Perform Complete |
Applying a procedure
to a familiar task. For example, carrying out the procedure to install an
applications package on a PC. |
|
Implementing |
Use Apply Implement |
Applying a procedure
to an unfamiliar task. For example, using applications software to solve a
given problem or writing a piece of code to perform a specific
task. |
|
Analyse: Break material into
its constituent parts and determine how these parts relate to one another
and to the overall structure or purpose. | ||
|
Differentiating |
Select Choose Discriminate |
Identifying
similarities and differences, and important and unimportant attributes of
objects or systems. For example, choosing a computer system (from two or
more provided) for a specific task, or selecting a specific data structure
to model a given problem. |
|
Organising |
Arrange Find Structure Organise |
Determining how
elements fit together within a system. For example, constructing a
flowchart to represent a given problem description or producing a data
flow diagram to model a supplied case study. |
|
Attributing |
Assign Attribute Deconstruct |
Determine a point of
view, bias, values or intent. For example, determining the point of view
of an author of an essay on the social implications of IT. |
|
Evaluate: Make judgements
based on criteria and standards. | ||
|
Checking |
Check Verify Confirm Monitor Test |
Determining
inconsistencies or fallacies within a process or product. For example, dry
running a given algorithm to check its correctness or testing a program to
locate errors. |