Using the revised Bloom’s Taxonomy
for the creation of examination questions

 

his document is written for individuals who are involved in writing examination questions. The techniques suggested here can be applied to the creation of questions at any level (from access to post-graduate) and any subject – although the examples relate to Computing and Information Systems. The document has two objectives:

 

v     update setters on the revision to Bloom’s taxonomy

v     suggest way of applying the revised taxonomy to the production of exam questions.

 

It is primarily intended for individuals who set questions (‘setters’) for the Scottish Qualifications Authority, but may also be of interest to individuals who simply require a brief introduction to the revised taxonomy for personal purposes.

 

Table of contents

1. Background

v     The original taxonomy

2. The revised taxonomy

v     The taxonomy table

Ø      The knowledge dimension

Ø      The cognitive dimension

3. Constructing question papers

v     Creating questions

v     Classifying questions

v     Creating question papers

v     Analysing question papers

v     Balancing question papers

4. Summary

1. Background

The original Bloom’s Taxonomy was published in 1956. It was designed to provide a common framework for individuals who were responsible for constructing assessments – particularly written examinations within universities. The original framework provided a means of classifying academic ability into one of six categories:

                                                   

Knowledge Comprehension Application Analysis Synthesis Evaluation.

Figure 1 - Bloom's original Taxonomy

 

Bloom further sub-divided these categories. For example, knowledge was broken down into: knowledge of specific facts, knowledge of ways and means of dealing with specifics, and knowledge of universals and abstractions. Even some of the sub-categories were sub-divided. For example, knowledge of specifics was broken down into: knowledge of terminology, and knowledge of specific facts. In total, Bloom proposed 21 categories of cognitive ability, ranging from ‘knowledge of terminology’ to ‘judgements in terms of external criteria’.

 

The taxonomy is hierarchical in that each category builds on the one underneath. So, for example, analysis depends on comprehension which in turn depends on underpinning knowledge (see figure above). In other words, you can’t analyse something until you comprehend it, and you can’t comprehend something until you know about it.

 

Bloom’s Taxonomy has been the basis of test paper construction for 50 years. A full treatment of his Taxonomy can be found here.

 

Return to table of contents

2. Revised Taxonomy

Bloom’s Taxonomy has remained unchanged since 1956. However, a revision of his taxonomy was published in 2001 entitled A Taxonomy For Learning, Teaching & Assessment written by Lorin Anderson and David Krathwohl. The revised taxonomy focuses on teaching and learning, rather than assessment, but can be applied to the construction of test items. The core of the revised taxonomy is a taxonomy table.

 

Return to table of contents

Taxonomy table

The taxonomy table is a two-dimensional system for classifying knowledge and cognitive skills (see Table 1). One dimension classifies knowledge and the other classifies cognition.

 

Knowledge dimension	Cognitive dimension
	1. Remember	2. Understand	3. Apply	4. Analyse	5. Evaluate	6. Create
A. Factual knowledge
B. Conceptual knowledge
C. Procedural knowledge
D. Meta knowledge

Table 1 - Taxonomy table

Taken together, the two dimensions (or classifications) allow you to categorise learning – or assessment. For example, a specific question might assess the recall of factual knowledge – this test item would be placed in cell 1A; another question might assess the evaluation of procedural knowledge – which would be placed in cell 5C.

 

The table has a sense of complexity and hierarchy. Complexity can be measured by how far the test item is placed from the top left-hand corner. In the above example, the recall of factual knowledge (cell 1A) is less complex than the evaluation of procedural knowledge (cell 5C). Of course, this is a crude generalisation and assumes that both questions relate to the same knowledge domain – the recall of factual knowledge relating to nuclear fusion is probably more complex than the application of procedural knowledge of multiplication tables.

 

The table also has a sense of hierarchy. Factual knowledge underpins conceptual knowledge which underpins procedural knowledge which underpins meta knowledge. This is illustrated in Figure 2.

 

Figure 2 - Knowledge dimension as a hierarchy

The above diagram provides an example of the hierarchy of knowledge in the context of computer programming. Knowledge of a programming language’s syntax and semantics (factual knowledge) is required for a student to understand program constructs (conceptual knowledge) which is needed before a student can write a complete program (procedural knowledge) and thereby know his/her limitations as a programmer (meta knowledge).

 

Each of the dimensions in the table will now be explained.

 

Return to table of contents

Knowledge dimension

The knowledge dimension has four categories:

 

1. factual knowledge
2. conceptual knowledge
3. procedural knowledge
4. meta knowledge.

 

Each category can be further broken down. For example, factual knowledge has two sub-categories – knowledge of terminology and knowledge of specific details (see Table 2).

 

Category	Examples
Factual knowledge: The basic elements candidates must know to be acquainted with a discipline.
Knowledge of terminology.	Technical vocabulary, knowledge of symbols, knowledge of measures, knowledge of acronyms and abbreviations.
Knowledge of specific details.	History of the Internet, descriptions of features of specific WP program, sources of information, knowledge of a programming language.
Conceptual knowledge: The relationships between components or systems.
Knowledge of classifications.	Types of programming language, types of computer system.
Knowledge of systems.	Basic structure of a computer, ISO reference model, knowledge of a specific operating system.
Knowledge of principles and generalisations.	Stored program concept, programming techniques, Moore’s Law.
Knowledge of theories, models and structures.	Program testing strategies, SSADM, program design, JSP.
Procedural knowledge: How to do something, methods of research, criteria for using methods and techniques.
Knowledge of subject-specific skills and algorithms.	Knowledge of how to use an application package, knowledge of how to write a computer program, sorting and searching algorithms.
Knowledge of subject-specific techniques and methods.	Top-down program design, normalisation, structured programming, systematic fault-finding.
Knowledge of criteria for using procedures.	Knowledge of when to use a specific algorithm, knowledge of criteria for selecting a type of applications package.
Meta knowledge: Knowledge of knowledge.
Strategic knowledge.	Knowledge of learning strategies, knowledge of the use of heuristics, knowledge of mind mapping.
Knowledge about cognitive tasks.	Knowledge about the relative complexity of different procedures, exam technique.
Self knowledge.	Awareness of personal strengths and weaknesses, awareness of extent of own knowledge about a particular topic.

Table 2 - Knowledge dimension

Most types of knowledge can be classified under one of these categories (or sub-categories). We will look at how these classifications can be used to assist with the production of exam questions in the next section.

 

Return to table of contents

Cognitive dimension

The cognitive dimension has six categories:

 

1. remembering
2. understanding
3. applying
4. analysing
5. evaluating
6. creating.

 

Cognitive ability	Keywords	Definitions and examples
Remember: Retrieve relevant knowledge from memory.
Recognising	Identify Match	Matching descriptions with visual representations. For example, identifying the components of a microcomputer system.
Recalling	State Define Describe	Retrieving knowledge from long-term memory. For example, stating four characteristics of information or defining the meaning of an acronym.
Understand: Construct meaning from instructions.
Interpreting	Estimate Convert Translate	Changing from one form of representation to another. For example, interpreting an advert for computer hardware or converting one unit or measurement to another (e.g. bytes to megabytes).
Exemplifying	Give examples Illustrate Demonstrate Show	Finding a specific example of a concept or principle. For example, relating a specific package’s features to the generic features of a type of package.
Classifying	Arrange Classify Categorise Sort	Assigning something to a specific class or category or re-ordering a list. For example, classifying specific software products by software type (freeware, shareware, commercial etc.).
Summarising	Summarise Review	Abstracting a general theme or major points. For example, writing a short review of a specific software product.
Inferring	Predict Deduce Extrapolate	Drawing a conclusion from presented information. For example, given a number of specific cases, produce rules using an expert system.
Comparing	Compare Contrast Evaluate Map	Detecting correspondences between ideas and/or objects. For example, contrast two programming languages in terms of their data structure facilities.
Explaining	Give reasons Explain Justify	Constructing a cause-and-effect model of a system. For example, give reasons for the emergence of the Internet.
Apply: Carry out or use a procedure in a given situation.
Executing	Carry out Perform Complete	Applying a procedure to a familiar task. For example, carrying out the procedure to install an applications package on a PC.
Implementing	Use Apply Implement	Applying a procedure to an unfamiliar task. For example, using applications software to solve a given problem or writing a piece of code to perform a specific task.
Analyse: Break material into its constituent parts and determine how these parts relate to one another and to the overall structure or purpose.
Differentiating	Select Choose Discriminate	Identifying similarities and differences, and important and unimportant attributes of objects or systems. For example, choosing a computer system (from two or more provided) for a specific task, or selecting a specific data structure to model a given problem.
Organising	Arrange Find Structure Organise	Determining how elements fit together within a system. For example, constructing a flowchart to represent a given problem description or producing a data flow diagram to model a supplied case study.
Attributing	Assign Attribute Deconstruct	Determine a point of view, bias, values or intent. For example, determining the point of view of an author of an essay on the social implications of IT.
Evaluate: Make judgements based on criteria and standards.
Checking	Check Verify Confirm Monitor Test	Determining inconsistencies or fallacies within a process or product. For example, dry running a given algorithm to check its correctness or testing a program to locate errors.