The Levels of Learning*
Their Effects
And Study Questions

Synthesis

Evaluation

Evaluation

Application

Application

Application

Analysis

Analysis

Analysis

Analysis

Comprehension

Comprehension

Comprehension

Comprehension

Comprehension

Knowledge

Knowledge

Knowledge

Knowledge

Knowledge

Knowledge

The Levels of Learning --
*aka Bloom's Taxonomy of Learning Objectives

The categories listed above and discussed below describe the different tiers at which a person's ability to manipulate facts can exist, from shallow to profoundly deep, from the least to the most complex, from the easy to the effort-and- time-intensive. They are also operations that people perform on sets of information in various situations.

Activities in any setting, tasks, questions, and assignments can all be classified under one or another of these divisions. Giving one's name, for example, when meeting someone for the first time is a simple knowledge task. A person just remembers what his or her name is and offers it. In education, teachers design class activities to achieve one or another or several of these objectives.

These skills represent activities students perform when they are learning. Since teachers use them to design all activities, students can more effectively prepare for class discussions, quizzes and tests by thinking about the course content in as many levels as possible. That way, s/he can also anticipate test questions and study more effectively for them. When a quiz or test question stumps a student, s/he can decide what kind of question it is and sometimes find the way to answer it. The best teachers introduce concepts and test mastery using as many of the higher learning level skills as possible.

There are six levels of action. All constitute operations performed on facts. Each one challenges the mind more rigorously than the one before and includes each of previous ones.

Looking at these six levels can help students understand a major way in which college differs from high school, provide a way to approach exams, and supply a basis for understanding why some assignments may seem harder than others.

The discussion below defines each level, provides indicator words for each, and gives a sample test or activity question. A link at the end of the initial discussion leads to additional sample questions and analysis of each. The reason for any variation from the traditional ranking is explained below also.

Know

Knowledge is the learning level that consists of being able to state previously learned facts and details about a topic on demand.

• It requires that students use the memory to recall and/or recognize information unaided.
• Questions are designed such that if the student accurately remembers the response to the prompt (question) and offers it, the task is completed.
• Matching definitions on an objective test fall into this category, as do most objective quiz and test questions.
• Memorized facts constitute the first step in each learning level.

Rote learning has some benefits. Memorizing facts has

• Inherent value -- Modern culture holds that it is good in and of itself to be able to know facts.
• Practical value -- facts "provide building blocks for generalization, laws, and principles" (Sanders), and as such, this ability forms the foundation for the other levels.
  
  Having facts at hand also saves time and trouble -- people should know the best places to find their favorite cars, food, and clothes.
  
  In addition, having the basic information that underlies one's business saves the time of looking it up and entitles one to the credibility that attaches to that profession.
  
  Last, the basic facts satisfy a primal need to know the landscape of one's identity, world, and individual place in it.
• Social value -- one appears uncultured if s/he is not familiar with the markers common to those of his or her culture. The more one shares the language, values, goals, behaviors, and appearance of the group one is seeking to join, the easier it is to get in.

Learning cannot be limited to memorizing information, though, despite the fact that to many, intelligence means the number of facts stored in one's head. These are the limitations to rote memorization.

• Arbitrarily chosen and isolated facts are quickly forgotten.
• Recall does not necessarily signal the ability to do something. Just because someone can restate a concept does not mean s/he can understand its implications, perform with it, or create something new based on it.
• Essential information is stored either in print or electronically and is so easy to access that it is no longer the most efficient use of the culture's resources to store it mainly in a person's head.
• What teachers tell first-year medical students may also be true of many other fields:

  Half of what you are taught as medical students will
  in ten years have been shown to be wrong, and the
  trouble is, none of your teachers knows which half.
  

  C. Sydney Burwell (1893–1967) British Medical Journal 2, 113, 1956 (Oxford Companion to Medicine). Facts change.
• Overemphasizing memory tasks supplant the exercise of the higher order thinking skills. It gobbles up time and energy that might have been more productively spent on activities that benefit people and society more in the long run. To learn to make inferences develops a skill that lasts. But the value of memorized inferences of others will fade. Memorized inferences also have limited application. The skills of induction and deduction have wide application, whereas applying the inferences of others results in meager success. If intellectual progress had stopped at rote memorization, none of the technological or social advances of modern times would have occurred. In third-world countries, much of what passes for learning is nothing more than rote memorization.

Knowledge Key Words

When a teacher uses these words on a test question or assignment, s/he is asking students to answer the question at the knowledge level. If they can do it correctly, they show they know the information.

Explain in your own words
Paraphrase
Summarize
Give the exact denotation

Sample Knowledge Question

List the three parts of an essay.

Key word: list

Goal: To name the main sections into which an essay can be divided

Limiting factor: include only the major parts of an essay as previously learned and nothing more. The student is not expected to write an essay.

Back to top

Comprehend

Comprehension is the learning level that consists of understanding the significance and purpose of a set of information that one knows. When a person knows something, s/he may also understand most of it, but comprehension also goes deeper to look for hidden meanings, significances, and overall themes.

It includes activities such as translation and interpretation.

• Translation -- The intellectual process of changing ideas from one form or medium to another with a minimum of personal spin. For example, a spoken story can be written down or transcribed into another language. The paraphrasing of a text is a translation task.
• Interpretation -- The intellectual process of changing ideas from one form or medium to another with a freedom to add responsible personal spin. Sanders adds, "To discover or use a relationship between two or more ideas" by relating "facts, generalizations, definitions, and values on a common sense level." One example is to show what a given short story would be if it were music -- a waltz, rap, or hardcore heavy metal tune. The form of the final statement allows for only a few reasonable spins before it is no longer related to the original.

Every interpretation starts with a translation and so includes it.

Comprehension Key Words

When a teacher uses these words on a test, s/he is asking students to answer the question at the comprehension level. If they can do it correctly, they show they comprehend the information.

State the thesis of
State the theme of
Find the main claim
Read between the lines
Find the main idea
In no more than six words, write a complete interpretation of

Sample Comprehension Question

Summarize the main features of the Freedom of Information Act.

Key word: summarize

Goal: to describe briefly the major kinds of access to government documents the Freedom of Information Act allows

Limiting factor: main features -- do not list all of the features of the act, just the ones that appear to be most important.

Back to top

Analyze

Analysis is the learning level that consists of seeing not only how parts make up a whole, but also how each part relates to each of the other parts with an awareness of the rules and dynamics of the relationship. Sanders says,

In other words, a respectable academic analysis makes two demands:

• One must know the rules that go with the process one is using to break information down into its parts (the various types are listed below)
• One must watch him- or herself as s/he does it to make sure it is being done correctly.

Those two factors make analysis seriously labor-intensive. The graphic at the top which lists the skills illustrates the difference in energy required. There, a large space separates analysis from the first two, low-energy skills, knowledge and comprehension.

The Seven Types of Analyses

Analysis features seven types of relationships.

• Comparison -- are the ideas or texts identical, similar, different, unrelated, or contradictory? What is the evidence?
• Relationship of implication -- "an implication is an idea that follows inevitably from specified evidence" (Sanders)
• Relationship of an inductive generalization to supporting evidence
• Relationship of a value, skill, or definition to an example of its use
• Numerical relationship
• Cause and effect relationship
• Classification/Division

Analysis Key Words

When a teacher uses these words on a test question or assignment, s/he is asking students to answer the question at the analysis level. If they can do it correctly, they show they can analyze the information.

Compare
Contrast
Cause
Effect
Group
Divide
Classify

Categorize
Draw
Outline
Point out
Arrange
Dissect
Sort

Analyze
Diagram
Differentiate
Compile
Find the difference
Illustrate visually
Discover relevant information

Sample Analysis Question

Compare the conditions that caused British Columbia to become part of Canada in 1871 to those that caused Newfoundland's entry in 1949.

Key word: Compare

Goal: To look for the reasons each entered the union and focus on the similar ones

Limiting factor: Examine conditions in British Columbia before 1871 and in Newfoundland before 1949 to find the similarities.

Back to top

Apply

Application is the learning level that consists of transferring training by independently selecting the behaviors appropriate to a concrete task.

It involves a hands-on, concrete activity. Anytime someone does something, it is an application task. S/he takes what s/he knows and applies it to a concrete operation of some type.

People train for them in two ways, which leads to its placement in different locations of the taxonomy.

• Informal application -- the task is learned through an apprenticeship or other one-on-one modeling. One human being models the steps to changing the car's oil while the apprentice watches and learns.
  
  This type involves no analysis, just mimicking, so it is placed after comprehension and before analysis in some listings of the learning levels.
• Academic application -- a person learns to perform the task by following a set of instructions on paper, videotape, or other electronic medium only, without the literal physical presence of the author.
  
  This type involves breaking the task down into the steps of a process analysis, following them, monitoring progress each part of the way, going back to correct missteps, and adjusting until the outcome is satisfactory. The analysis and increased effort call for placing it at least after analysis. Because it involves evaluating progress, an academic application fits best after or on the same line as evaluation. The list including an academic application looks like this: know, comprehend, analyze, apply/evaluate, synthesize.
  
  The order is important only so that students understand the degree to which the amounts of energy involved increase and how to perform the actions.

Often in an academic setting, which is privileged here, an application task involves scrutinizing a problem, selecting a known situation with significant matching parallels, applying them to a new setting, and arriving at the solution from that action. The information is discovered by the student -- the answer itself has not ever been discussed in class or in the reading assignment. The student must rely on the fact that at least one set of information needed to find the solution has been presented in class. S/he must use that plus personal experience and the analytical relationships process to find the way to the answer.

The assignments in college classes come in three types of applications.

• The instructor provides all of the content that the student must have to respond. If two sets of information are needed, the professor provides both. All the student has to do is discover the areas in which the two bodies of information intersect, discern the relationships, and find his/her way to the answer.
• The instructor provides only one of two or more sets of information that the student must have to respond. The instructor, whether s/he knows it or not, expects the student to locate the rest of the information. Then s/he must discover the areas in which the bodies of information intersect and find his/her way to the answer. The professor may or may not be willing or able to give guidance on finding or generating the second set of information.
• The instructor provides neither of two or more sets of information that the student must have to respond. In a class such as this, the book has nothing to do with the class discussions, and the exams have nothing to do with either the text, class discussions, or the lectures.
  
  The professor may or may not know it, or care when s/he is made aware of it. Some instructors who do it intentionally say that the student who cannot handle it does not belong in college. It is one way used to weed people out. The professor, whether s/he knows it or not, expects the student to locate all sets of information, discover the areas in which the bodies of information intersect, and find his/her way to the answer. The teacher's willingness and ability to give guidance on finding or generating the second set of information may vary.
  
  The situation may really be as it seems above, but it is also possible that the connections are there, just not user friendly. Whether they are or not, past students in such classes have worked out one way to approach the course when they absolutely have to have it for their degree plans. They audit the course with that professor in one semester, then sign up for it again the next semester and take it for credit.
  
  The assignments for this class most often resemble the first two types. If it ever seems as if the third one is going on, please email me at the address below. It's a sign of a communication problem, not an effort to weed anyone out.

Application Key Words

When a teacher uses these words on a test question or assignment, s/he is asking students to answer the question at the application level. If they can do it correctly, they show they can apply the information.

Imagine
Make believe
Use your knowledge of ___ to
Put yourself in the situation of

Sample Application Question

Use your knowledge of the three types of organization and sort the eight methods of development in a row under the type each most often uses.

Key word: use your knowledge of the three types of organization

Goal: To create a chart listing the three main ways in which any set of information can be organized and sort the methods of development into it according to the order pattern it most often uses

Limiting factor: information on the three types of organization as indicated by the transitions used most often to arrange information, the eight methods of development, and which type used most often by each. An analysis of the information in linked files above provides the two sets of information that carry the matching sets of data needed.

The resulting chart carries this information in table form, either this way or in columns:

Time -- narration, process

Space -- description

Order of importance -- classification, comparison, cause and effect, illustration, and definition

Back to top

Evaluate

Evaluation is the learning level that consists of setting up appropriate standards or values according to a purpose and determining how closely the idea or object meets them.

In the classroom, evaluation most often takes the form of being asked,

• "How did you like the book (or movie, or story)?"
• "How likely is the proposed solution to work for a given situation?"
• "Is the book (or movie, story, or behavior) good (or bad)?"
• "Is what the people are doing going to get them where they want to go?"

If neither the question nor the instructor provides the criteria by which to perform the evaluation, it is up to the student to name them in the answer.

Some listings of the learning levels place evaluation after synthesis. But because one can evaluate without creating something new, it is placed before synthesis in the taxonomy here. An artist in any medium has to pick and choose which concepts to illustrate, which media to use, and which creative paths to follow. All are evaluation tasks. One has to evaluate to synthesize. But once the artist finishes the project, no action absolutely needs to come after it. The creation can just be. If one necessarily had to measure the worth of creations, evaluation would necessarily follow synthesis, but since such is not the case, it is placed before.

Evaluation Key Words

When a teacher uses these words on a test question or assignment, s/he is asking students to answer the question at the evaluation level. If they can do it correctly, they show they can evaluate the information.

Sample Evaluation Question

According to the number of safety-related features each contains, determine which compact car is safest to drive.

Key word: Determine

Goal: To find out which compact car meets criteria that indicate a lower risk

Limiting factor: Cars must be compact. Examine only safety features.

Back to top

Synthesize

Synthesis is the learning level that consists of drawing information from relevant sources and arriving at a well-formed, coherent whole that includes parts of others, yet forms something entirely new.

• It requires imaginative, original thinking
• Great freedom characterizes the search for solutions
• The answer is not always necessarily known
• There may be more than one correct answer -- in the humanities which privilege socially beneficial actions, it must be
• Intellectually defensible
• Justifiable
• Yield a good life
• Succeed in the classroom, and/or
• be profitable in the working world

Society values synthesis the most because it is usually the key factor in solving most problems and the way that is most likely to solve major ones. To paraphrase Einstein, it is harder to solve a problem as long as one is still viewing it as s/he did when s/he created it. The fresh perspective is the first step to the solution.

Synthesis consumes energy at a magnitude that is so much greater than all of the previous five learning levels combined that it is given extra space where they are listed at the top of the page. The separation would be much larger if it wouldn't make the file so much bigger.

Synthesis Key Words

When a teacher uses these words on a test question or assignment, s/he is asking students to answer the question at the synthesis level. If they can do it correctly, they show they can create something original out of pre-existing parts.

Sample Synthesis Question

Rearrange the amendments to the U. S. Constitution in order from least to most important in everyday life. Also, devise any new amendments that you think need to be added to improve everyday life.

Key word: Rearrange, devise, and improve

Goal: To discover which amendments seem most important and to create those which are needed to make life better

Limiting factor: Deal only with amendments to the Constitution, not the other parts

The person whose education includes a healthy number of tasks at the different levels and whose education takes, changes significantly. These are some characteristics.

Works Cited

Parker, John. Workshops for Active Learning. Delta, BC, Canada:

Sanders, Norris. Classroom Questions: What Kinds? New York:

Back to top

The Effects of Systematic
Practice in the Learning Levels

People whose education includes a healthy number of tasks at the different levels and whose training takes approach life differently. The list at left describes characteristics often seen in people who have either not interiorized or else not been exposed to academic training. The items on the right are features more often exhibited by people who have assimilated their training.

The descriptions below do not justify value judgments or intellectual snobbery. They just highlight differences in behaviors of people who cannot or choose not to activate analysis and the other high-level skills.

Walter J. Ong in his 1982 book, Orality and Literacy, attributes the differences to aliteracy as opposed to regular and thoughtful reading and writing. It is also possible that the differences come from being hard-wired to prefer the ears over the eyes -- from absorbing information that one hears more readily than the learning that one sees. In any case, people act differently. It may be because of the amount they read and write -- or don't.

The person who has not interiorized academic training:

1. In the face of a significant, life-changing decision, consults only people.

The person who has assimilated academic training:

1. Uses print also as a point of reference (point of reference: a resource one consults when making a life-changing decision to measure, confirm, or establish one's position).

2. Is externally motivated -- s/he waits for things to happen or for others to provide direction.

2. Is internally motivated -- s/he recognizes her/his own responsibility to make things happen.

3. Writes what everyone knows in general terms, with narration as the main mode of learning

3. Writes and thinks in the methods of development, that is, analytically -- narration, yes, but also description, illustration, process, comparison, classification, cause and effect, and definition -- in useful, specific terms.

4. Wipes memory clean from semester to semester

4. Sees learning as cumulative -- keeps past learning in active memory, seeing how new learning depends on it, relating new information to that learned earlier.

5. Generally neither questions anything nor follows up to verify that something works, regularly relying on groupthink as a basis for action. (Groupthink: a mode of thinking that people engage in when they are so deeply involved in a cohesive in-group that their need for acceptance and sense of belonging overrides their motivation to appraise alternative courses of action realistically.)

5. Thinks critically. (Critical thinking: the practice of examining the thinking process carefully to clarify and improve understanding. The critical thinker appraises alternative courses of action realistically, discussing them in an organized way, implementing one, monitoring the result for effectiveness, and making changes until s/he achieves the desired outcome or changes the goal.

6. Reaffirms the status quo with little or no recognition of its limits.

6. Acknowledges the status quo with an awareness of its limitations and with an eye to change where possible.

8. Responds more readily to emotion and power arguments, granting least weight to mind appeals

8. Responds less readily to emotion and power arguments, granting most weight to mind appeals

_______________________________________________________________

Mechanism for going from the not to the one who has: University -- an institution of higher learning in which students and faculty participate in the making of knowledge.

Works Cited

Ong, Walter J. Orality and Literacy. London: Methuen Press, 1982.

Back to top

Study Questions for Learning Levels

The questions below test mastery over the above materials. The questions are matching definitions and multiple choice/short answer.

Back to top

Terms

_____ a. Interpretation

_____ b. Evaluation

_____ c. Point of reference

_____ d. Groupthink

_____ e. Synthesis

_____ f. Application

_____ g. Analysis

_____ h. Comprehension

_____ i. Critical thinking

_____ j. Knowledge

_____ k. Translation

_____ l. University

_____ m. Credit tag

.

Definitions -- Match up the terms at left with the phrases below. To write them in, copy the file into a word processor and insert or make a printout.

1. The learning level that consists of transferring training by independently selecting the behaviors appropriate to a concrete task.

2. The intellectual process of changing ideas from one form or medium to another with a minimum of personal spin.

3. The learning level that consists of understanding the significance and purpose of a set of information that one knows.

4. A mode of thinking that people engage in when they are so deeply involved in a cohesive in-group that their need for acceptance and sense of belonging override their motivation to appraise alternative courses of action realistically.

5. Words in a sentence that indicate the source of a quotation, paraphrase, or summary.

6. The learning level that consists of drawing information from relevant sources and arriving at a well formed, coherent whole that includes parts of others, yet forms something entirely new.

7. The learning level that consists of being able to state previously learned facts and details about a topic on demand.

8. The learning level that consists of setting up appropriate standards or values according to a purpose and determining how closely the idea or object meets them.

9. An institution of higher learning in which students and faculty participate in the making of knowledge.

10. The practice of examining the thinking process carefully to clarify and improve understanding by appraising alternative courses of action realistically, discussing them in an organized way, implementing one, monitoring the result for effectiveness, and making changes until one achieves the desired outcome or changes the goal.

11. A resource one consults when making a life-changing decision to measure, confirm, or establish one's position

12. The intellectual process of changing ideas from one form or medium to another with a freedom to add responsible personal spin.

13. The learning level that consists of seeing not only how parts make up a whole, but also how each part relates to each of the other parts with an awareness of the rules and dynamics of the relationship

Back to top

Multiple Choice and Short Answer Questions

These questions focus attention on the significant facts and concepts relating to the learning levels.

1. Bloom's taxonomy of learning objectives is a synonym for

2. In the diagram at the top of the page, the extra spaces between analysis and the first two learning levels and between synthesis and all the others are there to indicate

3. The learning levels are

4. The diagram at the top of the page does not list the six learning levels in a straight line. What structure or item in the world might the way they are listed resemble? What special characteristic(s) about the six levels might the structure you see illustrate?

5. In the diagram at the top of the page, what concept does placing knowledge at the bottom of each column illustrate?

6. In the diagram at the top of the page, what concept does placing all six levels in order in the last column illustrate?

7. Anything a person asks another to do

8. Knowing the levels can help a student when taking a test or doing anything else if s/he

9. The best teachers introduce concepts

10. What is the highest level at which Question 3 above in this multiple choice/short answer section operates?

11. Read this sample comprehension question: Summarize the main features of the Freedom of Information Act. If one responds to the question by listing all of the features, what is the highest level at which that answer operates?

12. Which learning level requires that one develop (in a healthy way) a split personality who is not only doing something, but monitoring what s/he is doing to make sure s/he is doing it right?

13. Read this sample analysis question: Compare the conditions that caused British Columbia to become part of Canada in 1871 to those that caused Newfoundland's entry in 1949.

14. Finish this rhyme as a person with two personalities would:

15. When a question is an evaluation, what should a student do if neither the question nor the instructor provides the criteria for making the judgment?

16. Read this evaluation question: According to the number of safety-related features each contains, determine which compact car is safest to drive. Assume that a student includes these statements along with several other sentences in the answer: "A Lamborghini gets 11 miles to the gallon." What two problems having to do with the rules for doing an evaluation arise in the sentence quoted?

17. Read this sample synthesis question: Rearrange the amendments to the U. S. Constitution in order from least to most important in everyday life. Also, devise any new amendments that you think need to be added to improve everyday life. By definition, synthesis draws on several sources. List the two sources that must be accessible at minimum to answer the question.

18. Read the sample synthesis question above again. By definition, synthesis calls for creativity. What part of the question requires inventiveness to answer the question?

19. What is the highest level at which questions 11 and 16 above operate?

20. Judging from what you've seen, at what level is most education conveyed at the high school level?

21. Think about what most people mean when they talk about education and intelligence. At which level does what they mean operate?

22. Judging by your experience, at what level do most people operate?

23. What is the highest level at which this question operates: Design a plan for your life that results into the greatest good for the greatest number of people, including yourself.

24. State two concepts illustrated by the way the names of the learning levels are organized in the lists at the beginning of this discussion.