Infidelity to Truth: Education Malpractices in American Public Education: Chapter Six

By Duane Swacker

About Duane

Chapter 6

Of Standards and Measurement

Truth is not only violated by falsehood; it may be equally outraged by silence. Henri Frederic Amiel

How can anyone be against having standards in the classroom, standards for behavior or learning?  Kind of hard to argue against, eh!  What is so wrong with holding students accountable to educational standards?  Nothing!  Except when the term standard is inappropriately and incorrectly used to mean one thing while purporting to signify another, in other words lacking fidelity to truth.

Surely we need to and must measure student achievement.  How are we going to know how the student stands up to the standard?  How are we to know how the students in one class, district or state do in comparison to other classes, districts or states if we don’t measure student achievement?

The silence is deafening in regard to the lack of logical thought and the abuse of the language that permeates educational discourse in the standards and measurement movement.  The standards and measurement meme in public education has been a part of policies and practices for at least the last quarter of a century.  Even before NCLB, state departments of education were making and disseminating “standards” as guides for classroom curriculum.  And the emphasis was being guides and not some supposed “standard” against which educational outcomes could be supposedly “measured” for not only the student but teacher, school and district.  It wasn’t until the passage of NCLB in 2001 that the standards and measurement meme has come to completely dominate not only school life but the policy and practice arenas from the legislatures to state departments of educations to district boards and into the schools.

The standards and measurement movement is choking the life out of our public school classrooms!

It is causing innumerable harms to students, distorting curriculum and the teaching and learning process, many times into a year-long test prep program, causing districts to drop many electives, foreign languages, band, choir, and many others not related to the two main tested subjects English and Math.  Not only that but in the elementary level many students are now deprived of much needed recess/play time, gone are learning stations in favor of drill and kill methods of attempting to raise test scores.  Ever increasing test scores have become the predominant driver of curriculum since NCLB was signed into law.

In order to untangle this mess of educational malpractices that standards and measurement discourse has brought about we first need to examine exactly what are standards and measurements in a broader logical context which then will enable us to ascertain just how damaging the misuse of language, the twisted use of logic that makes the standards and measurement movement appear to be THE way to improve the teaching and learning processes in American public schools.  It will then be shown that using the false and error filled practices of educational standards and standardized testing contravene the fundamental purpose of public education causing, at times, irrevocable harm to the student in not guaranteeing “to promote the welfare of the individual so that each person may savor the right to life, liberty, the pursuit of happiness, and the fruits of their own industry.

The word standard is in the top 1000 most used words in American English and the Miriam Webster online dictionary gives the following definitions:


1:  a conspicuous object (as in a banner) formerly carried at the top of a pole and used to mark a rallying point especially in battle or to serve as an emblem

2a:  a long narrow tapering flag that is personal to an individual or corporation and bears heraldic devices  b:  the personal flag of the head of state or of a member of a royal family  c:  an organization flag carried by a mounted or motorized military unit  d:  banner

3:  something established by authority, custom, or general consent as a model or example:  criterion <quite slow by today’s standards>

4:  something set up and established by authority as a rule for the measure of a quantity, weight, extent, value, or quality

5a:  the fineness and legally fixed weight of the metal used in coins  b:  the basis of value in a monetary system <the gold standard>

6:  a structure built for or serving as a base of support

7a:  a shrub or herb grown with an erect main stem so that it forms or resembles a tree  b:  a fruit tree grafted on the stock that does not induce dwarfing

8a:  the large odd upper petal of a papilionaceous flower (as of the pea)  b.  one of the three inner usually erect and incurved petals of an iris

9:  a musical composition (as a song) that has become a part of the standard repertoire

For the purposes of this discussion, obviously definitions 1, 2, 5, 6, 7, 8, and 9 do not concern us.  It is the somewhat similar and perhaps inter-confusing definitions of 3 and 4 that interest us.

As mentioned above before NCLB the definition of standard as used in the individual state’s curriculum standards and even today in curriculum standards promulgated and promoted by subject area organizations such as the National Council of Teachers of Mathematics or the American Council of Teachers of Foreign Languages the term standard as used fell/falls under definition three as they were never meant to be used as “a rule for the measure of a quantity, weight, extent, value, or quality” as in definition four but as a model for teachers to use.  Confusing indeed!

Another way to look at the concept of standards is that there are two accepted types of standards, metrological and documentary.

Metrology is the science of measurement and a metrological standard “is an object, system, or experiment that bears a defined relationship to a unit of measurement of a physical quantity.  Standards are the fundamental reference for a system of weights and measures, against which all other measuring devices are compared. Measurements are defined in relationship to internationally-standardized reference objects, which are used under carefully controlled laboratory conditions to define the units of length, mass, electrical potential, and other physical quantities.

A documentary standard “is a document established by consensus and approved by a recognized body, that provides, for common and repeated use, rules, guidelines or characteristics for activities or their results, aimed at the achievement of the optimum degree of order in a given context.

Many governmental departments promulgate documentary standards, for example the Food and Drug Administration (FDA) or the Environmental Protection Agencies (EPA) while at the same time being the certifying agent to ensure that the standards are followed.  The ISO promulgates international standards but is not the certifying agency, other agencies do the certifying of companies compliance with their standards.  From the EPA:

“When developing regulations, the first thing we do is ask if a regulation is needed at all. Every regulation is developed under slightly different circumstances but this is the general process: 

Step 1: EPA Proposes a Regulation 

The agency researches the issues and, if necessary, proposes a regulation, also known as a Notice of Proposed Rulemaking (NPRM). The proposal is listed in the Federal Register (FR) so that members of the public can consider it and send their comments to us. The proposed rule and supporting documents are also filed in EPA’s official docket on 

Step 2: EPA Considers Your Comments and Issues a Final Rule 

Generally, once we consider the comments received when the proposed regulation was issued, we revise the regulations accordingly and issue a final rule. This final rule is also published in the FR and in EPA’s official docket on 

Step 3: The Regulation is Codified in the Code of Federal Regulations 

Once a regulation is completed and has been printed in the FR as a final rule, it is codified when it is added to the Code of Federal Regulations (CFR). The CFR is the official record of all regulations created by the federal government. . . . ” 

The ISO has strict rules for making and issuing standards.  The key principles in standard(s) development:

1.  ISO standards respond to a need in the market.

ISO does not decide when to develop a new standard, but responds to a request from industry or other stakeholders such as consumer groups. Typically, an industry sector or group communicates the need for a standard to its national member who then contacts ISO.

2. ISO standards are based on global expert opinion.

ISO standards are developed by groups of experts from all over the world that are part of larger groups called technical committees. These experts negotiate all aspects of the standard, including its scope, key definitions and content.

3. ISO standards are developed through a multi-stakeholder process.

The technical committees are made up of experts from the relevant industry, but also from consumer associations, academia, NGOs and government.

4. ISO standards are based on a consensus

Developing ISO standards is a consensus-based approach and comments from all stakeholders are taken into account.

The Common Core State Standards (CCSS) and all other state educational standards might be considered a documentary standard but in the development of the standards no procedures have followed the formal protocol and processes as outlined by the OSI or government agencies in their development.

In addition to that and perhaps even worse is that the proponents of these standards claim that the CCSS are standards against which ‘student achievement’ can be measured.  In doing so educational standards proponents claim the documentary standard (definition three) as a metrological standard (definition four).  In doing so they are falsely claiming a meaning of standard that should not be given credence.

This confusion is compounded by what it means to measure something and the similar misuse of the meaning of the word measure by the proponents of the standards and testing regime.  Assessment and evaluation perhaps can be used interchangeably but assessment and evaluation are not the same as measurement.  Word usage matters!

The Merriam-Webster dictionary definition of measure includes the following:

1a (1):  an adequate or due portion (2):  a moderate degree; also: moderation, temperance (3):  A fixed or suitable limit:  bounds <rich beyond measure> b:  the dimensions, capacity or amount of something ascertained by measuring c:  an estimate of whit is to be expected (as of a person or situation d: (1):  a measured quantity (2):  amount, degree

2a:  an instrument or utensil for measuring b (1):  a standard or unit of measurement—see weight table (2):  A system of standard units of measure <metric measure>

3:  the act or process of measuring

4a (1):  melody, tune (2):  dance; especially:  a slow and stately dance  b:  rhythmic structure or movement: cadence:  as (1):  poetic rhythm measured by temporal quantity or accent; specifically:  meter (2):  musical time c (1):  a grouping of a specified number of musical beats located between two consecutive vertical lines on a staff (2):  a metrical unit:  foot

5: an exact divisor of a number

6:  a basis or standard of comparison <wealth is not a measure of happiness

7:  a step planned or taken as a means to an end; specifically:  a proposed legislative act

Measure as commonly used in educational standard and measurement discourse comes under definitions 1d, 2, and 3, the rest not being pertinent other than to be used as an obfuscating meaning to cover for the fact that, indeed, there is no true measuring against a standard whatsoever in the educational standards and standardized testing regimes and even in the grading of students.  What we are left with in this bastardization of the English language is a bewildering befuddle of confusion that can only serve to deceive many into buying into intellectually bankrupt schemes that invalidly sort, rate and rank students resulting in blatant discrimination with some students rewarded and others punished by various means such as denying opportunities to advance, to not being able to take courses or enroll in desired programs of study.

The most misleading concept/term in education is “measuring student achievement” or “measuring student learning”.  The concept has been misleading educators into deluding themselves and others that the teaching and learning process can be analyzed/assessed using “scientific” methods which are actually pseudo-scientific at best and at worst a complete bastardization of rationo-logical thinking and language usage.

There never has been and never will be any “measuring” of the teaching and learning process and what each individual student learns in their schooling.  There is and always has been assessing, evaluating, judging of what students learn but never a true “measuring” of it.

The TESTS MEASURE NOTHING, quite literally when you realize what is actually happening with them. Richard Phelps, a staunch standardized test proponent (he has written at least two books defending the standardized testing malpractices) in the introduction to “Correcting Fallacies About Educational and Psychological Testing” unwittingly lets the cat out of the bag with this statement:

Physical tests, such as those conducted by engineers, can be standardized, of course, but in this volume, we focus on the measurement of latent (i.e., nonobservable) mental, and not physical, traits.

Notice how he is trying to assert by proximity that educational standardized testing and the testing done by engineers are basically the same, in other words a “truly scientific endeavor”.  The same by proximity is not a good rhetorical/debating technique.

Since there is no agreement on a standard unit of learning, there is no exemplar of that standard unit and there is no measuring device calibrated against said non-existent standard unit, how is it possible to “measure the nonobservable”?


Finally, what the proponents of the educational standards and standardized testing regime don’t appear to understand is that in many areas of human interactions and feelings there cannot be any measurement.  How does one measure the love of one’s spouse, children, parents or friends?  How does one measure what is going on in the heart and mind of a distressed person who has just lost a loved one?  Why do we even begin to think that we can measure what goes on in the body and brain of the student who is learning any subject matter considering all the various hormonal and endocrinal influences occurring outside the individual’s control, with the hundreds of millions if not billions of neuronal firings going on at any given moment that partially influence what happens in the mind of the student in a teaching and learning situation?  How do we believe that the thousands and thousands of environmental influences on each individual could begin to be measured and accounted for?  Are proponents of the educational standards and standardized testing “measurement” regime that arrogant, hubristic and presumptuous to believe that they hold the key to supposedly measuring the teaching and learning process or more specifically, the learning, aka, student achievement, of an individual student?

Considering the facts of the misuse of language, logic and common sense as outlined above, the only wise course of action is to immediately cease and desist, to abandon, those malpractices that harm so many students and contravene the state’s responsibility in providing a public education for all students.  The billions of dollars spent by states on the educational standards and standardize testing regime would then be freed up to provide a better education for all students through perhaps such things as smaller class sizes, needed social services, foreign language instruction, arts programs, etc.  And the state, by approving and mandating the fake standards and false measuring of student learning that are the malpractices of educational standards and standardized testing, by not adhering to a regimen of fidelity to truth is surely guilty of not promoting “the welfare of the individual so that each person may savor the right to life, liberty, the pursuit of happiness, and the fruits of their own industry.

  1.  I purposely used “do” and not “are learning” as the teaching and learning process is not amenable to simplistic comparisons.
  2.  Yes, play time.  The research on the importance of play time for elementary (K-8) students in social and academic development is overwhelming.
  3.  And that dominance was greatly enlarged by the Obama administration’s “Race to the Top” federal mandates in 2009.  What a bizarrely craven way to attempt to improve the teaching and learning process, by having states compete for monies only if they enacted certain unethical malpractices such as using standardized testing not only for sorting and ranking students but also evaluating teachers and schools districts.
  4.  From the Oxford Online Dictionary, see:
  5.  See:
  6.  See:
  7.  From the National Institute for Standards and Technology
  8.  ISO is the French acronym used in describing standards.  In English the ISO means the International Organization for Standardization.
  9.  From the EPA website:
  10.  See:
  11.  The scope of this study does not include a discussion of the nefarious process by which the CCSS were made and forced upon the states through what might easily be considered monetary extortion by the Federal Department of Education.
  12.  Now whether that claim, a documentary standard as a metrological one, is intentionally misleading or not I leave up to the reader.  Personally I don’t believe they have the knowledge to understand the difference.  And while that attempt may be well intentioned we know that “The road to hell. . . . “
  13.  See:
  14.  The suggestions only begin to scratch the surface of what might be done and which must be determined by each public school community.
  15.  The false and misleading language embedded in these practices surely must qualify them for the designation of malpractice.



Ignorance of Laws Is No Excuse

The Treehorn Express

….on behalf of kids. …against the nasty prejudices of testucating, inexperienced, indifferent dyslexics.

“Ignorance of the Laws is No Excuse.”

Schools are institutions that have been established as happy, secure places for young children to maximize their creative intelligence. Right? They encourage the pupils within to reach any goal that they want to reach, unhindered and at the pace of the learner. Learning how-to-learn-skills and a quest for excellence in all life-skills are built into each child’s psyche through the natural desire to learn at each one’s level of cope-ability . Learning is a pleasureable pursuit and a happy adventure for all .

Well, they are supposed to be. BUT! The 21st century style of schooling has ignored too many fundamental laws. Such lack of knowledge is hardly an excuse for expert, professionally active, learning transmitters in schools. Each of us should know more about these laws, whether we are a rookie in our first year of teaching or a redundant, geriatric has-been, as I am. Agree?

Lets check :

CAMPBELL’S LAW : The more any quantitative social indicator is used for social decision-making, the more subject it will be open to corruption pressures and the more apt it will be to distort and corrupt the social pressures it is intended to measure.

GOODHART’S LAW : When a measure becomes a target it ceases to become a good measure. [Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.]

McNamara’s Law : Decisions based on quantitative variables and ignoring all other variables, are artificial and misleading.
[ They usually 1. Measure only whatever can be easily measured. 2. Disregard that which can’t be measured 3. Presume that which can’t be measured easily, isn’t important.] 4. What can’t be easily measured doesn’t exist.]

Settledge’s Definition of Mark, Score or Grading: An inaccurate report of an inaccurate judgement by a biased and variable judge of the extent to which an undefined level of mastery of unknown proportions of an inadequate mount of material has been completed.

Lucas Critique: If we wish to predict the effects of an educational policy change, we should first model the deeper parameters, the micro-foundations of learning.


Standardised Blanket Testing: A dysfunctional offspring of an unhappy liaison between megalomaniacal greed and crazed testucating politics. Disruptive and depressing as a learning teaching/device.



Ignorance of such reliable, empirical laws is no excuse. We should know better. Their infallibility is unquestionable.

For further definitions, CHECK HERE
Phil Cullen [….for kids] 41 Cominan Avenue, Banora Point Australia 486 07 5524 6443

A True Schooling Conference

The Treehorn Express 

Treehorn story?

Theme Song :

The Treehorn Express is dedicated to the cessation of Kleinist NAPLAN testing in Australia.  Kleinism is a New York version of fear-driven schooling which uses the blanket-testing ‘wmd’ called NAPLAN [its only learning-motivational weapon] to destroy the  reputation of teachers and schools. This weapon was forced on schools in Australia in 2009. It separates ‘haves’ from ‘have-nots’; and opens the door for mega-bank-rolling by known publishers, measurers and technocrats for absolute control of schooling in this country. It disrespects school pupils, devalues teachers’ professionalism, threatens Australia’s developmental future and is just no good.  More politely described, it stinks. Although some ‘education’ groups support it, ideologically, NAPLAN is immoral, unprofessional, politically driven, unrequested by the profession, curriculum destructive, extremely costly, wasteful and divisive. It has a background of malicious intent.


For further information, click on the official description



Last month on 23 October, 2011, the creator of Treehorn passed away at 92 years of age. Florence Parry Heide was the mother of five children, grandmother of seven and a great-grandmother.  She was one of the most popular authors of children’s books – ever. Her most popular was The Shrinking of Treehorn [1971]. illustrated in macabre fashion  by Edward Gorey who also illustrated her two other Treehorn books, Treehorn’s Treasure [1981] and Treehorn’s Wish [1986].


She was going to call our hero Harold, but decided on a home-made version from the surname of one of her friends.That makes his name unique; but his appalling ailment is shared by all children of his age – primary school age. They are universally ignored. Everybody: parents, teachers; all adults ignore children at school when things get serious. Most adults just don’t know what to do when politics and schooling get mixed. Young pupils never complain, so it’s ‘let’s just go with the flow’ for the adults.  What a monumental shame.

Vale Florence Parry Heidi. God bless you for giving us such a reminder of our frailties.



Following a conference last week at which Peter Garrett, Minister for Education etc., led a group of principals on ways to improve scores on the Naplan tests, there will be another one on Monday, 14 November at which he will lead a conversation with parents live and on-line for the same purpose. Improving the scores is important for Peter G. for political purposes.  The integrity of the spirit of mathematics and literacy curriculum is not discussed. If you are interested in checking this out by ‘attending’, try 

I do hope that the parental right to exclude children from the Naplan annual tests is emphasised.  I share a feeling with others that it may not. Fair play? Why aren’t schools encouraged to have a permanent fixture on their school newsletters some statement like : “The professional staff of this school disapproves of national blanket testing and would like to encourage parents not to participate in Naplan testing. A note to this effect, addressed to the Principal will be sufficient. We value on-the-spot shared evaluation of progress with each pupil.”

I’ll miss the on-line show, but I’d like to ask: “Why, dear Peter, shouldn’t you encourage such statements, so that parents know?”

The Gillard government, as indeed might an Abbott government, feels quite secure that schooling-by-fear is here to stay. One commentator suggests that this is part of our endemic militarism. Political parties feel encouraged by the silence of teachers and other adults (Hi Treehorn), with quite weird active support from principals’ formal groups. Few remember that the importation was a political stunt and nothing else. It had nothing to do with improving learning.  Now, at the centre of origin the majority of school principals [New York] want the whole business terminated.


Peter, Julia and their neo-naplan friends want to keep it !!!

There’s a teacher or two who believes that the effects of Naplan can be softened…like softening the attitude of a bull-pit, half-way through its leap at your throat.  Professionalism!  What’s happened to it?  Say “Stop it” for children’s sake. Completely.

Les Treichel says, “I fear that the ‘window of opportunity’ is regrettably now firmly closed. Our only saving grace will be when Principals and teachers at the workface demonstrate the courage to stand up and be counted! Of course they must firstly convince the Mums and Dads that NAPLAN is a debilitating disease that will ultimately kill the love of learning and have disastrous consequences for the harnessing of each and every child’s full educational potential. This will be no easy challenge as parents tend to be highly gullible and prone to be readily brainwashed by our political masters.”

There is no one in Australia more experienced in the schooling of remote school children than Les Treichel. Very, very few would come anywhere near him. After over fifty years of active hands-on school experience in the far west of Queensland, he is speaking, no doubt, on behalf of the thousands of children and parents that he has known and worked with.

I’d like to see a real Schooling Conference with the likes of Les Treichel, Paul Thomson [Kimberley College Principal], Anne Nelson [Spensely Street State Primary School], Allan Alach [Hokowhitu Primary School] and others of rich experience, wide reading and deep thinking about teaching and learning in the school context, in conversation with other practising chalk-face teachers and principals, discussing what really happens in classrooms and how we can do better. We would have a TV show of epic proportions….a signature event.  No lectures. Just plain, honest-to-goodness talk, broadcast and recorded for the benefit of those who can’t attend.  (Nobody seems to talk about ‘teaching and learning in the school context’ any more, do they?.) I’d like to see Prof. Michael Dunkin there too. What Australian academic has studied and described nitty-gritty classroom interactions in depth more than he has?

A cynic might suggest that one such conference be run for those teachers who vote Labor; one for Conservatives[post 2013?] and one for the apolitical.  The big controllers might then take notice of the report.  Might as well take politically controlled schooling to its extreme. Cynic!  In any case such a conference would be a stand-out…talking about teaching and learning in classrooms!  Unique.


Valid Testing

Australia’s best known and most respected literacy educator with over fifty years on the job, Prof. Brian Cambourne is concerned about validity aspects of the NAPLAN tests of literacy. He suggests that questions need to be asked about what some psychometricians call “consequential validity” [what happens as a consequence] and what all psychometricians call “ecological [or external] validity” [measuring the effectiveness]. The later assumes that “NAPLAN and effective literary behaviour are synonymous”.  More later.


Pedagogy of the Oppressed 

Allan Alach applies this wonderful book by Brazilian Paulo Friere, written in 1970, to the New Zealand context.  Its currency is maintained for a number of countries. Pick yours. Allan’s article appears in this week’s Education Today, a really wonderful New Zealand magazine [amongst the world’s best] that dares, in its current issue, to have politicians makes pre-election statements. Great idea. Make sure you read the comments. Editor Doug Hislop has given permission to copy Allan’s article for Treehorn. Hang in there.


Occupy DOE 

The occupying bug has spread. American teachers and parents are planning to occupy the Department of Education in Washington next year to protest the stupidities allied to blanket testing. Check it out.


Is there a mining magnate or corporate leader out there who cares for kids?

There are people in Victoria who would like to produce enough car-stickers ‘BAN NAPLAN’ for each teacher in Australia to put on their car.

Get in touch.

Attention Tom : “How do you eat an elephant? One bite at a time.”


Thousands of thanks to Allan Alach for his advice and references.

Other Treehorns ? :   Check Recent Posts and Archives in the sidebar.

Phil Cullen

41 Cominan Avenue

Banora Point

Australia 2486

07 5524 6443

Validity & Reliability of Tests

until blanket testing is ‘dead, buried and cremated’.

The Treehorn Express

If Treehorn, the hero of Florence Patty Heidi’s The Shrinking of Treehorn. was to set a test for those adults who constantly judged him and made certain assertions about him and his condition…as pro-Naplanners are wont to do with all children…one has to wonder just what questions he might ask! Would you care to send some to me at  ….apart from the obvious :”Why don’t you take any notice of me?”

Please send one or two or more. I‘d love to list them.


Validity & Reliability of Tests

Blanket testing is a device used by the unprincipled and inexperienced to annoy children, with whom they won’t or can’t discuss individual learning progress.

{Called NCLB in U.S.A.; National Standards in N.Z.; National Testing in the U.K.; Naplan in Australia}

The tests pretend to measure some half-dozen or so hard  competencies of young children, too intimidated to explain anything to anybody.

Even though they know of its evils, the unscrupulous use their findings to make gross statements about pupils’ general competencies, teacher ability and over-all school performance. They allow the publishing of the names of the ‘best’ schools and the ‘worst’ schools.

These classroom sciolists, camp followers and professional illiterates believe that the testing is valid and reliable ; and that the tests can distinguish the holistic differences between children, teachers and schools accurately. They have the political power to pretend. Basic professional ethics, human child feelings and parent concern just don’t matter.

– – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – –

Since I am a convert from an earlier-generation Kleinism [fear-based schooling], I can describe blanket testing with such politeness. I didn’t worry about the validity or reliability of those term and annual tests that I gave to pupils in my developmental-principal years. Tagging kids with numbers and scores was good enough for most parents. That’s all they seemed to want. Since then, wider experience, deeper professional reading and a colleague-developed, deeply-entrenched ethical standard have assured me that such blanket tests are useless, evil and dangerous. They should not exist. They tag everything with numbers!!

Now…with reform-based Kleinism,  those, more expert than I, believe that the validity and reliability aspects need to be considered more deeply than they are…

Professor Brian Cambourne, Australia’s distinguished literacy guru says:

This acceptance [especially by the media and education bureaucrats] has given NAPLAN high ‘face’ validity with the general public and this face validity has been conflated to equate with what psychometricians call ‘construct’ validity. Nor has what some new breed psychometricians call ‘consequential’ validity ever been researched. (Consequential validity addresses the question ‘Are the consequences of applying this test worth the pedagogical costs of using it?’)

Brian goes on to express the hope that the community itself would come to question the validity of NAPLAN. He expresses concern about…

1. The number of kids who have been classified as “failing” or “poor” readers who are avid effective readers of complex books, web sites, etc. I’ve a;lready met a number of parents and teachers who look at their children’s NAPLAN results and shake their heads in amazement because they know that these kids are very effective readers. (Wouldn’t it be wonderful if we could somehow collect and share hundreds of these stories?)

2. I’ve seen some eye-movement comparisons of effective readers and ineffective readers reading normal book-based or paper-based text of appropriate level and standardised test texts. The evidence is that reading the standardised test is a substantially different process from reading a “normal” text.

3. One school I visit is worried that its kids did poorly in the spelling section of NAPLAN, yet in their writing at school they clearly demonstrate high spelling ability. Have you looked at the spelling section if NAPLAN? It’s not a test of the kind of spelling knowledge that supports written communication.

Brian reckons that if he could find $50K for the eye-movement technology mentioned in#2 above, he could collect some VERY HARD data which shows just how invalid NAPLAN is. Anybody know a mining magnate or CSG operator who could spare this amount out of Petty Cash?

He believes that there is a need for  detailed research into the nitty-gritty details of the processes and assumptions underpinning the construction, use, application and scoring of NAPLAN type tests.

Seriously, if you have the address of a philanthropic rich person who is looking for a children-benefit project…why not send him this Treehorn and highlight this section. You never know! It seems that SOMETHING MUST BE DONE.

Thanks to Dr. Marian Lewis of USQ, support for this view comes from the U.S…..

Whoever Said There’s No Such Thing As a Stupid Question Never Looked Carefully at a Standardized Test

It can’t be repeated often enough: Standardized tests are very poor measures of the intellectual capabilities that matter most, and that’s true because of how they’re designed, not just because of how they’re used. Like other writers, I’ve relied on arguments and research to make this point. But sometimes a telling example can be more effective. So here’s an item that appeared on the state high school math exam in Massachusetts:

n 1 2 3 4 5 6

tn 3 5 __ __ __ __

The first two terms of a sequence, t1 and t2, are shown above as 3 and 5. Using the rule: tn = (tn-1) plus (tn-2), where n is greater than or equal to 3, complete the table.

If (a) your reaction to this question was “Huh??” (or “Uh-oh. What’s with the teeny little n’s?”) and (b) you lead a reasonably successful and satisfying life, it may be worth pausing to ask why we deny diplomas to high school students just because they, too, struggle with such questions. Hence [Deborah] Meier’s Mandate: “No student should be expected to meet an academic requirement that a cross section of successful adults in the community cannot.”

But perhaps you figured out that the test designers are just asking you to add 3 and 5 to get 8, then add 5 and 8 to get 13, then add 8 to 13 to get 21, and so on. If so, congratulations. But what is the question really testing? A pair of math educators, Al Cuoco and Faye Ruopp, pointed out how much less is going on here than meets the eye:

The problem simply requires the ability to follow a rule; there is no mathematics in it at all. And many 10th-grade students will get it wrong, not because they lack the mathematical thinking necessary to fill in the table, but simply because they haven’t had experience with the notation. Next year, however, teachers will prep students on how to use formulas like tn = tn-1 + tn-2, more students will get it right, and state education officials will tell us that we are increasing mathematical literacy.[1]

In contrast to most criticisms of standardized testing, which look at tests in the aggregate and their effects on entire populations, this is a bottom-up critique. Its impact is to challenge not only the view that such tests provide “objective” data about learning but to jolt us into realizing that high scores are not necessarily good news and low scores are not necessarily bad news.

If the questions on a test measure little more than the ability to apply an algorithm mindlessly, then you can’t use the results of that test to make pronouncements about this kid’s (or this school’s, or this state’s, or this country’s) proficiency at mathematical thinking. Similarly, if the questions on a science or social studies test mostly gauge the number of dates or definitions that have been committed to memory — and, perhaps, a generic skill at taking tests — it would be foolish to draw conclusions about students’ understanding of those fields.

A parallel bottom-up critique emerges from interviewing children about why they picked the answers they did on multiple-choice exams — answers for which they received no credit — and discovering that some of their reasons are actually quite sophisticated, which of course one would never know just by counting the number of their “correct” answers.[2]

No newspaper, no politician, no parent or school administrator should ever assume that a test score is a valid and meaningful indicator without looking carefully at the questions on that test to ascertain that they’re designed to measure something of importance and do so effectively. Moreover, as Cuoco and Ruopp remind us, rising scores over time are often nothing to cheer about because the kind of instruction intended to prepare kids for the test — even when it does so successfully — may be instruction that’s not particularly valuable. Indeed, teaching designed to raise test scores typically reduces the time available for real learning. And it’s naïve to tell teachers they should “just teach well and let the tests take care of themselves.” Indeed, if the questions on the tests are sufficiently stupid, bad teaching may produce better scores than good teaching.


1. Cuoco and Ruopp, “Math Exam Rationale Doesn’t Add Up,” Boston Globe, May 24, 1998, p. D3.

2. For examples (and analysis) of this kind of discrepancy, see Banesh Hoffmann, The Tyranny of Testing (New York: Crowell-Collier, 1962); Deborah Meier, “Why Reading Tests Don’t Test Reading,”Dissent, Fall 1981: 457-66; Walt Haney and Laurie Scott, “Talking with Children About Tests: An Exploratory Study of Test Item Ambiguity,” in Roy O. Freedle and Richard P. Duran, eds., Cognitive and Linguistic Analyses of Test Performance (Norwood, NJ: Ablex, 1987); and Clifford Hill and Eric Larsen,Children and Reading Tests (Stamford, CT: Ablex, 2000).

– – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – –

Next Issue

Since the holiday period allows more time for deep professional reading, I hope to concoct a list of appropriate readings that you might enjoy. Aren’t we lucky these days to have so much available at our finger-tips [so to speak] that gives meaning and pride to the teaching-learning enterprise?

Don’t Forget

Can you send a question that Treehorn will put on his test for his adult ‘carers’?   AND  If you do know the address of a magnate or person who might sponsor research into aspects of NAPLAN mentioned above, you can send it to me, if you prefer. I shall send it on. It could help our kids and show them that we like them.

Phil Cullen

41 Cominan Avenue

Banora Point  2486

07 5524 6443

Enhanced by Zemanta