Seen today on my Google+ feed:

“Even failed experiments and theories teach valuable lessons.”

Not entirely disconnected, here is an email I recently received from Jason Knapp of Ohio:

For what’s it worth – I have a great amount of respect and admiration for what you’re doing, and what you have achieved with your Academy. My son loves your Number Theory book – it’s his favorite out of the AoPs set.

[anecdote for your classroom]

I have a mathematics background, and I direct research for a small biotech company. I’m honestly dismayed by the ever increasing difficulty of finding people with good problem solving skills – from finding interns to hiring PhD level scientists. I have actually recommended some of the AoPs books to some of our interns (kids working on their masters – smart, but poorly trained). Not because they need to know power-of-a-point or sum-of-cubes, but because the process of solving those types of problems is remarkably the same as that used to solve *real* problems: getting organized, searching for patterns, thinking of a simpler problem, changing your point-of-view, etc …

When class looks like this, you have the trust of your students that it’s going to be a good class:

Hat tips: Bill Dorminy and Patricia Li

This evening I arrived at my office to write student evaluations. I’ll likely be here until around midnight because I want to get them out as soon as possible and focus on more interesting work – like designing more interesting lessons!

But I’m quickly derailed in my goal. Gustavo Lacerda sent me this article and asked for my thoughts.

Unfortunately I have a number of thoughts on this article and feel compelled to be an insufferable blowhard blog them.

My first thought is that the post/article needs to be longer because I don’t feel like the subject is adequately explored (or even defined well). Maybe. Or maybe it’s fine as it is – short enough that more people will read it and think about it. But the points discussed are important. After all, they point toward the design of our educational system, the mental health of our children, and possibly billions or tens of billions of dollars misspent in the educational system. Not that Gray takes the time to highlight all of these points, and maybe he doesn’t need to.

But I need to. Like a compulsion. Because I love trying to solve these kinds of education problems. I love optimization problems. So let’s dive in.

The article begins with a critique of an educational system that has packed more and more into the days of elementary school children. I strongly agree that children should not spend all their time at school work. We are a long way from developing a curriculum for life better than childhood play and socialization.

But I’m not going to pretend that the educational system developed purely as a mechanism for directed play and socialization. That such a system might be better than our doesn’t mean that we’ve explored the best possible world!

And that may be my biggest gripe with this article – an article that I do praise for (re)raising important issues. Yes, we put mathophobes in charge of…teaching math to the children. Yes, exactly. Right out of the gates, the damage is done. Then those children grow up and run the educational system reinforcing a vicious cycle. Then the article, along with the research it reports on, make one enormous leap – reform the system by dropping arithmetic altogether! As if no other solutions are possible!

I admit that other solutions are difficult, if not simply unviable, but I’ll come back to that.

It is simply incorrect to say that Benezet’s experiment removed arithmetic from the curriculum. As I see it, Benezet’s experiment changed the following variables:

• Arithmetic was taught contextually, based on topics in which the students already showed interest.
• The teachers acted as social intermediaries instead of…machines of dull hate.
• Though the article did not say as much, both students and teachers likely did less work.

The third point is not so minor as it may seem. After all, the article begins with the supposition that students are overworked. It then transitions into a picture of teachers of elementary school students as disinterested in math if not terrified of it. In other words, there is a mental health component.

Under this reinterpretation of Benezet’s experiment, I am happy to discover that I follow Benezet’s philosophy to the extent that I teach EVERYTHING contextually. Okay, maybe not every little thing, but I teach contextually as much as possible. I tell stories. I encourage students to tell stories. When students get stuck with problems, I make up a story in which the approach to the storyline problem illuminates the problem they’re working on. I teach base numbers by asking students to image how we’d pack widgets to send them to Springfield, where the Simpsons and all those eight-fingered people dwell. I teach the locus of points equidistant from two intersecting lines (angle bisector) by having the students imagine that the universe is reflected over a line, then asking what happens if I drop them to the ground from a point on the reflection line. I have fun and make jokes and encourage them to do the same.

But I do teach advanced to very advanced math students, and they do need to learn the notation and formality along the way. I just make sure that conceptual understanding and joy of the subject matter take center stage.

I’d also point out that while some of the parents of my students pressure me to give more homework, I resist. I give some homework – enough that I know that the students are exploring some hard problems and getting comfortable with the mechanics. But I give very little to my middle school students during most of the year.

The students who really want to achieve some measure of perfection with their skills can and do work more problems. No external pressure required.

So, here’s where I stand with (and part from) Benezet. Yes, math education is done so poorly at the elementary school level that it’s more beneficial – at least to very many students – to get rid of the dull exposition of arithmetic by mathophobes. That’s different from not including arithmetic in curriculum – it’s just “taught” at a contextual level. So long as the vast majority of elementary school teachers are mathematically illiterate, this solution is fine with me.

And my fiscally conservative side loves the fact that we might even save money by reducing the number of hours in the school day!

But let’s not get carried away! One size never fits all. What about the kids who are wired to find math to be the most amazing art in all this great big universe? You can probably tell from the construction of that last sentence that I’m talking about me. Math – weeeeeeeeeeeeeee! Math was my play time at school. I like music and I liked it then, but singing in choir got old much quicker than learning how to count quickly through numbers that were 3 more than multiples of 7. Recess was great. I was a pro at running over tires half-buried in the sand and leaping off swings from twelve feet in the air. But I also liked computing the sales tax at the grocery store before the cashier would ring it up. I craved more time to explore and think about mathematics. And I truly loved those occasions in elementary school when a teacher taught me some math I hadn’t learned from my father or discovered myself. Both of them!

To some students, math is play. And that’s both important to mathophiles who’d like to also be represented in the conversation, but important to a world so complex that we need highly trained mathematical minds to engineer all the Asimovian magic around us.

Gifted class saves some kids from missing out on interesting math. Math team starting at the middle school level catches a lot of kids and inspires them. But even before that, we can give kids a taste of math as an art, and see who gets hooked.

This is the solution to the Mathematician’s Lament!

And even if only a portion of the students want class time in that art called “mathematics”, we absolutely do need to provide for it.

After all, haven’t we discovered that you can’t be Tiger Woods if you start playing golf in middle school, Lebron James if you start dribbling at the age of 11, or Venus Williams if you haven’t handled a tennis racket by kindergarten? Should we expect the art and sport of mathematics to be any different? Should we hold back Terrance Tao from explorations until he’s in the sixth grade?

It’s hard for me to defend such a position.

Addendum (because I’m so sleep deprived that I just forgot to say it):

So, no, I can’t support the idea of making Benezet’s solution universal.  I just think we need to make elementary math into something like art class.  And, just like art, let the kids who are great at it have the resources to take them as far as they’d like.  I think this can be done without needing more than a few elementary school teachers to have a good sense of math and math education.

And now it’s nearly midnight and I have successfully found a way to enjoy my time at work. I’ll get my evaluations done tomorrow and I’ve had a more fun and inspiring night for having thought this through. Thanks for reading.

My friend Alexis posted this TED talk on FaceBook a little while ago.

While this particular subject is jazz and freestyle hip-hop, I’ll make the leap to mathematics.  It’s one thing to memorize and perform a song.  It’s another to create one.  It’s one thing to memorize a formula or mathematical circumstance.  It’s another to create one.

I don’t want to say that memorization is not a legitimate skill.  It is.  But creativity is a profoundly more complex and valuable skill.  Typical school curriculum and even some of the math competitions promote the skill of memorization.  My goal is to foster mathematical creativity.

After reading about Bem’s experiments into precognition, my wife Amanda and I were discussing what we felt was likely wrong with the study.  So, she pulled up a copy of the study.  We had a thorough laugh of it.  She wrote an article summarizing her thoughts.

A few days ago I read about this:

Parapsychologists have made outlandish claims about precognition – knowledge of unpredictable future events – for years. But the fringe phenomenon is about to get a mainstream airing: a paper providing evidence for its existence has been accepted for publication by the leading social psychology journal.

My thoughts are similar to these here:

I don’t believe a word of it because a) let’s face it, it’s about precognition and b) there’s simply no effort to propose a mechanism that might support such an outrageous claim (and no, ‘quantum mechanics’ is not a mechanism).

That may sound like a terrible confirmation bias.  But I have similar biases as to whether or not my nephew’s army men are going to suddenly come to life like Liliputians invading Homewood.

But I’m hoping Bem is right.  If collectively, people can guess future events at around a 53% rate, then either all people have the gift, or some people have it more strongly.  I’m guessing the latter because in my experience people’s abilities or judgment to apply their abilities varies.  That would be we can weed out the ones who do poorly and get an even higher rate of precognition.  Maybe in the mid-50′s or higher.

Naturally, I started thinking about which casino I’d take down first with such a cadre of psychics.  Craps is a game which, played properly, gives the house a fraction of a percent of edge.  Some casinos allow bettors to play odds behind their bets which can reduce the edge.  Even a little bit of precognition is enough to turn the tables.  Certainly 53% is enough, and with the higher quality psychics…I could imagine a way to revolve between tables and venues so that we can make millions before getting noticed.

We can mix in some black jack without even counting cards, which means we can revolve so many people into the game that the casino would never notice.  In fact, the whole adventure would be a lot like…just gambling a bunch.  How could the casinos ever complain?

Who’s in?

Faked Research?

On a more serious note, I wonder how such research could be faked.  It’s hard to fake research like this — it takes a creative mind that understands mathematics or computers and how to shield participants and other researchers from the rigging procedure.  I suspect Daryl is very clever.  After all, one of his children was a gold medalist at the International Mathematical Olympiad (we were at the training program together for two summers) and works/worked at Google.  If somebody could fake research like this even after some review, it’s somebody with high mathematical capability, which Daryl likely has.

There are already several failed attempts to replicate the research.

Were I a professor at Ithaca, I would want to look into the research myself if for no other reason than to protect the reputation of my community.  Any time faked research moves the scientific community in a new direction, even for a brief period of time, it undermines trust in science and in scientists.

But perhaps the research wasn’t faked and is still wrong.  After all, statistical significance cannot ever rule out outlier statistics.  Maybe Bem was just the luckiest scientist ever…who happened to be researching in an area most often described as fantasy.

It’s tough to build a strong foundation in the face of erosion.  That’s what I feel like I’m doing in Alabama at times.  In one year, four of my most talented students who have together many first place awards, have moved away or are moving soon — including three who qualified for the AIME as middle schoolers (the other is only a rising 7th grader).  Yet Alabama math team continues to get stronger at a substantial pace because a small number of hard working students is still much better than keeping all the talent in a culture where only a few students are guided toward achieving their potential in rigorous subject areas that involve mathematics.

At the end of this most recent school year I was quite exhausted.  ARML and the attitudes of my students has already restored a lot of my energy.  This coming school year even more will change for the most motivated students in Alabama.  I am finding more ways to get the most motivated students together to learn more, and hopefully enjoy themselves while they’re at it.

There are a lot of things that I look forward to next year: seeing if the MathCounts team improves so much for a third year in a row, seeing if the ARML team can improve more than any team at ARML, seeing the extremely competitive Algebra divisions (I and II) in Alabama math team, and seeing which students find their rhythm.

Sometime after I began reading his blog, I met Patri Friedman through mutual friends when I lived in California.  He was working for Google at the time.  He blogged a number of times about how seasteading might be a hope for better government worth investing in.  From what I can tell, Patri and I have somewhat similar views on government as a form of technology (social software?), and we both feel that great improvements can be made — particularly in regards to government encroachment on liberty which has all kinds of effects even outside of a fundamentalist view of basic or natural rights (if you buy in to the construct of natural rights theory, which I often do “as a temporary organizational good” but not “absolutely”).

When Patri founded the Seasteading Institute, I was critical.  The idea intrigues me, and I hope that it works out well.  After all, I have similar values in what he’s trying to achieve for society.  But I am cynical that government would simply coopt the seas the moment seasteaders began to compete (with government) as distributed engineers of society.  Since then I have seen a number of famous economists and bloggers make similar and other criticisms.

Criticism from ones peers is a good thing, and anyone giving a grand experiment a try should be very happy to receive criticism.  If there were no good criticisms of an experiment, it probably would have been tried already.  I haven’t communicated with Patri in several years now, but I hope that he views critiques of his movement this way.  I hope that he constantly refines his approach as he churns other people’s thoughts over in his mind.

Although my criticism of seasteading is based on its chances of success, I don’t necessary think that it’s a bad thing for Patri to be doing something that I believe has a modest probability of success (at best).  Suppose that you could invest a billion dollars in research that would have a 1% chance of resulting in a source of energy that costs half of what current sources costs and pollutes less.  Would you take that gamble?  I certainly would!  The expected value of the result is in the trillions (perhaps in the hundreds of trillions of dollars worth of resources or much more in the long run).

Experimentation is vital to progress.  This is one of the reasons why I prefer a government that is federalist in many regards — because (at least in certain areas) the states can be wonderful laboratories for government experimentation (so long as the voters pay attention to the results).  This is the way I think of scientific experimentation as well — we must invest wisely, but we must certain invest even in low probability results, so long as the expected payout is high enough to pay for the cost of investment (and volatility of returns).

Over the past three years or so I’ve seen an increasing number of articles written about seasteading, mostly centered around the Seasteading Institute.  In some cases I’ve known the authors of these articles and I’ve discussed seasteading with them and with many of my friends (including again mutual friends of Patri’s).  I don’t recall every hearing anyone say that they think that there is a better chance of success than failure.  Yet, there is clearly great respect for Patri’s work.  Because smart people do value great experimentation.

And if he succeeds, he’ll be a hero.  And he’ll deserve to be a hero.  But what smart people know is that success or not, it is experimenters of all varieties who are collectively heroic as they sacrifice for their experiments.

One of my personal heroes is Richard Feynman whose explanations of math, science, and life in general strike as more clear and well-reasoned than those of just about any human being I’ve ever read. His book Surely You’re Joking, Mr. Feynman! (Adventures of a Curious Character) is one of the few books I have read and reread as an adult (I am a slow, dyslexic reader). One of the chapters in that book is called Cargo Cult Science, and to give an idea as to the topic, here are some excerpts (which link to a page containing the entire chapter):

…I meet lots of people who sooner or later get me into a conversation about UFO’s, or astrology, or some form of mysticism, expanded consciousness, new types of awareness, ESP, and so forth. And I’ve concluded that it’s not a scientific world.

Most people believe so many wonderful things that I decided to investigate why they did. And what has been referred to as my curiosity for investigation has landed me in a difficulty where I found so much junk that I’m overwhelmed. First I started out by investigating various ideas of mysticism and mystic experiences. I went into isolation tanks and got many hours of hallucinations, so I know something about that. Then I went to Esalen, which is a hotbed of this kind of thought (it’s a wonderful place; you should go visit there). Then I became overwhelmed. I didn’t realize how MUCH there was…

…I began to think, what else is there that we believe? (And I thought then about the witch doctors, and how easy it would have been to check on them by noticing that nothing really worked.) So I found things that even more people believe, such as that we have some knowledge of how to educate. There are big schools of reading methods and mathematics methods, and so forth, but if you notice, you’ll see the reading scores keep going down — or hardly going up — in spite of the fact that we continually use these same people to improve the methods. There’s a witch doctor remedy that doesn’t work. It ought to be looked into; how do they know that their method should work? Another example is how to treat criminals. We obviously have made no progress — lots of theory, but no progress — in decreasing the amount of crime by the method that we use to handle criminals.

Yet these things are said to be scientific. We study them. And I think ordinary people with commonsense ideas are intimidated by this pseudoscience. A teacher who has some good idea of how to teach her children to read is forced by the school system to do it some other way — or is even fooled by the school system into thinking that her method is not necessarily a good one. Or a parent of bad boys, after disciplining them in one way or another, feels guilty for the rest of her life because she didn’t do “the right thing”, according to the experts.

So we really ought to look into theories that don’t work, and science that isn’t science.

I think the educational and psychological studies I mentioned are examples of what I would like to call cargo cult science. In the South Seas there is a cargo cult of people. During the war they saw airplanes with lots of good materials, and they want the same thing to happen now. So they’ve arranged to make things like runways, to put fires along the sides of the runways, to make a wooden hut for a man to sit in, with two wooden pieces on his head to headphones and bars of bamboo sticking out like antennas — he’s the controller — and they wait for the airplanes to land. They’re doing everything right. The form is perfect. It looks exactly the way it looked before. But it doesn’t work. No airplanes land. So I call these things cargo cult science, because they follow all the apparent precepts and forms of scientific investigation, but they’re missing something essential, because the planes don’t land.

Feynman, not one to moralize without first sucking his audience in with a good story, appropriately placed this chapter last in the book. I suspect nearly everyone who first picks this book up finishes every page.

Making Education Scientific

Not to be confused with scientific education, Feynman introduces us to the possibility that education is a process that can be improved by science. Yet we seem to do very little to replicate positive results. When top math team programs spring up around the country, whose students get rapidly swept into elite universities by the droves, these programs and their methodologies get little attention compared to programs backed by politicians or teachers unions. Instead of standing up for more reasonable programs, most people follow along with the witch doctor remedies, never knowing better.

Here’s a set of questions just to display the lack of scientific inquiry:

How did Pizitz Middle School and Mountain Brook Junior High raise a batch of students from an area that covers only a few square miles, that in 1991 beat the entire state of New York, the state of California, Massachusetts, and the rest of the states at the highly competitive national MATHCOUNTS competition? Why does nobody try to study that success for replication?

Why does Mountain Brook Junior High no longer compete in most math competitions?

When a Wall Street whiz retires to pursue life as a teacher, and in a few years has his Bucholz High School (Florida) team near the top in national competitions, why does nobody try to replicate the process of hiring professional math whizzes to run math programs?

When a single school system in Texas can defeat the entire state of California (and all the other states) in a competitive national math competition year after year, why isn’t there a media frenzy covering the teams of scientists trying to determine exactly what makes Jeff Boyd’s program different?

The simple answer is that politics get in the way of good education, which is true and obvious, but equally true is that too few people have a good enough understanding of what science really is in order to interpret the world scientifically. After all, it’s this human weakness that beaurocrat’s exploit. And so any educational institution too closely entwined with the witch doctors seems to stagnate.

Experiment. Observe. Refine. Play a role by taking a scientific view of educational results.

One of the biggest problems in education is that programs and methods of instruction are rarely tested to determine levels of effectiveness. My hope is to create some clear benchmarks from which to judge the success of MIST Academy, but also to design testing scenarios from time to time and see how students perform.

Currently, my Introductory C class is about a third of the way through an advanced geometry curriculum. I am very proud of the group of thirteen students, all middle schoolers, who have worked hard in this class in addition to their regular school work. I suspect that for many of them, this experience will serve to soften the increased level of work and expectations they will meet with in high school, and broaden their mathematical horizons.

A Modest Testing Experiment

During the first eight weeks of class, I have tested the students twice. It is unfortunately very difficult to design a standardized test for highly advanced students, so I decided to use tests from the Mu Alpha Theta national convention. Of course, it is impossible for me to know which of the two tests is truly more difficult, so I designed a testing experiment as follows:

I divided the students into two equal groups of 6 (the 13th student was late to class the first day, leaving an equal number for each testing group). Call the first group of students A, B, C, D, E, and F. Call the second group U, V, W, X, Y, and Z.

During the first testing session, I gave the first group Test 1, and the second group Test 2.

During the second testing session, I have the first group Test 2, and the second group Test 1.

Each test is 30 problems of multiple choice (A – E answers) scored 5 points per correct answer and 1 point per problem skipped (so guessing had no expected advantage or disadvantage).

Between testing dates, the students had three weeks of instruction on course materials, which amounts to around 15% of the advanced geometry curriculum we are covering. Students were not allowed to keep their exams (until after both tests were complete) and were instructed not to discuss any of the problems.

Here are the results listed as Student (Test 1, Test 2):

A (100, 132)

B (56, 61)

C (44, 78)

D (54, 84)

E (94, 128)

F (74, 88)

U (82, 49)

V (40, 38)

W (89, 100)

X (40, 41)

Y (39, 43)

Z (94, 118)

Remember that half the students took each test during each testing period.

Results

The average score on the first test was 67.17. The average score on the second test was 80. So, the second test should be regarded as significantly less difficult than the first.

The average score during the second testing session was 12 points higher than during the first testing session.

Adjusting for difficulty by adding 12.83 points to each score on the first test (a debatable method, but reasonable for modest purposes), only two students failed to increase their scores from the first test session to the second test session. Student B missed two of the three classes between testing. Student Z took the easier of the two exams first and missed one of the three classes. Every student who attended all three classes improved in score. Removing students B and Z from the sample, the average improvement of students missing no class was 16.3 points, with no students failing to improve.

Interpretation

I personally see these results as significant. There was near across-the-board improvement by students over a relatively short period of time. The average increase was 10% of available points (30*5 – 30*1 = 120 point spread between a perfect score and the score of a student who submits no answers) during a span of around a month (three classes between testing) covering 15% of the curriculum. At this point, on average, these middle schoolers are scoring on par with the high school juniors who took those exams at the Mu Alpha Theta national convention.

In addition to seeing overall improvement in student ability, my own interpretation is that the students who are working the hardest generally improved the most. In particular, student U has been highly participatory during the three weeks of instruction and improved dramatically, scoring much higher on the harder test during the second testing period. Student C seems like a consistently hard worker and made vast improvement. Student E seems to be working harder lately and improved greatly. Students A and D are consistently high performers and have good work ethics.

All these students are very talented, but it is clear that the ones who put in more effort get more out of the class.