A college is on the market. That’s rare and usually it only happens when the school's model doesn’t work. Hampshire is a fixer-upper in terms of management, but it’s got great bones. I should know, because I spent four years as a student there and two as a professor. If you care about credential inflation, colleges that optimize for rankings instead of learning, and the collapse of trust in higher education, this is your chance to help.

It’s not every day that you can save a campus and an institution that would cost $700M+ to build from scratch, for the measly low price of ~$175M. We have a once-in-a-generation opportunity to buy a 800-acre campus for a fraction of replacement cost. You cannot build this again, at any price, on any timeline. Try getting the permits to assemble 800 contiguous acres in Amherst today.

The deadline is September 2026, Hampshire’s tender date, when it would be on the hook for buying its bonds back. After that, the situation becomes significantly more complicated. The discussion about selling the campus is already underway, and things may start happening in a matter of weeks. Now is the time to move.

We can relaunch Hampshire with the financial foundation the original institution never had, and prove that this model is viable when properly managed. Don’t think of it as rescuing a failed institution. This is about building a new institution on the most valuable real estate in experimental education. This is also for parents who worry about watching their kids disappear into the credential machine. Hampshire is the alternative: a bet that your kid is ready to do original research, and shouldn’t waste their youth on grades and transcripts.

If you have capacity at the $10M+ level and this interests you, email me at <ethanludwinpeery@gmail.com>. But first, read on:

What Hampshire Was

I was a student at Hampshire College for four years, from 2009 to 2013. I went on to get my PhD at NYU, after which I went back and spent two years as a professor at Hampshire, teaching cognitive science from Fall 2022 to Spring 2024.

Hampshire was founded on the premise that the standard model of higher education is actively hostile to real learning. Grades, tests, required courses, and academic departments are more than just dead weight — they make things worse. So Hampshire threw them out.

Instead of letter grades, students at Hampshire receive written evaluations, sometimes a page or more. Grades can be gamed with last-minute cramming or strategic test-taking, but evaluations reflect whether you have actually learned the material, as judged by the person who taught it.

At a normal school you can get a B+ by showing up and being reasonably smart, and you can often get a good grade simply by avoiding attention. A Russian friend recently told me about her experience in university. “Students are focusing on points instead of gaining knowledge,” she said. “On our ‘team work’ guys pick the simplest topic, and I said, ‘let’s pick something challenging, this is too easy’. And they said, ‘well yeah, that’s the plan, to have A and do nothing’.”

Grades encourage this nonsense, but written evaluations make that strategy impossible. Faculty are likely to write, this student chose the simplest topic while everyone else in the class picked something challenging.

There are no quizzes or tests, so there’s no cramming and no pointless questions about “will this be on the exam”. Instead, students read primary sources and do projects to build up their skills. At most schools a senior thesis is exceptional, but at Hampshire, it is the minimum. Every student has to do a year-long senior thesis in order to graduate.

Hampshire has no majors, no departments, and no required courses. You can’t coast, and there is nowhere to hide. You can’t just take the courses as demanded by your department or major. Instead, your curriculum is a negotiation between you and your faculty advisor about what your education should actually contain. There is a lot of freedom, but it is structured freedom. Students who thrive at Hampshire are a specific type who find conventional academic structures frustrating because conventional coursework moves too slowly.

The hands-on learning that is promised at other schools is actually real at Hampshire. It is one of the few colleges on earth that still has a fully equipped machine shop that any undergraduate can walk in and use.

People think of Hampshire as artsy and activist-y, but the arts-and-film narrative the school leaned on is not remotely the full truth. You’ve probably heard of alumni like documentary filmmaker Ken Burns, or actor Liev Schreiber. I also think of alumni like Eugene Mirman, voice of Gene Belcher in Bob’s Burgers and author of the greatest commencement speech of all time.

But Hampshire also has a remarkable tradition of entrepreneurship. Per official metrics, one in four Hampshire graduates go on to start their own business or nonprofit. You probably recognize some of the founders from their companies, like Stonyfield Farm Yogurt (founder Gary Hirshberg) or Duolingo (founder Jose Fuentes). You may have encountered jackets from “superhuman streetwear” brand Volante Design, but you probably didn’t know that it was founded by two Hampshire alumni, David and Willow Volante, who met at Hampshire.

Hampshire’s history of producing astonishing STEM research opportunities has never gotten enough credit. Sure we have alumni like Academy Award-nominated singer-songwriter Elliott Smith. But we also produced alumni like space plasma physicist Cynthia Cattell, theoretical physicist and popular science author Lee Smolin, founding member of Ubuntu and Debian Benjamin Mako Hill, and AI researcher Gary Marcus. Since you’re reading this on Substack, there’s a good chance you’ve also heard of writer, neuroscientist, and Hampshire alum Erik Hoel, and his Substack The Intrinsic Perspective.

This is not a throwback to a brighter age: Hampshire continued producing amazing STEM graduates right up to this year. One of my former students was always coming to my office to show me her microscope work, despite the fact that it was much too advanced for me to understand. She was consistently at the top of the biology classes she was taking at Smith and UMass, and her senior thesis was applying microscopy techniques to examine alpha-crystallin B, a protein associated with Alzheimer’s disease. I know some of those words. This is a real Hampshire grad and you can hire her right now.

This isn’t to say that I look at how Hampshire has been run and would rather that “STEM-types should be in charge”. If hippyism is an excess of right-brain thinking, STEMlordism is an excess of left-brain thinking.1 Something like “renaissance humanism” is the more interesting synthesis, and you see a lot of that at Hampshire. Take me for example: I studied cognitive science, but I spent enough time in the shop that everyone thought I was studying blacksmithing.

“When I first came to Hampshire, I thought I was gonna do writing and sculpture,” reflects Nicole DelRosso, “but I wanted to take a neuroscience class, because in my high school I didn’t have access to interesting courses like that. I was just dying to know more about the brain.” She switched to studying neuroscience, and six years later she won a Fulbright scholarship.

Or take my friend Talia Adi: “I knew I wanted to study neuroscience,” she said in a recent interview, “but I had trained as a ballet dancer and wanted to have the flexibility to pursue a multidisciplinary education. … I was also able to continue dancing as part of the Five College Dance Program and even took a year off after my first year at Hampshire to move to New York and dance full time.” After graduating she worked at a lab at Yale, and is now on the cusp of finishing her M.D./Ph.D. program.

People love Hampshire despite its many flaws, and they give up ostensibly better opportunities to go there instead. I recently saw a good example in Christopher Benfey’s New York Review of Books piece A Clearing of the Ground, where he shares this story about the early days of the school:

Those who were there at the origin never forgot the heady excitement of the place. My older brother turned down a Yale scholarship to enter Hampshire’s second class in 1971, where he read Chaucer, built a pottery kiln, and took off after a year for a trip around the world before becoming a prominent molecular biologist.

Hampshire has had a high transfer rate, especially in the last decade, but there are also many students who transferred to Hampshire. These students had other options, but rejected that path to finish their studies at Hampshire instead.

One of my good friends at Hampshire started out as a Mount Holyoke student, but she kept coming to more and more Hampshire events, started taking Hampshire classes, and eventually decided to transfer. More recently, I advised a student who had started at Clark University, but found himself disappointed with the slow pace of his classes there. So he transferred to Hampshire, where no one would keep him from working as fast as he wanted.

Finally, I should mention that Hampshire is part of the Five College Consortium, so students can take classes at Amherst, Smith, Mount Holyoke, and UMass. This means access to one of the richest academic ecosystems in the country, while still doing things none of those schools are ambitious enough to try.

Elite Human Capital

Back in 2005, Paul Graham wrote that undergraduates are undervalued. Not on average, because the average 20-year-old is not very capable, but in the spread:

The most productive young people will always be undervalued by large organizations, because the young have no performance to measure yet, and any error in guessing their ability will tend toward the mean. What’s an especially productive 22 year old to do?

His advice was to start a startup: go around employers entirely, start a company, let the market judge you directly.

I basically agree, but this advice only works for aspirations that fit neatly into the market. If you have a 20-year-old Darwin, you don’t want him trying to start a startup. You want him joining a community of scholars, finding mentors, and doing the slow work that compounds over a lifetime. You want him planting seeds that will take decades to bear fruit.

Even technologists and entrepreneurs are rarely ready to do this at 20. Meeting your co-founder at college and dropping out together is a classic for a reason, the college part still did something. Those who are 100% ready should probably just go start their startup and skip college entirely. But those who are 80% or 90% ready should have a place to go that takes them more seriously than the average school treats the average undergraduate.

This brings me to what Hampshire actually was at its best: a magnet for an unusual kind of student talent. My friend Adam Mastroianni, who went to Princeton and then to Oxford and then got his Ph.D. at Harvard where he was a resident advisor, certainly has seen a lot of talented students. But when he visited my class and talked to a handful of my students, he came away genuinely surprised.

“I pretty much had my pick of the most talented undergrads in psychology from across the country for years,” he told me, “and didn’t find people who were as interesting to talk to.” But what really struck him was their initiative. When given an opening, my students immediately asked how they could help, what they could learn, what they could build. “Is every Hampshire student this high agency,” he asked me, “or is it just the three that I talked to?”

“Yes they are really like that,” I said. “They are all like that.”

Paul Graham mentions that what surprises him most about undergraduates is how conservative they are. “Not politically, of course. I mean they don’t seem to want to take risks. This is a mistake, because the younger you are, the more risk you can take.”

This explains why Hampshire’s selection effect is so strong. Most college applicants are deeply risk-averse; they’re optimizing for credentials, safety, legibility. Hampshire is beautiful because the entire model filters that out. Choosing a school with no grades, no majors, and an uncertain future is about the riskiest move possible at 18. You’d have to be totally bold or wildly ambitious to take that choice. You’d have to not care about collecting credentials at all.

The students who end up at Hampshire demonstrate something important about themselves, just by showing up. Adam was surprised because he spent years working with student pools that were filtered for achievement, but Hampshire filters for something much more rare and hard to measure.

Subscribe now

Why Higher Ed Needs Exactly This Right Now

Higher Ed in the US sucks right now, and its greatest weaknesses have historically been Hampshire’s greatest strengths. A revived Hampshire College, with intelligent leadership, could punch hugely above its weight.

Grade Inflation

Grade inflation has reached the point where the median grade at many elite universities is now an A, making transcripts nearly useless as signals of actual ability or effort. When everyone gets an A, the grade stops meaning anything.

Narrative evaluations, which Hampshire has been doing for 50 years, already solve this problem. Grades are hackable because they’re produced by a known algorithm, which the professor gives you in the syllabus on day one. Once you understand the rubric, you can optimize for getting the highest grade, instead of learning the most material or developing the deepest understanding. In fact, students who prioritize real learning or understanding get worse grades than their classmates who just focus on gaming the system.

Narrative evals can’t be hacked because there’s no rubric to game. You can still butter up the professor, but at least you can’t do it algorithmically. Unlike a letter grade, evaluations require a professor to describe what you did, so they convey real information. When they’re good, they are much stronger than an A. When I was a student, one of my evals said, “Ethan writes like a second-year grad student”. You can imagine how that looked when I was applying to grad school.

When they are bad, they are much worse than a D. I never got a “bad eval”, but in my third year when I took Cognitive Psychology, I skipped the readings on one single day of class, and Jeremiah “Jay” Trudeau called me out for it in my evaluation. Jay, I want you to know that I think about this comment to this very day. He also called my final experiment proposal “a little dry”. You’re killing me here man.

Project-based learning solves this even more neatly than evaluations do. If an evaluation conveys more information than a grade, the work itself conveys more information than the evaluation. A Hampshire education is valuable not for the degree per se, but for the body of work you created during your time with the school.

AI in Education

AI cheating is rampant in higher education, and worse, it’s surprisingly defensible. If you ask students why they use LLMs to cheat, they’ll tell you that they use AI as a weapon against tedium and busywork. It’s hard to blame them when they know that no one cares about their five-paragraph essays. The standard conversation around AI — “How do we catch students, how do we stop them?” — makes everyone paranoid and antagonistic.

Without grades and busywork, LLMs stop being such a threat, and you can completely sidestep the endless arguments about cheating. It puts you in the nice position of not having this faculty-student war. Using ChatGPT to do derivative, passable work makes a lot of sense when you are aiming at a rubric, which is why students at traditional schools do it all the time. It doesn’t make sense when you don’t have grades. Submit something mid, who cares. Your eval will say, this student’s writing was mid.

Hampshire doesn’t need to be an “anti-LLM police state”, nor does it need to be a handwavy “AI-first education”. It can just be what it has always been, a place that asks students what they’re passionate about and what difficult, ambitious things they want to make. That was a good idea in 1970, and it looks even better when every traditional institution is tearing itself apart over ChatGPT.

Practical and Intellectual Skills

Conventional education produces students who are helpless, both intellectually and in the physical world. My Irish exchange student once told me, “I love how you guys do classes with a lot of debate. Even when I started my bachelors back home it was more drilling of ‘knowledge’ and barely any room for debating or deep discussions.”

The practical side is no better. At NYU, I discovered that most students couldn’t identify a goose. As part of a lecture on categories, the professor would put a goose on screen and ask what it was, and students would say, “a duck.” I had NYU students who saw me eating carrots and were concerned for my health that I didn’t peel them first. These students weren’t stupid, but their education had kept them so far from the actual world that basic reality had become unfamiliar.

Before grad school I worked at Harvard for a few years. One of our RAs was a 4.0 Harvard senior, very talented kid. But at one point we learned that she couldn’t cut limes for drinks at her graduation party, because she didn’t know how to use a knife. Her parents never let her.

In contrast, Hampshire students make knives. Sometimes professionally, like Emiliano Carrillo, who you may have seen on season 3 of Forged in Fire. They can fix a power hammer, they can lay a bead, they can cut a lime. (Specialization is for insects.) They can even sell hot dogs out of the window of their first-floor dorm room. The senior thesis that every Hampshire student has to complete to graduate demands that you do something real, even if it sucks. Not write about something real. Do it.

Better Teaching Is Available

The higher education market is more distorted than most people realize. Schools hire professors based on prestige and arcane internal politics, not teaching ability, and application processes are stupidly onerous. The result is thousands of talented PhDs either unable to find work or trapped in grueling back-to-back one-year contracts. This is concerning, but also a tremendous opportunity. A school that was committed to great teaching and advising could have its pick of exceptional people.

For example, Dr. Bret C. Devereaux of the blog ACOUP, in his wonderful Academic Ranks Explained Or What On Earth Is an Adjunct?, says:

Most adjuncts in order to make ends meet have to stack multiple heavy course-loads due to the shamefully low pay they receive, and so while many adjuncts are dedicated teachers they are rarely able to give each class the time it needs. That is compounded by the fact that the short-term nature of adjuncts means they have little freedom in what they teach, since getting a new course ‘on the book’ takes time and is thus impossible for an adjunct with short-term appointments. I have been repeatedly asked by students when I would teach a course on Greek or Roman warfare and the answer is ‘never’ despite tremendous student demand because I am never in an appointment long enough to propose and get approval for the course to be on the catalog.

One of the things that the old Hampshire got right is that faculty were able to teach whatever they wanted. Bret would clearly do better, and his students would be better off for it, if he had the same freedom. Offering that freedom when recruiting faculty would sweeten the deal for them and improve things for the students, who would finally get the courses on Greek or Roman warfare that they so crave. There are thousands of adjuncts with proven track records like this who would be motivated to take a better deal; which is a low bar, since they are treated so poorly.

In short, you could approach whoever you wanted and offer them a very sweet package: You don’t have to apply to the job, we want YOU. Teach whatever you want within your discipline. We don’t care about your publishing or any of that other academic nonsense. Here’s a five-year contract, long enough to really develop your curriculum. As long as the college appears stable enough to last even a few years, many people would take that deal.

You could also hire some professors of practice, like universities used to. These are people who have been successful in a professional field who are brought in to teach, based on their real-world expertise rather than research credentials. A journalist, executive, or electrical engineer might teach as a professor of practice, either part-time or after retirement. They typically don’t have the normally required degree, like a PhD, and they’re not on the tenure track. Or you could hire people straight out of grad school, that traditional institutions are too cowardly to hire. But as always, a market inefficiency is an opportunity for those who are not irrational, and this time, it could be us.

Competition on Model

Almost every other college is interchangeable. The selective ones compete only on status, the non-selective ones compete only on price. There is basically no competition on model.

At every other college, all students are given basically the same experience. There’s some honors stuff, but everything has to be aimed at the lowest common denominator student. If you’re ready for graduate work sophomore year, Harvard and Yale are not really set up for you to do that.

Deep Springs (total enrollment: 26 undergrads) and Minerva (sophomores are in Tokyo, juniors are in Buenos Aires, seniors are in Taipei, faculty are all remote) seem to be the only thing close. And these are probably good comparisons. Both programs seem successful despite many reasons they should not be, probably because they offer an experience you can’t get anywhere else.

In 2024, Genomics PhD Ruxandra Teslo gained some recognition writing about higher education with her piece, The flight of the Weird Nerd from academia. Later she wrote an even more popular follow-up called The Weird Nerd comes with trade-offs, where she offers the following rule: “Any system that is not explicitly pro-Weird Nerd will turn anti-Weird Nerd pretty quickly.”

Hampshire was special because even though the people in charge didn’t always understand the mission, the school was structurally set up so that no one could stop weird nerds from prospering. “No one told you what classes to take, and as a result, none of you know math,” joked alum Eugene Mirman in his graduation speech roast. It’s funny because there’s some truth in it, but it is a joke. I know, I’ve taught statistics.

Double down on that model, and staff it with an administration that is pro-Weird Nerd, and you will see something amazing.

The fundamental problem with higher education? Students want to learn things and they want to do things, and the schools don’t seem to believe that. The “structural changes” that need to happen are simple: schools need to hire faculty who believe that students want to learn, and who have some sense of what students actually want to pursue.

If you read that and thought “quirky electives”, then your intuitions are also wrong. When I taught at Hampshire, the two largest classes at the entire school, so popular they were overenrolled every time, were my courses Introduction to Psychology and Negotiation.

Share

The Vision

The motto of Hampshire College is Non Satis Scire: “to know is not enough”. When Hampshire works, it works because someone is living this motto. It’s not enough for a history student to know how a medieval suit of armor was built, they need to make one themselves. It’s not enough for an entrepreneurship student to know how to design a business model, they need to start their own food truck and staff it with other students.

The unofficial nickname of Hampshire College has long been “grad school for undergrads”, because of its insanely high grad school placement rate (an average 8.4 out of every 100 Hampshire alums earn research doctorates, placing Hampshire in the top 1.4% of colleges nationwide), because every student completes an essentially graduate-level year-long senior thesis, and because the average Hampshire class is the same format and level as a graduate seminar anywhere else. Seriously, Hampshire alumni are often surprised when they take their first Ph.D. seminar, because it’s identical to the classes they were taking at Hampshire, only slower and easier.

If you’re not familiar with grad school, you may think this means, “more classes, and harder”. But in fact, it’s an entirely different approach to education. In undergrad you are there to consume knowledge, which is why you read textbooks and take exams to demonstrate that you’ve absorbed what others discovered. In graduate school you are there to produce knowledge, which is why you read primary sources and conduct original research.

Most PhD programs have only two years of classes — for the rest of your time there, you’re a scholar doing scholarship. Hampshire’s model applies that same logic to undergraduates. Get up to speed as fast as possible, then start doing the real thing.

In recent years, the college has not taken these mottos seriously. Students are encouraged to complete projects, but these are mostly class projects, designed by faculty with little to no input from students. They write papers, but mostly class papers. Every student does a senior thesis, but not until their final year. Where is the school that is ready to take college students seriously as scholars and craftsmen?

Grad School for Undergrads

Therefore, my vision for the revived Hampshire College is a college where students do real scholarly and professional work.

In most colleges, you read textbooks about people who did real work. You don’t do real work, and you don’t even read the primary sources of the people who did the work. Maybe if you are lucky, you get to help with real work when you are in your last semester, as some kind of research assistant or something. I can’t believe we have let this become our dominant model of higher education.

Why would you want to spend all your time reading textbooks? Ugh! And memorizing them? Ugh! You want to get stuff that you don’t know, that anybody else doesn’t know, and try to figure out how to study it and how to think about it. (Hampshire founding faculty member Lynn Miller)

Whatever you study at Hampshire, you should be doing the real version of that thing. Hampshire biology students should be publishing their biology research. Hampshire journalism students should be starting papers, zines, and newsletters. Hampshire entrepreneurship students should be starting small businesses. It could hardly be more obvious, or more ambitious.

When I taught at Hampshire, I tried to do this with my classes. Instead of teaching a traditionally boring psych methods course, I taught Hampshire College Butchers the Psychology Classics, where we replicated several classic psychology studies with only two weeks of time allotted to complete each, and somehow succeeded. These replications were real contributions to the psychology literature, and some of them even had larger sample sizes than the originals.

Most people don’t believe that undergraduates can do this kind of work. They are wrong. Admittedly even I was unsure that my class, mostly not psych students and mostly with no research experience, could do a good job in only two weeks per study. But I was wrong too.

Hampshire advertised this “real-work-first” learning, but especially in recent years, it didn’t always deliver. It worked better for some sections of the student body, like art students, who always got to put on their own gallery show for their senior theses. In other subjects, students would often find themselves trapped in traditional coursework until they learned to use open-ended tools, like independent studies, to break free. Credit to Hampshire for having those tools at all; but in a revived Hampshire, real work should not be something students have to fight for the chance to do. It should be the default.

If you read the New York Times article about the purported end of the college, you’ll see a quote from Hampshire alum and filmmaker Ken Burns, who said that Hampshire offered “sort of medieval guild-like tutors and apprenticeships.”

This is also something we could enhance. Senior students should apprentice to faculty, beginner students should apprentice to the more senior. Again, Hampshire always did this a little, but it mostly did it on a small scale and often did it poorly. The founding faculty did it, but they are now all gone; no wonder Hampshire faded.

If you read this piece about founding faculty Lynn Miller (which you should, it’s amazing), you’ll see how Lynn treated his students as apprentices. See how he describes Jan Term:

We used to have a January term, three weeks long and the perfect opportunity for a lab course. In the first week our students would learn how to use the tools . . . pipettes, centrifuges . . . and how to run gels and handle themselves in a laboratory. At the end of that week, the instructors — I’d have several advanced students as instructors — would talk about their own projects, so students could then switch to working on a project that looked interesting to them. In three weeks we turned out polished, beginning molecular biologists. They could walk into a molecular biology lab anywhere in the country and be comfortable and do good work, right from the beginning. We got wonderful letters back from the faculty they were working with. They said, “Ah, you trained these kids well!” Because they knew how to handle themselves in the laboratory.

Every year there are a few hundred 18-year-olds who are ready to do graduate-level work in their field, and what are they supposed to do? Right now, they have to suffer through four years of undergrad before they are supported in doing real work. One of the only workarounds was for them to go to Hampshire College, where they could start doing graduate-level work without being bothered. The college should be retooled for this specifically.

If you hate credentialism, this is the model for you. Hampshire alums kick ass because every single one of them leaves the college with more than a piece of paper and letter grades that may or may not mean what they say. They all have in hand their senior thesis, an example of what they can do when given a full year to work on a project of their own choosing, which is more impressive (or more damning!) than any GPA. This is why they find it so easy to get into graduate programs, and this is why it would be a shame to let this model vanish from the earth.

Imagine an admissions department built around these principles. Dear talented high schooler. We noticed your original work in your subject area. Very impressive. If you go to most colleges they’ll make you take a bunch of survey courses on subject matter you’ve already mastered, and you will be so bored that your eyes will roll all the way up into the back of your head. Did you ever think about coming to Hampshire, where you could take graduate-level courses in your first year and continue your original work without being bothered?

More Online

Speaking of market inefficiencies, I’m kind of an internet maximalist. The internet is powerful and, in many cases, good. Like, there’s a reason that you’re reading this on a blog. My vision for Hampshire would be extremely online, with a special focus on learning how to share and market your work.

I don’t mean that Hampshire would become a school for influencers. But the internet is more than the future — it is the present, and most of higher education seems unaware of that fact. Past your first or second year, your academic work shouldn’t be produced just for your teachers, your classmates, or even your peers. It should be produced for the world, and the way to share your work with the world is to put it on the internet where people can find it. No matter how wise or educated you become, you can’t fulfill your ambitions alone. Writing about your ideas online is part of how you find your future collaborators, the people you need.

Being very online and publishing your work should be part of the new academic culture. Hampshire could have cultural dominance of the “young people doing real work” brand, which would attract all kinds of new students and new opportunities. If you’re doing projects that translate well to video, you should be putting them on YouTube. If you’re producing interesting writing or analysis, you should have a blog, or ten blogs. Bandcamp, Twitch, Steam, Substack, whatever it is. You should ship it, and learn to ship it well, because shipping your work is a skill.

Similarly, many faculty hiring decisions can be sourced from the internet. I plan to approach the chemistry YouTuber Explosions&Fire (he has an actual PhD) to see if he will be the chair of our new chemistry department.

There are a lot of other changes I would make, like a real student government with powers and responsibilities, and student involvement in the running of the college, like they did at Goddard College (see this Teen Vogue piece for more).

We had some of this when I was a student, though it was gone by 2022. We had student EMTs, and when I was sent to the hospital after my freak blacksmithing accident (blacksmithing is normally very safe), a student EMT was first on the scene. We had an IT department with student IT workers, which a previous administration confusingly closed, despite the fact that working in the student IT department led several of my close friends to get high-paying IT jobs right after graduation. I would revive all of these programs.

I could tell you how we should offer a 5-year BA/MA program — other colleges already offer these, and everyone at Hampshire already does a senior thesis, so why not make the senior thesis two years long and have the tuition go to Hampshire instead of some lesser MA program? But I won’t bore you with the details now.

Share

What Went Wrong

One thing that can get lost in the stories of crisis is that the model was working. Between Fall 2022 and Fall 2024, enrollment grew 68%. Students were finding Hampshire, choosing Hampshire, and staying at Hampshire. I taught dozens of those students. The demand was there. It was going so well that in early 2023, they sent around a “Survey on Ways to Address the Increase in Student Numbers” to all the faculty.

Then in 2024, the administration slashed the admissions department, cutting one third of admissions staff. Budget cuts are sometimes necessary and never easy, but you don’t cut the department that’s producing some of your only good news while the school is fighting for its life. This is just eating your seedcorn. By late 2025, enrollment was back down to 750 students from a high of 833, and at the time of the announcement of the closing, the college was down to 625 students.

This was easy to see coming, which makes the decision all the more confusing. Admissions experts saw this choice and commented, “Without the admission office, there are no students. Without students, there is no revenue. Without revenue, there is no college.” Sure enough, cutting admissions was followed by reduced enrollment and a rapid slide towards collapse.

Some people want to make this a financial mismanagement story, but the truth is that the college missed its enrollment target this year by nearly half. The school is perfectly sustainable at an enrollment of 1000 students, which was the enrollment in 2013 when I graduated. Any financial problems come from falling tuition, which is all downstream from cutting the admissions department, weakening the academic program, and letting the student experience deteriorate.

Faculty are the lifeblood of an educational institution, but Hampshire kept making terrible faculty hiring decisions. Over and over again Hampshire would find good faculty, hire them, then cut them. A good professor can make the difference between a student dropping out and staying, especially at Hampshire, where students work so closely with advisors. But as a student at Hampshire, you could never tell if your favorite professor would be there next semester. Some of my friends had four different advisors over their four years at Hampshire. Faculty who taught courses that were massively popular with the students would often discover without warning that their contracts weren’t being renewed.

For years, students would show up at the college because they heard that it had a great game design program, only to discover that the school could not hold on to a single game design professor. Students heard about the great entrepreneurs among Hampshire alums and assumed we had the faculty to support that tradition. “I was under the impression it was an acceptable school for business majors,” one student told me while transferring away, “but it definitely is not.”

Faculty who could leave usually did, especially when the school nearly closed in 2019. Great professors found new jobs elsewhere. Teaching and advising quality suffered for these losses. Students came to Hampshire expecting a project-based model that would work with their unusual sleep disorder or chronic health issue, only to find themselves in classes with professors who would threaten to fail them for missing class during a two-week hospital stay.

Governance Model

My friends who have read drafts of this piece all ask me some version of the same thing: if you brought Hampshire back, how would you organize the new version of the school, and how would you make sure the same thing doesn’t happen again?

But as serious as it might make me look to have a “governance and operating model” appendix stapled to this blog post, I just don’t believe that having 7 board members instead of 9 board members will make a material difference, either in terms of fundraising or in terms of long-term success.

It’s worth having a theory of what went wrong, because without one it’s hard to understand how things went from working to not working, and even harder to know what would stop it from happening again. In many institutional failures, there’s a story that makes sense. Tough decisions under uncertainty, reasonable people who turned out to be wrong. Everybody thinks they are acting reasonably, so what did they think they were doing?

But the truth is that sometimes people just make bad decisions. And as best as I can tell, that is what happened at Hampshire.

Hampshire consistently had a balanced budget before 2019. Looking at Hampshire’s 990s, 2013 net tuition was $36.3M, or ~$51.5M adjusted for inflation. In fact, Hampshire had surpluses from 2013-2016.

Then in 2018 the board hired Mim Nelson as president, who pursued a secret merger with UMass, keeping the board and community almost entirely in the dark about what she was doing, until her surprise announcement in early 2019 that Hampshire would stop accepting new students while seeking a “strategic partner.” This led to a collapse in enrollment. Students who were already at Hampshire transferred out in droves, and tuition revenue disappeared with them.

If you went back to 2016 and tried to save the college, the advice would be pretty basic: don’t hire Mim in 2018. If that somehow happened anyway, at least don’t send what one faculty member described as an “utterly bizarre letter” to the 77 already-accepted students warning them that by fall the college might have no dining hall, no dorms, few faculty, and none of the services one could usually expect from a liberal arts college. Because as that same faculty member put it, the letter “said to us that not only was the college discouraging an entering class, but, having seen that letter, it seemed to be a spur for all the rest of the students to transfer, get out, as quickly as they possibly could, absolutely eliminating, or virtually eliminating, our revenue base.”

You can infer a lot about the board’s judgment from this one letter. The good news is that the president, the board chair, the vice chair, and “several other trustees” resigned, so the board since 2019 has not been the same board that made these decisions. But the college has made many confusing decisions since.

The school could simply not have sent such an apocalyptic letter, just like they could have chosen to not eliminate the IT department in 2023, or not cut ⅓ of the admissions department in 2024. Maybe these ideas made sense in some context, but if so you would have to read pretty hard between the lines, because the college’s justifications for such decisions don’t hold up to scrutiny.

I do think some structural things should change. Having faculty be in charge of faculty hiring decisions seems like it caused a lot of harm last time. When faculty have total choice over their membership, the school eventually comes to serve the needs of the faculty over the needs of the students, which leads to collapse. I wouldn’t give students total control of faculty hiring decisions, but you would need a more open process.

But writing out all those details now is like picking the paint colors for the new buildings we’ll name in our top donor’s honor. If there are funders who are interested, I would need to hash those details out with them, so why write the bylaws before I know who my business partners will be? I believe the vision matters the most, because the vision will attract (or repel) future students, staff, and faculty, and good people are more important than anything else.

The Runway

The bottleneck on this idea is simple: it would require a lot of money. By my estimate, around $175 million (for details, see the appendix).

We would need to secure the campus. We would need a couple of years of runway to keep things going until tuition stabilized. We would need additional funds to recruit and retain new faculty, more funds for accreditation and regulatory compliance. And ideally we would need some kind of endowment so that this kind of thing doesn’t happen again in the future.

We would probably keep the institutional identity, because a completely new institution wouldn’t be able to simply inherit Hampshire’s accreditation, and NECHE regional accreditation for a new college would take around five to seven years. Petitioning NECHE for a teach-out-and-continue arrangement rather than a full new-institution review would make a lot more sense.

You would need serious faculty recruitment to build the new, stronger teaching program, and you would need new administrators. The staff are amazing, and most of them could be retained.

Some people might be skeptical that it’s possible to raise the needed funds, since Hampshire has been trying to raise money for years now and failing. My answer is simple: Hampshire as an institution has been pretty dysfunctional. It’s no surprise that they had a hard time raising money. But a return to Hampshire’s original model of education, under competent leadership, might be able to raise the funds that the current institution could not.

All this is difficult but not impossible. Donors give billions of dollars in donations to higher education every year. Traditionally this money has gone to schools like the Ivy League, but with grade inflation rampant even at places like Harvard, donors who care about real education might like somewhere more ambitious to send their donations. It’s just crazy enough to work.

A single transformational donor — Somewhere there is a billionaire for whom Hampshire is an obvious bet. They’ve probably already funded a school, or a fellowship, or a learning program, because they believe from their own experience that conventional education takes the ambitious young people most capable of doing something original and redirects their energy towards getting very good at gaming tests.

In Hampshire they will find a fifty-year proof of concept, a model that when poorly funded and totally mismanaged has still produced scientists, entrepreneurs, artists, and thinkers across every field, an alumni base that testifies to its impact, and a price tag (a $200-300 million endowment) that is, for the right donor, an entirely tractable problem.

A coalition of tech/creative philanthropists — The Hampshire model appeals strongly to people who distrust credentialing, and few groups feel that as much as the founders, engineers, and investors who built the tech industry. The people most likely to feel the Hampshire model in their bones are the ones who succeeded despite conventional credentialing or succeeded outside it entirely. I’m talking college dropouts, self-taught programmers, and people who got their first job because of something they built rather than a degree they held.

For them, Hampshire is a vindication of their view on how learning and doing actually works. Traditional school actively trains you to optimize for formal systems and to please gatekeepers. It trains students to focus on grades rather than mastery, producing people who are good at paperwork and mediocre at everything else.

An existing institution — I don’t want to see Hampshire sold to UMass. But Hampshire is a member of the Five Colleges, and shouldn’t be shy of asking for help.

Amherst, Smith, Mount Holyoke, and UMass all have institutional reasons to want Hampshire to survive. Hampshire’s edge is part of what makes the consortium interesting to ambitious students who are choosing between the Pioneer Valley and, say, the competing schools in Boston. If Hampshire closes, many of those students leave the valley entirely. The consortium loses the particular kind of cross-disciplinary, high-agency student that Hampshire attracts, the kind of student who enriches classes at the other four schools too.

A collective loan guarantee of $10–20M from UMass, Amherst, Smith, and Mount Holyoke, in exchange for governance representation, protects something none of them could replicate on their own, and is an extraordinarily good deal.

Bequests — Hampshire’s oldest alumni are only now in their early 70s, and bequests have been few compared to what its older peer institutions receive. If we couldn’t quite make it to $175 million right away, we could supplement with charitable bequest pledges, where donors commit to leaving money to the institution in their wills.

This shows up on the books as a “gift expectancy” rather than cash, and they are revocable, so lenders and accreditors won’t treat them as assets. Donors can always change their wills. You can’t use them to satisfy requirements from NECHE, but you can use a portfolio of bequest commitments as evidence of long-term community support, to secure more favorable terms from donors and lenders on operations.

This is surprisingly achievable. A sum of $175M is a large but not impossibly large capital campaign. People raise this kind of money for higher education all the time some of the time. The cause is right, so with the right donors, in the right moment, it could happen.

Hampshire is a superior model of higher education. At its worst, underfunded, mismanaged, making administrative decisions that defy explanation, Hampshire produced scientists, entrepreneurs, and thinkers that rival any conventional university. Its failure was a recoverable own-goal rather than a structural inevitability, and because of AI and credentialism, the model is more relevant now than it was at founding.

This is a once-in-a-generation acquisition opportunity, and the window created by the bond deadline makes this moment unique. I won’t pretend I’m the obvious person to run the revival of a $175 million institution. But I believe in the mission, I’ve spent a career here as student and teacher, and I’m the person making this case. I’m ready to make it happen.

I almost wrote Hampshire a eulogy. But I like this kind of eulogy better.

If this resonates and you have the capacity to act on it, my email is <ethanludwinpeery@gmail.com>. Hampshire’s tender deadline is September 2026, and the window is narrow.

Appendix: One Hundred Seventy-Five Million US American Dollars

The Campus

Hampshire owns roughly 800 acres in South Amherst, with academic buildings, two dorm halls, three residential complexes, a farm, and a large solar array. According to its fiscal 2025 audit, the college’s total net assets stood at $37.9 million, with an operating deficit of $3.7 million.

This is not a bankruptcy. The board of trustees voted to permanently close Hampshire College following the fall 2026 semester in a voluntary wind-down, not a court-supervised liquidation. That distinction matters enormously, because the board retains control of the asset disposition process, which means they can negotiate directly with a prospective buyer rather than having a bankruptcy trustee run a fire sale auction.

Hampshire’s land in Amherst is simultaneously very large, partially constrained by wetlands and zoning, and encumbered by the bond debt and the bondholder’s mortgage lien. A naive estimate of campus value would start with the following. Rural/semi-rural land in the Pioneer Valley runs roughly $15,000–$40,000 per acre depending on use and zoning. Hampshire has roughly 40 buildings totaling perhaps 500,000+ square feet; at $200–300/sqft replacement value, buildings alone could represent $100–150 million in replacement cost, though market value for a specialized campus is far lower.

Comparable distressed college campus sales in recent years (e.g. Burlington College in Vermont, College of Saint Rose in Albany) have typically closed in the range of $20–60 million for the whole institution. On that basis you might estimate $35M as a midpoint acquisition price. Add $25M to retire the bond debt, and a naive buyer is looking at roughly $60M just to get started.

But the actual number is probably lower.

Remember, the encumbrance on the campus is the bond debt. According to recent reporting, Hampshire holds approximately $25 million in bond debt, with lenders holding a mortgage on campus properties as collateral. Retiring that debt is the primary obligation any buyer must satisfy.

More urgently, Hampshire faces a September 2026 tender date, at which point it would be obligated to buy its bonds back. That is a hard deadline that creates real time pressure, and real leverage for a buyer willing to move quickly. The trustees aren’t looking to turn a profit, they only need to satisfy the bondholders and wind up cleanly. An educational buyer who assumes the bond debt and moves before September has a strong negotiating position.

The practical implication is that acquisition cost may be considerably lower than a standard distressed campus sale would suggest. A buyer willing to assume the ~$25M in bond debt, move before the September tender date, and make a credible case for continuing Hampshire’s educational mission could potentially acquire the campus for close to the bond value alone. Call it $25-30M for the campus rather than $60M, a meaningful difference in the overall capital requirement.

The Runway

So, with ~$25 million to assume the bond debt and acquire the campus from a board that has no reason to extract maximum value, with a comfortable margin we’re looking at roughly $30 million to start.

Add $30 million for two years of operating runway while enrollment builds. In year one, a relaunched Hampshire might enroll 250 students, a conservative target given that Hampshire enrolled 306 students in fall 2023 and 275 in fall 2022, both under a struggling institution with a damaged reputation and limited resources. At a sticker price of roughly $60,000 and a discount rate of around 50-60%, expect net tuition revenue of somewhere between $6M and $7.5M. The founding endowment (more on this below) contributes another $5 million per year at a 5% draw rate, for total annual revenue of roughly $11-12.5 million. Against operating expenses of around $25 million at a lean restart (most costs are fixed regardless of enrollment size) that’s an annual gap of about $12.5-14 million. Two years of that bridge with a margin for error is $30 million.

Add $5 million for faculty recruitment and retention, $3 million for accreditation and regulatory compliance, and we’re at $68 million.

Finally, add a founding endowment of $100 million to make sure this kind of nonsense never happens again. At 5% draw rate, $100 million generates $5M/year, supporting the operating budget and providing a buffer.

This conservative estimate puts us at $168 million. Round up to $175 million to be careful and account for various margins, including the possibility that the bond negotiation goes less favorably than hoped or that the September tender deadline creates cost pressure. If we want to be really conservative, call it $200 million. But the honest number, if acquisition goes well, is closer to $175M, and possibly lower. Give or take, this is all it would take to do it right.

The window here is narrow. The September 2026 tender date is the moment at which the bond situation becomes significantly more complicated and expensive. The discussion about selling the campus is already underway. A buyer who moves soon is in a fundamentally stronger position than one who waits.

People have recently discovered LLM translation tools that let you translate between more than just the standard set of human languages.

Instead of being limited to boring old languages like “English” and “Swahili”, these tools let you translate from anything you want to anything else. For example, you can set the target language to “LinkedIn”, plug in the Declaration of Independence, and recover such gems as, “after a long series of blockers and non-consensual acquisitions, it’s clear the current ‘King of Great Britain’ brand is a total mismatch for our culture. It’s giving absolute tyranny. 🚩”

Because you can put anything you want in the “to” field, the target language can be anything you imagine. LinkedIn is just one of a million options. You could also pop in a normal English sentence and get it back in the style of Matt Yglesias, Bernie Sanders, Barack Obama, or even George Costanza himself. “My favorite is the Gen Z translator,” writes one user. “I can finally communicate with my teenage daughter!” This is how you learn that government of the people, by the people, for the people, will never fall off.

This phenomenon is known as style transfer. And one surprise of the current AI era is that for some reason, style transfer is something Deep Learning1 does really really well.

This isn’t only possible in text — image tools can do the same thing. Pop in an image, name a style, and watch your dog slowly turn into a Monet. This one has been around for a while: check out this James Gurney blog post all the way back from 2016 marveling at the effect.

And it’s probably no mistake that one of the first practical uses of LLMs was in machine translation tools like Google Translate. Because as you can likely guess by now, Google Translate is also doing a form of style transfer. From the model’s perspective, “¿Quién eres y qué has hecho con mis zapatos?” is the same subject as “Who are you, and what have you done with my shoes?”, just in a different style.

I’m not sure why deep learning is so good at style transfer. Maybe it’s because LLMs are already generating a pastiche of the text that went into them, so it’s easy to ask for your pastiche in a different genre. Maybe it’s because neural networks see all things as existing at certain points in high dimensional spaces, so they find it easy to push a concept far out along any dimension you might choose, even if that dimension is “Golden Gate Bridge”.

Subscribe now

Large language models have recently gotten much, much better at coding. In just a few years, models went from “struggling to write basic functions” to “building complete applications with minimal oversight”. They’re getting better fast, and show little sign of stopping. It’s natural to wonder how good they will be six months from now, and to wonder where the ceiling is, if indeed there is a ceiling at all.

There are a lot of software engineers and they’re all paid quite well, so as LLMs get better at writing, debugging, and reviewing code, people worry that all these high-paying programming jobs might disappear, possibly overnight. This could begin with something as simple as jobs lost and industries disrupted, but it also might spiral into tanking the whole economy. This concern is real and worth taking seriously, but there are bigger, more serious questions here that aren’t really about jobs at all.

Most of the time when technology improves, it improves incrementally. Cars get faster, commercial flights get cheaper, computers get smaller. For a long time, LLMs followed the same pattern. Language models got better and better at producing plausible-sounding text, but that was basically all they could do.

In 2016, LLMs would write recipes that started: “4 caam pruce 6 ½ Su ; cer”. So close! By 2021 they could take the string “I decided to call my blog…” and complete it as: “I decided to call my blog Vodka Logic because Vodka is the liquid I would choose for a well-balanced life.” This is a huge improvement, but it still looks like very good autocomplete. It was hard to beat the stochastic parrot accusations.

LLM coding seems different. Code has a serious logical structure. If you want to write code, you have to model the problem you’re trying to solve, catch internal contradictions, and reason about states that don’t yet exist. The bar on functional code is even higher. If the logic or syntax isn’t sound, it just doesn’t work. Coding seems like it requires abilities beyond sophisticated pattern matching. And now that language models have gotten good at coding, it’s getting harder to maintain the feeling that these systems are just very good autocomplete.

This is a new and not entirely comfortable situation to be in. LLMs look less and less like very sophisticated lookup tables and more like they’re starting to do something that resembles genuine thought. Maybe LLM coding is a sign that AGI, “artificial intelligence that can do everything humans can do on a computer” is nearly or already here, and humans are not only out of a job, but possibly on the verge of being obsolete.

But a different possibility is that LLM coding is simply another example of style transfer.

For humans, writing code is a long and tricky process of intricate logical reasoning. You start with a common-sense idea of what you want the program to do, but to actually make the program, you have to hold layers upon layers of rules in your head, work through recursive commands, account for double binds and ambiguities, and so on. If you have a brain that’s made out of meat, this process can be slow and difficult.

In fact, it’s difficult in a really familiar way. It’s difficult in almost exactly the same way as translating a sentence into a foreign language. In both cases, you start with something structured and meaningful, then you go through a complex process to re-render the same meaning in a new structure.

When you translate a sentence into a new language, you keep the meaning of the original, but change the surface features and the grammar. There’s a new format, but it’s fully mapped over the same content. To a language model, the Hungarian sentence „A légpárnás hajóm tele van angolnákkal”, the Japanese sentence “私のホバークラフトは鰻でいっぱいです”, and the English sentence “My hovercraft is full of eels” are all the same idea, just expressed in three different styles.

So it’s easy to imagine that to an LLM, the sentence “check to see if a number is odd” has the same core meaning as this code, with the only difference being the expressed style:

def is_odd(number):
    if number % 2 == 1:
        return True
    else:
        return False

These two bits of prose look different on the surface, but they’re just different mappings of nearly-identical terms and logic. The only difference between them is that one of them is in the style “English” and one of them is in the style “Python”. This is just style transfer again, the same thing we saw in the LinkedIn translator.

If you find this kind of hard to swallow, it might be just because language feels intuitive while, unless you’ve spent a truly staggering amount of time with it, code does not. But the style transfer / language translation of it all becomes somewhat more apparent when we compare code to code. Translating between programming languages looks a lot like translating between natural languages. Look at the two functions below — aren’t these just style transfer?

function isOdd(number) {
    if (number % 2 === 1) {
        return true;
    } else {
        return false;
    }
}

FUNCTION IsOdd(number)
    IF number MOD 2 = 1 THEN
        IsOdd = TRUE
    ELSE
        IsOdd = FALSE
    END IF
END FUNCTION

This might be less surprising when you consider that all LLMs are in some sense descended from Google Translate. The transformer architecture that powers your local chatbot was originally developed to solve machine translation problems, and the 2017 “Attention Is All You Need” paper that kicked off the modern LLM era came straight out of Google’s translation work. It’s all translation.

Subscribe now

If LLMs are translation machinery that has been aggressively expanded to everything else, should it be any surprise they’re especially good at the task they were originally built for? If anything, it’s more of a surprise that you can push software designed for translation to the point where it will write you a grocery list or pretend to be your boyfriend.

So of course LLMs can nail a translation/style transfer — that’s what they were built to do, and everything else is an extension of that capability. Coding, making an email seem more business-casual, re-writing a parking ticket in the style of Jane Austen.2 These are all the same operation, just with more or less exotic target “languages”.

If LLM coding is just style transfer from natural language to code, then we shouldn’t find the recent advances in LLM coding all that surprising. The advances would still be very useful, but they wouldn’t represent a categorical step change in what LLMs are able to do. If LLM coding is “the same as translation,” that might still be impressive, but it’s not a new kind of reasoning. We already knew that LLMs are good at style transfer; this would just be evidence of them getting better, or of learning new styles or languages.

In a recent test, LLMs were asked to write code in esoteric programming languages like Brainfuck or the Shakespeare Programming Language, a language “designed to make programs look like Shakespearean plays”. Despite their strange nature, these languages are Turing-complete and use the same fundamental logic as Python or C#. Any coder who understands the logical core of programming should be able to chug along in these languages, even if it’s slow and difficult. But compared to Python or C#, they have many thousand times fewer GitHub repos, and orders of magnitude fewer examples for an LLM to learn from. This is a natural test of the question of understanding vs. translation: the exact same problems, just with radically less training data.

Sure enough, when asked to write code in these esoteric languages, LLMs perform abysmally. If they really had a logical understanding of code, they would be able to apply that logic to any arbitrary new syntax, however strange. If they’re just doing very sophisticated style transfer, then they would need many thousands of examples to be able to render an existing idea in a new language. That seems to be what’s happening.

A human coder should be able to ship functional code in an unknown language in a matter of days or hours. It might be a challenge, but they wouldn’t need millions of hours of experience or hundreds of thousands of examples in a new language. But LLMs do, so they are probably doing something different than the human is.

That said, I don’t know how far this argument can go. When an LLM turns instructions into a single python function, that does seem a lot like translation. But working code more than a few functions long requires some intellectual heavy lifting. Writing a large program involves maintaining coherent state across thousands of lines of code. And recent LLMs sometimes make connections and suggestions that go beyond translation, and seem like genuine flashes of insight.

But when you use English sentences to ask for a function or a piece of code, I think we can understand that process as being a lot like asking the LLM to translate an English argument into Hindi, Greek, or Italian. It’s something that comes naturally to them — as natural as taking their “Golden Gate Bridge” vector and turning it up to 11.

I liked The Great Gatsby a lot more on my second read-through than I did when I read it in high school. I think it deserves its status as a Great American Novel. I’m also grateful to F. Scott Fitzgerald’s editor, Maxwell Perkins, who convinced him that “The Great Gatsby” was a better and more marketable title than his original idea, “Trimalchio in West Egg”.1

But my main takeaway is that I can’t believe that we make everyone read this novel in high school. It’s great literature, but I don’t know if teenagers can be expected to relate just yet, because The Great Gatsby is a book about what it feels like to survive World War I and then turn 30.

A quick recap of the plot: It’s 1922, and Nick Carraway has just moved to New York City. He is a Yale graduate and a World War I veteran. Lacking any idea of what to do with his life, he becomes a bond salesman, and rents “a weather-beaten cardboard bungalow at eighty a month” on Long Island. But he never shows any interest in the job; it’s a fantasy. He buys “a dozen volumes on banking and credit and investment securities” but never reads them. And he acts a bit like he’s roughing it, but he’s not. “Father agreed to finance me for a year,” he reports.

Nick starts making connections. Daisy Buchanan, his second cousin once removed, and her husband Tom Buchanan, an enormously wealthy Yale football star who Nick already knows from college, happen to have just moved from Chicago to a mansion across the bay, for no apparent or particular reason. Nick is soon invited to their house, where he meets flapper and golf champion Jordan Baker, and learns all kinds of nasty details about Tom and Daisy — like that Tom has a mistress, the wife of a local mechanic.

Back at the bungalow, Nick’s neighbor is the enigmatic millionaire Jay Gatsby, who spends all summer hosting gigantic parties. For a long time Nick watches from the outside, but one morning he receives an invitation and ends up meeting Gatsby at the party.

Just like Nick, Gatsby is a WWI veteran. The first thing he asks Nick is, “Weren’t you in the First Division during the war?” and they talk for a moment about “some wet, grey little villages in France”.

The first time they have a serious heart-to-heart, Gatsby unloads on Nick about how suicidal he was:

“Then came the war, old sport. It was a great relief, and I tried very hard to die, but I seemed to bear an enchanted life. I accepted a commission as first lieutenant when it began. In the Argonne Forest I took the remains of my machine-gun battalion so far forward that there was a half mile gap on either side of us where the infantry couldn’t advance. We stayed there two days and two nights, a hundred and thirty men with sixteen Lewis guns, and when the infantry came up at last they found the insignia of three German divisions among the piles of dead. I was promoted to be a major, and every Allied government gave me a decoration—even Montenegro, little Montenegro down on the Adriatic Sea!”

Through Jordan, Gatsby reveals that he has a history with Daisy. They met in 1917 and fell in love, but when Gatsby went off to war, Daisy married Tom instead, though she almost backed out at the last moment when she received a mysterious letter. They enlist Nick in a conspiracy to reunite Gatsby and Daisy, who meet at his bungalow and begin having an affair. But this isn’t enough for Gatsby. “He wanted nothing less of Daisy,” reports Nick, “than that she should go to Tom and say: ‘I never loved you.’”

The Great Gatsby is about turning 30, and it isn’t exactly subtle. The climax of the novel comes in a suite in the Plaza Hotel, where Gatsby insists that Daisy renounce Tom to be with him instead. “Your wife doesn’t love you,” He tells Tom. “She’s never loved you. She loves me.” But Daisy insists that she loves both of them, and in the end she rejects Gatsby in favor of Tom. At exactly this moment, Nick Carraway, the narrator, remembers that he has just turned 30:

“Nick?” He asked again.

“What?”

“Want any?”

“No… I just remembered that today’s my birthday.”

I was thirty. Before me stretched the portentous, menacing road of a new decade.

It was seven o’clock when we got into the coupé with him and started for Long Island. Tom talked incessantly, exulting and laughing, but his voice was as remote from Jordan and me as the foreign clamour on the sidewalk or the tumult of the elevated overhead. Human sympathy has its limits, and we were content to let all their tragic arguments fade with the city lights behind. Thirty—the promise of a decade of loneliness, a thinning list of single men to know, a thinning briefcase of enthusiasm, thinning hair. But there was Jordan beside me, who, unlike Daisy, was too wise ever to carry well-forgotten dreams from age to age. As we passed over the dark bridge her wan face fell lazily against my coat’s shoulder and the formidable stroke of thirty died away with the reassuring pressure of her hand.

So we drove on toward death through the cooling twilight.

The Great Gatsby is definitely about men trying to paper over their fears of aging and dying by trying to capture an elegant woman. But this feeling is not exclusive to the gentlemen. When they hear the sounds of jazz being played for a wedding in the hotel below, Daisy remarks, “We’re getting old. If we were young we’d rise and dance.”2

Subscribe now

I didn’t ever survive World War I, but as someone who turned 30 a few years ago, I enjoyed this book. But I can’t believe we expect high schoolers to appreciate it, much less understand it. I didn’t understand this book the first time I read it, and I don’t think my classmates did either. The whole thing must have gone over our heads.

In retrospect, the things we must have missed are kind of astonishing. Jokes and references about the book mostly run along the lines of “Gatsby parties”, or saying “old sport” over and over again. Now “old sport” does admittedly appear in the novel a total of 45 times, I counted. But the book is so much more than that.

I was surprised during my re-read to discover that Daisy’s husband Tom is a turbo-racist:

“Civilization’s going to pieces,” broke out Tom violently. “I’ve gotten to be a terrible pessimist about things. Have you read The Rise of the Coloured Empires by this man Goddard?”

“Why, no,” I answered, rather surprised by his tone.

“Well, it’s a fine book, and everybody ought to read it. The idea is if we don’t look out the white race will be—will be utterly submerged. It’s all scientific stuff; it’s been proved.”

“Tom’s getting very profound,” said Daisy, with an expression of unthoughtful sadness. “He reads deep books with long words in them. What was that word we—”

“Well, these books are all scientific,” insisted Tom, glancing at her impatiently. “This fellow has worked out the whole thing. It’s up to us, who are the dominant race, to watch out or these other races will have control of things.”

“We’ve got to beat them down,” whispered Daisy, winking ferociously toward the fervent sun.

“You ought to live in California—” began Miss Baker, but Tom interrupted her by shifting heavily in his chair.

“This idea is that we’re Nordics. I am, and you are, and you are, and—” After an infinitesimal hesitation he included Daisy with a slight nod, and she winked at me again. “—And we’ve produced all the things that go to make civilization—oh, science and art, and all that. Do you see?”

There was something pathetic in his concentration, as if his complacency, more acute than of old, was not enough to him any more.

This appears to be Tom’s new obsession. Nick remarks that, “the fact that [Tom] ‘had some woman in New York’ was really less surprising than that he had been depressed by a book. Something was making him nibble at the edge of stale ideas as if his sturdy physical egotism no longer nourished his peremptory heart.” Even when he’s fighting with Gatsby over Daisy’s love, Tom’s thinking jerks in this direction, and he says, “Nowadays people begin by sneering at family life and family institutions, and next they’ll throw everything overboard and have intermarriage between black and white.”

Either I missed this entirely when I read it in high school, or I totally forgot it in the years since. Even so, that’s kind of surprising; I remembered lots of other details about the book! How did I miss that?

Do you remember Meyer Wolfsheim, Gatsby’s Jewish friend, mentor, and business partner? Do you remember that near the end of the novel, when Nick goes to see Wolfsheim to invite him to Gatsby’s funeral, the name of Wolfsheim’s business front is “The Swastika Holding Company”?

The morning of the funeral I went up to New York to see Meyer Wolfsheim; I couldn’t seem to reach him any other way. The door that I pushed open, on the advice of an elevator boy, was marked “The Swastika Holding Company,” and at first there didn’t seem to be anyone inside. But when I’d shouted “hello” several times in vain, an argument broke out behind a partition, and presently a lovely Jewess appeared at an interior door and scrutinized me with black hostile eyes.

This is some weird accidental oracle shit on Fitzgerald’s part. At the time, the swastika didn’t have antisemitic connotations to most people (though the Nazi party had adopted the swastika in 1920). It was instead a popular symbol of luck and success, “much like the four leafed clover”, and probably was intended to give Meyer Wolfsheim an exotic, oriental air, just like how his cuff buttons are made of “oddly familiar pieces of ivory”, which Wolfsheim calls attention to and reveals as the “finest specimens of human molars”.

But again, it’s very strange that I don’t remember this standing out to us as high schoolers. You’d expect we would pick up on the irony of this one Jewish character having a company named after the swastika. But apparently not.

But even if they don’t fully appreciate it, it’s probably good to have the young’uns read this book, and not just because it is one of the last touchstones giving us a shared national identity. Not even because it might encourage them to try calling each other “old sport” over tater tots in the cafeteria.

History rhymes more than I would like it to. So far we have dodged having another great war, but we had our own major pandemic in 2020 to mirror the Spanish Flu of 1918, and I worry we are facing a new jazz age, except this time without the comfort of actual advancements in jazz. Too many of the same social issues are coming around again; Tom would fit in disconcertingly well on the more racist parts of Twitter.

I was struck by Wikipedia’s description that: “In Fitzgerald’s eyes, the [Jazz Age] represented a morally permissive time when Americans of all ages became disillusioned with prevailing social norms and obsessed with pleasure-seeking.” When I saw that, I thought of this Wall Street Journal poll from 2023:

Everyone remembers the last line of The Great Gatsby: “So we beat on, boats against the current, borne back ceaselessly into the past.” But I find myself more drawn to the paragraphs just before that.

“His dream must have seemed so close that he could hardly fail to grasp it. He did not know that it was already behind him, somewhere back in that vast obscurity beyond the city, where the dark fields of the republic rolled on under the night.” I wonder if Fitzgerald was self-aware about the fact that this novel was to be published on the eve of the 150th anniversary of the United States.

Happy 2026.

I originally posted this essay in July of 2023 — this is a new version with minor edits to update the language and improve clarity.

I used to be in favor of open-notes tests. But after seeing them in action for a while, I realized that I don’t think they’re a very good idea.

Traditional tests aren’t great, so it’s nice to see people exploring other directions. But the open-notes approach is a mistake, because it doesn’t fit very well with the strengths of test taking.

Tests have some natural strengths and some obvious weaknesses. If we understand these strengths and weaknesses, we can design tests that let us meet more of our goals. Settling for the open-notes approach keeps tests from becoming all they can be.

Knowledge

The traditional argument in favor of open-notes tests is that having access to your notes is true to life. In the real world, it’s rare to find yourself locked in a room without your books, forced to answer questions under a time limit. For most tasks in real life, you’ll have access to whatever resources you need, and can look things up as you go.

Einstein famously was unable to remember the speed of sound, when given the Edison Test. Why memorize such facts, he remarked, when one could easily look them up in a textbook?

Einstein was right. Skill comes from more than just what you carry around in your head. Experts use all the tools they need and refer whatever sources they want when they’re solving a problem. In many ways, skill in a domain is just skill at using the reference works of that domain. Hence the old joke that programming be renamed “Googling StackOverflow.”

Take this view too far, however, and you end up with absurdity. It’s clear that experts don’t carry everything around in their head. But it’s also not true that they carry nothing around in their head.

A physicist may not be able to tell you the speed of sound without looking it up. But every physicist will be able to tell you who Maxwell and Newton were, and a little bit about their contributions. If someone doesn’t know what F = ma means, they’re probably not a physicist.

A programmer won’t be able to recall from memory the exact workings of every function they’ve ever used. But every programmer will be able to tell you the syntax needed to write a for loop in their favorite languages. If someone can’t tell you the syntax of an if statement, they’re probably not a programmer.

An expert is someone who is able to do both. Some things they will know by heart, and some things they will be able to accomplish only given time and resources. You need both to have mastery of a skill. We might call these two forms of knowledge what you carry around in your head and what you can accomplish.

Subscribe now

Mastery

We don’t expect students to leave a class as an expert in their field, but we do expect them to have mastery of the material.

What does mastery mean? I think that mastery involves both of these skills.

Someone who can accomplish a task but doesn’t carry any of that knowledge around with them is following a guide, or a set of instructions, without any understanding. Someone who can tell you important facts about a field but can’t accomplish anything is a fan, not an expert.

Students shouldn’t be expected to memorize everything. But we should expect them to carry certain facts around in their head wherever they go. I don’t care if a student leaves my statistics class without memorizing the equation for a t-test. They can look that up. But if they can’t read a scatterplot, that’s a problem.

To evaluate a student’s mastery of a subject, we want to measure both kinds of knowledge. We should give them the chance to demonstrate real skill in the field, but we should also require them to show that they have internalized some of the most important facts and concepts.

We already have good ways of doing both.

Tests separate the student from their resources, and have the potential to measure the information that the student actually carries around in their head.

Class projects (and depending on the subject, papers) allow the student to use whatever they want in the solving of an actual (if usually artificial) problem, and have the potential to measure the student’s ability to accomplish practical work in the field.

When tests and projects are designed with this in mind, a class can run smoothly. When they are not, the result is disaster.

Tests

What are the important features of a test? Well, they happen in a controlled environment. You can’t choose what you’re working on; all questions have been decided for you. You have a limited amount of time. You’re not allowed to collaborate with other people. And you’re not allowed to look anything up.

Open-notes tests relax this last criterion. Some of them relax it in a small way, like giving students a formula sheet, or allowing them to bring a note card as a cheat sheet. Sometimes tests are truly open notes, and students are allowed to refer to whatever they like. Sometimes students can even bring their laptops, and make use of the entire internet.1

It’s good to evaluate a student’s skill at solving problems without restrictions. But tests aren’t a good way to evaluate this kind of knowledge, because they unnaturally restrict the student in other ways. The student isn’t given the kind of time they would have if they were solving a real problem. They don’t get any choice of what problem to work on. They can’t collaborate with others, or go to peers to discuss some aspect of the problem that’s troubling them, which is a huge part of solving problems in the real world. The format of a test hamstrings them.

This is tragic. Tests are naturally suited to evaluating the knowledge and skills that a student has internalized. We should use tests to see if the things you want your students to carry around in their heads have actually ended up in there.

When designing a test like this, you should figure out what you want your students to walk around with, and only include questions about those facts and skills. If it’s information they’d be better off just looking up (dates, exact values, trivia, etc.), that shouldn’t go on the test.

A simple way to evaluate this kind of test is to give it to other experts, and make sure that they can easily answer all the questions without looking up the answers. If experts in the field can’t casually ace your test, then it isn’t a good test of what experts should be expected to carry around in their heads.

This standard may even be slightly too harsh; you probably don’t need your students to walk out of the class on the same level as an expert. Another way to benchmark a test is to pick a student who you know reasonably well, who seems to have mastered the subject, and see how they do on your test.

A test made on these principles should be simple and easy, something that an expert would be able to breeze through. No cruft and no trick questions. Just an evaluation of how much knowledge they are carrying around.

Subscribe now

Projects & Papers

For most subjects, class projects or papers are the right way to test the skill of what you can accomplish. Don’t shoehorn open-notes into a test format, it doesn’t fit. Just have them do a project. Projects are inherently open-notes; who ever heard of limiting the resources that can be brought to bear on a class project?

Projects provide a better environment for testing what you can accomplish because they don’t unrealistically hamper the student, as even the most liberal open-notes test will. Students have some level of control over what project they choose, how they approach it, what techniques they use, and who they call on for help. That’s a fair test of their abilities as a whole.

Exceptions

Does this advice apply to all subjects? I don’t think so. Foreign language courses are almost entirely about internalization. If you need to look anything up, you haven’t really learned the language. So testing makes a lot of sense in a language course. On the same note, I’m not sure if projects have any place in introductory language courses — though once you get to composition courses, projects start making more sense again.

There may be other reasons to have students do projects. Here I’ve mostly approached projects as a form of evaluation, but projects can also be an important teaching tool. Having students complete a project as an alternative to readings or lecture is a good idea, but that’s a different use case.

There are also some subjects where tests make no sense at all. For many hands-on skills, like writing or sculpture, you could conceivably make a test, but the real proof will be in creation.

Testing is a good way to examine internalized knowledge, but there are some kinds of internalized knowledge that aren’t easily measured by a test. Exactly how to hold your hammer and chisel, just what the dough looks like when it’s ready to go in the oven — these are things that an expert will have internalized, but which would be difficult to put on a test.

So there are some kinds of internalized knowledge that are better measured by projects. It seems like this is especially true for crafts, and for courses beyond the beginner level, as the student begins to pick up these hard-to-measure intutions.

Generally, the more advanced the course, the less of a role there is for testing. While every subject has a core base of knowledge that all experts will know by heart, specialists will internalize knowledge that sets them apart even from other specialists. People already seem to understand this at some level, and most advanced courses tend to go light on the tests.

In the Beginning

In the beginning, science was all about metaphysics. A huge number of our questions were metaphysical. How do time and space work? What is stuff made of? How many different kinds of plants and animals are there? Is heat the absence of cold, or is cold the absence of heat, or are they two different fluids? That kind of thing. 

The scientific revolution came out of these projects. Descartes convinced most physical scientists that the material universe is made up of microscopic corpuscles, and that all natural phenomena can be explained in terms of the shape, size, motion, and interaction of these tiny snooker balls. Newton and Flamsteed and others took these metaphysical ideas and started framing out modern physics. 

You saw the list of elements go from four entries to ???, uhh, actually we’re still figuring that out, get back to you in 100 years. There might be a couple different kinds of air, but we’re pretty sure that the list of elements includes fire and light. 

Copernicus did the same thing to astronomy. The Earth, which used to be the center of the universe, became a planet. The sun, which had been a planet, became a star. The moon, which had also been a planet, was enough of a problem that we needed to invent an entirely new kind of thing for it to be, a satellite. 

Some of these projects had better results than others. John Wilkins spent his time inventing a Philosophical Language, which he planned would be a system for classifying every possible thing and notion in the entire universe. In his system, all possible concepts are divided into forty Genera, which are each divided into several “Differences”, which are further divided into Species. 

Each Genus comes with a two-letter code, so in this system, any word starting with “Zi” is in the Genus of "beasts". Wilkins then divides “beasts” into six Differences: “Whole footed”, “Cloven Hoofed”, “Non-Rapacious”, “Feline”, “Canine”, and “Egg-laying”. Each of these adds a letter, and then each species gets another letter after that. So all feline beasts start with “Z-I-P”, which is why lion is Zipa, cat is Zipi, and the most famous feline beast of all, the beaver, is Zipyi.

You may not have heard of Wilkins’ Philosophical Language, but you’ve probably heard the parody by Borges. In his short essay, “The Analytical Language of John Wilkins”, Borges invents “a certain Chinese encyclopedia entitled Celestial Emporium of Benevolent Knowledge” that divides all animals into different categories. You know the list: 

(a) those that belong to the Emperor; (b) embalmed ones; (c) those that are trained; (d) suckling pigs; (e) mermaids; (f) fabulous ones; (g) stray dogs; (h) those that are included in this classification; (i) those that tremble as if they were mad; (j) innumerable ones; (k) those drawn with a very fine camel’s-hair brush; (l) etcetera; (m) those that have just broken the flower vase; (n) those that from a distance resemble flies.

So some of this metaphysics was more successful than others. But all of it was serious — John Wilkins wasn’t some nut, he was one of the founders of the Royal Society. Without him, there would be no science as we know it today. And all of it was metaphysical. 

Subscribe now

What is a Metaphysics

Ok hold up, let’s go back: what exactly is metaphysics? The term is a little hard to pin down, and there are a few different ways you can try to define it. 

One answer is academic: Metaphysics is the branch of philosophy that deals with the basic structure of reality. This includes the nature of existence, and the features that all entities have in common.

That answer is kind of dry, so here’s an answer from a Sci-Fi author. Seriously, I think this is a  great definition from Neal Stephenson. He says,

A straightforward way of defining metaphysics is as the set of assumptions and practices present in the scientist’s mind before he or she begins to do science. There is nothing wrong with making such assumptions, as it is not possible to do science without them. 

A more prosaic definition is that metaphysics is about categories. When you look at the universe, you have to divide it into categories, you know, dog, tree, car, rhombus, et cetera. And sometimes we decide that our categories are wrong, that a whale isn’t a kind of fish, but rather more of a very large, very wet dog. When we think about these issues of categorization, that’s metaphysics.

But “categories” is not quite right, since categorization is about dividing up the things that we already believe in. Metaphysics is a little deeper — it’s also about what sorts of entities the universe does and does not contain in the first place.

You can’t divide microscopic particles into different categories until you believe that there are microscopic particles at all. You can’t divide infectious diseases into categories based on the infectious agent until you believe that diseases can be caused by things like bacteria and viruses, and you can’t do it at all without concepts like “disease”. So metaphysics is about categories, but also about what kinds of things there are to categorize in the first place. 

Victims of our own Success

You don’t hear about metaphysics all that much anymore. And I worry that our successes have made us think that metaphysics isn’t important. 

We’ve gotten satisfying answers to many of the metaphysical questions we started out with. We have a periodic table for our elements, a quantum model for our physics, cell theory for our medicine, and DNA for our biology. The success of these questions makes people assume that metaphysics is behind us. We think we slew that dragon, that we can beat these swords into plowshares, and spears into pruning hooks. 

But we shouldn’t let our guard down yet. The answers we’ve come up with are definitely useful. In some sense, they may even be correct. I’ll leave that question to philosophy hour. But we can’t forget that they are human inventions, and like all human inventions, they may be incorrect. 

We keep the periodic table because it works. And maybe it will work forever, in which case, great, fundamental questions of chemistry over. But if we eventually found some place where it stopped making sense, well, we would need to throw it out. If that day comes, we should make sure we remember how to tackle these questions. This used to be a skillset at the heart of science. If you wanted to hack it as a chemist in the 1780s, you needed to be able to think about metaphysics. But we no longer think this skill is important so we don’t teach it.   

We now have some kind of answer for all the metaphysical questions someone might ask. What is matter, what is heat, what is light, what is a species, et cetera. And our answers for most of these questions haven’t changed in our lifetimes, so they can seem like they’re final. But having an answer is very different from having the right answer. And just because an answer has stuck around for a while doesn’t mean it’s the right one. We should be much more paranoid. 

Metaphysics seems fake to us because we think we live at the end of scientific history, but it was all very real for the scientists of just a few generations ago. If you lived through the turn of the century, you saw your metaphysics overturned several times. 

In the 1880s, atoms were indivisible, physics was deterministic, and space and time were both absolute. Fast forward to the 1920s, and all of that has changed. Atoms are no longer indivisible, they’re made up of smaller subatomic particles. In fact, atoms are mostly empty space. Physics is no longer deterministic clockwork, quantum mechanics has made it all probability. Space and time are no longer absolute, they’re relative. And guess what, they’re distorted by mass. That’s metaphysics.

And that’s just a sample of the chaos of that era. If you were a mathematician, you lived through Frege, Russell, Hilbert, Gödel, and Turing. The last few decades of the 19th and first few decades of the 20th century were a wild time to be a scientist, or to be a human being in general. Let’s not forget the massive changes in political science! You couldn’t just take the nature of the world for granted, couldn’t assume that metaphysical questions were all settled, because you had seen your metaphysics get trashed over and over again. 

There’s a passage I really like from Albert Einstein. It’s a couple lines from his 1916 obituary for physicist and philosopher Ernst Mach,1 one of Einstein’s personal heroes. He says, 

Concepts that have proven useful in ordering things easily achieve such authority over us that we forget their earthly origins and accept them as unalterable givens. … The path of scientific progress is often made impassable for a long time by such errors. Therefore it is by no means an idle game if we become practiced in analysing long-held commonplace concepts and showing the circumstances on which their justification and usefulness depend … Thus their excessive authority will be broken. They will be removed if they cannot be properly legitimated … or replaced if a new system can be established that we prefer for whatever reason.

As much as we would like to, we can’t escape from metaphysics. We live in an age that no longer believes in metaphysics, but even that can’t stop it, you can’t get away from it. 

Many of you probably will have strong opinions about this one: is Pluto a planet, or not?2 This isn’t quite as profound a question as the ones raised by Einstein and Rutherford, but it’s still metaphysical. We have to carve up the world in some way, have to draw category boundaries somewhere, but as much as we would like to believe that this is a rational and not at all a political question, as much as we would like to believe that there are objective answers to these carvings, there are not.

Outstanding Metaphysics

Let me give you a few examples besides Pluto.

#1: For years, we’ve classified cancers by the location they appear in the body. And you probably still find it natural to talk about breast cancer, lung cancer, pancreatic cancer. But lately, oncologists have started classifying cancers based on the genetic mutations that caused them (e.g. BRAF V600) rather than the location where they first show up. So instead of saying, “you have lung cancer,” a doctor might now say, “you have cancer with this specific genetic mutation, and it happens to be in your lung.”

It turns out that trying to understand cancer cells in terms of “what part of the cell is malfunctioning?” is a better approach than trying to understand the cells in terms of “what organ did we find them in?” How a tumor behaves and how it responds to treatment usually has more to do with the mutations that created it than what part of the body happened to house the cells that developed those mutations. Two patients with “breast cancer” might benefit from very different treatments, while two patients with tumors in different parts of the body might nevertheless benefit from the same treatment. We’re finding that it’s better to carve up this part of the world based on mutation than based on location. 

#2: When astronomers look at the universe, they find two surprises. First, it’s clumpier than it should be. Galaxies stick together more than the gravity from visible matter can explain. Second, it’s expanding faster than it should be, as if some unseen force is pushing everything apart. These are weird and they also seem like they contradict each other: more gravity on one hand, more repulsion on the other. 

To explain this, some physicists have come up with the idea of "dark matter", secret mass that could explain the clumping. But this mass would have to not emit, absorb, or reflect light. And they came up with "dark energy", which would explain why everything is flying apart. 

New entities with strange properties? That’s metaphysics. And whether or not dark matter and dark energy end up being a good answer, it’s clear that something is missing from the standard model, so metaphysics is needed here one way or another.

#3: Medicine loves to diagnose and carve up the world of human health into different categories, but medical metaphysics is a mess. What counts as a disease, and how we define it, are hotly debated and unstable over time. Remember that “syndrome” means “collection of symptoms that go together but we don’t know why”.

There’s no telling if most of our diagnoses will still be recognizable 100 years from now, and lots of evidence they won’t, in part because most of our diagnoses from 100 years ago aren't recognizable now. To take just one example, there are signs that what we call “rheumatoid arthritis” may turn out to be more than one disease.

And forget about nutrition. Did you know that “Vitamin E” is actually a family of eight different compounds, four tocopherols and four tocotrienols? Plus a ninth synthetic compound, tocofersolan?

Psych Me Out 

The younger a science is, the more likely it hasn't gotten good answers to its basic metaphysical questions. And the less you have a working answer to these questions, the more important metaphysics becomes.

So sure, I’m gonna call out economics, sociology, and yes, even medicine. Blocked. Blocked. Blocked. None of you are free of metaphysics.

But I think this is especially important in my home field of psychology. Psychology has a lot of problems, and we usually try to solve them by taking better measurements, or by being extra rigorous in our stats and methods. But taking better measurements doesn’t help when you’re asking meaningless questions, and it’s easy to ask meaningless questions when your metaphysics is all confused. The replication crisis led to a focus on improved methodology: pre-registration, better statistical practices, larger sample sizes. But this doesn't address conceptual weakness.

The easy example is mental health diagnoses. Almost everyone suspects that depression, anxiety, OCD, all these categories are poorly formed. There’s probably more than one kind of depression; or maybe depression is the wrong concept entirely, we should toss it out and start over from scratch. 

Again I feel like we’re a victim of our own success here. We have other lists, like a list of chemical elements. People see a list of diagnoses and assume that it’s just as good.

The periodic table and the DSM are both human inventions, but people take the wrong lesson from this. It shouldn’t make you trust the DSM more, it should make you trust the periodic table less. Both of them are up for negotiation at any time. The difference is that the periodic table has so far resisted attempts to find better alternatives. The DSM survives only because we haven’t found a good alternative to rally around, and even so, it changes all the time.

Mental health categories are more like the problems of categorization in medicine. The debate is metaphysical, but it’s mostly about where to draw the lines. But there are also more fundamental questions open in psychology, the kind of profound metaphysics that we haven’t touched in a long time. 

Like, what is an emotion? We use a lot of words casually when we’re talking about the mind — nouns like thought, memory, emotion, even “mind” itself, verbs like perceive, think, recall, remember, interpret. These are all very familiar. But medicine used to talk about melancholic and sanguine humors, and chemistry used to talk about four elements and three principles. Those were familiar too. 

Thank you.

I learned that the universe is large and cruel, and that I am soft and small. Or at least, I learned that the universe can be as cruel as it decides. There are no checks on its power. 

I learned that there are some things you can’t power through. In bad conditions, a task that would take a dozen hours in normal weather can take you more than five days. If you push harder, you faint or die.

For important tasks, I always give myself much more time than I need. But I learned that planning, being careful, and giving yourself lots of slack can’t always protect you. The weather alone can rob you of a week. 

I learned that the rules are made up (I kind of knew this already, but it was a nice reminder). If it’s hot during the day, you can just move your furniture at night. 

But I also learned that in Massachusetts, it can be 88 °F at 10:00 PM, hours after the sun has gone down. I really didn’t know that was possible. In Sri Lanka maybe, but not around here.

I learned that I need other people, that only the help of those around me can keep me safe and only their kindness can keep me alive. I learned that it’s not enough to be smart or careful or strong or to have the grit needed to power through, I cannot make it on my own.1

I learned that intense heat can really addle your thinking. The experience has been like being on two shots of vodka all day, only much worse. I still feel like I can’t think straight. 

I learned that I am emotionally attached to my furniture and other belongings. But I also learned that I am happier sleeping on a fold-out futon in an empty room, with my laptop on a square college-style ottoman, than in a room full of all my books and my things, with a real bed and a good desk.

I learned that it can get so hot that not only does standing directly in front of a box fan not make you feel any cooler, standing in front of an A/C unit doesn’t make you feel any cooler either. 

I learned that I am more sentimental than I realized, that there are many things I have a hard time giving up. There’s often no reason to rush a decision, and delaying is often a good tactic, but I learned that I still have a lot to learn about the balance between delaying a decision until you have to decide, and being decisive up front. 

I learned that I have a lot of houseplants, and I learned that I am the guy who makes the living room feel like a Victorian parlor when I move in. I didn’t know I was that guy, but moving makes some things impossible to miss. Moving offers you contrasts. I learned that moving can teach you who you are. 

Subscribe now

Every red-blooded American loves the freedom of the highway. But with a long enough drive, even the wide open road can grow a little stale. How do you pass the time?

I recently spent two years working at a job where I had to make the same two-hour drive, two times a week. This wasn’t a problem; it’s a beautiful drive. But after the first dozen runs it got a little old, so I started working on ways to help pass the time. 

The sad truth is that I’m not much of a podcast or an audiobook guy, so my main tool was music. At first I thought I would be able to just listen to my usual catalogue. But I was surprised to find that many songs I love under normal conditions are a total drag on the highway — while other songs, that I might not usually appreciate, are mesmerizing on a long drive.

Soon I realized that I had accidentally developed a new theory of the best songs for highway driving. Today, I share that theory with all of you.

Long Time

My first discovery is simple: highway songs should be as long as possible. 

On a long drive, you want to pass the time and keep the seconds from dragging. Longer songs make the drive feel shorter and tend to be more pleasantly hypnotic. 

Clock time passes in minutes, but your perception of time isn’t strictly bound to reality. The mind measures things in intervals. This is why, other things being equal, six 10-minute meetings feels more frantic than one 60-minute meeting. You had to do six things in an hour instead of just one! And it’s related to why 200 words is an easier read when it’s presented as five paragraphs than when it’s dropped on you as one great big wall of text.

When you’re listening to 3-minute pop songs, a two-hour drive is 40 songs long. But if you’re listening to 6-minute ballads, the same drive is only 20 songs long. This will make it feel about half as long, or at least makes it feel like it took half as many songs. 

The songs still have to be good, of course. But good & long songs pass the time better than songs that are good & short. Based on extensive empirical testing, I’ve found that highway songs should ideally be 4 or 5 minutes long, though longer songs are better. 

I think there must be diminishing returns at some point. A single 120-minute jam won’t make your drive feel like it was “one song long”, no matter how much of a banger it might be. But in general, listening to longer songs makes the drive feel shorter. You enjoy a few of your favorite tracks, and before you know it, you’ve arrived. If you’re listening to shorter songs, you burn through dozens of tracks in an hour, move from album to album, you have to switch playlists, it’s a hassle.

So while it is a masterpiece, on a road trip I can’t recommend that you listen to Pet Sounds, which has an average track length of less than three minutes. The longest track, “I Just Wasn't Made for These Times,” runs a mere 3 minutes 12 seconds.

Subscribe now

The Roadtrip

Ok, longer songs better. On top of that, I noticed two other features that great highway songs tend to have in common.

The best highway songs have, no pun intended, a real driving beat. The beat needs to be fast enough that you feel like you’re truly rolling, but not quite so thumping that you want to constantly accelerate. 

The right tempo seems to be close to 100 BPM, plus or minus ten. Any slower and things start to get boring. Any faster and you’ll want to floor it, and soon you’re going 95 on MA Route 2, which is a problem.

Finally, almost all the best driving songs have a long instrumental prelude. In the best songs, there’s as much as a full minute of run-up before you hit the lyrics. 

Why do preludes make for such good highway driving? If I had to guess, I’d say that the prelude is hypnotic. It helps ease you into things. Without lyrics, the prelude doesn’t measure time so much, so it passes the time more quickly. You do tend to lose yourself in these songs.

In some ways, the goal of a highway song is to enter something like a shamanic trance. Think about what defines the highway experience: you’re not making turns, you’re not making decisions. You’re on a constant journey. You’re going somewhere far away.

Highway: The Playlist

Right now I have 81 songs on my “Highway” playlist, giving me 6 hours and 25 minutes of driving jams. I won’t bore you with the full playlist today — just the hits. Here are the top 10 tracks, take a look:

Read more

A while ago, I published an essay called Why Psychology Hasn’t Had a Big New Idea in Decades. This was inspired by my experience teaching a class about psychological paradigms at Hampshire College, and by thinking about this problem for the majority of my adult life. 

My basic argument was that psychology isn’t making progress because it doesn’t have a paradigm, and we can’t make any real progress until we have one. I spend the body of the essay describing what we know about how sciences get their first paradigm, and I close it out by listing some directions that psychology might pursue next, mostly as inspired by discussions I had with my students.

Despite it being so dang long, I got a number of thought-provoking comments on this post, and I wanted to respond to a few of them here. Here goes.

Colin M. Fisher writes: 

I’m with you and Adam Mastroianni that psychology has only proto-paradigms. But I think the comparison to chemistry takes us awry. Even if we have a stronger paradigm (and I think its stronger than it was), we still won’t be like chemistry. Because people are changed by the awareness of their own psychology. And those situations are human creations. So we’ll always be shooting at a moving target, paradigm or no. And that will forever keep us from being like chemistry.

 I really don’t get this argument. 

Compare: The chemicals we use are changed by our awareness of chemistry. As we learn more, we make new chemicals that are human creations. So we’ll always be shooting at a moving target, which will forever keep chemistry from becoming like chemistry. Huh? 

My best guess about the disconnect here is that Colin might see psychology as the study of specific people and specific situations. 

And yeah, if you think of it this way, it’s easy to see why it might feel endless. Specific people and specific situations really are moving targets. How do people behave at the grocery store? How do people behave at dinner? How do people behave at the movies? Look out, there are a lot of different situations. Just when you think you’re getting a handle on things, someone goes and invents a new situation, “brunch”, and you have to start a whole new line of research. Then the pandemic hits and you have to learn how people behave in a “zoom room”. When you see the job of psychology as enumerating these million million things, it can seem kind of hopeless!

But I see psychology instead as the study of the nature of people and situations, and the laws that govern them. I think it’s possible to ask questions like: what do dinner and brunch have in common? Why do people talk at dinner but not usually at the movies? Why do people mostly not talk at the movies, but sometimes they do? Where do the exceptions come from? When “zoom room” is invented as a new situation, we don’t have to approach it as something totally new. It clearly has some things in common with other situations. 

Taking Colin’s concerns as stated, if people are changed by the awareness of their own psychology, we can still ask how they are changed by the awareness of their own psychology. If situations are human creations, what kinds of situations can exist, how do situations differ from each other? Why do humans create some situations and not others? Are there exotic situations that we could be creating, but that no one has yet discovered? Can I try them?

“Paradigm” is a famously hard concept to define, but one gloss is that it can be read as, “way to talk about [whatever]”. I think most of us agree that we have better ways to talk about psychology today than we did 100 years ago, when professionals still used terms like “neurasthenia” and “hysteria”. Colin seems to agree: he says, “it’s stronger than it was”. 

I think there are going to be even better ways to talk about psychology in the future. And I think chemistry is one of the right models to look at, because of how huge their subject matter seemed at first, and how much success they’ve had at breaking it down into manageable parts. When psychology gets a strong enough paradigm, we will be like chemistry.

The fact that psychology seems so impossibly complex and nuanced makes this a good comparison, not a bad one — chemistry also seemed impossibly complex and nuanced, until we found a paradigm that let us break down the confusion into individual parts.

Chemistry started with the goal of understanding every possible kind of matter in the universe. This seems like a pretty tough ask — even if you limit yourself to just, say, kinds of mud, there’s a lot of variation. How are you supposed to find a way to describe the difference between every different kind of mud?

But chemists eventually found that every material thing (not just mud) is made of combinations of only about 90 different kinds of things: the 90-ish naturally occurring elements. Then we learned that all the elements are actually made up of only three different things — matter is just different combinations of protons, electrons, and neutrons.

So, what pieces are the mind made of? Maybe it’s more than 3 things, maybe it’s more than 90 things, but chemistry has a pretty clear message for us. When you stare at something for long enough, and refuse to give up on the question, “what pieces is it made of?”, eventually you get somewhere. You can start with “every kind of material body in the universe” and then discover that it’s all made up of just 90 or so things. And then you can discover that those 90 or so things are made up of just three things in different combinations. The mind is complicated, but I’m not sure it’s as vast a problem as “the nature of every substance in the universe”. This makes me optimistic.

And as a parting example, “situations” may be 100% the wrong way of framing these questions. In fact, I strongly suspect it’s the wrong approach, something we inherited from a failed attempt at a paradigm (one that I hope to write more about soon!). For someone like Colin, tossing out the concept of “situation” might be the start of the paradigm shift.

Subscribe now

I got several more comments when my essay was generously re-posted by The Seeds of Science. Here I want to focus on one comment in particular, from Becoming Human, who said:

What a fascinating read! And it wasn't short, so getting to the bottom says something (about how compelling it was, and about my attention span these days).

I will poke things and say you are mixing apples, oranges, and aircraft carriers. Chemistry, Physics, and even Biochemistry are discrete sciences that operate at relatively low scale with discrete, relatively predictable units (let's not get into the whole quantum thing). They are paradigms because they yield to relatively mechanistic reduction.

Psychology (or thinking or mind) is not in that category. If it is anything more than an oddly stable phenomenon, it is a complex system (again, let us not quibble that complexity is indefinable). While we suspect that whatever the psyche appears to be, it is a manifestation of activity at more granular scales, there is no decisive proof. As such, it is bearishly difficult to break it down into subsystems.

In this way, psychology is in the family with biology, ecology, sociology, and economics (which you allude to above). These, and related fields like nutrition, are so complex that we can only evaluate them by looking at tendencies and correlations, not underlying mechanisms. Sometimes we find little machines, such as efficient market theory or particular diets. Still, they are almost always gross approximations that fall apart under meaningful scrutiny, leaving us with only broad generalizations like "people tend to die without food."

Some folks will insist that the psyche is an emergent property of materialist principles, but that conclusion is an article of faith, not science. Much of what you list above, like DNA as memory or embodied cognition, makes the "mind as computer" argument a little slippery, and should one keep sliding to panpsychism, all semblance to science goes in the sh*tter.

As such, it is probably more instructive to allow psychology to operate as a form of complex systems analysis and stop expecting it to become math or chemistry. This is not to say stop looking for better answers, but we should stop looking for paradigmatic approaches if that means coherent, self-contained systems for analysis.

I think what BH is saying is that the study of the natural world can be broken up into two categories: 

Some fields are “discrete sciences”. This includes chemistry, physics, and parts of biology, like genetics. These fields study “discrete, relatively predictable units” which are usually very small. Because these small discrete things obey clear rules, you can figure out those rules, and use the units to model other parts of the world in a relatively mechanistic way. In these fields, reductionism works.

Other fields are “complex systems”. This includes ecology, sociology, economics, and other parts of biology, like nutrition. These fields study real systems, but the systems they study are so complex that they can’t be easily broken down into simple laws. You can learn things about these systems, like "markets are mostly efficient”, or “iron is good for you”, but there are always exceptions. To use BH’s metaphor, some fields are aircraft carriers — impossible to reduce to simple principles.

Complex systems are inherently arbitrary, like studying bumper cars at the amusement park. You can absolutely learn more about bumper cars, you really can become an expert. There are some distinctions that are very useful — most bumper cars are powered either through an overhead system (OHS) which uses a conductive floor and ceiling, or through a floor pick-up (FPU) which uses alternating strips of metal separated by insulating spacers. But people can and do invent other ways of powering their bumper cars, because the system is arbitrary. There are laws of chemistry and physics, but there are no true laws of bumper cardom.

I think this is a pretty good way of looking at things. When studying the natural world, some fields study the parts. Other fields study the complex systems. People mistake these two kinds of fields for each other all the time: both are scholarly, both study the natural world, and they’re both empirical, they involve taking measurements and running experiments. So they both look like “science”. But the two kinds of fields are different in really critical ways. Complex systems have immediate practical uses and the list of questions is never-ending, so there’s always more to do, they are a good investment in time and energy. But studying the parts leads to the greatest successes of the sciences, a profound understanding that cannot be matched.

So far so good. But here’s where I start to disagree.

The way we study psychology today mostly does look like BH’s “complex systems”. We search for tendencies and correlations, little machines, not underlying mechanisms. Our findings are almost always gross approximations and broad generalizations. We have some knowledge that is useful, but almost no knowledge that is profound. 

But it doesn’t have to be this way, because every study of complex systems has to be matched by a field that studies the parts of that system. Machines aren’t always simple, but they are always made of something. Words are made of letters, letters of strokes. Bodies are made of cells. The electrical grid is a complex system, maybe too complicated to be studied by reduction. But it still has parts that can be studied in isolation, it’s all current and wires. Systems can be non-reducible in function, but still reducible in structure.

So take psychology: yeah, there are an infinite number of possible thoughts. Thoughts are complex. But what are thoughts made out of? They must be made of some kind of parts, what parts are they made of? There are an infinite number of possible memories, but the memories are stored on some substrate using some compression algorithm. What can we learn about those parts? There are an infinite number of different personalities, but the personalities must differ in some ways, and the differences will ultimately be expressed in differences in their parts, whatever pieces personalities are made of.

BH says, “some folks will insist that the psyche is an emergent property of materialist principles, but that conclusion is an article of faith, not science.” And yeah, maybe this is an article of faith, but I also literally can’t imagine how it could be some other way. In fact, I would say that the faith that the world can be understood as an emergent property of various parts is the core article of faith in science. If things can’t be understood as a system, then what are we doing here? 

BH says, “While we suspect that whatever the psyche appears to be, it is a manifestation of activity at more granular scales, there is no decisive proof.” I never claimed any proof, but again, it’s hard for me to imagine how it could be any other way.

If you look at a Jeep and say, I think it can drive forwards and backwards because of the interactions of the parts inside, you would be right. If you look at a person sick with tuberculosis and say, I think they are sick with these symptoms because of the interactions of the parts inside, you would be right again. 

Maybe these examples are too easy, because we already know that Jeeps and humans are made of granular parts, but I think the logic holds even for a system with unknown parts. If an alien computer crashed to Earth, capable of amazing functions we didn’t understand, I still think it would be reasonable to think that it accomplishes those functions through the interactions of some parts inside. I’m not sure what the opposing position would even look like: “it does those things through a homogenous process that can’t be broken down at all”? Honestly this position seems incoherent. It would have to entirely reject the idea of internal causal mechanisms, which is hard to even think about.

So when people tell me that the mind isn’t the result of activity at a smaller scale, I don’t really know what they mean. Literally what else could it be? Everything else works that way, why not the mind?

Even if you can only explain some things in the aggregate, even if some problems can’t be understood by reducing them to the action of smaller parts, the smaller parts are still there. You may not be able to predict the action of an airplane by studying the screws that hold it together, but the airplane still has screws. 

This is the difference between the naturalist and the biologist: There are basically an infinite number of different bugs; the naturalist’s job is never done. But the biologist can discover what bugs are made of (DNA, proteins, goo, etc.) and build a system that will encompass all bugs, even the ones that haven’t been discovered, even the ones that don’t exist yet. You still need the naturalist to go out and catalogue all the things that can be built out of DNA and proteins. But that doesn’t mean you can’t do biology. There are parts underneath. 

So yeah, I think psychology has discrete, relatively predictable units that will yield to mechanistic reduction. We don’t know what the units are yet, but it’s gotta be made of something. Eventually we’ll find out what that is. I think it’s what we should be looking for now!

Subscribe now

Final comment today is less of a comment, more of an essay, one from Jared Peterson, called Rethinking the Edges of the Mind. Jared told me (personal communication) that this piece was directly a response to my question of "why psychology hasn't had a big new idea in decades".

For starters, Jared offers a great perspective on how paradigms can be different from each other:

We need both: lenses to reveal, and boundaries to isolate. The boundaries prevent distraction and dilution; the lenses help us detect detail and structure. Together, they allow science to temporarily ignore the complexity of the whole in order to understand a part. Because if a psychoneuroendocrinologist had to account for bioengineering and evolution every time they ran a study, they’d never get anything done.

The two questions I wish to ask in this essay are the following: has psychology placed its boundaries in the right place? And does it have the right lens?

This definitely lines up with the list of paradigm ideas in my essay. Some of them are clearly about redrawing the boundaries of the field, like when I ask if psychology needs to include plants. Others are clearly about lenses, like when I suggest that we could focus more on the details of memory storage.

Jared is most interested in re-drawing boundaries, and he frames boundaries in terms of context leaks:

The leaking of context into psychology is positively diluvian. Everywhere you go, you hear a magic phrase; context matters. A phrase that is a tacit acknowledgement that, in our attempt to bracket out the rest of the world and put some dam boundaries around our subject of study, there are gaps large enough for the world to leak back in and disrupt our focus. It’s a phrase that means that what we have left outside the dam boundaries is flooding back in making it difficult to study the thing we are interested in. A phrase that means the dam boundaries are not holding.

Context leaks always happen, but they happen in psychology a lot. This is part of why our ongoing conversation around replication has been so messy. Every replication is different from the original study. It will be conducted in a different room, by a different team of researchers. Even if by some coincidence it is conducted in the same room by the exact same team, it will still take place in a different time, a different political situation, etc. Are those differences meaningful to the effect we’re studying? Nobody knows, and there’s no obvious way to come up with a system of those differences. We don’t know how to keep the context out.

Jared takes this leakage as a sign that our boundaries are wrong. If we drew them better, they wouldn’t leak:

This is one way to understand the difference between the hard and soft sciences. In the hard sciences, like physics and chemistry, the units are more easily isolated, and so context doesn’t leak in and overwhelm the model. You can actually do math because the units of analysis are independent enough to behave according to the basic rules of arithmetic.

This strikes me as basically the same idea as BH’s discrete sciences vs. complex systems.

I guess I’ll bite the bullet here and say that I think that psychology as it’s currently done is indeed a soft science. The units of analysis that psychologists use are so big and so messy that you can’t model them. The boundaries around psychological constructs, at least the ones we use today, are not clear. Right on. 

But this is why I keep coming back to alchemy. Alchemy was also a soft science! The units were not isolated, you could not do math if you tried. All you had were weird findings about how gold wouldn’t separate in a fire but would dissolve in Aqua Regis. But with a lot of effort, people took those weird findings, figured out the units, developed methods to isolate them, and eventually started doing math to the whole thing. They created a hard science, and I don’t see any reason we can’t do it again. At the very least we should aspire to it!

Of course, psychology isn’t quite as lost as alchemy. I think that is a tad too insulting. 

I know that alchemy is thrown around as a dirty word sometimes, but I don’t mean the comparison to be insulting. In fact, I find it uplifting. People can go from alchemy to chemistry in just a couple generations. All you need is some relatively minor shifts in perspective! That makes me feel very good about our odds of making psychology into a mature science some time before I die (in the 2440s, of course).

In mature sciences, claims come with relevant provisos. “Sugar dissolves in water” assumes you don’t freeze it instantly. “Vinegar and baking soda create a volcano effect” assumes you’re in earth-like conditions, and so it doesn’t count as a replication if you do it on Pluto. These provisos are an implicit part of the paradigm, left unstated simply because there are simply too many of them.

I hear this kind of reasoning a lot, but I think it’s wrong.

“Sugar dissolves in water” is not how a chemist really thinks. It’s a gloss, chemical understanding translated very approximately into common language. A chemist actually thinks something more like, “crystal lattices held together by intermolecular forces (mainly hydrogen bonds) undergo physical dissolution in polar solvents through intermolecular interactions. The molecules in the crystal lattice are solvated, disrupting the lattice without breaking covalent bonds.” That’s closer, but it’s still a rough gloss to keep things to just two sentences.

The point is, it’s not that chemists have 1001 provisos that tell them what’s in and what’s out. It’s that they have a system, and that system has specific constraints. A chemist has a model of the world in their head. The “assumes you don’t freeze it instantly” and “assumes you’re in earth-like conditions” aren’t provisos or exceptions. They’re left unstated because they are baked into the model.

In Settlers of Catan, someone might say, “you can’t build roads if you don’t have access to brick”. Someone else might object that you can in fact build a road without brick if you have enough cards that you can trade for a brick card — for example, if you have 3 sheep and a 3:1 harbor of the appropriate type. But this is all baked into the rules, and the original statement wasn’t made in ignorance of these interactions.

This is why generalization has to be handled on a case-by-case basis. A good behavioral scientist never assumes a finding generalizes because all our theories are sometimes theories.

People are loss averse…sometimes. People obey authority figures…sometimes. Jared is agreeable…sometimes.

No psychological theory is as law-like as anything in chemistry because we don't know the provisos. Context leaks in, and we can’t cleanly isolate causes, or pin down universal mechanisms. Many of are most important findings are just glorified averages. (e.g., people tend to be Loss Averse)

And maybe part of the reason for this unsatisfactory situation is that we’re stuck using categories as confused as earth, water, air, and fire—studying the phases of the elements of reality, but not the elements themselves. If we are not ever studying the right type of things, of course the models will fall apart. Of course we can't define the provisos, consistently replicate findings, or predict how things will generalize.

I like this metaphor. Concepts like “agreeableness” definitely seem more like the phases of psychology than the elements.

It’s easy to see when someone is agreeable, just like it’s easy to see when something is a liquid or a gas. But it’s hard to see what little bits are behind the agreeableness, just like it was hard to find the elements. 

And all this makes me wonder what would it take to redraw our boundaries? To move beyond endlessly repeating "context matters" without ever saying exactly what that context is? To build models that can replicate not by ignoring context, but by learning to incorporate it?

Maybe psychology’s leakiness is inevitable. Maybe, like Kevin Munger suggests, we need to abandon replication as the cornerstone of the social sciences. After all, if physicists struggle with modeling three bodies in space, why should psychologists—who study something far more multi-causal—expect to ever have anything less ambiguous “people tend to be loss averse”?

And yet, I can't shake the sense that psychology systematically underfits the complexity of human behavior by treating context as noise rather than structure. That our models leak not just because human behavior is messy, but because we drew our dam boundaries in the wrong damn place. Because if it changes behavior, then shouldn’t it be the very content that we study?

One of the most hopeful things I know is that in high-dimensional spaces, local minima are very rare. You can always find some kind of improvement just by moving around. So while it may be hard to find the right boundaries, and there may be some discontinuity, I think there’s a lot of promise in just trying to re-draw our boundaries in as many ways as we can, and seeing what happens. I offered a few suggestions in my original piece, but we could easily come up with more.

Abandoning replication, on the other hand, is the worst idea I’ve ever heard. If you think a question is important enough to bother running a study about it, you should also want to know if other people find the same thing when they run a similar study. And more than that, no one else should have to take your word for what you found in your study — they should run their own version of the study and make up their mind for themselves.

Perhaps the reason cognition and behavior don’t appear in Wimsatt’s chart is that they are not the result of compositional layering. Wimsatt’s chart shows levels that emerge by combining entities of the same kind: atoms into molecules, molecules into cells, cells into organisms, etc. Each level nests within the next, preserving ontological similarity.

But cognition isn’t built that way. It doesn’t arise from the stacking of like onto like. Instead, it emerges from the interaction between different kinds of things, namely, the organism and its environment.

Cognition, in this view, isn’t a “higher level” built atop the others. It’s a relational phenomenon—something that exists between levels rather than above them. That may be why it resists being plotted on the chart: it breaks the compositional metaphor that underlies it.

I strongly disagree. I think cognition does work by combining entities of the same kind, we just don’t know what those entities are yet.  

This line of argument makes about as much sense to me as saying, “auto engineering is a relational phenomenon, it emerges from the interaction between different kinds of things, namely, the truck and the road.” This is true in some sense — obviously the truck was designed with the road in mind, obviously it responds to and is shaped by the road. But it’s also true that the truck is built by combining entities of the same kind. You really can study the nature of the truck!

The fact that organisms interact with their environment doesn’t mean that you can’t study biology as a hard science. You can invent cell theory, disprove spontaneous generation, discover the base pairs, etc. Biology is made of entities of the same kind, even though it’s also relational.

To zoom in on one section in particular:

…shows levels that emerge by combining entities of the same kind: atoms into molecules, molecules into cells, cells into organisms, etc. Each level nests within the next, preserving ontological similarity.

People enjoy this pattern a lot, how it seems like you can explain biology in terms of chemistry, and chemistry in terms of physics. But I think this pattern, while cute, has been misleading. Scientific ontologies don’t have to nest inside each other.

I think psychology may be more like evolution, an information science. Evolution was discovered in the context of biology, but the paradigm doesn’t just apply to DNA, it applies to everything that suffers variation and selection. That’s why we can, at least potentially, apply evolutionary thinking to things like memes, or software.

Computers can be built out of anything, as far as we can tell: silicon, gears, hydraulics, you name it. The arrangement and the function matter more than the materials you use. Cognition is also probably less about what the pieces are, and more about how pieces are arranged. You can make logic gates out of swarms of crabs — so you can probably make the pieces of the mind out of different substrates, if you put your mind to it.

Last week, after years of fraud allegations, and an 18-month investigation that found her responsible for research misconduct, Harvard Business School revoked the tenure of professor Francesca Gino. 

It started with a series of blog posts from the psychologist-bloggers at Data Colada, alleging fraud in her papers. “We believe that many … Gino-authored papers contain fake data,” they wrote at the time. “Perhaps dozens.” Since then, several of her papers have been retracted, her data has been picked over by outside forensics, and Gino has sued both the bloggers and Harvard for $25 million.

Life has many strange coincidences, and one of mine is that I worked for Francesca Gino. I was fresh out of college, and had just gotten a part-time job as a research assistant at Harvard Business School. The professor I was working with had just started collaborating with Gino, and they wanted to bring me on to do some of the legwork. So I joined the project.

I can’t remember what kind of RA work I did, but I do remember I had at least one meeting with Gino, maybe more than one, where we discussed the research. That part of the memory is unusually clear; I could probably go back to HBS and find the room where it happened.

I barely knew Gino at all, but in a way, I was strangely close to these events. When I was studying psychology as an undergrad, Gino’s research was some of my favorite. I knew all her papers. My senior thesis project used materials from one of her studies. When I sent Gino a cold email to ask for the number-matching task she sometimes used to measure cheating, she responded right away, and kindly sent me the MS Word doc containing the task. I’m even in the acknowledgments of one of her papers. Look:  

Subscribe now

Remember Amy Cuddy? Champion of “power posing”, until that entire literature fell apart? Star of the erstwhile most popular TED talk of all time (currently at 74.5 million views)? Practically the poster child for the replication crisis?

I briefly worked for her too. This was 2012, and I was a summer research assistant at a lab in the Harvard school of government. But HBS professors sometimes ran their studies in our lab space, and the interns were sometimes loaned out to do RA tasks for people at different places around the university.

So for about one month, my part-time research services were on loan to Amy Cuddy. According to my notes from the time, I spent those hours coding videos, though I don’t recall the videos or what I was coding. Another study that summer involved saliva collection — I remember asking participants to spit into the little tubes. This was probably for cortisol testing, a repeat feature of power posing research, so the saliva study might have been a Cuddy project too.

I met Cuddy herself at least once. I walked across the Anderson Memorial Bridge and onto the obscenely beautiful Business School campus, and ended up in a meeting room with glass walls. I didn’t put it together at the time, but this must have been shortly after she recorded her famous TED talk, maybe just days later. I remember one of the postdocs asked her, “How was Edinburgh?”, and she told him about her flight back.

To cap it all off, that summer I went to a talk by Dan Ariely (a major Gino co-author, who has also been accused of fabricating data), and got him to sign my copy of his book. I was just excited to meet the author of some of my favorite studies. I had no idea I was in the thick of it.

It’s easy to think that the takeaway of the Gino story is about the psychology research, or about the allegations of misconduct. That’s what most of the coverage leads with. And it’s easy to see why — “Professor Who Studied Dishonesty Under Investigation For Dishonesty” is a catchy headline.

Even so, I think those parts of the story are incidental. Francesca Gino happened to be at the center of the story in our timeline, but it could have been someone else. Many famous professors are just a hair away from being investigated for fraud. 

To me, the real story is that for the first time in approximately 80 years, Harvard has revoked a professor’s tenure. In fact, it’s the first time Harvard has revoked any professor’s tenure since it introduced the concept of tenure in the 1940s. It’s almost unprecedented.

There have been a lot of high-profile scandals in the fifteen or so years of the replication crisis, but all of those stories ended differently. Admitted fabricator Diederik Stapel resigned his professorship. So did Brian Wansink of Cornell. Marc Tessier-Lavigne resigned as president of Stanford, but remained a tenured professor. Amy Cuddy “quietly left” her tenure-track job at Harvard. But Gino fought and forced their hand.

It was a different time. Remember when TED talks used to have cultural gravity? But this is now. Things are changing for higher education, for journalism, for the institutions of science. I think it is genuinely a different world.

I can’t tell you how this will end, but I do know something about how it began. Because for reasons I don’t totally understand, I was there.

This summer at FractalU Boston, I’m teaching a 6-week course called Welcome to the Replication Crisis. We’ll do close reads of the hits — the landmark papers, blog posts, and takedowns behind psychology’s recent identity crisis. You’ll leave the course with a forensic eye for p-hacking, replication failure, and the forces reshaping how science gets done.

If you’re in the Boston area this summer, there’s still room to sign up! I’d love to have you join us.

A science needs a paradigm because we underrate a simple fact: there are an infinite number of things you can study and infinite ways to study them. Without some sense of which things are in bounds and which things are out of bounds, you end up totally confused. "A dog's bark is louder than a cat's meow, is that important?" "I kept track of how often I think about giardiniera peppers and it's 1.4 times a month, is that something?" "The maximum speed of my car doesn't change when I paint it different colors, maybe that's relevant?"

Without a paradigm, all you have is a topic. But a paradigm gives you shape. Chemists agree that mass can’t be created or destroyed, so when you’re studying chemistry, you make sure to weigh everything very carefully. In the physics department, they measure how much energy they put behind a cannonball. They don’t measure what color they painted it.

Under normal conditions, scientists working inside a paradigm don’t question these commitments any more than Formula 1 drivers question the fact they're about to race cars instead of bobsleds. Instead, the common language of the paradigm makes it easy to work together, make sense of the world, and go really really fast. 

Since a paradigm lets you go so fast, you might not notice you’re inside one at all. Scientists often work inside a paradigm without having any idea what a paradigm is. They might even deny they're doing any such thing. But you actually can’t get away from this. "Paradigm" is a description of what your community assumes and values; you have one whether you like it or not.  

For better or worse, no paradigm is perfect. Over time, people run across observations that don't fit neatly into the framework. Kuhn called these "anomalies”. At first, scientists assume that these are interesting puzzles that will eventually be solved using current methods, so the anomalies are downplayed or ignored. But as the list of unsolved problems grows, doubts about the paradigm begin to spread.

Anomalies do more than just accumulate, they clash with the deeply held commitments of the paradigm. And they’re more than just awkward, anomalies eventually make the paradigm’s commitments impossible to defend. When anomalies can’t be ignored any longer, the science enters a period of crisis that ends only when the old paradigm collapses, and a new one takes its place.

When a science switches out its old paradigm for a new one, we call it a scientific revolution. This isn’t “cute” or “heartwarming”, sciences only do this when they’re in extreme distress.

Paradigms shape everything scientists do and think, so the revolutions that overthrow them are profound. The new paradigm doesn’t just answer the old questions in a new way—it changes how scientists think about their problems. It redefines what details are important, how you measure success, and what truths are worth pursuing. 

A paradigm shift is like going from playing checkers to playing chess. Oh, you thought there was one kind of game piece and they all behave the same way? No, there are six different kinds of pieces, and they follow different rules. Some can only go forward, some can jump over other pieces, and some cause the game to end if they get captured.

This may sound silly, but in the chemical revolution we went from thinking there were 4-7 elements to thinking there might be dozens of elements. Chemists went from thinking there was one kind of air to having to invent a new word for the quickly growing category of air-like substances, AKA the category of “gas”.

So as a psychologist, I can’t help but spend some time thinking about psychology’s next paradigm. 

The State of the Art

Generously, the current paradigm in our field is cognitive psychology. This basically boils down to, “perhaps the mind is like a digital computer”.

Ulric Neisser, the psychologist who wrote the book Cognitive Psychology, asks us to “consider the familiar parallel between man and computer”. He tells us,

The task of a psychologist trying to understand human cognition is analogous to that of a man trying to discover how a computer has been programmed. In particular, if the program seems to store and reuse information, he would like to know by what “routines” or “procedures” this is done.

…

Although a program is nothing but a flow of symbols, it has reality enough to control the operation of very tangible machinery that executes very physical operations. A man who seeks to discover the program of a computer is surely not doing anything self-contradictory!

This particular framing is a defense against psychology’s previous attempt at a paradigm, behaviorism. Hardline behaviorists argued that because behavior is the only thing that can be observed, behavior is the only part of psychology that can be studied scientifically. You can’t observe a thought, so you can’t possibly measure it or run studies on it. Behavior only.

Cognitive psychology was a direct reaction to these ideas, and part of why the cognitive revolution was successful is that the computer, so obviously full of programs and subroutines, made it respectable to speculate about thinking again. Neisser says: 

There were cognitive theorists long before the advent of the computer. … But, in the eyes of many psychologists, a theory which dealt with cognitive transformations, memory schemata, and the like was not about anything. One could understand theories that dealt with overt movements, or with physiology; one could even understand (and deplore) theories which dealt with the content of consciousness; but what kind of a thing is a schema? If memory consists of transformations, what is transformed? So long as cognitive psychology literally did not know what it was talking about, there was always a danger that it was talking about nothing at all. This is no longer a serious risk. Information is what is transformed, and the structured pattern of its transformations is what we want to understand.

Cognitive psychologists, or at least the ones who think about such things, certainly feel that this was a scientific revolution in Kuhn’s terms. In the 2014 Introduction To The Classic Edition of Neisser’s book, the psychologist Ira Hyman wrote:

Much like political revolutions, scientific revolutions need rallying cries. The cognitive movement was a scientific revolution and [Neisser’s book] Cognitive Psychology became the rallying cry for the cognitive revolution. In the 1950s and 60s, Psychology needed a scientific revolution. In Kuhnian terms, the field was ready.

Before behaviorism, psychology passed through what I can only describe as “early psychology”, a period where everyone was obsessed with the words “consciousness” and “introspection” — even though as far as I can tell, none of them agreed what those words might mean. 

Less generously, psychology doesn’t have a paradigm and never has. My friend Adam Mastroianni makes the case for this position in his essay, I’m so sorry for psychology’s loss, whatever it is:

Psychology doesn’t exactly have a paradigm; we’re still too young for that. But we do have … proto-paradigms. We’re currently stuck with two proto-paradigms that were once useful but aren’t anymore, and one proto-paradigm that was never useful and will never be.

The first of the formerly useful [proto-paradigms] will be familiar: this whole cognitive bias craze. Yes, humans do not always obey the optimal rules of decision-making, and this insight has won two Nobel Prizes. …

The second formerly useful proto-paradigm is something like “situations matter.” This idea maintains that people's contexts have immense power over their behavior, and the strongest version maintains that the only difference between sinners and saints is their situations. …

The third proto-paradigm has never been scientifically productive, and won't ever be. It's also a little harder to explain. Let's call this one “pick a noun and study it.”

I’m with Adam. Despite some honest attempts, psychology has never had a paradigm, only proto-paradigms. We’re still more like alchemy than chemistry. And we won’t be like chemistry until we have our first paradigm.

This leads us to the obvious question: how might we go about getting our first paradigm?  

Active Leads

Thomas Kuhn was trained as a physicist. He had three degrees from Harvard, all of them in physics. Physics has had a paradigm since the Ancient Greeks, so the question of where a science gets its starter paradigm was not very natural to him. Kuhn was mostly interested in revolutions from one paradigm to another. 

On top of that, Kuhn’s first book was about the Copernican Revolution in astronomy. If anything, astronomy has had a paradigm for longer than physics has! In Kuhn’s experience, sciences always have some kind of paradigm waiting when you show up, and the main question is why there’s this cycle where they sometimes throw out the old one and bring in a new one. 

The political metaphor, which he uses when discussing these ideas, doesn’t help. Political revolutions are always conducted in reaction to an existing government and always replace that government with a new one. Where the state got its first government is a different kind of question entirely.

As a result, Kuhn didn’t say much about how a science gets its first paradigm. 

But he did say a little. In the introduction to The Structure of Scientific Revolutions, Kuhn talks about how the pitiful state of psychology is part of what inspired his ideas about paradigms in the first place. He mentions “the controversies over fundamentals”, “the number and extent of the overt disagreements about the nature of legitimate scientific problems and methods”, and so on. He could tell that psychology was conspicuously missing something that was not missing in astronomy, physics, chemistry, or biology.

In a later essay, Kuhn said,

There are many fields—I shall call them proto-sciences—in which practice does generate testable conclusions but which nonetheless resemble philosophy and the arts rather than the established sciences in their developmental patterns. I think, for example … of many of the social sciences today.

So it’s clear that Kuhn always had psychology in the back of his mind. However, when it came time to give advice about how we might get out of this hole, he was pretty unhelpful. He said: 

I conclude, in short, that the proto-sciences, like the arts and philosophy, lack some element which, in the mature sciences, permits the more obvious forms of progress. It is not, however, anything that a methodological prescription can provide. Unlike my present critics, Lakatos at this point included, I claim no therapy to assist the transformation of a proto-science to a science, nor do I suppose that anything of the sort is to be had. If, as Feyerabend suggests, some social scientists take from me the view that they can improve the status of their fields by first legislating agreement on fundamentals and then turning to puzzle-solving, they are badly misconstruing my point.

Let’s start with the parts I agree with. Kuhn is right that trying to force a paradigm is a mistake. When people try to legislate agreement, you get unforced blunders, like saying that Pluto isn’t a planet.1

But I think he’s wrong that nothing can be done. I think there are many things we can do if we want to kickstart the revolution. If nothing else, we can keep ourselves aware that it’s a possibility. 

This is part of why I’m so interested in the transition from alchemy to chemistry, even to the point of sometimes trying alchemy experiments for myself. I think the alchemists didn’t have a real paradigm — alchemy was a proto-science. But when a paradigm was discovered, that gave us chemistry. This makes the transition from alchemy to chemistry one of the few examples of a science getting its first paradigm that’s actually well-documented.

Whether he means to or not, Kuhn does give us a couple of leads. In my opinion, here are the main two: 1) most of the time, the new paradigm is an old idea that’s brought back from the archives, and 2) a change in paradigm usually goes hand-in-hand with some kind of change in scope. 

Old Hats

Kuhn points out that new paradigms usually come from old ideas, dreamed up back in some misty past, and resurrected only when the field is in crisis. “Often a new paradigm emerges,” he says, “at least in embryo, before a crisis has developed far or been explicitly recognized.”

Sometimes the proposal was developed during the last crisis, but was passed over at the time:

In cases like these one can say only that a minor breakdown of the paradigm and the very first blurring of its rules for normal science were sufficient to induce in someone a new way of looking at the field. 

Sometimes the proposal was invented when the field wasn’t in crisis at all, so the idea of switching to a new paradigm seemed crazy at the time:

The solution [to many crises] had been at least partially anticipated during a period when there was no crisis in the corresponding science; and in the absence of crisis those anticipations had been ignored.

The new paradigm isn’t drawn directly from the void. Instead, it’s usually an idea that was discussed, dismissed, and later called back into service. Every science has a few benchwarmers, some of them warmer than others. Or occasionally, it’s a similar idea from a neighboring science. 

Psychology is pretty young as far as sciences go; our bench is not all that deep. Unlike physics and astronomy, we can’t just dig down and find a different set of foundations laid out 2000 years ago by the Ancient Greeks. But we can look back to previous eras of crisis, go through their waste bin, and try to figure out if any of the ideas they passed over still have some juice. 

Scope

A new paradigm always means a new scope for the field.

Early psychology was all about “consciousness” which was often studied through introspection. If that sounds impossibly fuzzy, to the point where you have a hard time believing it, I can’t blame you. But here’s William James delivering the same message with insane confidence (emphasis in the original): 

Introspective Observation is what we have to rely on first and foremost and always. The word introspection need hardly be defined - it means, of course, the looking into our own minds and reporting what we there discover. Every one agrees that we there discover states of consciousness. So far as I know, the existence of such states has never been doubted by any critic, however sceptical in other respects he may have been.

With this set of commitments, the natural scope of psychology was human beings. It’s not clear if other animals are conscious, and even if they are, you can’t ask them to introspect about their experience and report back. So you couldn’t do psychology on mice and dogs.

After the proto-revolution that gave us behaviorism, the scope shifted. Introspection was out, and behavior was the only law. Suddenly, animals were acceptable research subjects for psychology. 

In fact, animals were now better research subjects than humans. Behaviorists believed that all of an organism’s behavior came from the wiring between stimuli and response, associations formed over a lifetime of experience. This meant that any experience outside the lab would mess with your research. If the organism had ever encountered a stimulus before, it might have already formed some associations.

The ideal research subject had to be raised in a controlled environment from the moment of birth, and you weren’t allowed to do that with humans, at least not until the glorious behaviorist revolution was complete. So with the exception of a few infamous experiments involving human infants, behaviorists found that in the meantime it was best to confine themselves to studying animals, especially rats and pigeons. This is where the stereotype of psychologists studying rats in mazes comes from, which persists even though that stereotype is more than a generation out of date.

After the cognitive revolution, thinking was back in. Humans mostly, but not entirely, replaced animals as the subject of serious psychological research.

It’s clear why humans were acceptable again, but I don’t really understand why we stopped studying animals. To a cognitive psychologist, both human and animal minds are kinds of programs, or computers, and you can run basically the same studies on either. So why did we switch almost entirely back to people? Here are some theories: 

Social psychologists, who did pretty well riding the wave of the cognitive revolution, wanted to study social relationships. Humans arguably have more complex social interactions than other animals, and social psychologists were interested in social questions very specific to humans, like “can we cooperate well enough to avoid nuking the planet?” For them, studying dogs or mice was a non-starter.

We are also, I think wrongly, paranoid about anthropomorphising animals. There’s this superstition that it’s unscientific to speculate about whether a rat working her way through a maze is having a good time. You have to frame the minds of animals in terms of behavior, or in terms of hormones, or else it isn’t “rigorous”. Meanwhile it is entirely scientific to hand a human a scrap of paper that says I AM ENJOYING THIS DAY 1--2--3--4--5--6--7 and treat their response like it came from a particle accelerator. 

Another answer is more mundane. Today, the most common research method is the survey. These weren’t an option under behaviorism, but when behaviorism fell, they became an acceptable method again. Surveys have many advantages, but one drawback is that it’s surprisingly hard to give a research survey to frogs, birds, or dolphins. This also favored human subjects.

And of course, the most common research subject today is not just any human, it’s a very specific kind of human: the undergraduate psychology major at a top US research university. These subjects are special because they’re really close at hand. Unlike rats and pigeons, you don’t have to house, feed, or raise them. You don’t even have to go find them. If you are a professor at a research university, they sign up for your classes; they actually pay you! This explains why so many psych research papers use undergrads; or at least they did, until paying people to take online surveys became even easier. Perhaps convenience eats all worldviews

The transitions from introspection to behaviorism to cognitive psychology are good examples of what it looks like to change scope. But I don’t think any of these were successful revolutions, because none of them gave psychology a real paradigm (though behaviorism came surprisingly close).

These kinds of border skirmishes are common enough. The study of heat and light used to be considered a part of chemistry; now they’re both squarely inside physics. But real revolutions tend to be much more upsetting. So maybe these revolutions in psychology weren’t successful because they didn’t change the scope enough.

Kuhn usually acts as though every revolution involves just one science that has clear boundaries. A crisis comes along, and you swap out the old paradigm for a new one — but you have one paradigm before the revolution, and one paradigm afterwards, and the boundaries don’t change much. This does describe a few cases, like the Copernican Revolution. But the reality wasn’t usually so simple.

When science was just getting started, people seemed to expect that it would stay united in one big happy field, which they called natural philosophy. But they were wrong. Science ended up splitting up into a bunch of different fields. Within a couple of generations they had already specialized into physics, chemistry, biology, and various others. Today, even sub-disciplines like organic vs. inorganic chemistry arguably have different paradigms.

This didn’t seem very painful. My impression is that people trying to figure out natural philosophy came up with a bunch of different directions, people signed on to whichever one they liked best, and 100 years later people discovered they were in different fields. They diverged, but it was pretty quiet and amicable. Not much of a revolution!

More famous is the exact opposite. 

Newton’s physics combined mechanics (“the area of physics concerned with the relationships between force, matter, and motion among physical objects”) with astronomy. This seems unremarkable to us, but it’s hard to express just how surprising it was at the time. 

Nothing could be so obvious as that the Earth and the heavens are subject to separate laws. The heavens are perfect and eternal, while the Earth is changing and impermanent. The laws that govern Earth are the laws of gross bodies (which change, decay, and rot), while the laws that govern the motion of the spheres are divine (the spheres are probably pushed around by angels or something). 

But in Newton’s paradigm, mechanics and astronomy were governed by the exact same set of laws. Worse, instead of being formed of pure celestial fire, the stars were made of the same kind of crude material substance as the Earth. And worse still, Newton’s theory of gravity said that not only did the stars and planets exert an influence over the Earth (acceptable), but the Earth itself had an influence over the stars and planets (unacceptable), in fact the exact same kind of influence in the same measure (practically blasphemy). But Newton’s paradigm was so useful that eventually there was no sense in seeing things any other way.

Other examples are out there if you look. Something similar happened when James Clerk Maxwell unified electricity, magnetism and light. We thought these were three largely different things, and physics changed when Maxwell showed they could all be described by the same laws.

If psychology doesn’t have a full scientific paradigm right now, then the next paradigm won’t come from a real scientific revolution, because revolutions only happen when a mature science switches between one complete paradigm and another. This does limit how much we can learn by looking at stories from other revolutions. 

But as the examples from revolving in and out of behaviorism show, we should expect that any serious attempt at building a paradigm will involve some kind of change in scope. 

We may not be able to predict the next paradigm. But if we assume that the next paradigm, whatever it might be, will mean some kind of change in scope, we can look for our next paradigm by thinking about what kinds of changes in scope seem possible. We can consider how we might shift our boundaries, what divisions we can ignore, what new distinctions we can adopt, what neighboring fields we might be absorbed by, and even what fields we might be able to absorb. 

If psychology’s first paradigm does come from a major revolution, there’s a good chance it will involve the field splitting up, or different fields coming together. “Psychology” as a field may not survive any more than Alchemy or Natural Philosophy did. But it wouldn’t be wrong if we tear down our walls and use the stones to build something better, and we shouldn’t be afraid of the possibility that the resulting science might have a new name or new boundaries. 

Subscribe now

And the Paradigm is….

In a moment I’m going to show you a list of some possible paradigms, directions that might be a hint at the future of psychology.

This particular list is based on discussions I had with students in CS-0257 Psychological Revolutions, a class I taught at Hampshire College in Fall 2023. So before diving into the list, I want to give special thanks to my students for their part in coming up with these ideas.

This list isn’t the only list of possible futures. I won’t even claim this is my list per se. But after a semester of conversation, these are the ideas that had the best showing, and I figured an artifact like that should be preserved for posterity. 

Psychology + Neuroscience

Uniting psychology and neuroscience is a very popular idea. Laypeople often assume that psychology and neuroscience are already one field, and are surprised to hear they’re not.

But to psychologists, combining the two fields would be revolutionary. The very suggestion challenges a core assumption about implementation, or the details of how the brain is wired up. You might think that the wiring is very important, but behaviorism and cognitive psychology both agree that implementation doesn’t matter, and they’re very explicit in this belief.

Behaviorists were interested in the physical structures of the body. But they saw the nervous system as not special in any way, no more deserving of their study than any other part of anatomy. John Watson, who wrote the book Behaviorism and kind of founded the field, said: 

Should We as Behaviorists Be Especially Interested in the Central Nervous System? Because he places emphasis on the facts of adjustment of the whole organism rather than upon the working of parts of the body, the behaviorist is often accused of not making a place in his scheme for the nervous system. … Many of our so-called physiological psychologies are filled with pretty pictures of brain and spinal cord schemes. As a matter of fact we do not yet know enough about the functioning of brain and spinal cord to draw diagram about their functions. For the behaviorist the nervous system is, 1st, a part of the body—no more mysterious than muscles and glands; 2nd, it is a specialized body mechanism that enables its possessor to react more quickly and in a more integrated way with muscles or glands when acted upon by a given stimulus than would be the case if no nervous system were present.

Elsewhere in the same book, Watson gives this example:

Responses may be wholly confined to the muscular and glandular systems inside the body. A child or hungry adult may be standing stock still in front of a window filled with pastry. Your first exclamation may be “he isn’t doing anything” or “he is just looking at the pastry.” An instrument would show that his salivary glands are pouring out secretions, that his stomach is rhythmically contracting and expanding, and that marked changes in blood pressure are taking place—that the endocrine glands are pouring substances into the blood.

Watson was a bit unusual, even for a behaviorist, and some of his positions were extreme. But this position survived into mainstream behaviorism. B. F. Skinner expressed the same idea, sometimes even more bluntly, like when he said: 

In regarding behavior as a scientific datum in its own right and in proceeding to examine it in accordance with established scientific practices, one naturally does not expect to encounter neurones, synapses, or any other aspect of the internal economy of the organism. Entities of that sort lie outside the field of behavior as here defined.

Or when he said:

Cognitive psychologists like to say that “the mind is what the brain does,” but surely the rest of the body plays a part. The mind is what the body does. It is what the person does. In other words, it is behavior.

Behaviorism kind of goes back and forth between saying neurons are outside of psychology, and saying that neurons are included, but only a small part of a whole-body approach. But the idea that behavior can be reduced to the actions of neurons alone is absolutely rejected. 

However, I think Skinner is misrepresenting the cognitive psychologists in that last quote, because they don’t care much about the brain either. Yes, these days they will gladly stick undergrads in an fMRI machine so they can put “brain scans” in their papers; this makes it easier to win grants and get published in fancier journals. But they don’t include these as a core part of their science — scanning brains is only useful insofar as it helps you get closer to mental states (and win grants and publish papers). The actual brain is no more interesting to them than the details of the ear, skin, or eye. As I heard one psychology professor say, “I don’t know my axons from my dendrites.” 

If you go back to the foundations of the field, you can see that cognitive psychology doesn’t encompass neuroscience either, and that this exclusion is a real commitment. In the book Cognitive Psychology, Ulric Neisser wrote: 

Psychology is not just something “to do until the biochemist comes” (as I have recently heard psychiatry described); … [the psychologist] will not care much whether his particular computer stores information in magnetic cores or in thin films; he wants to understand the program, not the “hardware.” … Psychology, like economics, is a science concerned with the interdependence among certain events rather than with their physical nature. … We must be careful not to confuse the program with the computer that it controls. Any single general-purpose computer can be “loaded” with an essentially infinite number of different programs. On the other hand, most programs can be run, with minor modifications, on many physically different kinds of computers. 

Many people are surprised to learn that there’s some real math behind this idea. The Church–Turing thesis is the name for the idea that there’s only one kind of computation: If you can run some program on one computer, you can run it on any other kind of computer, and it doesn’t matter if that computer is made of gears, levers, hydraulics, silicon — or yes, neurons. 

This is where you get all those silly stories about how you can build computers using swarms of crabs, or how you can build a computer that can play Minecraft inside Minecraft. As long as it’s Turing-complete, you can run anything on anything (though if your computer is composed of swarms of crabs, it will be slower and buggier than other computers). 

This thesis hasn’t been strictly proven in the way mathematicians usually like, but we’ve never found a program that can run on one kind of computer but not on another, or a kind of computer that can work through algorithms that other kinds of computer can’t figure. It would be an enormously big deal to discover some kind of incompatible program, or totally original kind of computer. But we have never discovered such a thing, and there aren’t any leads.

(To the best of my understanding, quantum computers are not a different kind of computation — they’re able to do some things much faster than classical computers, but mostly in the same way that your silicon computer is much faster than a computer made up of swarms of crabs.)

I’m not sure what to think of the idea that neurons don’t matter. On the one hand, even though most people find it unintuitive, treating implementation like it doesn’t matter is a core commitment of both behaviorism and cognitive psychology. And it has pretty solid support from information theory and what we understand about computation. Whatever algorithms might govern the mind, you should be able to run the same algorithms on a PS4, they will just take slightly longer. 

On the other hand, any major assumption shared by both behaviorism and cognitive psychology is exactly the kind of assumption that might be holding us back! Any shared assumption is likely to be shared because it is a blind spot. Maybe this is one of those sacred mysteries — their commitments disagree with common sense only because if they didn’t have something counterintuitive to say, laypeople wouldn’t feel the need to listen to them at all.

I also think it’s possible to take the information theory arguments too seriously. It’s true that (as far as we know), any algorithm can be run on any computer, regardless of physical substrate. But that doesn’t mean that all physical systems compute identically. Computers made out of silicon are much smaller and run much faster than computers made out of brass gears, which is why you’re reading this on a silicon computer and not a brass one. The fact that the brain is made of neurons probably can tell us something about the mind; though it’s not clear how much it can tell us, or how important that information might be. 

What would it even mean for neuroscience to subsume psychology? I think it would be too nitty-gritty. Doing psychology as neuroscience seems like trying to describe the subway system without being able to talk about trains, stations, or riders — instead you can only talk about atoms and molecules. Trains, stations, and riders are all made up of molecules, but this seems like the wrong level of analysis. 

My intuition is that the marriage of psychology and neuroscience simply isn’t weird enough. Newton didn’t invent his paradigm by switching to a more common-sense perspective on physics. He got there by putting together pieces no one else expected to go together, and ending up with a result no one imagined, a result that almost everyone found shocking and offensive. I don’t think psychology will drop such a bombshell, but I do think whatever happens will be pretty strange. 

Psychology + Economics

Psychology and economics are both fields about people — their thoughts, their behavior, and how they decide what to eat for lunch.

The main reason to suspect these could become the same field is that when people have tried to combine the fields in the past, it’s gone pretty well. There’s no Nobel Prize in psychology, but when psychologists win Nobel Prizes, they win them in economics; one in 2002 and (arguably) one in 2017. Some of the studies that came from this collaboration are among the most famous and robust in psychology; that shows a lot of promise. Psychologists don’t win Nobel Prizes in medicine or chemistry, or at least we haven’t yet. 

However, the main reason to suspect these won’t become the same field is… all that promising work. The research that won those Nobel Prizes was mostly conducted in the 1970s, and arguably that was when the relationship peaked. If a union between psychology and economics were going to happen, you’d think that after 50 years of effort, we would have seen more progress. At some point you have to accept that the relationship is over, and move on. 

There's good reason to think that the union of economics and psychology was a weekend fling rather than a marriage: the entire relationship hinged on the falsification of the Rational Actor Hypothesis. That was the one thing they had in common, nice while it lasted, but now they're running out of things to talk about.

But this isn’t a knock-down argument. Economics doesn’t have a paradigm either, and the overlap is hard to ignore. Perhaps we’re just smushing together psychology and economics in the wrong way. If we smush together different parts of the fields, maybe that will prove a more successful smushing.

Psychology + Medicine

An underrated fact about psychology is that the brain is an organ like the kidneys, lungs, and spleen.

Some people will say that while it’s true the brain is an organ, it is a much more important organ than the spleen. To these people I say:

As a hypothesis, “there’s a set of laws that governs all biological organs” sounds decently plausible. Whatever these laws are, they’ll govern the brain as much as they govern the spleen. If the laws are trivial, like “organs are wet and inside the body”, that won’t make for much of a paradigm. But if there are profound similarities between all the organs, that begins to sound like a real science.  

The main problem here is that medicine doesn’t have a paradigm of its own, so we can’t freeride on their hard work. You’d have to invent a new paradigm that includes some kind of commitments about the function of organs, and gives constraints that apply equally to the brain and the kidneys.

I’d find this outcome kind of amusing, because it reminds me so much of behaviorism’s whole-body approach to psychology. That commitment was abandoned when behaviorism fell, but strictly speaking it’s not incompatible with cognitive psychology. The mind can be a computer whether all of that computation happens in the brain or elsewhere. So even though I think behaviorism was wrong, this gives the possible union with medicine a little more promise, because we know the idea used to be popular and there was no obvious reason to abandon it. 

Psychology of Plants

Under behaviorism, psychologists spent a lot of time studying non-human animals. When behaviorism fell, animals were mostly dropped. These days, almost no psychologists study animals. 

One change we can tinker with is making this circle of research subjects smaller or wider. It’s hard to see how you could make the circle smaller. Psychologists these days claim to study humans, but they mostly study adults. Actually, mostly just adults in their 20s. Actually, mostly college-aged psychology majors studying at American research universities.  

So it seems more promising to think about how we could make the circle wider. We could expand it back to the scope of the behaviorists, and include all animals. Or we could expand it even further.

Everyone knows that Venus flytraps snap shut their jaws on their prey, and that daisies turn to follow the sun. Isn’t that behavior? You can point out that a Venus flytrap only closes its jaws when something touches its trigger hairs. But I don’t see how this is meaningfully different from a statement like, “my dog only bites me when I poke it in the eye”. There’s a motor action that occurs in response to a stimulus, and something happens in between. Go to the Wikipedia article on plant arithmetic, thou sluggard. Consider her ways and be wise. 

While these behaviors are definitely simple, to my mind this is so clearly behavior that no other examples are needed. But in fact we do have more complicated examples of plant behavior. 

Take the Boquila, a flowering vine that grows on a host tree and imitates the host’s leaves, including the size, shape, and color. They can mimic over 20 species of plants, and don’t need to touch the host trees to mimic their leaves. They’ll attempt to mimic the leaves of their host even when that host is a fake plant made out of plastic. Some people have looked at these facts and concluded that this is evidence for plant vision.

Even the humble pea plant (Pisum sativum) may be able to sense the size of a stick before touching it. Peas are climbing vines — as they grow, they reach out and grab on to surrounding objects for support. This spinning search (circumnutation) is too slow to see under normal circumstances but is perfectly visible in timelapse. And the findings of that one paper, at least, suggest that plants spin faster when they are trying to grab a thin stick than when they are trying to grab a thick one. This could be more evidence of plant vision, but the authors think it might actually be evidence for plant echolocation. 

It’s easy to oversell these results. A lot of people miss the message and go straight for “are plants intelligent?”2 I think the answer, based on these behaviors at least, is obviously no. These plants are not translating Ovid; they probably couldn’t even catch a fly ball. But to the question “do plants think?”, the answer is obviously yes. Clearly some kind of computation is happening here. The plant is sensing something about the world, it makes some kind of internal comparison(s), and this affects its behavior. That’s thinking in my book. 

These examples could also be better documented. It’s possible that some of these claims may not hold up. But that’s the point: plant behavior should be better documented. And plant behavior is the domain of plant psychologists

To my mind, the question isn’t whether we decide to expand the scope of psychology to plants. The question is whether there’s any prospect at all of keeping plants out! They may not be as quick as animals, their cognition may not look all that much like ours, there may not be anything happening “upstairs”, but there is clearly some kind of computation happening somewhere. 

I don’t think this is enough to tell us what psychology’s first paradigm will be. But I do think that whatever the eventual paradigm of psychology is, it will have to include plants. So any idea for a paradigm that would exclude plants is on shaky footing. And any idea that clearly includes plant psychology is all the more promising for that inclusion.   

If you take the other side of that wager, psychology won’t include plants, that still gives you an interesting starting point. Try to imagine what paradigm could possibly exclude plants. Behaviorists and cognitive psychologists have mostly ignored plants, but that was an oversight — these paradigms don’t rule out plants, psychologists just didn’t think of it. I think you’ll find that it’s surprisingly hard to draw the lines in a way that leaves plants on the outside while keeping all the normal parts of psychology in.

In any case, a few notes about the practical implications:

It’s hard to recruit human participants for anything other than surveys, and hard to house a colony of five hundred rats, but relatively easy to grow your own research subjects when they are tomatoes. That means plant psychology is good for replication. A packet of snow peas is cheap. You can run studies on plant vision or plant echolocation in the comfort of your own back garden. 

One reason people choose to do research with animals is that you can do things to animals that you can’t do with humans, like keep them in cages for their whole lives, or cut apart their brains and see what happens. By similar logic, with plants you can run studies that you could never ethically or practically run on animals. You can do things to plants that you would never consider doing to an animal, and grow them in any situation you need, with no danger to your conscience. At least, that is, until we figure out whether or not plants can feel pain. 

Psychology of Memory

In a computer, each file is stored in a specific file format, a set of rules that govern how the data inside the file is organized. This is where we get JPEG, WAV, PNG, PDF, TXT, and GIF. All of these are different formats for storing different kinds of information. 

Most file formats include some kind of compression so they can hold the same information (or a good approximation of the same information) using less storage. This compression is sometimes characteristic: you can tell the file format by noticing signs of the compression. You can recognize a JPEG by the characteristic chunky compression, like so:

It’s a little subtle in this example, because the compression isn’t that extreme. But if you look closely you’ll see lots of little square regions in the image, some of them with strange wave patterns. These are “JPEG artifacts”, a flaw that reveals the nature of the compression technique.

You can find a description of the techniques behind JPEG here, and you can play with the algorithm using a tool by DefenderofBasic that’s hosted here. But I’m going to skip over most of the details, we’re not here to talk about JPEGs. We’re here to talk about how file formats sometimes show their fingerprints on the data. In this case, JPEG compression starts by breaking the image into 8×8 pixel squares, and if you check a JPEG, any artifacts should be in chunks of 8x8 pixels. 

In the brain, memories are stored as…??? Honestly, who knows. In my opinion, this is one of psychology’s major blindspots. 

I think the answer I’m supposed to give is “memories are stored as associations”. But this doesn’t hold up to closer scrutiny. First of all, associations between what and what? If “the Eiffel Tower” is associated with “Paris”, what does the association represent? Does that mean the Eiffel Tower is in Paris, or that Paris is in the Eiffel Tower? Association by itself is too simple to represent even the most basic information.

You can argue that we store memories in the kind of associations used in Large Language Models (LLMs). But LLMs actually make very characteristic errors, like not being able to count the number of R’s in the word “strawberry”, which comes from the fact that their inputs are tokenized. This is characteristic of LLMs just like how 8x8 square artifacts are characteristic of JPEGs, and if humans stored our memories the same way, we would make similar errors. 

If you read an intro psych textbook, you will see something like: memory consists of encoding, storage, and retrieval. But this is true of any sort of storage. Index cards are also encoded (on paper, in the English language, using approximations of Roman characters written in pen), stored (in little boxes to keep them from getting wet or getting eaten by insects, which would damage their information), and retrieved (using the labels written on the boxes).

The landmark studies in memory are mostly on a single theme, they mostly demonstrate different ways that memory is not perfect. While overturning common misconceptions is an important job, and one we’re reasonably good at, this is short of the abilities of a mature science. Some hints about memory come from studies that show that people can remember more things, or remember them better, when they use certain techniques or are in certain situations. But I’ve never encountered a study that uses this to argue for or against a particular kind of encoding. 

The very best evidence about the nature of memory comes from people with severe brain damage — most famously, HM. HM could hold on to new information for a couple of minutes, but soon forgot, and couldn’t make any new long-term memories. Cases like this make it pretty clear that there are at least two kinds of memory, which we sometimes call short-term and long-term. That’s important, but it’s a lot like saying a computer contains two kinds of file types. It doesn’t actually tell us how those file types work.

This is a tough nut to crack, but if someone did start getting close to a theory of the file format(s) of memory, there’s a good chance it would give us our first paradigm. Memory isn’t everything, but it’s pretty central. We make decisions in the light of our previous experiences. We perceive things in the context of things we’ve seen before. Everything is shaped by memory; maybe that’s where we should hang our hat. 

We might be able to get there by looking at plants. Plants seem to form memories, but they don't have brains or even neurons, so how are they storing that information? Because it’s easier to do invasive research on living plants than on living animals, we might be able to look inside living plants and actually see the memories being formed.

But there are other angles too. One of the most surprising passages in the book Cognitive Psychology is one where Neisser shares a theory of memory that has certainly fallen out of the mainstream, but for all I know, may still be plausible. He says:

No one would dispute that human beings store a great deal of information about their past experiences, and it seems obvious that this information must be physically embodied somewhere in the brain. Recent discoveries in biochemistry have opened up a promising possibility. Some experimental findings have hinted that the complex molecules of DNA and RNA, known to be involved in the transmission of inherited traits, may be the substrate of memory as well. Although the supporting evidence so far is shaky, this hypothesis has already gained many adherents.

Psychology of Value

Behaviorism and cognitive psychology are more similar than they would like to admit. While the cognitive revolution stressed their differences, they share a lot of commitments.

For one, the two schools share a similar concept of value. Both schools think that value is an important part of psychology, that value itself runs the gamut from reward to punishment, and that behavior is driven by some kind of function that aims to increase reward and decrease punishment. Sometimes they call this the “hedonic principle”. 

Because it’s so central, and has gone unquestioned for so long, I think a new paradigm might come out of a new way of thinking about value. A similar possibility would be a new paradigm that gives value a different kind of role in psychology, where it’s less central, or performs a different function. 

I’ll give just one example of why I think this could be the right direction. There’s a 2015 paper from a team of psychologists led by Tim Wilson which found that if you ask people to sit in a room for 15 minutes, with a button they can press to receive a painful electric shock (they had already experienced the shock in an earlier stage of the study, so they know it’s painful), 25% of women and 67% of men will give themselves at least one shock before the 15 minutes is up. One woman shocked herself nine times, but the real outlier is the man who gave himself 190 shocks, slightly more than one shock every five seconds. 

The authors kind of go back and forth on how to interpret this. Usually they stick to the standard cognitive psychology model of value: a shock is always a punishment, so if people chose to shock themselves, they must have been doing it to avoid some greater punishment. In the abstract they write that many people “[prefer] to administer electric shocks to themselves instead of being left alone with their thoughts”. 

They also interpret the main finding as “what is striking is that simply being alone with their own thoughts for 15 min was apparently so aversive that it drove many participants to self-administer an electric shock that they had earlier said they would pay to avoid”, and they go to some lengths to show that people don’t enjoy being alone with their thoughts. 

But other times they admit that being alone with your thoughts (BAWYT) doesn’t seem like it could be all that bad. At one point they write, “many participants elected to receive negative stimulation over no stimulation”, suggesting that the value of BAWYT is zero rather than negative. And their attempts to show that BAWYT is agonizing fall pretty flat. They say that “participants did not enjoy the experience very much”, but across six studies, the average rating for BAWYT was 5.12 on a 9-point scale. This is the definition of middling, and doesn’t suggest it was all that negative. 

Why did that one guy shock himself so many times? You can cling to the standard way of thinking about value, and say that sitting quietly must have been worse than the shocks. Or you can see this as evidence that the old way of thinking about things is wrong, and that we need to find a new model for value, a model where shocking yourself 190 times in a row makes perfect sense. Because to at least one participant, it does.

Psychology Splits

I said a bit about this before, but I want to say it again at the close: One of the biggest assumptions is that when this all shakes out, there will be just one field that is still called “psychology”.

But it may not go that way. In the next revolution, psychology may split up. Psychology might even split three or four ways. We shouldn’t try to squeeze everything in one jacket if the jacket doesn’t fit.

The study of the human mind is already divided between different fields. Many parts are given to economics, sociology, neuroscience, medicine, and so on. Maybe these divisions will just get more and more useful. Psychology might even go the way of Poland in 1795 and cease to exist as a sovereign field, its territory divided up between its more powerful neighbors.

Personally, I think this is unlikely. Many fields would like to conquer psychology; but none have succeeded. I think there’s a unique field or two in here somewhere.

But that’s the thing, there might be more than one. There are many divisions within psychology, like decision-making, sensation, and emotion. Maybe these sub-areas will prove the most productive, and will split the field from within. Maybe memory and perception end up together in one new field, while personality and motivation end up in another.

Psychology might also divide in more complicated ways, maybe based on some of the ideas above. For example, memory might form a new field with neuroscience (since the storage of memories might be closely connected to how neurons work), that is neither traditional psychology nor traditional neuroscience, but some kind of new field based on the study of particular methods of storing information. Meanwhile, motivation might form a new field with medicine, as the functions of the brain and other organs are found to go hand in hand with the reasons why we find it easy to drink sugar water, and hard to pretend to pay attention during the quarterly staff meeting.

The only thing we should expect for sure is this: when the revolution comes, it will be surprising. And, you know what? I think it will also be delightful. 

I.

If you’re looking for the whipping boy for all of medicine, and most of science, look no further than Galen of Pergamon.

As early as 1605, in The Advancement of Learning, Francis Bacon is taking aim at Galen for the “specious causes” that keep us from further advancement in science. He attacks Plato and Aristotle first, of course, but it’s pretty interesting to see that Galen is the #3 man on his list after these two heavy-hitters.

Centuries went by, but not much changed. Charles Richet, winner of the 1913 Nobel Prize in Physiology or Medicine, said that Galen and “all the physicians who followed [him] during sixteen centuries, describe humours which they had never seen, and which no one will ever see, for they do not exist.” Some of the ‘humors’ exist, he says, like blood and bile. But of the “extraordinary phlegm or pituitary accretion” he says, “where is it? Who will ever see it? Who has ever seen it? What can we say of this fanciful classification of humours into four groups, of which two are absolutely imaginary?”

And so on until the present day. In Scott Alexander’s review of Superforecasting, he quotes Tetlock’s comment on Galen:

Consider Galen, the second-century physician to Rome’s emperors…Galen was untroubled by doubt. Each outcome confirmed he was right, no matter how equivocal the evidence might look to someone less wise than the master. “All who drink of this treatment recover in a short time, except those whom it does not help, who all die,” he wrote. “It is obvious, therefore, that it fails only in incurable cases.”

Scott then says,

After hearing one too many “everyone thought Columbus would fall off the edge of the flat world” -style stories, I tend to be skeptical of “people in the past were hilariously stupid” anecdotes. I don’t know anything about Galen, but I wonder if this was really the whole story.

This strikes me the same way. The more I read about history, the more I realize that it was made up of people just like us. They didn’t always have the same tools we do, and they made some weird mistakes, but they weren’t any less intelligent than we are. Some of them were pretty smart.

I’m also concerned that this criticism doesn’t pass the sniff test. If Galen was really a moron, why did he have such a lasting impact? Most philosophers, including great ones, aren’t even known in their own time. There must have been something about Galen’s works that kept them around for thousands of years.

All this makes me very suspicious. Was Galen actually a dope? Or did someone pull an intellectual hit job on the guy? If so, why? What was he actually like, and why did he have such a huge influence?

When you press a seashell into the sand, it leaves an impression that is its own shape. For some reason culture isn’t like this. Great works and people seem to leave impressions in the culture that are different from their own shape. People who read Frankenstein are often surprised to learn that “Doctor” Frankenstein was about 19 (Victor made the creature during the summer after his second year at university; he was an undergrad), and that his “monster” can not only speak, but is articulate and sensitive. Whatever Galen was actually like, the modern stereotype of him is probably inaccurate.

So for these reasons and others, I decided to review one of Galen’s books.

II.

First, a little biographical context.

Aelius Galenus or Claudius Galenus (henceforth “Galen”) was born in Pergamon, a town in modern-day Turkey, in 129 CE. At the time, it was a part of the Roman empire, and a major intellectual center. Galen’s father was an architect; while rich, he was not considered to be particularly high status. Since there was little pressure for his son to go into a traditional career, instead of the “safe” subjects of literature and rhetoric that most Romans studied, Galen got an unusual education in mathematics and geometry.

(His father’s patient encouragement has its foil in his mother, who “flew into rages and bit her servants, a practice of which Galen disapproved.”)

When Galen was a teenager, however, his father had a dream where the god of medicine appeared and told him that his son should study medicine, so Galen started training as a doctor.

During this training Galen became familiar with the writings of Hippocrates, who had lived about 600 years earlier. Hippocrates had introduced the idea of the four humors to medicine — four fluids that congeal together to form our flesh and organs, and which co-mingle in our veins in their liquid form. Hippocrates came up with this system, but Galen would be the one to make it world-famous.

I could try to describe the theory myself, but actually Hippocrates does a great job on his own:

The Human body contains blood, phlegm, yellow bile and black bile. These are the things that make up its constitution and cause its pains and health. Health is primarily that state in which these constituent substances are in the correct proportion to each other, both in strength and quantity, and are well mixed. Pain occurs when one of the substances presents either a deficiency or an excess, or is separated in the body and not mixed with others.

All disease and illness, in this system, were the result of an imbalance in the four humors. From this perspective, treatments like bloodletting make perfect sense. By opening up the veins, the excessive humors drain away, leaving the patient more balanced — in better humors.

Long-term trends towards any of the humors were responsible for what we would call personality. Hence the terms sanguine, phlegmatic, melancholy and so on for different personal traits and emotional conditions.

This is the theory that he would put his weight behind, and which he would eventually be responsible for bringing to the majority of the western world.

When Galen was 19, his father died, leaving him independently wealthy. Hippocrates wrote that a good doctor should travel, so Galen ended up spending a decade studying with medical experts from various schools in cities all around the Mediterranean, including Alexandria.

After this, he came back to Pergamon where he got a job as the doctor treating the gladiators of the city. This was an unusual step for someone of his wealth and education, because despite their popularity as a form of entertainment, gladiators at the time were considered extremely low-class.

It’s not clear why he took this job, but it seems likely that it influenced how he thought about medicine. Spending long hours stitching gladiators back together gave him a detailed knowledge of human anatomy, which other doctors of the time lacked. It sounds like he did a great job, too, because only five of the gladiators died during his time there — compared to 60 under the guy who had the job before.

Eventually all roads lead to Rome, of course, and Galen arrived in 162 CE. His lectures and demonstrations made such an impression, and ruffled so many feathers, that he was afraid of getting poisoned by the Roman doctors and eventually left to save his life. In 169 CE, however, a great plague (probably smallpox) broke out, and Marcus Aurelius summoned him back to Rome to serve as court physician. Marcus Aurelius died the next year (according to some sources, of the plague), but Galen ended up with a longterm post in Rome as physician to the new Emperor, Commodus.

Galen himself died some time between 199 and 216 CE, at the the ripe old age of between 70 and 87.

It’s hard not to notice just how famous Galen was in his own time. Marcus Aurelius described him as “primum sane medicorum esse, philosophorum autem solum” — first among doctors and unique among philosophers (one wonders if Galen might have influenced the Emperor’s own philosophy). Forgeries and unscrupulous editions of his work were such a problem during his lifetime, he had to write a book called On My Own Books to try to sort it all out. Among other things, he complains that his servants were stealing private letters he had written to friends and circulating bootleg copies as medical advice.

Galen was an incredibly prolific writer. Wikipedia claims that he produced more works than any other author in antiquity, maybe up to 600 treatises, and possibly employed 20 scribes at one point. While these particular claims are hard to substantiate, he did leave behind a whole lot of books.

Fires and the various other mishaps that are guaranteed to happen to classical texts destroyed many of his works. Some of this even happened during his own lifetime, and in On My Own Books he seems surprisingly relaxed about so many of his works being lost:

The books of many others perished at that time, as did all those of mine which were located in that storehouse; and none of my friends in Rome admitted to having copies of the first two books. Since, then, my followers prevailed upon me to write the same treatise again, I thought that I should give this explanation regarding the previously distributed books, in case anyone in the future finds them and wonders why I should have written a treatise twice on the same subject.

Even with these losses, huge amounts of his work has survived. It’s hard to get an exact count, but Claudii Galeni Opera Omnia by Karl Gottlob Kühn, compiled around 1833 and for a long time the definitive edition, contains 122 different works in 22 volumes. That’s a lot.

Despite this, I was surprised how hard it was to get my hands on primary source copies of his works (in English). Because of our own plague, I was limited to finding sources online — but for most classical works, this is pretty easy. Marcus Aurelius was a contemporary of Galen, and it’s not too hard to find multiple different translations of Meditations (though admittedly Marcus may have a slightly wider appeal).

Part of this might be that Galen’s works are very badly organized. Every secondary source I read on the Galenic corpus is full of griping about how confusing the whole thing is. Galen wrote in Greek, but many of the original versions of his books are lost, leaving us only with Arabic or Latin translations, or Latin translations of earlier Arabic translations. Some of the books appear under different titles in different places, and sometimes the works are only indexed under abbreviations of those titles. Some of them probably were never intended for publication (those bootleg letters I mentioned above), and so may not have official titles or versions at all. Forgeries of his works in various languages continued well on into the Renaissance.

Galen himself was very unclear on how to think about the documents he produced. At one point in On My Own Books, he starts off by talking about a piece of writing he did “as an exercise for myself”, and then immediately turns around and mentions that he gave it to friends, who in turn gave it to their friends. Needless to say, the whole thing is a mess, the scholars seem very agitated.

I chose to review the longest piece I could find, which is On the Natural Faculties, specifically the translation by Arthur John Brock, which was the only translation I was able to track down. This also seemed like a good choice because, instead of being a treatise on a more limited topic like diet, the pulse, or bones, this book serves as more of an introductory textbook to what today we would call biology.

Subscribe now

III.

On the Natural Faculties is divided into three books, though if the three books have any structure to them, I wasn’t able to figure it out.

Galen is pretty straightforward in naming his pieces, and this book is about him trying to describe all of the “natural faculties”. This doesn’t really correspond to any modern concept, but essentially he means the fundamental or basic biological functions common to all living things. He begins by contrasting the functions of the soul, like feeling and voluntary motion (we might say “mental functions”), which occur only in animals, with the natural functions common to both animals and plants. You could maybe translate “natural faculties” as something like “basic biological functions”.

I had always heard that Galen was a Hippocrates stan, but right from the get-go he’s mentioning Aristotle in the very same breath (though he reminds us that Hippocrates “lived much earlier than Aristotle”). When describing the natural faculties, he seems to base them off of Aristotle’s physics more than Hippocrates’ humors.

Aristotle’s physics is a system I mostly know secondhand from the descriptions offered by Thomas Kuhn (for an example, take a look at this piece). Kuhn stresses that this system is hard for a modern mind to understand and even harder to explain, so I was surprised at how intelligible Galen’s account is. Maybe reading Kuhn’s description prepared me to understand what Galen has to say, but either way, it’s great. I think Galen does a better job than Kuhn. Basically he says, look, there are different kinds of motion:

If that which is white becomes black, or what is black becomes white, it undergoes motion in respect to colour; or if what was previously sweet now becomes bitter, or, conversely, from being bitter now becomes sweet, it will be said to undergo motion in respect to flavour … when a warm thing becomes cold, and a cold warm, here too we speak of motion; similarly also when anything moist becomes dry, or dry moist.

He goes on to suggest that the natural faculties are more advanced forms of motion, possibly built up out of the combination of simpler forms of motion.

(Kuhn treats the Aristotelian perspective as if it was the common sense of the ancient world, but the fact that Galen has to describe it in such detail makes me wonder if that was really the case.)

That’s the framework. What the exact set of natural faculties are, however, is less clear.

In Book One he focuses on three faculties in particular — genesis, growth, and nutrition — and provides lots of arguments that (for example) the body’s ability to grow is different from its ability to sustain itself. In Book Three he gives a different list of four — the attractive, retentive, expulsive, and alterative faculties — but he also suggests that these are “handmaids of Nutrition”.

Elsewhere he says that genesis is not “a simple activity of Nature” but instead is “compounded of alteration and of shaping.” He also mentions faculties like “adhesion” and “presentation”. The particulars are pretty confusing, but the general gist is clear. Galen wants to lay out all the different faculties and their sub-faculties (and sub-sub-faculties?) so that the reader can understand the workings of the body.

Galen makes it pretty plain that he thinks that diseases are caused by failures or overactivity of the different principles. For example, he says that in leprosy “there is adhesion of the nutriment but no real assimilation”. One faculty is working but the other is disordered. If you want to be a good physician, he says, you need to understand all these faculties so you can identify diseases (tell what faculties are misfunctioning) and treat them — “how are you going to be successful in treatment, if you do not understand the real essence of each disease?” he says.

The four humors do make their way into this mix eventually, especially in the second and third books. (Though the translator often insists on translating “humor” as “juice”, which makes me very uncomfortable.) The relationship seems to be that the humors are the building material of the body, but that all the activity is carried out through the natural faculties. The student needs to know the humors to understand what is being moved around, but the humors are primitive. To Galen, biology is all about these faculties shuffling, transforming, and combining different humors.

VI.

Anyways, that’s what Galen wants to be talking about. But about halfway through Book One, he goes entirely off the rails and never really gets back on track.

The thing that sets him off is other schools of medicine. It’s clear that Galen cannot stop thinking about them. They invade his every thought; he is beleaguered by them. I would seriously believe that he loses sleep over them.

Some of the commentators I’ve read suggested that Galen was an arrogant man — one said he saw in Galen “the blind assumption that he alone was graced with the ability to bring Hippocrates’ work to completion”. My sense of Galen was that he is a man who is constantly exasperated. He is just trying to write basic pieces about how to be a good physician and philosopher, and people keep descending on him with the most unbelievably pedantic arguments. Book One of On the Natural Faculties is divided into 17 sections, and he spends half of the first section hedging around ways people could potentially take his words in the wrong ways.

These sound more than a little like intrusive thoughts, and it’s tempting to think that he’s blowing this all out of proportion. But from what I know about Galen’s life, it seems likely that he really was getting into disagreements all the time, and probably really did need to worry about people quoting his work out of context. One article in The Lancet describes him as “a public figure, known and recognised by many, accosted in the streets, challenged to debate.” It’s easy to imagine how being accosted in the streets might work its way into your head.

Either way, these concerns absolutely consume him. He keeps getting drawn off on different tangents, before trying to return to the main thread with statements like:

I said, however, that I was not going to enter into an argument with these people, and it was only because the example was drawn from the subject-matter of medicine, and because I need it for the present treatise, that I have mentioned it.

Let us pass on, then, again to another piece of nonsense; for the sophists do not allow one to engage in enquiries that are of any worth, albeit there are many such; they compel one to spend one’s time in dissipating the fallacious arguments which they bring forward. What, then, is this piece of nonsense?

Now, we usually refrain from arguing with people whose principles are wrong from the outset. Still, having been compelled by the natural course of events to enter into some kind of a discussion with them, we must add this further to what was said…

Since, then, we have talked sufficient nonsense — not willingly, but because we were forced, as the proverb says, “to behave madly among madmen” — let us return again to the subject of urinary secretion.

But, as I have said, one is driven to talk nonsense whenever one gets into discussion with such men. Having, therefore, given a concise and summary statement of the matter, I wish to be done with it.

Of course, in the very next paragraph, he is immediately drawn back into a discussion of their shortcomings!

In some ways, On the Natural Faculties is less of a medical treatise and more of a fascinating snapshot of the state of the academic medical world in the latter half of the second century CE. The tone sounds really contemporary in a lot of ways, and has a quality of acrimonious quibbling that is more than a little familiar, though I don’t think modern physicians are likely to be poisoned by their colleagues (but what do I know).

Subscribe now

V.

We’ve established that Galen has a problem with other experts and schools of medical thought. That leaves us wondering how justified he is. Is he criticizing them for real problems in their work, or is this just partisan squabbling?

What are the things that he takes such issue with from these other schools? I think there are two things he’s mostly complaining about.

The first thing that really sets Galen off is sectarian dogmatism. “Everyone becomes like the first teacher that he comes across,” he says, “without waiting to learn anything from anybody else.” He bemoans sectarian partisanship and, in classic doctor fashion, uses a weird hygiene metaphor, calling it “excessively resistant to all cleansing process”. It is “harder to heal than any itch”.

The fact is that those who are enslaved to their sects are not merely devoid of all sound knowledge, but they will not even stop to learn!

This is kind of tragicomic, because two of the main things Galen is accused of are 1) blindly following whatever Hippocrates said about medicine and 2) leading centuries of physicians to blindly follow whatever he wrote!

It’s hard to know how blindly Galen is following the teachings of Hippocrates. On the one hand, he does refer to him as “most divine Hippocrates” at least once. On the other hand, he is open to pointing out the (rare) cases where he thinks Hippocrates has overlooked something, and even talks about how he wishes his opponents would criticize Hippocrates more directly! When someone disagrees with a whole suite of his intellectual heroes, he says, “now, one cannot be blamed for not agreeing with all these great men, nor for imagining that one knows more than they; but not to consider such distinguished teaching worthy either of contradiction or even mention shows an extraordinary arrogance.”

Maybe other physicians really did follow Galen’s writing blindly in the centuries following his death. I’m not sure anymore. But Galen certainly can’t be blamed for it. He could not be clearer in stating that this is exactly what the student of medicine should avoid doing.

It would be tempting to pass this all off as one-sided; “stop listening blindly to your teachers and listen blindly to me!” I don’t get that sense.

First, we know that Galen studied all over the ancient world, so he was exposed to all sorts of ways of doing medicine. He practiced what he preached. It’s hard to know how fair a representation he’s giving of the other schools of thought, but he writes as though he has them all memorized, and he certainly was in a position to frequently get into debates with them. When he tells us that they’re uncritical, I’m tempted to believe him.

Second, Galen makes a serious point to try to convince the reader of his positions. He’s not just stating “facts” and expecting you to bow down at his feet. He’s engaging with opposing points of view and trying to make compelling arguments that he thinks will convince his readers.

VI.

Finally, I don’t buy this because nowhere is Galen asking people to listen blindly to anyone, least of all himself. Because the second thing that REALLY sets Galen off is when people aren’t empirical enough! He constantly ridicules, in pretty harsh language, those who remain unconvinced by observation and experiment.

Asclepiades, one particularly hated adversary, is charged with “bidding us distrust our senses where obvious facts plainly overturn his hypotheses.” Asclepiades has rather unusual opinions about the urinary system, and in one particularly strong example, Galen asks rhetorically (and sarcastically!),

I do not suppose that Asclepiades ever saw a stone which had been passed by one of these sufferers, or observed that this was preceded by a sharp pain in the region between the kidneys and the bladder as the stone traversed the ureter, or that, when the stone was passed, both the pain and the retention at once ceased. It is worth while, then, learning how his theory account for the presence of urine in the bladder, and one is forced to marvel at the ingenuity of a man who puts aside these broad, clearly visible routes, and postulates others which are narrow, invisible—indeed, entirely imperceptible.

Other schools are also attacked for denying “observed facts” or even “obvious facts”. Meanwhile, people who draw incorrect conclusions but respect the facts are praised.

Galen cares a lot about physicians basing decisions on empirical observation. We know that he’s serious about this because of the many disturbing vivisection experiments he describes in great detail.

In discussing digestion, he says, “I have personally, on countless occasions, divided the peritoneum of a still living animal and have always found all the intestines contracting peristaltically upon their contents.” He describes an experiment where you vivisect an animal, cutting away different coats of the esophagus, “then give the animal food and you will see that it still swallows although the peristaltic function has been abolished”. When describing the action of the stomach, he suggests that you can fill an animal with liquid food — “an experiment I have often carried out in pigs” — and cut them open “after three or four hours.”

He really seems to want his readers to try these macabre exercises at home. “You may observe this yourself,” he says, “if you will try to hit upon the time at which the descent of food from the stomach takes place.” Fellow physicians are criticized for their lack of anatomical experience in the same way. “If he had ever practised anatomy, he might have known that the outer coat of the bladder springs from the peritoneum and is essentially the same as it.”

The most extreme example comes from a debate with the disciples of Asclepiades about the function of the ureters, trying to convince this rival school that urine flows from the kidneys to the bladder through these channels. After exhausting his rhetorical options, Galen turns to empirical anatomy. First he shows them, in a dead animal, that the ureters connect the two structures. This isn’t enough. Next he shows them “in a still living animal, the urine plainly running out through the ureters into the bladder.” This doesn’t change their minds either.

Next he takes a live animal, ligates the ureters, bandages the animal up, and lets it go. When he opens it up again later, he finds the ureters “quite full and distended”, and when he removes the ligature, everyone can see the urine flow into the bladder.

You’d think the story would end there, but not so. Instead, says Galen, “tie a ligature round [the animal’s] penis and then … squeeze the bladder all over.” He points out that nothing goes back through the ureters to the kidneys, demonstrating that the conveyance is a special, one-way action. He goes on like this for a while. Let the animal urinate and tie a ligature around one ureter but not the other. Cut open both the ureters and see the urine “spurt out of it”. Bandage the animal up and open him up later to discover his insides full of urine and the bladder empty. “Now, if anyone will but test this for himself on an animal,” Galen concludes, “I think he will strongly condemn the rashness of Asclepiades.”

Today we know that Galen was wrong, and that humorism isn’t a great way to think about medicine. But whatever Galen might have been lacking, it certainly was not the empirical bent. He was no armchair philosopher, and was more than happy to cut up lots of animals to make a point about the function of the ureters.

This is funny because, again, this is the opposite of the story we’re told about Galen. He’s described as a pre-scientific or even unscientific thinker, who believed that experimentation and investigation are a waste of time. Clearly this isn’t the case, and he made full use of all the resources available to him. We know that human dissection was prohibited in the empire, but Galen worked with gladiators, so we know that he had firsthand experience with human anatomy. He certainly was unafraid, even eager, to practice animal dissection and vivisection. Other doctors of the time didn’t seem to do either of these things, or at least didn’t do nearly as much, and so Galen starts looking more and more like a lone light of empiricism in the wilderness.

(However extreme and disturbing his methods may be.)

VII.

In view of this, it’s extremely depressing to see Tetlock write, “yet Galen never conducted anything resembling a modern experiment.” Galen isn’t here to respond, but if he were, I imagine he would say: and yet Tetlock never conducted anything resembling a basic literature review!

Galen definitely isn’t as charitable as we might want him to be. He calls some of the ideas he disagrees with “impossible, nay, perfectly nonsensical”, or “stupid—I might say insane”. His intellectual rivals “are like slaves” he says, “caught in the act of stealing … quite bewildered, and while the one says nothing, the other indulges in shameless lying.”

But I’m pretty sympathetic to Galen’s position, because his contemporaries really do sound like idiots. Of course, all this is being filtered through Galen’s own account, but if he’s describing them with any accuracy, he is totally fair in saying that they have no idea what they are talking about.

Some of the positions he argues against include:

• Urine passes into the bladder in the form of vapors, rather than being secreted by the kidneys and passed through the ureters to the bladder. Galen argues against this first by pointing out that the kidneys and bladder are connected by the ureters (which must have some purpose), and second by the extensive evidence from vivisection that I mentioned above.

• Fetuses are constructed piecemeal in the uterus, instead of beginning small and growing to the size they are at birth. Galen argues against this by describing an observation mentioned by Hippocrates, where he saw a very early miscarriage in which he noticed that the heart of the fetus was “so small as to differ in no respect from a millet-seed, or, if you will, a bean.”

• Swallowing is powered by the “impulse from above” (I think he means gravity), rather than the action of the muscles. Galen argues against this first by pointing out that the “oblique situation of the gullet” means that gravity wouldn’t be sufficient, and second by describing vivisections he had conducted that showed swallowing to be caused by the action of different parts of esophagus. He also mentions something about pouring water down a dead man’s throat, but I didn’t understand that part of the argument.

Galen is certainly right about some things and wrong about other things. He mentions that the veins and arteries are connected to each other, something that had apparently escaped everyone else’s notice. But he also thinks that air is taken in through the capillaries near the skin. He has some unusual ideas about pregnancy, and reports that corn will spontaneously draw water out of nearby earthen jars and increase in weight, which I’m pretty sure is not true. But compared to his contemporaries, he seems to be very much on the right track.

In fact, I was surprised by how modern Galen’s reasoning seems. At one point he discusses magnets, and argues against an atomic explanation of their action. Before you laugh, remember that the modern explanation is electromagnetic fields, which look a lot more like one of Galen’s natural faculties (a tendency for mutual attraction over distance) than they do Epicurus’ view that “the atoms which flow from the stone are related in shape to those flowing from the iron, and so they become easily interlocked with one another”.

In making this argument, Galen first points out that the idea that nature has “powers which attract” is sufficient to explain the observations. Next he says that since the proposed entanglement can’t be observed, it’s not clear why anyone would prefer this explanation to another. Even if we do allow it, he says, it doesn’t explain why a piece of iron that has previously touched a lodestone will then go on to attract other pieces of iron. The atomic explanation predicts that this shouldn’t happen!

Then he brings out cases from his own experience, saying, “I have seen five writing-stylets of iron attached to one another in a line, only the first one being in contact with the lodestone, and the power being transmitted through it to the others.” Then he uses a thought experiment! “Now, if you imagine a small lodestone hanging in a house, and in contact with it all round a large number of pieces of iron, from them again others, and so on…” He continues on in this vein for a number of paragraphs.

This strikes me as a very modern way to make an argument. It’s how I would make an argument today — none of the intellectual arsenal seems to be missing! In fact, Galen uses thought experiments quite frequently. There’s one in particular he loves, about children making balloons out of the bladders of pigs, which he uses a couple times to demonstrate that only Nature has the ability to actually make things grow, rather than just distend them.

Nature (always capitalized) is a concept Galen comes back to time and time again, which he uses as an underlying principle for his arguments. Usually he talks about how Nature exercises forethought for the animal’s well-being and survival, or talks about how Nature is “artistic”, making everything for a purpose. A constant criticism he makes of other schools is against their claims that various ichors, tissues, or organs exist without a purpose. Nature doesn’t make things without a purpose! he shouts.

“Nature”, described in this way, ends up sounding a lot like evolution. Of course, Galen had no concept of evolution, but it’s very interesting to see that he was using a functionally equivalent concept hundreds of years before Darwin. It’s not clear if this is a concept he developed himself or if he inherited it from some earlier thinker, but clearly someone, in looking at biology, noticed that the principle “everything exists for a purpose” was a pretty good starting place for making arguments that turned out to be true.

Nature takes on the exact same role in his arguments that evolution would take on in ours:

“Your school of medicine says that the ureters don’t do anything? Then why are they there? We know that this animal was created by Nature, why would she include these structures if they were truly useless?”

“Your school of medicine says that the ureters don’t do anything? Then why are they there? We know that this animal evolved through natural selection, wouldn’t selective pressure remove these structures if they were truly useless?”

VIII.

Galen’s idea of humorism is more detailed and nuanced than the picture of humorism I’ve gotten from secondhand accounts. For one thing, he’s not limited to just four humors. He refers to different types of blood and suggests that there is a “sweet” phlegm and a “bitter or salty” phlegm, the former useful to health and the latter dangerous. At one point he mentions that Praxagoras describes a theory of eleven humors, and even says, “this is not a departure from the teaching of Hippocrates; for Praxagoras divides into species and varieties the humours which Hippocrates first mentioned”.

As much as modern authors like to describe the humors as imaginary, Galen routinely talks about them as if he were actually collecting and measuring them! In a section where he talks about the properties of different drugs, he describes at length how they will evacuate more or less of the various humors in different seasons or from people of different ages.

One might be tempted to dismiss this as metaphor or inference — “probably Galen means that you can see the evidence of evacuation” — but how are we to take it, then, when he describes the humors in terms of the volume produced? “Who does not know,” he says, “that if a drug for attracting phlegm be given in a case of jaundice it will not even evacuate four cyathi [about 4 oz.] of phlegm?” I’m not sure what is being evacuated here, or how he collected it, but he certainly seems to be measuring something.

Galen makes even more sense when talking about other drugs. He has a whole digression on antivenoms, where he concludes by saying that after the treatment has been applied, “we can, in fact, plainly observe these poisons deposited on the medicaments.” He describes a case where manual traction was insufficient to remove a thorn from a young man’s foot, but some kind of “medicament” removed it easily. He mentions that some people claim that the thorn only falls out in this case because the medicament reduces inflammation. Not so, says Galen, who points out that while there are anti-inflammatory drugs as well, those don’t do anything to draw out embedded bodies.

He doesn’t offer enough detail and I don’t know enough medicine to discover how many of these claims are correct, but it’s certainly more nuanced reasoning than he is usually given credit for.

Remember that quote that Tetlock used to dunk on Galen? “All who drink of this treatment recover in a short time, except those whom it does not help, who all die. It is obvious, therefore, that it fails only in incurable cases.”

I haven’t been able to confirm its source. Everything attributes it to Galen, but they all point to earlier secondary sources — people who were not Galen but who were supposedly quoting him. Tetlock doesn’t cite one of Galen’s many works — he cites Druin Burch’s 2010 book, Taking the Medicine: A Short History of Medicine’s Beautiful Idea, and Our Difficulty Swallowing It. Now, Taking the Medicine does include that quotation, but as far as I can tell, it doesn’t cite a source at all.

The other secondary sources I found seem to lead back to the 1998 book, Where’s the Evidence? Debates in Modern Medicine, by William A. Silverman, but he doesn’t say which of Galen’s works this quote supposedly comes from. In fact I can’t find any sources from before 1998 that include any fragment of this quote at all. It’s not looking good for the authenticity of this quote.

(I was going to suggest that maybe this quote might somehow be a mistaken bastardization of the “dormitive potency” line from Molière’s The Hypochondriac, since that’s what it reminds me of. But I looked and I actually wasn’t able to find that line in the play, so… [Update: That line is from the Third Interlude at the end of the play.])

As with many other things, it’s ironic that people haven’t actually read Galen, because this is yet another thing that Galen complains about in this book. Back in his day, the guy no one read but everyone pretended to was Aristotle. “The fact is, these people seem to me to have read none of Aristotle’s writings, but to have heard from others how great an authority he was on ‘Nature’…” I guess little has changed in the past 1,800 years.

You know, reading about Galen reminded me of someone… a physician… trained all around the world… a prolific writer… constantly drawn into disputes about philosophy and empirical practice… who am I thinking of…

IX.

Ok, so it looks like there was an intellectual hit job on Galen. Why?

I see two possibilities. The first is that Galen’s followers may have really been as bad as people say. To take one example: Jacobus Sylvius, in the 16th century, was a huge supporter of Galen. When his student Vesalius called into question Galen’s knowledge of anatomy, by performing his own dissections, Sylvius rushed to Galen’s defense. Despite being the first professor to teach anatomy of the human corpse in France, Sylvius said that Galen had not erred, but “probably the human body had changed since then.” Further, he said that advance beyond Galen’s understanding was impossible.

(I’ll note, though, that the oldest source I can find for these claims is a Catholic book from 1913, so if there’s some reason to suspect that the Catholic Church wanted to put out an intellectual hit on Jacobus Sylvius, then maybe disregard all this.)

If Galen’s other supporters were anything like this — and my sense is that they were — then it’s no surprise that there was a reaction. Galen’s teachings had ossified, and when the old guard was swept out, it damaged Galen’s reputation just as much as it destroyed his school of thought. All this despite the fact that Galen probably would have endorsed the work of Vesalius, who was actually performing dissections, over the work of those who merely studied his texts!

Another is that Galen may have been a casualty of the fight between a corpuscular and an essentialist worldview. You’ll remember that Galen’s thought was as much based in the work of Aristotle as it was based on the work of Hippocrates. All things can be explained through different kinds of “motion”, or faculties. In this paradigm, other kinds of explanations are unnecessary.

Earlier I mentioned a passage where Galen makes an argument about how magnets work, and how he takes issue with the atomic explanations offered for this phenomenon. It’s not just magnets; Galen is strongly opposed to this entire worldview. In a particularly comic moment, he says:

According to Asclepiades, however, nothing is naturally in sympathy with anything else, all substance being divided and broken up into inharmonious elements and absurd “molecules.”

Now, you may remember from Scott’s review of The Structure of Scientific Revolutions that part of Descartes’ claim to fame is inventing the idea of a purely materialist world, where matter interacts with other matter directly, with no recourse to hidden “forces” or “essences” — the classic billiard balls or clockwork universe. This is part of why Newton’s theory of gravity caused such an uproar. Saying that it is in the nature of all matter to attract other matter sounded uncomfortably essentialist! Only Newton’s constant defense and the inherent strength of the theory eventually managed to win it acceptance.

It’s easy to imagine that Galen was a target of the Cartesian revolution in the same way that Aristotle was. As far as I know, Aristotle never explicitly argued against atomism, just in favor of essentialism. While Galen was prolific, his work still wasn’t as broad or as foundational as Aristotle’s. These factors might have left Galen more open to criticism. While grudging respect for Aristotle continued, it might have been enough to discredit Galen in the mind of most scholars.

It’s hard to make the timeline line up for this version of the story, however. Vesalius was arguing against Galen’s anatomical texts in the 1530’s and 1540’s, and Francis Bacon criticized him so roundly in 1605. Yet bloodletting was endorsed by major medical organizations as late as the 1830s, and during these decades France and England were still importing (!) millions of leeches a year. Possibly the criticism of Galen has always been stronger in scientific than in medical circles. In fact, my experience with modern pieces written about Galen is that those from a scientific perspective are full of criticism, while those of a more strictly medical background tend to mention his discoveries and the treatments he pioneered.

Whatever the cause of this hit job was, it seems to have been unjustified. Galen doesn’t seem like a very nice or pleasant guy to be around, but he was an empiricist if ever there was one, and he was trying to understand biology with absolutely every tool available to him at the time, so he could provide the best treatment possible. He was wrong about humorism, but after reading this book, it’s more understandable how at the time, humorism might have seemed like the best theory around. If more of us were like Galen (minus all the vivisection), we would be much better off.

British lads hit each other with chair

At a mere 1 min 7 sec, this is one of the best narrative videos of all time. People have shot million-dollar movies with weaker plots than this clip. In fact, most films ever made have worse plots than this clip. Holly…

Read more

Here’s a story: at some point during the last years of his presidency, Barack Obama loses his cell phone at a basketball game.

The phone is lost forever, but about 100 years later, some of the data resurfaces on the ‘net. Most of it appears to be fragments from Barack’s notes app — paragraphs that were supposed to end up in his memoir, quotes from books he’s reading, notes to himself, reminders, even song lyrics and lines from movies.

Over the coming centuries, people bundle these scraps together and start treating the collection like it’s a real book, which becomes widely admired. By the year 3868, this document is still being passed around. People attribute the quote “I'm the first person to be alone on an entire planet” to Obama’s great wisdom, not knowing that it is in fact a line from the 2015 movie The Martian, which by the 39th century has been lost to the ages.

This is what it’s like to read Meditations by Marcus Aurelius.

II. 

Marcus Aurelius Antoninus was born in 121 AD. He became co-Emperor of Rome in 161 AD along with his adopted brother Lucius Aurelius Verus (Marcus apparently insisted). When Lucius died in 169 AD, Marcus continued ruling alone until his own death in 180 AD. He was the last of the rulers later known as the Five Good Emperors, and most people consider the end of his rule to mark the end of the Pax Romana.

Meditations isn’t really a book. For one thing, it has no actual title. Early versions are called “Treatise to Himself” (Greek: τὰ εἰς ἑαυτὸν ἠθικά), or referred to by terms like “a directing of his own life by Marcus the Emperor” or “the exhortations of Marcus”. Today we usually call it Meditations, but that doesn’t come from the author. 

Instead, it’s a collection of random stuff. Some of the entries are miniature essays on philosophical topics. Some are literal reminders (many entries begin, “keep in mind…”). Some are quotations from plays or books. Some of the entries seem to be self-help (“Remember not to…”). Some entries appear to be rough drafts for later entries. Some entries are inscrutable. 

Meditations is esoteric, fragmented, and weirdly repetitive because it’s excerpts from his various pieces of stationery, especially from workbooks where Marcus is trying to develop his understanding of philosophy. He reframes the same ideas over and over again because he’s trying to make them stick. It’s a collection of random stuff because it really is just Marcus’s own private notes to himself.

This is more obvious in some translations than in others. For example, compare entry 8.38 from the Collier translation: 

If you are so discerning, says the Philosopher, make use of your Talent to some purpose; and let your Subject be proportionable to your Parts.

…with this translation of the same passage from the 2002 translation by Gregory Hays, which is what I read: 

The stench of decay. Rotting meat in a bag. Look at it clearly. If you can.

Comparing translations like this got me into some trouble. I was sitting in the park reading my print version of the Hays translation, with a different translation (probably the Project Gutenberg version) pulled up on my phone. A guy jogged by and, seeing me with a book in one hand and my phone in the other, quipped, “can’t decide, huh?” I shouted after him, “they’re two translations of the same book!” but it was too late, I had already been painted as a rube. 

Meditations has a reputation for being eternal wisdom or advice to the ages, but that’s clearly at odds with the text. Marcus didn’t intend the work to be published, and probably never expected anyone else would even read it. Take for example entry 8.25, which reads: “Veras, leaving Lucilla behind, then Lucilla. Maximus, leaving Secunda. And Secunda. Diotimus, leaving Epitynchanus. Then Epitynchanus. Faustina, leaving Antoninus. Then Antoninus.” This may have meant something to other Romans of the time, but even just a few generations later, it would be quite hard to catch all those references. 

Other passages wouldn’t even make sense to other Romans. They clearly mean something to Marcus and Marcus alone. Like entry 1.7, where he remembers letters “like the one [Rusticus] sent my mother from Sinuessa”. Or entry 1.17, where he thanks the Gods for “remedies granted through dreams … [like] the one at Caieta.” Or entry 1.16, when he reminds himself of “most of the things at Lanuvium” and “the way [my adopted father] accepted the customs agent’s apology at Tusculum”. We have no idea what happened between the custom agent and Antoninus Pius at Tusculum, and neither would Marcus’s contemporaries.

The text is divided into 12 books, but there’s no clear structure. It’s not clear why the books appear in the order that they do, not clear if they were put in that order by Marcus or by a later editor. The divisions between books may just represent the individual papyrus rolls that Marcus originally wrote on, and we are in some sense receiving a copy of 12 of his scrapbooks. When one roll filled up, he started another.

The books are usually divided into individual entries, and we can use entry numbers like “1.11” to refer to the eleventh entry in Book 1. But it’s not always clear where one entry ends and the next one begins, and in some cases this is a question Marcus himself wouldn’t have been able to answer. So different translations sometimes disagree on where to put the divisions between entries.

The only book with any clear structure is Book 1, where Marcus provides something like an autobiography. He lists seventeen influences on his life (mostly his teachers and relatives, but the last entry is “The Gods”) and writes about what they taught him. This seems the most like an actual book or at least an actual essay, and may have been composed later than the other parts of Meditations. 

III. 

Marcus was maybe a fine emperor, but it’s not clear why we should look to him for advice. 

He wasn’t a tyrant, but neither were his policies all that different from previous emperors. He didn’t even preside over an era of great peace — he spent most of his reign fighting wars to keep the Empire from collapsing. And he botched one of the most important parts of the job when he passed the throne on to his son Commodus, a tyrant so unfit for the role that he was eventually murdered by his own people.1

Marcus seemed to view being Emperor as a duty, even a burden. What he really loved was philosophy. 

One of the things I learned from reading Meditations is that the term “Stoic” comes from the Greek word for porch. Stoicism was named after the Stoa Poikile (ἡ ποικίλη στοά), or "painted porch", a portico on the north side of the Agora in Athens where Zeno of Citium and other early Stoics hung out and discussed their philosophy. Even in English, Stoicism used to sometimes be called “the School of the Porch”.

So a more accurate word for “Stoic” might be “Porchie”. Combine this with the fact that “cynic” is just the Greek word for “dog”, and you start to realize that a different standard of translation could just as easily have given us "Porchies" and "Doggies" instead of "Stoics" and "Cynics."

If Marcus is your introduction to Stoicism, I can see why you would find things to like here. But despite his reputation as the quintessential Stoic, Meditations doesn’t contain that much Stoicism. There’s only one explicit reference, and a few quotes from Stoic philosophers. Many of his influences weren’t Stoics, and Marcus is constantly referencing other schools of philosophy, like Cynics, and even *gasp* Epicureans.

On top of this, Marcus didn’t make any original contributions to Stoic philosophy. The Stoicism in Meditations is all stuff he’s recording from other thinkers, sometimes even in the form of direct quotations. If you want to know about historical Stoicism, you’re better off going straight to Epictetus, who Marcus is reading and copying down (e.g. entry 4.41 among others).

As for his personal life, Marcus isn’t exactly an inspiring moral example. Like all upper-class Romans, he owns many slaves, and doesn’t seem to think there’s anything wrong with that. 

In entry 1.16, he writes that he admires his adoptive father for avoiding “bathing at strange hours, no self-indulgent building projects, no concern for food, or the cut and color of his clothes, or having attractive slaves.” Make up your own mind, but one of these things seems much less admirable than the others to me, and it seems weird to compare them. 

In entry 1.17, Marcus prides himself on not having taken sexual advantage of two of his own slaves (Benedicta and Theodotus). But he doesn’t seem to think it would have been wrong to have taken advantage of them, he seems to be proud because this restraint shows that he has mastered his passions. 

This intense detachment, even to the point of dissociation, is a weird red flag throughout the book. “Perhaps the most depressing entry in the entire work,” says translator Gregory Hays, “is the one in which Marcus urges himself to cultivate an indifference to music”:

Analyze the melody into the notes that form it, and as you hear each one, ask yourself whether you’re powerless against that. That should be enough to deter you. (11.2)

It’s also weird when he thanks the gods for the fact “that I wasn’t more talented in rhetoric or poetry, or other areas,” because, “if I’d felt that I was making better progress I might never have given them up.” (1.17)

There is some good stuff in here, especially in Book 1, where Marcus is listing people who have been important in his life and the “debts and lessons” that he owes to them.

Like 1.10, which many people could stand to hear: 

THE LITERARY CRITIC ALEXANDER [taught me to] not to be constantly correcting people, and in particular not to jump on them whenever they make an error of usage or a grammatical mistake or mispronounce something, but just answer their question or add another example, or debate the issue itself (not their phrasing), or make some other contribution to the discussion—and insert the right expression, unobtrusively.

This one (1.12) also stuck with me:

ALEXANDER THE PLATONIST [taught me to] not to be constantly telling people (or writing them) that I’m too busy, unless I really am. Similarly, not to be always ducking my responsibilities to the people around me because of “pressing business.”

When it comes to Marcus’s personal contribution, this is why I find Obama makes for a good comparison. Even several hundred years in the future, I suspect people will still be interested in a glimpse of the private thoughts of an American President. These are the reflections of one of the most powerful men of his era — it’s hard not to be at least a little curious.

So, what do we learn about such a man from reading his notes? Mostly, Marcus seems depresso. People treat Marcus like a font of wisdom, but there’s a lot of “did you ever notice how everyone sucks?” and “isn’t it hard to get out of bed in the morning?” 

Trouble with getting out of bed is a recurring theme, like in entry 2.1:

When you wake up in the morning, tell yourself: the people I deal with today will be meddling, ungrateful, arrogant, dishonest, jealous, and surly. They are like this because they can't tell good from evil.

Sometimes he is morbid, like in entry 8.34, where he asks, “Have you ever seen a severed hand or foot, or a decapitated head, just lying somewhere far away from the body it belonged to…?”

Marcus constantly reflects on death. If anything, he seems relieved by the thought of its eventual arrival, like in entry 8.2: “For every action, ask: How does it affect me? Could I change my mind about it? But soon I’ll be dead, and the slate’s empty.”

There’s a lot of what can only be called vagueposting, like entry 4.6, which begins, “That sort of person is bound to do that.” Then he goes back to thinking about death: “You might as well resent a fig tree for secreting juice. (Anyway, before very long you’ll both be dead—dead and soon forgotten.)”

He shows a lot of contempt for his fellow humans, like in entry 4.48: “In short, know this: Human lives are brief and trivial. Yesterday a blob of semen; tomorrow embalming fluid, ash.”

Sometimes he is ???, like in entry 9.37: “Enough of this wretched, whining monkey life.”

He even talks back and lectures himself, like in this absolute classic dialogue from Book 5:

At dawn, when you have trouble getting out of bed, tell yourself: “I have to go to work—as a human being. What do I have to complain of, if I’m going to do what I was born for—the things I was brought into the world to do? Or is this what I was created for? To huddle under the blankets and stay warm? 

—But it’s nicer here.... 

So you were born to feel “nice”? Instead of doing things and experiencing them? Don’t you see the plants, the birds, the ants and spiders and bees going about their individual tasks, putting the world in order, as best they can? And you’re not willing to do your job as a human being? Why aren’t you running to do what your nature demands? 

—But we have to sleep sometime.... 

Agreed. But nature set a limit on that—as it did on eating and drinking. And you’re over the limit. You’ve had more than enough of that. But not of working. There you’re still below your quota. 

You don’t love yourself enough. Or you’d love your nature too, and what it demands of you. People who love what they do wear themselves down doing it, they even forget to wash or eat. Do you have less respect for your own nature than the engraver does for engraving, the dancer for the dance, the miser for money or the social climber for status? When they’re really possessed by what they do, they’d rather stop eating and sleeping than give up practicing their arts. 

Is helping others less valuable to you? Not worth your effort?

IV.

Many people seem to think that becoming famous would give them a kind of deep and lasting satisfaction. Even when they hear famous people complain about how fame is a burden, how you can’t leave your house to go to the deli, they still imagine that for a celebrity, not being able to go to the store is an experience of sublime melancholy. And when you cut your toenails as a celebrity, you’re not just cutting toenails, they’re famous person toenails. In reality, it’s simply the normal bullshit of not being able to leave your house to go to the store.

I’ve never been Emperor of Rome, but my limited experience with gaining positions of respect tells me that it is not profoundly satisfying in the way people imagine. In fact, it can be quite stressful. What Marcus writes definitely matches this experience. 

Meditations is supposedly a favorite of presidents and ministers, and I can see how modern politicians would find his reflections reassuring. They worked hard to become senator or president, thinking it would fix them. But instead they just get piled on with more crap. The private moments of their lives contain the same tedious tasks as they did before. It must be a relief to read something that confirms it’s not just you, becoming important at work has sucked for the last several thousand years.

I’m reminded of Robert Cottrell’s review of the book Crassus by Peter Stothard, where he gives the following summary.

In brief: Crassus crushes Spartacus, makes a fortune in real estate, bankrolls Julius Caesar, partners up with Pompey, worries about seeming wussy by comparison, decides to invade Parthia for no particular reason, gets killed.

The book is part of Yale's Ancient Lives series, a series which hopes to persuade us that the big questions in life “have changed very little over the course of millennia”. I finished Crassus thinking exactly the contrary.

The ancient Rome of Gibbon is so varnished over with irony and high style that it reads like fiction. The ancient Rome of Stothard reads more like a newspaper delivered two thousand years late. Everything that happens in Crassus is described in admirably straightforward terms. But I can scarcely guess at the psychology of the people involved, nor model the choices that they faced.

The richest person on Earth invades a big foreign country in person, at the head of his own army, on spec, just for the lulz, and loses? Not to mention the six thousand slaves he crucified earlier in life along the Appian Way? I suppose if you could graft stem-cells from Vladimir Putin on to stem-cells from Yevgeny Prigozhin then you might grow something that ticked most of the boxes, but I don't understand those people either.

Ultimately, that’s what I got out of reading Meditations.

I didn’t have to read any dead Romans during my education, which may have been an error on the part of my teachers. But it used to be that if you got any sort of education at all, it involved reading a lot of dead Romans.

That probably made sense in the first few centuries. But a couple of hundred years after the fall of Rome, these guys might have started coming across as pretty alien to the average reader. Maybe this is why Marcus stood out.

Marcus lived almost 2000 years ago. He’s from a different time and a very alien culture. He was Emperor of Rome for almost two decades, one of the most powerful men who has ever lived. Other Ancient Romans, like Crassus, behave strangely and might seem inscrutable. But in his private thoughts, Marcus shows human follies and weaknesses that are uncomfortably familiar, even to a person of the 21st century. He’s very relatable, even if I can’t always personally relate.

Subscribe now

Today Pasteur is best known for the process that bears his name, pasteurization. But at the time, Pasteur was most famous as an opponent of spontaneous generation, the idea that living creatures commonly arise from nonliving matter. Pasteur describes the idea as: “Mightn't matter, perhaps, organize itself? Or posed differently, mightn't creatures enter the world without parents, without forebears? This is the question I seek to resolve.”

The ancients thought that snakes and frogs were spontaneously generated out of mud; that anchovies came from sea foam; and that fleas were created from dust, instead of from other fleas. There was a lot of fighting over the details, but almost everyone agreed that many animals came directly from slime. 

By Pasteur’s time, biologists like Redi and Spallanzani had shown that most living things came from parents. The remaining battlefield was whether or not microbes spontaneously generated from organic matter, or if they had parents too. 

Pasteur and the Irish physicist John Tyndall conducted what were eventually considered to be the conclusive experiments showing that spontaneous generation doesn’t happen, not even at the microscopic level. This is the work that Pasteur describes in his speech. 

In the first couple pages of the speech, Pasteur sets the background of the theory in order to challenge it, sharing some “absurdities” of spontaneous generation’s ridiculous history. His main example is a seventeenth-century account from "the celebrated alchemical physician Van Helmont", who he quotes as saying:

Carve an indentation in a brick, fill it with crushed basil, and cover the brick with another, so that the indentation is completely sealed. Expose the two bricks to sunlight, and you will find that within a few days, fumes from the basil, acting as a leavening agent, will have transformed the vegetable matter into veritable scorpions.

The quote as it appears in Pasteur’s original French can be found here, where the instructions are rendered:

Creusez un trou dans une brique, mettez-y de l'herbe de basilic pilée, appliquez une seconde brique sur la première, de façon que le trou soit parfaitement couvert, exposez les deux briques au soleil, et au bout de quelques jours, l'odeur de basilic, agissant comme ferment, changera l'herbe en véritables scorpions.

Pasteur gives the following source for this van Helmont quote:

Les œuvres de Jean-Baptiste VAN HELMONT, traduction de Jean LE CONTE. Lyon, 1671. in-4°. Première partie. Chap. XVI : La nécessité des ferments pour les transmutations, p. 103-109.

At first I despaired of finding such an ancient document, and I assumed that I would have to rely on Pasteur’s account. But I should have known better — of course the Internet Archive has a complete copy, with the relevant passage on page 105. Here’s a screenshot:  

This version is a translation by Jean Le Conte (presumably a translation from Latin; I haven’t been able to find an earlier version), and it matches an alternate French translation of these instructions that Pasteur (or perhaps his editors) offer in a footnote to the transcribed speech:

L'odeur enfermée dans la semence du basilique produit l'herbe basilique, avec l'esprit qui est dedans. Si elle se moisit en quelque endroit, elle change de nature, et produit des véritables scorpions. Ce que les incrédules pourront apprendre en mettant l'herbe contuse dans un trou qu'ils auront fait au milieu d'une brique, puis qu'ils joignent exactement une autre à celle-Iù, et qu'ils l'exposent au soleil.

You'll notice that this is actually somewhat different than the version Pasteur quoted in his speech, with some interesting differences. I’ll render it in English as best I can. Here’s my rough translation: 

The scent enclosed in the basil seed produces the basil herb, from the spirit within. If the seed becomes moldy in any place, it changes its nature, and produces veritable scorpions. Unbelievers can educate themselves on the matter by putting the bruised herb in an indentation that they have made in the middle of a brick, which they join exactly with another one, and expose them to the sun.

As they say, big if true. 

I don’t think it’s very likely that basil will turn into veritable scorpions, even under ideal conditions. 

However, I do think it is our responsibility as scientists to attempt to replicate empirical claims from other scientists when reasonably possible. This study is easy to replicate — it won’t take me much effort to put some crushed basil between a pair of bricks and leave them in the sun for a while. I might as well try it.

It’s especially our responsibility to attempt to replicate empirical claims when the original scientist invites us to try. Here, van Helmont specifically asks skeptics to try his procedure for themselves. He really makes it sound like he has run this experiment and gotten the results he expected. We should match his freak. 

Despite how crazy this all sounds, van Helmont is not a known crank. In fact, one of his studies is considered a “milestone in the history of biology”: He planted a willow tree and measured the weight of the tree, the soil, and the water he added. He found that after five years the tree had gained more than 150 pounds, with almost no change in the weight of the soil. He concluded that the plant mostly did not “eat” the soil, that most of the weight of the tree came from its water (in fact, most of the weight of a plant comes from carbon from the air). That’s the kind of attention to detail that, one would think, might prevent the recording of imaginary scorpions. He also invented the word “gas”. 

If van Helmont’s procedure does work as described, that would be a pretty big deal. Re-opening the book on spontaneous generation would admittedly be a bit of a headache. But it’s the kind of headache we should be open to. If spontaneous generation is real, I wanna know. And I’m happy to spend an afternoon making the world’s most terrible caprese sandwich in an effort to find out. 

Methods

Bricks 🧱

I started by collecting a number of bricks.

This was harder than expected. I discovered that the local hardware store doesn’t sell bricks, and also discovered that bricks weigh a lot more than you might think, even for bricks. In the end I was able to collect five bricks from vacant lots. This made for two pairs of bricks, plus a backup. 

The instructions call for the bricks to have “an indentation”. I decided to try two different kinds.

For the first brick (left), I hollowed out an indentation by repeatedly hitting the face of the brick with a small CRAFTSMAN® claw hammer (bottom center). 

I also found a Stiles and Hart Brick, which comes with a built-in indentation (right).

Basil 🌿

On Monday, June 24, I got two kinds of basil from the local Stop & Shop. 

The first was Nature's Promise Organic Basil in a flat plastic container (left). The other was Goodness Gardens Fresh Hydroponic Basil (right). I figured that these two provided a decent amount of diversity, in case the kind or quality of basil makes a difference for producing scorpions.

My basil obtained, I then had to decide how to prepare it.

Pasteur uses the word “pilée”, from the verb piler, usually translated as “to crush”, so this would be “crushed basil” or possibly “smashed basil” (compare glace pilée, crushed or shaved ice). 

Jean Le Conte uses the word “contuse”. Confusingly, this does not appear to be used in modern French. However, the English version of this term comes from Latin contundo, meaning beat, pound, or bruise. Le Conte likely got the word from this same root, so a reasonable translation might be “bruised” or “pounded”. 

I accomplished this crushing/bruising by putting each type of basil inside a separate ramekin, along with a small amount of water, and crushing it with a spoon. As above, the Nature's Promise Organic Basil is on the left and the Goodness Gardens Fresh Hydroponic Basil is on the right.

I then put each basil mixture inside a brick, the Nature's Promise Organic Basil inside the hand-hollowed brick and the Goodness Gardens Fresh Hydroponic Basil inside the Stiles and Hart Brick.

Once the basil pilée/contuse was inside the bricks, I covered each brick with a brick of equivalent size, completely trapping the basil within the indentations, and I set these brick configurations outside in direct summer sunlight. 

Sunlight ☀️

Sunlight was easy enough to get, as the next several days were to be sunny. So the main question at this point was how long it might take to turn the basil into scorpions. 

Pasteur’s translation suggests “within a few days” / “au bout de quelques jours”, while Le Conte’s translation doesn’t seem to suggest anything about how long this should take, saying simply, “expose them to the sun” / “qu'ils l'exposent au soleil”. However, he does suggest that the basil needs to become moldy (moisir).

Based on this, I planned to leave the bricks in the sunlight for 3-5 days before very carefully opening them up and checking for scorpions. If they failed to produce scorpions in that time, I might leave them for longer, especially if the basil was not visibly moldy by that point.

Subscribe now

Results 

I put the basil between the bricks on Monday, and first pulled them out on Saturday. 

I invited my friend Tim over to witness the results, in case my findings were to be the kind that might provoke disbelief. I also gave him a small tupperware vessel in which to contain the scorpions for further study, should any appear. 

However, when I opened the bricks, there were no scorpions. The basil in the brick on the left was still moist, while the basil on the right was dry and crispy.

While there were no scorpions to be found, I noticed that the basil had not moldered yet, at least not in the sense of becoming literally moldy. This seemed like it might be a problem, since the Le Conte translation implies that it was the moldering of the basil that led to the production of scorpions (“Si elle se moisit en quelque endroit, elle change de nature…”). So I covered the bricks back up with the bricks and put them back in the sunshine. 

On the following Wednesday, after more than a week in the sun, I opened them once more. There were still no scorpions, and the basil had not visibly changed. While the basil had still not moldered, I decided to end the study here.

Discussion

I was ultimately unable to reproduce the results van Helmont described, but I have a few guesses about why. 

Instructions Unclear, Scorpions stuck in Brick

The instructions passed down to us from Pasteur and Le Conte are pretty sparse, so there might have been any number of errors in my method. I could have used the wrong kind of basil, the wrong kind of bricks, even the wrong kind of sunlight. A lot has changed since the 17th century. Maybe you need to use bricks made from clay from the right part of Belgium. Maybe the variety of basil that turns into scorpions has gone extinct. Maybe I just needed to leave them out in the sunlight for longer. 

There are even some signs in the instructions that this kind of detail might be missing. The French is ambiguous, but suggests that the “semence” of the basil is what changes and produces scorpions. Now semence can be translated figuratively as “essence”, but it can also be translated more literally as “seed". So another interpretation is that basil stems and leaves don’t produce scorpions, and that I should have sealed up basil seeds between two bricks, and that their moldering might have successfully produced veritable scorpions. 

But if special conditions are required to produce scorpions, it’s van Helmont’s responsibility to tell us. He doesn’t say anything about how you need to use this kind of brick or that kind of basil to get things right. He doesn’t even say anything about how long you need to leave them out in the sunlight. Just “[put] the bruised herb in an indentation … made in the middle of a brick … [join it] exactly with another one, and expose them to the sun.”

This implies that van Helmont thinks this scorpion generation recipe is pretty robust. If the type of brick or variety of basil really mattered, he would have included that in the instructions. Since he didn’t say anything about them, he’s suggesting that you can use any bricks or basil, leave them in the sun for however long, and it should still work. Our responsibility is to take other scientists’ writing seriously. 

Translation Errors

All I have to work with are French translations of the original text, which was probably in Latin. On top of that, I have to translate the French into English. This means there are at least two places where translation errors might sneak in. The French looks pretty straightforward to me, but there might be all kinds of subtleties hidden in the jargon.  

The two French translations don’t even agree with each other, and it’s not clear why. Are they two independent translations from the original source? Or is Pasteur paraphrasing Le Conte in his speech, not quoting him directly? All these questions are hard to settle without the original source. If anyone can find van Helmont’s original instructions, please let me know. 

Scorpions from Elsewhere

A final possibility is that Pasteur was right, and basil doesn’t turn into scorpions under any conditions, that scorpions only ever come from other scorpions. If this is true, we have to wonder why van Helmont made such a wildly inaccurate claim.

It certainly could be the case that he was simply lying. But this doesn’t seem very likely — van Helmont was otherwise a respected scientist and intellectual. If we can’t trust the man who invented the word “gas”, who can we trust? 

It’s also possible that van Helmont never tried this design for himself, and was relying on someone else’s report. This also seems unlikely, especially given that he suggests “unbelievers” try it at home. That really makes it sound like he’s seen these scorpions with his own eyes. 

You might also think that van Helmont did try the design, and did find scorpions between the bricks — they just came from outside the bricks, rather than being spontaneously generated from basil. Perhaps there was some basil crushed into a brick, and van Helmont noticed a few days later that scorpions had found their way inside. Maybe the bricks were porous and there were already scorpion eggs coincidentally in the brick, or maybe moldy basil attracted a bunch of little bugs, which attracted the scorpions.

The problem with this idea is that van Helmont is from the temperate, northern European nation of Belgium. His Wikipedia page does say that he spent some time in other parts of Europe, like Switzerland, Italy, France, Germany, and England, but none of these places are famous for their scorpions. Maybe he saw a demonstration abroad, but more likely he tried the procedure at home in Brussels, and I don’t expect there are scorpions crawling around the gardens of northern Europe. 

Or at least, that’s what I thought at first, before discovering Euscorpius italicus, the Italian small wood-scorpion. Despite its name, E. italicus is found in countries across Europe — including very occasionally in Belgium. This paper from 2002 mentions eight sightings of Euscorpius scorpions “found in houses or in surrounding areas” around Belgium. And the Global Biodiversity Information Facility provides this map of Euscorpius italicus observations, including two near Brussels:

Wikipedia further says, “this species can be found in a variety of warm habitats, such as ruins, buildings, under household furnishings, and in crevices of walls. It is most often encountered near humans or places that humans have considerably changed. In nature, it hides under rocks.” They are quite small, sometimes smaller than a housekey. That definitely sounds like the kind of scorpion that might hide between two bricks. 

Or van Helmont might have encountered Tetratrichobothrius flavicaudis, the European yellow-tailed scorpion. T. flavicaudis is even less bothered by the cold than E. italicus, having made its way as far north as the British isles, where there is now a colony of around 10,000 scorpions. As with E. italicus, they can be quite small — see this photo from a very confused UK redditor. Wikipedia says, “in the UK, the scorpion occupies cracks and holes in walls where the mortar pointing has crumbled away.” I was even able to find this photograph of a T. flavicaudis happily chilling in the gap between two bricks! 

If I have any Belgian or UK readers, it might be interesting to replicate this procedure in your back garden — set up some basil and bricks, and see if they attract your local scorpions. 

Assuming the scorpions are finding their way in from the outside, and not being created directly from the spirit of the basil, you should be able to prevent their appearance by putting the bricks inside some kind of tupperware, or other scorpion-proof container. Comparing the rate of scorpion appearance between bricks inside a sealed container and bricks sitting around outside might provide a definitive test of where van Helmont’s scorpions were coming from, putting this question to rest once and for all. 

For now, I think the most likely explanation for van Helmont’s results is that E. italicus, T. flavicaudis, or a similar species found its way between his bricks when he wasn’t looking, and not that they were spontaneously generated from basil.

EDIT 10/9/24: The basil → scorpions connection may have been a common belief at the time. Adam Mastroianni recently sent me this passage from Chapter 7, Book 2 of Sir Thomas Browne’s Pseudodoxia Epidemica:

Many things are delivered and believed of other Plants, wherein at least we cannot but suspend. That there is a property in Basil to propagate Scorpions, and that by the smell thereof they are bred in the brains of men, is much advanced by Hollerius, who found this Insect in the brains of a man that delighted much in this smell. Wherein beside we find no way to conjoin the effect unto the cause assigned; herein the Moderns speak but timorously, and some of the Ancients quite contrarily. For, according unto Oribasius, Physitian unto Julian, The Affricans, Men best experienced in poisons, affirm, whosoever hath eaten Basil, although he be stung with a Scorpion, shall feel no pain thereby: which is a very different effect, and rather antidotally destroying, then seminally promoting its production.

The task is not to see what has never been seen before, but to think what has never been thought before about what you see everyday.

— Erwin Schrödinger (or possibly Arthur Schopenhauer)

The way to get somewhere in science is to ask dumb questions about things that are totally obvious.

Looking at obvious things and asking dumb questions is the source of our greatest achievements. Some guys were standing around and were like, hey, uh, what is air? What is the moon? Why does it look like that? Where did it come from? What is gold made of? Why do fleas bite us? What are sparks? When something gets hot, what’s happening to it? What is heat? What is light? Where do colors come from? And so on.

All these things — air, the moon, gold, fleas, sparks, heat, light, color — are pretty obviously real. They are for-sure real things, things that are definitely out there. This makes them great starting points. 

I used to call these kinds of topics and questions “natural”, after I found the following snippet from Paul Bloom: 

[In this course] we’re going to ask questions nobody would have otherwise thought to ask, where the common man wouldn’t address, and this is, of course, standard in all sciences.

The first step to insight is to ask questions like why do things fall down and not up? And I imagine the first person who articulated the question aloud probably met with the response saying, “What a stupid question. Of course things fall down.” Well, yes, of course things fall down, but why? Why is our flesh warm? Why does water turn solid when it gets cold? These are natural facts about the universe, but the naturalness needs to be explained and not merely assumed.

But calling a subject “natural” sounds rather judgmental, and carries the suggestion that other research questions might be “unnatural”. On top of that, the term “natural science” is already taken. I’ll agree that these are natural facts, but if we call this quality “natural”, things will get confusing. So to crib a phrase from Solomon Asch, who I suspect was referencing Euclid, topics and questions that are this stupidly obvious (and thus ideal for study) might be called self-evident. 

To naïve thought nothing is less problematic than that we grasp the actions of others, but it is precisely the task of psychology to remove the veil of self-evidence from these momentous processes.

— Solomon Asch (1952)1

Subscribe now

Unicorn Hunters 

Studying self-evident phenomena is great because whatever you do ends up being useful. If you make careful observations or develop a theory about something that’s self-evident, like “what are stars?” you are guaranteed to be gesturing towards something. 

You may still be seriously off-track. To the ancients, planets like Mars and Venus were stars, and the Sun was not a star,2 so the theories they had about stars were pretty far off the mark. But even if your theory is entirely wrong, the stars are clearly out there. And the observations the ancients made about stars were still useful, even if they thought that stars were holes in the orb surrounding the universe through which you could faintly glimpse the fires of Heaven. Whatever kind of thing they are, everyone can agree that the little points of light in the night sky are a for-real thing. 

Likewise, anything you discover about mountains is guaranteed to echo down through history (assuming you write it down) because mountains definitely exist, so any new observation, no matter how small, will contribute to an understanding of this very real thing. Anything you “discover” about unicorns is guaranteed not to echo down through history, because unicorns aren’t real. Sorry. 

Keep One Hand on Reality at All Times

Science is a lot like a murder mystery. The whole point of the mystery is to figure out how the dead body became dead. If everyone’s alive, there’s nothing to explain. 

Every self-evident phenomenon is a dead body. We don’t know where they came from, but we know that they’re there. Our job is to find out who done it. Detectives Newton and Maxwell and friends pretty much solved the murders of light and color, Detectives Pasteur and Koch cleaned up in the case of the microbiology murders, but there are still lots of dead bodies lying around for the rest of us to investigate.

You don’t go looking for bodies no one has ever seen before and which may not exist, when you are already tripping over the murder victims you already have. Yes, there are probably some missing persons that no one has noticed. You can choose to spend your time dragging the river to see if you turn up any new bodies, but why would you do that when there are so many cold cases already on the books?

Sometimes when you’re trying to solve a murder, you stumble across a dead body that you didn’t expect. And when this happens, it is pretty exciting. When Wilhelm Röntgen accidentally discovered the X-Rays, it was like coming across a whole new murder victim. Thrilling stuff. But Röntgen didn’t bring the police force to an anonymous patch of farmland outside the city limits and say, I think we are about to discover a new body, dig here. More like, he was investigating the murder of electricity, and when he went to interview a witness, he found an unexpected skeleton in their closet. 

The really exciting part of Röntgen’s discovery wasn’t even the new body, though that did get a lot of attention. The shocking twist came when detectives discovered that the new victim (X-rays) was the missing piece that tied together the murders of electricity, magnetism, and light, cases that up to that point, they had thought were totally separate! These were all the work of one serial killer, and his name is electromagnetism. Ok maybe this metaphor is getting a little strained.  

Subscribe now

My Backyard

I bring this up because despite the earlier quotes from psychologists, my home field of psychology has a surprisingly hard time staying focused on things that are self-evident. 

Instead of studying broad, clearly visible topics, psychologists tend to be interested in topics that are narrow, subtle, and sometimes invisible. Blue-ribbon findings in psychology include “being on a rickety bridge makes an attractive woman more attractive”, “people eat more soup when their bowl is secretly rigged to refill itself”, and “doing word scrambles that include words like ‘wrinkle’ makes you walk slower”. These projects do have tangential connections to self-evident topics like emotion, attraction, eating, and… I dunno, stereotypes? But they’re not really steps towards finding the answers to these mysteries. They’re just flags in the sand, claiming the discovery of previously unheard-of effects.

Also, all three of these examples fail to replicate. Their findings may not be real at all. But the actual problem is this: I can’t bring myself to care. These studies wouldn’t be important even if they were true. They just don’t tell us enough about the self-evident mysteries of psychology, and they don’t stake out a strong enough claim to present a new mystery. “Misattribution of arousal” has nothing on X-rays, which you could confirm by taking pictures of your bones. 

This gives me pretty mixed feelings about the replication crisis, which is a weird position to be in. I think the replication crisis is important, and I think we should teach students all about it, which is why I teach students all about it. In particular, the replication crisis has taught us a lot about the correct (and incorrect) uses of statistics and methods. I’m also glad that some very [redacted] psychologists were exposed. We do want some minimum of quality control.

But we also waste a lot of time arguing about whether or not particular effects are real, and whether or not they exist. I’ll grant some exceptions, but for most of the specific effects in question, who cares? I certainly don’t care if briefly exposing people to “a faint image of $100 bills” leads them to endorse “free-market systems and social inequality” more than people who saw a blurred version of the same image. I want to know what an emotion is, how many come included with purchase, and how many are available as DLC.

Maybe you disagree with me about whether or not these effects are self-evident, or whether these studies have any bearing on questions that are self-evident. That’s fine. I just want to remind you that our job is not to invent new concepts or discover new effects. Our job is to explain the things that are obvious.

Instead of spending all this effort trying to gin up discoveries, or playing whack-a-mole with the fraud of the week, psychologists should spend more time going after the many self-evident murders taking place in our beautiful psychology city. We should be asking questions like: Why do different people want different things? What kinds of different things can they want? Why do we get along well with some people and not others? Why does depression have the particular list of symptoms that it has?

The subjects of all these questions are self-evident. People want things, and different people want different kinds of things, in ways that are hard to categorize. Some people become fast friends and others become bitter enemies, and it’s hard to predict in advance how this will shake out. Depression officially includes symptoms like “insomnia” and “trouble concentrating” — those seem fairly different, why would it include both of these things? These are topics with solid foundations.  

Psychology doomers will tell you that all of psychology is bunk, people chasing phantoms and abstractions. They’re wrong. Bad studies get all the press, and yes, they get lots of the blue ribbons. But there is real self-evident murder-mystery psychology out there. All I’m saying is: I want more of it! I want us to focus on our real successes. For example:

People definitely make predictions about how they’ll feel in the future, and they often make decisions based on their predictions; this is self-evident. Dan Gilbert and his collaborators study these predictions. 

People definitely choose what to do, and at any given moment there are an infinite number of things they could do. They must narrow this list down somehow; this is also self-evident. Fiery Cushman and his collaborators think this probably has something to do with how the mind represents value. 

People definitely think some things are good (vacation) and other things are bad (COVID-19). That seems self-evident to me, so I tried to study it with my friend Adam Mastroianni. But we ended up finding out something unexpected about what happens when you ask people to think about how things could be different. Even so, this new topic also seems self-evident. People definitely spend some time thinking about how things could be different, and this is something we don’t understand all that well. 

Maybe in time we will return to the question of why some things rule (burritos) and other things drool (Congress). Until then, it’s a self-evident problem just waiting to be solved. Your move, detective. 

[Thanks to Paul for insightful comments on an early draft of this post.]

I HAVE BEEN in the middle of composing a pamphlet making a Summary of Alchymy’s Greatest Hits and my recent broadside addressed to the publick (the cause of some Complaints), expressed no Doubts to the strength of our Art, but was a meer measure to see whether there were any Discovery or Phænomena I had miss’d. Indeed I profess and doubt not that Alchymy has right to several major recent Discoveries, and these same Discoveries I did Bear in Mind at the time of my asking.

Perchance you hold suspicion that I am no honest Philosopher but instead a base Mountebank, so if You will give me leave, I shall Briefly propose to you a list of ten (seek my treatise Chymie (soon out in octavo!) for more details on each). All of these are robust findings, discovered (or at least substantially built upon) by Alchymists in the last Centuries, and I hold that most of them are interesting to divers Modern Naturalists, Physitians, and Philosophers. 

1. Gold is a Body so fix’d, and wherein the Elementary Ingredients (if it have any) are so firmly united to each other, that we finde when Gold is expos’d to the Fire, how violent soever, it does not discernably lose any of its fixednesse or weight, so far is it from being dissipated into those Principles.

2. However, the same Gold will also by common Aqua Regis, and (I speak it knowingly) by divers other Menstruums be reduc’d into a seeming Liquor, in so much that the Corpuscles of Gold will, with those of the Menstruum, pass through Cap-Paper, and with them also coagulate into a Crystalline Salt.

3. On sealing up a Mouse and a Candle lit a-Flame in a close Glass-Vessel, and watching to see what happens, the Mouse invariably dies before the Candle.

4. On inverting a Glass-Vessel over the Flame of a Candle and surrounding the Vessel's neck with Water, first the Water rises some height into the Neck of the Vessel, then after some time the Candle dies. For it seems very considerable, that parts of the Air in the Vessel are converted to Fire, and the little nimble Atoms of Fire escape through the Pores of the Glass.

5. After an eruption of Vesuvius, the inclination of a Magnetic Needle in the same Town made a considerable Swing. 

6. You may Wonder that a red-hot piece of Iron is attracted (as they speak) by a Magnet, though the Magnet shows no attraction for the Iron.

7. Impure Silver and Lead being expos’d together to a moderate Fire, will thereby be colliquated into one Mass, and mingle; whereas a much vehementer Fire will drive or carry off the baser Metals (I mean the Lead, and the Copper or other Alloy) from the Silver, though not, for ought appears, separate them from one another.

8. Nothing could seem so simple as the Burning of Wood, which the Fire Dissipates into Smoake and Ashes. But both may afterwards be Separated by other Degrees of Fire, whose orderly Gradation allowes the Disparity of their Volatileness to Discover it self. The latter of these is Confessedly made up of two such Differing Bodies as Earth and Salt; and the Former being condens’d into that Soot which adheres to our Chimneys, Discovers it self to Contain both Salt and Oyl, and Spirit and Earth (and some Portion of Phlegme too). 

9. Oil of Vitriol, upon a rough Bladder-Stone in a Glass or Earthen Vessel, remains and acts as a Menstruum.

10. The Dutch-Men that were forc’d to Winter in that Icie Region neer the Artick Circle, call’d Nova Zembla, relate that there was a Separation of Parts made in their frozen Beer about the middle of November, but of their Sack, which was later Frozen very hard, yet was not divided by the Frost into differing Substances, after such manner as their Beer had been. Thus even Cold sometimes may Congregare Homogenea, & Heterogenea Segregare.

I’m not actually a 17th-century alchemist working on a pamphlet — I composed this in response to fellow-psychologist Paul Bloom, who marshaled a list of ten “major recent discoveries” in psychology in support of the argument that psychology is ok (itself a response to an argument by my friend Adam). I think his list looks a lot like mine, though the prose is slightly more modern.

All of the alchemical findings above are paraphrased examples from real alchemists or early chemists. In several cases I have cribbed directly from Robert Boyle’s explanations of discoveries made around or before his time.

To the best of my limited chemical knowledge, all of the findings on this list ended up being both true and important. And the list is more than just a collection of scattered one-off findings, it shows some real direction. For example, items 3 and 4 are clear forerunners to the eventual understanding of combustion and oxygen. Items 1, 2, 7, and 8 all touch on questions of what elements exist (e.g. is gold an element or is it made up of other elements) and how they can be distinguished and studied. My list excludes alchemical bloopers like “the magnet briskly rubbed with garlic or placed near a diamond loses its force, which, however, can be fully restored if boar’s blood is poured on to the magnet”. Only the good stuff.

The truth of the matter is that for hundreds of years alchemy did produce major discoveries, reliable and robust findings, and many of these findings later turned out to be very important. (They also produced lots of crap, but that doesn’t matter, because science is a strong-link problem and over time the cream floated to the top.) We shouldn’t dismiss the alchemists. 

But are we to conclude from this list that “Alchemy is ok”? I don’t think so. At no point was alchemy ever ok. There was no agreement on basics like 1) are bodies made of elements, or elements and principles, or “Materiall Ingredients”, or something else, 2) assuming there are elements and/or principles or something else, what and how many are there, 3) how do you figure out whether or not something is an element or principle, or if it is a mixed body, 4) do you have to conduct division of bodies by fire alone or are there other methods of studying mixed bodies, and 5) what else do you need to measure to make sure your alchemy is replicable, viz. things like position of the sun and moon and planets within the zodiac, direction and speed of the wind, and temperature (which was still not very well understood). They needed Boyle, Lavoisier, Dalton, and a host of others to give them shape and turn them into chemistry. In Kuhn’s terms, they needed a paradigm. 

"Paradigm shift" is now so widely used that it has lost much of its force. People think paradigm shifts work along the lines of, "chemists used to wear periwigs and now they don't." But Kuhn meant something much more than just changes in fashion. He meant changes in basic concepts, and beliefs about “the fundamental entities of which the universe is composed” (his words). If you asked Galen of Pergamon which of the four humors you’re deficient in, he would gladly tell you. If you asked a modern doctor the same, they literally won't know how to answer. “I need more phlegm, how do I get more phlegm?” no longer means anything in medicine, and forget about asking for black bile.1

This is why paradigms don't spontaneously generate from big stacks of findings. You need a revolution, and often you can’t make full sense of your most important discoveries until you have one. Boyle knew it was important that gold never lost weight no matter how much you heat it up, but he didn’t know why it was important. That's why a "psychology is ok" attitude is self-defeating: if we keep doing what we're doing, we'll only get more of what we already have. 

Psychology today is about where alchemy was in the year 1661. Like the alchemists, we do sometimes produce true and important findings, but we have no way of making sense of them or fitting them together. We need a paradigm to give us shape and turn us into a mature science. No number of discoveries, however robust or interesting to non-specialists, can make that happen.

It was important to notice that a candle under a glass draws water up into the vessel, then dims and goes out. This was an early clue about the nature of combustion, and a sign of the element that would come to be known as oxygen. The effect is so robust that you can easily try it at home. But Philo of Byzantium was no chemist — he looked at this and saw atoms of fire escaping through pores in the glass! You can run experiments like this for hundreds of years without ever really understanding what's going on.

As Bloom points out, we know that all sorts of psychological traits (or at least measures we conceive of as related to those traits) are heritable, that people overestimate the frequency of rare but conspicuous events, and that some fears are shared across all humans. But how these facts fit together remains an open question. When we figure out the psychological equivalent of discovering the elements, we'll have more than just a collection. We'll have a paradigm.2

How do we do that? Good question, and a hard one, but I hope to write more about it soon. In the meantime, you can always go back and try to figure out how people like Newton, or Lavoisier, or Darwin did it. We have something very useful that none of those guys did: their example. What they wrote is in the public domain — you can read it, and sometimes you can practically see the gears turning. The more history of science you have, the more you can understand how it works, and how to make it better. But first you have to read it.  

Around this time last year (i.e. in Fall 2022), I taught a class about the replication crisis. This is a topic pretty close to my heart, because the replication crisis kicked off right around the time I started getting serious about psychology, so a big part of my education involved watching this apocalypse unfold in real time. I tried to re-create that experience for my students, just slightly sped up. We read a lot of the things I encountered at the time, and we read them in about the same order.

The class was taught as a seminar, with a simple format: for each day, students would read a couple primary sources. Then we would discuss those sources in class, where I’d give them context for the readings and help them make sense of how each reading fits in with the larger conversation. I can’t give you all that context here, but I can at least share the reading list. Here it is.

One of the most interesting parts of the replication crisis is how much of it happened on the internet, over "informal" channels, sometimes at lightning speed. The result is that the best pieces of writing on any subject are often on someone's blog. I tried to emphasize this aspect, because I think it is important. So while the syllabus does include some research papers, you’ll see that many of the assigned readings are pieces of journalism, blog posts, or even posts on social media.

Next time I teach this course, I’ll have to add readings about topics like the recent allegations surrounding Francesca Gino’s research, Gino suing Harvard and the Data Colada team in response to these allegations, more recent work on the sociology and incentives of academia, and more recent critiques of peer review. But that’s between me and the class of 2027.

Wednesday, 7 September

Overview

Readings Due:

Andrew Gelman – What has happened down here is the winds have changed

The Nib – Repeat After Me (comic)

Richard Feynman – Cargo Cult Science

Optional Readings:

Andrew Gelman – Why is the scientific replication crisis centered on psychology?

‎

Monday, 12 September

Fabrication

Readings Due:

The Nation – Disgrace: On Marc Hauser

Chapters 1 and 6 of Faking Science by Diederik Stapel, translated by Nick Brown.

Optional Readings:

Science – Harvard Misconduct Investigation of Psychologist Released

Any other Stapel Chapters

‎

Wednesday, 14 September

Sociology and Incentives of Academic Science

Readings Due:

Chapter 5 of Faking Science by Diederik Stapel, translated by Nick Brown. This chapter is pretty long and the first half isn't super relevant to today's topic, feel free to skim up to page 82, "I was pleased with my shovel and bucket in Chicago. Down in the basement..."

Erik Hoel – Publish and Perish

Étienne F.D. – Book Review: Making Nature

This SMBC comic

Optional Readings:

Tal Yarkoni – No, it’s not The Incentives—it’s you

Chapter 4 of Faking Science by Diederik Stapel, translated by Nick Brown. In this chapter, Stapel talks a little more about the "under the table" elements (his words) of pre-replication-crisis psychology research. Informative if you want to read more about how things used to be.

‎

Monday, 19 September

p-Hacking and QRPs

Readings Due:

Simmons, Nelson, & Simonsohn – False-Positive Psychology

538 – Science Isn’t Broken

Sacha Epskamp – Questionable Practices by Researchers and Teenage Wizards

Optional Readings:

Data Colada – Help! Someone Thinks I p-hacked

‎

Wednesday, 21 September

p-Hacking responses

Readings Due:

The Atlantic – The Myth of Self-Correcting Science

Pashler & Harris – Is the Replicability Crisis Overblown? Three Arguments Examined

Optional Readings:

Gelman & Loken – The garden of forking paths

Nature – Psychologists do some soul-searching

‎ ‎

Monday, 26 September

Power Posing

Readings Due:

Watch: The Original TED Talk

Data Colada – Power Posing: Reassessing The Evidence Behind The Most Popular TED Talk

Dana Carney (the first author on the original power posing paper) – My position on “Power Poses”

Optional Readings:

The Original Power Posing Study

The Replication Attempt

NPR Coverage

‎

Wednesday, 28 September

Power Posing Discussion & Scientific Criticism

Readings Due:

The New York Times – When the Revolution Came for Amy Cuddy

Simine Vazire – Criticizing a Scientist’s Work Isn’t Bullying. It’s Science.

Data Colada – Menschplaining: Three Ideas for Civil Criticism

Optional Readings:

A new analysis from Cuddy et al. in 2017

‎

Monday, 3 October

Statistical Power and p-Curve

Readings Due:

Data Colada – Samples Can't Be Too Large

Data Colada – MTurk vs. The Lab: Either Way We Need Big Samples

Simonsohn, Nelson, & Simmons – p-Curve Paper

Optional Readings:

Data Colada – We cannot afford to study effect size in the lab

Data Colada – "The" Effect Size Does Not Exist

Nature – Power failure: why small sample size undermines the reliability of neuroscience

www.p-curve.com

‎

Wednesday, 5 October

YOM KIPPUR NO CLASS

‎

Monday, 10 October

OCTOBER BREAK NO CLASS

‎

Wednesday, 12 October

Social Priming

Readings Due:

John Bargh – Nothing in Their Heads

Ed Yong – A failed replication draws a scathing personal attack from a psychology professor

Sanjay Srivastava – Some reflections on the Bargh-Doyen elderly walking priming brouhaha

Optional Readings:

Bargh, Chen, & Burrows – Automaticity of Social Behavior: Direct Effects of Trait Construct and Stereotype Activation on Action

Doyen, Klein, Pichon, & Cleeremans – Behavioral Priming: It's All in the Mind, but Whose Mind?

Daniel Kahneman – A proposal to deal with questions about priming effects

‎

Monday, 17 October

Feeling the Future

Readings Due:

The New York Times – Journal’s Article on ESP Is Expected to Prompt Outrage

Ritchie, Wiseman, & French – Failing the Future: Three Unsuccessful Attempts to Replicate Bem's ‘Retroactive Facilitation of Recall’ Effect

Andrew Gelman – A new Bem theory

Optional Readings:

Data Colada – Pilot-Dropping Backfires (So Daryl Bem Probably Did Not Do It)

‎

Wednesday, 19 October

Preregistration

Readings Due:

Data Colada – Preregistration: Not just for the Empiro-zealots

Data Colada – How To Properly Preregister A Study

Optional Readings:

Data Colada – AsPredicted: Pre-registration Made Easy

‎

Monday, 24 October

Many Labs I

Readings Due:

Klein et al. – Investigating variation in replicability: A “many labs” replication project.

Data Colada – "Many Labs" Overestimated The Importance of Hidden Moderators

Optional Readings:

Slate – Why Psychologists’ Food Fight Matters

Data from Investigating Variation in Replicability: A “Many Labs” Replication Project

‎

Wednesday, 26 October

On Failed Replications

Readings Due:

Jason Mitchell – On the evidentiary emptiness of failed replications

Tom Stafford – What Jason Mitchell's 'On the emptiness of failed replications' gets right

‎

Monday, 31 October

Growth Mindset

Readings Due:

BuzzFeedNews – A Mindset "Revolution" Sweeping Britain's Classrooms May Be Based On Shaky Science

Scott Alexander – Should Buzzfeed Publish Claims Which Are Explosive If True But Not Yet Proven?

Alex Tabarrok – Growth Mindset Replicates!

Optional Readings:

What did they say about it on twitter?

‎

Subscribe now

‎

Wednesday, 2 November

Reproducibility Project: Psychology

Readings Due:

Open Science Collaboration – Estimating the reproducibility of psychological science

Gilbert et al. – Comment on “Estimating the reproducibility of psychological science”

‎

Monday, 7 November

Reproducibility Responses

Readings Due:

Anderson et al. – Response to Comment on “Estimating the reproducibility of psychological science”

Gilbert et al. – A RESPONSE TO THE REPLY TO OUR TECHNICAL COMMENT ON “ESTIMATING THE REPRODUCIBILITY OF PSYCHOLOGICAL SCIENCE"

Data Colada – Evaluating Replications: 40% Full ≠ 60% Empty

Optional Readings:

Sanjay Srivastava – Evaluating a new critique of the Reproducibility Project

Gilbert et al. – More on “Estimating the Reproducibility of Psychological Science”

‎

Wednesday, 9 November

Facial Feedback Hypothesis

Readings Due:

Bayesian Spectacles – Musings on Preregistration: The Case of the Facial Feedback Effect

Registered Replication Report: Strack, Martin, & Stepper (1988)

‎

Monday, 14 November

Ego Depletion

Readings Due:

Scott Alexander – Book Review: Willpower

Carter & McCullough – Publication bias and the limited strength model of self-control: has the evidence for ego depletion been overestimated?

Hagger et al. – A Multilab Preregistered Replication of the Ego-Depletion Effect

Optional Readings:

Simine Vazire – fifty million frenchmen can eat it

Carter, Kofler, Forster, & McCullough – A series of meta-analytic tests of the depletion effect: Self-control does not seem to rely on a limited resource

Lurquin et al. – No Evidence of the Ego-Depletion Effect across Task Characteristics and Individual Differences: A Pre-Registered Study

‎

Wednesday, 16 November

Many Labs II

Readings Due:

Klein et al. – Many Labs 2: Investigating Variation in Replicability Across Samples and Settings

After 10 Years, ‘Many Labs’ Comes to an End – But Its Success Is Replicable

‎

Monday, 21 November

Brian Wansink

Readings Due:

Science – Cornell nutrition scientist resigns after retractions and research misconduct finding

Brian Wansink – The Grad Student Who Never Said "No" (Take a look at the comments on this blog post as well)

WIRED – This Thanksgiving, Ditch the Food Psychology

‎

Wednesday, 23 November

THANKSGIVING NO CLASS

‎

Monday, 28 November

Data Thugs

Readings Due:

Science – Meet the ‘data thugs' out to expose shoddy and questionable research

Nature – Meet this super-spotter of duplicated images in science papers

Optional Readings:

James Heathers – I Quit. And I’m OK With That

Joe Hilgard – Smell you later

‎

Wednesday, 30 November

Dan Ariely

Readings Due:

Data Colada – Evidence of Fraud in an Influential Field Experiment About Dishonesty

BuzzFeedNews – Dan Ariely Retracts Honesty Study Based On Fake Data

Optional Readings:

Science – Fraudulent data raise questions about superstar honesty researcher

Forbes – An Influential Study Of Dishonesty Was Dishonest

Original Paper

‎

Monday, 5 December

Generalizability and Paradigms

Readings Due:

Tal Yarkoni – The Generalizability Crisis

Thomas Kuhn – What Are Scientific Revolutions?

‎

Wednesday, 7 December

Peer Review & Publication

Readings Due:

Michael Nielsen – Three myths about scientific peer review

Vox – Meet Alexandra Elbakyan, the researcher who's breaking the law to make science free for all

Simine Vazire – Opening the Black Box of Peer Review

Optional Readings:

Vox – Let's stop pretending peer review works

WIRED – Peer-Reviewed Scientific Journals Don't Really Do Their Job

Smith (former editor of the British Medical Journal) – Classical peer review: an empty gun

Hate the peer-review process? Einstein did too

[I originally wrote this in August 2020, when I was teaching courses as a PhD student at NYU, and I’m reposting it here for reference. This is Part III; Part I is here and Part II is here.]

Exams were my white whale for this course.

My design goals were clear (see Part II). Someone who knows their stuff should be able to prove what they know and walk out of the class. Students should be encouraged to learn as fast as they can, and they should be rewarded for getting ahead of the class if they want to. And there should be almost no consequences for failure, so that students can experiment without torpedoing their grade.

But exams are famously plagued with problems. Rescheduling exams for students who are sick or have to miss a day. Deciding who is allowed to do make-up exams. The endless questions about exam format — “professor, will this be on the final?” Somehow, we complain about all this but take it for granted. Why not come up with a way to make these problems a thing of the past?

1. Final-First Exams

These days, professors have gotten more comfortable experimenting with exam formats. Lots of exams are open notes, open book, or even take-home. Some classes let you drop your lowest exam score. I’ve even heard of professors giving five exams and dropping your worst two.

Dropping tests is cool, because it fixes some of the classic problems. Have to miss an exam? No problem, just drop that one. No need for make-up exams. If you bomb an exam, just drop it.

This is the right direction, but we can go further. What else can we tinker with, to make exams even better?

I thought back to the cumulative format, and why it doesn’t work for teaching. Why have cumulative exams, then? Doesn’t it just serve to obscure your expectations? My class format was fractal (see Part I), so that students could see what’s coming, know what’s expected of them. Why not use this approach with exams, too?

Dropping one exam isn’t cool. You know what’s cool? Dropping ALL the exams.

I call the format Final-First, because your first exam is a final exam. In fact, every exam is a final exam, meaning every exam covers all of the material covered in the whole course. The exams have nearly identical formats, differing only in the particulars. I swap out the numbers and some of the details on the questions, but once you’ve seen one final, you have a pretty good sense of all of them.

This course was six weeks long, and I gave them a final exam at the end of every week. This means they had a final exam at the end of Week 1, at the end of Week 2, at the end of Week 3, and so on…

Since these were all final exams, I didn’t expect most of them would do very well on the first exam. But that’s ok, because we dropped all their exam scores except for the best one. The exam grade, as it contributed to their grade for the class as a whole, was entirely based on their best exam. Other exam grades didn’t contribute at all.

If a student gets a 90% on the third final, it doesn’t matter how they did on the first two. Why should a student suffer if they get a 10% on the first exam but manage to nail it with a 90% later on? Clearly that student has done a great job and learned all the material we wanted them to, even though they struggled at first. In fact, isn’t that more impressive?

This format has some great features, which are beautifully in line with my design goals:

• Good Incentives: If you understand the material quickly, you should be rewarded. Students who succeed are rewarded with more freedom. No one who has mastered the material should be forced to go through the motions. If you get a grade you’re happy with, you can choose to skip the rest of the exams with no downside.

• Safety Net: Each exam offers a new chance to set a minimum threshold for your grade. Once you get a 85 on one exam, you can rest easy that your grade won’t go any lower. With this design there are no consequences for failure. You can bomb (or miss) as many exams as you want without any risk to your final grade.

• Low Anxiety: Students who are able to get a good grade on one of the early exams will be able to worry about things other than cramming for the next exam. Maybe they’ll use it to study more, or maybe they’ll just go to the beach. I don’t care. If you can get an 80 on the final exam in week two of a six-week class, you deserve to go to the beach.

• Transparency: With this format, there’s no more need for, “what will be on the test?” Once you have taken the first final, you will know (approximately) the format of all the other finals. This has the added benefit of:

• Context: Seeing all the material at once will allow you to begin building a tapestry of ideas in your head. You will never be blindsided by new material, things you didn’t realize were expected of you. Once you’ve seen one final exam, you’ve seen them all, and being exposed to all the material early on will help you learn it better.

• Feedback: You will be able to tell what skills you have mastered and which you need to work on. This will allow you to spend your study time wisely. Previous exams become a great tool for review. You can go over your performance with the TA or professor and be able to see exactly what you need to work on for the next exam, because the next exam is so similar.

I was really happy with this design. It hit all of my design goals, and it resolves a lot of the classic problems with exams.

Other people liked the idea too. I was on a date with another PhD student and we were talking about teaching, so I told her about this design. She said, “that sounds a bit insane upfront, but not so much when you think about it.”

Now there was nothing to do but try it out. For this class, I made the exam 50% of the final grade. Normally, making a single evaluation a huge chunk of the grade is unfair. But with this format, the exams are the best one of six evaluations, and besides, the exams test what I really want them to know.

1.1 The Results

Final-First exams worked really, really well.

I was worried that students would be confused by the format, or would be terrified when they failed the first Exam, but I actually got very few questions about it. Students seemed to understand what I was trying to do.

It really did solve all the usual exam problems. No one ever asked me for a makeup exam. Only once did I have to clarify what would be on the exam. When students wanted to meet to go over their answers, we were able to make real progress, because it was immediately clear to me what parts of the material they had mastered and what they were still struggling with. In many cases we could look back over two or three different exams and see the same thing tripping them up every time over multiple weeks.

Most people improved steadily over time. The average grade went from 60% on Exam 1 (this was by design; see below) to 85% on Exam 6. Students took the exams pretty freely. Some of them took every exam, but on average they took only 4 of the 6 exams.

A few students actually got their best grade quite early on. On the first final, at the end of the first week of class, the highest grade was an incredible 88% (!!!). This student kept taking exams, though, and was able to eventually beat her record with a 92.5% on Exam 5.

The student who got the second-highest score on Exam 1 got a 84%, again very high for having taken only three classes. This student chose to skip most of the other exams. He did take Exam 5, but only got a 75.5%, so in the end his final grade was actually based on his exam score from the first week of class!

I was a little surprised that more students didn’t try to get a solid grade early on. When I think about this format, one of the most exciting things to me is the idea that you can teach yourself all the material, get ahead of the class, get a great exam grade halfway through, and not have to show up to class anymore. But while a few students got great scores on Exams 3 and 4, that was the exception. It might be different in a semester-long class. Six weeks is just not much time to teach yourself, even if you really commit to it!

These are extreme cases of the safety net working as intended, but the design worked equally well for students with less extreme grades. To my surprise, only 26 of the 39 students took Exam 6, the final final exam. I think this means that by the end of the class, many of them were satisfied enough with their exam grade that they chose not to take this last final. Of those who did take Exam 6, only 18 got a better grade on the final final than on any previous final, which means that 8 people didn’t improve their grade at all on the final final.

The best exam grade in the entire course, a 97.5%, was actually earned on Exam 5. Perhaps unsurprisingly, that student chose not to take Exam 6.

These grades are really impressive, because the exams were not easy. I came in with specific expectations of what a student should know by the end of intro stats. These expectations were reasonable, but they were also pretty high. We expect too little of undergrads, and we underestimate what they are capable of doing and understanding.

I didn’t change my expectations at all during this course. Every student who earned a 90% on an exam met my expectations, and every student who did better than that exceeded my expectations. In my opinion, a good grade means that they mastered the material.

1.2 Student Opinion

Students really liked the exams. Some of the most positive feedback was about this part of the class. Take a look:

“This was one of my favorite aspects of the course because it genuinely did relieve a lot of stress. My biggest fears for this course revolved around completing it and not only doing poorly, but also learning nothing. I think the weekly exams allowed me to continually refresh and apply what we had reviewed without the anxiety of failing the course.”

“I thought the idea of getting graded based on the best exam was exceptional since we learn more as we continue taking the class.”

“To be honest, this is the best [exam] format I’ve ever taken! It really gives me the motivation to study harder each time without getting too stressed out.”

Other comments were much the same. As you’ll notice, the experience students had with the format was exactly the experience I was aiming for. A few other notes of interest were:

“I found myself studying ahead of time to supplement the material I have not learned yet”

“Towards the end it was fine, but the first few were pretty stressful for me.”

The one complaint, which I did see a few times, was that the Exams tested them on questions they didn’t recognize and hadn’t seen before. But of course, this was by design, because I wanted to see if they really understood the concepts.

Some students seemed to understand this, with one noting, “[Ethan] helped us prepare as best as we could without actually giving us the answers.” And once again I’ll point to their excellent exam grades as proof that the difference in format wasn’t actually a problem.

Subscribe now

2. Exam Design

This format is certainly the most interesting part of the exams. But the design of the exams and the exam questions is worth discussing as well.

The Final-First exam format doesn’t work if you don’t pay close attention to the design of the exams. Exams need to be nearly identical, so that students always know what’s coming on the next one. But they can’t be too similar, or else students will memorize them by rote. You need to keep mixing it up.

I had a plan for the exams going in. As I argued in What You Want from Tests, exams should be used to test the knowledge that students carry around in their heads, the bits that an expert will internalize. That’s what I was aiming for in this class. Research reports would cover their ability to actually do stats, and exams would cover their memory and intuition for the most important concepts.

Then, of course, the whole course was forced online. I immediately knew this meant that exams would de facto be open book, open notes, and really, open Google. So I knew that I would have to pivot away from my original plans. I couldn’t just focus on internalized knowledge.

(I never explicitly told students that the exams were open notes, but I never told them not to look things up either.)

I actually think this ended up improving the exams. I stand by what I said in What You Want from Tests, but it can be more complicated than I imply in that essay.

2.1 Exam Structure

The structure of the exams mirrored the structure of the course — after all, every exam was a final. Each exam was 50 points in total. Of that, 15 points had to do with basic data skills, 15 points went to descriptive statistics, and 15 points were on the use and interpretation of inferential statistics. Just like the course, the exams were divided into these three sub-topics.

The remaining 5 points went to what I called “advanced topics”. These were questions about things I mentioned in lecture but that were slightly outside the scope of the class, more complex questions about the use of core concepts, or questions that tested their intuitions in ways we had hinted at, but hadn’t explicitly discussed.

An interesting consequence was that a student who mastered all the core material, but hadn’t yet achieved that deeper understanding, would only get a 90% on the exam, because the advanced section was the last 10% of the exam grade. A grade of higher than 90% means that a student understood not only all of the material at the expected level, but was making progress into understanding it more completely.

This is why I am so confident that the students who got above a 90% on their exam grade not only met my standards, they exceeded them. That last ten percent came from questions that were, by design, more difficult than an intro stats student should be able to answer.

2.2 Exam Difficulty

Maybe other teachers already know this, but something I had never realized before was that a teacher has a lot of control over the difficulty curve of an exam. I knew that a professor could make an exam more or less difficult, but I didn’t understand that you have a lot of control over the distribution of scores.

This was particularly important for a class using the Final-First exam format. In this system, most students take a final exam in Week 1, and of course most of them will bomb it. There’s a big difference in morale, however, between bombing an exam with 50% and bombing it with 5%!

I wanted to encourage students to do well. I wanted to make sure they felt like they could succeed from the very beginning. To make this happen, I designed the exam so that it was easy to get a decent score, but hard to get a great score. (For those of you who are statistically inclined, compare item response theory.)

(This is also how I asked Liz to grade the research reports. Make it easy to get a decent grade but hard to get a perfect grade, I said.)

I had already decided that 15 points, or 30% of the exam, was devoted to data skills. This stuff is pretty easy, and so I knew that most students would be getting a good chunk of points from this section right from the start. In the other two sections, I made sure to include a couple easy questions, to keep the baseline grade relatively high.

The fact that the average score on Exam 1 was 60% shows that I was successful. In fact, even in Week 1, the lowest exam grade was only a 40%. That doesn’t sound like much, but considering that we were only 17% of the way through the class, I think it’s pretty good.

I used some other tricks for this as well. One was that the exam was almost entirely multiple-choice. A classic problem with multiple choice questions is that students always have a decent chance to get the right answer by just guessing. For example, a student guessing on a multiple-choice question with four answers will get the right answer 25% of the time. An exam with nothing but 4-answer multiple choice questions has a baseline grade of 25%. It’s even worse for an exam that’s all true/false, which has a baseline of 50%. This is why up until 2016, the SAT took off 1/4 a point for each wrong answer. Statistically, it meant that a student who did nothing but guess would get a score of about zero.

But we can turn this same force to our advantage. To adjust the baseline score, I can change the number of answers I include for my multiple choice questions. This is exactly what I did. For the Data section, which I wanted to be a score-booster, all the multiple choice questions had only a few answers each. For the Advanced section, where I wanted students to earn points only if they really knew their stuff, most of the multiple choice questions had 8 or more response options! And for the other sections, which I wanted to land somewhere in between, I included a mix.

Of course, there are limits to how lenient we want to be. In particular, true/false questions seem too easy — a baseline of 50% just from guessing is way too high. One idea that I really like is True / False / Can’t Tell questions. At a shallow level, these are just true/false questions with three options instead of two. But at a deeper level, this encourages students to engage with the question in a new way. Instead of just determining which answer is right, they have to think about whether they even have enough information to make that call. It literally adds another dimension to the question. This is especially well-suited to statistics, which is all about making informed guesses based on limited information.

I used a similar approach in some of my short answer questions. I’ve noticed that in class, students are often much more comfortable telling you why something is wrong than trying to give you the right answer themselves. I translated this into “What’s wrong with…” questions. Students would be given a short paragraph that described some statistics. For the most part these were perfectly normal paragraphs, but I had always inserted at least one error. For example, sometimes I would say that a variable wasn’t skewed, but I would report a mean and median that were strikingly different (which is the classic sign of a skewed variable). Students would have to pick out the mistake and tell me why it was wrong.

This is a really important skill in real life. A big part of the practice of using stats as a scientist is noticing when something is wrong in an analysis, whether you’re checking your own analysis or looking over someone else’s work.

I included one of these questions in the Data section for almost every exam, since they are a good way to ask about data features like skew and range without just asking students to regurgitate the definitions. I also included a few in the Descriptive Statistics sections, and I think that added some nice variety. You know a student doesn’t understand correlation when you report r = 1.2 and they don’t catch it.

I realize now that I never included any of these questions about inferential statistics. This was a mistake, since catching errors in the reporting of tests is something that comes up all the time. If I taught this class again, I would put “What’s wrong with…” questions in all three sections of the exam.

Another way to control exam difficulty is with paired questions. You include two questions about the same topic, but one is easy, and one is harder. For example, in my descriptive statistics sections, I always included two questions where I described some data and asked students what plot or chart they should use to represent that data. By design, the first of these was always pretty easy, and the second was, while not exactly hard, a more sincere test of their understanding.

This has some great features. First, it helps raise their baseline score. A student who understands the idea even a little will usually get the first question right, and this will boost their grade. They essentially get partial credit on that concept, even though the question is multiple choice. (They say you can’t give partial credit on multiple choice questions, but what do they know?) But a student only gets full credit if they can answer the more challenging question. Again we see that the design makes it easy to get a decent grade, but hard to get a perfect grade.

Second, it helps with feedback. For any topic on the exam, if a student gets neither question right, they clearly do not understand the topic at all. If they get the easy one right but not the harder one, they understand the basics but haven’t quite got the whole idea. And if they get both right, it’s clear they understand it at the level I want them to. If they somehow get the hard question right and the easy question wrong, this tells you that they were probably guessing. You can look at the exam and see exactly how students are doing with each of the core skills.

2.3 Difficulty Over the Course of the Class

As important as the difficulty curve within an exam is, it’s also worth mentioning difficulty curves over time. Part of the reason to make an exam easy to pass but hard to ace is that this is good for student morale, while still being an accurate measure of their ability. With a Final-First exam, you also want to worry about difficulty over time.

Students shouldn’t get a good grade on the first final unless they really know their stuff. Early on, exam grades should be pretty low. But if exam grades go down with every exam, or even if they fail to go up, that’s bad for morale. It tells the students that they aren’t learning anything from the class. That shouldn’t be true, and even if it is, you shouldn’t be telling them that!

My recommendation is that your hardest exam should go first, and your easiest exam (while still staying true to what you want them to get out of the class) should go last, with the other exams in between in decreasing order of difficulty. And of course, for the reasons described above, your hardest exam should still be designed so that on average students do decently on it. If the average score on the first final is less than 50%, you’ve probably done something wrong.

One thing that I would like to do someday is create a way to generate exams automatically. These exams are formulaic by design, so it would be relatively easy to write a script that could mix & match components and spit out as many exams as you want. Not only could this make the exams more fair and regular, you could do things like share multiple practice exams with your students.

3. Exams Online

As with everything else, I was worried about exams being online. There were the concerns around cheating, as I mentioned in Part I, and also just around giving an exam remotely.

I was wrong. Holding exams online is one of the best things I’ve ever done for a class. It was so easy that I am seriously considering using online exams for in-person classes in the future.

I ended up running all my exams through Qualtrics, a survey software I use in my research. Qualtrics is flexible and it has a lot of nice features that are helpful for exams, but I suspect you could run online exams with other survey platforms.

Exams were run every week. Since my students were located all around the world, and since many of them had jobs or other responsibilities, I opened the exam for a full 24 hours. Lectures were Monday / Tuesday / Wednesday, and every week the exam was open from 5:00pm EST Thursday to 5:00pm EST Friday. Using the survey software, it was easy to leave it open all day and let them drop in whenever they wanted. I also liked how this didn’t cut into class time.

Qualtrics automatically records the time when a session is opened and when it is submitted, so I used that to time their exams. The exam would begin as soon as a student clicked on the link, since that prompted Qualtrics to record the session start. I recommended that they time themselves to ensure that they didn’t go over. We compared their start and their submit times to see if they followed directions. Some of them did go over by a little, but we were lenient, and graded those exams too. To my surprise, no one tried to sneak in a much longer exam session.

After some pilot testing with my sister, I ended up making the exam only 45 minutes long. This isn’t much time, but I figured it would be easy to add time later if I had to. I was worried that students would complain, and fully expected that I would have to bump it up to 60 minutes after the first few exams. But this ended up being unfounded too. I didn’t get any complaints about the exam length — students never mentioned it! — and so I kept it 45 minutes long for the whole course.

Short exams also fit my design goals. There’s no need to belabor an examination. As long as it’s accurate, it should be as short as possible. Once again, I imagined how it would be if, through some horrible clerical error, I was forced to take the class myself. I knew I would be able to ace the exam in about 15 minutes, so I wouldn’t be forced to waste more than a tiny amount of time. That’s how it should be.

Running exams online also gave us huge benefits on the backend. Exams were incredibly simple to grade. Once all the scores were in, I would take the exam myself, putting in all the right answers and writing ANSWER KEY as the student name at the end. Then, when Liz downloaded all the responses for grading, she could just use Excel functions to compare each of their answers to the responses I put for the answer key, and automatically assign points that way. There were always a few short-answer questions to grade by hand, but the majority of the grading, for every single student, could be accomplished in just a few minutes.

And unlike working with scantron or paper forms, there is no headache when it comes to digitizing the results. Answers and scores were in a spreadsheet from the beginning.

It was easy to make answer keys for the same reason. Admittedly I didn’t know this at first — all the credit goes to Liz. It turns out that you can make Qualtrics generate a PDF of all the answers given by a specific person, so all we had to do was get it to spit out the ANSWER KEY responses and, surprise, there was the answer key. Again your mileage may vary, but online systems can be very powerful.

The online format does offer students the opportunity to cheat. But as I already mentioned, I don’t think they did, and I don’t think it would matter either way. There are things you could do to help prevent this, if you were worried, like giving a narrower exam window or putting out multiple versions of the exam to prevent crosstalk, the sorts of things we already do in the classroom. You could make projects a bigger part of their grade. But I think it’s to everyone’s advantage to trust the students.

With a well-designed exam, it will be easier to learn the material than it will be to cheat. The same goes for open notes. If you make a good enough exam, students will actually find it easier to leave their notes closed.

5. What I Didn’t Get To

I got to put almost everything I wanted in this course, but there were a few things I missed.

I’ve always wanted there to be a bigger role for teams, but the teams in this class didn’t work very well. It seems like there should be ways to encourage students to help one another out, reward them for working together. But all the ideas that come to mind, like giving students bonus points for helping their teammates, have obvious problems. So while I want to incentivize teamwork and peer support, I haven’t come up with a way to make it happen yet.

Students would also really benefit from giving and watching presentations. I was able to do this for my RA, and it’s clear to me that she gained a lot from making the presentations and from getting feedback. Criticizing presentations and giving feedback is also good practice for statistical literacy, and it might be less intimidating for the average student.

But it would be difficult to have every student give a presentation. It’s probably impossible for large class sizes, and it doesn’t seem like it would work well online. During the semester, you might be able to do it in recitation, either for extra credit, or in small teams.

But the real problem is that giving a single presentation is like answering a single math problem. It’s just not that much practice. Unless the class size were very small, you probably couldn’t set it up so that every student got to present multiple times. This might be better suited to an advanced course. The breakout room activities, given that they include small and regular “presentations”, might be the best we can do here.

6. Concluding Remarks

I’ve heard a lot about the things you can and can’t do when teaching stats. I’ve heard that you can’t get students to pay attention. That you can’t make them care about the subject. That they’re all cheating on their assignments. That they aren’t smart enough to learn how to use statistical software on their own.

Things are bad in education today, but they’re not bad because of lack of funding, or because students are unmotivated. Things are bad because educators lack vision.

What else do you call it when everyone knows what the problems are, but no one manages to dream up solutions? We have the ability to make education work for us, and nothing special is required, just careful thought and patient experimentation.

In particular, there are huge gains to be had in developing approaches that let students and teachers stress less over the material and waste less time. This may free them to spend more time learning, but it may also free them to have a life outside the classroom. A class with more hours of homework, longer tests, and more fiendish questions is not a better class. In most cases it is a worse one.

What could be better than learning more, with less effort, and in less time? Let us celebrate academic laziness. Perfection comes not when there are no more assignments to add, but when there are no more assignments to take away.

Students have almost no control, of course, but it’s confusing how teachers continue to design classes with backbreaking grading loads for themselves. Just give fewer assignments, shorter assignments, assignments that are easier to grade. You can do this without making your class worse. In fact, you can do it while making your class better.

So many teachers teach classes that they themselves would hate. If you wouldn’t want to take your class, if you wouldn’t find it easy, then what are you doing? It seems unnecessarily cruel. Make your classes enjoyable. If you can’t make them enjoyable, at least make them easy. If you can’t make them easy, at least make sure they’re not a huge pain.

So many teachers are paranoid about students cheating, collaborating, or doing too well on tests. Are you a teacher, or a mall cop? When classes are fair, students don’t cheat. Even when classes are rigged, most students still refuse to cheat. Taking this approach creates a system where the most honest students are the ones who have the most to lose. I have seen too many honest students fail what should have been an easy class.

It’s August as I’m writing this, and online I have seen many examples of college professors sharing heavy-handed “how to be ok pages” or “COVID pages” that they plan to attach to their syllabi for the fall semester. These pages contain assurances that you can come to the professor with anything, that you can get extra time when you need it, and so on. Professors love these pages because it makes them feel like they’re doing something to make a difference. But these promises are hot air and all your students know it. If the structure of your class is cruel, this kind of statement becomes a sick joke. And if the structure of your class is kind, then you don’t need a page at the front of your syllabus trumpeting it. It’s the fundamental rule of communication: show, don’t tell. Put your good intentions in the structure of your class or not at all.

Just make a class that doesn’t suck.

Subscribe now

[I originally wrote this in August 2020, when I was teaching courses as a PhD student at NYU, and I’m reposting it here for reference. This is Part II of III; Part I is here.]

Grades are stupid. But at the end of the day, my university forces me to give everyone a final grade. And you do want to evaluate your students based on something, so they can know what they mastered and how they can still improve.

1. Design Goals

To begin with, I tried to work out my design goals. I started by thinking about the ways that classes normally fail and decided to work backwards from there.

One of the most blatant failures in the education system is when students are forced to take a class that they’ve already taken, or on a subject they already know. So my first goal was that someone who really knows the topic should be able to get a 100 with very little effort. There’s an easy way to check if this works: the course should be designed so that if, as the professor, I were to take it, I would ace it easily.

And not just ace it. Someone who really knows the material should, after demonstrating their knowledge, be able to walk out of the course entirely and never have to come back. Once you know the material, you shouldn’t be forced to waste your time regurgitating it.

A related problem is forcing students to waste time on concepts they already understand; or conversely, moving on to new material before a student is ready.

This is tricky because students really do learn things at different speeds. We can’t tailor the lectures to every student, but we can do things to help. Students should be given freedom to focus on the problems they find challenging. Once a student has mastered something, we should try not to bother them about it.

Similarly, most classes don’t incentivize students to learn things on their own. There’s no point getting ahead of the rest of the class. You’ll just be bored, and it might even hurt you, since it will be taking away from the time you could be using to cram the old material. This is a perverse incentive. If a student is ready to go further on their own, we should let them.

Basically, if a student wants to speedrun my class, who am I to complain? Let them do it.

Another classic way that classes screw up is by making students afraid of failure. With traditional grading, students have no room to experiment with different ways of learning, studying, and understanding. The class format pushes them to obsess about every evaluation, and encourages them to do the minimum amount required to get the grade, to take no risks. If they try something interesting and fail, their GPA plummets. This leads students to obsess over pointless minutiae, like what precisely is on the test, and exactly how to word their answers.

I wanted to save students the time they normally spend thinking about this nonsense. If they choose to spend that saved time studying, so much the better. If they don’t, then all we are losing is their anxiety. Either way, we should reward students for taking risks and attempting to go deeper with the material, not punish them.

In the end I came up with three ways to evaluate student progress.

First, I had a system to replace class participation and attendance, based off of small team activities, which counted for 30% of the final grade.

Second, I had students independently analyze two simple datasets of their choice, and write up a report about each. Together the two reports counted for 20% of the final grade.

Third, I invented a new exam format (covered in the next post), which counted for 50% of the final grade.

2. Teams & Breakout Rooms

I really hate attendance.

Taking attendance is undignified. It’s disrespectful of students, who are assumed to be incapable of making informed decisions about their education, and of the professor, who is implicitly supporting that assumption. Sometimes students get sick, have a family emergency, or need to go to the dentist, and they should be able to take care of these things without worrying about their grade. They shouldn’t have to send me an email with a doctor’s note. I don’t like getting those emails—just stay home if you’re sick—and I’m sure students don’t like sending them.

All of this is doubly true of online teaching. All the lectures are recorded. Students can watch and re-watch my presentations as many times as they want. Why should any of us care about them being “in class” when that means almost nothing in a virtual classroom?

When I taught Introduction to Psychology last summer (2019), I tried using a participation-based system. Rather than taking attendance, I had my TA mark down when students spoke in class. The idea was that this would encourage them not just to show up, but to participate in class discussions. I also hoped it would encourage them to do the assigned reading, which we discussed each day.

This didn’t work. Students would speak up even when they had nothing to add, just to get the grade. The quality of discussion suffered for it. Some very shy students didn’t speak at all, and lost points, despite the fact that they were doing great in the class otherwise. It was a huge pain for my TA to keep track of it all. This system didn’t do anything I hoped it would, and I think it was a failure.

We could just chuck attendance altogether. But on the other hand, it’s good to have some kind of incentive for students to show up to class. Recorded lectures are about as good as live ones, but if students show up to class most of the time, they can ask questions and I can get a sense of what they do and don’t understand. It would be good to encourage most of them to be there most of the time. Can we come up with a way to make this happen?

2.1 Enter the Zoom Room

One of the things that everyone learned early on in the pandemic is that video calls suck. Jumping onto a Zoom call is excruciating, and afterwards you feel drained of all will to live. Turning off your camera helps, but not by much.

At first this seemed universal. People speculated that it was something inherent to the Zoom platform. There were theories that the video latency, however subtle, was unnatural and jarring. But over time, I noticed two exceptions. The first was direct calls, with smaller groups. Hanging out with one or two friends over Zoom, while not as much fun as hanging out in person, didn’t make me want to tear my eyes out the way a Zoom call with several people did.

The other exception was playing virtual trivia. Early on in the pandemic, my friend Liz from my PhD cohort set up a virtual trivia night for students in our program. In virtual trivia, we would start off all together in one Zoom room. For each round, teams would be sent off into individual breakout rooms for 10-15 minutes to answer questions. Then we would all come back to the main room for scoring. We’d do this process every round, with a couple of trivia rounds each night.

This was infinitely better than every other group call I had been on, and it wasn’t just that we were a group of PhD students drinking late into the night. The breakout rooms were just as relaxed as being on a small call, and they broke up the evening in a way that made the main room much more fun, even though the full group was pretty large.

When I started thinking about how to run an online class, I knew I would have to include something like this.

(Liz also happened to be my TA for the stats course!)

I had been wanting to incorporate something about teams for a while, and this seemed like the perfect way to do it. Instead of sending teams off for rounds of trivia, I would send them off to do breakout room activities, and call them back to discuss the answers.

These activities took different formats depending on the topic we were covering each day, but most of them worked something like this. I put up a question or a task on the slides, and then sent the students into breakout rooms for about 10 or 15 minutes. When they came back, I randomly chose a couple teams to share their answers.

Getting the correct answer wasn’t the point. If the group provided an answer that engaged with the activity, the group got credit, even if their answer was incorrect. The only way to get no credit was to not engage with the question or to give no answer at all. If I didn’t call on a team, that activity didn’t affect their grade.

This seemed to be the perfect replacement for attendance. At least one member of every group would need to be there every day, while individual members could come and go if they needed to. But part of their individual success would come from helping to make sure that the whole team was successful, so it was still in their interest to show up and help out whenever possible. I didn’t need to keep track of who was there, I just needed to give activities and ask them for their answers. And I didn’t even need to grade their responses, just record if they made an attempt.

I also hoped that this would give them some level of social support for the class — the kind of friendship they would normally get from the students sitting next to them, and people to go to if they needed help or support.

Another benefit was that this broke up the huge lectures into smaller chunks. I had already added intermissions to break the 2.75-hour classes into two sessions of about 1 hour 15 minutes. With breakout room activities, days could end up being four sessions of about 30 minutes each, with activities and an intermission in between. That’s a lot better.

This was also meant to be a grade boost. A whopping 30% of their final grade came from their team grade, and because all you had to do was show up and try to answer the questions, I expected most teams to get 100%. I included this grade boost because I didn’t want them to worry about their final grade too much. This way, they would still have to work to get an excellent grade, but a student who did a decent job wouldn’t have to worry about failure. (As I mentioned earlier, I think that grades are kind of a joke.)

I shared a brief stats experience survey with my students the week before class, and I assigned them to teams based on their responses. I wanted to make sure that each team had a diverse collection of skills — that there was at least one student in every group who was comfortable with public speaking, at least one with decent math skills, and so on. The idea was that every team would have the skills they needed to succeed, and they would all have someone to turn to for help on any subject. I ended up with eight teams of five students each.

2.2 How did Breakout Rooms Work?

The grading worked just as planned. Seven of the eight teams got perfect marks on their breakout room activities. The other group missed one day (none of them showed up) and got about 90% on the team grade. But in general this provided exactly the padding I intended.

Or, almost. In retrospect, 30% was way too much. Students got really good grades anyways, and it wasn’t all thanks to the team grade — remember, more than 50% got an A! Making the team grades only 20% or even only 10% wouldn’t have changed their grades by very much, because they were all doing so well on other parts of the class. I think it should have counted for less than 30%, because it’s a shame that so much of their grade came from something unrelated to their understanding of the material. I am very happy so many of them got a 95 — I just think it would be better for them to get a 95 from nailing the assignments and exams than showing up and participating! It’s something I would do differently next time.

The activities worked really well. Lectures can be, let’s face it, pretty boring, and I think having these class exercises helped keep students from falling asleep. There’s also no better way to learn something than doing it yourself, so following each lesson with an exercise was a good idea. And it was nice on my end to take a quick break, wait a few minutes, and see how they had done when they came back.

You do have to be careful with the activities, though. Activities work well if they are a simple problem, something the students couldn’t do when they showed up, but can do now that they’ve seen the day’s lecture. This helps the lesson stick in their memory, and demonstrates why what they just learned is actually useful. Activities can also take a “don’t take my word for it, see for yourself” approach, and I liked this when I was able to use it.

No matter what though, the activities have to be easy. They aren’t a challenge or an exam; they exist to round out the lecture and serve as a teaching aid. It’s ok if students struggle with the details; it can be good for them to get a sense of their own limitations. But if they get stuck, can’t do the activity, or reach a dead end, then they don’t learn anything. The implicit message is that they can’t handle it, and that’s not the right message to send them. They can handle things that you’ve prepared them for; don’t give them assignments you haven’t prepared them for.

Students had mixed opinions of the teams. I got feedback like, “there was zero accountability for the breakout rooms … Most of the time, my teammates wouldn’t show up” and “as the days progressed, my group became unresponsive to the point where I was simply doing the work and presenting it on my own.” A few of them did have positive things to say about the teams, but that was clearly the minority opinion.

Most students liked the breakout room activities, though. “I was able to apply the material and then receive feedback (if called on) instantly. The breakout rooms presented a great opportunity to work through what was being discussed,” one student said. Another wrote, “Breakout rooms really allowed me to understand the application of concepts. I don’t think I would have been able to work through the research reports (or the finals) with as much ease had we not gone through related work individually and then as a class.”

The only complaint I saw about these activities was that I gave students too much time to work on them. I find this confusing, because I assumed students would be happy to have an extra 5-minute break to go and make a sandwich or something. Either way, I mark this idea as another success. It does seem like it helped the concepts and skills really stick with them.

Some students suggested that the activities be designed to more directly prepare them for the exams — basically, to have the activities be examples of the kind of questions that appeared on the exams. I can see why they proposed this, but I don’t like it. The exams are designed to try to see if students can generalize stats concepts to new situations. (And from their grades, it’s clear that by the end they could!) If I give them practice with questions of a similar format, I think that would defeat the purpose.

Obviously then, the problem is the teams, and it’s not clear to me what the solution is. Students suggested that I could have them do the work as a team but then call on individual students for the answers. That’s a little too invasive for my taste. One reason to have teams is to help less confident students — you know, the kind who would hate being called on.

I could imagine making the teams larger, maybe groups of 7-10. With more students, it’s more likely that some of them would show up. I could also make the teams smaller, maybe just 2 or 3 people per team. This would lead to less diffusion of responsibility. In either case, I’m sure there would still be slackers. Students don’t like having slackers on their team, but if everyone is getting a 100% on their team grades anyways, I don’t mind if there are a couple freeloaders. Maybe teaching this in person, if that ever happens, would change the dynamic and solve the whole problem.

If I were to teach this in a classroom rather than online, I would have them do more class activities, but have each activity be smaller/shorter. Sending people to breakout rooms on Zoom is a bit of a commitment. It takes a minute to send them out and to re-orient on coming back, so you want them to get their money’s worth. But teaching in person, it would be better to just give them more diverse tasks. Rather than giving them a 10-minute worksheet, I would do something like throw three histograms up the board and give them 3 minutes to tell me what values you could and could not reject from each.

Subscribe now

3. Research Reports

About a year ago, I wrote an essay called What You Want from Tests, where I outline two kinds of knowledge that you need to have mastery over a skill. The first is the sort of things that every expert carries around inside their head, and this is what I argue you should try to examine with exams and quizzes. The other kind of knowledge is the ability to actually use the skill. Without the ability to use the skill, any knowledge is just trivia. You’re not an expert, you’re just a fan.

Statistics is a skill-based course, so the second kind of knowledge is really important. I didn’t just want my students to memorize a bunch of facts about statistics, I wanted them to learn how to actually use statistics.

A few years ago I was working with an undergraduate who had volunteered to be my research assistant. She was an exceptionally bright and curious student, who always asked remarkably insightful questions. She was also very diligent, and had already taken several stats classes before she started working with me. She had even taken some MA-level stats courses, which is unusual and impressive for an undergrad.

Despite all this, I discovered that she did not really understand stats. She didn’t understand many of the concepts. She had a hard time conducting even basic analyses. Despite her excellent grades, almost nothing from the classes had stuck with her.

I already knew that she was gifted, and I was aware of the shortcomings of the usual stats education approaches, so I reassured her that it was not her fault, and I offered to help her do something about it.

At this point I had already done a lot of thinking about how to do a better job teaching stats, and I realized that people always forget to teach this practical side of the skill, even though the practical side is what actually matters. Now, there’s no mystery about how to teach skills. I learned stats by struggling through real analyses for projects that were actually important to me, and everyone agrees that working on a project you genuinely care about is the best way to pick up a new skill.

But this doesn’t work in every situation. Even for me, it was a struggle, and this sink-or-swim approach is too harsh for the classroom. It’s also inefficient for beginners, because real data is messy and confusing. If students bring in a real problem, the correct approach might be too advanced for an intro class. And scale makes it impossible. Do we expect every student in an intro course to be able to bring in a project they’re thrilled about? They don’t know anything about the topic yet, so they don’t know what a good project would be.

I realized that all these problems could be fixed by using fake datasets. It’s easy enough to generate data, and you can make it look however you want. And unlike a real project, you can introduce concepts one at a time so that the student is always ready for them.

So that summer, I made a bunch of practice datasets for my RA to work with. I wrote a set of R functions that would automatically generate datasets to my specifications. At the start of each day, I would give my RA a short lesson on a stats concept, and then send her a couple datasets. Naturally, most of the datasets would be in some way related to that day’s lesson. She would work on them all morning, prepare some slides, and at noon, before we broke for lunch, she would give us a presentation on what she found out. I let my other RAs give feedback first (giving critique is great training as well), and then I would ask questions and give her feedback.

The first datasets were extremely simple, and they gave her no trouble at all. Once she was comfortable with conducting simple analyses on her own, I introduced complications, the sort of wrinkles one would expect to find in a real dataset. First I introduced the concept of statistical power, and gave her some critically underpowered studies, so she could learn to interpret those null results as inconclusive. Then we had a discussion of outliers, when and when not to exclude them, and the datasets for that day included different kinds of outliers. We covered causal inference, interactions, p-hacking, and many other concepts in the same way. The concepts in these lessons were cumulative. Once we had covered outliers, for example, I would sometimes put outliers in the datasets later on.

The datasets at the start of the semester were really easy. The datasets by the end were almost as tricky as real-world data. But at no point did my RA work on anything that was too hard for her. Each new complication was just one step up from something she had already mastered, so she was always prepared to tackle it.

3.1 Class Projects

I knew I wanted to do something similar for my class, to give them the same kind of practice with the practical side of things. In particular, I like this approach because for each dataset, you have to figure out what statistical test to run on the data. This is one of the stats skills you use most often in the real world, and it’s often the first question you ask when thinking about an analysis. Yet somehow, intro stats classes almost never teach this skill. At best, students get handed an extremely confusing flowchart. I knew I could do better.

Unfortunately the approach I used with my RA doesn’t exactly scale. I couldn’t give them the same kind of step-by-step training. I couldn’t have them all give a presentation on every dataset, and of course, many students are terrified of presenting to begin with.

Still, I figured I could come up with something that captured most of the benefits. I took several of the simpler datasets that I had made for my RA and I put them in a folder on the class website. Rather than having to analyze all of them, students were required to pick two of these datasets and write a research report about each of them. They could do these two reports at any point during the class, but since they weren’t taught how to do most analyses until about halfway through, I expected most of them to do these assignments during the second half of the course.

Students are taught to write long. This is a bad habit, especially when working with such simple datasets. I limited research reports to a maximum of one page long, including any graphs and/or tables. Students should learn to be concise, and besides, I didn’t want Liz to have to sift through dozens of extra pages when grading.

Each research report was 10% of the final grade, so these assignments were 20% of their grade in total. They were free to analyze the data however they wanted, but in particular we thought that R, SPSS, and Excel/Google Sheets were good choices, so I included one session for each of those approaches in the lectures.

This wasn’t much training, to be sure. A lot of people might have seen this as a big risk — you’re expecting them to use R or SPSS with barely more than an hour of training each? But I wasn’t worried about it. Somehow I knew that they were up to the task.

Originally, I was planning to let students do up to two additional research reports for extra credit. But in the week before class, one of the students suggested that instead of doing research reports for extra credit, we could let them re-do research reports that they weren’t satisfied with. This basically translated to “do 4 research reports, get your grade from the best two”.

I liked this for a couple of reasons. First, it let them make mistakes on early research reports without huge consequences, which was one of my design goals for the class. Second, students who were struggling would be encouraged to do additional reports, which would give them the extra practice they need, while students who didn’t need additional help wouldn’t be bothered.

I implemented this change, with the requirement that the do-overs would have to be on new datasets. Students would get feedback from Liz about how to do better, but they would have to apply those lessons in a new context. I limited them to two of these do-overs at most. I wanted them to be able to learn from their mistakes, but also I didn’t want each of them doing 10 reports.

The research reports were not really about the grades. They weren’t so much intended as evaluations. Really, they were more like practice, or lessons. What I really wanted them to get out of the research reports was, “I can do this and it’s not scary”, because I think it will help set them up to be confident when using these skills in real life (and on the Exams). It wasn’t about challenging or testing them, it was about giving them the opportunity to try things for themselves.

About halfway through the course, one student emailed me to ask for more guidance on how to format the reports. At the very least, she said, I should give them an example of what one would look like. I told her:

This assignment is designed to mimic what doing analysis is like in the real world. Data is emailed to you in a confusing format, and the file is poorly organized. The people who have hired you to conduct the analysis don’t know exactly what they want and can’t tell you what kind of test to conduct; after all, that’s what they hired you for. I’m trying to give you a controlled version of this experience — not nearly so confusing as real life, but where you are asked to exercise your judgment and the knowledge we’ve covered in class. Giving you any more guidance on how to conduct the analysis or write the report would defeat the purpose of the assignment.

To this student’s credit, she totally understood my point and ended up getting a 98 on both research reports.

A final reason to like the research reports is that they capture my “walk out of class once you’ve mastered the material” goal. If you already took stats but you were for some reason forced to take my class, or if you decide to teach yourself all the material in the first week, then you can just throw together two one-page reports, get an A+ on both of them, and forget about this part of the class entirely.

3.2 How did they do?

Students really surprised me on the research reports. When I first looked at the grades, I thought that maybe Liz had been too lenient. Almost all of them had gotten A’s! But when I looked closer, I saw that the students had earned them. The reports weren’t perfect, but they showed serious critical thinking and really creative engagement with the datasets. All very impressive for a subject they had been studying for less than six weeks!

When I looked back, I saw that on their first submissions, many students had gotten B’s and C’s. Liz wasn’t being too lenient at all. In fact, her feedback was intensely detailed. But this helped the students enormously. It’s clear that the students took that feedback and turned it around for their do-overs, and that’s what ended up earning them those A’s.

Some students, I was happy to see, didn’t need the do-overs. One student did her first two, got a 98 and a 99, and unsurprisingly, chose not to submit any more. Another student, who had said in class that she was terrible at math, gave it a shot and to her great surprise earned a 93 and a 90. She decided that was good enough for her, and didn’t send in another. The system works.

I especially liked how diverse the reports were. Students used all sorts of weird charts and phrased their results in all sorts of unusual ways. Not wrong per se, just the sort of thing an expert would never do. I think this demonstrates genuine understanding. Rather than just copying someone else’s approach, they had come up with their own, often slightly bizarre perspective, and then applied it. That’s what mastery looks like, folks.

How about the software? Some of them came to me or to Liz for help, but honestly, not as many as you might expect. For the most part they seem to have taught themselves.

When I was looking through the reports, I saw that most of them chose to use R for their research reports, and almost all of them did a solid job of it. This was a big surprise, but it’s very encouraging.

In conversations about how to teach stats, I’ve often heard, “It would be great if we could teach the students R or python. But you just can’t teach the average student a programming language in only one semester. It would take up too much of the lecture, and there would be too many questions for the TAs to handle. We should stick to SPSS worksheets and formulas for now, that’s the sort of thing that students can deal with.” I’m happy to have evidence that, in my opinion, proves this entirely false. Apparently students can learn the basics of R with almost no instruction, and in less than six weeks, as long as you give them the right environment for it.

(And I later heard from the person who TA’d statistics during the semester that this happened:)

I’m pretty happy with the research reports. Is there anything I would do differently next time? Well, one thing Liz pointed out to me is that while I gave them 24 different options to choose from, most of the reports people submitted were analyses of the same 4 or 5 datasets. These were some of the most straightforward datasets, and most of them involved analyses of correlation between two variables.

Now, as I said before, the research reports are not really about challenging students. I’m fine with them doing two easy reports, since doing any independent report at all is great for intro stats. But conducting correlation tests both times does slightly defeat the purpose of doing two reports.

A better system would be to break the research reports up into different bundles. Bundle A could be the easy ones and Bundle B could be more challenging. Or Bundle A could include all the correlation tests and Bundle B could include the others, so that every student would have to use at least two different tests. You could maybe include a Bundle C of advanced datasets. These could either give you extra points just for attempting them, or they could be strictly for extra credit. In any case, adding some more structure to the research reports would probably improve them.

Subscribe now

[I originally wrote this in August 2020, when I was teaching courses as a PhD student at NYU, and I’m posting it here for future reference. This is Part I of III.]

This summer (2020) I taught Statistics for the Behavioral Sciences.

The course was unusual for a number of reasons. I’ve wanted to teach stats for a long time, so I came into this class with a collection of unorthodox ideas that I’ve been sitting on for a few years.

I always suspected that students are capable of more than they’re given credit for, so I started out with high expectations. At the same time, I worried that might be a problem, maybe I was expecting too much. I didn’t shift my expectations, or make the class easier halfway through, but more than half (!) of my students got an A or higher. These grades mean that most of the students either mastered the material to my satisfaction or came very close to doing so. This approach worked and I would definitely recommend it.

1. Course Format

1.1 Being Online

The big curveball for this class was the pandemic, which made it necessary to teach the class online. I’ve never taken a course online, and I had never expected to teach one that way. Going into this, I had almost no experience with online classes. When we transitioned to online instruction in March, I was TA’ing for a class, so I got to see how that went. But that was about it.

I’m confident in my skills, but there were a few things in particular that I was worried about.

One of the really rewarding parts of teaching is getting to know your students. But Zoom isn’t that great, so I was worried that there might be no personal connection. Partly I was worried that the class would be less enjoyable. People like making friends and knowing that the instructor cares about them. But part of it was also practical. Without that sense of the classroom and knowledge of the students, I was concerned that I wouldn’t be able to tell when students didn’t understand the material. Maybe I wouldn’t be able to explain things as well when they had questions.

The other major concern I had was cheating. I knew that in the transition towards online classes brought on by the pandemic, many schools forced students to install unsettling exam-monitoring software on their personal devices. This sort of thing is pretty evil. While I would never spy on my students, it did make me worry about cheating on exams. With online exams, it seems like it could be a real problem. But I also know, from being a TA, that students cheat a lot less than professors think they do. In the end I took no special steps to prevent cheating. I don’t really care about or believe in grades, and I decided to trust the students.

1.2 Personal Connection

It turns out that both of these concerns were unfounded.

Admittedly, there was very little personal connection. I didn’t get to know most of my students. I would recognize their names, but I never even saw most of their faces.

But no one seemed to suffer for it. In the end we still developed the rapport that you need for good teaching. In their evaluations, students said things like:

“Ethan was a great teacher! He clearly loved the subject, and wanted to try and teach it in a more accessible way”

“Ethan specifically explained things very well and was so real. It was nice hearing examples in ‘layman’s terms’ that were more approachable”

“I really felt as if this teacher wanted us to do well, and helped us learn as much as possible in the clearest way possible. … Great great teacher!”

and

“Ethan is cool”

This experience has changed my mind about classroom engagement, and makes me doubt some of the common wisdom about teaching.

Is getting to know your students a reasonable expectation? Certainly we can get to know our students. But is it appropriate? Students aren’t in your class to be your friends, and you’re not there to be their pal. People are in the classroom to, hopefully, learn something.

“Personal connection” often seems to be used as a proxy for respecting your students and treating them like human beings. But — surprise! — you can respect your students and treat them like human beings without necessarily having a friendship with them, or even knowing their names. Students are sensitive to this difference. They care about being treated with respect, but don’t seem to care about the other stuff.

A cynical take would be that professors use the excuse of “getting to know their students” to push students into having an unnecessarily friendly relationship. But pretending to be equals when you are in a position of power over someone is at best dishonest, and at worst is a way of denying that you have a responsibility to them.

I do think there are things you can do to drive “engagement”. But I don’t know if it really matters. My students got really good grades and displayed surprisingly deep understanding of the material, so the lack of personal connection didn’t hurt their education. And many of them told me that this was one of the most enjoyable courses they have ever taken, so it didn’t seem to make learning any less fun.

1.3 Cheating

I was even more wrong about cheating. I didn’t see any evidence of cheating on exams or assignments, and there was plenty of evidence that they weren’t cheating. Students made lots of simple mistakes, which if they were cheating, they could have avoided. Exam scores improved incrementally over time, just as you would expect from honest learning. Their assignments and answers on the tests were idiosyncratic, not the carbon copies you might expect if they were sharing answers. If students were cheating, they didn’t leave any trace of it, and so I’m inclined to believe that they didn’t.

The lack of cheating is a little weird. When I was a TA, I would catch students cheating all the time. They usually do a bad job hiding it — they forget that I was a student not too long ago, and so they don’t realize that I know most of the tricks. So the fact that we didn’t see any of the classic signs is strong evidence that they didn’t really cheat.

So why didn’t they cheat on my class, when they do cheat during the semester? I think it has to do with trust. In the exit survey for the class, one student wrote down, “no feeling of being ‘cheated’ by the prof”. Another student wrote, “My biggest fears for this course revolved around completing it and not only doing poorly, but also learning nothing.”

Students stoop to cheating when they think, often correctly, that there is no other way to do well in the course. When professors are unclear about expectations, or make examinations needlessly difficult, the students feel cheated by the professor, and will turn to cheating themselves. When you see an exam filled with trick questions, it’s hard not to feel like the game is rigged. But to their credit, even in this situation, most students still won’t cheat.

Teachers have a lot to learn about cheating. If you don’t cheat your students, most of them won’t cheat on your assignments. It’s about trust. Not your trusting that they won’t cheat on assignments — their trusting that you won’t cheat them in their education.

This all makes it especially disappointing that, during this pandemic, so many schools are engaging in unethical surveillance of their students in the name of academic honesty. Students just don’t cheat all that much, even when they can definitely get away with it.

Subscribe now

2. Course Content

So much for the course format. What was I actually teaching?

2.1 What’s Wrong with Stats?

Statistics education is pretty terrible, and everyone knows it. All the professors who teach stats agree: students come into class, usually manage to pass, but retain almost nothing.

Everyone is looking for the magic bullet. But even so, no one thinks it’s a great mystery. Professors and TAs will all tell you the same thing: the problem is motivation. The majority of students, they say, simply aren’t interested in learning this form of esoteric math. As a result, most of the proposed solutions are motivational: find a way to make it fun and interesting, or at least find the right set of rewards and punishments.

But when I was a TA for intro stats, I noticed that this didn’t match what I saw at all. The students in my recitations were engaged, and really wanted to understand statistics. They asked insightful and sophisticated questions, and were always pestering me for more detail. Yet somehow they seemed to come back every week having forgotten everything we discussed the week before. This isn’t the behavior of students who are checked out — this is the behavior of students who are trying, and repeatedly failing, to build a model of what is going on around them.

Even if I had been wrong about most of them, there were a few students who were clearly both able and motivated. These students got perfect scores on multiple tests and assignments, regularly came to my office hours, and discussed many of the concepts in great detail. They showed me the extensive, meticulous notes they had taken in lecture. But when it came to answering simple questions about the material in a new context, they always came up blank.

These students weren’t lacking in motivation or intelligence. So it must be external; something about the class was failing them. Even if everyone in the class were as motivated as these high-achievers, we would still be having trouble with comprehension and retention.

2.2 Driver’s Ed

I think the motivation story is all wrong. The problem is that statistics is taught at the wrong level.

Imagine you are taking a driver’s ed course, and have just shown up to the first day of class. The professor gets up and says, “Hi everyone, in this class you’re going to learn all about cars. Cars are really amazing. Some people use cars to get to work. Some people use them to get to school. Some people use them to go on vacation! There are a lot of kinds of cars. The big ones are called trucks. Those ones carry things like fruit and gravel. In this course you’ll learn all the different kinds and their uses, and we’ll talk a bit about the history of cars.”

You raise your hand, “Excuse me, professor. I’m here because I want to learn how to drive. I didn’t come here to learn about the types or history of automobiles. I’m sure that knowledge will come in handy in some ways, but it’s really not my focus. How do you actually drive?”

“Worry not,” he says. “To drive, move the wheel back and forth.”

So you leave that course and you sign up for a different one. You show up to the new class, and the professor gets up and says, “Hi everyone, in this class we’re going to learn all about cars. We’re going to be starting with the drivetrain. It’s important that you be able to describe and identify all the parts. Look at this diagram. Here’s the gearbox (which you can see is constant-mesh), clutch mechanism, the flywheel, the differential…” You get up and walk out of the room.

Neither of these classes will teach you how to drive. And sadly, this is a pretty good metaphor for how statistics is usually taught. Some statistics courses give students an overview of probability theory and a brief sense of the history, without teaching them how to actually conduct an analysis. Others throw the equations right on the board and start discussing the terms without any context. All too often, a single class will try to include both of these approaches. This is probably worse than either of them alone.

Students don’t want to learn a list of tests, the life history of Ronald Fisher, or the exact meanings of the terms in the formula for the pooled standard deviation. These are all things one naturally picks up over time, but none of it is useful without the core knowledge. Students want to learn what statistics is and how we actually use it. But somehow they seem to come away from our courses without having been taught either of these things.

Driver’s ed focuses on the point of contact: how to use the car. Similarly, the main goal of this class was statistical skills and how to use them.

I wanted students to become statistically literate. Most students won’t end up being researchers or statisticians in the same way that most people who take driver’s ed won’t end up being auto mechanics or engineers for GM. We still benefit from knowing what a car is and how to operate it. Similarly, students benefit from knowing what statistics is and how to use it. For those students who do want to go on to use statistics professionally, this will still give them a strong foundation. Auto mechanics don’t suffer from having taken driver’s ed in high school.

The focus was limited and practical. Students were taught how to recognize different kinds of variables and data, interpret standard plots and graphs, read and understand statistical reports, and conduct basic analyses using statistical software. I alluded to other subjects of interest in my lectures, but in lessons and evaluations, I focused on these basic skills.

We can also talk a little bit about what I didn’t want to cover. The history of stats is interesting, but most of the time it doesn’t help you be a better statistician. The most important thing to know about the history is that these tests and concepts were just invented by a few guys not all that different from you and me. Anyone can make up a concept or design a new test. You assign it a Greek letter and suddenly it sounds official, but for all we know, Fisher came up with it while sitting in the bathtub. Besides that, most of the details don’t matter. Aside from a couple of helpful examples, I didn’t teach them anything about the history of statistics.

You do need to know a few symbols to be able to interpret tests, but I didn’t want to cover much in the way of formatting. I don’t care if students report a number as 0.02 or .02 or 0.0212; I don’t care if they write “p-value” or “p-value”. Time is limited, and I don’t want to waste their time or mine going over this nonsense. If by the end of the class, they know the concepts but not the formatting, then I have succeeded. If they know the formatting but not the concepts, I have definitely failed. So I decided to focus on the concepts and, as much as possible, ignore the formatting.

2.3 Fractal

So that’s what I wanted to teach. How do you actually teach something like this?

Most courses take a cumulative approach. You start with the basics, and the material slowly becomes more and more complex. Each lesson builds on all the previous lessons. At the end you finally tackle the most advanced material. Then you take the final exam.

In my experience, this falls apart by the second week of class. Students who miss even a single lecture are cut adrift, left to founder or drown. Even if you make it to every class, your safety isn’t guaranteed. If you don’t understand the explanation they give in lecture for some topic, you’re out of luck, because the class is never going to come back to that topic again.

Rather than being cumulative, my course approach was fractal. A fractal is a figure or function where every part has the same character as the whole. Every part contains copies of the whole thing. That’s how I structured the course: every part of the course was nested within other parts of the course.

You could be the best teacher who ever lived, with the most beautiful slides imaginable. It doesn’t matter — students just can’t learn something in one go. This is especially true in statistics. The classic learning pattern for the subject is brief flashes of insight, a feeling of sudden understanding, and then losing your hold on it and slipping back into confusion. This is normal.

For some reason, people don’t understand this. Everyone thinks there is going to be a shortcut explanation for these ideas, but we don’t think that way about other skills. We don’t think that painters will master three-point perspective in a single session, and we don’t expect programming students to master for loops in a single day. Maybe you can get the gist after the first introduction, but really understanding these topics takes time. Somehow we see stats differently. In particular, there is a whole genre of articles and blog posts all about how to explain p-values. These assume that the concept can be distilled into a single statement, or a single lesson. But that’s crazy. You can’t understand p-values in one hour, no matter how good the explanation is.

I think of statistics as really being three closely-related topics: a language for talking about data in general terms, descriptive statistics for talking about individual variables, and inferential statistics for making educated guesses about the world on the basis of limited samples.

The structure was built around these topics. The first day of class was an overview of the entire course, introducing all three topics in very general terms. Day 2 and Day 3 were another microcosm: again we covered the whole course, this time in slightly more detail.

Week 2 covered data in more detail. Week 3 covered descriptive statistics. Weeks 4 and 5 covered inferential statistics. Finally, in Week 6, we went even deeper into inferential statistics, exposing the math behind each test, and how it works.

This means that students see every single topic many times before the end of the course. For example, the two-sample t-test appears a total of six times in the lectures. It appears first in day one, during the complete overview, again in the lectures for day three, and then again in weeks three, four, and six.

It doesn’t matter if you don’t understand the two-sample t-test the first time, or the second time, or even the third time you see it. It doesn’t matter if you miss a few classes. It doesn’t matter if one of the examples I use doesn’t make sense to you. We will come back to this concept again, in a new context, with new examples. By the end of the class, you will get to see it from every angle.

These things take time. Mastery of a subject comes only when you return to an idea over and over, seeing it in new situations and becoming more familiar with it, building your own understanding. The structure of the class needs to support this, or students won’t be able to learn a damn thing.

2.4 Context

My influences in this were the Snowflake Method, and Progressive Rendering from It’s Time For An Intuition-First Calculus Course. Both of these perspectives emphasize understanding the gist of an idea before getting stuck in the details. To quote the reasoning from It’s Time For An Intuition-First Calculus Course:

The “start-to-finish” approach seems official. Orderly. Rigorous. And it doesn’t work.

What, exactly, do you know when you’ve seen the first 20% of a portrait in full resolution? A forehead? Do you even know the gender? The age? The teacher has forgotten that you’ve never seen the full picture and likely can’t appreciate that you’re even seeing a forehead!

Progressive rendering (blurry-to-sharp) gives a full overview, a rough approximation of what the expert sees, and gets you curious about more. After the overview, we start filling in the details. And because you have an idea of where you’re going, you’re excited to learn. What’s better: “Let’s download the next 10% of the forehead”, or “Let’s sharpen the picture”?

Let’s admit it: we forget the details of most classes. If we’ll have a hazy memory anyway, shouldn’t it be of the entire picture? That has the best shot of enticing us to sharpen the details later on.

Sometimes I think of this course as Intuition-First Statistics. “Intuition-first” doesn’t mean our goal is to teach good statistical intuitions, though hopefully students do get some of that. It means that we should start by working with intuitions, and everything else will follow from that. Because, although it may sound surprising, students actually have pretty strong statistical intuitions.

The problem is context. The cumulative or start-to-finish approach makes perfect sense to the instructor, but only because they already know what is coming. They can see the context; how everything is connected.

The students don’t have any of that. They just get hit in the face with new material that they never saw coming. Every day it’s some new bullshit. They have no idea what is up next, what it means, or how it all is related. They’re always being knocked off-balance by new topics you didn’t prepare them for, and they never have time to figure out how it’s all connected.

This is a huge problem, because context really matters for comprehension and memory. A great example comes from research by Bransford & Johnson (1972). In their studies, participants heard a paragraph like the one below. Take a look at this passage and see if you can figure out what it is all about:

The procedure is actually quite simple. First you arrange things into different groups. Of course, one pile may be sufficient depending on how much there is to do. If you have to go somewhere else due to lack of facilities that is the next step, otherwise you are pretty well set. It is important not to overdo things. That is, it is better to do too few things at once than too many. In the short run this may not seem important but complications can easily arise. A mistake can be expensive as well. At first the whole procedure will seem complicated. Soon, however, it will become just another facet of life. It is difficult to foresee any end to the necessity for this task in the immediate future, but then one never can tell. After the procedure is completed one arranges the materials into different groups again. Then they can be put into their appropriate places. Eventually they will be used once more and the whole cycle will then have to be repeated. However, that is part of life.

One third of the participants heard the paragraph without any context. It didn’t make much sense to them, and they had trouble recalling what they had heard.

The next third of the participants, before hearing the paragraph, were told that it was about doing laundry. To these participants, the paragraph made perfect sense, and they had very little trouble recalling the details.

The final third learned the topic only after they’d heard the entire paragraph. These participants also found the paragraph confusing, and even having been given the context, weren’t able to recall much about it. Context alone isn’t enough; you need to see the context up front.

Something similar happens in class. Without context, even the most motivated students have trouble remembering the material. They have a hard time memorizing tests or equations because they don’t understand what a test is used for, let alone how it works. I don’t have trouble with the equations, but only because I understand what the tests were created to do. It’s easy to put things into their proper categories if you have a good grasp of the system; it’s impossible if you don’t even know what categories are possible.

The fractal approach solves this problem. The first two or three times I went over the material, I didn’t expect them to remember any of it. We cover all the material early on, because being introduced to everything at a shallow level prepares students to understand the material in depth once it comes back around again.

Subscribe now