Striving for elegance

Experimental design is my favorite aspect of being a scientist. It requires a perfect balance of creativity and analytic thinking. I was originally trying to think of some general guidelines for creating a tightly-controlled experiment, but I realized that everyone's process may be a little different. So, I decided to highlight three key approaches I incorporate in every experimental design.

When I was in my early years of graduate school, I remember seeing some experiments that others described as elegant. I thought, wow, I want to design experiments like that. It took a few years, but by the time I was a postdoc, I received a review on a manuscript that actually described my experimental design as elegant. The reviewer even noted "This conjunction of ideas is nice - a kind of Occam's Razor applied to the complexity of different aspects of memory updating." How did I acheive the best reviews I could possibily hope for? Here I have compiled 3 approaches I have applied to honing my trade of creating elegant experimental designs.

Establishing the appropriate controls

So you have a hypothesis you want to test. For me, this has often involved questions about how memory influences behavior. Let's use this question as an example: How does remembering a specific event influence later memory for that event?

What's a good control for remembering? It turns out years of research has found a pretty good control condition: restudying learned material. Typically, it's done like this: in a retrieval condition, subjects are shown a picture of a face and prompted to recall the person's name. In a restudy condition, subjects are shown both the face and name together. Sounds like a pretty good control, right? Well, there are a few problems.

  • The retrieval condition is harder. It requires effort that the restudy condition does not require. Unfortunately, this may be a problem that is difficult to fully get around. However, I have developed some ways to keep subjects relatively engaged during restudy.
  • You don't know what subjects are doing in the restudy condition. Are they rehearsing the association over and over again in their head? Are they passively reading it and not really paying much attention to the information? Are they even reading it at all? If there is no response required that somehow monitors their behavior, you will have no way of knowing what subjects are actually doing.
  • Subjects might get the answer wrong in the retrieval condition. Unless performance is at the floor, this might not be a problem. You could just cut those incorrect trials from analysis. On the other hand, you could take advantage of memory errors to study updating! (see post on Memory is Malleable)

Here's how I addressed these problems. 1. I changed the manipulation, so that it was more matched across the two conditions. For this, I had subjects drag objects to specific locations on the computer screen using a mouse. In the active retrieval condition, subjects had to remember where the object had originally been, and place it in that spot. In the passive condition, I placed a small box in the object's location and told them to place it there. In these two conditions, subjects had to engage with the object -- they had to physically move it with the mouse. Even though the active condition is still more effortful, this manipulation forced subjects to pay attention to the task at hand. 2. In my studies of spatial recall, I took advantage of the fact that subjects always placed the objects back in a slightly wrong spot. Why try to force subjects to act a certain way when it's not in their nature? The data told a different story -- errors are inherent in our memories. Unfortunately, memory is typically studied in a black and white fashion -- its either correct or incorrect. But memory doesn't behave this way. Some details are remembered perfectly, while others are forgotton or distorted. This happens too often for it to be some sort of anomaly.

Collecting data from multiple sources

More to come soon, including combining eye movements + behavior, and neural activity + sophisticated experimental design, and different patient populations.

Measuring phenomena in multiple ways

Here I will discuss different ways to measure the same phenomena. This approach can give you greater confidence that you have a real effect. Some examples include button press response selection vs. distance measure. Another example is to use behavioral responses and eye movement measurements to measure memory accuracy.