Since October, I and nearly 3,000 other volunteers have been participating. We view a "movie" of blood circulation in the brain of a living mouse and rate whether there is or is not a "stall" in any of the blood vessels. This preliminary phase compares the citizen scientists' results with a sample rated by professional scientists at Cornell, to determine whether we laypeople en masse are as accurate as the professionals' and can be used in the research.It has long been known that reduced blood flow in the brain is associated with Alzheimer’s disease and other forms of dementia. However, new imaging techniques have only recently enabled our collaborators at the Schaffer-Nishimura Lab (Cornell University) to make important discoveries about the mechanisms that underlie this reduced blood flow.
For instance, one such cause seems to be capillaries becoming clogged by white blood cells. By sticking to the inside walls of blood vessels, white blood cells cause “stalls” – instances where blood is no longer flowing. It seems that around 2% of the tiniest blood vessels in the brain can become stalled in Alzheimer’s, causing up to 30% reduction in overall blood flow. This is likely to contribute to further disease progression and typical Alzheimer’s symptoms. In fact, the researchers at the Schaffer-Nishimura Lab have recently demonstrated that reversing stalls in mice also reduces Alzheimer’s symptoms, such as cognitive decline and mood changes.
While the research at the Schaffer-Nishimura Lab is promising, it is also incredibly time-consuming. In fact, the data that takes about one hour to collect, takes about a week for a trained scientist to analyze. At this rate, it could take decades to find functional Alzheimer’s treatment candidates. Fortunately, the data curation step, though still too complicated for machines, involves perceptual tasks that are very easy for humans. We aim to crowdsource the data analysis to the general public through a game-like activity, which would drastically speed up the research.
The evaluation stage is over, and we passed. Not only did we do nearly as well as the experts, in some cases we did better.
This means we volunteers have actually been doing scientifically valid work on the research project and have moved it ahead significantly. We haven't been wasting our time - to the contrary, we've been saving the professionals' time by providing them with a large amount of data so they can concentrate on what they alone can do.To be considered as good as experts, the Crowd [us] should reach at least 95% specificity (not catching too many stalls that aren't actually there - false positives) and 95% sensitivity (ability to catch stalls when they are present, not missing any - false negatives). The Crowd beat the target at both of these values. We demonstrated that crowd-based analysis is sufficiently accurate to replace lab-based analysis. In other words, we have now scientifically validated the use of Stall Catchers as a reliable and accurate way to analyze the Alzheimer's data. Indeed, the crowd-based analysis was so good that in a few cases it revealed errors in the original lab-based data to which it was being compared!
In practice this means we can move forward confidently in analyzing Alzheimer's research data without any expert intervention. Of course experts may spot check the data from time to time to make sure we are on track, but we have answered the basic question of "can Stall Catchers replace laboratory analysis" with a resounding YES!
If you'd like to join in, play the Stallcatchers game and add to the body of data, check out this link, register, and start playing!