New motorcycle safety study: Valid number of crashes to study is unknown

How many crashes make a valid study? (Photo: Uriah Welcome)

http://www.flickr.com/photos/precision/ / CC BY 2.0

How many motorcycle crashes must be studied to yield statistically valid data about motorcycle crashes in general? To judge from recent statements from various quarters, that number would be 1,200, or perhaps 900.

"The true answer to that question, today nobody knows," says Dr. Samir Ahmed, the Oklahoma State University researcher conducting the "Motorcycle Crash Causation Study" that will be funded through a mix of federal government and private dollars.

The question of how many crashes to study for the data to be statistically valid is not academic because the number, and the funding to study that number of crashes, have split the motorcycle community. The Motorcycle Safety Foundation (MSF), which had been the focal point for raising money to match $3 million put up by Congress, has announced that it will not contribute its $2.8 million.

"But with a limited sample size of approximately 300, we believe the study will not provide sufficient statistical significance of the OECD identified study variables and the MSF Board of Trustees has determined that MSF must continue to make its commitment of funds contingent upon a sample size of at least 900 cases," said the MSF in a statement on the matter.

The American Motorcyclist Association (AMA), on the other hand, favors carrying out the study even if only 300 crashes are included.

"This is the first time in 30 years that Congress has offered any money to study motorcycle crash causation, and we don’t know when or if it will ever happen again. We can’t afford to miss this opportunity to gain insights into why people crash, and we can’t pass up a very good opportunity waiting for the theoretically perfect one," said AMA spokesman Peter Terhorst.

Statistical analysis underway

In order to settle the issue and establish a sound basis for the study, Dr. Ahmed and his colleagues are currently conducting a statistical analysis to determine how many crashes will be needed for statistical validity. He said that he expects they will have the answer "in about a month."

Dr. Ahmed explained that 900 crashes were initially presumed to be necessary for statistical validity because that was the number used in the Hurt study conducted 30 years ago. It was presumed that the Hurt study researchers had done statistical analysis to determine the number needed and that was their conclusion.

That presumption was incorrect, he said.

In the meantime, the clock was ticking on the federal funding. If the study had not gotten underway by Oct. 1, 2009, the $3 million would have been returned to the general fund and there would have been no study at all. Consequently, the study was begun with an understanding that current funds might not be enough and if that proves to be the case, said Dr. Ahmed, "We'll keep asking for more."

Once possible source of additional funding Dr. Ahmed mentioned is the U.S. Department of Defense.

"There are lots of Marines who get killed on motorcycles, and those Marines are expensive to train," said Dr. Ahmed. Supporting this study might be one way to reduce the number of those deaths.

Crashes also cost states money in direct costs as well as loss of wages and taxes. Although states may be strapped for cash now, presumably the economy will improve during the four years the study may take, and Dr. Ahmed sees them as possibly being interested in helping to fund the study at some point.

Of course, the biggest boost to study funding would be if the MSF, with a statistically sound target number of crashes to study, were persuaded that its $2.8 million could be used to successfully produce a study that did "provide sufficient statistical significance."

Related articles

New motorcycle safety study in pilot stage
New motorcycle safety study: Valid number of crashes to study is unknown
Some particulars clear, others still unclear in new motorcycle safety study
Motorcycle Safety Foundation open to reconsidering safety study funding
Analysis says valid motorcycle crash data achievable with available funding
Rationale of motorcycle crash study statistical analysis explained
MSF funding for crash study still looks like a no-go
MSF's motorcycle crash study reluctance challenged
MSF, VTTI to conduct 'naturalistic' motorcycle safety study
Motorcycle crash study will proceed without MSF money
MSF President Tim Buche discusses motorcycle safety studies