Full Content is available to subscribers

Subscribe/Learn More  >

A Scalpel Not a Sword: On the Role of Statistical Tests in Design Cognition

[+] Author Affiliations
Mark Fuge

University of Maryland, College Park, MD

Paper No. DETC2015-46840, pp. V007T06A008; 11 pages
  • ASME 2015 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
  • Volume 7: 27th International Conference on Design Theory and Methodology
  • Boston, Massachusetts, USA, August 2–5, 2015
  • Conference Sponsors: Design Engineering Division, Computers and Information in Engineering Division
  • ISBN: 978-0-7918-5717-5
  • Copyright © 2015 by ASME


The number of design studies using statistical testing has increased dramatically over the past decade. While this has benefits, statistical testing requires scrutiny to protect against common errors and misconceptions. To illuminate how these issues affect design, this paper provides a comprehensive analysis of the past decade of studies within the DTM community. Specifically, the paper 1) reviews the background of statistical testing across multiple fields, highlighting recommended practices, 2) discusses its use in the Design community, and 3) provides concrete methods for authors and reviewers to evaluate statistical tests employed in Design Cognition studies.

The analysis identifies recurring issues with: ignoring multiple comparisons; deficiencies in study and result reporting; inadequate defense of modeling assumptions; unavailable plots, data, and analysis files for replication; and lack of interpretation of statistical results with respect to practical outcomes or alternate forms of scientific inquiry. Based upon practices already adopted in other research communities, we put forth: 1) checklists that help authors and reviewers verify data reporting, analysis, and statistical assumptions; and 2) design guidelines for creating more reproducible design experiments. Ultimately, we argue that design researchers, reviewers, and editors should view statistical testing less like a sword and more like a scalpel — a specialized tool best used in concert with other techniques — to gain a more complete picture of Design Cognition.

Copyright © 2015 by ASME
Topics: Design



Interactive Graphics


Country-Specific Mortality and Growth Failure in Infancy and Yound Children and Association With Material Stature

Use interactive graphics and maps to view and sort country-specific infant and early dhildhood mortality and growth failure data and their association with maternal

Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In