Reaves.dev

v0.1.0

built using

Phoenix v1.7.12

Design Studies

Stephen M. Reaves

::

2023-09-19

Notes about Lesson 6 of CS-6310

Summary

Overview

Design is about making decisions

Decisions can come from multiple places

Design Study := rigorous and systematic evaluation of the factors that influence a design and consists of a comparison of various possible approaches

Begin with relevant criteria, metrics, and threshold

Design Spaces

Explore a space of possibilities

Factors to examine:

Teaching and Learning

Design can’t be taught, only learned by doing

Experiments

Overall goal of design study is repeatability

Report

Objective

Dispassionate

Multiple sections

  1. Context
  2. Research questions
  3. Subjects
  4. Experimental conditions
  5. Variables
  6. Method
  7. Results
  8. Discussion
  9. Conclusions

Context

Background or motivation for study

Necessary information to understand the study

Research Questions

Examines tradeoffs between various non-functional requirements

Questions should be neutral and numbered

Subject

Subjects are the thing being studied

Studies typically include multiple subjects

Experimental Conditions

A Software design study normally means running several versions of a program, making measurements and evaluating the results

Configuration information should be explicitly documented

Variables

Independent and dependent variables must be identified and appropriate metrics specified

3 column table

Research QuestionIndependentDependent

Method

Number of trials

Measurement devices and tools

Randomization technique

Significant digits

Explicit statement of which subjects will be run and with what arguments

Briefly describe any statistical techniques you will use

Data

The point of a study is to produce data

This section presents the data collected and their statistical analysis

Discussion

Opportunity to interpret the data you collected and provided a discussion of its implications

Reflect on the experiment itself

Conclusions

Summarize your results and draw conclusions

Provide explicit answers to each of the research questions