RUNNING BEHAVIORAL STUDIES WITH HUMAN PARTICIPANTS -FRONT COVER

RUNNING BEHAVIORAL STUDIES WITH HUMAN PARTICIPANTS

BRIEF CONTENTS

DETAILED CONTENTS

LIST OF FIGURES AND TABLES

PREFACE

ACKNOWLEDGMENTS

ABOUT THE AUTHORS

CHAPTER 1: INTRODUCTION

1.1 Background

1.2 Overview of the Research Process

1.3 Overview of the Running Examples

1.4 Further Readings

1.5 Questions

Summary Questions

Thought Questions

CHAPTER 2: PREPARATION FOR RUNNING EXPERIMENTS

2.1 Reading Literature in the Area

2.2 Preparing the Apparatus and Materials, Design,and Procedure Through Piloting

2.2.1 Care, Control, Use, and Maintenance of Apparatus

2.2.2 The Testing Facility

2.2.3 Choice of Dependent Measures: Performance, Time, Actions, Errors, Verbal Protocols, and Other Measures

2.2.4 Plan Data Collection With Analysis in Mind

2.2.5 Run Analyses With Pilot Data

2.2.6 Write About Your Experiment Before Running

2.3 Preparing for Recruiting

2.3.1 Choice of a Term: Participants or Subjects

2.3.2 Recruiting Participants

2.3.3 Subject Pools and Class-Based Participation

2.4 Institutional Review Board (IRB)

2.4.1 What Needs IRB Approval?

2.4.2 Preparing an IRB Submission

2.5 Preparing to Run the Low Vision HCI Study

2.6 Preparing to Run the HRI Study

2.7 Conclusion

2.8 Further Readings

2.9 Questions

Summary Questions

Thought Questions

CHAPTER 3: POTENTIAL ETHICAL PROBLEMS

3.1 Preamble: A Simple Study That Hurt Somebody

3.2 The History and Role of Ethics Reviews

3.3 Recruiting Subjects

3.4 Coercing Subjects

3.5 Risks, Costs, and Benefits of Participation

3.6 Sensitive Data

3.7 Plagiarism

3.8 Fraud

3.9 Conflicts of Interest

3.10 Authorship and Data Ownership

3.11 Interpersonal Conflicts Within a Research Team

3.12 Potential Ethical Problems in the Low Vision HCI Study

3.13 Potential Ethical Problems in the Multilingual Fonts Study

3.14 Conclusion

3.15 Further Readings

3.16 Questions

Summary Questions

Thought Questions

CHAPTER 4: RISKS TO VALIDITY TO AVOID WHILE RUNNING AN EXPERIMENT

4.1 Validity Defined: Internal, External, Construct, and Surface

4.2 Risks to Internal Validity

4.2.1 Power: How Many Participants?

4.2.2 Experimenter Effects

4.2.3 Participant Effects

4.2.4 Demand Characteristics

4.2.5 Equipment and Setup Effects

4.2.6 Randomization and Counterbalancing

4.2.7 Abandoning the Task

4.3 Risks to External Validity

4.3.1 Task Fidelity

4.3.2 Representativeness of Your Sample

4.4 Avoiding Risks in the Multilingual Fonts Study

4.5 Avoiding Risks in the HRI Study

4.6 Conclusion

4.7 Further Readings

4.8 Questions

Summary Questions

Thought Questions

CHAPTER 5: RUNNING A RESEARCH SESSION

5.1 Preparing to Run a Research Session

5.1.1 Preparing the Space for Your Study

5.1.2 Piloting

5.1.3 Experimenter Dress Code

5.1.4 Preparing and Using a Script

5.1.5 Before Subjects Arrive

5.2 Running a Research Session

5.2.1 Welcome

5.2.2 Talking With Subjects

5.2.3 Concluding a Session

5.2.4 Running Simulated Subjects

5.3 Other Issues

5.3.1 Missing Subjects

5.3.2 Other Problems and How to Deal With Them

5.3.3 Chance for Insights

5.4 Running the Low Vision HCI Study

5.5 Running the Multilingual Fonts Study

5.6 Running the HRI Study

5.7 Conclusion

5.8 Further Readings

5.9 Questions

Summary Questions

Thought Questions

CHAPTER 6: CONCLUDING A STUDY

6.1 Data Care, Security, and Privacy

6.2 Data Backup

6.3 Data Analysis

6.3.1 Documenting the Analysis Process

6.3.2 Descriptive and Inferential Statistics

6.3.3 Planned Versus Exploratory Data Analysis

6.3.4 Displaying Your Data

6.4 Communicating Your Results

6.4.1 Research Outlets

6.4.2 The Writing Process

6.5 Concluding the Low Vision HCI Study

6.6 Concluding the Multilingual Fonts Study

6.7 Concluding the HRI Study

6.8 Conclusion

6.9 Further Readings

6.10 Questions

Summary Questions

Thought Questions

AFTERWORD

APPENDIX 1: A CHECKLIST FOR PREPARING STUDIES

APPENDIX 2: EXAMPLE SCRIPTS FOR RUNNING STUDIES

A2.1 A High-Level Script for a Human–Computer Interaction Study

A2.2 A More Detailed Script for a Cognitive Psychology Experiment

APPENDIX 3: EXAMPLE CONSENT FORM

APPENDIX 4: EXAMPLE DEBRIEFING FORM

APPENDIX 5: EXAMPLE INSTITUTIONAL REVIEW BOARD APPLICATION

APPENDIX 6: CONSIDERATIONS WHEN RUNNING A STUDY ONLINE

A6.1 Recruiting Subjects

A6.2 Apparatus

A6.3 Gaming Your Apparatus

A6.4 Further Readings

REFERENCES

INDEX