Opening the ‘Black Box’ of Simulations: Transparency of Simulation Models and Effective Results Reports Through the Systematic Design of Experiments
Hamburg University of Technology (TUHH) , Research Paper Series No. 001
39 Pages Posted: 9 Jun 2011 Last revised: 25 Jan 2015
Date Written: May 1, 2011
Many still view simulation models as a black box. This paper argues that perceptions could change if the systematic design of experiments (DOE) for simulation research was fully realized. DOE can increase (1) the transparency of simulation model behavior and (2) the effectiveness of reporting simulation results. Based on DOE principles, we develop a systematic procedure to guide the analysis of simulation models as well as concrete templates for sharing the results. A simulation model investigating the performance of learning algorithms in an economic mechanism design context illustrates our suggestions. Overall, the proposed systematic procedure for applying DOE principles complements current initiatives for a more standardized simulation research process.
Keywords: communication, design of experiments, simulation, standards, transparency
Suggested Citation: Suggested Citation