Broken Windows Investigation Replication Package

This is the replication package used to investigate the Broken Windows Theory of Software Engineering.

The package contains a presentation as well as the source for the whole analysis performed.

Notes

If you have any questions or feel that anything in the replication package is amiss, please open an issue.

1. Most models have been run with moment matching to improve pareto-k diagnostics. We found no consistent errors in the loo estimates and did not include the moment matching in the replication package due its resource intensiveness.
2. Seeds have been used where possible to reduce the effects of randomness for those who wish to execute this replication package themselves. All steps does however not support seeding (especially loo and posterior sampling) and may therefore produce slightly different results on each execution. Seeds were selected as the date of the day they were introduced on.

Sub pages:

• Overview - Provides an overview if the replication package.
• Setup - Data preparation, utility function and library imports.
• Data description - Plots and data descriptions not relating to the outcomes, those are found in “Baysian Models”.
• Models - All our models grouped by which outcome they measure.
  • Logic reuse - Exploration and models related to code reuse.
  • Variable naming - Exploration and models related to variable naming.
  • Sonarqube issues - Exploration and models related to introduced sonarqube issues.
  • Implemented utility methods - Exploration and models related to the implementation of utility methods.
  • Documentation state - Exploration and models related to the state of the validation documentation.
  • Task completion - Exploration and models related to dropouts and task completion.
  • Time to complete task - Exploration and models related to the time outcome.
  • System quality rating - Exploration and models related how participants rathed the quality of the scenario.
  • Self-reported submission quality - Exploration and models related to how participants reported the quality of their own work.
• Environment - Description of the computational environment used to perform the analysis.