Simulating a cost estimate can be used to get an understanding of the potential range of outcomes. Cost estimates by their nature contain uncertainty and lend themselves well to simulation. In this article we'll look at how to simulate an estimate and some items that need to be considered.
Estimating costs for simulation comes in two flavors: expert estimates used to apply a probability distribution or using historical data to fit a distribution. We'll look at an example development project where we need to estimate the project cost. The project has three primary costs:
- Engineering and project management cost.
- Test and prototype construction labor.
- Material/component cost to build prototypes and test units.
It's not quite as simple as assigning probability distributions to random variables and simulating the cost estimate model as we'll see later. To start, we'll create a simplified estimate that treats each of the three cost factors as a one random variable for each.
As you can see with this model, engineering and project management time is lumped together as a single random variable. If we simulate this basic model for 25,000 iterations we get the following result.
Now, let's say we want to get more granular in our estimate of engineering/pm time. We'll break down the estimate into each discipline and apply a random variable to each. Then we'll sum the random variables to get a total for engineering/pm.
Again, if we simulate this detailed model for 25,000 iterations, we get the following result.
Comparing the Two Models
The box and whisker plot below provides a comparison of the two simulations. Note the wider spread for the basic model. Clearly there are differences between the approaches. This difference can be explained by correlation among the various disciplines. The basic model lumps all times together and basically shows perfect correlation between the time for each discipline.
The detailed model assumes each discipline's time is independent of all other disciplines. Mechanical may be low while software may be high, thus cancelling the extremes out. This results in a tighter range of outcomes.
Adding Correlation to the Detailed Model
We can show the effects of correlation in the detailed model by introducing correlation into the model during simulation. The detailed model is shown again, but this time we've added a correlation matrix that will be used during simulation.
To show the effect of correlation, we'll run two simulations. The first will have a Spearman Rank correlation coefficient of 0.5 between all disciplines. The second will have a correlation coefficient of 0.999 between all disciplines. 0.999 was used instead of 1 to avoid an error during simulation. We'll add these simulation results to the box and whisker plot for comparison.
As correlation increases, the spread of outcomes also increase. Note that 0.999 correlation is almost identical to the basic model.
Which Way to Proceed?
This is a tricky question, where judgement comes into play. It's certainly conceivable that there may be correlation between disciplines. If the mechanical engineers require a redesign, this can affect electrical and software as well. Not to mention additional project management work.
The main problem is there is likely no data that would allow you to determine the correlation between disciplines. You either need to guess, or go the conservative route and lump the times together or use very high correlation in a detailed model.
In this article we've kept it simple and only focused on engineering/pm times. There could also be correlation between engineering/pm and the other two cost factors. Extensive redesign may require more prototypes and more testing.