Rapid iterations and long running programs

So recently I have to run a program that takes a long time to complete. I don’t mean the program is slow. It’s just that whatever it’s doing takes a long time because there is a lot of data to process.

I’m talking days, stretching into almost 2 weeks.

When I was younger (and more naive), I thought pfft there could never be programs that had to run more than an hour. Computers are so fast!

I was wrong.

When I was writing my honours thesis in university, I had to write a computer virus infection simulation program (it forms the basis of my thesis results). I had nodes serving as computers. I put edges in the graph (mathematical term for network) as connections between the “computers”. Then I let my virus logic run amok in the network.

When I found out how long it took to complete, or at least reach a stable infection equilibrium, I was shocked. Dismayed. Dread even. It took slightly over an hour to complete. Sometimes two hours.

So I’d run the program and go for lunch. Then come back and see what my results were. I learnt to make changes and note them down meticulously because re-running the program was expensive, time-wise.

And so it is with my current long-running program. One mistake, or one forgotten consideration, or one miswritten line of code, and I’d have to rerun the program. By that time, I might have wasted a day.

As they say, measure twice cut once.