Birthdays and paradoxes

I was sitting down, thinking of an amusing article to write, and I didn’t quite feel up to writing. It’s been a long weekend, and besides, it’s a special day for me today.

32 years old

So I thought, “What the heck”.

Then I thought I should at least leave you with something. So I’ll introduce you to the concept of the birthday paradox. Basically, given a group of 23 people, what’s the percentage chance that any 2 persons in that group share the same birthday? (hint: it’s higher than you think. Here’s a less clinical explanation of the paradox than the one from Wikipedia.).

How long does your program live? Part 1

Have you considered the lifespan of your programs? You might have thought of scaling issues, security concerns and hardware settings. What if your program is still used 5 years down the road? 10 years? 20? Will it still work?

I had the occasion to work on a system. The thing was, it dealt with financial years. In case you’re wondering, a financial year (typically?) starts on April to the following March. So financial year 2009/2010 starts from April 2009 to March 2010. Calendar years are the “normal” ones, starting from January to December of that year.

Where was I? Oh right, the system with the financial years. So the Unix shell script that runs as an interface displayed up to the latest 5 financial years. There’s an accompanying set of C programs, as well as a corresponding set of database tables. It’s not well thought of, but it’s the best under the circumstances.

What happened was, every time a new financial year arrived, a new set of database tables was created (appropriately named of course). And a new set of C programs copied and pasted and edited (to insert, update and delete from the new database tables). And new lines of code added to that Unix shell script. This happens in late February and March, to prepare for the imminent arrival of April (April has more clout than August, even though August is well respected…)

So I did that for one year. Then the next year arrived, and I found myself with 7 financial years worth of program options, but only needed to display 5. The manager and senior colleague were already set to go for the path of least resistance: doing whatever worked last year. I decided otherwise.

I went through the code for the shell scripts, database scripts, stored procedures and C programs. The code were all flexible in that only the financial year portion was changed. So I told them that I’m going to change all the code so that the code took in the financial year as a parameter, or that it automatically calculated the correct financial years. The manager gave me the go ahead, and I went to work quickly, for it was February, and it’s unknown if I could get it to work in time.

There were some problems, and I’ll tell you about them in the 2nd part of this exciting mini-series…

Our future – The Sims and The Matrix

I was wondering, what would happen if we run out of resources on Earth? We’d probably all die, which is kind of depressing. So I thought of how not to run out of resources in the first place.

Earth plant
[image by Andrejs Zemdega]

I have 2 obvious answers. One is regeneration of our resources, through recycling, reusing and reducing. That doesn’t appear to be enough, since we consume too much.

The second answer is intergalactic colonisation. “We can’t control our own planet’s resources, so we’ll just take over another planet!” Even if we disregard the moral and selfish motivation of that reason, there’s still one tiny problem. Our scientific space program isn’t up to par yet. We might run out of resources before our science and technology is advanced enough.

So I came up with another answer. Now, if you assume that humans are driven by needs and wants, then… ok, let me define needs and wants first. Needs are critical to human survival, or at least make it easier to survive. Wants are wishes… let me give examples then. You need food, shelter and clothing (to protect from weather). You want to travel, to play video games and to wear Gucci products.

The distinction between a need and want is subjective (does steak qualify as a need or want?). We don’t need the distinction to be obvious anyway. So what’s the answer? Push the wants into a virtual world.

Create a virtual world where the wants can be satisfied by bits and bytes rather than carbon, wool, oil, water and other natural resources.

For it to work, the experience must be indistinguishable from what can be experienced now. Hence The Matrix. What we call reality, is just a sum total of what our senses tell us anyway. Simulate the senses correctly, and you get “reality”. The game franchise “The Sims” has been steadily heading towards this end. Its limits are the number of objects available (including other Sims), the number of interactions available and the number of sensory inputs.

With the wants satisfied in the virtual world, there should be more resources in the real world. If nothing else, this should give the option of regeneration more time to regenerate resources. Or more time to spread our human genes to other galaxies.

Is the virtual world solution easier to implement (we programmers have much to do then)? Will it upset the economy (most definitely)? Will the virtual world consume more resources (servers have carbon footprints too)?

If you miss, little else matters

I’ve played 2 more game sessions since I last failed at rolling dice. I’m starting to get the hang of playing Dungeons and Dragons. In case you’re not a D&D person, I’ll keep most of the deep references out.

Here’s my general observation: It’s actually very easy to miss. Generally speaking, in a neutral setting, where there are no bonuses to hit, the percentage to hit range from 20% to 40%. Let me give you an example.

Suppose our hero fighter is engaged in battle with a kobold minion. He swings his sword at the kobold as a melee attack. In DnD terms, this means to roll 15 or higher on a d20, 15 being the kobold’s armour class (or AC). In mathematical terms, that’s a 6 out of 20 chance of hitting, or 30% chance.

And most of the hit rolls are like this. The most one could hope for, was to need to roll only an 11 or higher on a d20. That’s a slightly less than 50% chance to hit. And that’s the best case scenario.

My experience with video game role playing games was that, you issue an attack and it hits. Usually. Most of the time. I’ve never found the use of support skills or spells significant. I would cast a protection spell to reduce the amount of damage, and that would be the extent.

So I was in a bit of a quandary when playing Dungeons and Dragons. I like magical stuff. The characters wielding magic are controller types, meaning they can deal damage to multiple enemies but do less damage. They are supposed to slow the enemies, daze them, teleport them, immobilise them, lower their defenses, increase their vulnerability and so on. Basically supportive skills.

Now for my previous game, I was offered help in creating a character. So I took it, and let the helper create whatever he deemed fit. And I got a character geared towards those supportive skills. I thought, “Interesting. I thought boosting damage might be better. Oh well.”

And I truly saw the error of my ways for that game. The enemies had super high defenses. The miss rate was like 70% or 80%. Until the bonuses started to stack.

We had 6 players (which was large), and we buffed the 1 or 2 players with high damage. Because if we didn’t, we would never get any of the enemies killed, because we would never hit them. A +1 to your next attack roll, a -2 to the enemy’s defense, a +2 because of that power, a +2 because I used mine.

As for me, I shone at one particular part, where a few enemies bunched up together, and my spell disintegrated them in one shot. Ok, maybe not all of them, but it cleared most of them. Because it allowed the other players to concentrate on that demon with tons of hit points.

And for the last battle, the buffing really helped. There was this ranger, who could hit using two weapons with one of his skills. After we buffed him, he was able to hit with anything better than 1 (because a 1 was an automatic miss). He hit, and started rolling dice for damage. I believe he needed to roll 6 d10’s (and an additional d6 or d8 because he had a critical hit). After stacking all his damage bonuses, he dealt 73 points of damage. He simply needed to hit.

So from the few games I played, I finally realised the power of team work in DnD. As the characters I favoured, I was to harass the enemies, and disrupt them from harassing my team members. My job was to make it harder for the enemy, and make it easier for us to destroy them. My supportive skills are crucial to this.

Because if my team members miss the enemy, little else matters.

Sort algorithm choice is immaterial for small numbers

I was debating with myself whether I should do a time comparison for the sort algorithms. Then I decided other people had already explained it much better than I could (and the real reason was that I was too lazy to code it and measure it. But you didn’t hear it from me…). So I’ll tell you a story instead.

Manager at desk
[image by diane39]

It was a few years back. I was just a rookie programmer back then. My manager didn’t understand why his superiors hired me. I didn’t even have a proper degree in computer science. He seemed to think my degree in applied mathematics and computational science was irrelevant. In fact, I think he’s afraid of me on some level.

Every day, I walked into my cubicle, sat in my chair, and waited for the phone to ring off its hook. It was a boring existence, doing small maintenance code changes to the C source code, helping users with their queries and basically making sure the System continues to stand on its rickety feet.

I was too “lowly” to be given work in the frontend, the web programming parts, the “high value” work.

One senior colleague, relieved of the maintenance duties by me, got the chance to work on higher level design stuff. A new project arrived, Highly Perceptible Project Opportunity or HiPPO for short. The manager got 3 more new staff to help out. The backend programs would be done by one of the new staff and the senior colleague (since the business processing logic was entrenched there), and the frontend web application would be handled by the rest of the team.

Me? I was given the task of making sure the System continues to stand on its rickety feet. I was considered too “lowly” to handle high level design work or web application development, yet too senior and knowledgeable about the System to remove me from the System.

One day, while I was looking at an email trying to figure out what the user wanted, one of the web application developers came over to me. Let me repeat that. One of the web application developers came into my cubicle to talk to me! I barely even get to see anyone from the HiPPO team.

Anyway, Tyler (as I’ll call him) asked me if I knew anything about sorting. In particular, how to implement a sorting algorithm. What? I went over to Tyler’s computer (wow, I’ve never been in that area…) and took a look at his code, and what he needed to implement. I told him to send me the source code, and I’ll try something out first before getting back to him.

I went back to my desk, elated that I had something to work on other than update statements and Excel spreadsheets. Tyler sent me the source code (we didn’t have source control, and even if we had, I doubt I’d be given access), and I tried sorting using bubble sort.

After testing the results, the bubble sort implementation worked fine for the requirements, so I sent the code file back to Tyler. Tyler’s happy that the sort worked, and I went back to my boring existence.

The next day, my manager called me to go into his office. I went in, and he gestured me to sit down. He looked at me for a few seconds, a furrow forming between his eyebrows.

“Vincent, I heard from Tyler from this morning’s HiPPO project update that you helped him with his sorting implementation.”

“Yes.”

He looked at me wordlessly for another few seconds.

“Is it fast? Have you looked at other sorting algorithms? There’s quicksort and heapsort I believe.”

My manager had obviously done his homework.

“Yes, I know of the other algorithms. I have chosen bubble sort because it’s easy to implement, and based on the requirements, it’s also fast enough.”

“I’m concerned about the speed. It has to be fast.”

“I’ve already tested with the higher ranges of the number of expected records. The performance is good. Tyler had also tested with the records from the test database, and it works fine.”

“But I’m worried about the user finding the web application slow.”

Now it’s my turn to look at him for a few seconds. I took a deep breath.

“We only need to sort maybe 3 or 4 records. Bubble sort is fast enough.”

[Story had been unbelievably distorted and exaggerated to make it interesting.]

Are they recruiting secret agents?

So I told you about the time when the security guard stopped me because I was too hot. I went for an interview, and it ended with stepping into the company building. This is where the story continues…

[What follows had been highly exaggerated for entertainment purposes. About 72.59% of it was created out of thin air.]

Bright lobby
[image by Freezingtime]

The gigantic lobby was brightly lit. The company representative had said nothing other than “Follow me” since I passed the security heat scan. He moved purposefully to the lifts and pressed a button.

I noticed the receptionist was staring at me. Not with adoration, sadly, but with an ever so slight frown on her face. The lift doors opened, and she went back to looking at her computer screen. The representative had already went into the lift, and I moved in quickly.

He pressed the button “B3”. We’re going below ground? He faced the button panel in a stiff manner, and I didn’t dare to engage him in conversation.

The doors opened, and I saw a long hallway stretching away from me. He exited the lift and I followed suit.

*Glung glong glung glong* The sound of our footsteps echoed emptily in the hallway. The tiled floor was clinically white. The walls were a uniform washed out grey, devoid of decorations.

I was half-expecting a door on the side to break the monotony when the representative stopped. A glass door was in front of us! There was a security panel on the right.

“Please move to behind me.” It wasn’t a request.

I practically leapt behind him, for fear of lasers sweeping the hallway. But the curiosity in me won. I took a quick peek at what he’s doing at the security panel. I mean, it’s not everyday you see retinal and fingerprint scans in action. But he’s just entering a few digits on the security panel. Oh, he just didn’t want me seeing him entering his passcode.

The glass doors opened with an almost imperceptible swoosh. I waited for him to take the lead, but he stayed where he was.

“This is as far as I can take you.”

I stared at him incredulously. “Uh, how do I get to where I’m supposed to go then?”

“Go down the hallway. When you reach the fork, turn right. There’ll be a room at the end. Wait there. Your interviewer will be a bit late.”

I almost wanted to ask him how was I supposed to return to civilisation when he turned abruptly and walked swiftly back to the lift. I went past the glass doors and they swooshed shut. My throat constricted a bit.

Following his instructions, I found the door leading to the room he mentioned. My hands shook while I turned the door knob. I entered the room.

It was a fairly large room. There was another door on the other side of the room. The lighting was uneven, where the brightest part was on a corner. Coincidentally, that’s where a table and two chairs reside (the only pieces of furniture in the room). A ceiling light flickered at the darker regions, momentarily allowing me to see… nothing there. The walls were the same pale grey, and even the light switches blended into the walls. The only thing missing in the room was a polygraph near the table. I swallowed.

There was a sign on the wall, near the table. I moved closer to read it, happy to see any form of decoration. It was a “No camera phones allowed” sign. My heart jumped, and I instinctively touched my pockets where my camera phone rested.

“I am so dead.”

After waiting for ten minutes, I decided they wouldn’t kill me if I sat down first. I placed my briefcase against the leg of the table, and mentally rehearsed what I was going to say. I had to tell the interviewer about my camera phone, of course. Checking that I had extra copies of my resume and academic accomplishments did nothing to calm me.

I jumped when a door opened. The interviewer came in from the other side of the room. “Sit down,” he said, a smile on his face. “I’m sorry I was late. Held up by a meeting.”

“I have to tell you this. I have a camera phone, but I’ve already switch it off.”
“Oh. It’s ok.”

He proceeded with what I felt were standard interview protocol. I was still in shock, when he threw an unexpected “I have a grandfather’s clock and it’s faster than my watch. Should I shorten or lengthen the pendulum?” I stared at him for a few seconds before answering. He must have sensed my confusion, because he said, “Oh it’s not part of the interview. I’m just curious.”

Then he showed me a piece of code on string concatenation and asked me how to make it faster. Then he showed me another piece of code, and asked about the rounding errors of the monetary calculations (oh I know that one!).

Finally, it ended. He told me he had to attend to something, and he couldn’t take me back (to the surface). But it should be easy that I retrace my steps back to the glass doors. Someone would pick me up from there. He exited via the same door he came in. I packed my stuff and left the room too.

Retracing my steps, I found the glass doors. And the representative waiting for me. How did he know when my interview would finish? As I walked towards the glass doors, he entered his passcode, and the doors slid open.

He took me wordlessly through the (endless) hallway, back to the lift, back to ground floor, past the (cute) receptionist, and out into the afternoon sun. “Thank you for your time, and we’ll be in touch.” was all he said.

I’m just glad I’m still alive.

If you try exceptional code, make sure you catch the exceptions

Recently, an application failed on me. And I didn’t receive any notification. I only knew it failed because the next hour, I received a notification that it couldn’t start. Let me tell you more about the application first.

The application was supposed to run every hour. At the start, it would update a datetime column in the database. At the end, it would update another datetime column in the database. On start, before continuing with the bulk of its operations, the application checked that the start date was earlier than the end date (from the previous operations). This way, it won’t accidentally run once more while presumably another copy of itself was running.

Anyway, if it did happen to hit that start-date-later-than-end-date condition, then it failed. And a notification would be sent to me. The details were logged in a log file. Due to some complexities of its implementation, I had to check the log file of the earlier run to get the details.

And I found nothing. I mean the log file obviously wasn’t complete. But the error message wasn’t there. I was perplexed. I remembered there was a giant try-catch clause in the program, so I should at least see something. Can you see the flaw?

try
{
    // do stuff here
}
catch (SqlException se)
{
    log.WriteLog(se.ToString());
}
catch (OleDbException oe)
{
    log.WriteLog(oe.ToString());
}

What if the exception wasn’t database-related? And that’s exactly what happened. There was an input file, and it wasn’t properly formatted, thus creating all sorts of parsing errors. It was an IO problem. Since the action taken was the same in exceptional cases, I rewrote the above to this:

try
{
    // do stuff here
}
catch (Exception e)
{
    log.WriteLog(e.ToString());
}

If you’re going to catch specific exceptions, make sure you catch all of them.