Stereotypes and Programming

Generally, stereotypes are bad. They’re useful only as a tool to fill in gaps, but most people use it the wrong way, which is they rely *only* on their version of the stereotype.

Also, all software frameworks are stereotypes. Also, the sun was being dramatic.

Teaching classes

I’ve been asked what possible career could I have with a maths degree. The asker usually already have an answer. Be a teacher.

When I graduated from university, it was about a year after Sep 9/11. Finding a job was hard. My friends, driven to desperation, decided to see if teaching was a viable career. I almost wanted to try too, but then I deliberately decided not to be a teacher (at least in Singapore), so I just followed my friends to the application centre.

I spent maybe an hour looking at the architecture of the centre while my friends filled in the application forms. So in Singapore, before you’re approved as a teacher, you had to go study at National Institute of Education (NIE) first. This is on top of whatever degree you’ve already earned. NIE teaches you how to be an effective teacher (we take education seriously here in Singapore).

After NIE, then you get assigned to schools. I don’t know the details, but I believe you don’t get a choice about the assignment.

Out of my university friends, only one actually became a teacher. There was once I mentioned that I forgot the double angle formulas of sine and cosine. She lectured me on that, together with another friend who was a part-time maths tutor.

I have since learned my lesson. (By not saying anything that’s maths-related. What, you think I’d go memorise the double angle formula just in case? Sheesh…)

So it’s a weird turn of events that I contemplated teaching a class of my own. It’s when I realised that classes on professional subjects are different than academic subjects. I’ve gone to a few Visual Studio courses (because they were the only courses I was approved to attend, and upper management pressured me to “Go take some courses dammit Vincent!”. Well, my manager didn’t actually say that, but the tone was there…), and the main difference was that you actually have to get something done.

So now, I’m teaching a programming course. Apparently, there are many tutorials out there teaching you how to do something in an Open XML spreadsheet (and for Word documents), but no one really teaches you how to know what settings to use.

The goal of the course: Given an Open XML spreadsheet (say, as a user requirement), reproduce that spreadsheet (with data changes of course). It’s called “OpenXML Spreadsheet Boot Camp”, and you can find out more here.

Multi-use variables or multiple variables?

So I’ve been working on a software project of mine. I’ll tell you more about it soon enough, but for now, it’s enough to say that I’m writing source code that generates source code.

One thing I’ve noticed is variable declaration. There are 2 extremes.

One variable used multiple times

This is the memory-efficient version. If you need the use of an integer variable, you just declare one variable. For example,

int i;
i = DoSomething() + DoSomethingElse();
DoAlpha(i);
i = DoThis() + DoThat();
DoBeta(i);

That’s just for illustrative purposes. If you’ve written a fair amount of code, I’m sure you can think of better examples. Which are probably (and usually) more elaborate and lengthier.

The drawback to this is that the variable is temporary. As the code continues its execution, previous values stored in that variable are considered to be unimportant to future executions. That’s why the value can be discarded and the variable overwritten.

Multiple variables but one-off use

Then there’s the “declare as many variables as you can (or think you need)” method. For example,

int i1 = DoSomething();
int i2 = DoSomethingElse();
int i3 = DoThis();
int i4 = DoThat();

This has the advantage of keeping the variable values “alive” through that section of code. The drawback is that you use more memory, even if seemingly trivial. I mean, that’s like 12 more bytes of memory (assuming integers still take up 32 bits when you’re reading this). That hardly makes a dent in the computer’s memory space.

The hybrid

The above 2 are extreme cases. What happens when you write code is probably a hybrid, somewhere in between the 2 extremes. For example,

int iSubtotal;
int iTotal;
iSubtotal = DoSomething();
iTotal += iSubtotal;
iSubtotal = DoThis() + DoThat();
DoSomethingElse(iSubtotal);
iTotal += iSubtotal;

You know what you declared those variables for, so you have an idea how many “unique” variables you need. This have the benefits of using the least number of variables (sort of), balanced with keeping the least number of “live” variable values around.

So why am I talking about this?

Auto-generated source code cannot generate hybrids

When you’re writing code, you have one very important advantage: You have context. A program that generates source code, such as a decompiler, does not have that.

When you’re writing code, you make variable decisions such as naming, naming conventions, how many you need and so on.

A decompiler has difficulty making decisions like those, so it has to choose one of the extremes. Typically the multiple variables route, because that’s the safest. All a decompiler can do is detect that a variable is needed, and so writes out the variable declaration in the resulting source code. It cannot decide on whether this part of the code can reuse one of the variables it has already declared (or at least has difficulty doing so).

Ok, so the cat’s out of the bag. I’m writing a decompiler. That’s not exactly true but will suffice for now (I promise I’ll tell you more soon!).

Anyway, that’s what I discovered while working on my software project. I have decided to go the multi-use variable route, because of a human (and programmer) behaviour. A human programmer has difficulty holding on to many separate variables in his head.

When a section of code requires many variables, I tend to try to limit the number of variables I remember in my head. Maybe there’s a pattern. I might remember there’s fFinancialYear1 up to fFinancialYear7. I might decide to refactor the code such that I only need one fFinancialYear floating point variable (assuming the appended numeral makes sense, and not just laziness in naming). I might separate the code section into several sections, so each section has a limited number of variables.

Maybe that’s not how most programmers work, but I find it “friendlier” than having thisIsAnAwesomeClass1 through thisIsAnAwesomeClass20, and I can’t remember which awesome class does which. I tend to work with tighter variable names (where possible and logical), and write code that’s as tight in scope as possible. So the variable values can be discarded, which means I don’t have to keep track of whether that value is still needed, even if the computer doesn’t mind having to keep track of it.

So how do you write your code where variables are concerned?

Colossal computer coding

Jumping fish, lazing cat, right-handed piano keyboard playing and more than 2000 lines of source code written.

You can try to guess what’s the music piece I’m practising on. I’m almost ready… and I’ll have a recording up then.

Modularity in programming guides

I’ve read many programming guide books and tutorials. The one thing I’m looking for is, “I want to do X. What is the code I need to write to do just that?”

Many times, the author of the book or tutorial had mixed in other code or concepts into that. I want to know the simplest way to print “Hello World!”. I don’t want to include any extra libraries that don’t help with that. I don’t want any custom functions that makes printing a string any easier. I just want to print a string, ok?

The point is, the author already knows how to accomplish that task you want to learn. It’s when he gets, I don’t know, bored, that he adds other concepts to make it, I don’t know, interesting.

I’m not looking for the least number of code lines to write that accomplishes that task, although it’s usually that. I’m looking for the lines of code that just do what I want.

Because sometimes, I can’t differentiate the important from the extraneous. I don’t know, that’s why I’m learning, remember? This is especially important when I need to mix and match different concepts. If what I learnt has other extra stuff mixed in, then the resulting code has “more” extra stuff.

It’s like I want to mix X and Y, but got (X + dx) and (Y + dy). And I don’t know which parts are dx or dy.

Some authors make it clear which parts are the actual lines of code to accomplish X. Some authors are great at explaining stuff. I’m saying there are many others who don’t or can’t.

So when I wrote my Open XML spreadsheet programming guide, I made sure each chapter was modular. If not, I had sufficient comments and explanations so the reader knows which parts are the important parts. Each chapter was modelled after a major feature/function in the Excel software. How to style text, how to insert images, how to add more worksheets, that kind of thing. The Excel user mixes and matches those functions, so I want the programmer using the guide to be able to do a similar thing.

I got an email from a programmer who bought my guide that he liked to pick apart code to figure it out. I wrote a few custom functions but only because it made the code more readable. The full source code is given, so the reader is free to pick apart those functions and write his own (to better suit his needs).

I believe this is attributed to Albert Einstein:

Make things as simple as possible, but not simpler.

Be careful of encapsulating too much into just one function call.

Spreadsheet Open XML V2

I was rushing to get this out. The updated version of my programming guide is out! I first launched it on 17 Jan this year, so I was hoping to meet the personal-and-unseen deadline of 17 May, so it’s a nice 4 months interval. Ah well, I’m the only one who cares anyway…

I’ve added loads more content to help you with your Open XML spreadsheet needs. Version 1 was 53 pages. Version 2 is 147 pages. There are a lot of pages with screenshots, but still… 147 pages!

You can find out more here.

The last few weeks had been interesting while I rushed to get working source code and write explanations for the guide… I need to sleep… wait, I’ve got a magazine deadline! *sigh*

First programming product almost done

As I mentioned earlier, I’m preparing a guide to creating Excel files using just code and the Open XML SDK. I’m calling it “Excel Open XML From Scratch” (nice name, huh?). All the source code (C# and VB.NET) had been written and tested. They work! Yay!

So now, I’m writing the accompanying PDF to explain the code and concepts. As mentioned before, I’m targeting a “before Christmas” launch. Now that we’re nearer the date, I can confidently tell you that it’s going to be sometime within the next few days. It will be released on 17 December. This gives you a week before Christmas to get this as a present for yourself or a programmer friend.

The price will be set at USD 17 (see actual price on product page), which I believe is a fair price considering that it will save you hours of work. Think about how much you’re paid per hour. Probably more than the price of this product. Hmm… I might even have to increase the price…

This is my first programming-related product, so I’m really excited by this. If you don’t really care about C#, VB.NET, Open XML or Excel, then I apologise. Just ignore any related posts for the immediate couple of weeks.

Is math important to programming?

It depends.

Now that we’ve gotten the short answer out of the way, let’s discuss this further. Recently, Jeff Atwood asked Should Competent Programmers be “Mathematically Inclined”? I don’t get the need for the quotes, but never mind…

The summary of his article is that the problems involving math that most programmers deal with are the “balancing your checkbook” kind. Meaning simple math is required, meaning you don’t need to be a math whiz to solve that problem with programming. And I agree with him.

My work currently puts me in contact with a lot of numbers. As in quantities in the 10 digits range, with values also in the 10 digits range. And the finance and credit departments of my employer are very interested in those numbers…

Anyway, the manipulation of those numbers require very little math, despite their math origins. How hard can it be to sum up figures, do discrete prorating, calculate percentages or find out the price based on existing rates?

The initial reading of Jeff’s article made me feel indignant (goes off to check that I indeed write about math and programming). How dare he view math as frivolous to programming! How dare he reduce math to insignificance! (wipes spittle from mouth)

Then the cold hard truth of my professional programming experience knocked my senses back into place. I had to remember that the work I did back in university just wasn’t… useful (directly) to my present work. I worked with 3D, image rotations and optimising an airflow simulation. Not quite the business related programming I do now.

I still believe math is useful to programmers. Just as you should learn C, even if you don’t use C in your regular work. I’ll tell you how math and C are related in a bit, but first…

Math is elegant

I agree with Steve Yegge. They teach math all wrong in schools. This is despite his American background and my Singaporean background. I guess errant math education has no borders

The earlier parts of math education focus on calculations and memorisations. Oh, there are some proving questions (much to my distaste) alright, but there’s not a lot of them. In university, my math education started to take on a distinct shift; there were fewer numbers in the questions.

I remember there was this linear algebra tutorial. There were only 4 questions in it. I took a grand total of 4 hours to complete it. Even then, I didn’t fully answer the questions. It was proving this, or ransacking notes to find out which theorems were applicable, or wandering into higher dimensional linear spaces (I think I hated the professor at one point during my struggle). I think I even skipped 1 (or 2!) question. That’s how different the questions became.

With this shift, I found math to become more … elegant. Suddenly, plugging in numbers into calculation formulas aren’t important anymore. Even remembering proofs and theorems take backstage. Figuring out which method, proof or theorem to use to solve a question in the simplest manner is paramount.

I started to solve problems elegantly, be they math or programming.

And for me to find elegant solutions, I needed to think more. Much more. Sometimes this fails, and I end up with a less-than-ideal-elegance solution. But that’s ok. If I don’t aim for elegance, I’ll never reach it.

In my earlier stages of education, math was binary. Either my solution was right, or it was wrong. Later on, it was right. And there’s another solution that’s also right, and shorter. Or easier to understand. Math solutions became a little fuzzy.

What I’m saying is that although math isn’t directly useful to my programming, it certainly shaped the way I solve programming problems. Because programming solutions are also a little fuzzy.

Many programming solutions are sub-optimal. And they don’t need to be optimal. They already solve all normal occurring cases, and maybe the edge cases don’t matter that much. Or maybe there isn’t an optimal solution. In which case trying to find the optimal solution is a waste of time. So much for elegance.

So is math really important? I can’t say for you (despite writing an article on it), but C isn’t that important for programmers either.

Learning C makes you think

It’s sometimes hilarious to see Jeff and Joel argue about the importance of learning C. Joel does have a point. Learning C makes you think harder about solving programming problems (pointers and all). It doesn’t mean you’ll ever use any of the solutions (or C for that matter), but it trains your mind to think.

And in this respect, that’s what C and math are to a programmer. Learning both makes you think. You’ll think about just solving the problem. You’ll think about a more elegant solution. Maybe coming up with a less convoluted solution but eminently understandable by your fellow programmers.

But if you never go through the extremes of “slap messy but amazingly it works” and “elegant one liner but takes forever to understand”, then you might find it hard to find a happy in-between state. Because you just don’t know what’s possible.

“So is math important to programming?” an obvious exasperation in your question.

Well … it depends. On you.

A typical month work load

Climber by Bettina Ritter
[image by Bettina Ritter]

I don’t really have a typical work day. It doesn’t mean I have an exciting job. It just means I can’t tell you what I’ll be doing the next day, because I don’t know for sure what I’ll be doing. What I can tell you is what I do in a typical month, generally speaking. There’s a point to all this, and I’ll start with…

From 8:30 am till 6 pm

Those are my work hours. Except Friday. On Fridays, I get to go off at 5:30 pm. My current job title is IT Analyst, changed from Systems Analyst. And if you think that’s vague, you’re right. My job scope is quite varied. Basically, my contractual terms require me to “do whatever the boss tells you to do”.

The current company I’m working for isn’t a software company. I just work in the IT department. What it means is, programming isn’t as highly regarded as I want, as what I read about in those programming blogs and sites. It kinda sucks, but it keeps me fed.

Let me tell you about the tools I use at work. I’m the “online guy”, which means any user interface related development comes to me. I use Visual Studio 2003 (C# and VB.NET) for all the web applications, console programs and a few software tools I create to help me. I also use Visual Studio 2005 for one particular application, with a graphical user interface. It’s too tedious to explain why I use both versions. It’s enough to know that I do.

I also use PowerBuilder for some Windows applications. It’s really, really painful to work with PowerBuilder code. I tell myself it’s the previous programmer’s skill that’s to blame, not the language, but I frequently fail. Tracing and debugging PowerBuilder code takes a lot of work for me. I really hate PowerBuilder… I think this calls for a separate rant article.

I’ve been asked to investigate C and C++ code on Unix machines too. So yes, I understand make files, shell scripts and cron jobs. I even know how to use the vi editor! I used to telnet to the Unix machines with TerraTerm, which is now abandoned for a more secure client application. Can’t remember the name because I rarely use it, because I rarely need to telnet.

Database admin, server admin and LAN admin

Despite the fact that I’m completely ignorant of SQL and databases in my formal education, I’m thrust full force into it at work. I’ve worked with the Oracle, SQL Server and Sybase databases, know most of the nuances between them on SQL syntax, and understand how to use stored procedures. I handle them all.

I am also completely in charge of a few Windows servers and the SQL Server databases running on them. Server maintenance, backup schedule and tapes, security patches, SSL certificates, IIS configuration, server performance.

Then I’ve got to know about the opening of ports for security purposes, who to notify when there are application or server changes. I need to know ping and tracert and ipconfig and other network related stuff.

All of that maintenance and administration is on top of my development work.

I don’t need to connect to Oracle databases now, but I used to do so with TOAD. There’s a limit to the number of licenses, so I wrote my own database connector program. It only does retrieval of data, basically the select statement, but it’s enough for current tasks. The Oracle databases belong to another team, and they’ve only needed me to help out rarely.

I use the Enterprise Manager and Query Analyzer for the SQL Servers. They’re great tools, and they come with the database installation, which is cool. There’s also another tool that has saved me many times. It’s the DTS, Data Transformation Services. I’ve used it to transfer data interchangeably between Oracle, SQL Server, Sybase and get this, Excel. Users take to Excel much better, so I need to use their form of “database”.

Designers, comparers and reflectors

I’m also a web designer. I suck at it, but I’ve been lucky enough to muddle through, and my users and their customers think my user interface looks awesome. I use Paint.NET (and sometimes the inbuilt Windows Paint program) for my image editing tasks. Plus I’ve got some colour tricks up my sleeve.

Some time ago, I had to verify some old code by another programmer. He can’t remember what he changed, and I obviously don’t know what could possibly be changed. I needed help! Fortunately, I found CSDiff. It allows you to compare two files (or even folders) and lists down differences between them. Much better than checking line after line of code by inspection.

And if you do .NET work, you must get the Reflector by Lutz Roeder, which had been taken over by Red Gate Software. It allows you to get back code from compiled .NET DLLs and programs. The result might not be the prettiest code, but with sufficient talent and patience, you can get something out of it.

I’ve used it on my own code and other team members’ code to check for disparity. Sometimes, you forget which version you’ve compiled that code into… Sometimes, it’s for self study, to understand what others have done.

The phone calls. Oh the phone calls.

My phone rings a lot. There are over 10 people in my immediate vicinity. I can tell you that, if you add up all the phone calls all of them ever receive in a month, it would still be less than what I alone receive in that same month.

Remember I told you I’m the “online guy”? That means a lot of users know me, and I don’t know all of them. Since they usually interact with the application interface, any problem is routed to me. Whether it’s data inconsistency, business logic query, application error or failure, all of them come to me. I’m a one-man helpdesk I tell ya.

It was so bad that sometimes, I’ve had to solve and handle user queries for entire days on end. Due to the nature of my work, the start and end of the month are particularly busy for my team. The number of times my phone rings goes through the roof. Maintaining decent phone etiquette starts to be a strain…

Wait, there’s something missing…

Where’s the source control software I’m supposed to be using? Well, I’m the source control. My team is very small in size. Company directives dictate we send work to our offshore colleagues. I think those (typically recent graduate) colleagues have some problems of their own, let alone set up a source control system that works across geographic boundaries.

I’ve not been with development teams at other IT departments, but I think we would totally fail at the Joel Test. Totally.

Despite these circumstances, I still manage to do development work, sometimes with surprising and outstanding results. I believe good task management is crucial to my balancing act. Which brings me to…

Holistic approach to programming

If you’re working at a software company, or on something focused on software and programming, I envy you. I really do. You’d probably get to talk with other programmers on interesting topics. Your work is really appreciated, because it goes to the bottom line.

I might not be programming exclusively, but I get to see the bigger picture. I get to liaise with people from sales, marketing and customer service. I get to talk with upper management and even the actual customers. I get to see the kinds of products and services offered, and how it’s implemented and supported by software.

Programming is kind of … an elite thing. When I was studying C programming in university, I was surprised that many of my fellow students struggled with it. I took to it like a fish in water. After a while, I realised that most people cannot grasp the thinking required in programming, even if they opted to study it themselves.

So I’m going to state this. Many people are not going to understand how great that piece of code you’ve written. Many people think software can make their lives easier, but fail to realise that not everyone can write good software.

This is where all your other skills come in. You have to sell what you’re doing to other people. Convince them that it’s useful, that it’s awesome, that it’s relevant, that what you do and what you propose is important.

Sell your ideas. Market your ideas. Your software is more useful if you see it from a bigger-picture point of view, from other people’s point of view. That requires you to understand other concepts. Concepts that aren’t related to programming at all. And you synthesise them together to make your code better.

And that, is my point.

Reverting to base nature

This is an article on observation and human behaviour. Faced with a situation outside a person’s comfort zone, how will that person react? Can that person’s reactions tell you anything about his base nature?

Handling hot tea

Coffee guy by archives @ iStockphoto

I read this in a detective comic book. In one of the episodes, there’s a senior detective giving tips to the protagonist, Q (yes, that’s his name). So the senior detective went about his activities while engaging Q in conversation. He casually prepared two cups of tea*, and served one to Q.

After Q drank his tea, the senior detective smiled and told Q that he had just given Q a quick test. He had deduced two pieces of information

  • Q was right-handed
  • Q was in a state of calm

When a person is (suddenly/unexpectedly/casually) given a cup of hot drink, in order to carefully handle the cup, the person uses the dominant hand (usually). Q had used his right hand to grab the handle.

When a person is sipping a cup of hot drink, to test the temperature and prevent scalding, the person usually relaxes his facial expression, and shows his real expression. Because of the focus on the hot drink, any false expressions used to mask his feelings disappear.

For example, a person is smiling at you. When that person sips a hot drink, and suddenly furrows his eyebrows and his lips sag a little downward, he may be worried about something, but puts on a happier front.

Because of sipping a hot drink, a person revealed his base nature.

* can’t remember if it’s tea or coffee.
And I don’t know if this is true. I haven’t conducted nearly enough observations to conclude…

The truth trailing tough times

I’ve read and heard that one of the ways to determine if that special someone really fits you, is to go travelling with that person. Or go hiking, or camping, or any activity where both persons are under moderate stress.

Suddenly, one finds out how the other person handles flight delays, inconsiderate hikers, missing toothbrushes, a scratch, a cramp, and decisions affecting both persons. During tough times, a person’s base nature shows up.

Flight delays become “How about that cup of coffee?”, and cramps become “Well, at least the view here is beautiful. Ouch, ouch…”

Or missing toothbrushes become “How could you forget that? I distinctly remember telling you to bring it!”, and decisions affecting both persons become a one-sided dictatorship (Amazing Race had plenty of those).

Coding under stress

What happens when you code under deadlines, under new and unheard of requirements, under strange and different environments? You revert to your base nature.

If you have bad coding practices, then under stress, those bad coding practices will show. If you’re lazy about debugging, then under stress, your programs are going to be chock full of bugs. If you’re plain terrible at programming, and had always relied on friends and colleagues, then under stress, well, somebody’s going to notice it.

For example, copying and pasting. I’m sure you had copied and pasted similar pieces of code, never mind the rule of reducing redundant code. This is where the base nature of a programmer shows. The disciplined programmer would copy and paste that code, then quickly go through each line to make sure it’s appropriate to the context (variable names, conditional checks and so on). The careless programmer would just give it a cursory check if it compiled.

The shortcut of manually typing out each line was taken to fulfill our innate programmer laziness. Yet it’s the understanding of the code copied and the actions taken after pasting, that distinguishes the better programmer.

Improving your base nature

Practise your desired qualities when in a calm state. Practise the use of printfs, Console.WriteLines, MessageBoxes or whatever can be used to display variable values. Practise spotting errors before your compiler does. Practise the use of your programming language constructs (how to loop, how to control decision branches) in a non-critical program under a non-critical state.

When your desired qualities become second nature to you, it’s still not enough. Because in the face of adversity, that second nature might still fall off. Practise until it becomes your base nature.

Then reverting to your base nature is ok, because your base nature is already great.