Financial reports must be untouched by human hands

People handling billing or financial data are usually very uptight. Especially when it comes to numbers. If you thought mathematicians, statisticians or economists were protective of their numerical figures, go talk to someone who works in the financial department.

PI is an elegant number. It goes to about 3.14159265, and describes the ratio of the circumference of a circle to its diameter. It doesn’t mean much to a finance person though.

But the bits have mercy if their report shows that there’s a $3.14 missing.

Sometimes, in the course of my work, I get requests to dump data from the database. Well, more like generating ad-hoc reports based on whatever was needed. Excel seems to be the preferred output format, since it displays all the data in nice little columns and rows. And let’s face it, the user is probably more skilled in Excel macros and functions than you and I are. Let them do their little calculations and predictions and pie charts, I say.

But noooo… Finance people will have none of that.

“I cannot manipulate data.” was the usual answer.

Wait, when I churned out those data, wasn’t I manipulating data, of sorts? I was doing SUMs, GROUP BYs and ORDER BYs with the database queries. If the user wanted a sum over that particular column, and I didn’t provide it, just use the inbuilt Excel sum function.

“Oh no, I cannot manipulate data.”

Apparently, whatever financial reports must be completely generated by the computer. All financial reports must be untouched by human hands. Let’s hope you got that algorithm right. Wait, aren’t you human? Would that mean the reports went through human hands?

If not using the database, please disconnect

I’m maintaining some Windows programs created by the PowerBuilder software. The original developer didn’t plan for the programs to be used by many people. So the instant one of the programs was run, a database connection to the Sybase database was opened. And left there.

As more programs were created in this manner (and added to the suite of programs my team is in charge of), the number of total users also increased. Since the connections were held in place, table locks between users became a real problem, because a user could be done with an operation, but still hold onto the table. This also meant the database became clogged up with connections, usually non-active.

The better solution is to open the connection when you’re going to do any database operation, and then close it once you’re done. But the original programs were developed like eons ago. If I understand it correctly, client programs back then assumed they had total control over the database. Contrast that with the web applications of today, and let me just say that, I have my work cut out for me…

I decided to write something on this after reading Raymond’s article on cookie licking. So if you’re not using any database functions, please disconnect.

FAHTR aka Free And Happen To Remember

The following completely fake conversation was inspired by an actual conversation with an internal company user.

User: The web application doesn’t let me do XYZ. I can’t download ABC data for customer PQR.
Me: Well, your customer database don’t have JKL. The code changes will take some time.

User: So how long will it take?
Me: It’s not urgent, right?

User: Well…
Me: Then I’ll do it on a FAHTR basis.

User: What father?
Me: FAHTR. It means I’ll do it when I’m free and I happen to remember to do it.

Satellite and maritime data calls

Let’s talk maritime communication data. That’s what my users work hip-deep in.

“Wait! Didn’t you say you’re in the satellite business the last time?”

Oh yeah… so, to me (and my users), they’re effectively the same thing. You see, out at sea (ooh, alliteration), captains and sailors still need to talk with their loved ones back on terra firma. And surf the Internet (for you know, uh stuff…). And send (short message service) SMSs to their friends (and sweet phone messages to lovers). And report back to headquarters of course.

[actually they probably don’t frivolously send love notes. Because their company is paying for it, and paying close attention to their usage. See below…]

They do that by using a device, which sends data up to space to one of the supporting satellites (not our moon, one of those man-made ones), which then sends it back to Earth to be received by a satellite dish, which then relays that data out the normal way. Here’s a simple diagram to show how it works:

Basic land-sea data transfer via satellite

I totally suck at drawing diagrams…

The ships are equipped with a terminal (no it’s not installed at the fore of the ship like in the diagram. It’s probably in the command centre or communication centre or… you know, I have no idea where they install it…), which contains one (possibly more) subscriber identity module card or SIM card. These SIM cards aren’t the kind we use in our mobile phones though.

And each SIM card has one, possibly more, mobile numbers. And each mobile number is tied to a specific service, such as voice call, or SMS, or fax. These mobile numbers are used to track usage. What do I mean by that?

For the purposes of our discussion, the word “call” refers to any initiated communication. So an SMS is a call. Browsing to a particular website is a call. Faxing a piece of paper is a call.

And each call comes with associated data, and the entire thing is referred to as a call detail record, or CDR (or call record in short). There’s a lot of data in one call, so I’m going to tell you about the ones my users and customers care most about (because they track usage religiously. I know, because I wrote the web application for it.) You may encounter other similar terms.

  • Calling number – the mobile number making the call
  • Called number – the name’s a bit misleading. It can actually be an IP address or URL, other than a receiver’s mobile number or fax number. It’s also known as destination number.
  • Unit type – “messages” for SMS, bits/bytes/KB/MB (and other variants) for IP data, sec/min for voice data
  • Call duration – a value corresponding to the related unit type, such as 2.13 MB or 27 seconds
  • Call start date/time – when the call was made

The calling number is the unique identifier. The main thing it identifies is the company to whom the billed is sent (very important, this billing thing). The calling (mobile) number is tied to a billing account in the billing system.

The call duration is especially important, because the customer is billed accordingly. This is why I’m adamant about the use of fixed point database types such as numeric(12,4). The floating point errors will scale out of hand otherwise (millions of these call records are created every month). Which will cause the total billed amount to the customers to be out of hand. Money is a very serious topic… Do not underestimate the power of 2 numbers not tallying by a difference of 2 cents in the billing report…

That’s the basics of satellite data. Oh right, the terminals…

There’s this service called Broadband Global Area Network, or BGAN for short (pronounced bee-gan), and I even presented at a product launch hosted by my employer. The BGAN service is provided by Inmarsat, a satellite telecommunications company, and they provide me with most of the data I work with. And they (and their data servers) are based in London. Which is why I obsess over timezones and formatting dates and times.

So (customer) companies buy these BGAN terminals and install them on their ships. And the captains and crew members get to communicate with people on land, and even with other ships. All made possible by routing data through the satellites.

And my users sell these terminals, associated paraphernalia (such as the SIM cards), and data usage (where price plans come in). Inmarsat deals with telecommunications companies (like my employer), and the telecommunications companies take care of the customers.

There are other types of terminals of course. Which means different services, and different types of data. Which gives my team many headaches. Because my users will be telling the customers “We support this!”, and turn to my team to actually support the new data, new price plans, new billing calculations, new entries in bills, new everything. There are now more than 10 sources of call records which we handle every day…

I believe that’s enough information for now. Any questions?

Are corporate programmers also polymath programmers?

I’ve been writing for more than 2 years now, and I realised that I never told you much about my professional job. Let’s start with a brief summary of how I started…

Beginning of professional career

I started working a few months after I graduated in 2002. I had a science degree in applied mathematics and computational science, but that didn’t stop me from applying for the software engineer, software developer, application programmer, systems analyst and other confusing variants of the job title. I like programming.

Once, I worked in a startup company, writing software to deal with patents. One of the requirements was “must know regular expressions”, used for searching through the patent text, so I put my mind to learning it. I got the job. The funny thing was, the CEO handed that part of the coding to an intern. Oh wait, a PhD intern. Maybe I wasn’t qualified to touch regex…

Somewhere in that career history of mine was a software development house. The team I was in, was assigned to develop an enterprise product for a Japanese company. It’s an internal website, and handled work flow processes, task assignment and other enterprisey functions.

Half the time, I was helping to complete some sections of a business class. A third of the time, I was doing the testing, because I seem to be the only (expendable) one with more than coding skills. Because I had experience talking with users, coming up with specifications, database design, setting test environments and oh yeah, coding.

They even offered me a role in translation, after they heard I knew Japanese. Correction I told them, I knew a little teensy bit of Japanese. They had an overflow of programmers I guess… (it was a team of 10 programmers. Or was it 12?).

In the present

Currently I’m working in a telecommunications company in Singapore. Actually, I started my first job in the same company too. After the first few years, I joined the startup. Then left and joined the software house. Then left, and rejoined my current company, but in a different team (still in the billing support department though).

The interesting thing is that I’m dealing mostly with satellite data, not mobile phone data (one of the core business of telecommunications companies). And my users are like a complete company, albeit on a small scale.

They have their own departments on customer service, marketing, sales, and to a certain extent, their own IT team. And that IT team is composed of 3 people. And I’m one of them. I have a supervisor and a colleague who handles the backend programs (mostly C and C++ on Unix).

Me? I’m the frontend guy, and the everything-else guy.

Because of the unique service (compared with the rest of the company) dealing with satellite data, my team is involved with everything from the call records of the customers, to the configuration of their price plans, to billing the customers, to settling profit with the satellite providers. The satellite business is quite interesting, and I’ll tell you more in a future article.

The other team mates focus on collection of the raw file containing the call records, processing them, calculations of the bills and other backend stuff. I handle all the stuff that users will see and interact with, like Windows applications and web applications. And I manage the Windows servers the web applications are running on. And there’s the updating of the SSL certificates for the web applications. Once I was at the data centre during a power maintenance, because someone needed to be there to flip the on-off switch of the servers and check on the equipment.

My work is seen by a lot of people, which includes the customers, the sales staff, the marketing staff, the product managers, the administrative staff and the customer service officers. This means I get a lot of queries.

If a number doesn’t tally with the total, someone sends me an email.
If the font size is too small, someone sends me an email.
If the Excel download fails, someone calls me.
If the data cannot be found, someone calls me.
If they can’t open an FDF file, they call me.
If the offshore Chinese colleague have questions on business logic or web design or code design, he/she calls/emails me.

If [something happens], someone [calls/emails] me.

In short, I get interrupted a lot. Nice private offices? Not a chance.

And I still have to write code so the projects actually ship before deadlines.

There was once where I had 1 month to come up with a new website for the customers to view their call records (brand new .NET web application, new user interface, completely new database schema). That was a December (the sales people had to move fast to get that contract before the start of the next year, which meant I had to work fast). And another colleague from another team was reassigned, and his work was handed to me (basically I was covering for 2 teams). And I received calls and queries from the users of both teams.

And the new website was out in the world on time. *whew*

Do programmers who work in a corporate environment go through similar experiences? Do they handle many non-programming related work? Are corporate programmers also polymath programmers?

I’m stumped, since I probably have this naive notion of programmers just programming (to a large extent). My friend once asked me what I do at work. I didn’t know how to answer, so I rattled off a few tasks mentioned above, and he’s surprised I do so many different tasks. Particularly when I sometimes have email correspondence directly with customers, which I try to avoid for business reasons. The sales and customer service teams are supposed to be the frontline, not me.

Helping corporate programmers on time management

This brings me to a personal project. I’m creating a product about time management for corporate programmers, working title “Time management for corporate programmers – Handling interruptions, removing distractions and getting the Flow”. Or some such.

All the writings and articles on this site will continue to be free. I’m writing an ebook to help fellow corporate programmers (maybe even programmers in general) with what I know. And this endeavor might just be able to support this site. The articles are fun to write, and I love the brilliant comments by you (even the ones that tell me I’m wrong. Those are awesome).

Actually “time management” is too specific. I’ll be writing on discipline, control and health. There will be some quick tips on data migrations and doing your miscellaneous tasks with free tools (because you know, companies are always cutting the budget). My notes actually look like a jumbled mess, but maybe that’s the nature of polymaths… or just me.

So do you have any questions about time management in a corporate setting? Or even time management in general? What are the typical tasks you do? How can I help you do your job, the tasks you really want to do (it’s coding and shipping software, right?), better?

*whispers* Psst, it’s ok if you’re a student/coder. I was a student/coder once. Just ask.

Let me know in a comment, or if you’re shy, use my contact form to email me privately. You can also talk with me on Twitter (@orcasquall).

I’m really excited about this project.