Financial reports must be untouched by human hands

December 28, 2009 By

People handling billing or financial data are usually very uptight. Especially when it comes to numbers. If you thought mathematicians, statisticians or economists were protective of their numerical figures, go talk to someone who works in the financial department.

PI is an elegant number. It goes to about 3.14159265, and describes the ratio of the circumference of a circle to its diameter. It doesn’t mean much to a finance person though.

But the bits have mercy if their report shows that there’s a $3.14 missing.

Sometimes, in the course of my work, I get requests to dump data from the database. Well, more like generating ad-hoc reports based on whatever was needed. Excel seems to be the preferred output format, since it displays all the data in nice little columns and rows. And let’s face it, the user is probably more skilled in Excel macros and functions than you and I are. Let them do their little calculations and predictions and pie charts, I say.

But noooo… Finance people will have none of that.

“I cannot manipulate data.” was the usual answer.

Wait, when I churned out those data, wasn’t I manipulating data, of sorts? I was doing SUMs, GROUP BYs and ORDER BYs with the database queries. If the user wanted a sum over that particular column, and I didn’t provide it, just use the inbuilt Excel sum function.

“Oh no, I cannot manipulate data.”

Apparently, whatever financial reports must be completely generated by the computer. All financial reports must be untouched by human hands. Let’s hope you got that algorithm right. Wait, aren’t you human? Would that mean the reports went through human hands?

Filed Under: Work Stuff Tagged With: , , ,

If not using the database, please disconnect

December 3, 2009 By

I’m maintaining some Windows programs created by the PowerBuilder software. The original developer didn’t plan for the programs to be used by many people. So the instant one of the programs was run, a database connection to the Sybase database was opened. And left there.

As more programs were created in this manner (and added to the suite of programs my team is in charge of), the number of total users also increased. Since the connections were held in place, table locks between users became a real problem, because a user could be done with an operation, but still hold onto the table. This also meant the database became clogged up with connections, usually non-active.

The better solution is to open the connection when you’re going to do any database operation, and then close it once you’re done. But the original programs were developed like eons ago. If I understand it correctly, client programs back then assumed they had total control over the database. Contrast that with the web applications of today, and let me just say that, I have my work cut out for me…

I decided to write something on this after reading Raymond’s article on cookie licking. So if you’re not using any database functions, please disconnect.

Filed Under: Work Stuff Tagged With: , , ,

FAHTR aka Free And Happen To Remember

November 24, 2009 By

The following completely fake conversation was inspired by an actual conversation with an internal company user.

User: The web application doesn’t let me do XYZ. I can’t download ABC data for customer PQR.
Me: Well, your customer database don’t have JKL. The code changes will take some time.

User: So how long will it take?
Me: It’s not urgent, right?

User: Well…
Me: Then I’ll do it on a FAHTR basis.

User: What father?
Me: FAHTR. It means I’ll do it when I’m free and I happen to remember to do it.

Filed Under: Work Stuff Tagged With: , , , ,

Satellite and maritime data calls

October 1, 2009 By

Let’s talk maritime communication data. That’s what my users work hip-deep in.

“Wait! Didn’t you say you’re in the satellite business the last time?”

Oh yeah… so, to me (and my users), they’re effectively the same thing. You see, out at sea (ooh, alliteration), captains and sailors still need to talk with their loved ones back on terra firma. And surf the Internet (for you know, uh stuff…). And send (short message service) SMSs to their friends (and sweet phone messages to lovers). And report back to headquarters of course.

[actually they probably don't frivolously send love notes. Because their company is paying for it, and paying close attention to their usage. See below...]

They do that by using a device, which sends data up to space to one of the supporting satellites (not our moon, one of those man-made ones), which then sends it back to Earth to be received by a satellite dish, which then relays that data out the normal way. Here’s a simple diagram to show how it works:

Basic land-sea data transfer via satellite

I totally suck at drawing diagrams…

The ships are equipped with a terminal (no it’s not installed at the fore of the ship like in the diagram. It’s probably in the command centre or communication centre or… you know, I have no idea where they install it…), which contains one (possibly more) subscriber identity module card or SIM card. These SIM cards aren’t the kind we use in our mobile phones though.

And each SIM card has one, possibly more, mobile numbers. And each mobile number is tied to a specific service, such as voice call, or SMS, or fax. These mobile numbers are used to track usage. What do I mean by that?

For the purposes of our discussion, the word “call” refers to any initiated communication. So an SMS is a call. Browsing to a particular website is a call. Faxing a piece of paper is a call.

And each call comes with associated data, and the entire thing is referred to as a call detail record, or CDR (or call record in short). There’s a lot of data in one call, so I’m going to tell you about the ones my users and customers care most about (because they track usage religiously. I know, because I wrote the web application for it.) You may encounter other similar terms.

  • Calling number – the mobile number making the call
  • Called number – the name’s a bit misleading. It can actually be an IP address or URL, other than a receiver’s mobile number or fax number. It’s also known as destination number.
  • Unit type – “messages” for SMS, bits/bytes/KB/MB (and other variants) for IP data, sec/min for voice data
  • Call duration – a value corresponding to the related unit type, such as 2.13 MB or 27 seconds
  • Call start date/time – when the call was made

The calling number is the unique identifier. The main thing it identifies is the company to whom the billed is sent (very important, this billing thing). The calling (mobile) number is tied to a billing account in the billing system.

The call duration is especially important, because the customer is billed accordingly. This is why I’m adamant about the use of fixed point database types such as numeric(12,4). The floating point errors will scale out of hand otherwise (millions of these call records are created every month). Which will cause the total billed amount to the customers to be out of hand. Money is a very serious topic… Do not underestimate the power of 2 numbers not tallying by a difference of 2 cents in the billing report…

That’s the basics of satellite data. Oh right, the terminals…

There’s this service called Broadband Global Area Network, or BGAN for short (pronounced bee-gan), and I even presented at a product launch hosted by my employer. The BGAN service is provided by Inmarsat, a satellite telecommunications company, and they provide me with most of the data I work with. And they (and their data servers) are based in London. Which is why I obsess over timezones and formatting dates and times.

So (customer) companies buy these BGAN terminals and install them on their ships. And the captains and crew members get to communicate with people on land, and even with other ships. All made possible by routing data through the satellites.

And my users sell these terminals, associated paraphernalia (such as the SIM cards), and data usage (where price plans come in). Inmarsat deals with telecommunications companies (like my employer), and the telecommunications companies take care of the customers.

There are other types of terminals of course. Which means different services, and different types of data. Which gives my team many headaches. Because my users will be telling the customers “We support this!”, and turn to my team to actually support the new data, new price plans, new billing calculations, new entries in bills, new everything. There are now more than 10 sources of call records which we handle every day…

I believe that’s enough information for now. Any questions?

Filed Under: Work Stuff Tagged With: , , ,

Are corporate programmers also polymath programmers?

September 24, 2009 By

I’ve been writing for more than 2 years now, and I realised that I never told you much about my professional job. Let’s start with a brief summary of how I started…

Beginning of professional career

I started working a few months after I graduated in 2002. I had a science degree in applied mathematics and computational science, but that didn’t stop me from applying for the software engineer, software developer, application programmer, systems analyst and other confusing variants of the job title. I like programming.

Once, I worked in a startup company, writing software to deal with patents. One of the requirements was “must know regular expressions”, used for searching through the patent text, so I put my mind to learning it. I got the job. The funny thing was, the CEO handed that part of the coding to an intern. Oh wait, a PhD intern. Maybe I wasn’t qualified to touch regex…

Somewhere in that career history of mine was a software development house. The team I was in, was assigned to develop an enterprise product for a Japanese company. It’s an internal website, and handled work flow processes, task assignment and other enterprisey functions.

Half the time, I was helping to complete some sections of a business class. A third of the time, I was doing the testing, because I seem to be the only (expendable) one with more than coding skills. Because I had experience talking with users, coming up with specifications, database design, setting test environments and oh yeah, coding.

They even offered me a role in translation, after they heard I knew Japanese. Correction I told them, I knew a little teensy bit of Japanese. They had an overflow of programmers I guess… (it was a team of 10 programmers. Or was it 12?).

In the present

Currently I’m working in a telecommunications company in Singapore. Actually, I started my first job in the same company too. After the first few years, I joined the startup. Then left and joined the software house. Then left, and rejoined my current company, but in a different team (still in the billing support department though).

The interesting thing is that I’m dealing mostly with satellite data, not mobile phone data (one of the core business of telecommunications companies). And my users are like a complete company, albeit on a small scale.

They have their own departments on customer service, marketing, sales, and to a certain extent, their own IT team. And that IT team is composed of 3 people. And I’m one of them. I have a supervisor and a colleague who handles the backend programs (mostly C and C++ on Unix).

Me? I’m the frontend guy, and the everything-else guy.

Because of the unique service (compared with the rest of the company) dealing with satellite data, my team is involved with everything from the call records of the customers, to the configuration of their price plans, to billing the customers, to settling profit with the satellite providers. The satellite business is quite interesting, and I’ll tell you more in a future article.

The other team mates focus on collection of the raw file containing the call records, processing them, calculations of the bills and other backend stuff. I handle all the stuff that users will see and interact with, like Windows applications and web applications. And I manage the Windows servers the web applications are running on. And there’s the updating of the SSL certificates for the web applications. Once I was at the data centre during a power maintenance, because someone needed to be there to flip the on-off switch of the servers and check on the equipment.

My work is seen by a lot of people, which includes the customers, the sales staff, the marketing staff, the product managers, the administrative staff and the customer service officers. This means I get a lot of queries.

If a number doesn’t tally with the total, someone sends me an email.
If the font size is too small, someone sends me an email.
If the Excel download fails, someone calls me.
If the data cannot be found, someone calls me.
If they can’t open an FDF file, they call me.
If the offshore Chinese colleague have questions on business logic or web design or code design, he/she calls/emails me.

If [something happens], someone [calls/emails] me.

In short, I get interrupted a lot. Nice private offices? Not a chance.

And I still have to write code so the projects actually ship before deadlines.

There was once where I had 1 month to come up with a new website for the customers to view their call records (brand new .NET web application, new user interface, completely new database schema). That was a December (the sales people had to move fast to get that contract before the start of the next year, which meant I had to work fast). And another colleague from another team was reassigned, and his work was handed to me (basically I was covering for 2 teams). And I received calls and queries from the users of both teams.

And the new website was out in the world on time. *whew*

Do programmers who work in a corporate environment go through similar experiences? Do they handle many non-programming related work? Are corporate programmers also polymath programmers?

I’m stumped, since I probably have this naive notion of programmers just programming (to a large extent). My friend once asked me what I do at work. I didn’t know how to answer, so I rattled off a few tasks mentioned above, and he’s surprised I do so many different tasks. Particularly when I sometimes have email correspondence directly with customers, which I try to avoid for business reasons. The sales and customer service teams are supposed to be the frontline, not me.

Helping corporate programmers on time management

This brings me to a personal project. I’m creating a product about time management for corporate programmers, working title “Time management for corporate programmers – Handling interruptions, removing distractions and getting the Flow”. Or some such.

All the writings and articles on this site will continue to be free. I’m writing an ebook to help fellow corporate programmers (maybe even programmers in general) with what I know. And this endeavor might just be able to support this site. The articles are fun to write, and I love the brilliant comments by you (even the ones that tell me I’m wrong. Those are awesome).

Actually “time management” is too specific. I’ll be writing on discipline, control and health. There will be some quick tips on data migrations and doing your miscellaneous tasks with free tools (because you know, companies are always cutting the budget). My notes actually look like a jumbled mess, but maybe that’s the nature of polymaths… or just me.

So do you have any questions about time management in a corporate setting? Or even time management in general? What are the typical tasks you do? How can I help you do your job, the tasks you really want to do (it’s coding and shipping software, right?), better?

*whispers* Psst, it’s ok if you’re a student/coder. I was a student/coder once. Just ask.

Let me know in a comment, or if you’re shy, use my contact form to email me privately. You can also talk with me on Twitter (@orcasquall).

I’m really excited about this project.

Filed Under: Work Stuff Tagged With: , , ,

Certifications, standards or just codswallop?

March 23, 2009 By

Recently, I attended a 1 day course on Information Technology Infrastructure Library or ITIL. It’s just an awareness course, so I only learned what it is, what it’s used for, why it should be used and so on. Basically ITIL is a set of best practices for doing IT work.

Truth be told, I didn’t want to attend the course. Due to management targets and budget considerations, I was “strongly encouraged” to attend the course (it is the end of the financial year…). Ah well, so I did. Man, did I have to rearrange my coding deadlines… One never knows how much 1 day of coding means, till one loses that 1 day…

Anyway, I learnt what ITIL is, and what the associated ISO 20000 is too. ITIL, as my instructor mentioned, is “documented common sense“. Also, an organisation (or process) can’t be certified in ITIL, as there might be some recommendations that conflict with each other within your business scope. An individual can be ITIL certified though.

Businesses can be ISO 20000 certified. The process of certification, as I understand it, is tedious. You need documentation of processes, and you need proof that you’ve been following the standards and your own documentation. And the first thought that came to my mind was Evidence Based Scheduling and FogBugz. I talked with the instructor and he agreed that, having a system that’s part of the workflow, and that it captures proof as it goes, is the best solution. And you need at least 3 months worth of proof.

I can see 2 benefits to certification. Boost in credibility (people trust companies with certifications?) and actual improvements in financial bottom lines (whether it’s cost savings or profit).

Frankly speaking, I don’t know why I’m attending the course. That I’ll push for a more structured IT environment back at work, following ITIL recommendations? That I can be part of the certification process team? I’m a messy person. What do they think I can do?

Ok, the course wasn’t a complete loss for me. At least I learnt about the certification process. Still, I keep wondering, could the entire thing be condensed into the Joel Test?

At the team I’m working at, we’d maybe pass 3 out of the 12 points on the check list. If we can’t pass that, we can’t pass ITIL, and we certainly can’t pass ISO 20000. I’m just saying…

[Vincent is currently preparing to go to the land of the rising sun, and hopefully see sakura blossoms. Comments and responses will be slower. I'll be taking over for a while, and remember, I'm just a blog.]

Filed Under: Work Stuff Tagged With: , , ,

Every user query is a puzzle

February 27, 2009 By

The user didn’t really mean to give you a level 5 puzzle to solve. But that’s what it is. Every time the user asks you to help him solve a software related problem, he’s giving you a puzzle.

Sometimes, it’s a level 1 puzzle, where you don’t even need to fire up your code editor to glance through the code to answer, or give more than a few seconds to think through. Sometimes, you have to really push for lots of hints, like clarifying the user’s query, ask for screenshots (even if sometimes they aren’t enough…), and dig through code archives.

I got a level 1.5-ish puzzle lately… The user said he couldn’t use the delete button. I fired up the application, selected a record, and indeed the delete button was disabled. On a whim, I double-clicked. The details of the selected record came up, and the delete button was enabled.

I have never seen or used that particular section of the application before. The user on the other hand, was supposed to be familiar with the application. Perhaps I’m more willing to try out typing randomly or aimlessly clicking the mouse on an application screen. I certainly typed enough asdf’s…

So, have you encountered any high level puzzles? Share in the comments.

Filed Under: Work Stuff Tagged With: ,

Brain dead

February 9, 2009 By

I can’t think. I can’t focus. My thoughts are scattered, flying around of their own volition, out of my reach, out of my control.

I believe this is what people call “burnt out”.

For the past couple of months, I’ve been coding for, or working on, a few projects. I was either the sole developer, or a major contributor to the project. Their deadlines had been either back-to-back or overlapping. And this unhealthy schedule stays for the next couple of months.

I may be a Polymer, but I am only one.

There had been a few instances in the past where projects bunch up together. It was tough, but I weathered through. This time, it took its toll on me. I’ve never been out of the Zone for so long.

All my usual remedies failed. Reading. Hanging out with friends. Listening to upbeat music while working (but I already do that on a regular basis). Tea. Playing video games. Exercise. Coffee (mocha. I can’t withstand the strength of “normal” coffee…). Going for walks.

I was about to attempt unfettered hedonistic pleasures when I remembered I could take a break. Ok, it was more like “Take a day off!”, “Stop working!” and “Fight for Freedom!”. Ok, maybe not the last one…

I heeded those subtle hints, and took a day off here, a day off there. The Chinese New Year, which was just over, lengthened a weekend. So many people were taking days off too. Hint: don’t take the same days off as the other people if you can help it. Take those days when they’re back in office. This way, you get more done because you don’t get interrupted.

Stop interrupting me!

This episode was seriously difficult to recover from.

It came back, surprising me with its arrival. My coding groove. There were no announcements, no hints. One day I was down in the slumps, the next I was able to think cogently for a few hours straight. That gave me enough time to sift through the requirements, because much of programming isn’t really writing the code. It’s thinking.

I’m still feeling terrible, but at least I’m able to get the projects moving faster now. All the tricks for getting my energy back failed abysmally. In the end, my mind and body provided the best solution.

It was time. I just needed to rest. Sometimes, the fastest way is to stop.

Filed Under: Work Stuff Tagged With: , ,

Random Quote – Program X not social networking site

October 29, 2008 By

Today, my users got into a tangle. All of them got into a database deadlock.

Now my users have this habit of opening multiple windows so as to “appear busy”. Granted, my application wasn’t designed and programmed with many concurrent users in mind. Still, if more than 2 of them happen to access the same table, or heaven forbid, performing an update or delete operation on that table, the database throws up its hands in defeat and promptly denies further access. This includes other users accessing other tables.

So I did an sp_who (it’s a Sybase database) and found out the perpetrators. Then I wrote an email telling those who were doing non-critical tasks (and not still performing database transactions) to get the heck out of my database and log out of the application. And not return until half an hour later. Nicely coached in diplomatic language of course.

One of them asked if another colleague could continue, since that person was still performing a transaction. I said “Of course.” I also added something in the email that I thought was interesting (on hindsight):

Program X is not a social networking site. Log in, do your thing and log out.

Yes, I was a bit peeved and slightly mischievous at the time.

Filed Under: Work Stuff Tagged With: , , , , ,

A typical month work load

October 27, 2008 By

Climber by Bettina Ritter
[image by Bettina Ritter]

I don’t really have a typical work day. It doesn’t mean I have an exciting job. It just means I can’t tell you what I’ll be doing the next day, because I don’t know for sure what I’ll be doing. What I can tell you is what I do in a typical month, generally speaking. There’s a point to all this, and I’ll start with…

From 8:30 am till 6 pm

Those are my work hours. Except Friday. On Fridays, I get to go off at 5:30 pm. My current job title is IT Analyst, changed from Systems Analyst. And if you think that’s vague, you’re right. My job scope is quite varied. Basically, my contractual terms require me to “do whatever the boss tells you to do”.

The current company I’m working for isn’t a software company. I just work in the IT department. What it means is, programming isn’t as highly regarded as I want, as what I read about in those programming blogs and sites. It kinda sucks, but it keeps me fed.

Let me tell you about the tools I use at work. I’m the “online guy”, which means any user interface related development comes to me. I use Visual Studio 2003 (C# and VB.NET) for all the web applications, console programs and a few software tools I create to help me. I also use Visual Studio 2005 for one particular application, with a graphical user interface. It’s too tedious to explain why I use both versions. It’s enough to know that I do.

I also use PowerBuilder for some Windows applications. It’s really, really painful to work with PowerBuilder code. I tell myself it’s the previous programmer’s skill that’s to blame, not the language, but I frequently fail. Tracing and debugging PowerBuilder code takes a lot of work for me. I really hate PowerBuilder… I think this calls for a separate rant article.

I’ve been asked to investigate C and C++ code on Unix machines too. So yes, I understand make files, shell scripts and cron jobs. I even know how to use the vi editor! I used to telnet to the Unix machines with TerraTerm, which is now abandoned for a more secure client application. Can’t remember the name because I rarely use it, because I rarely need to telnet.

Database admin, server admin and LAN admin

Despite the fact that I’m completely ignorant of SQL and databases in my formal education, I’m thrust full force into it at work. I’ve worked with the Oracle, SQL Server and Sybase databases, know most of the nuances between them on SQL syntax, and understand how to use stored procedures. I handle them all.

I am also completely in charge of a few Windows servers and the SQL Server databases running on them. Server maintenance, backup schedule and tapes, security patches, SSL certificates, IIS configuration, server performance.

Then I’ve got to know about the opening of ports for security purposes, who to notify when there are application or server changes. I need to know ping and tracert and ipconfig and other network related stuff.

All of that maintenance and administration is on top of my development work.

I don’t need to connect to Oracle databases now, but I used to do so with TOAD. There’s a limit to the number of licenses, so I wrote my own database connector program. It only does retrieval of data, basically the select statement, but it’s enough for current tasks. The Oracle databases belong to another team, and they’ve only needed me to help out rarely.

I use the Enterprise Manager and Query Analyzer for the SQL Servers. They’re great tools, and they come with the database installation, which is cool. There’s also another tool that has saved me many times. It’s the DTS, Data Transformation Services. I’ve used it to transfer data interchangeably between Oracle, SQL Server, Sybase and get this, Excel. Users take to Excel much better, so I need to use their form of “database”.

Designers, comparers and reflectors

I’m also a web designer. I suck at it, but I’ve been lucky enough to muddle through, and my users and their customers think my user interface looks awesome. I use Paint.NET (and sometimes the inbuilt Windows Paint program) for my image editing tasks. Plus I’ve got some colour tricks up my sleeve.

Some time ago, I had to verify some old code by another programmer. He can’t remember what he changed, and I obviously don’t know what could possibly be changed. I needed help! Fortunately, I found CSDiff. It allows you to compare two files (or even folders) and lists down differences between them. Much better than checking line after line of code by inspection.

And if you do .NET work, you must get the Reflector by Lutz Roeder, which had been taken over by Red Gate Software. It allows you to get back code from compiled .NET DLLs and programs. The result might not be the prettiest code, but with sufficient talent and patience, you can get something out of it.

I’ve used it on my own code and other team members’ code to check for disparity. Sometimes, you forget which version you’ve compiled that code into… Sometimes, it’s for self study, to understand what others have done.

The phone calls. Oh the phone calls.

My phone rings a lot. There are over 10 people in my immediate vicinity. I can tell you that, if you add up all the phone calls all of them ever receive in a month, it would still be less than what I alone receive in that same month.

Remember I told you I’m the “online guy”? That means a lot of users know me, and I don’t know all of them. Since they usually interact with the application interface, any problem is routed to me. Whether it’s data inconsistency, business logic query, application error or failure, all of them come to me. I’m a one-man helpdesk I tell ya.

It was so bad that sometimes, I’ve had to solve and handle user queries for entire days on end. Due to the nature of my work, the start and end of the month are particularly busy for my team. The number of times my phone rings goes through the roof. Maintaining decent phone etiquette starts to be a strain…

Wait, there’s something missing…

Where’s the source control software I’m supposed to be using? Well, I’m the source control. My team is very small in size. Company directives dictate we send work to our offshore colleagues. I think those (typically recent graduate) colleagues have some problems of their own, let alone set up a source control system that works across geographic boundaries.

I’ve not been with development teams at other IT departments, but I think we would totally fail at the Joel Test. Totally.

Despite these circumstances, I still manage to do development work, sometimes with surprising and outstanding results. I believe good task management is crucial to my balancing act. Which brings me to…

Holistic approach to programming

If you’re working at a software company, or on something focused on software and programming, I envy you. I really do. You’d probably get to talk with other programmers on interesting topics. Your work is really appreciated, because it goes to the bottom line.

I might not be programming exclusively, but I get to see the bigger picture. I get to liaise with people from sales, marketing and customer service. I get to talk with upper management and even the actual customers. I get to see the kinds of products and services offered, and how it’s implemented and supported by software.

Programming is kind of … an elite thing. When I was studying C programming in university, I was surprised that many of my fellow students struggled with it. I took to it like a fish in water. After a while, I realised that most people cannot grasp the thinking required in programming, even if they opted to study it themselves.

So I’m going to state this. Many people are not going to understand how great that piece of code you’ve written. Many people think software can make their lives easier, but fail to realise that not everyone can write good software.

This is where all your other skills come in. You have to sell what you’re doing to other people. Convince them that it’s useful, that it’s awesome, that it’s relevant, that what you do and what you propose is important.

Sell your ideas. Market your ideas. Your software is more useful if you see it from a bigger-picture point of view, from other people’s point of view. That requires you to understand other concepts. Concepts that aren’t related to programming at all. And you synthesise them together to make your code better.

And that, is my point.

Filed Under: Work Stuff Tagged With: , , , , , , ,
« Older Posts