Estimating and Timeboxing

In the summer of 2001, I was a lowly System Administrator managing a farm of IBM AIX servers running a payment gateway for a major MasterCard franchise. My network was segregated from the rest of the organization using dedicated firewalls. Firewall logs were in a very raw format: Source IP#, Source Port #, Destination IP#, Destination Port # and a timestamp. I had repeatedly asked for some networking and monitoring intelligence tools to help parse the firewall logs and turn them into more human understandable data but was turned down. I even offered to write some PERL scripts to do some of this work myself if I could have a company supplied cell phone to send any alerts to. Nada.

When the "Code Red" worm was released, my manager freaked. "I want you to read every line of those firewall logs and make sure nothing is getting through!" he said. I responded that Code Red only affected Windows servers running IIS. He didn't care. "Read them!" he insisted. I countered that there were about 500,000 lines of digits a day coming through the logs; his task was impossible.

His order was a desperate attempt to assert managerial oversight in a domain he really didn't understand. He was a colonel in the Army reserves and had never been an IT person. I think he got the job when the CEO asked his executive, "Who wanted to manage the IT department?" and Malcolm stretched his arms for a yawn at the same time.

It would never have been possible to ever convince him that Code Red wasn't a threat. He believed in management by reaction. Ordering something to not happen meant that it couldn't.

This memory came back to me this past week when my current CIO asked me to assign a member of my team to complete our Software Asset Registry. The Registry is a spreadsheet containing the list of all software we have in our organization determined by polling our IT departments and business units. We needed a Business Analyst to fill in the missing data in the table. This data would then be used to feed our operational plan such as deciding which software as at risk of falling out of support; which software should be retired or replaced; and who were the business owners for an application and did they even still use it. It seemed like a simple, straightforward request.

Opening the Registry for the first time, I saw that it contained over 600 software applications.Some quick spreadsheet functions helped me assess that there were over 2000 missing pieces of data. While some of this data would be easy to discover ("What is the manufacturer's current release version of this package?"), other questions would take some time ("Should we retire or upgrade this application?"). Easy questions might take 3 minutes for a web search; hard questions could take hours of meetings to get an answer.

Within 10 minutes of opening the spreadsheet, I realize the task ahead was daunting. It wouldn't be like reading 500,000 lines of firewall logs a day, but it would take some time. I estimated that each missing datum would take an average of 15 minutes to research an answer. At this rate, the entire task would take 73 person days of work.

Did I think this would be time well spent? That's not really my question to answer. Did I think my CIO knew the scope of the effort required to complete the Registry before he assigned it to my team? That's the question I have to ask. I don't think he realized how poor the initial dataset actually was.

Asking me to complete the Registry made perfect sense at the time. But deciding to proceed with the work once without validating the scope and effort did not make any sense at all. Is this Registry really worth 73 person days of work at the labour rate we pay for our BAs?

I put the question back to our CIO with some options. I shared my time estimate with him and asked him:

  1. Should we modify the scope of the work to reduce its duration and effort?
  2. Should I validate my 15 minute per task estimate by taking (say) 20 applications and determining exactly how long they take a BA to populate the missing fields? or
  3. Should I assign someone to do this work anyway, now that he knows the effort involved? And if so, can we hire a summer student to do this?

In my experience, too many managers either do whatever their bosses want--even if, upon sober second thought, it seems ridiculous; or they refuse to do what their boss asks hoping that it will go away, be forgotten, or be assigned to someone else. Perhaps this last option is why the Registry's data is in such a sorry state to begin with.

When faced with tasks that seemed reasonable but quickly devolve to a "three Tylenol day", I suggest giving a manager or business client some concrete estimates of what it will cost to fulfill his or her request and some options should he or she agree that the expense of the request is too high. In this way, we can let the requester "save face" by not doing silly things that seemed simple enough in the first place, yet also give them the ownership of the issue to do a silly thing if they really want to. By also giving them the facts they need to make a decision, we're counting on them coming to the same conclusion on their own.