Sometimes a Check Sheet is all the Data you Need

Being data-driven in a sea of anecdotes

As the company’s new COO, I was reviewing the list of programming projects. One of them was over a year old and estimated to take nearly 6 months to complete. I called the VP of development.

Finding the problem

“Can you tell me about this project?”

“Oh yeah,” she said. “That’s a full rewrite of the billing system. It’s a huge project. It’s likely to break a whole lot of things so the testing is significant. And we aren’t even sure yet whether it will fix the problem.”

“What exactly is the problem?” I asked.

“Sometimes, the billing system does this weird thing with this fee. It’s a small amount of money but customer service gets a ton of calls and it’s creating a lot of work. We know we need to get it fixed. Plus we hate the way the billing system works, so we might as well rewrite it while we’re at it.”

Problem #1. Scope creep. Developers are amazing learners. They’ll build something the way they know how to build it, then in 6 months realize there was a better way to program it. So they spend the next 6 years looking for a project that will give them the opportunity to “fix” their old programming, even if it isn’t broken.

“How big is the problem?” I asked naively.

“We can’t tell. It doesn’t happen consistently, so we can’t search for it. I guess we could tell how often they adjusted it manually, but this isn’t the only reason they do that, so that won’t really tell you.”

“Fair enough. Can you pull that number for me though, along with anything else that might tell us the reason they adjusted it manually? Maybe we can get a ballpark number.”

After agreeing on a follow-up date, I went to see the Customer Service manager.

Researching the problem

“I’m looking at this project and trying to understand…”, I started.

“Oh great,” she interrupted. “That is such a huge problem! The team will be so happy you are going to get it fixed.”

I shuffled in my seat a bit. “Well, before we try to fix it, I want to understand it better. It’s a big project and we aren’t even sure we know what needs to be done yet. How often is this happening?”

“Oh, it’s constant! Bob spent an hour with one customer who was so mad! Everyone takes these calls all the time. It’s a huge problem.”

Problem #2. Empathetic managers without data. This manager obviously cared a lot about her people and the customers, amazing traits. But this wasn’t the first time I’d heard her blow a problem out of proportion without first identifying how large it actually was. She wanted to rescue the team instead of improving it.

“Okay. We can’t programatically identify when this happens. Given the scope of the project and the potential negative impacts it could have, I’d like to understand it more. What if we asked the team to keep track of these calls on a check sheet for a few weeks, just a quick scratch kind of thing?”

She became even more animated. “Oh no! They HATE check sheets. I have one already but I’m constantly having to remind them to use it and I know it’s not right.”

“Oh, well if you already have one, can I see what they’re already tracking?”

She handed me a sheet that had obviously been copied over and over and over… It had 25 categories on it. A full sheet of things like “customer asked for the manager”, and “billing issue”. Some items were overly specific, some were apparent duplicates, and some were so vague they were totally useless.

“What do you do with these after they fill them out?”

“I add them up every month and send them to our old VP.”

I confirmed that she believed the information was inaccurate and that she didn’t use it herself. Then I then told her to stop today.

“Go get them off their desks right now. Put a post-it on their monitor with Fee Error written on the top. That’s the only thing I want them manually counting for the next two weeks. Have them bring them to your next team meeting and invite me please.”

I gained a lot of points that day. Apparently those check sheets were the bane of their existence and this fee problem was their biggest complaint. With one change, I’d addressed both.

Problem #3. Old ineffective manual processes grow. Every time anyone had a complaint, a new line was added to the check sheet. She was trying to do the right thing — look at the data. The manager had done the only thing she knew to do, and her manager had never bothered to follow through.

The quick fix makes the problem harder

While the counting progressed, I hit the last leg of this three-legged problem. Most problems have 3 key contact points: the problem, the short-term solution, and the long-term solution. We were evaluating the problem and I understood the potential long-term solution. Now I needed to understand the short-term patch that was in place. So I headed to accounting to find the woman who did all the manual adjustments.

“Yeah, that’s such a pain,” she started. “I spend hours every week fixing them.”

Uh-oh. Maybe it was a bigger problem than I thought.

“Hours? How many do you think you do each week?”

“Oh, it varies. I get at least a couple a week, but sometimes as many as five!”

I squirmed again. Hours, to fix a couple a week?

“How much money do most of them involve?”

“Again, it varies, but most end up being $10 or $12.”

Problem #4. Task instead of impact focus. Most people do the task without thinking about the real costs. This accountant’s time was worth much more than the money we were saving.

I spoke with the accountant and her manager. Most of the time waste was from chasing approvals. The manager worried about having a junior accountant reducing fee income without oversight. A good concern to have. She’d taken the fast and easy option of having all adjustments preapproved by her. A rational decision for a very small company, but we were no longer a very small company. They needed a simpler process.

The first step was batching approvals. Instead of individual emails for each approval, they batched them. The manager reviewed twice a day at regular intervals. That reduced the review time for the manager without too much delay for the customer, and it sped things up for the accountant.

The second step was reviewing the amounts of the requested transactions. The large majority were for amounts under $25. We created an end of day report that the manager could scan and approve in batch for these smaller amounts. Then we added some month-end reports that would tell us what the total cost was of the adjustments. The manager and I agreed to review every month to make sure we weren’t losing too much from the changed process.

In a few weeks, the junior accountant reported that her time for these went from hours a week to a single hour.

Problem #5. Old processes don’t fit anymore but don’t get reviewed. Sometimes, a very appropriate process is started because of a problem. But when the problem is resolved or the situation changes, that old process remains.

Back to the source

Two weeks later at the Customer Service meeting, the check sheets revealed the true scope of the problem. There was a consistent flow of this problem, but it was quantifiably less often than it “felt”.

When faced with the numbers, the team turned to emotion.

“But the customers hate it still! Remember that one time when…”

There were a lot of these “remember when” stories. With each one, I asked if the new short-term process would have helped. They agreed that the shorter and simpler adjustment process would have helped.

I described for them what the long-term fix might look like. A complete redesign of the billing system. It would probably fix this problem and maybe some other small issues. However, it would mean learning a whole new system, and likely create new issues we hadn’t even considered yet.

We agreed to kill the “fix it” project. I agreed to be the escalation point for any complaints they weren’t comfortable handling. I assured them that I was frustrated that we couldn’t fix it, but I believe the cure would be worse than the problem. I thanked them for taking the time to help me estimate the size of the problem. And for helping us find a smoother “patch fix” alternative. And most of all for their continued customer focus.

Problem #6. Fixing a problem without considering the alternatives usually causes more problems. In their haste to eliminate complaints, companies often jump to risky projects. Take the time to understand the problem and generate alternatives. You can often find a simpler answer. Whenever you are faced with a problem, quantify, validate, and innovate.

Sometimes the fix is more obvious

Some small but loud problems need fixes. This consistent complaint was resolved with a picture.

A customer sent us this photo of a huge stack of paper. His letter explained that every time we made a change to his account, we mailed him this huge stack of paper. His mailman was starting to complain.

Off to customer service again.

“How often does this happen?”

“I’m not sure but I think it’s every time we….”

“Great. Pull 5 examples for me and let’s meet at the end of the week.”

Sure enough, we had a consistent problem. Fortunately, it was easier to identify when it happened. But again I hit the “it’s too hard to fix” wall.

“What if we didn’t fix it, but just automated the workaround?”

It took only a few weeks to program a new process. We let the system do whatever it wanted. But we added a new process. The problem situation was identified and the extra pages deleted from the file before it printed for mailing. Not an elegant solution, but a quick fix to an irritating problem. Customers were happier, and we saved a big chunk of printing and postage costs.

Data is everywhere

When a problem hits, a lot of managers rely on emotion and relationships to get it solved. Their heart is in the right place. Their team is unhappy, the customers are unhappy, and they want to make it better.

But drama is not a solution.

You can always find, build, and sometimes even buy the data you need to scope a problem. There is always a better and safer solution than the one you first think of. Big projects are risky, and they take away resources that your company needs for bigger impact things.

The final review

Problem #1. Scope creep. Keep your projects focused on the solution. Unless there is a conscious decision to leverage a project for code improvement, don’t do it.

Problem #2. Empathetic managers without data. Teach your managers to collect and review data to understand the problems their teams face.

Problem #3. Old ineffective manual processes grow. Review your manual processes routinely. Eliminate what you can, streamline what you still need, and automate the critical ones.

Problem #4. Task instead of impact focus. Teach your employees the value of their time. Help them balance the time they spend with the hard cash they are trying to save.

Problem #5. Old processes don’t fit anymore but don’t get reviewed. Review your processes routinely. Eliminate what you can, streamline what you still need, and automate the critical ones.

Problem #6. Fixing a problem without considering the alternatives usually causes more problems.

For any problem:

• Quantify the problem (even if you have to use a check sheet)
• Validate the options that could solve it
• Innovate new solutions that improve the overall situation.

Thanks for reading. If this was helpful or interesting to you, please clap (multiple times!), comment, or contact me at [email protected]. If you need help moving your company from anecdotes to a data-driven problem-solving machine, let’s talk.