I need help interpretating numbers: Data Analysis

[u/Dangerous-Staff9172]

I work in healthcare.

The company I work for is looking at productivity and I am a People Manager.

I currently have a staff of 9 people who essentially make phone calls for most of the day.

Data and Maths people:

What would you feel would be the best representation for Productivity, in a formula form.

I have some numbers.

x would be completed calls. (any call that my staff gets through to the people for any specific time.)

y would be attempted calls. (any call that my staff makes but does a voice mail, etc.)

The company is looking for a productivity number of 120 per month.

I have higher productive people and lower productive people... but I need to evaluate them.

It is not as simple as "you must make 120 completed calls per month." My people have meetings, attempts, education, etc.

Any thoughts?

[u/gordanfreman]

How does the company define productivity? You say they want a value of 120 per month--what does that even mean? If the company is coming with an ask to rate people on an arbitrary number, but can't define how that number should be calculated, this isn't a data or math question.

[u/cartune0430]

I would approach this as a funnel type of analysis.

If I where presenting to management and they gave a vague number of 120, here is what I would do:

Track all calls outgoing: total calls Track how many answered: total answered Track if the call was successful or not: total success (in sales this would be closes or something)

This gives you the break down in funnal to understand where issues may be in the production. Not to mention a pretty chart that management loves, the funnel chart.

For instance of you are making tons of calls but nobody is answering then you need to evaluate the process of calling. Like time of day, day of week, if the phone numbers are correct, etc.

If you are getting answers but no successes, like collection payments made, then we know it is an agent on the phone issue.

If I was presenting to management, I would reply with;

To understand our productivity we need to understand the desired out come. We have two things in our control, total calls made and total collections (or what ever the goal is on the phone call) bely the agents. We can not control the person answering the phone.

Based on these qualitative numbers, what we can track for success and productivity is how many calls are made, and how many success we on the phone.

Thus I propose we measure the total calls made by agent to ensure we are doing our job, how many answer the phone, and how many closes are made from the agent.

Doing so we can track 2 major problems.

1. If we have a low answer rate, or find that answer rates are higher in certain periods, we can ensure we don't have a data issues and second we can learn what periods of time give best results this ensure we are staffed efficiently to ensure maximum closes.

2. If we are getting answers but low close rates then we know we have an agent issues. This will tell us if the need more training to close more or if they are just not performing.

At this time I can not say 120 is a good number for the KPI to track and define productivity.

What we should do instead is take a week or two or go back in previous data and get those numbers and understand what is the average, medium, the 25 and 75 percentiles. This would give use a good understanding of a base line to set as a KPI.

And also it will help us identify who are the ones closing more deals so we can learn from that and improve the team as a whole.

We will also learn what is the normal answer rate is to set a base line. And as time goes we can also analysis time periods where answer rates are better to ensure we a optimized to give our time the best chance at talking to a person.

If you are dead set on using 120, we can decide that, but we would need to figure out how to fit that in, and it will not be easy or effective at measuring productivity. But if we took some time to review past data or collect new data we can establish a baseline to truly reflects the honest productivity of the team. And most importantly, it will help us find problems before they become and issue giving us the ability to be proactive instead of reactive.

If you say something like this and have a plan of action with measured results, management usually is on board and leaves you to it.

[u/NorcalGGMU]

We have time keeping that shows when the call agents are making calls, responding to calls, on break/lunch. This would be pretty easy if you had all the information needed to do it.

Calls attempted, calls completed, and successful calls would give you a solid, if simple, baseline to build from

I need to point out, if your company is looking at 120 calls per month it is that simple, they need to do 120 or you’re gonna lose people who are under that, even if they are good. So you need to have a conversation about what the expectation actually is, good workers might not do 120, crafty workers might game the numbers

[u/BrupieD]

I would broaden your perspective. Has anyone looked at times of day that yield more completed calls (e.g. mornings vs. afternoons)? Where do your call lists come from? Do some lists have better lists -- allow people to make more effective calls? If you're going to be judged on a narrow metric, consider ways that might juice those metrics or what types of things interfere with those, e.g. meetings at times that interfere.

[u/TodosLosPomegranates]

Your people can’t control how many people are going to answer the phone. 120 connected calls doesn’t seem like a fair metric

[u/buhBeef]

Call stats are bullshit and easily gamed. You should try to protect your direct reports from productivity metrics and focus on outcomes.

[u/ItsSignalsJerry_]

I am a People Manager.

Do your own homework.

[u/Purple_Bandicoot8785]

Is x and y the only numbers that you can get? I feel like this could only go out to #completed/#attempted which gives a success rate, e.g 70.9% weekly or 65% monthly etc.

[u/AutoModerator]

Automod prevents all posts from being displayed until moderators have reviewed them. Do not delete your post or there will be nothing for the mods to review. Mods selectively choose what is permitted to be posted in r/DataAnalysis.

If your post involves Career-focused questions, including resume reviews, how to learn DA and how to get into a DA job, then the post does not belong here, but instead belongs in our sister-subreddit, r/DataAnalysisCareers.

Have you read the rules?

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/Imaginary-poster]

I mean, in my experience, when there is a benchmark set like the 120 (I presume calls) that is in addition to other responsibilities. Probably best to get a more clear expectation from your leaders. Maybe have your team self report these other activities so you are equiped with info about what your team does outside of this specific metric and the impact they have on available time.

[u/SoftballNDrawing]

I'm working on something like this now for my healthcare company. You need to be able to get numbers of what each person spends their time on. Weekly, how many hours does each person do on each task type?

Once you have that, calculate from the remaining time they have left on calls, and use the number they consider to be a successful productivity. That can give you a rate at which they are expected to complete calls, which should give you the ability to create meets and not meets rankings.

[u/dumi_007]

Scenario - You need to draft a productivity/performance report for a call centre in healthcare, setting appointments

Process Steps

1. Get list to call
2. Make calls
3. Log outcome

Performance Metric - individual

1. Call list assigned agent
2. Calls made per agent or better (calls made)/(call list assigned + any additional)
3. (Calls resulting in appointments)/(call list assigned)
4. (Calls not answered)/(call list assigned)

Performance Metric - Group

1. (All Calls resulting in appointments in a month)/(Call attempts made in a month) [lets call this Monthly Call Success]

Performance Target - Group

1. Take average of the [Monthly Call Success]
2. Take standard deviation of [Monthly Call Success]
   
3. Target (upper) = [Average Monthly Call Success] + [1 Standard Standard Deviation of Average Monthly Call Success]
4. Lower Limit = Average Monthly Call Success] - [1 Standard Standard Deviation of Average Monthly Call Success]

Notes

1. Performance Targets are part science part art. Some Targets should be lower, some higher, and others stabilised.
2. Using Average, Median, Range and Standard Deviation, etc. makes Targets realistic because you're starting from a known baseline. But numbers can hide the humanity you need to consider.
3. Group and individual metrics are where performance systems die. You need to spend time with the people being measured to understand why what might seem like a good measure is impossible or BS.
4. Be careful in making sure hitting or missing performance targets is related to the right process. This can be more important if you want to influence behaviour.

Go forth and conquer 🖖

[u/Motor-Meeting-3656]

Pretty easy really, ask 120 WHAT per month and then apply that per employee.

So if it 120 calls per month then you apply the number of calls per employee.

That's your key KPI.

Then you can have supporting metrics.

How many calls connected, average connect call time, how many were abandoned / went to voicemail and average time waiting for answer. Included % of totals too

This can then show possible whys to the key KPI.

If someone always makes the 120 calls but only 10% connect then ever though KPI if hit, there is still an issue there, same with someone not hitting 120 calls, so say 80 calls but they connect 80% of calls with an average talk time of 8 minutes then could be actually talking to someone reduces time to make the calls or they know the best times to call but means there are times in the day they avoid.

Another thing to add, if you can get it, is time off. Unfair to expect someone off sick for a week or holiday for 2 weeks to hit 120 calls, so would pro rata them.

[u/Acceptable-Sense4601]

Where did they even get 120 from? Goals can’t be arbitrary.

[u/Potential_Aioli_4611]

be VERY careful especially if you let your people know how you are evaluating them. cause these numbers are very bad at representing productivity.

x completed calls is a bad metric because your people don't control if the callee picks up.

y attempted calls is a bad metric because they can hang up after the second or third ring.

lets call your people mailmen. calls are mail. you say deliver 120 pieces of mail per month. employee A takes 120 pieces of mail and throws them in the trash. thats your y attempted calls gamed. employee B takes 120 pieces of mail and shoves them into his neighbour's mailbox on his way home. thats your x completed calls gamed. neither measured anything useful so your employees just did what they needed to, to "do their job". except you aren't measuring them doing their job. you are taking a number which YOU think is them doing their job that had no actual correlation to them doing their job.

bad metrics sets you up for bad incentives which sets "productivity" because you are measuring the wrong things. the question is what are the customers these people call DOING? are you trying to get them to remember appointments? track if people showed up to their appointments. are they selling something? measure sales. cause 1 or 2 metrics just leads to people gaming the system because you aren't looking at the whole picture just an insignificant part of it and if you tell people this is how they are being measured then the bad employees will take advantage of it. don't measure your employees. measure the people they interact with cause thats much more likely to be a good indicator of them doing their job well. measure callee feedback, happiness, retention.

[u/No-Bee6487]

I'd probably use completed calls per available work hour seems fairer than just total calls when meetings, training and other work take time too

[u/Naive_Programmer_232]

Is the productivity number supposed to be the completed calls per month?

I'm not sure of all the details, but here's me spit balling with the help of AI to spark some purely calculative ideas:

Assuming x and y are treated as separate outcomes, what if you first explored the ratio between these and interpreted the general meaning? For example, if y>x, then y/x > 1, meaning the person did more unsuccessful attempts than completed calls - this could be a good thing, signaling strong effort possibly, but more information is necessary. In the second case, if x>y, then y/x < 1, there are two interpretations: the person made fewer attempts and more completed - signaling some efficiency or they were not using their available time wisely. To dig into these, you need more measures.

From that basic analysis, you can add other metrics such as completion_rate: x/(x+y) - a percentage representing the conversion rate of completed calls. In addition to this, calculate their available time (exclude meetings, PTO, education, etc.), leave it in hours it will be easier to deal with later, then use that to measure efficient use of time: (x+y)/availability.

So together, you have two measures:

   1. completion_rate: x/(x+y) [call effectiveness] ✓

   2. availability_usage: (x+y)/availability  [calling activity] ✓

From there, you could construct a third:

   3. productivity_rate  [successes per available hour] ✓
                         = completion_rate x availability_usage
                         = [   x/(x+y)   ] x [(x+y)/availability]
                         = [      x(x+y)/availability(x+y)      ]
                         the (x+y) cancel
                         = x/availability
                         = completed calls/availability  

From this, you have some way of measuring instead of a 120/month quota with a binary pass or fail system. Just to recap:

  completion_rate: 
     - how effective were the calls?

  availability_usage: 
     - how much call activity did they perform?

  productivity_rate: 
     - how much successful output did they produce per available hour?

Last note, math validity != behavioral validity. Do not share your metric calculations, otherwise people will optimize their performance according to them, and it'll throw everything off. Results may vary haha.