Training doesn’t lead to experience it only acts as a baseline to what should and should not happen. With something as complex as tax credit claims then tacit domain knowledge is going to be key to the decision making process. So at the start of the project, before staff get to use any system, the domain experts should be able to define the process and the expected decisions any system should be making. With something like the tax credit system there are multiple sources of data on which a decision is made.

Even with tacit domain knowledge not every case handled is going to pass through a system without error. Some will be needed to be routed to a domain expert for analysis for a final yes/no on the claimant. The key to issues like this is how the knowledge is put back into the system to enable the team to make better decisions in the future.

A Case For Machine Learning?

Well I believe there is here, what I don’t believe though that it’s a black and white solution. Here machine learning should aid the process and make a recommendation on the final decision, ultimately though it’s up to a qualified employee voice to make the final call based on the data presented.

Since the introduction of tax credits in April 2003 there will be a trail of data and decisions, therefore there is historical data to train a system to make decisions. Like I say, this sort of system shouldn’t be making the final decision but merely aiding those who do based on previous cases.

With this sort of scenario it’s going to based on claimant documents so scanning and text mining is going to be playing a key part. Couple this with a decision trees and you have the basis for a decision making process.

The most important part of the mix though is not to have the system guess when it can’t decide but rather making the domain expert decide and thus enabling the system to learn from the new experience. This is important especially in borderline cases where the final decision is not clear.

The advantage of using machine learning in the training stage, along with a domain expert, is that at a guess 66% of claims would fall within the average (i.e. 1 standard deviation from the mean) it’s the outlier cases that take time on analysis.

Human Factors

Let’s not beat around the bush here. This is call centre staff being put under pressure to perform.

“They also say they are being encouraged to hit a target of making 20 decisions a day, or about three an hour, on whether to stop, amend or leave a tax claim unchanged.”

Automation can help in such cases and a trained system can certainly lead to better decision making once the machine learning training has been performed and evaluated.

Balancing the Cost Of Machine Learning

Machine Learning systems don’t come cheap and they also take time to develop, train and refine before being let out in the real world. The implementation of process also takes time and money to implement. The costs should only strengthen the benefits of the final process. I say all this but there is a “but”…

“The company is not paid on the number of letters issued, but on the basis of savings to public finances arising from correcting tax credits claims that are incorrect.”

It seems to me though that designing a system that would decide based on prior evidence and training may not work to the advantage of Concentrix (or any other company doing this work under these contractual measures) as the payment is based on money saved to the public purse. It would make sense for such a system NOT to exist in this case, this is a real shame.

Summary

There is a strong case here for a process driven algorithmic system to deliver aid in the final decision making process. Removing part of the analytical side away from staff means they can spend time with the customer, final decisions that require more analysis then are sent to a domain expert, the knowledge gained from that case is then fed back into the system for learning. The more cases the better the learning patterns.

Issues do arise though when the bottom line is based on money saved and not the amount of customers processed. This leads to rules being relaxed in the favour of the service company (though there is no evidence for that here may I add).