It is not a good idea. Knowing that a conversation is monitored introduces a bias that makes your model practically useless. I don’t even know how can they measure performance decently. There is a reason double blind studies exist.

I know exactly what is going on here. A business school type of exec came up with a completely stupid idea. He did a great power point presentation to the board, decision was taken up bottom, a data science team agreed because “I am paid well above average salary, who give a f”, they deliver something with a dashboard for monkey execs.

Unfortunately it is so common in data science… After few years the exec will find out the model is useless garbage, will blame data science team, will do some “restructuring” and will go around exec conferences saying that “AI” doesn’t work. Meanwhile data scientist who implemented the original model already left for a better company

Edit. I checked, the guy is from a business school and was talking about an idea he has… They are so predictable…