The conversation ended
👋 Welcome to my paid subscriber-only edition of Empathy Engine (🔒 Leader’s Dispatch). Each week I build evidence-forward tools for product leads who need to say no, defend tradeoffs, and lock in decisions before they get rewritten later.
Maya Chen walked out of the one-on-one feeling good about it. (Maya and Dev are fictional, built to illustrate this series.)
She had prepared for two days. Dev, a senior analyst on her team, had been sending customer-segment forecasts to stakeholders before checking the assumptions underneath them. She brought three dated examples and one ask: confirm the source definitions before any draft leaves the team.
Dev asked a sharp question about the second example, then said, “That makes sense. I’ll check the definitions first.” Back at her desk, Maya typed two words into her notes: landed well.
Research Binder: the receipts, methodology notes, and source boundaries are compiled at the bottom of this post.
Eleven days later, the next forecast arrived. The assumption she had flagged was still in it. A new unchecked one sat in the section Dev had reworked.
On the drive home, Maya replayed the conversation, hunting for the sentence that went wrong. But the replay only reviewed the conversation. The question she needed was simpler and harder. What changed? That question has four parts:
The message. What was said?
The reaction. What did each person report?
The work. What happened afterward?
Other explanations. What else could explain it?
“Landed well” answered none of them. It recorded Maya’s impression of the meeting. How it felt and what changed are separate questions.
A finished meeting gives you a date, not the rest of the answers. Look at comparable work afterward, and where evidence is missing, write Unknown.
When someone agrees with me, I still need to understand what they think we agreed to.
A positive average, widely different results
Before going further: feedback is not the villain of this series.
In 1996, Avraham Kluger and Angelo DeNisi published a meta-analysis in Psychological Bulletin covering 131 papers and 607 effect sizes. Each effect compared performance with feedback against performance without it. The studies were published before 1996, and most were laboratory research.
On average, feedback helped: the weighted average effect was d = 0.41. That is a standardized effect size, not a 41% improvement in anything.
And over 38% of those 607 effects were negative. “Negative” here means the feedback group performed worse than the no-feedback comparison group. It does not mean an unhappy reaction, or a drop from someone’s own earlier performance.
Think of d as a shared ruler for comparing results across studies. It measures how far apart the feedback and no-feedback groups performed, using a common scale. A d of 0 means no difference; d = 0.41 means the feedback group did better on average, not 41% better. But over 38% of the effects went the other way. The average doesn’t tell us what will happen after one feedback conversation.
Both facts are true at once, and neither forecasts Maya’s conversation. The 38% is a share of effect sizes from mostly older laboratory research, not a failure rate for today’s one-on-ones.
The tail wasn’t a fluke. When the authors removed outlying data, about a third of the remaining effects were still negative. They also found that, across studies, messages coded as discouraging showed lower average effects. They interpreted that as attention shifting from the task toward the self. Attention itself was never measured, so treat it as a theory about a pattern, not a window into Dev’s head.
The practical move is simpler: check what happened in the work, and keep missing evidence Unknown.
Two checks, not one
Maya had a record of her impression. She did not have Dev’s account, and she had his reaction only as she remembered it: a sharp question, then “That makes sense.”
Separate doesn’t mean unrelated. In one field study of 145 leaders receiving 360-degree feedback, more negative immediate reactions went with worse ratings from others about a year later, after accounting for where each leader started. One study, ratings only, an association rather than a cause. Reactions matter. They just aren’t the same measurement as the work.
Accounts diverge, too. In several studies, people receiving negative feedback judged it as less accurate than the people who gave it. That is disagreement, not proof that either side is wrong. Someone can leave a meeting feeling respected and still be unclear about what was supposed to change.
A good feeling does not establish a better result, and a tense meeting does not establish a worse one. Describe the recipient’s reaction in their own words, without labels like defensive or unmotivated. Examine the work against stated criteria. Neither check establishes the cause.
When Maya asked for Dev’s account, it was specific: “I thought the definitions check was for anything going to the board. This one was an internal draft.” That is a different understanding of the same ask. Both accounts go on the page. Neither has to win.
Was the message accurate?
Then the question that can feel settled once you’ve prepared carefully: was the message accurate? Preparation measures effort, not correctness. When Maya reopened her three examples, two held up against the files. The third did not. That assumption had been checked, and the check was documented in a tab of the shared model she hadn’t opened. Dev knew, and he hadn’t mentioned it in the room. Two examples still support her concern. Whether they show a broader pattern, three documents can’t say.
Scullen, Mount and Goff studied developmental multi-rater ratings of managers in two large samples. Their model attributed 62% of rating variance in one sample and 53% in the other to idiosyncratic rater effects, meaning patterns specific to the person doing the rating. The managers’ measured performance accounted for 21% and 25%.
Those percentages describe variation in ratings, not feedback conversations, and they don’t say what share of ratings were wrong. Rater-specific patterns can include valid differences in what people observed. Calling all of it bias would go beyond the finding.
What it does support: a rating is not a direct reading of performance, and neither is a manager’s summary of a draft. Make the judgment inspectable: the work, the criteria and both accounts. That is how Maya’s third example fell apart.
What else changed?
Maya’s first conclusion was that the feedback didn’t work. Maybe. But Kluger and DeNisi excluded simple before-and-after comparisons from their analysis because of serious threats to internal validity. A sequence tells you when two things happened, not why.
So Maya listed what else had moved in eleven days, separating what she observed from what she guessed.
The task. Observed: the new forecast covered a segment with messier source data. Unknown: whether that made the check harder.
The leader. Observed: Maya canceled their mid-week check-in while traveling.
The conditions. Observed: a stakeholder pulled the deadline forward three days. Reported by Dev: source-file requests go through another team’s queue. Unknown: how long his took.
Other changes. Observed: her deputy had reviewed Dev’s earlier drafts. Maya reviewed this one herself.
Dev’s part. Observed: he didn’t raise the problem with example three, and the new unchecked assumption was his.
