Reference file

Evidence and analysis

evidence-and-analysis.md

Evidence and analysis

Rule out the system, gather four kinds of evidence, then read the 360 correctly.

Evidence and the system check

A 360 measures perception. Leaders are judged by what they do. World-class diagnostics combine four kinds of evidence and rule out the system before assessing the person.

Rule out the system first

Many apparent leadership gaps are system gaps. Coaching a leader around a broken system wastes the sprint and damages trust. Check the environment before the individual.

Layer Environment questions Individual questions
Information Does the leader have clear expectations, timely data, and feedback on results? Does the leader have the knowledge and skill?
Resources Do they have the tools, budget, headcount, and time the role needs? Do they have the capacity: time and energy not consumed elsewhere?
Incentives Do rewards and consequences reinforce the behavior being asked for? Are they motivated to change?

Work the left column first. If the leader is told to coach but measured only on personal bookings, the gap is in the system. Fix it, then reassess the person.

The four kinds of evidence

Evidence What it shows Minimum
Time allocation Where the leader actually operates, against the stage standard 4 to 6 representative weeks of calendar and activity data
360 How the leader is experienced by each group around them Manager, 3 to 5 peers, 3 to 5 direct reports, 2 to 3 cross-functional partners
Direct observation What the leader does in the room One 1:1, one team meeting, one forecast or pipeline review; plus a sample of work product such as coaching notes, emails, or decks
Practice test How fast the leader learns One role-played situation tied to the likely headline gap, run twice with feedback between

Where sources agree, confidence is high. Where they disagree, the disagreement is a finding.

The practice test

The feedback-and-repeat test works for sitting leaders as well as candidates.

  1. Set up a realistic situation tied to the likely headline gap: a hard performance conversation, a coaching session with a struggling manager, a resource negotiation with a peer.
  2. Let the leader run it without interruption.
  3. Give two or three specific points of feedback.
  4. Invite questions.
  5. Run it again immediately.
  6. Score the change, not just the first attempt: listening, clarification, application, adaptability, self-awareness.

A leader who applies the feedback visibly is ready for an aggressive sprint. A leader whose second attempt is unchanged has shown that the gap is not knowledge.

The learning question

Ask every leader: "What is the most important thing you have changed about how you lead in the last year, and what triggered it?"

A specific answer with a trigger and a result shows an active learner. "Nothing major" or a vague answer means the leader has stopped developing, and that matters more than any single competency score.

Rater quality controls

  • Response floor. Report a rater group only with 3 or more responses, and only if at least 70% of invited raters responded.
  • Leniency check. Flag raters who give every item the top score. Weight examples over ratings.
  • Same raters. Use the same raters at baseline and at re-measurement.
  • Examples required. A rating without an example carries less weight than one with an example.

360 and gap analysis

Raters

Group Count Why
Self 1 Baseline for blind-spot detection
Manager 1 Role expectations
Peers 3 to 5 Influence without authority
Direct reports 3 to 5 Coaching, communication, trust
Cross-functional partners 2 to 3 Collaboration and change leadership

Report a group separately only with 3 or more responses and a response rate of at least 70%. Otherwise fold it into "others" to protect anonymity. Rater quality controls are in the section above.

Questions per competency

Ask each rater for a 1 to 4 rating and one specific example. Then ask two open questions across the whole survey:

  • What is one thing this leader should keep doing?
  • What is one thing this leader could do differently that would make the biggest difference to you?

The four reads

Read Rule What it means
Blind spot Self rates 1 or more points above others The leader does not see how this lands. If blind spots appear on several competencies, self-awareness becomes the headline gap regardless of other scores
Hidden strength Others rate 1 or more points above self Under-used capability; often the fastest lever in the plan
Overused strength Self rates 4 and others describe overuse behaviors A strength becoming a liability. Harder to fix than a gap, because it requires turning something down
Group split Leading-self and leading-others averages differ by 0.5 or more Development belongs on the lagging group, usually leading-self first

Read distributions, not averages

Ratings Average Story
2, 2, 3, 3 2.5 Consistent, moderate gap. Broad development need
1, 1, 4, 4 2.5 Two groups experience a different leader. Find out which raters scored low and why
4, 4, 4, 1 3.25 Strong overall with one damaged relationship. Often the real story

Always look at which rater group produced the outliers. Direct reports scoring low while peers score high is a different problem from the reverse.

Perception is the target

360 data is perception, not truth. It is still the leader's reality. If most direct reports experience the leader as unapproachable, the intention behind the behavior does not matter. The development target is what people experience.

Derailers under pressure

Strengths overused under stress become derailers. Name what the leader becomes when a quarter goes badly and what triggers it.

Derailer Looks like under pressure Common trigger
Controlling Takes over deals and decisions; managers stop deciding A miss in a quarter that matters
Avoidant Delays hard conversations; problems surface late Conflict with a peer or a top performer
Abrasive Sharp in meetings; bad news stops flowing upward Board or CEO pressure
Retreating Disappears into analysis and planning Ambiguity with no clear answer

Ask raters: "When things go badly, what changes about how this leader behaves?" The answers are often more predictive than any competency score.