Chapter 01
Reliability Explained
Reliability is not accuracy; it is repeatability. An instrument can be reliably wrong. Reliability is a necessary, not sufficient, condition for trust.
GI-003Assessment Science™Foundational Publication25 min read
Reliability in Board Assessment
Why Consistency Matters When Measuring Governance Intelligence™
Executive Summary
Reliability is the property of an instrument to produce the same reading for the same underlying reality. Without it, no other virtue — validity, actionability, benchmarking — can hold.
This monograph explains the three layers of reliability that matter most in board assessment: internal consistency, inter-rater reliability, and longitudinal stability.
Board Chair Brief
If your board's assessment scores swing materially between cycles without a corresponding change in board behavior, reliability — not the board — is the likely culprit.
Chapter 02
Internal Consistency
Items intended to measure the same construct should move together. When they do not, the construct is either poorly defined or poorly instrumented.
Chapter 03
Inter-Rater Reliability
Different directors observing the same board should — within reason — produce convergent signals. Divergence is data; unstructured divergence is noise.
Chapter 04
Longitudinal Stability
Absent real change in board behavior, scores should be stable across cycles. Stability across time is the strongest test of instrument reliability.
Chapter 05
Reliability Pyramid™
BEACON models reliability as a pyramid: internal consistency at the base, inter-rater reliability in the middle, longitudinal stability at the apex. Each layer depends on the ones beneath it.
Chapter 06
Board Examples
Two boards, similar in composition, produce very different reliability profiles when their assessment instruments differ in item construction and rater calibration.
Interactive Graphic
Reliability Pyramid™
Three dependent layers of reliability. Each apex depends on the layers beneath it.
- Internal Consistency
- Inter-Rater Reliability
- Longitudinal Stability
BEACON Perspective™
Reliability is invisible when it is working and catastrophic when it is not. Its absence is the most common — and most under-diagnosed — cause of board assessments losing credibility.
Board Reflection™ · Discussion Questions
- Would we trust this year's scores enough to change committee assignments based on them?
- How much of our year-over-year variance is instrument noise?
FAQ
- What is an acceptable reliability threshold?
- Conventional psychometric thresholds (e.g., Cronbach's alpha ≥ 0.80 for internal consistency) are a useful floor, but boards should demand documentation, not just a number.
References
- Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika.
- Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations. Psychological Bulletin.
Download Center
Distribute this monograph
Executive-ready formats for board packets, briefings, and citation.
GI-003 · Foundational Publication
Related Publications
More from Volume I
Primary CTA