A school district surveys its families about a new schedule. Eleven hundred responses come back, the overall satisfaction mean lands at 3.8 out of 5, and the report writes itself: families are broadly satisfied. What the mean does not say is that families in one part of the district skipped the survey at three times the rate of everyone else, and they are the families the schedule change hits hardest. The 3.8 is not wrong. It is just not describing the district. It is describing whoever answered, and the people it no longer includes are exactly the people the decision is about.
This is not a rare failure. It is the ordinary condition of applied research. Nonresponse never falls evenly. Convenience samples drift from the populations they stand in for. And when a group comes back with only seven cases, the standard move is to fold it into "other" or drop it from the table, a decision about who exists in the findings, made silently by a software default. Every dataset stands in for people, and the distance between the sample and the people is where findings quietly go wrong.
The check that lives in the limitations section
Researchers know this. It is why limitations sections exist. But notice where the knowledge lives: at the end of the paper, in prose, after the analysis is done and the tables are built. The software that ran the analysis never asked. Statistical packages have been built for decades around the mechanics of computation: get the data in, run the test, print the table. Who the data represents was left to the researcher's judgment, at the exact moment in the workflow when judgment is busiest with everything else.
The result is a strange gap. We have software that will flag a violated normality assumption in a heartbeat, and nothing that flags a missing community. A skewed residual gets a warning. A skewed sample gets a footnote, if someone remembers.
What the machine could be doing
None of this is hard to compute. Whether nonresponse falls evenly across groups is a table. Whether a sample's composition matches population benchmarks is a goodness-of-fit test that has existed for a century. The reason these checks are missing from the workflow is not difficulty. It is that nobody built the workflow around the question. A modern Mac can run every one of these checks in the time it takes the file to open, which means the only real decision is whether the software treats "who is in this data" as an analytical question or as sentiment for the acknowledgments.
Where we put the question
We built it into the Data Quality stage of Quanta, before any test is trusted. Missingness by group shows whether nonresponse fell evenly or concentrated in one community, with meaningful gaps flagged. Sample vs population takes the benchmarks you enter, tests your sample's composition against them, and names which groups are over- and under-represented. Grouped descriptives keep every group visible: a group under ten cases gets flagged and marked to be read descriptively, never silently merged away. The district survey above stops being a clean 3.8 the moment the missingness card renders, because the uneven nonresponse is sitting right there, before the first test runs.
The same position runs through the other tools, because the human context is not only a sampling problem. In Quala, the quote bank preserves voices verbatim so themes stay anchored to what people actually said, codes carry definitions rather than just labels, and the audit trail records how every interpretation was made. In MM Studio, joint displays hold a number and a narrative side by side so neither erases the other, and every integrated claim carries an evidence-strength rating that says how hard it can be leaned on. Across all three, ReliCheck Intelligence suggests and never auto-applies, because reading meaning in human data is human work.
These checks ship in the base apps, not a premium tier. Who your findings describe is not a premium question.
What changes for the study
The honest version of the district report reads differently. Satisfaction averaged 3.8 among respondents, nonresponse concentrated in the neighborhoods most affected by the change, and the findings should be read as describing the families who answered. That paragraph costs one glance at a card and it changes what the school board does next. It might mean a follow-up effort in the under-heard neighborhoods. It might just mean a decision made with open eyes. Either way, the people who did not answer are back in the story, which is the only place findings about them belong.
The full statement of how we think about this, across all three apps, is at relicheck.com/human-context.
ReliCheck Quanta is a native Mac statistics app for the social, behavioral, education, and health sciences, with representation checks built into Data Quality and a published validation record. Free 30-day trial at relicheck.com/quanta.