Research
Measurement, and the decisions people make with what we measure
Two lines of work — one on how behavior is measured, one on how people act on model-generated evidence — and the software that makes both of them possible to study.
How I got here
I spent three and a half years at Amazon Health building systems pharmacists use to decide whether a prescription is correct. The engineering question was how to compare four kinds of evidence quickly and reliably. The question that stayed with me is a different one: what does a person need to see in order to judge that evidence well, and how would we know if they were judging it badly?
Alongside that, I have worked on large-scale education research with Dr. Rongxiu Wu and coauthors — multilevel models of PISA data, and interpretable machine learning on survey data about STEM career interest. That work is where I learned how much rests on the measure. In the PISA 2022 study of pre-sleep digital self-regulation, the central construct is captured by a single self-report item, and the paper’s own limitations section proposes adding behavioral measures such as smartphone logs.
So I built one. PhoneMood records phone use, samples mood in context, and keeps the missing observations visible in the analysis. Building research tools is not a detour from research for me — it is how I get to ask the question at all.
Two lines of work
Behavioral measurement of self-regulation and wellbeing
Survey items, device records, and momentary reports rarely agree about the same behavior. I work on what that disagreement does to the conclusions we draw about self-regulation, sleep, and digital wellbeing.
- Question
- How closely do reported and recorded behavior correspond, and how much do our substantive findings depend on which one we measured?
- Work so far
- A PISA 2022 multilevel study of pre-sleep digital self-regulation and emotional control across 19,779 students in 996 schools in Canada and Hong Kong (preprint), and PhoneMood, an Android instrument that pairs recorded phone use with usage-triggered mood reports and within-person analysis.
- My contribution
- On the PISA study I am credited with methodology, software, formal analysis, and investigation. On PhoneMood I am the sole author: research questions, sampling protocol, Android implementation, statistical engine, and reproducible export.
- Doctoral questions
- When a retrospective questionnaire and a device record disagree about the same behavior, which one relates to the outcome we care about — and does the answer differ by who is reporting?
- Usage-triggered sampling cannot observe the moments it does not trigger on. Which sampling rules recover brief, repeated checking, and how far do within-person estimates move when the rule changes?
- Unanswered prompts and monitoring gaps are unlikely to be missing at random. What should an analysis of sparse, irregular momentary data assume, and how sensitive are the conclusions to that assumption?
Human–AI decision making in consequential workflows
I have built the kind of system where a model produces evidence and an accountable expert decides what to do with it. Evaluating the model alone leaves out most of what determines the outcome.
- Question
- When a system presents model-generated evidence to an accountable expert, what should it show, what should it withhold, and how do we evaluate the person and the model together?
- Work so far
- A multimodal pharmacy verification copilot with specialist agents, structured cross-checks, gating, and escalation, and PharmacyLite, the review platform where pharmacists act on that evidence and the record of their decision is kept.
- My contribution
- I built pipeline components and comparison logic for the copilot and developed its checking and escalation controls with pharmacists and applied scientists; I architected the review platform across its interface, APIs, and workflow orchestration.
- Doctoral questions
- Decision-support systems are usually evaluated on their own outputs. What does a joint evaluation of the model and the human decision look like when the person, not the system, is accountable for the result?
- An inconclusive comparison and a confident mismatch call for different actions. How should a system represent uncertainty so that experts neither over-trust it nor discard it — and does that change as they work with the system over months?
Two further interests follow from this work but do not yet have a project behind them: how AI-generated feedback changes what a learner does next, and how people direct their own learning when a system is always available to answer. I list them as directions I would want to develop, not as work I have done.
Contribution
What I can contribute to a research group
Each of these is something I have already built, not a capability I expect to grow into.
Mobile and web research prototypes
Build the app or site a study runs on, including the parts that are tedious: permissions, background collection, prompts, and consent screens.
PhoneMood
Behavioral data collection pipelines
Turn a sampling protocol into software that records what happened, what was asked, what was answered, and what was missed.
Instrument architecture
Reproducible processing and analysis
Construct the analysis sample with the inclusion rules written down, and export the data, model inputs, and estimates together.
Methodology
AI integration and evaluation systems
Integrate multimodal models, build the interface that presents their output, and set up the evaluation that says whether it is good enough.
Verification copilot
Backend services and research infrastructure
Stand up and operate the services, storage, and deployments a lab needs, without asking someone else to keep them alive.
Cost estimation service
Quantitative modeling and error analysis
Fit multilevel and within-person models, compare predictive models, interpret them with SHAP, and check where they break.
Publications