Program Sensing
Can frontier AI decide what deserves leadership attention? An original evaluation of model judgment on portfolio prioritization, measured against experienced program leaders taking the same test, under the same rules.
In every complex organization I've worked in, leadership time is the scarcest resource. Filtering information up is a perpetual challenge: However clear the escalation process, leadership attention stays contested. Endless status conversations, side channels, and meetings that end before the things that mattered got discussed.
As organizations push to become flatter, the question gets sharper. To what degree can AI complement or replace human coordination mechanisms?
Program Sensing tests one concrete version of that question: Whether AI can reliably prioritize the leadership-intervention needs of a complex Tech portfolio. The setting is the weekly portfolio review, the one hour where a VP's attention gets allocated. Real-time, AI-driven prioritization could give leadership faster, sharper visibility into what actually needs them. If AI delivers. That's what this evaluation measures.
A realistic instrument
You act as Chief of Staff to a VP of Engineering at a Fortune 500 company: Weekly status updates from a 30–50 program portfolio, and one decision. Which programs get the VP's limited time, and in what order? Everyone gets the same fixed packets; the prioritization rules stay visible throughout.
An expert human baseline
The baseline isn't crowdworkers. It's experienced program, portfolio, and engineering leaders taking the identical test: No AI, no internet, no outside help. Pooled, their calls form the human standard the models are measured against.
Frontier models, same rules
Leading models take the same instrument with the same standing rules and no web access. Nobody gets hints, and no answer keys are published while the study is in the field.
Open results
The report publishes here: Per-dimension analysis plus a pseudonymized open dataset of the human responses. Participants can compare their own score against their peers, and against every model tested.
I spent a decade deciding what deserved leadership attention across a $30M+ technology portfolio. This is the evaluation I wished existed: Grounded in the judgment the job actually requires, not abstractions. I designed the study, authored the instrument, and built the evaluation harness end to end.
At publication, the methodology report and an open subset of the data ship here, with enough to check my work, rerun the numbers, and fork the approach for your own portfolio.