NBA Shot Selection: Offline RL on Real Tracking Data

I built this because I love basketball. I wanted to see whether a machine could learn shot selection the way the analytics era rewrote it: stop settling for long two-pointers, live at the rim and the three-point line. It is a passion project I pushed well past what it needed to be, and the part I find most interesting starts before any model runs, with the data. None of it comes ready to use. I stitch it together from three separate public sources.

During the 2015-16 season the NBA had SportVU cameras in every arena tracking where all ten players and the ball were about 25 times a second, and the raw logs are mirrored on a public GitHub repository, so I can see exactly where everyone stood at the moment of every shoot-or-pass decision. On their own those positions don't tell me what happened next, so I merge in a second public source, the play-by-play log: a plain record of who shot and whether it went in. A third source, the NBA's official stats site, gives each player's shooting accuracy broken down by distance. Merging the cameras with the play-by-play is the move that matters: it lets the agent see the whole scene at a decision and be graded on whether the real shot went in, so it learns from reality instead of from a guess.

Shot selection is one of the most consequential decisions in professional basketball: at each step of a possession the ball-handler must decide whether to attempt a shot or pass, conditioned on defender positioning, teammate availability, shot-clock pressure, and their own shooting ability. The question this project asks is whether a learned policy, trained on real NBA tracking data, can outperform the actual decisions NBA players made.

This is an offline RL problem with three pathologies that break standard DQN: (1) the agent learns entirely from a fixed dataset and cannot explore, (2) the four pass actions are variable-identity: the same action index points to a different teammate at every timestep, depending on who is closest to the ball-handler, which corrupts standard Bellman backups, and (3) shoot is a terminal action that occurs once per episode while passes can occur 2–10 times, creating asymmetry that interacts destructively with conservative offline-RL penalties like CQL. As I'd find out, the real adversary turned out to be a fourth pathology: a reward and an evaluation that quietly graded the agent against its own assumptions.

The first version of this project produced a headline I was proud of: a shot-quality metric (EPSA, expected points per shot attempt above league average) on which the Dueling agent scored +0.273 versus +0.044 for NBA players' actual choices, roughly 6× better. It was the kind of clean, citable number that ends a slide deck. So I tried to break it instead.

Two things broke:

The metric was circular. EPSA scored shots using the same contest-adjusted Expected-Points-Value (EPV) proxy that the agent was rewarded on during training. The agent wasn't being measured against reality; it was being graded by the very function it was optimizing, so a high score was nearly guaranteed and couldn't falsify the policy. A circular metric can't be wrong, which is exactly what makes it worthless.

The data was corrupt. Auditing the 78-dimensional possession features surfaced two silent bugs: distance-to-basket was measured to the wrong basket on a chunk of possessions, and the "terminal" state was the last 2 Hz tracking sample of the possession rather than the actual shot release. When I re-scored the celebrated agents on real logged shot outcomes, the verdict inverted: they anti-selected, endorsing shots that went on to score worse than the players' own choices (−0.09 PPP). The 6× win was an artifact, top to bottom.

Superseded

Rather than patch the symptom, I rebuilt the foundation. The possession segmenter was rewritten to re-derive the true shot-release frame using a direction-free nearer-basket distance, chosen over two alternative "join" fixes specifically because it is the only one that is leak-free. I regenerated all 636 games into a corrected substrate (processed_possessions_v2) and gated it on basketball sanity: shots beyond 30 ft at 4.5%, three-point rate ~30%, FG% monotonically decreasing with distance, and 100% agreement between the derived made/missed flag and the play-by-play log.

The reward was regrounded on reality. The circular PBRS/EPV proxy was replaced with real points at the logged terminal shot, minus a constant selectivity bar so the agent is penalized for endorsing below-average looks. Critically, passing earns no shaped teammate-EPV bonus; its value flows only through the Q-bootstrap from whatever terminal shot it leads to, which removes the last circular shortcut. Then I retrained six configurations from scratch on the clean data.

The variable-identity action problem is solved by scoring each teammate individually through a shared encoder rather than mapping Q-values to fixed action slots. Two shared encoders (phi_defender and phi_teammate) embed each of the 5 defenders and 4 teammates independently. Defender representations are mean-pooled into a defender-context vector; teammate representations are kept per-entity. The shoot head takes the ball-handler context + pooled defender context and outputs Q(shoot). The pass head is shared across teammates and outputs Q(pass_i) per teammate, ensuring permutation equivariance and consistent Bellman targets regardless of which physical player occupies slot i.

The Dueling architecture decomposes Q(s,a) = V(s) + A(s,a) − mean(A). The standard mean-subtraction biases the network against shooting when there are four pass actions and one shoot action: the mean is dominated by the pass group. The fix is type-aware advantage normalization: pass advantages are mean-centered within the pass group only, while the shoot advantage is left unnormalized.

Double DQN target computation to mitigate overestimation bias. Hard target-net sync every 100 episodes (~570 gradient steps) rather than soft Polyak updates, chosen because soft updates destabilized the 450-player ID embeddings: most players appear rarely, so continuous small weight changes prevented the embedding table from converging. 100K-transition replay buffer with terminal oversampling to compensate for shoot-action rarity. Adam at 1e-4, Huber loss with gradient clipping at 1.0. Linear epsilon decay from 1.0 → 0.01 over the first 40K episodes; agents trained out to the full 100K.

The honest follow-up question: is the agent doing anything cleverer than "the three-pointer is worth more, so shoot more threes"? Most of the +0.2 gap is that three-point-value effect, which a one-line "shoot the best threes" rule also captures. But measured against that exact heuristic, 4 of 5 configs beat it by a statistically significant margin (bootstrap CI excludes zero): the high-selectivity agent by +0.094, the two plain outcome agents by ~+0.06, the model-reward agent by +0.028; only one ties. So there is a real, learned shot-selection edge beyond the trivial rule: modest, ~+0.06 PPP, but genuine and verified.

Selectivity, not volume, is the lever. The strongest agent (high selectivity bar) shoots on only 13.4% of decision points, an "only fire on elite looks" policy that lifts realized PPP to 1.347, while the model-reward agent shoots 38.5% at +0.206. Different points on the same selectivity frontier, all above the player baseline.

A standalone deliverable came out of the same data. Separate from the RL agent, I trained a supervised shot-quality model that predicts make probability from the pre-shot state. It reaches AUC 0.672 from tracking features alone and 0.733 once play-by-play context is added, a clean, verified model that's useful on its own as a shot-difficulty estimator.

The two non-obvious engineering wins were the per-entity architecture and the type-aware advantage normalization, corrections to subtle failure modes a standard implementation would silently learn around. But the real lesson was the audit. The most dangerous result in machine learning is the one that looks great, and the temptation to ship a single clean headline number (like the 90.1% accuracy I'd see clinical-ML papers cite; see the THA Pain Prediction case study for that one) is precisely when you should try hardest to falsify it. A metric that reuses the model's own assumptions can't be wrong, which is exactly why it tells you nothing.

Killing my own +0.273 was the most valuable thing I did on this project. What survived the audit is smaller and far more defensible: on real outcomes, a regrounded agent that out-selects NBA shot decisions by +0.19–0.32 PPP, with a modest but statistically real learned edge over the best simple heuristic, plus a verified shot-quality model. Real, scoped, and honest: the result, not the hype.