EIPO and curiosity

Wed, 03 Jun 2026 00:00:00 +0000

This LinkedIn Post poking fun at the #GreatAIReplacement reminded me of this now ancient paper (2022) Redeeming Intrinsic Rewards via Constrained Optimization on mathematical approaches to governing the balance between exploration (curiosity) and exploitation (using known approaches). The paper covers their approach to optimizing model training to play various ATARI games. The Extrinsic-Intrinsic Policy Optimization (EIPO) algorithm they proposed did seem to improve over other reinforcement algorithms.

It’s interesting to think of curiosity as a mathematically definable property. I tend to view it in a slightly less quantifiable frame. This essay CurioCuriosity Is No Solo Actd has an interesting take that I like, proposing that curiosity is communal. There’s a quote in the piece: “‘the eternal convergence of the world within any one thing,’ writes Carl Mika, such that ‘one thing is never alone and all things actively construct and compose it.’ From this perspective of deep holism, talk of knowing any one thing is ‘minimally useful.’ As such, knowledge is not properly propositional but instead procedural; it is less concerned with knowing what than with knowing how. And its wisdom lies in ‘sharing’ more than ‘stating.’”

Opinion on It Might be Working

EIPO and curiosity