A utility-maximization model for rats' willingness to work for water rewards

see it at the Society for Neuro-Economics Conference, Dublin, 2019

An animal must continually invest time and energy to obtain sufficient food and water, and to meet other immediate and long-term survival needs. Because time and energy are limited, considerable selection pressure would exist for animals to efficiently allocate their efforts, such that the effort directed toward each goal is sufficiently but not excessively energized. In the laboratory, rats will perform tasks to earn food or water rewards. It is well established that, given an alternative, rats choose options with larger rewards, and are willing to work harder for them. But little is known about the effect of average reward size on the total amount of work a rat is willing to do (the analog of labor supply), or the total amount of water the rat will consume (the analog of the market demand) at equilibrium.

We tested this in N=14 rats, which performed work to obtain water in a long-term closed economy (no other hydration source). The task was freely available 24 hours/day for months, over which the time of every unit of labor (trial) and quantity of every unit of earned water (reward) were recorded to ms and ul precision. The task difficulty was fixed, but the wage rate in ml/trial was varied. At any given time, only one reward size was offered, and the size of that reward was stable for weeks. We show that when wage rates were stably low, rats would do more total work per day -- even though this required investing more effort for smaller rewards. The simplest explanation of this would be that rats' demand for water is inelastic, such that they consume the same quantity of water per day regardless of its price in effort. But we find that rats' demand for water is surprisingly elastic: when the cost (trials/ml) is very high, rats consume enough for long-term health maintenance. But when the cost of water is low, rats will consume twice that amount or more. Utility maximization theory can account for both the back-bending labor supply curves and price elasticity of demand for water we observe in rats. We suggest an analytical form which fits all our steady state behavioral data.

We interpret the net marginal utility of the next trial as a time-varying signal in the brain driving moment-to-moment behavioral engagement. We suggest that this dynamic variable is represented in periventricular neurons in the brainstem. By experimentally measuring and constraining the temporal dynamics of work, we can expose multiple timescales affecting this motivated behavior. We propose an HMM model that switches between work and rest states depending on instantaneous marginal net utility, and show that this replicates stochastic temporal dynamics of the rat behavior.


(A) Marginal utility MU (left) and utility U (right) of water H (per ml, top row) and of labor L (per trial, second row) and the sum of these (bottom row), as a function of the number of trials performed, for different wage rates (ml/trial, colors). (B) The resulting isoutility curves predict (C) the observed back-bending labor supply (L* vs W) in data from rats.

Credits:

Mark Machina advised and helped derive functional forms for the utility equations. Neehar Kondapaneni did the initial work on the dynamic HMM model.