https://stackoverflow.com/questions/582056/getting-list-of-parameter-names-inside-python-function

Seb,

Thanks for getting started on this!

My first response is something that you emphasized to Pablo earlier: It's not possible to distinguish between state variables and control variables, because the transition equations mean that today's choice of a control is mathematically identical to the choice of tomorrow's state. It seems that something Dolo/DolARK is going to need is that in every period the list of variables needs to be the same. But a given variable, at different stages, can be either a state or a control.

It IS necessary, at every stage (= subperiod in a cycle; = point at which a decision is made) every variable needs to be designated as either a state or a control. But that is properly accounted for in the definition and solution algorithm for the stage -- being a "state" or a "control" is not an intrinsic property of a variable.

Again, the simple example is the portfolio problem. As of the end of the period, assets are a state variable (having been determined by the consumption choice at the beginning). But, at the beginning, the problem can be formulated as making the choice of what $a$ to end the period with.

I think it would be useful to step back from trying to instantiate these ideas in code, and to do it more abstractly, as pseudocode.

With a given number of cycles n>0, each n would require something like your structure above, but the natural way to think about it is to identify each variable in the given subcycle as a state or a control. (The realization of exogenous/forcing/random variables would be treated, in this typology, as a state).

The problem is then defined by:

a payoff, which can is a function of states and controls
transition equations, which translate this period's states plus this period's controls into next period's states
a (discounted) value function for the next period

and in the Bellman formulation, the problem is to find, for any configuration of states in the current subperiod, the choice of controls that maximizes the sum of the payoff today and the resulting value tomorrow.

One point I want to be absolutely clear on: The most fundamental thing that needs to be kept track of is the value function. The decision rule is derived from maximizing the sum of today's value and the expected value from the resulting states. Decision rules are very useful objects DERIVED from the value function, but they are defined by their ability to maximize value. There are many cases where the problem cannot be solved without the value function.