Log-space evaluation and subtracting the maximum.

Care must be taken when the random variable lives in a high dimensional space. For example, the naive importance sample estimate E_q[f(Z) p(Z) / q(Z)] involves the ratio of two terms p(Z) / q(Z), each of which must have tails dropping off faster than O(|z|^{-(k + 1)}) in order to have finite integral. This ratio would often be zero or infinity up to numerical precision.

For that reason, we write

Log E_q[ f(Z) p(Z) / q(Z) ]
= Log E_q[ exp{Log[f(Z)] + Log[p(Z)] - Log[q(Z)] - C} ] + C,  where
C := Max[ Log[f(Z)] + Log[p(Z)] - Log[q(Z)] ].


The maximum value of the exponentiated term will be 0.0, and the the expectation can be evaluated in a stable manner.