Next: Policy Sampling
Up: Evaluating One Policy With
Previous: Evaluating One Policy With
Importance Sampling
We have two sources D1(x) and D2(x) that produce differnt distributions.
We compute expectation of a function F(x) on one source while sampling the other source. The
expectation of F(x) with respect to distribution D is the sum of products of all values of X
with the probability that D assigns that value. In our case:
ED2[F(x)] =
=
=
ED1[(
)F(x)]
EXAMPLE 1
Input:
-
F(x) = k.
-
-
![$D_{2} = Prob[k] = \frac{3}{4}(\frac{1}{4})^{k-1}.$](img11.gif)
Computation:
- We find expectancy
ED2[k] from samples of D1.
- We check the equation by computing
ED2[F(x)].
=
=
=
ED1[k]
- One of the problems in importance sampling is the variance.
In this case.
=
=
= ![$\infty$](img20.gif)
Yishay Mansour
2000-01-07