We develop a framework for analyzing a finite-horizon investor's asset allocation problem in the case where not all risks can be hedged by financial instruments and markets are therefore incomplete. We assume that the agents assess their utility relative to a (stochastic) benchmark. The investment strategies we study are closedform approximations to the optimal solution. Using duality methods we compute explicit upper bounds for the optimality gap of our approximating solutions. In the examples we present, our approximations are very close to the optimal solution.