Upon impact, the free surface of a solid metal may eject a cloud of fast and fine particles. Photon Doppler Velocimetry (PDV) is one of the optical diagnostics used to characterize these ejecta. Although the technique provides a direct way to estimate the particle velocities in the single scattering regime, it has been shown that multiple scattering cannot be neglected in real ejecta. Here we derive a model for PDV measurements starting from first principles of wave scattering. We establish rigorously the relationship between the specific intensity and the measured signal, as well as the radiative transport equation (RTE) that describes the evolution of the specific intensity upon scattering and absorption in a dynamic ejecta, including the effects of inelastic scattering and inhomogenities in the optical properties. We also establish rigorously the connection between the Monte Carlo scheme used for numerical simulations and the solution to the RTE. Using numerical simulations, we demonstrate the crucial contribution of multiple scattering to PDV spectrograms as well as the effect of statistical inhomogeneities in particle size distribution. These results could substantially impact the analysis of ejecta by PDV.