Smooth convergence away from singular sets