The couple decided to invite a black man