“The origins of segregation live much earlier” than discriminatory public policies, says Colin Gordon, author of the book “Patchwork Apartheid.”