In the 1990s, there was no New York City skyline without the Twin Towers. They hobnobbed with Lady Liberty and the Empire State Building on any souvenir plate or T-shirt. They flanked Superman or supported King Kong on movie posters. They were such markers of Manhattan as to be critic-proof.

The centrality and iconicity that made the World Trade Center a target — the biggest buildings in the biggest city in the U.S., two for one — gave many a focus for tributes in the wake of their destruction, 20 years ago, on Sept. 11. From memorial flowers and candles on the Brooklyn Promenade overlooking their absence, to calls for rebuilding, to the twin searchlight beams of the Tribute in Light, to the eventual form of Michael Arad and Peter Walker’s 9/11 Memorial — where the towers’ exact footprints are rendered as eternal voids — the Twin Towers were celebrated as symbols of strength.

But it wasn’t always this way. Both the towers and their architect, Minoru Yamasaki, were criticized upon their completion, with racist and misogynist language dogging their Japanese American designer. The Twin Towers, which at first seemed a career-making commission, ultimately sabotaged Yamasaki’s career and confidence, their long double shadow putting everything he designed after 1973 in the shade.

When architect Yamasaki was awarded the commission to design the World Trade Center in 1962, he had been running his own firm for a dozen years. He was 50 years old. While he had achieved fame within the profession for his work on the striking concrete vaults of the Lambert-St. Louis Airport (1956) and the modernist Pruitt-Igoe public housing project (also 1956) in the same city, he was not known for skyscrapers and, in fact, had yet to design a building higher than 20 stories.

Meanwhile, before the towers were even completed, Pruitt-Igoe was demolished, an act which read as architectural criticism by dynamite. That two of Yamasaki’s major buildings would end up as rubble, one by politics, one by terrorists, seemed like the last word. And yet critics’ and historians’ views of the towers, as well as views of Yamasaki’s reputation, have also undergone a series of transformations.

Even as he ascended the ladder of the American architectural profession, Yamasaki continuously encountered racism via multiple avenues. A Seattle native, Yamasaki moved to New York after architecture school, and initially struggled to find professional opportunity during the Depression. After the 1941 bombing of Pearl Harbor, he sent for his parents to prevent their internment, living in a three-room apartment with them, his new wife and his brother. His boss at Shreve, Lamb & Harmon had to write a letter of support for Yamasaki to continue to work on the design for a U.S. naval training facility on Lake Seneca during the war.

Afterward, following a move to Detroit, he was not allowed to buy a house in suburban Birmingham or Grosse Pointe, like colleagues including Alexander Girard and Eero Saarinen, due to unwritten racial restrictions. His work was consistently interpreted as Japanese and Orientalist, despite the fact that his favorite pointed arches owe much more to Venice, and his arcaded courtyards to Spain and Italy, than to any influences from East Asia.