BUDAPEST–Hungary’s Reformed Church said it will look into the conduct of one of its clergymen after he unveiled a bust of Miklos Horthy, the country’s leader who in the 1920s and 1930s presided over the adoption of anti-Jewish laws and was a close ally of Nazi Germany.

The unveiling of the bust Sunday marked the 75th anniversary of a wartime accord with Nazi Germany which allowed Hungary to get back part of the territory it lost under the 1920 Treaty of Trianon, a peace agreement signed after World War I....