Let $O$ be the center of the circumcircle of $A_1A_2\cdots A_{2n}$.
Let $x$ denote the distance from $O$ to any side of $A_1A_2\cdots A_{2n}$ (since $x$ is constant for a regular polygon, there is no ambiguity.)
It is well known that $PH_i=2x$.
Let $M_1,M_2$ be the midpoints of $A_1A_2$ and $A_3A_4$, respectively.
Since $OM_1\parallel PH_1, OM_2\parallel PH_2$. Hence $\angle H_1PH_2 = \angle M_1OM_2=\angle M_1OA_2 + \angle A_2OA_3 +\angle A_3OM_2 = \frac{360^\circ}{n}$.
Similarly, $\angle H_iPH_{i+1}=\frac{360^\circ}{n}$ for any $i$ in modulo $n$. Also $PH_i=PH_{i+1}=2x$ , which implies all $H_iH_{i+1}$ are equal. Hence $H_1H_2\cdots H_n$ is a regular $n$-gon.