Minimal algebras with respect to their $*$-exponent