We consider a Zipf--Poisson ensemble in which $X_i\sim\poi(Ni^{ -\alpha})$ for $\alpha>1$ and $N>0$ and integers $i\ge 1$. As $N\to\infty$ the first $n'(N)$ random variables have their proper order $X_1>X_2>...>X_{ n'}$ relative to each other, with probability tending to 1 for $n'$ up to $(AN/\log(N))^{ 1/(\alpha+2)}$ for an explicit constant $A(\alpha)\ge 3/4$. The rate $N^{ 1/(\alpha+2)}$ cannot be achieved. The ordering of the first $n'(N)$ entities does not preclude $X_m>X_{ n'}$ for some interloping $m>n'$. The first $n"$ random variables are correctly ordered exclusive of any interlopers, with probability tending to 1 if $n"\le (BN/\log(N))^{ 1/(\alpha+2)}$ for $B<A$. For a Zipf--Poisson model of the British National Corpus, which has a total word count of $100{ ,}000{ ,}000$, our result estimates that the 72 words with the highest counts are properly ordered.

Categories: stat.ME math.ST stat.TH

Correct ordering in the Zipf-Poisson ensemble