o 0Gf@ @sdZddlmZmZddlmZddlZddlm Z ddl m Z dRddZ d d ZdSd d ZddZddZddZdTddZdTddZdUddZdTddZdTddZdTdd ZdVd#d$ZdWd&d'Zed(kred)ed*gd+Zgd,ZeD]ZeeeqyeD]Zeeeqeeeeeegd-Z e !d.e "d/e #d0e$dd!d1Z%e &e%ee%e !d.e "d2e #d0e$dd!d3Z%e &e%eeee%d!e%e'gd4gd5gd6gZ(e()dZ*e()d!Z+ee*Z,ee+Z-ee(Z.ee*e+e(Z/ee+e*e(Z0edej1e2e*e+Z3edej1e2e+e*Z4ee*e+e(Z5ed7ee,e-e.e/e0e3e4e5ed8e'gd9ZeeZ6eed:ed;ed<ed=ed>ed?e*ed@d!e*dZ7edAe7ee7d!e7gZ8edBe8edCe8e7e9de/fedDedE:e7e,d!e9dFedGedHe'gdIgdJgdKgZ;ee;edLedMerRcCst|stdt|}|dkr!t|}|dkrtd||S|Sdt|vs.|tjkr7t t | S||}t | }|dkrNdd||Sdd|td||S)as Renyi's generalized entropy Parameters ---------- px : array_like Discrete probability distribution of random variable X. Note that px is assumed to be a proper probability distribution. logbase : int or np.e, optional Default is 2 (bits) alpha : float or inf The order of the entropy. The default is 1, which in the limit is just Shannon's entropy. 2 is Renyi (Collision) entropy. If the string "inf" or numpy.inf is specified the min-entropy is returned. measure : str, optional The type of entropy measure desired. 'R' returns Renyi entropy measure. 'T' returns the Tsallis entropy measure. Returns ------- 1/(1-alpha)*log(sum(px**alpha)) In the limit as alpha -> 1, Shannon's entropy is returned. In the limit as alpha -> inf, min-entropy is returned. z+px is not a proper probability distributionrr$inf) rr+floatr1r#strlowerr r@rrr)r.alphar/measureZgenentrrr renyientropyksrFTcCsdS)a Generalized cross-entropy measures. Parameters ---------- px : array_like Discrete probability distribution of random variable X py : array_like Discrete probability distribution of random variable Y pxpy : 2d array_like, optional Joint probability distribution of X and Y. If pxpy is None, X and Y are assumed to be independent. logbase : int or np.e, optional Default is 2 (bits) measure : str, optional The measure is the type of generalized cross-entropy desired. 'T' is the cross-entropy version of the Tsallis measure. 'CR' is Cressie-Read measure. Nr)r.r6r7rDr/rErrrgencrossentropysrH__main__zQFrom Golan (2008) "Information and Entropy Econometrics -- A Review and Synthesisz Table 3.1)皙?rJrJrJrJ)S㥛?g;On?g'1Z?gK?gMbp?) gh㈵>g-C6?gMbP?g{Gz?g?g333333?rJg?g333333?gffffff?g?g??oZ InformationZ ProbabilityiZEntropye)rrUUUUUU?)qq?rPrP)gqq?rPUUUUUU?z Table 3.3zdiscretize functions)2g3333335@g@F@g?@g3@gLD@gYC@g333333&@g/@gfffff?@g9@g3333334@gffffff,@g8@g5@g&@g2@L0@g3333336@g333333@g;@rRǧA@-@g1@g333333<@gffffff0@g0@gG@g#@g2@g @@g:@g0@g333333@gffffff5@g4@gL=@rSg @g6@g)@gfffff:@g9@gfffff6@gffffff&@g333334@g333333:@g"@g%@g333333/@z0Example in section 3.6 of Golan, using table 3.3z'Bounding errors using Fano's inequalityz"H(P_{e}) + P_{e}log(K-1) >= H(X|Y)zor, a weaker inequalityzP_{e} >= [H(X|Y) - 1]/log(K)z P(x) = %sz?X = 3 has the highest probability, so this is the estimate Xhatz1The probability of error Pe is 1 - p(X=3) = %0.4gzH(Pe) = %0.4g and K=3z-H(Pe) + Pe*log(K-1) = %0.4g >= H(X|Y) = %0.4gzor using the weaker inequalityz+Pe = {:0.4g} >= [H(X) - 1]/log(K) = {:0.4g}z>Consider now, table 3.5, where there is additional informationz.The conditional probabilities of P(X|Y=y) are )rUg?)rOrOrO)rQrOrLz2The probability of error given this information iszPe = [H(X|Y) -1]/log(K) = %0.4gz+such that more information lowers the error)gV-?gV-?gw/?)g(\?g+?g%C?)gzG?rKgPn?)N)rN)r$)Nr$)rr$r?)rr$rG)>__doc__Zstatsmodels.compat.pythonrrZscipyrZnumpyr Z matplotlibrZpltZ scipy.specialrr rr!r#r'r)r1r2r9r<r=r>rFrH__name__printrYr pZsubplotZylabelZxlabelZlinspacexZplotZarraywrr.r6ZH_XZH_YZH_XYZ H_XgivenYZ H_YgivenXr&r0ZD_YXZD_XYZI_XYZdiscXZpeZH_per-formatZw2ZmeanZ markovchainrrrrs"    $ (   #   ( . 3                  6