o þ0Gf(Oã@sàddlZddlZddlZddlmZddlmm Z ddl m Z Gdd„dej ƒZGdd„deƒZGdd „d eƒZGd d „d eƒZGd d „d ejƒZGdd„de jƒZe  ee¡dd„ZGdd„deƒZeZeZeZeZdS)éN)Úmodel)ÚConvergenceWarningcó(eZdZdZ‡fdd„Zdd„Z‡ZS)Ú_DimReductionRegressionzB A base class for dimension reduction regression methods. c stƒj||fi|¤ŽdS©N)ÚsuperÚ__init__©ÚselfÚendogÚexogÚkwargs©Ú __class__©ú=lib/python3.10/site-packages/statsmodels/regression/dimred.pyrsz _DimReductionRegression.__init__cCs€t |j¡}|j|dd…f}|| d¡8}t |j|¡|jd}tj  |¡}tj  ||j¡j}||_ ||_ t  ||¡|_dS©Nr)ÚnpÚargsortr r ÚmeanÚdotÚTÚshapeÚlinalgZcholeskyÚsolveÚwexogÚ_covxrÚ array_splitÚ _split_wexog)r Ún_sliceÚiiÚxÚcovxZcovxrrrrÚ_preps  z_DimReductionRegression._prep)Ú__name__Ú __module__Ú __qualname__Ú__doc__rr#Ú __classcell__rrrrr s rc@s8eZdZdZddd„Zdd„Zdd„Z dd d„Zd S)ÚSlicedInverseRega0 Sliced Inverse Regression (SIR) Parameters ---------- endog : array_like (1d) The dependent variable exog : array_like (2d) The covariates References ---------- KC Li (1991). Sliced inverse regression for dimension reduction. JASA 86, 316-342. éc Ksêt|ƒdkr d}t |¡|jjd|}| |¡dd„|jDƒ}dd„|jDƒ}t |¡}t |¡}t  |j |dd…df|¡|  ¡}tj   |¡\}} t | ¡} || }| dd…| f} tj  |jj | ¡} t|| |d} t| ƒS)zÄ Estimate the EDR space using Sliced Inverse Regression. Parameters ---------- slice_n : int, optional Target number of observations per slice rz1SIR.fit does not take any extra keyword argumentscSóg|]}| d¡‘qS©r©r©Ú.0ÚzrrrÚ Józ(SlicedInverseReg.fit..cSóg|]}|jd‘qSr,©rr.rrrr1Kr2N©Úeigs)ÚlenÚwarningsÚwarnr rr#rrÚasarrayrrÚsumrÚeighrrrÚDimReductionResultsÚDimReductionResultsWrapper) r Úslice_nr ÚmsgrÚmnÚnZmncÚaÚbÚjjÚparamsÚresultsrrrÚfit6s"     & zSlicedInverseReg.fitcCsÆ|j}|j}|j}|j}d}t |||jf¡}t|jƒD]}t |j |dd…|f¡}|t  ||¡7}qt ||¡} tj   | ¡\} } t | t | j |j ¡¡} |j | } |t || |   d¡¡7}|Sr)Úk_varsÚ_covxÚ _slice_meansÚ _slice_propsrÚreshapeÚndimÚrangerÚpen_matr;rZqrr)r ÚAÚpr"rAÚphÚvÚkÚuÚcovxaÚqÚ_ZqdZqurrrÚ_regularized_objective[s  z'SlicedInverseReg._regularized_objectivec Cs:|j}|j}|j}|j}|j}|j}| ||f¡}dt |j j t |j |¡¡}| ||f¡}t ||¡} t || ¡} t | j | ¡} tj   | ¡} t  ||f¡} tj  | | j ¡}dg||}t|ƒD][}t|ƒD]T}| d9} d| ||f<t | j | ¡}||j 7}t | t || ¡¡ }t t || ¡|¡}|t | t || j ¡¡7}|t | tj  | t | j |¡¡¡7}|||||<qhqbtj  | t | j |j ¡¡}|t | |¡j }t|ƒD]@}||dd…f}||dd…f}t|ƒD])}t|ƒD]"}t |t |||||¡¡}|||fd|||8<qôqîqØ| ¡S)Néré)rIrNrJrrKrLrMrrrPrrÚinvÚzerosrrOÚravel)r rQrRrNr"rrArSZgrrWZcovx2aÚQZQiZjmZqcvZftrXÚrZumatZfmatZchZcuÚirVrTÚfrrrÚ_regularized_gradssP       $÷     "þÿz"SlicedInverseReg._regularized_gradr\Nédçü©ñÒMbP?cKsºt|ƒdkr d}t |¡|durtdƒ‚| dd¡}| dd¡}|jjd|} t |j ¡} |j| dd…f} | |   d¡8} t  | j ¡} t  | | ¡} dd „| Dƒ}d d „| Dƒ}t |¡}t |¡}|| ¡|_||_| jd|_||_| |_| |_||_|dur›t |j|f¡}t |¡|d|…d|…f<|}n|jd |kr¨d }t|ƒ‚|}t||j|j||ƒ\}}}|sÒ| | ¡¡}t t ||¡¡}d |}t |¡t||dd}t |ƒS)a© Estimate the EDR space using regularized SIR. Parameters ---------- ndim : int The number of EDR directions to estimate pen_mat : array_like A 2d array such that the squared Frobenius norm of `dot(pen_mat, dirs)`` is added to the objective function, where `dirs` is an orthogonal array whose columns span the estimated EDR space. slice_n : int, optional Target number of observations per slice maxiter :int The maximum number of iterations for estimating the EDR space. gtol : float If the norm of the gradient of the objective function falls below this value, the algorithm has converged. Returns ------- A results class instance. Notes ----- If each row of `exog` can be viewed as containing the values of a function evaluated at equally-spaced locations, then setting the rows of `pen_mat` to [[1, -2, 1, ...], [0, 1, -2, 1, ..], ...] will give smooth EDR coefficients. This is a form of "functional SIR" using the squared second derivative as a penalty. References ---------- L. Ferre, A.F. Yao (2003). Functional sliced inverse regression analysis. Statistics: a journal of theoretical and applied statistics 37(6) 475-488. rz3SIR.fit_regularized does not take keyword argumentsNzpen_mat is a required argumentÚ start_paramsr?r*cSr+r,r-r.rrrr1år2z4SlicedInverseReg.fit_regularized..cSr3r,r4r.rrrr1ær2r\z1Shape of start_params is not compatible with ndimz,SIR.fit_regularized did not converge, |g|=%fr5)!r7r8r9Ú ValueErrorÚgetr rrrr rÚcovrrr:r;rLrNrIrPrJrrKr^ÚeyeÚ _grass_optrZrdr_Úsqrtrr=r>)r rNrPr?ÚmaxiterÚgtolr r@rgrr r!r"Z split_exogrArBrFrYÚcnvrgÚgÚgnrGrrrÚfit_regularized¢sT *          ÿ z SlicedInverseReg.fit_regularized)r*)r\Nr*rerf)r$r%r&r'rHrZrdrsrrrrr)%s %/ÿr)c@seZdZdZdd„ZdS)ÚPrincipalHessianDirectionsa Principal Hessian Directions (PHD) Parameters ---------- endog : array_like (1d) The dependent variable exog : array_like (2d) The covariates Returns ------- A model instance. Call `fit` to obtain a results instance, from which the estimated parameters can be obtained. References ---------- KC Li (1992). On Principal Hessian Directions for Data Visualization and Dimension Reduction: Another application of Stein's lemma. JASA 87:420. cKsØ| dd¡}|j|j ¡}|j|j d¡}|r)ddlm}|||ƒ ¡}|j}t  d|||¡}|t |ƒ}t  |j ¡}tj  ||¡} tj  | ¡\} } t t | ¡ ¡} | | } | dd…| f} t|| | d}t|ƒS)aŸ Estimate the EDR space using PHD. Parameters ---------- resid : bool, optional If True, use least squares regression to remove the linear relationship between each covariate and the response, before conducting PHD. Returns ------- A results instance which can be used to access the estimated parameters. ÚresidFr)ÚOLSz i,ij,ik->jkNr5)rir rr Z#statsmodels.regression.linear_modelrvrHrurZeinsumr7rjrrrZeigrÚabsr=r>)r r ruÚyr!rvraÚcmZcxÚcbrCrDrErFrGrrrrHs"    zPrincipalHessianDirections.fitN)r$r%r&r'rHrrrrrts rtcr)ÚSlicedAverageVarianceEstimationa] Sliced Average Variance Estimation (SAVE) Parameters ---------- endog : array_like (1d) The dependent variable exog : array_like (2d) The covariates bc : bool, optional If True, use the bias-corrected CSAVE method of Li and Zhu. References ---------- RD Cook. SAVE: A method for dimension reduction and graphics in regression. http://www.stat.umn.edu/RegGraph/RecentDev/save.pdf Y Li, L-X Zhu (2007). Asymptotics for sliced average variance estimation. The Annals of Statistics. https://arxiv.org/pdf/0708.0462.pdf c sFtt|ƒj||fi|¤Žd|_d|vr|ddur!d|_dSdSdS)NFÚbcT)rÚSAVErr|r rrrras  ÿz(SlicedAverageVarianceEstimation.__init__cKs| dd¡}|jjd|}| |¡dd„|jDƒ}dd„|jDƒ}|jjd}|jsPd}t||ƒD]\}} t  |¡| } ||t  | | ¡7}q3|t |ƒ}n„d} |D] } | t  | | ¡7} qT| t |ƒ} d} |jD])}||  d¡}t |jdƒD]}||dd…f}t ||¡}| t  ||¡7} qzqj| |jjd} t  |¡} | | d| dd d}| d| dd d}|| || }t  |¡d t|ƒt |ƒ|}tj |¡\}}t | ¡}||}|dd…|f}tj |jj|¡}t|||d }t|ƒS) z“ Estimate the EDR space. Parameters ---------- slice_n : int Number of observations per slice r?é2rcSsg|]}t |j¡‘qSr)rrjrr.rrrr1zsz7SlicedAverageVarianceEstimation.fit..cSr3r,r4r.rrrr1{r2r\Nr[r5)rir rr#rrr|Úziprrkrr7rrOZouterr;rr<rrrrr=r>)r r r?rZcvÚnsrRZvmÚwZcvxZicvÚavÚcZvnr!rarbrVÚmZk1Zk2Zav2rCrDrErFrGrrrrHhsJ      ý " z#SlicedAverageVarianceEstimation.fit)r$r%r&r'rrHr(rrrrr{Is r{cs eZdZdZ‡fdd„Z‡ZS)r=aV Results class for a dimension reduction regression. Notes ----- The `params` attribute is a matrix whose columns span the effective dimension reduction (EDR) space. Some methods produce a corresponding set of eigenvalues (`eigs`) that indicate how much information is contained in each basis direction. cstƒ ||¡||_dSr)rrr6)r rrFr6rrrr·sÿ zDimReductionResults.__init__)r$r%r&r'rr(rrrrr=ªs r=c@seZdZddiZeZdS)r>rFÚcolumnsN)r$r%r&Ú_attrsZ _wrap_attrsrrrrr>½sÿr>cs|j\}}| ¡}||ƒ}d}t|ƒD]n} ||ƒ} | t | |¡|t ||¡8} t t | | ¡¡|kr9d}nI|  ||f¡} tj  | d¡\‰‰‰| ||f¡} t | ˆj ¡‰‡‡‡‡fdd„} d}|dkr| | ƒ}||ƒ}||kry|}|}n|d}|dksgq| ||f¡}|||fS) a Minimize a function on a Grassmann manifold. Parameters ---------- params : array_like Starting value for the optimization. fun : function The function to be minimized. grad : function The gradient of fun. maxiter : int The maximum number of iterations. gtol : float Convergence occurs when the gradient norm falls below this value. Returns ------- params : array_like The minimizing value for the objective function. fval : float The smallest achieved value of the objective function. cnvrg : bool True if the algorithm converged to a limit point. Notes ----- `params` is 2-d, but `fun` and `grad` should take 1-d arrays `params.ravel()` as arguments. Reference --------- A Edelman, TA Arias, ST Smith (1998). The geometry of algorithms with orthogonality constraints. SIAM J Matrix Anal Appl. http://math.mit.edu/~edelman/publications/geometry_of_algorithms.pdf FTrcs4ˆt ˆ|¡ˆt ˆ|¡}t |ˆ¡ ¡Sr)rZcosZsinrr_)ÚtÚpa©Zpa0ÚsrVZvtrrÚgeos$z_grass_opt..geog@g»½×Ùß|Û=r[) rr_rOrrrmr;rMrZsvdr)rFZfunZgradrnrorRÚdZf0rprYrqZgmZparamsmr‹ÚsteprˆÚf1rr‰rrlÇs8 &   ù€ rlcs:eZdZdZ‡fdd„Zdd„Zdd„Zd d d „Z‡ZS)ÚCovarianceReductiona/ Dimension reduction for covariance matrices (CORE). Parameters ---------- endog : array_like The dependent variable, treated as group labels exog : array_like The independent variables. dim : int The dimension of the subspace onto which the covariance matrices are projected. Returns ------- A model instance. Call `fit` on the model instance to obtain a results instance, which contains the fitted model parameters. Notes ----- This is a likelihood-based dimension reduction procedure based on Wishart models for sample covariance matrices. The goal is to find a projection matrix P so that C_i | P'C_iP and C_j | P'C_jP are equal in distribution for all i, j, where the C_i are the within-group covariance matrices. The model and methodology are as described in Cook and Forzani. The optimization method follows Edelman et. al. References ---------- DR Cook, L Forzani (2008). Covariance reducing models: an alternative to spectral modeling of covariance matrices. Biometrika 95:4. A Edelman, TA Arias, ST Smith (1998). The geometry of algorithms with orthogonality constraints. SIAM J Matrix Anal Appl. http://math.mit.edu/~edelman/publications/geometry_of_algorithms.pdf c sºtƒ ||¡gg}}tj|j|jd}| |j¡D]\}}| |  ¡j ¡| |j d¡qt |ƒ|_ d} t|ƒD]\} }| || || 7} q;| |j } | |_||_||_||_dS)N)Úindexr)rrÚpdZ DataFramer r ÚgroupbyrÚappendrjÚvaluesrr7ÚnobsÚ enumerateÚcovmÚcovsr€Údim) r r r r™r˜r€ZdfrYrTr—rbrrrr@s    zCovarianceReduction.__init__c Cs¦|jjd}| ||jf¡}t |jt |j|¡¡}tj |¡\}}|j |d}t |j ƒD]"\}}t |jt ||¡¡}tj |¡\}}||j ||d8}q.|S)zô Evaluate the log-likelihood Parameters ---------- params : array_like The projection matrix used to reduce the covariances, flattened to 1d. Returns the log-likelihood. rr[) r—rrMr™rrrrZslogdetr•r–r˜r€) r rFrRÚprojrƒrYZldetrcÚjrrrÚloglikeWs zCovarianceReduction.loglikec Cs¸|jjd}| ||jf¡}t |jt |j|¡¡}t |j|¡}|jtj  ||j¡j}t |j ƒD]%\}}t |jt ||¡¡}t ||¡}||j |tj  ||j¡j8}q2|  ¡S)a  Evaluate the score function. Parameters ---------- params : array_like The projection matrix used to reduce the covariances, flattened to 1d. Returns the score function evaluated at 'params'. r)r—rrMr™rrrr•rrr–r˜r€r_) r rFrRršZc0ZcPrqr›rƒrrrÚscorers  "zCovarianceReduction.scoreNéÈç-Cëâ6?c sЈjjd}ˆj}|dur$t ||f¡}t |¡|d|…d|…f<|}n|}t|‡fdd„‡fdd„||ƒ\}}}|d9}|sZˆ | ¡¡} t  t  | | ¡¡} d| } t   | t ¡tˆ|dd} || _t| ƒS) aö Fit the covariance reduction model. Parameters ---------- start_params : array_like Starting value for the projection matrix. May be rectangular, or flattened. maxiter : int The maximum number of gradient steps to take. gtol : float Convergence criterion for the gradient norm. Returns ------- A results instance that can be used to access the fitted parameters. rNcó ˆ |¡ Sr)rœ©r!©r rrÚ®ó z)CovarianceReduction.fit..cr r)rr¡r¢rrr£¯r¤éÿÿÿÿz/CovReduce optimization did not converge, |g|=%fr5)r—rr™rr^rkrlrr_rmr;r8r9rr=Úllfr>) r rgrnrorRrŒrFr¦rprqrrr@rGrr¢rrHs(   þ zCovarianceReduction.fit)NržrŸ) r$r%r&r'rrœrrHr(rrrrrs  'r)r8ZnumpyrZpandasr‘Zstatsmodels.baserZstatsmodels.base.wrapperÚbaseÚwrapperÚwrapZstatsmodels.tools.sm_exceptionsrZModelrr)rtr{ZResultsr=ZResultsWrapperr>Zpopulate_wrapperrlrZSIRZPHDr}ZCORErrrrÚs.  dAaÿQ'