o 0Gf2@sddlZddlmZddlmmZddlm Z  ddZ  dddZ dddZ GdddeZ GdddejZeee dS)N)Results)cache_readonlyc s@fdd}fdd}fdd}|||fS)ao Negative penalized log-likelihood functions. Returns the negative penalized log-likelihood, its derivative, and its Hessian. The penalty only includes the smooth (L2) term. All three functions have argument signature (x, model), where ``x`` is a point in the parameter space and ``model`` is an arbitrary statsmodels regression model. csN|j}dt|dd}|jtj|fi}| ||S)N)nobsnpsumZlogliker_)paramsmodelrZpen_llfZllf)L1_wtalphak loglike_kwds.nploglikecsD|j}d|}|jtj|fid |}||S)Nrr)rZscorerr )r r rZpen_gradZgr)r r r score_kwdsrrnpscore/s"z_gen_npfuncs..npscorecs@|j}d}|jtj|fid ||}|S)Nr)rr)rZhessianrr )r r rZpen_hessh)r r hess_kwdsrrrnphess5s&z_gen_npfuncs..nphessr) rr r rrrrrrr)r r rrrrr _gen_npfuncss  r coord_descentd?Hz>:0yE>FTc ( sjjd} dur indurindurintr*t| |dur4t| }n|}d}tjt|td} }d|d<| dd}d|vrr|ddurr|durht | d}n |t | d7}fd d t | D}d}t |D]}|}t | D]b}||rq|}d ||<t j|}|dur||7}jjjdd|ffd|i|}||\}}}t||||||||| d ||<|d krt|||krd ||<d||<qtt||}||krd }nqd |t||k<|s t|}||_t|St|} t| | f}!fddjD}t| d krbjjjdd| ffi|}"|"}#|#j|| <|#j|!t| | <njjjddd ffi|}"|"jd d}#t|#jtjr|#jj}$n|#j}$t |#dr|#j!}%nd}%j"j#}&}'t| _"j$j"_#|$||!|%d}d |_%||_||_&d|di|_'|&|'_"_#|S)a Return an elastic net regularized fit to a regression model. Parameters ---------- model : model object A statsmodels object implementing ``loglike``, ``score``, and ``hessian``. method : {'coord_descent'} Only the coordinate descent algorithm is implemented. maxiter : int The maximum number of iteration cycles (an iteration cycle involves running coordinate descent on all variables). alpha : scalar or array_like The penalty weight. If a scalar, the same penalty weight applies to all variables in the model. If a vector, it must have the same length as `params`, and contains a penalty weight for each coefficient. L1_wt : scalar The fraction of the penalty given to the L1 penalty term. Must be between 0 and 1 (inclusive). If 0, the fit is a ridge fit, if 1 it is a lasso fit. start_params : array_like Starting values for `params`. cnvrg_tol : scalar If `params` changes by less than this amount (in sup-norm) in one iteration cycle, the algorithm terminates with convergence. zero_tol : scalar Any estimated coefficient smaller than this value is replaced with zero. refit : bool If True, the model is refit using only the variables that have non-zero coefficients in the regularized fit. The refitted model is not regularized. check_step : bool If True, confirm that the first step is an improvement and search further if it is not. loglike_kwds : dict-like or None Keyword arguments for the log-likelihood function. score_kwds : dict-like or None Keyword arguments for the score function. hess_kwds : dict-like or None Keyword arguments for the Hessian function. Returns ------- Results A results object. Notes ----- The ``elastic net`` penalty is a combination of L1 and L2 penalties. The function that is minimized is: -loglike/n + alpha*((1-L1_wt)*|params|_2^2/2 + L1_wt*|params|_1) where |*|_1 and |*|_2 are the L1 and L2 norms. The computational approach used here is to obtain a quadratic approximation to the smooth part of the target function: -loglike/n + alpha*(1-L1_wt)*|params|_2^2/2 then repeatedly optimize the L1 penalized version of this function along coordinate axes. rNg-C6?)ZdtypeFZhasconstoffsetZexposurec sg|] }t|qSr)r.0r)r r rrrrr sz"fit_elasticnet..r)tol check_stepTrcsi|] }|t|dqSN)getattrr )r rr sz"fit_elasticnet..)maxiterscaler)r)Z iteration)(ZexogshaperZisscalarZonesZzeroscopylenboolZ_get_init_kwdspoplograngedot __class__Zendog_opt_1dabsmaxRegularizedResults convergedRegularizedResultsWrapperZ flatnonzeroZ _init_keysZfitr Znormalized_cov_paramsZix_ issubclasswrapResultsWrapperZ_resultshasattrr)Zdf_modelZdf_residrZ regularizedmethodZ fit_history)(r r=r(r r Z start_paramsZ cnvrg_tolZzero_tolZrefitr$rrrZk_exogr ZbtolZ params_zeroZ init_argsZ model_offsetZfgh_listr7itrZ params_saverZparams0rZ model_1varfuncgradhessZpchangeresultsiiZcovZmodel1Zrsltklassr)pqr)r r rrr rrfit_elasticnet>s J          $    rGcCs|}|||} |||} |||} | | |} |t| kr dS| dkr+|| | } n| dkr7||  | } ntjS|s@|| S||| ||t|| }|| |t|dkra|| Sddlm}|||f|d|df|d}|S)az One-dimensional helper for elastic net. Parameters ---------- func : function A smooth function of a single variable to be optimized with L1 penaty. grad : function The gradient of `func`. hess : function The Hessian of `func`. model : statsmodels model The model being fit. start : real A starting value for the function argument L1_wt : non-negative real The weight for the L1 penalty function. tol : non-negative real A convergence threshold. check_step : bool If True, check that the first step is an improvement and use bisection if it is not. If False, return after the first step regardless. Notes ----- ``func``, ``grad``, and ``hess`` have argument signature (x, model), where ``x`` is a point in the parameter space and ``model`` is the model being fit. If the log-likelihood for the model is exactly quadratic, the global minimum is returned in one step. Otherwise numerical bisection is used. Returns ------- The argmin of the objective function. rrg|=)brentr)argsZbrackr#)rr4nanZscipy.optimizerH)r?r@rAr startr r#r$xfbcdrf1rHZx_optrrrr3s(5      r3cs,eZdZdZfddZeddZZS)r6z Results for models estimated using regularization Parameters ---------- model : Model The model instance used to estimate the parameters. params : ndarray The estimated (regularized) parameters. cst||dSr%)super__init__)selfr r r2rrrSqszRegularizedResults.__init__cCs|j|jS)zR The predicted values from the model at the estimated parameters. )r Zpredictr )rTrrr fittedvaluestszRegularizedResults.fittedvalues)__name__ __module__ __qualname____doc__rSrrV __classcell__rrrUrr6fs  r6c@seZdZddddZeZdS)r8columnsZrows)r ZresidrVN)rWrXrY_attrsZ _wrap_attrsrrrrr8|s r8) rrrrNrrFTNNN)T)ZnumpyrZstatsmodels.base.modelrZstatsmodels.base.wrapperbasewrapperr:Zstatsmodels.tools.decoratorsrrrGr3r6r;r8Zpopulate_wrapperrrrrs&  ! T V