ge=dZddlZddlmZddlmZddlmZgdZ GddZ Gd d e Z Gd d e Z Gd de Z dS)z@Hessian update strategies for quasi-Newton optimization methods.N)norm)get_blas_funcs)warn)HessianUpdateStrategyBFGSSR1c*eZdZdZdZdZdZdZdS)ra]Interface for implementing Hessian update strategies. Many optimization methods make use of Hessian (or inverse Hessian) approximations, such as the quasi-Newton methods BFGS, SR1, L-BFGS. Some of these approximations, however, do not actually need to store the entire matrix or can compute the internal matrix product with a given vector in a very efficiently manner. This class serves as an abstract interface between the optimization algorithm and the quasi-Newton update strategies, giving freedom of implementation to store and update the internal matrix as efficiently as possible. Different choices of initialization and update procedure will result in different quasi-Newton strategies. Four methods should be implemented in derived classes: ``initialize``, ``update``, ``dot`` and ``get_matrix``. Notes ----- Any instance of a class that implements this interface, can be accepted by the method ``minimize`` and used by the compatible solvers to approximate the Hessian (or inverse Hessian) used by the optimization algorithms. c td)Initialize internal matrix. Allocate internal memory for storing and updating the Hessian or its inverse. Parameters ---------- n : int Problem dimension. approx_type : {'hess', 'inv_hess'} Selects either the Hessian or the inverse Hessian. When set to 'hess' the Hessian will be stored and updated. When set to 'inv_hess' its inverse will be used instead. z=The method ``initialize(n, approx_type)`` is not implemented.NotImplementedErrorselfn approx_types Glib/python3.11/site-packages/scipy/optimize/_hessian_update_strategy.py initializez HessianUpdateStrategy.initialize$"#9:: :c td)Update internal matrix. Update Hessian matrix or its inverse (depending on how 'approx_type' is defined) using information about the last evaluated points. Parameters ---------- delta_x : ndarray The difference between two points the gradient function have been evaluated at: ``delta_x = x2 - x1``. delta_grad : ndarray The difference between the gradients: ``delta_grad = grad(x2) - grad(x1)``. z>The method ``update(delta_x, delta_grad)`` is not implemented.r rdelta_x delta_grads rupdatezHessianUpdateStrategy.update6rrc td)PCompute the product of the internal matrix with the given vector. Parameters ---------- p : array_like 1-D array representing a vector. Returns ------- Hp : array 1-D represents the result of multiplying the approximation matrix by vector p. z)The method ``dot(p)`` is not implemented.r rps rdotzHessianUpdateStrategy.dotHs"#9:: :rc td)zReturn current internal matrix. Returns ------- H : ndarray, shape (n, n) Dense matrix containing either the Hessian or its inverse (depending on how 'approx_type' is defined). z0The method ``get_matrix(p)`` is not implemented.r )rs r get_matrixz HessianUpdateStrategy.get_matrixYs"#9:: :rN)__name__ __module__ __qualname____doc__rrr r"rrrr sZ0:::$:::$:::" : : : : :rrceZdZdZeddZeddZeddZddZd Z d Z d Z d Z d Z dZdS)FullHessianUpdateStrategyzKHessian update strategy with full dimensional internal representation. syrddtypesyr2symvautocL||_d|_d|_d|_d|_dSN) init_scalefirst_iterationrBH)rr3s r__init__z"FullHessianUpdateStrategy.__init__os-$ $rcd|_||_||_|dvrtd|jdkr"t j|t |_dSt j|t |_dS)r T)hessinv_hessz+`approx_type` must be 'hess' or 'inv_hess'.r9r,N) r4rr ValueErrornpeyefloatr5r6rs rrz$FullHessianUpdateStrategy.initializexsx $& 2 2 2JKK K  v % %VAU+++DFFFVAU+++DFFFrctj||}tj||}tjtj||}|dks |dks|dkrdS|jdkr||z S||z S)Nrr9)r<r absr)rrrs_norm2y_norm2yss r _auto_scalez%FullHessianUpdateStrategy._auto_scales&'**&Z00 VBF:w// 0 0 991 1 1  v % %R< < rc td)Nz9The method ``_update_implementation`` is not implemented.r rs r_update_implementationz0FullHessianUpdateStrategy._update_implementations!#9:: :rctj|dkrdStj|dkrtdtdS|jri|jdkr|||}nt|j}|jdkr|xj |zc_ n|xj |zc_ d|_| ||dS)rr@Nzdelta_grad == 0.0. Check if the approximated function is linear. If the function is linear better results can be obtained by defining the Hessian as zero instead of using quasi-Newton approximations.r0r9F) r<allr UserWarningr4r3rFr>rr5r6rH)rrrscales rrz FullHessianUpdateStrategy.updates 6'S. ! !  F 6*# $ $  #%0  1 1 1 F   )&((((*==do..6))%%#(D  ##GZ88888rc|jdkr|d|j|S|d|j|S)rr9rA)r_symvr5r6rs rr zFullHessianUpdateStrategy.dotsB  v % %::a++ +::a++ +rc|jdkrtj|j}ntj|j}tj|d}|j|||<|S)zReturn the current internal matrix. Returns ------- M : ndarray, shape (n, n) Dense matrix containing either the Hessian or its inverse (depending on how `approx_type` was defined). r9)k)rr<copyr5r6tril_indices_fromT)rMlis rr"z$FullHessianUpdateStrategy.get_matrixs[  v % %AAA  !!r * * *B"rN)r0)r#r$r%r&r_syr_syr2rNr7rrFrHrr r"r'rrr)r)gs >%s + + +D N6 - - -E N6 - - -E,,,4    :::$9$9$9L,,,&rr)c:eZdZdZ d fd ZdZdZdZxZS) raBroyden-Fletcher-Goldfarb-Shanno (BFGS) Hessian update strategy. Parameters ---------- exception_strategy : {'skip_update', 'damp_update'}, optional Define how to proceed when the curvature condition is violated. Set it to 'skip_update' to just skip the update. Or, alternatively, set it to 'damp_update' to interpolate between the actual BFGS result and the unmodified matrix. Both exceptions strategies are explained in [1]_, p.536-537. min_curvature : float This number, scaled by a normalization factor, defines the minimum curvature ``dot(delta_grad, delta_x)`` allowed to go unaffected by the exception strategy. By default is equal to 1e-8 when ``exception_strategy = 'skip_update'`` and equal to 0.2 when ``exception_strategy = 'damp_update'``. init_scale : {float, 'auto'} Matrix scale at first iteration. At the first iteration the Hessian matrix or its inverse will be initialized with ``init_scale*np.eye(n)``, where ``n`` is the problem dimension. Set it to 'auto' in order to use an automatic heuristic for choosing the initial scale. The heuristic is described in [1]_, p.143. By default uses 'auto'. Notes ----- The update is based on the description in [1]_, p.140. References ---------- .. [1] Nocedal, Jorge, and Stephen J. Wright. "Numerical optimization" Second Edition (2006). skip_updateNr0c|dkr|||_n/d|_n'|dkr|||_nd|_ntdt|||_dS)NrZ:0yE> damp_updateg?z<`exception_strategy` must be 'skip_update' or 'damp_update'.) min_curvaturer;superr7exception_strategy)rr`r^r3 __class__s rr7z BFGS.__init__s  . .(%2""%)"" = 0 0(%2""%(""122 2 $$$"4rc|d|z |||j|_|||z|dzz ||j|_dS)aUpdate the inverse Hessian matrix. BFGS update using the formula: ``H <- H + ((H*y).T*y + s.T*y)/(s.T*y)^2 * (s*s.T) - 1/(s.T*y) * ((H*y)*s.T + s*(H*y).T)`` where ``s = delta_x`` and ``y = delta_grad``. This formula is equivalent to (6.17) in [1]_ written in a more efficient way for implementation. References ---------- .. [1] Nocedal, Jorge, and Stephen J. Wright. "Numerical optimization" Second Edition (2006). aN)rXr6rW)rrEHyyHyss r_update_inverse_hessianzBFGS._update_inverse_hessian%sP"D2Iq"77BsFBE>177rc|d|z ||j|_|d|z ||j|_dS)aUpdate the Hessian matrix. BFGS update using the formula: ``B <- B - (B*s)*(B*s).T/s.T*(B*s) + y*y^T/s.T*y`` where ``s`` is short for ``delta_x`` and ``y`` is short for ``delta_grad``. Formula (6.19) in [1]_. References ---------- .. [1] Nocedal, Jorge, and Stephen J. Wright. "Numerical optimization" Second Edition (2006). g?rdrcN)rWr5)rrEBssBsys r_update_hessianzBFGS._update_hessian9sF38Q$&114#:rTV44rc"|jdkr|}|}n|}|}tj||}||}||}|dkr|||}|jdkr)|tj|jt z|_n(|tj|jt z|_||}||}||j |zkrN|j dkrdS|j dkr6d|j z d||z z z } | |zd| z |zz}tj||}|jdkr| ||||dS| ||||dS)Nr9r@r,rZr]rA) rr<r rFr=rr>r5r6r^r`rorj) rrrwzwzMwwMwrL update_factors rrHzBFGS._update_implementationKs  v % %AAAAA VAq\\ XXa[[ffQii #::$$Wj99E6))e!tJ?O?O"O O O F  v % %YYq}jDFYCCDFFFYYq}jDFYCCDFFFr)r\r0)r#r$r%r&r7rHrwrxs@rrrys]6%%%%%%DDDDDDDrr)r&numpyr< numpy.linalgr scipy.linalgrwarningsr__all__rr)rrr'rrrs2FF'''''' 3 2 2Y:Y:Y:Y:Y:Y:Y:Y:xEEEEE 5EEEPG9G9G9G9G9 $G9G9G9T4D4D4D4D4D #4D4D4D4D4Dr