&VfddlmZddlmZddlmZedgGddejZejdej e_dS) ) keras_export)adam) optimizerzkeras.optimizers.AdamWcBeZdZdZ d fd ZxZS) AdamWaOptimizer that implements the AdamW algorithm. AdamW optimization is a stochastic gradient descent method that is based on adaptive estimation of first-order and second-order moments with an added method to decay weights per the techniques discussed in the paper, 'Decoupled Weight Decay Regularization' by [Loshchilov, Hutter et al., 2019](https://arxiv.org/abs/1711.05101). According to [Kingma et al., 2014](http://arxiv.org/abs/1412.6980), the underying Adam method is "*computationally efficient, has little memory requirement, invariant to diagonal rescaling of gradients, and is well suited for problems that are large in terms of data/parameters*". Args: learning_rate: A float, a `keras.optimizers.schedules.LearningRateSchedule` instance, or a callable that takes no arguments and returns the actual value to use. The learning rate. Defaults to `0.001`. beta_1: A float value or a constant float tensor, or a callable that takes no arguments and returns the actual value to use. The exponential decay rate for the 1st moment estimates. Defaults to `0.9`. beta_2: A float value or a constant float tensor, or a callable that takes no arguments and returns the actual value to use. The exponential decay rate for the 2nd moment estimates. Defaults to `0.999`. epsilon: A small constant for numerical stability. This epsilon is "epsilon hat" in the Kingma and Ba paper (in the formula just before Section 2.1), not the epsilon in Algorithm 1 of the paper. Defaults to 1e-7. amsgrad: Boolean. Whether to apply AMSGrad variant of this algorithm from the paper "On the Convergence of Adam and beyond". Defaults to `False`. {{base_optimizer_keyword_args}} References: - [Loshchilov et al., 2019](https://arxiv.org/abs/1711.05101) - [Kingma et al., 2014](http://arxiv.org/abs/1412.6980) for `adam` - [Reddi et al., 2018]( https://openreview.net/pdf?id=ryQu7f-RZ) for `amsgrad`. MbP?Mbp??+?Hz>FNGz?adamwc tjd|||||||||| | | | | |d||jtddS)N) learning_ratebeta_1beta_2epsilonamsgradname weight_decayclipnorm clipvalueglobal_clipnormuse_ema ema_momentumema_overwrite_frequencyloss_scale_factorgradient_accumulation_stepszDArgument `weight_decay` must be a float. Received: weight_decay=None)super__init__r ValueError)selfrrrrrrrrrrrrrrrkwargs __class__s W/var/www/html/software/conda/lib/python3.11/site-packages/keras/src/optimizers/adamw.pyr!zAdamW.__init__5s&  '%+%$;/(C  !   &   $$  % $)rr r r r FNNNFr NNNr)__name__ __module__ __qualname____doc__r! __classcell__)r%s@r&rrsw++^ $$( !**********r'rz{{base_optimizer_keyword_args}}N) keras.src.api_exportrkeras.src.optimizersrrAdamrr+replacebase_optimizer_keyword_argsrr'r&r2s------%%%%%%******'())XXXXXDIXX*)Xv %%%y'L r'