o ={cA@sdZddlmmZddlmZddlmZddl m Z ddl m Z e dgdGd d d Z d"d d Zd#d dZddZddZe jfddZe dgdde jdfddZddZddZddZddZd d!ZdS)$z$Utilities related to loss functions.N)backend) keras_tensor)tf_utils) keras_exportzkeras.losses.Reduction)v1c@s8eZdZdZdZdZdZdZeddZ edd Z d S) ReductionV2aTypes of loss reduction. Contains the following values: * `AUTO`: Indicates that the reduction option will be determined by the usage context. For almost all cases this defaults to `SUM_OVER_BATCH_SIZE`. When used with `tf.distribute.Strategy`, outside of built-in training loops such as `tf.keras` `compile` and `fit`, we expect reduction value to be `SUM` or `NONE`. Using `AUTO` in that case will raise an error. * `NONE`: No **additional** reduction is applied to the output of the wrapped loss function. When non-scalar losses are returned to Keras functions like `fit`/`evaluate`, the unreduced vector loss is passed to the optimizer but the reported loss will be a scalar value. Caution: **Verify the shape of the outputs when using** `Reduction.NONE`. The builtin loss functions wrapped by the loss classes reduce one dimension (`axis=-1`, or `axis` if specified by loss function). `Reduction.NONE` just means that no **additional** reduction is applied by the class wrapper. For categorical losses with an example input shape of `[batch, W, H, n_classes]` the `n_classes` dimension is reduced. For pointwise losses you must include a dummy axis so that `[batch, W, H, 1]` is reduced to `[batch, W, H]`. Without the dummy axis `[batch, W, H]` will be incorrectly reduced to `[batch, W]`. * `SUM`: Scalar sum of weighted losses. * `SUM_OVER_BATCH_SIZE`: Scalar `SUM` divided by number of elements in losses. This reduction type is not supported when used with `tf.distribute.Strategy` outside of built-in training loops like `tf.keras` `compile`/`fit`. You can implement 'SUM_OVER_BATCH_SIZE' using global batch size like: ``` with strategy.scope(): loss_obj = tf.keras.losses.CategoricalCrossentropy( reduction=tf.keras.losses.Reduction.NONE) .... loss = tf.reduce_sum(loss_obj(labels, predictions)) * (1. / global_batch_size) ``` Please see the [custom training guide]( https://www.tensorflow.org/tutorials/distribute/custom_training) for more details on this. autoZnonesumZsum_over_batch_sizecCs|j|j|j|jfSN)AUTONONESUMSUM_OVER_BATCH_SIZE)clsr8lib/python3.10/site-packages/keras/utils/losses_utils.pyallQszReductionV2.allcCs*||vrtd|d|ddS)NzInvalid Reduction Key: z. Expected keys are "")r ValueError)rkeyrrrvalidateUs zReductionV2.validateN) __name__ __module__ __qualname____doc__r r r r classmethodrrrrrrrs. rc st|pdtsttstj}|j}j}|j}|durj|durj||}||dkrJ|jd drJt dgn||dkr_|jd dr_t dgfWdSt t }|dus|jd drt t |d|fddfdd|dus|jd drt t |d|fddfd dfWdS1swYdS) aHSqueeze last dim if ranks differ from expected by exactly 1. In the common case where we expect shapes to match, `expected_rank_diff` defaults to 0, and we squeeze the last dimension of the larger rank if they differ by 1. But, for example, if `labels` contains class IDs and `predictions` contains 1 probability per class, we expect `predictions` to have 1 more dimension than `labels`, so `expected_rank_diff` would be 1. In this case, we'd squeeze `labels` if `rank(predictions) - rank(labels) == 0`, and `predictions` if `rank(predictions) - rank(labels) == 2`. This will use static shape if available. Otherwise, it will add graph operations, which could result in a performance hit. Args: labels: Label values, a `Tensor` whose dimensions match `predictions`. predictions: Predicted values, a `Tensor` of arbitrary dimensions. expected_rank_diff: Expected result of `rank(predictions) - rank(labels)`. name: Name of the op. Returns: Tuple of `labels` and `predictions`, possibly with last dim squeezed. remove_squeezable_dimensionsNctdgSNrtfsqueezer predictionsrrz.remove_squeezable_dimensions..cSr rrr$rrr&crr r!rlabelsrrr&r'cr(r rrr*rrr&r))r name_scoperZis_tensor_or_extension_typer"convert_to_tensorshapendimsZdimsZis_compatible_withr#rankcondequal) r+r%Zexpected_rank_diffnameZpredictions_shapeZpredictions_rankZ labels_shapeZ labels_rank rank_diffr)r+r%rr]sX        $rc sj}|j}dur^j}|j}|dur,|dur,||dks$|ddkr+t\n2ttfddtdtdfdd}ttd|\durffSj}|j} | dkrufS|dur| dur| |dkrtdgn || dkrtdgfSt} | tfddfd d fd d } tt| dfd d| fS)aSqueeze or expand last dimension if needed. 1. Squeezes last dim of `y_pred` or `y_true` if their rank differs by 1 (using `remove_squeezable_dimensions`). 2. Squeezes or expands last dim of `sample_weight` if its rank differs by 1 from the new rank of `y_pred`. If `sample_weight` is scalar, it is kept scalar. This will use static shape if available. Otherwise, it will add graph operations, which could result in a performance hit. Args: y_pred: Predicted values, a `Tensor` of arbitrary dimensions. y_true: Optional label `Tensor` whose dimensions match `y_pred`. sample_weight: Optional weight scalar or `Tensor` whose dimensions match `y_pred`. Returns: Tuple of `y_pred`, `y_true` and `sample_weight`. Each of them possibly has the last dimension squeezed, `sample_weight` could be extended by one dimension. If `sample_weight` is None, (y_pred, y_true) is returned. Nrrcs tSr )rry_predy_truerrr&s z.squeeze_or_expand_dimensions..cstfddS)NcsfSr rrr5rrr&sz@squeeze_or_expand_dimensions....)r"r1r) is_last_dim_1 squeeze_dimsr6r7rrr&srcrr r!r sample_weightrrr&r'cs*fdd}ttd|fddS)Ncrr )r" expand_dimsrr:rrr&r'zMsqueeze_or_expand_dimensions.._maybe_expand_weights..rcr(r rrr:rrr&r)r"r1r2)Zexpand_weights)r4r;rr_maybe_expand_weightss z;squeeze_or_expand_dimensions.._maybe_expand_weightscsttdS)Nrr=r)r>maybe_squeeze_weightsr4rr_maybe_adjust_weightssz;squeeze_or_expand_dimensions.._maybe_adjust_weightscr(r rrr:rrr&r)) r.r/rr"r0r2r1r#r<) r6r7r;Z y_pred_shapeZ y_pred_rankZ y_true_shapeZ y_true_rankZmaybe_squeeze_dimsZ weights_shapeZ weights_rankZweights_rank_tensorr@r)r>r8r?r4r;r9r6r7rsqueeze_or_expand_dimensionssN         rAcCst|}tjj||ddS)a:Computes a safe mean of the losses. Args: losses: `Tensor` whose elements contain individual loss measurements. num_present: The number of measurable elements in `losses`. Returns: A scalar representing the mean of `losses`. If `num_present` is zero, then zero is returned. valuer3)r" reduce_sumZmathZ divide_no_nan)lossesZ num_presentZ total_lossrrr _safe_means rFcCsHtd}tjtj||d|jdWdS1swYdS)z3Computes the number of elements in `losses` tensor.Z num_elementsrC)dtypeN)rr,r"castsizerG)rEZscoperrr _num_elementss $rJcCs8|tjkr |}|St|}|tjkrt|t|}|S)z2Reduces the individual weighted loss measurements.)rr r"rDrrFrJ)weighted_losses reductionlossrrrreduce_weighted_losss  rNz/keras.__internal__.losses.compute_weighted_lossc Cs t||tjkr tj}|durd}t|pdb|tjj _ t |t j tjfs0t|}t |t j tjfs>t|}|jjsN|j}t|d}d}nd}t||j}t|d|\}}}t||}t||}|rst||}|WdS1swYdS)aComputes the weighted loss. Args: losses: `Tensor` of shape `[batch_size, d1, ... dN]`. sample_weight: Optional `Tensor` whose rank is either 0, or the same rank as `losses`, or be broadcastable to `losses`. reduction: (Optional) Type of `tf.keras.losses.Reduction` to apply to loss. Default value is `SUM_OVER_BATCH_SIZE`. name: Optional name for the op. Raises: ValueError: If the shape of `sample_weight` is not compatible with `losses`. Returns: Weighted loss `Tensor` of the same type as `losses`. If `reduction` is `NONE`, this has the same shape as `losses`; otherwise, it is scalar. N?Z weighted_lossfloat32TF)rrr rrr,r"compatrZget_default_graphZ_last_loss_reduction isinstancerZ KerasTensorZ RaggedTensorr-rG is_floatingrHrAZmultiplyrN) rEr;rLr3Z input_dtypeZ input_casted_rKrMrrrcompute_weighted_loss!s<          $rUcCs$tjj}|dkr|d|9}|S)zBScales and returns the given loss value by the number of replicas.rrO)r"Z distributeZ get_strategyZnum_replicas_in_sync)Z loss_valueZ num_replicasrrrscale_loss_for_distributionis  rVcstd|D](}|jjr$dus|jjjkr|jn |jhddhkr$d|jjr,|Sqr8fdd|D}|S)arCast a list of losses to a common dtype. If any loss is floating-point, they will all be casted to the most-precise floating-point loss. Otherwise the losses are not casted. We also skip casting losses if there are any complex losses. Args: losses: A list of losses. Returns: `losses`, but they have been casted to a common dtype. NZbfloat16Zfloat16rPcsg|]}t|qSr)r"rH).0rMZ highest_floatrr sz/cast_losses_to_common_dtype..)rGrSrIZ is_complex)rErMrrXrcast_losses_to_common_dtypeqs rZcCs t|ddS)zReturns Keras mask from tensor.Z _keras_maskN)getattr)y_prrrget_masks r]cCsD|dur t||j}|durt||d\}}}||9}|S|}|S)z2Applies any mask on predictions to sample weights.Nr:)r"rHrGrA)r\swmaskrTrrr apply_masksr`cCs\|dur(t||j}|tjtjfvr(tt||j}t|}|||9}t|||S)z;Redistribute sample weights considering only valid entries.N) r"rHrGrr rrIrDr`)rEr^r_rLtotalZvalidrrrapply_valid_masks   rb)rN)NN)rZtensorflow.compat.v2rQZv2r"ZkerasrZ keras.enginerZ keras.utilsrZ tensorflow.python.util.tf_exportrrrrArFrJrrNrUrVrZr]r`rbrrrrs2     A F\  G