Class QwenRMSNorm

java.lang.Object
smile.llm.model.qwen.QwenRMSNorm
All Implemented Interfaces:
Function<Tensor,Tensor>, Layer
public class QwenRMSNorm extends Object implements Layer
Qwen3.5 RMSNorm: rms_norm(x) * (1 + weight) with weight initialized to zeros.

  • Constructor Details

    • QwenRMSNorm

      public QwenRMSNorm(int dim, double eps)
      Constructor.
      Parameters:
      dim - feature dimension.
      eps - numerical stability epsilon.

  • Method Details

    • weight

      public Tensor weight()
      Returns the learned offset (the effective scale is 1 + weight).
      Returns:
      the learned offset (the effective scale is 1 + weight).

    • eps

      public double eps()
      Returns the numerical stability epsilon.
      Returns:
      the numerical stability epsilon.

    • forward

      public Tensor forward(Tensor input)
      Description copied from interface: Layer
      Forward propagation (or forward pass) through the layer.
      Specified by:
      forward in interface Layer
      Parameters:
      input - the input tensor.
      Returns:
      the output tensor.

    • module

      public MemorySegment module()
      Description copied from interface: Layer
      Returns the native ST_Module handle for this layer.
      Specified by:
      module in interface Layer
      Returns:
      the native ST_Module handle.