Enum Class WeightGemmBackend

java.lang.Object
java.lang.Enum<WeightGemmBackend>
smile.llm.quant.WeightGemmBackend
All Implemented Interfaces:
Serializable, Comparable<WeightGemmBackend>, Constable
public enum WeightGemmBackend extends Enum<WeightGemmBackend>
Weight GEMM backend selected from checkpoint format and GPU capability.

Policy (locked):

  • Dense → DENSE
  • Native FP8 on sm_90+ → FP8
  • Block-FP8 checkpoints on Ampere/Ada (sm_80–89) → MARLIN_FP8 (weights stay FP8 in memory)
  • Native NVFP4 on sm_100+ → NVFP4
  • GPTQ/AWQ on Ampere/Ada (8.0–8.9) → MARLIN
  • GPTQ/AWQ on Hopper/Blackwell → fail fast (no Marlin)

  • Enum Constant Details

    • DENSE

      public static final WeightGemmBackend DENSE
      Dense BF16/FP16 GEMM.

    • FP8

      public static final WeightGemmBackend FP8
      Native FP8 weight GEMM (Hopper+).

    • NVFP4

      public static final WeightGemmBackend NVFP4
      Native NVFP4 weight GEMM (Blackwell+).

    • MARLIN

      public static final WeightGemmBackend MARLIN
      Marlin INT4 failover (Ampere/Ada).

    • MARLIN_FP8

      public static final WeightGemmBackend MARLIN_FP8
      Weight-only Marlin GEMM over FP8 weights with bfloat16 group scales (Ampere/Ada failover for FP8 checkpoints).

  • Method Details

    • values

      public static WeightGemmBackend[] values()
      Returns an array containing the constants of this enum class, in the order they are declared.
      Returns:
      an array containing the constants of this enum class, in the order they are declared

    • valueOf

      public static WeightGemmBackend valueOf(String name)
      Returns the enum constant of this class with the specified name. The string must match exactly an identifier used to declare an enum constant in this class. (Extraneous whitespace characters are not permitted.)
      Parameters:
      name - the name of the enum constant to be returned.
      Returns:
      the enum constant with the specified name
      Throws:
      IllegalArgumentException - if this enum class has no constant with the specified name
      NullPointerException - if the argument is null

    • select

      public static WeightGemmBackend select(Device device, QuantFormat format)
      Selects the weight GEMM backend.
      Parameters:
      device - CUDA (or CPU) device that will run inference.
      format - checkpoint quantization format.
      Returns:
      the backend to use.
      Throws:
      IllegalStateException - on unsupported format/GPU combinations.

    • computeCapability

      public static int[] computeCapability(Device device)
      Returns {major, minor} for device, or {0,0} on CPU.
      Parameters:
      device - compute device.
      Returns:
      CUDA compute capability major/minor.

    • isFailover

      public boolean isFailover()
      Returns whether this backend is an Ampere/Ada Marlin failover (INT4 or FP8 weights).
      Returns:
      true when this backend is an Ampere/Ada Marlin failover.

    • isPrimary

      public boolean isPrimary()
      Returns whether this is a primary native low-precision path.
      Returns:
      true when this is a primary native low-precision path.