Record Class MiniMaxVisionArgs

java.lang.Object
java.lang.Record
smile.llm.model.minimax.MiniMaxVisionArgs
Record Components:
hiddenSize - ViT width.
numHeads - attention heads.
numLayers - encoder layers.
intermediateSize - MLP width.
patchSize - spatial patch edge in pixels.
temporalPatchSize - frames folded into one patch.
mergeSize - spatial merge factor (merge x merge patches become one token).
numChannels - input channels.
layerNormEps - LayerNorm epsilon.
ropeTheta - 3D RoPE base.
maxSegmentFrames - attention never spans more than this many temporal groups (longer grids are split).
textHidden - width of the language model embeddings (projector output).
projectorHidden - projector / merger hidden width.
imageTokenId - placeholder token id of image patches.
videoTokenId - placeholder token id of video patches.
public record MiniMaxVisionArgs(int hiddenSize, int numHeads, int numLayers, int intermediateSize, int patchSize, int temporalPatchSize, int mergeSize, int numChannels, double layerNormEps, double ropeTheta, int maxSegmentFrames, int textHidden, int projectorHidden, int imageTokenId, int videoTokenId) extends Record
Hyperparameters of the MiniMax-M3 vision encoder (CLIP-style ViT with 3D RoPE), the projector and the 2x2 patch merger, read from vision_config of config.json.

  • Constructor Details

    • MiniMaxVisionArgs

      public MiniMaxVisionArgs(int hiddenSize, int numHeads, int numLayers, int intermediateSize, int patchSize, int temporalPatchSize, int mergeSize, int numChannels, double layerNormEps, double ropeTheta, int maxSegmentFrames, int textHidden, int projectorHidden, int imageTokenId, int videoTokenId)
      Creates an instance of a MiniMaxVisionArgs record class.
      Parameters:
      hiddenSize - the value for the hiddenSize record component
      numHeads - the value for the numHeads record component
      numLayers - the value for the numLayers record component
      intermediateSize - the value for the intermediateSize record component
      patchSize - the value for the patchSize record component
      temporalPatchSize - the value for the temporalPatchSize record component
      mergeSize - the value for the mergeSize record component
      numChannels - the value for the numChannels record component
      layerNormEps - the value for the layerNormEps record component
      ropeTheta - the value for the ropeTheta record component
      maxSegmentFrames - the value for the maxSegmentFrames record component
      textHidden - the value for the textHidden record component
      projectorHidden - the value for the projectorHidden record component
      imageTokenId - the value for the imageTokenId record component
      videoTokenId - the value for the videoTokenId record component

  • Method Details

    • headDim

      public int headDim()
      Returns the per-head width.
      Returns:
      per-head width.

    • patchDim

      public int patchDim()
      Returns the length of one flattened patch (C * T * P * P).
      Returns:
      length of one flattened patch (C * T * P * P).

    • axisDim

      public int axisDim()
      Returns the rotary dimensions per axis (t, h, w each get this many).
      Returns:
      rotary dimensions per axis (t, h, w each get this many).

    • rotaryDim

      public int rotaryDim()
      Returns the rotated dimensions of each head (3 * axisDim); the remaining head dims pass through.
      Returns:
      rotated dimensions of each head (3 * axisDim); the remaining head dims pass through.

    • fromConfigFile

      public static MiniMaxVisionArgs fromConfigFile(File configFile) throws IOException
      Parses a checkpoint's config.json.
      Parameters:
      configFile - the file.
      Returns:
      the vision arguments, or null when the config has no vision_config.
      Throws:
      IOException - if the file cannot be read.

    • fromConfig

      public static MiniMaxVisionArgs fromConfig(tools.jackson.databind.JsonNode root)
      Parses the top-level config object.
      Parameters:
      root - the parsed config.json.
      Returns:
      the vision arguments, or null when there is no vision_config.

    • toString

      public final String toString()
      Returns a string representation of this record class. The representation contains the name of the class, followed by the name and value of each of the record components.
      Specified by:
      toString in class Record
      Returns:
      a string representation of this object

    • hashCode

      public final int hashCode()
      Returns a hash code value for this object. The value is derived from the hash code of each of the record components.
      Specified by:
      hashCode in class Record
      Returns:
      a hash code value for this object

    • equals

      public final boolean equals(Object o)
      Indicates whether some other object is "equal to" this one. The objects are equal if the other object is of the same class and if all the record components are equal. All components in this record class are compared with the compare method from their corresponding wrapper classes.
      Specified by:
      equals in class Record
      Parameters:
      o - the object with which to compare
      Returns:
      true if this object is the same as the o argument; false otherwise.

    • hiddenSize

      public int hiddenSize()
      Returns the value of the hiddenSize record component.
      Returns:
      the value of the hiddenSize record component

    • numHeads

      public int numHeads()
      Returns the value of the numHeads record component.
      Returns:
      the value of the numHeads record component

    • numLayers

      public int numLayers()
      Returns the value of the numLayers record component.
      Returns:
      the value of the numLayers record component

    • intermediateSize

      public int intermediateSize()
      Returns the value of the intermediateSize record component.
      Returns:
      the value of the intermediateSize record component

    • patchSize

      public int patchSize()
      Returns the value of the patchSize record component.
      Returns:
      the value of the patchSize record component

    • temporalPatchSize

      public int temporalPatchSize()
      Returns the value of the temporalPatchSize record component.
      Returns:
      the value of the temporalPatchSize record component

    • mergeSize

      public int mergeSize()
      Returns the value of the mergeSize record component.
      Returns:
      the value of the mergeSize record component

    • numChannels

      public int numChannels()
      Returns the value of the numChannels record component.
      Returns:
      the value of the numChannels record component

    • layerNormEps

      public double layerNormEps()
      Returns the value of the layerNormEps record component.
      Returns:
      the value of the layerNormEps record component

    • ropeTheta

      public double ropeTheta()
      Returns the value of the ropeTheta record component.
      Returns:
      the value of the ropeTheta record component

    • maxSegmentFrames

      public int maxSegmentFrames()
      Returns the value of the maxSegmentFrames record component.
      Returns:
      the value of the maxSegmentFrames record component

    • textHidden

      public int textHidden()
      Returns the value of the textHidden record component.
      Returns:
      the value of the textHidden record component

    • projectorHidden

      public int projectorHidden()
      Returns the value of the projectorHidden record component.
      Returns:
      the value of the projectorHidden record component

    • imageTokenId

      public int imageTokenId()
      Returns the value of the imageTokenId record component.
      Returns:
      the value of the imageTokenId record component

    • videoTokenId

      public int videoTokenId()
      Returns the value of the videoTokenId record component.
      Returns:
      the value of the videoTokenId record component