Class MiniMax

java.lang.Object
smile.llm.model.minimax.MiniMax
All Implemented Interfaces:
AutoCloseable, ModelExecutor, LanguageModel
public class MiniMax extends Object implements LanguageModel, ModelExecutor, AutoCloseable
MiniMax-M3 language model: loads a BF16 checkpoint and serves requests through the ModelExecutor step API (bind, prefill, decode, finish).

This is the single-device reference path: dense attention (no block selection yet), host-routed MoE, no CUDA graphs, no speculation. Quantized (MXFP8) checkpoints and chat templating are added in later phases.

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static final record 
    MiniMax.ClusterSpec
    Placement of this process's ranks in a multi-node model.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final String
    family
    Model family.

  • Constructor Summary

    Constructors
    Constructor
    Description
    MiniMax(String name, MiniMaxModel[] models, TensorParallelGroup group, MiniMaxTokenizer tokenizer, MiniMaxModelArgs params)
    Expert-parallel constructor.
    MiniMax(String name, MiniMaxModel model, MiniMaxTokenizer tokenizer, MiniMaxModelArgs params)
    Constructor.

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    attachCluster(ClusterLeader leader)
    Makes this process the leader of a cluster: from now on bind, prefill, decode, finish, evict and the prefix-reuse switch are mirrored to the followers, whose copies of the model execute them in lockstep.
    int
    bind(int[] prompt, int totalCapacity)
    Binds one request into the KV pool (multi-request safe).
    static MiniMax
    build(String checkpointDir, int maxBatchSize, int maxSeqLen, byte[] devices, double memFractionStatic, int pageSize)
    Builds a model from a HuggingFace BF16 checkpoint directory, sharding the routed experts of every MoE layer across devices (expert parallelism with an all-reduce; attention, dense FFNs, router and shared expert are replicated on every rank).
    static MiniMax
    build(String checkpointDir, int maxBatchSize, int maxSeqLen, byte[] devices, double memFractionStatic, int pageSize, String weightMode)
    Builds a model, choosing how MXFP8 weights are stored and multiplied.
    static MiniMax
    build(String checkpointDir, int maxBatchSize, int maxSeqLen, byte[] devices, double memFractionStatic, int pageSize, String weightMode, MiniMax.ClusterSpec cluster)
    Builds this process's part of a model whose expert-parallel ranks span several processes (and nodes).
    static MiniMax
    build(String checkpointDir, int maxBatchSize, int maxSeqLen, byte deviceId, double memFractionStatic, int pageSize)
    Builds a model from a HuggingFace BF16 checkpoint directory on a single device.
    static MiniMax
    buildLeader(String checkpointDir, int maxBatchSize, int maxSeqLen, byte[] devices, double memFractionStatic, int pageSize, String weightMode, int port, int followers, long timeoutMs)
    Builds the leader of a multi-node model: waits for followers follower processes (MiniMaxFollower) to connect, builds this process's ranks together with theirs, and mirrors every state change to them from then on.
    ChatCompletion
    chat(Message[] dialog, int maxGenLen, double temperature, double topp, boolean logprobs, long seed, GenerationListener listener, BooleanSupplier cancelRequested)
    Generates an assistant reply for a dialog.
    void
    close()
     
    String
    decode(int[] tokens)
    Decodes token ids to UTF-8 text.
    Tensor
    decodeStep(int[] requestIds, int[] lastTokens, int[] positions)
    One decode step for an active set (same or mixed positions; engine may cohort by position).
    int[]
    encode(String text)
    Encodes plain text (no chat template, no BOS/EOS).
    int[]
    encodeChat(Message... dialog)
    Encodes a dialog with the model chat template, leaving the assistant turn open for completion.
    int[]
    encodeChat(Message[] dialog, ChatOptions options)
    Encodes a dialog with optional tool-calling options.
    void
    evict(int requestId)
    Instant Eviction without radix insert.
    String
    family()
    Returns the architecture family label.
    void
    finish(int requestId, int[] sequenceTokens)
    Inserts into the radix tree (when enabled) and unbinds the request.
    ChatCompletion
    generate(int[] prompt, int maxGenLen, double temperature, double topp, boolean logprobs, long seed, GenerationListener listener, BooleanSupplier cancelRequested)
    Generates a completion from an already-tokenized prompt.
    boolean
    isMultimodal()
    Returns whether images and videos can be processed (the checkpoint shipped vision weights and they loaded).
    KvCachePool
    kvCachePool()
    Primary KV cache pool (rank 0 for TP).
    int
    maxSeqLen()
    Returns the configured maximum sequence length (max-model-len / context length).
    LanguageModel
    model()
    Returns the underlying language-model façade.
    String
    name()
    Returns the model instance / checkpoint name.
    MiniMaxModel
    network()
    Returns the network of rank 0.
    MiniMaxModel[]
    networks()
    Returns the network of every expert-parallel rank.
    int
    padToken()
    Returns the pad token id.
    MiniMaxModelArgs
    params()
    Returns the hyperparameters.
    Tensor
    prefill(int requestId, int[] prompt, int prefixLen)
    Prefills the unbound suffix of a prompt (activates this request alone).
    Tensor
    prefillChunk(int requestId, int[] prompt, int from, int to)
    Prefills prompt[from, to) (chunked Continuous Batching).
    Tensor
    prefillMultimodal(int requestId, MultimodalPrompt prompt, int from, int to)
    Prefills a multimodal prompt using vision embeds + interleaved mRoPE.
    int
    prefixLen(int requestId)
    Page-aligned matched prefix length for a bound request.
    MiniMaxMultimodal
    processMultimodal(Message[] dialog, ChatOptions options)
    Applies the chat template to a dialog with images and videos, preprocesses the media, and expands every placeholder to one token per vision patch group exactly as the reference processor does: an image becomes start, image * n, end; a video becomes, per two-frame group, a timestamp text, start, video * n, end.
    void
    serveCluster(ClusterFollower link)
    Follower main loop: executes the leader's mirrored calls on this process's ranks until the leader shuts the cluster down.
    void
    setPrefixReuseEnabled(boolean enabled)
    Enables or disables radix KV prefix reuse on every rank.
    int[]
    stopTokens()
    Returns stop / EOS token ids.
    String
    toString()
     
    String
    tryDecode(int[] tokens, boolean skipSpecial)
    Attempts a streaming decode; returns empty on incomplete UTF-8.
    MiniMaxWeightMode
    weightMode()
    Returns how linear weights are stored and multiplied.

    Methods inherited from class Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

    Methods inherited from interface LanguageModel

    chat, generate

    Methods inherited from interface ModelExecutor

    idleDecodeGraphPrefetch, speculateStep, speculateStep, supportsStepApi, warmPrefix

  • Field Details

  • Constructor Details

    • MiniMax

      public MiniMax(String name, MiniMaxModel model, MiniMaxTokenizer tokenizer, MiniMaxModelArgs params)
      Constructor.
      Parameters:
      name - model name.
      model - the network (weights loaded, KV pool installed).
      tokenizer - the tokenizer.
      params - hyperparameters.

    • MiniMax

      public MiniMax(String name, MiniMaxModel[] models, TensorParallelGroup group, MiniMaxTokenizer tokenizer, MiniMaxModelArgs params)
      Expert-parallel constructor.
      Parameters:
      name - model name.
      models - one replica per rank (weights loaded, KV pools installed), in rank order.
      group - all-reduce group over the ranks' devices, or null for a single rank.
      tokenizer - the tokenizer.
      params - hyperparameters.

  • Method Details

    • attachCluster

      public void attachCluster(ClusterLeader leader)
      Makes this process the leader of a cluster: from now on bind, prefill, decode, finish, evict and the prefix-reuse switch are mirrored to the followers, whose copies of the model execute them in lockstep.
      Parameters:
      leader - the connected cluster.

    • params

      public MiniMaxModelArgs params()
      Returns the hyperparameters.
      Returns:
      model args.

    • network

      public MiniMaxModel network()
      Returns the network of rank 0.
      Returns:
      the model.

    • networks

      public MiniMaxModel[] networks()
      Returns the network of every expert-parallel rank.
      Returns:
      one replica per rank.

    • close

      public void close()
      Specified by:
      close in interface AutoCloseable

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • family

      public String family()
      Description copied from interface: LanguageModel
      Returns the architecture family label.

      Examples: meta/llama3, alibaba/qwen3.5. This is not necessarily the public API model id used by serve.

      Specified by:
      family in interface LanguageModel
      Returns:
      the family label string.

    • name

      public String name()
      Description copied from interface: LanguageModel
      Returns the model instance / checkpoint name.
      Specified by:
      name in interface LanguageModel
      Returns:
      the instance name (e.g. directory or HF repo leaf name).

    • maxSeqLen

      public int maxSeqLen()
      Description copied from interface: LanguageModel
      Returns the configured maximum sequence length (max-model-len / context length).
      Specified by:
      maxSeqLen in interface LanguageModel
      Specified by:
      maxSeqLen in interface ModelExecutor
      Returns:
      the maximum number of tokens allowed for prompt plus completion.

    • isMultimodal

      public boolean isMultimodal()
      Returns whether images and videos can be processed (the checkpoint shipped vision weights and they loaded).
      Returns:
      true when processMultimodal(Message[], ChatOptions) may be called.

    • processMultimodal

      public MiniMaxMultimodal processMultimodal(Message[] dialog, ChatOptions options) throws IOException
      Applies the chat template to a dialog with images and videos, preprocesses the media, and expands every placeholder to one token per vision patch group exactly as the reference processor does: an image becomes start, image * n, end; a video becomes, per two-frame group, a timestamp text, start, video * n, end.
      Parameters:
      dialog - the messages; media parts are image or video URLs (data, http(s) or file).
      options - chat options.
      Returns:
      the expanded token ids and the patches to encode during prefill.
      Throws:
      IOException - if media cannot be read or decoded.
      IllegalArgumentException - if the request is malformed (no vision weights, audio, a media marker typed into text, an extreme aspect ratio, ...).

    • prefillMultimodal

      public Tensor prefillMultimodal(int requestId, MultimodalPrompt prompt, int from, int to)
      Description copied from interface: ModelExecutor
      Prefills a multimodal prompt using vision embeds + interleaved mRoPE. Default throws; Qwen and MiniMax override.
      Specified by:
      prefillMultimodal in interface ModelExecutor
      Parameters:
      requestId - request id.
      prompt - preprocess result.
      from - inclusive start (normally 0; prefix reuse disabled for VL).
      to - exclusive end.
      Returns:
      logits when to == inputIds.length, else null.

    • encodeChat

      public int[] encodeChat(Message... dialog)
      Description copied from interface: LanguageModel
      Encodes a dialog with the model chat template, leaving the assistant turn open for completion.
      Specified by:
      encodeChat in interface LanguageModel
      Parameters:
      dialog - ordered conversation turns (system / user / assistant).
      Returns:
      prompt token ids ready for LanguageModel.generate(int[], int, double, double, boolean, long, GenerationListener).

    • encodeChat

      public int[] encodeChat(Message[] dialog, ChatOptions options)
      Description copied from interface: LanguageModel
      Encodes a dialog with optional tool-calling options.

      Default ignores options and delegates to LanguageModel.encodeChat(Message...).

      Specified by:
      encodeChat in interface LanguageModel
      Parameters:
      dialog - ordered conversation turns.
      options - chat options; may be null.
      Returns:
      prompt token ids.

    • encode

      public int[] encode(String text)
      Encodes plain text (no chat template, no BOS/EOS).
      Parameters:
      text - the text.
      Returns:
      token ids.

    • weightMode

      public MiniMaxWeightMode weightMode()
      Returns how linear weights are stored and multiplied.
      Returns:
      the weight mode.

    • setPrefixReuseEnabled

      public void setPrefixReuseEnabled(boolean enabled)
      Enables or disables radix KV prefix reuse on every rank. The MSA index keys live in side caches addressed by the same slots, so shared prefixes reuse them too.
      Parameters:
      enabled - true to reuse cached prefixes.

    • model

      public LanguageModel model()
      Description copied from interface: ModelExecutor
      Returns the underlying language-model façade.
      Specified by:
      model in interface ModelExecutor
      Returns:
      language model.

    • kvCachePool

      public KvCachePool kvCachePool()
      Description copied from interface: ModelExecutor
      Primary KV cache pool (rank 0 for TP). May be null only for models without full-attention layers.
      Specified by:
      kvCachePool in interface ModelExecutor
      Returns:
      KV cache pool, or null.

    • padToken

      public int padToken()
      Description copied from interface: ModelExecutor
      Returns the pad token id.
      Specified by:
      padToken in interface ModelExecutor
      Returns:
      pad token id.

    • stopTokens

      public int[] stopTokens()
      Description copied from interface: ModelExecutor
      Returns stop / EOS token ids.
      Specified by:
      stopTokens in interface ModelExecutor
      Returns:
      stop token ids.

    • decode

      public String decode(int[] tokens)
      Description copied from interface: ModelExecutor
      Decodes token ids to UTF-8 text.
      Specified by:
      decode in interface ModelExecutor
      Parameters:
      tokens - token ids.
      Returns:
      decoded text.

    • tryDecode

      public String tryDecode(int[] tokens, boolean skipSpecial) throws CharacterCodingException
      Description copied from interface: ModelExecutor
      Attempts a streaming decode; returns empty on incomplete UTF-8.
      Specified by:
      tryDecode in interface ModelExecutor
      Parameters:
      tokens - token ids.
      skipSpecial - whether to skip special tokens.
      Returns:
      decoded text, or empty on incomplete UTF-8.
      Throws:
      CharacterCodingException - when the sequence is incomplete.

    • bind

      public int bind(int[] prompt, int totalCapacity)
      Description copied from interface: ModelExecutor
      Binds one request into the KV pool (multi-request safe).
      Specified by:
      bind in interface ModelExecutor
      Parameters:
      prompt - prompt token ids.
      totalCapacity - desired slots (prompt + generation).
      Returns:
      request id (> 0).

    • prefixLen

      public int prefixLen(int requestId)
      Description copied from interface: ModelExecutor
      Page-aligned matched prefix length for a bound request.
      Specified by:
      prefixLen in interface ModelExecutor
      Parameters:
      requestId - id from ModelExecutor.bind(int[], int).
      Returns:
      prefix length (>= 0).

    • prefill

      public Tensor prefill(int requestId, int[] prompt, int prefixLen)
      Description copied from interface: ModelExecutor
      Prefills the unbound suffix of a prompt (activates this request alone).
      Specified by:
      prefill in interface ModelExecutor
      Parameters:
      requestId - request id.
      prompt - full prompt tokens.
      prefixLen - cached prefix to skip (from ModelExecutor.prefixLen(int)).
      Returns:
      last-position logits shaped [1, vocab] (caller owns).

    • prefillChunk

      public Tensor prefillChunk(int requestId, int[] prompt, int from, int to)
      Description copied from interface: ModelExecutor
      Prefills prompt[from, to) (chunked Continuous Batching). Returns last-token logits only when to == prompt.length.
      Specified by:
      prefillChunk in interface ModelExecutor
      Parameters:
      requestId - request id.
      prompt - full prompt tokens.
      from - inclusive start position in the prompt.
      to - exclusive end position.
      Returns:
      logits [1, vocab] when the prompt is fully prefilled; otherwise null.

    • decodeStep

      public Tensor decodeStep(int[] requestIds, int[] lastTokens, int[] positions)
      Description copied from interface: ModelExecutor
      One decode step for an active set (same or mixed positions; engine may cohort by position). Tokens are [B] last tokens; positions are the write index for each row.
      Specified by:
      decodeStep in interface ModelExecutor
      Parameters:
      requestIds - active KV request ids (order = batch).
      lastTokens - token id per request.
      positions - absolute position per request.
      Returns:
      logits [B, vocab] (caller owns).

    • finish

      public void finish(int requestId, int[] sequenceTokens)
      Description copied from interface: ModelExecutor
      Inserts into the radix tree (when enabled) and unbinds the request.
      Specified by:
      finish in interface ModelExecutor
      Parameters:
      requestId - request id.
      sequenceTokens - prompt + completion (no pad).

    • evict

      public void evict(int requestId)
      Description copied from interface: ModelExecutor
      Instant Eviction without radix insert.
      Specified by:
      evict in interface ModelExecutor
      Parameters:
      requestId - request id.

    • generate

      public ChatCompletion generate(int[] prompt, int maxGenLen, double temperature, double topp, boolean logprobs, long seed, GenerationListener listener, BooleanSupplier cancelRequested)
      Description copied from interface: LanguageModel
      Generates a completion from an already-tokenized prompt.
      Specified by:
      generate in interface LanguageModel
      Parameters:
      prompt - prompt token id sequence.
      maxGenLen - maximum number of new tokens to generate (not including the prompt itself).
      temperature - sampling temperature; higher values increase randomness.
      topp - nucleus-sampling top-p threshold in (0, 1].
      logprobs - true to include per-token log-probabilities in the result.
      seed - optional RNG seed for deterministic sampling; 0 means non-deterministic.
      listener - optional progress callback; may be null.
      cancelRequested - when non-null and returns true, generation stops between decode steps with CancellationException.
      Returns:
      the completion for prompt.

    • serveCluster

      public void serveCluster(ClusterFollower link) throws Exception
      Follower main loop: executes the leader's mirrored calls on this process's ranks until the leader shuts the cluster down. Results (logits) are discarded: only the leader samples.
      Parameters:
      link - the connection to the leader.
      Throws:
      Exception - the first failing call (already reported to the leader).

    • chat

      public ChatCompletion chat(Message[] dialog, int maxGenLen, double temperature, double topp, boolean logprobs, long seed, GenerationListener listener, BooleanSupplier cancelRequested)
      Description copied from interface: LanguageModel
      Generates an assistant reply for a dialog.

      Equivalent to LanguageModel.encodeChat(Message...) followed by LanguageModel.generate(int[], int, double, double, boolean, long, GenerationListener).

      Specified by:
      chat in interface LanguageModel
      Parameters:
      dialog - ordered conversation turns.
      maxGenLen - maximum number of new tokens to generate.
      temperature - sampling temperature; higher values increase randomness.
      topp - nucleus-sampling top-p threshold in (0, 1].
      logprobs - true to include per-token log-probabilities in the result.
      seed - optional RNG seed for deterministic sampling; 0 means non-deterministic.
      listener - optional progress callback; may be null.
      cancelRequested - cooperative cancel flag; may be null.
      Returns:
      the completion for dialog.

    • build

      public static MiniMax build(String checkpointDir, int maxBatchSize, int maxSeqLen, byte deviceId, double memFractionStatic, int pageSize) throws IOException
      Builds a model from a HuggingFace BF16 checkpoint directory on a single device.
      Parameters:
      checkpointDir - directory with config.json, tokenizer.json and safetensors.
      maxBatchSize - maximum concurrent requests.
      maxSeqLen - maximum context length; <= 0 uses max_position_embeddings.
      deviceId - CUDA device index, or a negative value for CPU.
      memFractionStatic - fraction of device memory for weights + KV; <= 0 sizes the pool for tests.
      pageSize - KV page size in tokens.
      Returns:
      the loaded model.
      Throws:
      IOException - if the checkpoint is missing, quantized or incomplete.

    • build

      public static MiniMax build(String checkpointDir, int maxBatchSize, int maxSeqLen, byte[] devices, double memFractionStatic, int pageSize) throws IOException
      Builds a model from a HuggingFace BF16 checkpoint directory, sharding the routed experts of every MoE layer across devices (expert parallelism with an all-reduce; attention, dense FFNs, router and shared expert are replicated on every rank).
      Parameters:
      checkpointDir - directory with config.json, tokenizer.json and safetensors.
      maxBatchSize - maximum concurrent requests.
      maxSeqLen - maximum context length; <= 0 uses max_position_embeddings.
      devices - CUDA device indices, one rank each, or null for a single CPU rank.
      memFractionStatic - fraction of device memory for weights + KV; <= 0 sizes the pool for tests.
      pageSize - KV page size in tokens.
      Returns:
      the loaded model.
      Throws:
      IOException - if the checkpoint is missing, quantized or incomplete.

    • build

      public static MiniMax build(String checkpointDir, int maxBatchSize, int maxSeqLen, byte[] devices, double memFractionStatic, int pageSize, String weightMode) throws IOException
      Builds a model, choosing how MXFP8 weights are stored and multiplied.
      Parameters:
      checkpointDir - directory with config.json, tokenizer.json and safetensors.
      maxBatchSize - maximum concurrent requests.
      maxSeqLen - maximum context length; <= 0 uses max_position_embeddings.
      devices - CUDA device indices, one expert-parallel rank each, or null for one CPU rank.
      memFractionStatic - fraction of device memory for weights + KV; <= 0 sizes the pool for tests.
      pageSize - KV page size in tokens.
      weightMode - auto, marlin, fp8, fp8emu or dequant; null reads SMILE_MINIMAX_MXFP8 (default auto).
      Returns:
      the loaded model.
      Throws:
      IOException - if the checkpoint is missing, unsupported or incomplete.

    • build

      public static MiniMax build(String checkpointDir, int maxBatchSize, int maxSeqLen, byte[] devices, double memFractionStatic, int pageSize, String weightMode, MiniMax.ClusterSpec cluster) throws IOException
      Builds this process's part of a model whose expert-parallel ranks span several processes (and nodes). devices are the GPUs of this process; its ranks are cluster.rankOffset() .. of cluster.worldSize(), and every process must call this at the same time (the NCCL communicator is created first and returns when all ranks have joined). With a null cluster this is the single-process build.
      Parameters:
      checkpointDir - directory with config.json, tokenizer.json and safetensors.
      maxBatchSize - maximum concurrent requests.
      maxSeqLen - maximum context length; <= 0 uses max_position_embeddings.
      devices - CUDA device indices of this process, one expert-parallel rank each.
      memFractionStatic - fraction of device memory for weights + KV.
      pageSize - KV page size in tokens.
      weightMode - see build(String, int, int, byte[], double, int, String).
      cluster - this process's place in the cluster, or null.
      Returns:
      the loaded model.
      Throws:
      IOException - if the checkpoint is missing, unsupported or incomplete.

    • buildLeader

      public static MiniMax buildLeader(String checkpointDir, int maxBatchSize, int maxSeqLen, byte[] devices, double memFractionStatic, int pageSize, String weightMode, int port, int followers, long timeoutMs) throws IOException
      Builds the leader of a multi-node model: waits for followers follower processes (MiniMaxFollower) to connect, builds this process's ranks together with theirs, and mirrors every state change to them from then on. The leader owns world ranks 0 .. devices.length - 1.
      Parameters:
      checkpointDir - checkpoint directory on this node.
      maxBatchSize - maximum concurrent requests.
      maxSeqLen - maximum context length; <= 0 uses max_position_embeddings.
      devices - CUDA device indices of this process.
      memFractionStatic - fraction of device memory for weights + KV (the followers use the same).
      pageSize - KV page size in tokens.
      weightMode - see build(String, int, int, byte[], double, int, String); null for auto.
      port - TCP port the followers connect to.
      followers - number of follower processes.
      timeoutMs - how long to wait for the followers.
      Returns:
      the leader model.
      Throws:
      IOException - if the followers do not connect or the checkpoint cannot be loaded.