Package smile.llm.cluster

package smile.llm.cluster
  • Class
    Description
    ClusterFollower
    The follower side of a multi-node model: connects to a ClusterLeader, learns its ranks and the NCCL unique id, then executes every message the leader sends, in order, acknowledging each.
    ClusterFollower.Handler
    Executes one message; the follower acknowledges it afterwards.
    ClusterFollower.Welcome
    What the leader tells a follower on connection.
    ClusterLeader
    The control side of a multi-node model: the leader process owns the server and the engine, and every call that changes model state (bind, prefill, decode step, finish...) is mirrored to the follower processes, which hold the other ranks and execute the identical call so that the NCCL collectives of all ranks line up.
    ClusterLeader.PayloadWriter
    Writes the arguments of a message.