Interface SageMakerEndpointInput

The SageMakerEndpointInput interface defines the input parameters for the SageMakerEndpoint class, which includes the endpoint name, client options for the SageMaker client, the content handler, and optional keyword arguments for the model and the endpoint.

interface SageMakerEndpointInput {
    clientOptions: SageMakerRuntimeClientConfig;
    contentHandler: SageMakerLLMContentHandler;
    endpointName: string;
    endpointKwargs?: Record<string, unknown>;
    modelKwargs?: Record<string, unknown>;
    streaming?: boolean;
}

Hierarchy

  • BaseLLMParams
    • SageMakerEndpointInput

Properties

clientOptions contentHandler endpointName endpointKwargs? modelKwargs? streaming?

Properties

clientOptions

clientOptions: SageMakerRuntimeClientConfig

Options passed to the SageMaker client.

contentHandler

contentHandler: SageMakerLLMContentHandler

The content handler class that provides an input and output transform functions to handle formats between LLM and the endpoint.

endpointName

endpointName: string

The name of the endpoint from the deployed SageMaker model. Must be unique within an AWS Region.

Optional endpointKwargs

endpointKwargs?: Record<string, unknown>

Optional attributes passed to the InvokeEndpointCommand

Optional modelKwargs

modelKwargs?: Record<string, unknown>

Key word arguments to pass to the model.

Optional streaming

streaming?: boolean

Settings

Member Visibility

Theme

On This Page

Generated using TypeDoc