Handler for `POST /api/v1/llm/loaded-models`.
POST/api/v1/llm/loaded-models
Loads the model under the settings given, for this instance alone, over its default preset; the instance then serves every request for the model. A load the guardrail refuses is 409 with the estimate and the budget.
Request
Responses
- 200
- 404
- 409
- 422
- 503
Model loaded
Not a model of the model index
The load guardrail refused the model; confirm the load to go ahead
Not a language model, an unknown setting, or a setting for an MLX model
No local engine runs this model, or the engine did not start