monai.deploy.core.models.TritonModel#

class monai.deploy.core.models.TritonModel(path, name='')[source]#

Bases: Model

Represents Triton models in the model repository.

Triton Inference Server models are stored in a directory structure like this (triton-inference-server/server):

<model-repository-path>/
    <model-name>/
    [config.pbtxt]
    [<output-labels-file> ...]
    <version>/
        <model-definition-file>
    <version>/
        <model-definition-file>
    ...
    <model-name>/
    [config.pbtxt]
    [<output-labels-file> ...]
    <version>/
        <model-definition-file>
    <version>/
        <model-definition-file>
    ...
    ...

This class checks if the given path meets the folder structure of Triton:

The path should be a folder path.
The model folder must contain a config.pbtxt file.
1. A config.pbtxt file may contain model name. In that case, model’s name should match with the folder name.
The model folder may include one or more folders having a positive integer value as version. For the server side, the following is required, however, not required for the client side which parses only the model config.pbtxt file.
1. Each such folder must contain a folder or file whose file name (without extension) is ‘model’, unless an attribute is used to specify model file name.

If no version policy is specified, the latest version of a model is loaded and served. Model items identified would have a folder path, not a specific model file path.

This class encapuslates a single triton model. As such, the model repository folder will first be parsed by the named_model class, and then each sub folder by this class.

Initializes a TritonModel.

This assumes that the given path is a valid Triton model repository.

Parameters:

path (str) – A Path to the model repository.
name (str) – A name of the model.

Methods

`__init__`(path[, name])	Initializes a TritonModel.
`accept`(path)	Check if the path is a type of this model class.
`class_name`()	Return a name of the model class.
`connect`(netloc, **kwargs)	Connect to the Triton Inference Server at the network location.
`get`([name])	Return a model object by name.
`get_model_list`()	Return a list of models in the repository.
`items`()	Return an ItemsView of models that this Model instance has.
`register`(cls_list)	Register a list of model classes.
`registered_models`()	Return a list of registered model classes.

Attributes

`model_config`
`model_type`
`name`	Return a name of the model.
`net_loc`	<port>".
`path`	Return a path to the model.
`predictor`	Return a predictor of the model.

__init__(path, name='')[source]#

Initializes a TritonModel.

This assumes that the given path is a valid Triton model repository.

Parameters:

path (str) – A Path to the model repository.
name (str) – A name of the model.

classmethod accept(path)[source]#

Check if the path is a type of this model class.

Parameters:: path (str) – A path to a model.
Return type:: Tuple[bool, str]
Returns:: (True, <model_type>) if the path is a type of this model class, (False, “”) otherwise.

classmethod class_name()#: Return a name of the model class.

connect(netloc, **kwargs)[source]#

Connect to the Triton Inference Server at the network location.

Parameters:: netloc (str) – The network location of the Triton Inference Server.

get(name='')#

Return a model object by name.

If there is only one model in the repository or the model path, model object can be returned without specifying name.

If there are more than one models in the repository, the model object can be returned by name whose name matches the provided name.

Parameters:: name (str) – A name of the model.
Return type:: Model
Returns:: A model object is returned, matching the provided name if given.

get_model_list()#

Return a list of models in the repository.

If this model represents a model repository, then a list of model objects (name and path) is returned. Otherwise, a single model object list is returned.

Return type:: List[Dict[str, str]]
Returns:: A list of models (name, path dictionary) in the repository.

items()#

Return an ItemsView of models that this Model instance has.

If this model represents a model repository, then an ItemsView of submodel objects is returned. Otherwise, an ItemsView of a single model object (self) is returned.

Returns:: <model name>: <model object>.
Return type:: An ItemView of models

property name#: Return a name of the model.

property net_loc#

<port>”.

Returns:: The network location of the Triton Inference Server.
Return type:: str
Type:: Get the network location of the Triton Inference Server, i.e. “<host>

property path#: Return a path to the model.

property predictor#

Return a predictor of the model.

Returns:: A predictor of the model.

static register(cls_list)#: Register a list of model classes.

static registered_models()#: Return a list of registered model classes.

monai.deploy.core.models.TritonModel#

This Page