MedCPT generates embeddings of biomedical texts that can be used for semantic search (dense retrieval). MedCPT Article Encoder: compute the embeddings of articles (e.g., PubMed titles & abstracts). In this template, we will import the MedCPT Article Encoder on the Inferless Platform.

• Git. You would need git installed on your system if you wish to customize the repo after forking.
• Python>=3.8. You would need Python to customize the code in the app.py according to your needs.
• Curl. You would need Curl if you want to make API calls from the terminal itself.

Here is a quick start to help you get up and running with this template on Inferless.

Get started by forking the repository. You can do this by clicking on the fork button in the top right corner of the repository page.

This will create a copy of the repository in your own GitHub account, allowing you to make changes and customize it according to your needs.

Log in to your inferless account, select the workspace you want the model to be imported into and click the Add Model button.

Select the PyTorch as framework and choose Repo(custom code) as your model source and use the forked repo URL as the Model URL.

Enter all the required details to Import your model. Refer this link for more information on model import.

The following is a sample Input and Output JSON for this model which you can use while importing this model on Inferless.

{
  "inputs": [
    {
      "name": "articles",
      "datatype": "BYTES",
      "shape": [
        6
      ],
      "data": [
        "Diagnosis and Management of Central Diabetes Insipidus in Adults",
        "Central diabetes insipidus (CDI) is a clinical syndrome which results from loss or impaired function of vasopressinergic neurons in the hypothalamus/posterior pituitary, resulting in impaired synthesis and/or secretion of arginine vasopressin (AVP). [...]",
        "Adipsic diabetes insipidus",
        "Adipsic diabetes insipidus (ADI) is a rare but devastating disorder of water balance with significant associated morbidity and mortality. Most patients develop the disease as a result of hypothalamic destruction from a variety of underlying etiologies. [...]",
        "Nephrogenic diabetes insipidus: a comprehensive overview",
        "Nephrogenic diabetes insipidus (NDI) is characterized by the inability to concentrate urine that results in polyuria and polydipsia, despite having normal or elevated plasma concentrations of arginine vasopressin (AVP). [...]"
      ]
    }
  ]
}

{
  "outputs": [
    {
      "name": "embeds",
      "datatype": "FP32",
      "shape": [
        -1
      ],
      "data": [
        0.0413
      ]
    }
  ]
}

Following is an example of the curl command you can use to make inference. You can find the exact curl command in the Model's API page in Inferless.

curl --location '<your_inference_url>' \
          --header 'Content-Type: application/json' \
          --header 'Authorization: Bearer <your_api_key>' \
          --data '{
              "inputs": [
                {
                  "name": "articles",
                  "datatype": "BYTES",
                  "shape": [
                    6
                  ],
                  "data": [
                    "Diagnosis and Management of Central Diabetes Insipidus in Adults",
                    "Central diabetes insipidus (CDI) is a clinical syndrome which results from loss or impaired function of vasopressinergic neurons in the hypothalamus/posterior pituitary, resulting in impaired synthesis and/or secretion of arginine vasopressin (AVP). [...]",
                    "Adipsic diabetes insipidus",
                    "Adipsic diabetes insipidus (ADI) is a rare but devastating disorder of water balance with significant associated morbidity and mortality. Most patients develop the disease as a result of hypothalamic destruction from a variety of underlying etiologies. [...]",
                    "Nephrogenic diabetes insipidus: a comprehensive overview",
                    "Nephrogenic diabetes insipidus (NDI) is characterized by the inability to concentrate urine that results in polyuria and polydipsia, despite having normal or elevated plasma concentrations of arginine vasopressin (AVP). [...]"
                  ]
                }
              ]
            }
            '

Open the app.py file. This contains the main code for inference. It has three main functions, initialize, infer and finalize.

Initialize - This function is executed during the cold start and is used to initialize the model. If you have any custom configurations or settings that need to be applied during the initialization, make sure to add them in this function.

Infer - This function is where the inference happens. The argument to this function inputs, is a dictionary containing all the input parameters. The keys are the same as the name given in inputs. Refer to input for more.

def infer(self, inputs):
    prompt = inputs["prompt"]

Finalize - This function is used to perform any cleanup activity for example you can unload the model from the gpu by setting self.pipe = None.

For more information refer to the Inferless docs.