mit-ai-accelerator / c3po-model-server Goto Github PK

Summarization errors out if the prompt/input is too long. We also need to shorten the amount of time it takes to generate a summary or else it will time out on P1

Used in Minio storage api within app to express errors. extend from root AppError object.

MinioSaveError:
{ model_type, sha, time, ...rest of error }

MinioLoadError:
{ model_type, sha, time, ...rest of error }

Currently we have 2 pipelines, but the possibility of a 3rd if we convert the streamlit app UI to another pipeline.
How do we want to restructure the architectures?
Decide on the following:

• What will each pipeline be used for?
• Components/applications that should be included in each pipeline

While the current implementation works, it does risk erroring out if the underlying upload_file_to_minio implementation changes. Recommend adjusting to follow the mock pattern in the remainder of the same module. Specifically, adjust the following lines in test_upload_bertopic_embedding_pretrained_weak_learner_object_post_valid_request to directly patch the upload call:

    # Dump the object to the file object
    file_obj = io.BytesIO()
    pickle.dump(weak_learner_model_obj, file_obj)
    file_obj.seek(0)

    # Upload the file to the BERTopic Embedding Pretrained Model object
    response2 = client.post(
        f"/aimodels/bertopic/bertopic-embedding-pretrained/{embedding_pretrained_id}/upload/", files={"new_file": file_obj})

    os.remove(test_file)

How to do in P1
Brett Ostwalt commented:

Eric Robinson (AIA | 21 AS) alright connectivity should be working now.

in order to call the NITMRE services, the addresses are:

http://mattermost.nitmre:8065
http://nitmre-cpd.nitmre
If NITMRE is calling anything for transformers, the address is:

http://c3po-model-server.transformers
there should be less header information in general, but something that may be relevant is:

'x-forwarded-client-cert', 'By=spiffe://cluster.local/ns/nitmre/sa/default;Hash=;Subject="";URI=spiffe://cluster.local/ns/transformers/sa/default'
That is a call from transformers to nitmre so I imagine if it went the other way, then the spiffe values would be reversed

How to do locally

I just added a bunch of instructions to the readme in gitlab: https://code.il4.dso.mil/platform-one/products/581-swes/nitmre/nitmre-cpd/-/tree/master?ref_type=heads#run-locally-via-docker-compose

You should be able to build the docker image with nothing but docker installed. Alternatively, you can skip trying to build yourself and just change the provided sample docker compose (standalone-docker-compose.yml) to pull the latest image from P1 gitlab, the image url would be: registry.il4.dso.mil/platform-one/products/581-swes/nitmre/nitmre-cpd, I think you may have to lookup and specify the current tag (IIRC P1 does not allow using :latest).

David Milligan

Links for using AFRL tool
Hi Eric,

Here is the link to the ATICA code repository. Your user accounts were created.

https://github.rpi.edu/LACAI/ATICAv3

IntrucTOD llama2: https://drive.google.com/file/d/1OLlvpNiMO_MMx586_yfdZZXyAZUfor-Z/view?usp=sharing_eil_m&ts=64ca8ce8

all mini:

{
  "sha256": "178d15cd14cfa17b445cdb3f98815f6875be34c84f0cd2997cf51455abc7680d",
  "model_type": "sentence_transformers",
  "model_name": "all-MiniLM-L6-v2",
  "id": "9edf2ea6-cac0-4665-9b9a-3224e6e5d4f2",
  "uploaded": true,
  "version": 1,
  "originated_from": "app"
}

need to upload mpnet and add here too