(Constructs README): add Python code snippets about generative-ai-cdk-constructs HOT 11 CLOSED

Hi @krokoko, I would like to work on this issue.

from generative-ai-cdk-constructs.

Hi @martinezedgar , thank you ! Sure, I will assign the ticket to yourself.

2 main items for this ticket:

1. For python code snippets, you can add them to each readme file below the existing Typescript code snippet, something like:

Typescript

code snippet

Python

code snippet

2. Add python package to the existing table in each readme. It would look as follow:

Language	Package
Typescript	@cdklabs/generative-ai-cdk-constructs
Python	cdklabs.generative_ai_cdk_constructs

from generative-ai-cdk-constructs.

Perfect, I will do it like that.

from generative-ai-cdk-constructs.

Hi @krokoko, working in the issue I am facing an error with the Python implementation of HuggingFaceSageMakerEndpoint. This is how I am implementing the HuggingFaceSageMakerEndpoint in Python:

HuggingFaceSageMakerEndpoint(
            self,
            id='Mistral',
            props={
                'model_id': 'mistralai/Mistral-7B-Instruct-v0.1',
                'instance_type': SageMakerInstanceType.ML_G5_2_XLARGE,
                'container': DeepLearningContainerImage.HUGGINGFACE_PYTORCH_TGI_INFERENCE_2_0_1_TGI1_1_0_GPU_PY39_CU118_UBUNTU20_04,
                'environment': {
                    'SM_NUM_GPUS': '1',
                    'MAX_INPUT_LENGTH': '2048',
                    'MAX_TOTAL_TOKENS': '4096',
                },
            },
        )

After running cdk synth this is the error that I get:

jsii.errors.JavaScriptError: 
  @jsii/kernel.SerializationError: Passed to parameter props of new @cdklabs/generative-ai-cdk-constructs.HuggingFaceSageMakerEndpoint: Unable to deserialize value as @cdklabs/generative-ai-cdk-constructs.IHuggingFaceSageMakerEndpointProps
  ├── 🛑 Failing value is an object
  │      { '$jsii.map': [Object] }
  ╰── 🔍 Failure reason(s):
      ╰─ Value does not have the "$jsii.byref" key
      at Object.process (/private/var/folders/kv/rk20vrnn0xv4psrsn47ffq8h0000gn/T/tmpthv9aq8_/lib/program.js:11466:19)
      at Kernel._Kernel_toSandbox (/private/var/folders/kv/rk20vrnn0xv4psrsn47ffq8h0000gn/T/tmpthv9aq8_/lib/program.js:10444:25)
      at /private/var/folders/kv/rk20vrnn0xv4psrsn47ffq8h0000gn/T/tmpthv9aq8_/lib/program.js:10460:38
      at Array.map (<anonymous>)
      at Kernel._Kernel_boxUnboxParameters (/private/var/folders/kv/rk20vrnn0xv4psrsn47ffq8h0000gn/T/tmpthv9aq8_/lib/program.js:10460:23)
      at Kernel._Kernel_toSandboxValues (/private/var/folders/kv/rk20vrnn0xv4psrsn47ffq8h0000gn/T/tmpthv9aq8_/lib/program.js:10448:101)
      at Kernel._Kernel_create (/private/var/folders/kv/rk20vrnn0xv4psrsn47ffq8h0000gn/T/tmpthv9aq8_/lib/program.js:10108:115)
      at Kernel.create (/private/var/folders/kv/rk20vrnn0xv4psrsn47ffq8h0000gn/T/tmpthv9aq8_/lib/program.js:9779:93)
      at KernelHost.processRequest (/private/var/folders/kv/rk20vrnn0xv4psrsn47ffq8h0000gn/T/tmpthv9aq8_/lib/program.js:11696:36)
      at KernelHost.run (/private/var/folders/kv/rk20vrnn0xv4psrsn47ffq8h0000gn/T/tmpthv9aq8_/lib/program.js:11656:22)

I have tried multiple things but can't make it work, so I would like to know if there is something wrong with my Python implementation or if this is caused by another issue before submitting this implementation in the documentation.

Thank you in advance for your feedback.

from generative-ai-cdk-constructs.

Hi @martinezedgar , thanks ! I see you opened a PR, do you still have an issue with the HuggingFaceSageMakerEndpoint construct in Python ? Are you facing the same issue for the JumpStart version ? (I see the comment in your PR)

from generative-ai-cdk-constructs.

Hi @krokoko, yes I am still having the same issues in HuggingFace and JumpStart but I included the preliminary snippets for both in their respective README files. The other snippets work just fine, do you have any idea about what is going on?

from generative-ai-cdk-constructs.

I will have a look and try to reproduce

from generative-ai-cdk-constructs.

Hi @martinezedgar I was able to reproduce, will investigate and provide an update here, thank you for reporting this issue !

from generative-ai-cdk-constructs.

Thank you very much for your help @krokoko, let me know if there is anything I can help with.

from generative-ai-cdk-constructs.

@martinezedgar I just opened a PR to address the issue (#218), once it is merged you will be able to add the missing snippets

from generative-ai-cdk-constructs.

Perfect, I'll wait.

from generative-ai-cdk-constructs.