amazon-connect / ai-powered-speech-analytics-for-amazon-connect Goto Github PK

I notice that the agentAssist.html deployed via the cloudformation stack does not match what is in github. Is there an eta for getting these in sync?

I uploaded the template, followed the instructions on the blog post and admin guide but cannot get chat or audio transcripts to work. Currently the chat comes in, there is no detection in the custom CCP at all. I have a test chat to the aiChatFlow that comes in the s3 bucket. Please let me know what other information you need to help diagnose the issue.

Hi,
For some reason, I got the first 3 transcripts saved on my S3 bucket (inside "transcripts" folder). However, none of the calls after that got transcripts saved. The real-time transcripts was still working. All audios were still saved in "recordings" folder.
Could someone tell me where and which variables I need to check for this issues?
Here are some parameters I have set:
transcribeCall : TRUE
saveCallRecording: TRUE
CONSOLE_LOG_TRANSCRIPT_FLAG : TRUE
SAVE_PARTIAL_TRANSCRIPTS : TRUE
START_SELECTOR_TYPE : FRAGMENT_NUMBER
Thanks a lot.

I follow the steps in the deployment guide (unless I missed something). I open up the web app, but when i get the authentication popup from Connect, I get a "Session Expired Your session has expired. Please log in."

This happens if I'm logged into Connect on another tab or not.

Could this be related to the new version of connecting to Connect?

Hi,

I tried to used aichatflow for Chat with to Translate but it does nothing. chats dosent show on Translated Conversation and unable to send chat only native tool works. Sentiment is also not showing.

FYI - Voice works but translation is slow.

Please see the sample screenshot.

I am struggling from past few days to fix this but failed.

Please need your help.

Thanks in advance.

The whole flow remains untouched. Only changes are being made on AgentUI page to call certain APIs. Everything was working completely fine until yesterday afternoon. Suddenly the last bit stopped working on its own. Attaching screenshot of AgentUI backend console screen.

I see from the contact flow I can enable options to stream both agent and customer voice.
I need to set the speaker identification and to receive the transcription for both voices, I am not able to figure out how to achieve this ? Please guide me in setting it up ASAP.

References I went through-

1. Enabling two way streams
2. Setting diarization

A freshly deployed stack tested on chrome 81 (Version 81.0.4044.113 (Official Build) (64-bit)).

When getting a call in console these errors can be seen:

Wed Apr 22 2020 08:50:00 GMT+0300 (Eastern European Summer Time) " Message received from WS"
4lily-rtc.6f6f9376.js:9999 Uncaught Error: Failed to extract MediaRtpStats from RTCStatsReport for stream type audio_input
    at lily-rtc.6f6f9376.js:9999
(anonymous) @ lily-rtc.6f6f9376.js:9999
(index):517 contact refreshed
32lily-rtc.6f6f9376.js:9999 Uncaught Error: Failed to extract MediaRtpStats from RTCStatsReport for stream type audio_input
    at lily-rtc.6f6f9376.js:9999

Also, sample UI shows this status:

customerSentiment	Neutral
customerSentimentScore	0.ee9c92
languageCode	en-US
saveCallRecording	true
startStreamingAudioStatus	Failed
transcribeCall	true

In S3 audio bucket I can see empty transcriptions and non of the audio files. It is because the startStreamingAudio has failed. Any suggestions how to fix that?

More context:
Stack deployed to Frankfurt (eu-central-1).
Phone number picked from Lithuania pool.
Making a call from Lithuania.

Hi,

I deployed the stack in ap-southeast-1 and us-east-1 respectively and found out that in ap-southeast-1, KVSTranscribeStreamingLambda is failed to run due to the wrong endpoint address
"https://transcribestreaming." + TRANSCRIBE_REGION.getName() + ".amazonaws.com"

According to the official doc, the endpoint should be defined as below

private static final String TRANSCRIBE_ENDPOINT = "https://transcribe." + TRANSCRIBE_REGION.getName() + ".amazonaws.com";

Please kindly help to address it or correct me if any misunderstanding.

Thanks

Hi, I am following the steps given in the README.md to build the source.

Getting below error in the last step of the build-s3-dist.sh. It seems, the manifest-generator is missing in the source. Can anyone help on this issue?

./build-s3-dist.sh: line 262: cd: /Users/KK/Projects/ai-powered-speech-analytics-for-amazon-connect-1.1.2/deployment/../source/../deployment/manifest-generator: No such file or directory

Remediation as per Apache here, see sections "Fixed in Log4j 2.15.0" and "Fixed in Log4j 2.17.0"

Please follow the steps in README.md to make a new build and deploy with the latest versions of amazon-kinesis-video-streams-parser-library (which has been patched with log4j v2.17.0), org.slf4j:slf4j-api, and org.slf4j:slf4j-log4j12.

Please note that, at the time of writing this comment, v1.1.0 of amazon-kinesis-video-streams-parser-library was still not available on Maven, hence you would need to clone https://github.com/aws/amazon-kinesis-video-streams-parser-library first, and run mvn clean install to build amazon-kinesis-video-streams-parser-library. Once completed, run build in ai-powered-speech-analytics-for-amazon-connect.

For customers who originally deployed via AWS Solution Guide https://docs.aws.amazon.com/solutions/latest/ai-powered-speech-analytics-for-amazon-connect/template.html, we are currently working on AWS Solution patching, and will post an update as soon as possible.

I cloned the repo (tag v1.1.2) and followed the instructions in README to build distributable for customisation, got the following error:

./build-s3-dist.sh: line 262: cd: /Users/Gennadii_Aleksandrov/Documents/git/other/AmazonConnectSandbox/ai-powered-speech-analytics-for-amazon-connect/deployment/../source/../deployment/manifest-generator: No such file or directory

After that the build failed with this:

npm ERR! code ENOENT
npm ERR! syscall open
npm ERR! path /Users/Gennadii_Aleksandrov/Documents/git/other/AmazonConnectSandbox/ai-powered-speech-analytics-for-amazon-connect/source/package.json
npm ERR! errno -2
npm ERR! enoent ENOENT: no such file or directory, open '/Users/Gennadii_Aleksandrov/Documents/git/other/AmazonConnectSandbox/ai-powered-speech-analytics-for-amazon-connect/source/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/Gennadii_Aleksandrov/.npm/_logs/2022-08-12T16_47_09_881Z-debug-0.log
node:internal/modules/cjs/loader:959
  throw err;
  ^

Error: Cannot find module '/Users/Gennadii_Aleksandrov/Documents/git/other/AmazonConnectSandbox/ai-powered-speech-analytics-for-amazon-connect/source/app.js'
    at Module._resolveFilename (node:internal/modules/cjs/loader:956:15)
    at Module._load (node:internal/modules/cjs/loader:804:27)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
    at node:internal/main/run_main_module:17:47 {
  code: 'MODULE_NOT_FOUND',
  requireStack: []
}

Node.js v18.7.0

Hi, I added a third choice in the contact flow to add french transcription and set the attribute languageCode to fr-FR. Any idea why there is no french transcription in the CCP?

Also, where can I add french a an translation option?

Thanks

The suggested actions are not populating in the demo even when causing the sample utterances to appear in the transcription. Browser: Chrome 75

KVSの45分のリミットを超えるために、42分で再リクエストしていますが、
まだTranscribeが完了していないうちに再実行してしまうため、20秒程度のテキスト化の結果が欠落します。

また、稀に継続せずに顧客またはオペレータの文字起こしが終了してしまいます。
こちらに関しては、GetMedia API より返却されるペイロードから AWS_KINESISVIDEO_CONTINUATION_TOKEN を取得して、
そのトークンを指定して再度 GetMedia を行うよう修正することで対応出来ました。

I want the agent transcription in Real-Time along with customer transcription.

The maximum session duration (in seconds) of the IAM Role is set to 900 seconds in the initContactDetails Lambda Function. It need to be increased to run the solution beyond 900 seconds.

[Feature] Suggestion

For now, only en-US and es-US are available on kvs_trigger function. I would like to add more compatible language in order improve that solution.

I would like to also translate current flow (agent and contact) into those languages and give choice to user when deploying the stack to choose one.

When I launch the UI, For the agent assist section, I get below error message in browser

"To protect your security, ab2-roopali-connect-demo.awsapps.com will not allow Firefox to display the page if another site has embedded it. "
On opening in new browser, I get error -
"Access Error
This application has not been enabled for your directory. Please contact your Administrator for more details."

In developer tools, I can see below error -
[HTTP/1.1 403 Forbidden 0ms]
The loading of “https://*******/connect/login” in a frame is denied by “X-Frame-Options“ directive set to “SAMEORIGIN“.

I will appreciate if you can help with this error

Stack fails to delete due to a S3 bucket not being empty.

Replication:
Delete deployed stack.

Proposed resolution:
Consider adding code below to the createS3Bucket resource:

createS3Bucket:
    Type: 'AWS::S3::Bucket'
    DeletionPolicy: Retain
    UpdateReplacePolicy: Retain
    {...}

Although it seems like this code is fairly recently built, the Java SDK says that lambda.invokeAsync is deprecated, which appears in index.js of kvs_trigger. If we're trying to deploy this is a lambda function ourselves (with a few modifications), should we swap out the invokeAsync with invoke(request)? Is there a different version that ran with the CloudFormation template?

How can I get Outgoing Transcript of both agent and customer.

Is it possible to add more languages to translate into? I'm trying to add French, but I have no idea where I might have to modify the code.

Hello I am trying to get the transcription from customer call. I was able to get the audio in S3 bucket. But at the time of real time transcription I am not getting exact output.
Ex. When I say "balance" or anything else it is just storing Recorded,Courted or where did.
It is not storing the actual speech.
Please reply this is kind of urgent

There is a delay of almost 6-7 seconds between what I speak and the real time transcription that happens, i.e. the text comes up on screen after 6-7 seconds after I finish speaking. The network speed is almost 100 mbps, still this issue occurs.

I am able to save the transcript to DDB but not able to get it on frontend.
I tried to make a clone of lambda funtion ( Ai-powered-speech-analyti-wsrealtimetranscribeOnDD ) and set the trigger on tocustomer DDB but not able to get the data on frontend.
I think DDB scan method returning empty array instead of transcript .
can you help me with the same. what I need to do and get the agent transcript on front end.

@dfw100 You provided some help in the #24 issue but I am still very unclear. Can you please provide detailed steps to get the real time agent transcript?
@dfw100

@efh365
The solution currently provides the real time transcript of the customer only. I tried multiple things but to no avail. Can you please tell me how to get the agent transcript as well? @efh365

After communication with AWS Support Center, I made this issue.
In transcribeStreaming, languageCode korean(ko-KR) is not working.

When I tested KVSTranscriber in English(en-US) without any settings, There's no problem in process of speech-to-text.
In reference of other language (ex> en-GB, es-US), I set the languageCode to ko-KR, many errors occurred like below:

2021-03-12 04:32:18 INFO  KVSUtils:210 - StartSelector set to NOW
2021-03-12 04:32:18 INFO  KVSUtils:218 - GetMedia called on stream kinesis-stream-test-connect-gsss-contact-d5ddaae5-9aa5-4288-acc4-2d1599582727 response 200 requestId e2422832-3ede-be7d-ba86-d920ae539d76
2021-03-12 04:32:18 INFO  KVSUtils:210 - StartSelector set to NOW
2021-03-12 04:32:19 INFO  KVSUtils:218 - GetMedia called on stream kinesis-stream-test-connect-gsss-contact-d5ddaae5-9aa5-4288-acc4-2d1599582727 response 200 requestId d4803654-9f4b-4559-8c44-c7470fca807f
2021-03-12 04:32:21 INFO  KVSTranscribeStreamingLambda:159 - Calling Transcribe service..
2021-03-12 04:32:24 INFO  StreamTranscriptionBehaviorImpl:53 - 1615523544849 Received Initial response from Transcribe. Request Id: 52db6586-dd55-4931-9039-ae0ffd3c5c90
2021-03-12 04:32:24 INFO  StreamTranscriptionBehaviorImpl:53 - 1615523544869 Received Initial response from Transcribe. Request Id: 8d05bbc8-31bd-4eba-8636-b9fc908b0215
2021-03-12 04:32:40 INFO  TranscribeStreamingRetryClient:214 - Reached on error but doing nothingsoftware.amazon.awssdk.services.transcribestreaming.model.BadRequestException: Your request timed out because no new audio was received for 15 seconds. (Service: transcribe, Status Code: 400, Request ID: 8d05bbc8-31bd-4eba-8636-b9fc908b0215, Extended Request ID: null)
2021-03-12 04:32:40 INFO  TranscribeStreamingRetryClient:214 - Reached on error but doing nothingjava.util.concurrent.CompletionException: software.amazon.awssdk.services.transcribestreaming.model.BadRequestException: Your request timed out because no new audio was received for 15 seconds. (Service: transcribe, Status Code: 400, Request ID: 8d05bbc8-31bd-4eba-8636-b9fc908b0215, Extended Request ID: null)
2021-03-12 04:32:40 DEBUG TranscribeStreamingRetryClient:164 - Error occurred on channel AUDIO_FROM_CUSTOMER : software.amazon.awssdk.services.transcribestreaming.model.BadRequestException: Your request timed out because no new audio was received for 15 seconds. (Service: transcribe, Status Code: 400, Request ID: 8d05bbc8-31bd-4eba-8636-b9fc908b0215, Extended Request ID: null)
java.util.concurrent.CompletionException: software.amazon.awssdk.services.transcribestreaming.model.BadRequestException: Your request timed out because no new audio was received for 15 seconds. (Service: transcribe, Status Code: 400, Request ID: 8d05bbc8-31bd-4eba-8636-b9fc908b0215, Extended Request ID: null)
at software.amazon.awssdk.utils.CompletableFutureUtils.errorAsCompletionException(CompletableFutureUtils.java:60)
at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncExecutionFailureExceptionReportingStage.lambda$execute$0(AsyncExecutionFailureExceptionReportingStage.java:51)

I thought main problem is: Your request timed out because no new audio was received for 15 seconds.
But I certainly speak on the phone. The call test environment for korean was same with english.

I already know some preconditions are necessary for using added supported languages such as korean, japanese. I took those actions.

1. Set languageCode to korean(ko-KR) in Contact Flows
2. Set 'Start Media Streaming' block by 'from the customer' (can select both of options - from & to)
3. Change kvs_Trigger Function for LanguageCode Attributes
4. Update the newest version of transcribeStreaming sdk in build.gradle

AWS Support Center said "Some changes to the Transcribe SDK in the newer version that supports Korean is likely what is causing the issue". And I agree it.
So I want to track the change against this issue. And I want to know what is the origin problem that causes this issue?

Thank you. I'll be waiting the response.

Hi,

After deploying this solution I needed to update website files changing CCP domain from .awsapps to .my.connect.aws.

It seems that AWS has changed Connect domain and I think this project should be updated by changing CCPURL and LoginURL variables.

Thank you.
Regards.

Stack fails to delete due to a S3 bucket not being empty.

Replication:
Delete deployed stack.

Proposed resolution:
Consider adding code below to the createWebSiteS3Bucket resource:

createWebSiteS3Bucket:
    Type: 'AWS::S3::Bucket'
    DeletionPolicy: Retain
    UpdateReplacePolicy: Retain
    {...}

Hi,

I tried to used Chat with Translate but when I click on Send Message button it throws following error in AWS SDK js:

Uncaught TypeError: agentChatSession.sendMessage is not a function

It seems the issue is with import conflicts of ChatJS and AWS SDK, as ChatJS comes with baked in AWS SDK version.

Can you please help?

Thank you.

The solution currently provides the real time transcript of the customer only. How will I be able to get the real-time transcript of what the agent is saying? @aman-appcino @efh365

sample UI console error code :
Uncaught Error: Failed to extract MediaRtpStats from RTCStatsReport for stream type audio_input

my region : ap-northeast-1(dokyo)

sample UI shows this status:
Name | Value
kvsTriggerLambdaResult | Success
languageCode | en-US
saveCallRecording | true
startStreamingAudioStatus | Success
transcribeCall | true

Amazon Connect "Live Media Streaming" Enable Check Complete!!
Please solve the error code.

このサンプルで同時通話を実行すると、文字起こしが中断します。
おそらくスレッドでタスクを処理しきれなくなって終わってしまっているように見えます。
単にこのサンプルとして使うのであれば問題ないですが、実際のコールセンターでは使えません。

java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@30f751d0[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@166249ce[Wrapped task = com.amazonaws.transcribestreaming.KVSByteToAudioEventSubscription$$Lambda$440/0x000000010054e440@60ed4494]] rejected from java.util.concurrent.ThreadPoolExecutor@2ace3efd[Shutting down, pool size = 1, active threads = 1, queued tasks = 97, completed tasks = 0]

cd $deployment_dir/manifest-generator
npm install
node app.js

This is failing as there is no manifest-generator directory available and no app.js is available. is there any manually needs to be done to make it working?

I can't find js/aws-credentials.js. Where can I get this file?