Looking at the gestures.json file I see entries like this:

{
  "aggressive": {
    "queueOptions": [
      {
        "name": "in",
        "from": 0,
        "to": 43,
        "loopCount": 1
      },
      {
        "name": "hold",
        "from": 43,
        "to": 90
      },
      {
        "name": "out",
        "from": 90,
        "to": 158,
        "loopCount": 1
      }
    ]
  }

Currently I have only been using the gestures using the top level property name (e.g. - "aggressive"):

host.GestureFeature.playGesture('Gesture', 'aggressive');

Given the JSON block above, am I underutlizing the gestures library? Can I pass a dotted string and get access to what I'm calling the "sub-gestures" too (e.g. - "in", "hold", "out")? For example:

host.GestureFeature.playGesture('Gesture', 'aggressive.hold');

If not, then what is the purpose of the child elements like "in", "hold", "out"? If I can, then is my syntax correct and do the emotes similarly have sub-emotes?

@roschler We have the same issue raised by a customer - importing the GLTF into Blender and making the edits and re-exporting is generating a distorted mesh?

@c-morten Thanks. Side questions, is there a quick way to get rid of all the "stray hair" on the Cristine model? Hopefully some way that someone who is not an animator (massive understatement in my case) could do? I use the models frequently with their head position in profile and those hair elements look really bad from that view on that model.

Originally posted by @roschler in #19 (comment)

Is there an event I can hang a listener on or watch for in an interval timer that will tell me when a host has completed a gesture or an emote?

The sound is very weak and muffled.

How do I make it loud and clear?
Compare to generated wav files it pales in comparison.

what is the best way to have those hosts walk from pointA to pointB, is there any motion animation provided here or in any other repo?

if not, what is the best recommended way to achieve this?

I have noticed that if I try to play an animation with a host that is currently playing an animation, nothing happens. I can't play any new animations until the existing one stops. Is there a way to stop the existing animation so I can immediately play a new one?

Also, a giant thank you for creating this library. I have used it to create an automated, animated improv theater that uses OpenAI's GPT-3 natural language generation system to create animated scenes in real-time based on scene suggestions. If you want to see an example of this, check out this video:

https://www.youtube.com/watch?v=QL4QQ5NsPL8

Again, thank you for such a well designed and innovative software library!

I want to be able to halt any and all hosts that are currently talking immediately. What's the easiest way to do this?

Google Canary - Error: net::ERR_HTTP2_INADEQUATE_TRANSPORT_SECURITY

By following the Using the Dialogue Component and Amazon Lex to Build a Chatbot tutorial I made a new chatbot scene.
I use Windows 10 Pro (version 2004)
Using Google Chrome (Version 90.0.4430.93 (Official Build) (64-bit)) it's all good, working OK.

Using Google Chrome Canary (Version 92.0.4495.0 (Official Build) canary (64-bit)) I am experiencing errors.
This is important to fix since Canary code will make its way to standard Crome soon!

aws-sdk-2.657.0.min.js:71 POST https://runtime.lex.us-east-1.amazonaws.com/bot/BookTrip/alias/Test/user/us-east-1%3A350bd97c-3c63-47ad-8b99-eeceffda7c83/content net::ERR_HTTP2_INADEQUATE_TRANSPORT_SECURITY

The three.html sample shows how to make one host look at another:

host2.PointOfInterestFeature.setTarget(lookTracker1);

Can I make the host look at any other mesh object on a ThreeJS or perhaps in some arbitrary 3D vector, instead of at another host's look tracker location? If so, is there a function call for this, hopefully one that handles that handles any necessary world coordinate translations, and one that also provides tweening so you get the same smooth look-at operation you get with the PointOfInterestFeature.setTarget() call?

I am using the following text on the Sumerian test page:

<speak>This is a bad day at the shoe <mark name='{"feature":"PointOfInterestFeature","method":"setTargetByName","args":["camera"]}' /> counting department.</speak>

When I click the Play button, Luke looks off into space several clicks to the right of the camera, not at the camera itself. However, if I use this Javascript code:

host.PointOfInterestFeature.setTarget('camera');

It works fine. What is wrong with my embedded mark command?

I also tried same command with Luke to get him to look at the Alien character using :

<speak>This is a bad day at the shoe <mark name='{"feature":"PointOfInterestFeature","method":"setTargetByName","args":["Alien"]}' /> counting department.</speak>

But again, he stares off into space at the same place as the command with the "camera" as the target. What is wrong with my embedded PointOfInterestFeature command targets? What is the correct syntax to look at the "camera" or one of the other hosts?

@c-morten I need to swap voices for a host in real-time. Looking at the code, the voice to host assignment happens during the createHost() call:

		// Set up text to speech
		const audioListener = new THREE.AudioListener();
		camera.add(audioListener);
		host.addFeature(HOST.aws.TextToSpeechFeature, false, {
			listener: audioListener,
			attachTo: audioAttachJoint,
			voice,
			engine,
		});

Is there a way to change the voice on-the-fly, after the scene initialization and creation has occurred?

like title.
if i want continue play gesture, must refresh page.
can someone help me fix it?

I used the repo sample to build my Sumerian Hosts app. It's obvious as I watch the hosts in my app that they already have gesture maps attached to them. How do I remove those gesture maps or eliminate them? I have my own method for handling gestures and I don't want the existing ones to interfere.

@c-morten

In another issue you made this statement:

#19

"Another alternative would be to create your own message for the name of the mark tag and add a listener for that. Then you could use whatever logic you'd like to figure out which host you want to target inside your listener function."

Can you give me a code snippet that shows me how to add a listener for a custom mark tag name and how from within that function I would target a particular host's charGaze location?

When loading the app from a static site generator(Github Pages) I get the warning message
The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page. https://goo.gl/7K7WLu on line three.min.js:formatted:19644 and
The audio context is not running. Speech will not be able to be played until it is resumed. Use the "TextToSpeechFeature.resumeAudio" method to try to resume it after a user gesture. on line TextToSpeechFeature.js:26

I have install all modules and demo seems to start ok locally, but when I click on any of the demo links.. it shows Loading ... and doesn't go past that point. am I missing something? I installed all the modules.

bash-3.2$ npm run start-babylon

amazon-sumerian-hosts@ start-babylon /Users/xxx/workspace/amazon-sumerian-hosts
cross-env ENGINE=babylon NODE_ENV=development webpack-dev-server

[webpack-dev-server] Project is running at:
[webpack-dev-server] Loopback: http://localhost:8080/
[webpack-dev-server] On Your Network (IPv4): http://xxx.xxx:8080/
[webpack-dev-server] On Your Network (IPv6): http://xxx]:8080/
[webpack-dev-server] Content not from webpack is served from '/Users/xxx/workspace/amazon-sumerian-hosts' directory
[webpack-dev-middleware] wait until bundle finished: /packages/demos-babylon/src/
[webpack-dev-middleware] wait until bundle finished: /packages/amazon-sumerian-hosts-babylon/test/integration_test/Babylon.js/
assets by path ./packages/demos-babylon/dist/*.js 26.3 MiB
asset ./packages/demos-babylon/dist/customCharacterDemo.js 8.31 MiB [emitted] [minimized] (name: customCharacterDemo) 2 related assets
asset ./packages/demos-babylon/dist/chatbotDemo.js 5.99 MiB [emitted] [minimized] (name: chatbotDemo) 2 related assets
asset ./packages/demos-babylon/dist/gesturesDemo.js 5.99 MiB [emitted] [minimized] (name: gesturesDemo) 2 related assets
asset ./packages/demos-babylon/dist/helloWorldDemo.js 5.99 MiB [emitted] [minimized] (name: helloWorldDemo) 2 related assets
assets by path ./packages/amazon-sumerian-hosts-babylon/ 22 MiB
asset ./packages/amazon-sumerian-hosts-babylon/test/integration_test/Babylon.js/dist/animationTest.js 8.36 MiB [emitted] [minimized] (name: animationTest) 2 related assets
asset ./packages/amazon-sumerian-hosts-babylon/test/integration_test/Babylon.js/dist/textToSpeechTest.js 8.36 MiB [emitted] [minimized] (name: textToSpeechTest) 2 related assets
asset ./packages/amazon-sumerian-hosts-babylon/dist/host.babylon.js 5.27 MiB [emitted] [minimized] (name: host.babylon) 2 related assets
asset ./packages/amazon-sumerian-hosts-three/dist/host.three.js 249 KiB [emitted] [minimized] (name: host.three) 2 related assets
asset ./packages/amazon-sumerian-hosts-core/dist/host.core.js 240 KiB [emitted] [minimized] (name: host.core) 2 related assets
runtime modules 248 KiB 115 modules
javascript modules 12.5 MiB 1522 modules
json modules 2.81 MiB
./node_modules/aws-sdk/apis/metadata.json 15.9 KiB [built] [code generated]
./node_modules/aws-sdk/apis/sts-2011-06-15.min.json 3.71 KiB [built] [code generated]
./node_modules/aws-sdk/apis/sts-2011-06-15.paginators.json 27 bytes [built] [code generated]
./node_modules/aws-sdk/apis/cognito-identity-2014-06-30.min.json 7.4 KiB [built] [code generated]
./node_modules/aws-sdk/apis/cognito-identity-2014-06-30.paginators.json 155 bytes [built] [code generated]
./node_modules/aws-sdk/apis/acm-2015-12-08.min.json 5.13 KiB [built] [code generated]
./node_modules/aws-sdk/apis/acm-2015-12-08.paginators.json 161 bytes [built] [code generated]
./node_modules/aws-sdk/apis/acm-2015-12-08.waiters2.json 569 bytes [built] [code generated]
./node_modules/aws-sdk/apis/apigateway-2015-07-09.min.json 54.1 KiB [built] [code generated]
./node_modules/aws-sdk/apis/apigateway-2015-07-09.paginators.json 1.37 KiB [built] [code generated]
./node_modules/aws-sdk/apis/application-autoscaling-2016-02-06.min.json 6.63 KiB [built] [code generated]

Description

A typo error in the README.md is making the code appearing as a simple text

Version used

The typo error was seen in the mainline branch dans in the release v1.3.3

Section

In the section Getting started in three.js --> Step 10. Initializing the Hosts, in the idle Base animation description

Existing

Step 10. Initializing the Hosts

...
Next we add the AnimationFeature. This is what allows you to manage and play animations on the host character asset. The AnimationFeature is a layered animation system, below we'll take a look at each animation layer and animations we'll add to make the host appear to speak, gesture and look at the camera.

• // Base idle

host.AnimationFeature.addLayer('Base'); host.AnimationFeature.addAnimation('Base', idleClip.name, HOST.anim.AnimationTypes.single, {clip: idleClip}); host.AnimationFeature.playAnimation('Base', idleClip.name);

Here we have added a single [AnimationLayer](https://aws-samples.github.io/amazon-sumerian

...

Correction

Next we add the AnimationFeature. This is what allows you to manage and play animations on the host character asset. The AnimationFeature is a layered animation system, below we'll take a look at each animation layer and animations we'll add to make the host appear to speak, gesture and look at the camera.

• // Base idle
  host.AnimationFeature.addLayer('Base');
  host.AnimationFeature.addAnimation(
    'Base',
    idleClip.name,
    HOST.anim.AnimationTypes.single,
    {clip: idleClip}
  );
  host.AnimationFeature.playAnimation('Base', idleClip.name);

  Here we have added a single AnimationLayer to the host. Animation layers are containers that manage playback of a subset of animations. Each AnimationLayer can only play a single animation at any given time, but a host animation can also be a container for multiple animation assets. Here we are creating an animation of type HOST.anim.AnimationTypes.single, which means that the animation can only contain one Babylon.js AnimationGroup. The clip we've chosen is the stand_idle animation, which contains looping ambient motion in a standing position. This is the animation you'll see by default when the host isn't speaking. host.AnimationFeature.playAnimation is telling the host to start playing the idle animation on the Base layer we just created.

Pull request

#36

AWS-Infrastructure-Setup.md does not match the latest AWS Cognito work flow. Please provide documentation that is up-to-date and works with the amazon-sumerian-hosts-three demo, esp. for the mainline2.0 branch.

I am attempting to understand this code better. I really appreciate the effort to make an example project, but its convoluted and the code is too terse for an reference/example project. Someone really likes destructuring assignments. I think it would have been better to sacrifice the latest and greatest language features and focus more on readability.

I am currently stuck on the block below. I see these objects instantiated, but I don't see where the values of each array are populated. Look at the first item below, for example, "idleClips1". If I inspect the object after instantiating here, it contains the stand_idle_clip, which is great, BUT, how and where was it populated? Any insight will be appreciated.

        const [
          idleClips1,
          lipsyncClips1,
          gestureClips1,
          emoteClips1,
          faceClips1,
          blinkClips1,
          poiClips1,
        ] = clips1;

Hi, is it possible to use already synthesized audio streams instead of text in order to animate visemes/fonemes?
Or can we control viseme/foneme animations?
We connect to Polly to sythesize speech from our server and provide the frontend with ready audio streams.

Where can I find the complete list of embedded SSML feature strings? For example, like this one used to adjust the look-at target during speech:

mark(name='{"feature":"PointOfInterestFeature","method":"setTargetByName","args":["chargaze"]}')

I changed the three.html sample from using the Luke character over to Cristine. That was the only change I made:

        // ROS: const characterFile1 = './assets/glTF/characters/adult_male/luke/luke.gltf';
        const characterFile1 =
          './assets/glTF/characters/adult_female/cristine/cristine.gltf';

For some odd reason, instead of appearing where Luke does on the canvas, she appears way off the screen in the far upper left corner of the coordinate space, far outside the visible viewport. Looking at the code, I see this initialization Javascript:

        character1.position.set(1.25, 0, 0);
        character1.rotateY(-0.5);

Why does that code work properly for Luke but not Cristine? I have no experience with glTF related files, so is there something in those files that could affect the way the position of one character would be effectively rendered compared to another?

Everything else works fine, and the Alien character even looks off into space at her current location when she's talking. But as I said "she's out in the boonies". I have to use the mouse wheel quite a bit to zoom the camera outwards to bring her into view.

Also, is there a Slack, forum, or IRC chat room for asking questions about this project?

I run an educational institution and these characters would be a great tool for our Language Learning Centers and Online Classes.

Best regards

I have noticed in my scene that after about 5 to 10 minutes of generating audio, the audio gets "glitchy". In other words, after having my Sumerian Host characters (3 of them) talk for about 5 to 10 minutes straight, the audio starts to develop really bad ticks and pops in it, to the point it is unlistenable (think of a really old, scratchy vinyl record being played).. This usually ends up being an audio buffer handling problem somewhere. It happens every time and the symptoms are always the same. A few minutes of perfect audio, then a single tick or a pop here or there, and then once the ticks and pops start happening, it exponentially increases rapidly to the unlistenable stage to where it happens with every audio buffer being played.

I know this might be a Chrome thing, but just in case, is there a way to "patch" into the audio generation code of the Sumerian host library and play the audio with something else, in my case Howler.js, while maintaining the lip sync between the audio and the character/host animation?

Typescript & aws-sdk-js-v3

Can we expect a Typescript variant of this repo with the AWS SDK for JavaScript Version 3.x in the near future?

Hi folks
I uploaded luke.gltf into mixamo and assigned walking animation and exported it into fbx. And then I opened the fbx in blender and exported it into glb file. But I'm getting this issue

Please give me an advice
Regards

Hello,

I am pretty new with Sumerian and I cloned this repo in my personal mac and run the Demo hosts for Lex chatbot, but when I ask the host something nothing happen. I have few questions:

1 - Can I run this in a EC2 instance with a proper domain rather than my locahost?
2 - How can I interact with the Lex Bot?
3 - Where is the Sumerian dashboard in aws console? I tried several times but the only thing I see in any region is just 3 documentations links.
Any help please?
Tony

For both male and female genders, this repo contains 4 personas with models, animations and textures.

It also contains, for both genders, a subfolder called 'alt_outfits' with hoodie/polo/tshirt models and textures.

How do we use those though? if i open the cristina model, it contains clothing and head etc but if i open the alternative hoodie model, it just contain the clothing. What are we supposed to merge both to dress her with that alternative outfit?

Thanks in advance.

With respect to the following:

"You can clone the repository to obtain glTF character animation assets tailored to work well with the Host API. You can use them as-is, modify them in DCC software such as Blender or use them as a guide to develop your own 3D assets."

I am attempting to create custom materials and textures for the host model. I am able to import anyone of the models into Blender successfully (i.e. examples/assets/glTF/characters/adult_female/grace/grace.gltf) and can modify the materials in blender accordingly. The issue occurs when I export the model and attempt to use it in the ThreeJS example.

As you can see below, the grace.gltf model is completely distorted. This is what happens when I do not make any changes to the model, and simply import and export grace.gltf from blender. I listed some steps to reproduce below as well.

Blender 2.83.10 w/ Khronos glTF Blender I/O v1.2.75 -> This version was chosen because the .glTF models were created with v1.2.75

Result

Prerequisites
Download the amazon-sumerian-hosts project
Download & Install Blender

Steps to Reproduce:

1. Open Blender and Import examples/assets/glTF/characters/adult_female/grace/grace.gltf (any character model will do)
2. Export the file into a a new character folder i.e. examples/assets/glTF/characters/adult_female/grace_2/grace.gltf
   Make sure to use the .gltf Separate format in blender
3. Modify three.html so that the character from examples/assets/glTF/characters/adult_female/grace_2/grace.gltf is loaded.
4. Refresh the server and hard reload the page

I'm not sure if this is an issue with Blender or if this is an issue with how the .gltf files are parsed but I'm looking for some more information on what the workflow to customize hosts / create new ones are. Since it's mentioned that Blender can be used as a tool to do these customizations, I'm wondering what I'm missing here.

I want to display just the character, without any animations.
In babylon.html I commented out all the code responsible for loading the animations in the function createHost.
So basically just this line is left
const host = new HOST.HostObject({owner: character});
and the host is returned.
The character appears for a very short moment, probably during/after loadCharacter and then disappears.
Is it possible to show the character only?

Once any host completes it's speech the entire Babylon scene just quits rendering and hold a still image on the screen. It throws this error and then points to a variety of places in the different js files for the speech components:
Speech.js?2a41:128 Uncaught TypeError: Cannot set property currentTime of # which has only a getter

It does however keep playing speech when you click the start button, only the visuals are stopped from working. It started doing that around three weeks ago on my computer out of the blue. Essentially without changing a single thing in the code it just quit working overnight. Using an old version of the repo (the one where the poi feature was just released) does still work, but the newest version of it does not. I tried completely re-downloading and installing the repo several times, and get the same error each time. Any help is appreciated, thanks!

Hello all,

I am currently experimenting with this repo, the BabylonJS demos work really well and are very nice. Though, I was wondering how I would access these without the use of the web-dev-server. I haven't seen much documentation relating to this topic, if any documentation at all.

I have tried changing the npm build settings but nothing has been working. When I try to access any of the BabylonJS demos without going through port 8080, it will simply just not load. My ultimate goal is to get the demos working perfectly in production mode so I can run them on ports 80/443.

Any ideas? Any tips would be appreciated.

When I try to run the project I get the error

Uncaught TypeError: Failed to resolve module specifier "three". Relative references must start with either "/", "./", or "../".

Can you please help me with the point below? Someone asks me to demo host interaction with Lex and I can't do it with localhost I need my webserver.

1. Web applications that you build using the Sumerian Hosts NPM modules in this repository can be deployed to any regular web server, including a server running on EC2 or - even easier - AWS Amplify's drag-and-drop hosting solution. However, the demos we've included are not themselves directly deployable to a webserver. We realize this is inconvenient and is something we hope to change in the future. If you have an immediate need for deployable versions of the demos, let me know so I can look into a potential workaround for you. But if you're starting your own custom project from scratch using one of the guides we've provided in each module's README (like this README for the Babylon.js version) you should have no problem deploying your project to any web server.

Hello,

I need assistance on the following error.
I have installed all the dependencies & also setup the AWS credential.
Was facing some issues with proxy while installing npm install but got resolved after some workaround
Now when I try to run the npm run start-babylon I am getting an error as engine is not recognized by internal or external command as shown in figure.

We're using aws-amplify for authentication. There's a helpful API called Auth.currentCredentials() which will return valid credentials generated by the attached identity pool.

I have this effect sitting in my root component:

  /**
   * refresh the AWS credentials
   */
  useEffect( () => {
    const THIRTY_MINUTES_MS = 1800000;
    setInterval( async () => {
      const [creds] = await Promise.all([Auth.currentCredentials()]);
      const { accessKeyId, secretAccessKey, sessionToken } = creds;
      AWS.config.credentials = new AWS.Credentials({
        accessKeyId,
        secretAccessKey,
        sessionToken
      })
    }, THIRTY_MINUTES_MS);
  }, []);

Even with this code, it appears that credentials are expiring after an hour. How can I refresh the AWS credentials for this library?

Thanks.

There appears to be a memory leak somewhere in the new POI update. It usually takes around 10 to 20 minutes for it to build up to 5GB of memory used in chrome, upon which it closes itself and gives the chrome "out of memory" error screen. The only thing in the base code that I changed would be inputting my cognito ID. To double check that it wasn't something with the new default hosts, I swapped them out with Wes and Fiona and got the same results. Any help with finding where the leak is coming from is appreciated, thanks!

Since Babylon 4.1, the Offscreen Canvas API is supported to allow multithreading with Babylon programs.

https://doc.babylonjs.com/divingDeeper/scene/offscreenCanvas

This does not seem to be supported by this repository, as the Messenger class uses the window object for dispatching events. https://github.com/aws-samples/amazon-sumerian-hosts/blob/mainline/src/core/Messenger.js#L26

Is there a reason that this library needs to use the window for communication, as opposed to another pubsub implementation like the Node events library?

When trying to use a different language than what was initialized I’m getting the following error in the console:

sequential_executor.js:108 Uncaught (in promise) UnexpectedParameter: Unexpected key 'Language' found in params
    at ParamValidator.fail (http://localhost:3000/static/js/1.chunk.js:397338:37)
    at ParamValidator.validateStructure (http://localhost:3000/static/js/1.chunk.js:397365:14)
    at ParamValidator.validateMember (http://localhost:3000/static/js/1.chunk.js:397374:21)
    at ParamValidator.validate (http://localhost:3000/static/js/1.chunk.js:397322:10)
    at Request.VALIDATE_PARAMETERS (http://localhost:3000/static/js/1.chunk.js:395478:42)
    at Request.callListeners (http://localhost:3000/static/js/1.chunk.js:400999:20)
    at callNextListener (http://localhost:3000/static/js/1.chunk.js:400986:12)
    at http://localhost:3000/static/js/1.chunk.js:395431:9
    at finish (http://localhost:3000/static/js/1.chunk.js:391253:7)
    at http://localhost:3000/static/js/1.chunk.js:391271:9
    at CognitoIdentityCredentials.get (http://localhost:3000/static/js/1.chunk.js:391847:7)
    at getAsyncCredentials (http://localhost:3000/static/js/1.chunk.js:391265:24)
    at Config.getCredentials (http://localhost:3000/static/js/1.chunk.js:391287:9)
    at Request.VALIDATE_CREDENTIALS (http://localhost:3000/static/js/1.chunk.js:395423:26)
    at Request.callListeners (http://localhost:3000/static/js/1.chunk.js:400994:18)
    at Request.emit (http://localhost:3000/static/js/1.chunk.js:400966:10)
    at Request.emit (http://localhost:3000/static/js/1.chunk.js:399565:14)
    at Request.transition (http://localhost:3000/static/js/1.chunk.js:398898:10)
    at AcceptorStateMachine.runTo (http://localhost:3000/static/js/1.chunk.js:405036:12)
    at Request.runTo (http://localhost:3000/static/js/1.chunk.js:399274:15)
    at http://localhost:3000/static/js/1.chunk.js:399673:12
    at new Promise (<anonymous>)
    at Request.promise (http://localhost:3000/static/js/1.chunk.js:399661:12)
    at TextToSpeechFeature._synthesizeSpeechmarks (webpack-internal:///333:4916:71)
    at TextToSpeechFeature._updateSpeech (webpack-internal:///333:4829:42)
    at eval (webpack-internal:///333:4785:18)
    at Array.forEach (<anonymous>)
    at TextToSpeechFeature._updateConfig (webpack-internal:///333:4772:41)
    at TextToSpeechFeature._getSpeech (webpack-internal:///333:5019:21)
    at TextToSpeechFeature.play (webpack-internal:///333:5241:19)
    at eval (webpack-internal:///333:6067:140)

The way I’m doing it is by passing in options as follows:

  host.TextToSpeechFeature.play(text, {
      Engine: this.languageMap[this.language].engine,
      Language: this.languageMap[this.language].languageName,
      VoiceId: this.languageMap[this.language].voiceId,
}) 

It seems like it's a bug. Unable to switch languages after initialization without hardcoding the entire config object.

says it has to be defined in the webpack , but what if i dont want to manage webpack or try to implement the package with react or some other framework

.

Please describe the process you used. Esp. for generating the AnimationClips from the desired visemes.

After exporting the character from blender everything is perfect bar the camera clipping.

It seems to be activated a lot further out than it was pre-edit.

Do I need to change something in blender or is this code related?

safari/chrome on iPhone iOS 14.5.

the lips are moving but there is no sound.

(also, note the threejs version has to be specified in the import, for example below:)

<script src="https://cdnjs.cloudflare.com/ajax/libs/three.js/r125/three.js"></script>

Is it related to webAudio context? Any workaround? Thanks in advance.

If my application has a set list of hardcoded texts to be said by the virtual host, I shouldn’t need to call AWS Polly every time a new visitor comes on my webpage. I should just have the mp3/api-call-result files saved somewhere and link with those static files.

What is the easiest way to achieve this with this repo code? Many Thanks.

Hello,

many thanks for the publication under Open Source

In the test environment the example works wonderfully with three.js.

Unfortunately I have problems with the conversion to German.

.
.
.
const voice1 = 'Vicki'; // Polly voice. Full list of available voices at: https://docs.aws.amazon.com/polly/latest/dg/voicelist.html
const voiceEngine = ''; // Neural engine is not available for all voices in all regions: https://docs.aws.amazon.com/polly/latest/dg/NTTS-main.html

Unfortunately I have no more ideas how to switch the language output to German.

I am happy about tips and hints.

Ralf

I cloned this repo locally and then used a full text indexing package on its contents. I can't seem to find any samples that show how to use AWS scripting to do the following operations:

• Execute the Host component look-at feature (E.g.- look at camera)
• Execute the Host component gesture feature
• Execute the Host component emote feature

I need to be able to do these things from the new Preview Scripting environment, not the legacy scripting environment or from a IDE provided state machine. Where can I find sample code for these features?

With intentions to edit the avatar and animations I am trying to dissect how these character and animation files are made? Or how I can edit them?
As a test I imported the character into blender and exported straight away back into gltf/glb format.
Results:

• the .bin file completely different,
• the file size has increased from 50kb to 30mb
• the mesh is completely distorted after re-export (animations still work to an extent)
  Any help would be much appreciated thank you!

We are working on BabylonJS that integrates Sumerian Host. By calling Lex we received audio and message, how to sync Lex respond message with Sumerian host so that Host can play both audio and gesture. This could leave out the Polly and only need Lex. Is this possible?
Thanks.

The avatar is suddenly stops gestures, taking, camera following on babylon. Almost freezed. Maybe some new js update
OS:Windows 10
Browser: Chrome, Mozilla Firefox, Edge