interaction-lab / nri-svte Goto Github PK

• create initial publisher
• test head pan publisher
• fix if needed

• Check if virt/physical switch exists
  • If not, create issue and switch
• Decouple the visualization from the subscriber for capacitive touch
• Create data input (i.e., colliders on head mesh) in same form as captouch visualizer

• Create a publisher for the animations
• Implement animations so kuri can react to different events (head pat, bumper)

For @ipekgoktan :

• list out 2 messages interested in making new visualizations
• draw up some concept ideas
• organize google docs for meeting
• set meeting agenda and time for this week

• add blink button
• add smile button
• brainstorm other user controlled animation ideas

• receive cap touch data
• cap touch visualizations
• receive bumper data
• bumper visualizations
• receive wheel drop data
• wheel drop visualizations

• display camera feed using ROS#
• Use ROS# Image subscriber
• find out what image type kuri outputs
• transfer kuri camera data to ROS# ->UNITY

add audio manager which will play sounds when the text manager is updated.

• create a physical controller object with MRTK buttons
• allow user to pick up controller and move around with it
• Snap photo button to controller when it is time for photo gallery activity
• Snap visualization button to controller when it is time to turn them on and off

• Use a script to feed data to move Arrow (Kuri)
  • Move Arrow (Kuri) forward and backward using arrow keys
  • Turn Arrow (Kuri) left and right using arrow keys
• Have camera follow Arrow (Kuri) as it moves - Display 2
• Print out bumping data
  • When Arrow (Kuri) bumps into an object, print out what object it has collided into
  • Print out which sensor is bumped into
  • When Arrow (Kuri) bumps into an object, print out array of values from the colliders
• Pipe these values into the visualization
  • Only when the sensor bumps into object does the sensor change from grey (off) to a green color (on)

• choose a style guide for the Unity Project
• add an auto formatter to the repo

• make camera sound when kuri takes a picture
• have kuri react to getting pet
• react to bumper
• react to getting picked up
• react to detecting a user
• expression when color display is changed
• test all animations

This needs to be done by this Wednesday for the AmazonStudy branch

• RosSharp must work
• UI will need rescaling for sure
• Speed up image parsing

• Fix ROS# issues
• Fix rosbridge
• Send Face Detection data to Unity
• Scale the coordinates to the panel
• Mark faces on panel with 3D sphere

I think we should be able to make this public, does anyone see an issue with this? It also allow us to link the progress to the grant website/report.

• Create camera object
• Place camera object inside Kuri object
• Create panel
• Display the camera view on the panel using a render texture

also need to decouple the visualizer out maybe idk

will need to add amy to this

Currently using a plane as the Kuri Console which will display text, we should brainstorm a more permanent way to do this

• Add the correct message type to change kuri's chest lights
• write a LED publisher
• Implement MRTK Slider for color selection

• scale 3D marker when face moves forwards and backwards
• Change sphere to a more traditional shape (maybe a white square)
• make the scaling smoother

Making a ticket to keep track, kuri audio will randomly blow out, not sure why, when running animations.

Current fix is just to restart kuri to get sound back

Rolling tutorial

• create new scene
• complete roll-a-ball tutorial
• up and down arrow for Kuri to be picked up
• create plane to represent the ground
• print to Debug.Log() when kuri is picked up

MRTK issues prevent project from functioning correctly

refer to this link: https://docs.microsoft.com/en-us/windows/mixed-reality/design/surface-magnetism

make the pictures in the gallery stick to surfaces for a more immersive gallery experience

• Create button and add to the controller
  • Clicking button with virtual hand takes a screenshot of the current video feed
  • Creates a new 3d cube that stores that image
  • Each image must be of that current moment
  • Move picture objects close to user

merge branches to clean up repository

• merge photo gallery to main scene
• merge demo and amazon scenes
• resolve merge conflicts
• merge amazon with main
• resolve merge conflicts

First pull request to test github

• find which message type to subscribe to
• create unity subscriber for face detection
• update ROS# to incluse visual_msgs message types

• allow users to move the video display
• allow users to move kuri's console

• add audio python script to Kuri
• add audio message types to ros#
• create audio publisher
• have kuri play different sounds based on different interactions (ex: click sound when it takes a photo)

Extension of #31

• allow for infinite number of Meshrenders (maybe we can cap it but it will be a List instead of array_
• have way to delete images such as trashcan or x
• add in bounding boxes (maybe this is just within delete)

• learn how to store and display a single frame from kuri's vision_msgs
• create image displays when the user clicks on the controller's photo button
• allow users to place images to their liking so they can create their own gallery
• use an MRTK bounding box / bounds control to indicate that the images can be moved

• translucent kuri -> opaque
• public objetcs == null reference (show lasers)
• rename scenes
• navigation scene
• Cinemachine the scene
• remove touch sensors?
• check on namespaces?

• head
• eye
• maybe light?

other states like pat and bump should just be already added in

Create a PR template that shows up automatically when a PR is being made

• Created a docs folder
• Created the pull_request_template.md file that contains the template

• has all visualizations enabled
• just-in-time instructions from kuri text manager
• new color scheme

link is on the slack nri-svte chat microsoft/MixedRealityToolkit-Unity#6947

• fix msbuild issue using tutorial
• you need to find a manifest.json and delete a line
• make sure it works
• push the changes up

Need to put namespace defs around classes, e.g.,

namespace KuriSVTE{
  class VisualizerClass : Monobehavior{
  }
}