Write the introduction to our project.

• Introduction state
• Greeting state
• Acknowledge emotion state
• Searching state
• Farewell state
• (OpenDialog Generator)

Write a block post about our recorded gestures and show some examples. Title? Maybe "Botender learned to express emotions!"

Create component that provides the WebCam as GUI

Our gestures are terrible!

• Look at all the recorded gestures and discard all terrible ones
• Re-record gestures that are missing

Decide who writes which part of the report

There are still some todos left for the interaction system:

• @sanderjk5, @Grutschus: IntroductionState: Refactor the OpenAI implementation
• @sanderjk5, @Grutschus: Reactivate AcknowledgeEmotion State
• @sanderjk5, @Grutschus: Make the farewell messages more diverse (similar to greeting)
• @sanderjk5, @Grutschus: Implement the search state properly
• @ElectricUnit, @i2aoul : Add a state to ask for the general drink preferences of the user (type of drink and alcoholic/non-alcoholic)
• @ElectricUnit, @i2aoul: Add a feedback loop (maybe with OpenAI) for the drink preference state -> Maybe the user asks a question?
• @ElectricUnit, @i2aoul: Add a feedback loop to the drink recommendation state (maybe with OpenAI) -> Maybe the user is not happy with the first suggestion
• Everyone: Add appropriate gestures to ALL states
• Optional: Add an OpenDialog state for conversation with OpenAI
• Optional: Build an abstraction layer for OpenAI functions (e.g. to generate nicer responses in every interaction state)

Add gestures for the following occasions:

• concern
• happy
• idle
• laugh
• listening
• thinking
• understand issue

We need to record the final video of Botender, once everything is done.

Create Botender Framework structure
Define Interfaces

Create the user perception framework structure

• Create the architecture of the subsystem
• #31
• #32

Meeting Minutes from 24.11.2023

Attendees:

• Supervisor: Alessio
• Raoul
• Tea
• Lasse
• Till
• Jurek

Agenda

• Discuss the project specification
• Questions:
  • What does it mean that we have to extract our own features?

Notes

• We need to define the emotions we would like to detect before we train the detector. Avoid deciding on which emotions we'd like to detect based on the performance of the classifier.
• Justify why we chose these emotions
• More than anything, the grading takes the effort into account. We can achieve a 5 with only 4 emotions, as long as the final project is mature, well-documented, we have a nice blog, we really describe our effort in the final report, we justify our choices, etc.
• In the second feedback session we should have a working prototype - "Most of the systems should be in place"
  • Something to show for the first, something for the second
• The OpenAI integration could be something that actually increases our grades. However, we still need to have a rule-based system. We should maintain a version that runs without OpenAI.
• We could come with a different strategy and get creative for recommending drinks - everything we do on top of emotions can increase the grade
• "Extract own features": We just need to have some classifier of our own.

Follow-up / Summary / TODOs

I know this is purely cosmetic, but if someone has the time - why not :)

There are couple of tutorials on how to turn github pages into an actual blog:

• like this one
• or this one

Create the project specification according to the requirements posted on studium.

This is our work document: Overleaf

• Map coordinates of pyfeat and furhat
• Define grid
• Control furhats face movements
• Explore z coordinate
• Implement idle mode

• Analyse dataset
• Clean the dataset
• Feature extraction, standardization, ...
• Update / Increase grid search
• Evaluate results
• Use Pyfeat instead of own model?

Our code doesn't run on any of our devices properly.
Here are the things we should do:

• Convert emotion detection into a callable, so that we only detect emotions when necessary
• Alternatively try to only predict the emotion once every second
• Change the process communication:
  • First try to use a multiprocessing Manager to manage the queue
  • If that does not work try to use a shared and locked memory location

Create Interaction Component Framework
Set interfaces according to Botender definition

• Create the architecture of the subsystem
• Enable trigger of emotion detection

• What to say
• What to show
• Lengths of each section

Create a demo video of the working project.
Record a complete run through of the project.

1. Start the interaction
2. Talk to the bartender
3. Bartender recommends a drink
4. Customer accepts or rejects drink
5. Customer initiates small talk about (tbd)

After #58, we realize that the pyfeat emotion detector is superior to our classifier.

In this issue, we adapt Botender to use the pyfeat emotion detector.

• Switch subsystem 1 to use pyfeat
• Update Blog entry

Create coordinator to organize the current state of the conversation and activate the respective generator.

Maximum 5 pages - PDF

Final Report Structure

 Project title
  Authors

• Abstract (a one-paragraph summary of the project and the main results)

• Introduction (what objectives did you choose in the project proposal and why? What did each group member contribute?)

• #18

    Overall system design (what scenario did you choose? What were the high-level goals? What specific choices did you make to achieve this?)
    User Perception sub-system 
        Design (high-level overview of the system; what does it do?)
        Implementation (how did you achieve the intended design?)
        Results (analyse how well the subsystem works)
    Interaction sub-system
        Design (high-level overview of the system; what does it do?)
        Implementation (how did you achieve the intended design?)
        Results (analyse how well the subsystem works)
  

• General Discussion

      Discussion of the overall pipeline
      Challenges faced (what has worked and what has not)
      Use of ChatGPT or similar tools (if you use ChatGPT or similar tools, include here a description of what you have used it for, how, and how it benefited you)
      Ethical issues
  

• Conclusion

• References (it does not count towards the 5-page limit)

• Appendix (it does not count towards the 5-page limit; it must include your original project’s specifications/proposal; may include additional figures)

Write a Blogpost about our connection to the OpenAI Api, blocked by #63