We are pulling data from Snowflake and processing: summarize with AI and create image. Then storing it into Google Cloud storage. We expose as API /data it returns all emails: subject, body, sender, time, summary and image generated by ai. It is in Python.

Packages used:

• OpenAI API
• snowflake-converter
• csv

The DataTransfer class takes the following Snowflake credentials as input:

• snowflake_account
• snowflake_user
• snowflake_password
• snowflake_database
• snowflake_warehouse
• snowflake_schema
• snowflake_table

Using these credentials, the snowflake-converter API is leveraged to read data from the Snowflake database.

In a digital landscape saturated with content, the Multimodal Backend emerges as a pioneering solution, meticulously designed to refine how users discover and engage with YouTube videos that truly resonate with their interests. This platform distinguishes itself by utilizing scheduled content analyses, ensuring that users are consistently updated with relevant content without overwhelming them with real-time alerts.

•⁠ ⁠Scheduled Content Analysis: Central to the Multimodal Backend is its capability to perform in-depth analyses of video content at scheduled intervals. This approach ensures comprehensive coverage and relevance, allowing for the thoughtful curation of content that aligns with user interests.

•⁠ ⁠Customized Notifications: Leveraging advanced AI algorithms, the platform generates personalized notifications. These notifications are unique, comprising a custom image and a succinct summary that encapsulates the essence of the video's relevance to the user's interests.

•⁠ ⁠Seamless YouTube Integration: With a design that integrates effortlessly with YouTube, our backend system ensures timely notifications about new content from subscribed channels, specifically tailored to match user interests.

•⁠ ⁠Engaging and Informative Notifications: By synthesizing textual summaries with custom-generated images, we deliver notifications that are not only visually appealing but also rich in content. This multimodal notification strategy enhances the content discovery experience, making it both efficient and enjoyable.

 1.⁠ ⁠Interest Specification and Channel Subscription: Users specify their interests and subscribe to related YouTube channels via our platform.

 2.⁠ ⁠Scheduled Analysis: Our system conducts scheduled analyses of newly posted videos from these channels, identifying content segments that align with user interests.

 3.⁠ ⁠Content Summarization and Visual Creation: Key themes from these segments are extracted to craft a concise, engaging summary. Simultaneously, a visual representation of the video's theme is generated using a Dall-E prompt, creating a unique image for each notification.

 4.⁠ ⁠Personalized Notification Delivery: Users receive a personalized notification that includes both the custom image and the summary. This tailored approach acts as a personalized gateway to the content, ensuring relevance and enhancing user engagement.

The Multimodal Backend transcends traditional content discovery methods by adopting a scheduled analysis approach. This strategy not only respects the user's time and attention but also ensures that the content they receive is thoughtfully curated and highly relevant. Users are liberated from the chore of manually filtering through vast amounts of content, allowing them to enjoy a curated selection of videos that genuinely interest them.

The platform stands at the intersection of technology and user experience, providing a service that not only keeps users informed about their interests but also redefines the way they engage with digital content. By delivering personalized, multimodal notifications at scheduled intervals, the Multimodal Backend fosters a more meaningful connection between users and the content they love, setting a new standard for content discovery and engagement on YouTube.

NextJS app pulling from Chat, MultiModal and Data Backends.

• Collecting various data types(csv, json, jpeg, png, mp4, wav etc.) from multiple data data sources and storing to Google Cloud Storage.
• Running Airbyte cloud to extract and load it into Snowflake.