ARTIFICIAL INTELLIGENCE DEVICE FOR MULTI-MODAL RECOMMENDER AND CONTROL METHOD THEREOF

Smart overview of the Invention

An artificial intelligence (AI) device is proposed that enhances the recommendation process by integrating multiple modalities. It utilizes three distinct recommenders: behavior-based, review-based, and image-based. The device assesses the nature of a user's request—considering its modality, format, and content—to select the most appropriate recommender for generating personalized recommendations.

Functionality of Recommenders

The behavior-based recommender analyzes user preferences through a knowledge graph to predict potential likes for various recipes. It addresses challenges such as zero-shot inference, allowing it to make recommendations even for new users without prior data. The review-based recommender matches user queries with existing reviews to provide relevant suggestions, employing a hybrid approach that combines natural language processing (NLP) and knowledge graph embeddings for improved accuracy.

Image-Based Recommendations

The image-based recommender focuses on visual inputs by extracting key features from images. This recommender finds similar recipes or items based on latent representations of input images, guided by knowledge graph embeddings to enhance relevance in the results provided. This multimodal approach ensures that users receive tailored recommendations based not only on text but also on visual cues.

Recommendation Process

The AI device processes incoming requests by first identifying the type of recommender needed. After determining the appropriate modality, it transmits the request to the selected recommender. The system then generates a ranked list of recommendations based on knowledge graph embeddings and outputs these results to the user, ensuring a seamless experience across different types of queries.

Addressing User Needs

This multi-modal framework aims to improve user convenience and recommendation accuracy by addressing common issues faced in traditional systems, such as limited modalities and cold start problems. By integrating various recommendation strategies into a single device, it provides a comprehensive solution capable of catering to diverse user preferences across multiple categories, including food recipes, books, and more.