Grading rules 2024/2025

Assessment

The assessment for this course consists of two main categories of activities:

Qualification for the Final Exam: to be eligible to take the final exam, students are required to attain a minimum total score of 35 (Maximum 70) points across both the tutorial homework and the semestral project components.

Exam

As part of the assessment, there will be an oral exam with the following parameters:

Points breakdown:

Exam Topics (2023):

Evaluation scale

Your grade is determined by sum of points from tutorials and exam (at most 100 points).

Evaluation scale is according to The Study and Examination Code for Students of CTU in Prague.

Semestral Project

In the semestral project, your task involves conducting scientific/applied research to enhance existing PML methods, culminating in the creation of a concise scientific paper. The primary focus lies within the domain of Recommender Systems. However, potential applications of PML extend to various other fields. This paper aims to build upon the latest developments presented at the following conferences: RecSys, KDD, or WWW in the years 2022, 2023, or 2024.

Instead of a report, the scientific paper must follow the Association for Computing Machinery (ACM) template. The paper should range from 4 to 8 pages in length, excluding references. It is expected to extend and improve existing PML methods. This could involve enhancing the accuracy, efficiency, or applicability of the method. The extension should demonstrate originality and innovation.

A basic paper Structure:

Evaluation Criteria:

Students will be assessed based on the quality of their scientific paper, the novelty and significance of their research extension, and their ability to communicate their findings effectively.

Important Dates:

Note: Poster presentation + Paper submission will be evalutated in the maximum of 50 points. Lecture topics and guidance on the project will be provided throughout the semester to support students in their research and paper writing.

Examples of Semestral Project’s topic (2024)

Topic 1 - Enhancing Information Retrieval with Semantic Search: the project aim the development of a search system that understands the meaning behind user queries. By leveraging semantic search techniques and advanced language models, the goal is to retrieve contextually relevant results rather than just keyword matches, thereby improving the efficiency and accuracy of information retrieval. Example of query that needs semantic search: "a gift for my 10 yr old son".

Topic 2 - Cross-Selling Recommendations Using Language Models to Detect Correlated Items: This project involves creating a recommendation engine that utilizes large language models (LLMs) to identify and suggest items that correlate with each other. By detecting patterns and associations between products, the project aim to develop a system that enhances cross-selling opportunities, recommending complementary items to users based on their interests and purchasing behavior. Example: finding items that could be bought together by analyzing interactions and item descriptions.

Topic 3 - Personalized Clustering Recommendations Based on User Preferences: the project aiming to designing a recommendation system that clusters content according to specific user behaviors and preferences. The dynamic labeling of the clusters can be done using large language models (LLMs).

Topic 4 - Personalized Summarization of Reviews: This project aims to develop a tool that generates personalized summaries of product or service reviews. By applying natural language processing techniques, students will create a system that condenses extensive reviews into concise, relevant summaries tailored to the user’s specific interests or concerns, facilitating more informed decision-making.

Topic 5 - Explaining Recommendations with Language Models: In this project, the target will explore how to use large language models to provide clear and understandable explanations for system-generated recommendations. By generating natural language explanations, the goal is to increase transparency and user trust in recommendation systems, helping users comprehend why certain items are suggested to them based on their behavior and preferences.

Topic 6 - Personalized Assessment Systems for Education: this project aim to design a machine learning-based personalized assessment system that dynamically adjusts the evaluation based on a learner’s past performance and learning style, providing tailored feedback and progress tracking to enhance individual learning outcomes.

Topic 7 -Team Performance Prediction using PML: this project target it to develop a machine learning model to predict the outcome of sports events by analyzing historical data of two competing teams. The system will use features such as player statistics, team performance trends, and game conditions to forecast the likelihood of victory for a specific team in upcoming matches.

Topic 8: You could apply PML methods in areas less explored than recommendation systems.

Examples of Semestral Project’s topic (2023)

Topic 1: Sheshera Mysore, Andrew Mccallum, and Hamed Zamani. 2023. Large Language Model Augmented Narrative Driven Recommendations. In Proceedings of the 17th ACM Conference on Recommender Systems (RecSys '23). Association for Computing Machinery, New York, NY, USA, 777–783. https://doi.org/10.1145/3604915.3608829

Narrative-driven recommendation (NDR) presents an information access problem where users solicit recommendations with verbose descriptions of their preferences and context. For example, travelers may seek recommendations for points of interest while describing their likes, dislikes, and travel circumstances. This project consists of creating a semantic search-based recommendation system. For instance, a user might request 'a gift for my father,' and the recommendation system could suggest a wallet. The user’s preferences may play a relevant role.

Topic 2: Vančura, V., Alves, R., Kasalický, P., & Kordík, P. (2022, September). Scalable Linear Shallow Autoencoder for Collaborative Filtering. In Proceedings of the 16th ACM Conference on Recommender Systems (pp. 604-609). https://doi.org/10.1145/3523227.3551482

This paper proposes a shallow linear autoencoder. Shallow autoencoders provide interpretability, as demonstrated in the last figure of the paper. However, this interpretability is still based on structured variables, and little meaningful semantics can be extracted. This project should consist of proposing a semantic explanation of the models based on large language models.

Topic 3: Mostafa Rahmani, James Caverlee, and Fei Wang. 2023. Incorporating Time in Sequential Recommendation Models. In Proceedings of the 17th ACM Conference on Recommender Systems (RecSys '23). Association for Computing Machinery, New York, NY, USA, 784–790. https://doi.org/10.1145/3604915.3608830

Sequential models are designed to learn sequential patterns in data based on the chronological order of user interactions. However, they often ignore the timestamps of these interactions. Incorporating time is crucial because many sequential patterns are time-dependent, and the model cannot make time-aware recommendations without considering time. This article demonstrates that providing a rich representation of time can significantly improve the performance of sequential models. This project would consist of incorporating time into next basket prediction models, addressing the temporal aspect that differentiates it from traditional sequential recommendation systems. The next basket prediction problem focuses on predicting a user’s next set of items (in a next session) to purchase, taking into account temporal dependencies, while sequential recommendation systems primarily focus on recommending items (not set of items) in a single session.

Topic 4: You could apply PML methods in areas less explored than recommendation systems.