Enhancing Recommender Systems: A New Approach Using Collaborative Filtering with Bayesian Optimization and Gaussian Processes
Abstract
Recommendation systems frequently make use of collaborative filtering (CF). CF derives its strength from the fact that in order to profile its users, it does not require a lot of knowledge about them, but it depends on earlier users’ ratings in which they were choosing products to recommend. Although there has been progress in modeling users as well as items, calibrating CF algorithms’ hyperparameters will continue to be a difficult task. This paper has come up with a new way of doing this by the use of Bayesian optimization with Gaussian processes throughout hyperparameter tuning. The method in question automatically works on hyperparameters to make two fundamental and straightforward CF algorithms have solid results against three famous datasets (Netflix Prize, MovieLens 1M, MovieLens 10M), and two other datasets (Douban and Jester dataset 2). This solution not only exhibits solid performance in controlled experiments but also provides a simplified approach for practitioners, minimizing time- intensive manual adjustments while ensuring low overhead, therefore necessitating relatively low computational and devel- opmental resources. This results in expedited deployment and simplified integration into practical systems, hence enhancing the dependability and scalability of recommendation engines.
References
K. Srikumar, “A framework of agent-based personalized recommender system for e-commerce,” South Asian journal of management, vol. 11, p. 66, 2004.
Y. H. Cho, J. K. Kim, and S. H. Kim, “A personalized rec- ommender system based on web usage mining and decision tree induction,” Expert Systems with Applications, vol. 23, no. 3, pp. 329–342, 2002.
G. Adomavicius and A. Tuzhilin, “Toward the next genera- tion of recommender systems: a survey of the state-of-the-art and possible extensions,” IEEE Transactions on Knowledge and Data Engineering, vol. 17, no. 6, pp. 734–749, 2005.
S. Yan, “A collaborative filtering recommender approach by investigating interactions of interest and trust,” in Knowledge Engineering and Management, Berlin, Heidelberg, 2014, pp. 173–188.
C. P. Lam, “Collaborative filtering using associative neural memory,” in Intelligent Techniques for Web Personalization, Berlin, Heidelberg, 2005, pp. 153–168.
R. Zhang, Q.-d. Liu, Chun-Gui, J.-X. Wei, and Huiyi-Ma, “Collaborative filtering for recommender systems,” in 2014 Second International Conference on Advanced Cloud and Big Data, 2014, pp. 301–308.
G. Linden, B. Smith, and J. York, “Amazon.com recommen- dations: item-to-item collaborative filtering,” IEEE Internet Computing, vol. 7, no. 1, pp. 76–80, 2003.
S. Rendle, L. Zhang, and Y. Koren, “On the difficulty of evaluating baselines: A study on recommender systems,” ArXiv, vol. abs/1905.01395, 2019.
Y. Koren, “The bellkor solution to the netflix grand prize,” 2009.
——, “Factorization meets the neighborhood: a multifaceted collaborative filtering model,” in Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Dis- covery and Data Mining, ser. KDD ’08, New York, NY, USA, 2008, p. 426–434.
M. Imani, M. Imani, and S. F. Ghoreishi, “Bayesian op- timization for expensive smooth-varying functions,” IEEE Intelligent Systems, vol. 37, no. 4, pp. 44–55, 2022.
Y. Morita, S. Rezaeiravesh, N. Tabatabaei, R. Vinuesa, K. Fukagata, and P. Schlatter, “Applying bayesian optimiza- tion with gaussian process regression to computational fluid dynamics problems,” Journal of Computational Physics, vol. 449, p. 110788, Jan. 2022.
J. Snoek, H. Larochelle, and R. P. Adams, “Practical bayesian optimization of machine learning algorithms,” in Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 2, ser. NIPS’12, Red Hook, NY, USA, 2012, p. 2951–2959.
S. Molaei, A. Havvaei, H. Zare, and M. Jalili, “Collabora- tive deep forest learning for recommender systems,” IEEE Access, vol. 9, pp. 22 053–22 061, 2021.
M. F. Aljunid and M. Doddaghatta Huchaiah, “Multi-model deep learning approach for collaborative filtering recommen- dation system,” CAAI Transactions on Intelligence Technol- ogy, vol. 5, no. 4, p. 268–275, nov 2020.
Y. Huang and J. T. Kwok, “Collaborative filtering via co- factorization of individuals and groups,” in 2015 Interna- tional Joint Conference on Neural Networks (IJCNN), 2015, pp. 1–8.
Z. Li, J. Huang, and N. Zhong, “Exploiting user and item embedding in latent factor models for recommendations,” in Proceedings of the International Conference on Web Intelligence, ser. WI ’17, New York, NY, USA, 2017, p. 1241–1245.
F. Horasan, A. H. Yurttakal, and S. Gu¨ndu¨z, “A novel model based collaborative filtering recommender system via truncated ulv decomposition,” Journal of King Saud University - Computer and Information Sciences, vol. 35, no. 8, p. 101724, 2023.
Y. Chen, S. Mensah, F. Ma, H. Wang, and Z. Jiang, “Col- laborative filtering grounded on knowledge graphs,” Pattern Recognition Letters, vol. 151, pp. 55–61, 2021.
N. Khaledian and F. Mardukhi, “Cfmt: a collaborative filter- ing approach based on the nonnegative matrix factorization technique and trust relationships,” Journal of Ambient In- telligence and Humanized Computing, vol. 13, no. 5, pp. 2667–2683, May 2022.
H. Xia, Y. Luo, and Y. Liu, “Attention neural collaboration filtering based on gru for recommender systems,” Complex & Intelligent Systems, vol. 7, no. 3, pp. 1367–1379, June 2021.
S. Rendle, “Factorization machines with libfm,” ACM Trans. Intell. Syst. Technol., vol. 3, no. 3, May 2012.
R. Salakhutdinov and A. Mnih, “Bayesian probabilistic matrix factorization using markov chain monte carlo,” in Proceedings of the 25th International Conference on Ma- chine Learning, ser. ICML ’08, New York, NY, USA, 2008, p. 880–887.
P. Szabo´ and B. Genge, “Hybrid hyper-parameter optimiza- tion for collaborative filtering,” in 2020 22nd International Symposium on Symbolic and Numeric Algorithms for Scien- tific Computing (SYNASC), 2020, pp. 210–217.
R. Bardenet, M. Brendel, B. Ke´gl, and M. Sebag, “Col- laborative hyperparameter tuning,” in Proceedings of the 30th International Conference on International Conference on Machine Learning - Volume 28, ser. ICML’13, 2013, p. II–199–II–207.
M. N. Volkovs and R. S. Zemel, “Collaborative ranking with 17 parameters,” in Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 2, ser. NIPS’12, Red Hook, NY, USA, 2012, p. 2294–2302.
H. H. Hoos, Automated Algorithm Configuration and Pa- rameter Tuning, Berlin, Heidelberg, 2012, pp. 37–71.
P. N. Koch, O. Golovidov, S. Gardner, B. Wujek, J. D. Griffin, and Y. Xu, “Autotune: A derivative-free optimization framework for hyperparameter tuning,” Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2018.
Q. Lu, K. D. Polyzos, B. Li, and G. B. Giannakis, “Sur- rogate modeling for bayesian optimization beyond a single gaussian process,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 9, pp. 11 283–11 296, 2023.
P. Lualdi, R. Sturm, A. Camero, and T. Siefkes, “An uncertainty-based objective function for hyperparameter op- timization in gaussian processes applied to expensive black- box problems,” Applied Soft Computing, vol. 154, p. 111325, 2024.
M. A. L. Pearce, M. Poloczek, and J. Branke, “Bayesian optimization allowing for common random numbers,” Oper. Res., vol. 70, no. 6, p. 3457–3472, nov 2022.
P. I. Frazier, “A tutorial on bayesian optimization,” 2018.
B. Shahriari, K. Swersky, Z. Wang, R. P. Adams, and N. de Freitas, “Taking the human out of the loop: A review of bayesian optimization,” Proceedings of the IEEE, vol. 104, no. 1, pp. 148–175, 2016.
H. Wang, B. van Stein, M. Emmerich, and T. Back, “A new acquisition function for bayesian optimization based on the moment-generating function,” in 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2017, pp. 507–512.
M. Pearce, “Acquisition functions for simultaneous bayesian optimisation of multiple problems,” in 2017 Winter Simula- tion Conference (WSC), 2017, pp. 4618–4619.
J. Berk, V. Nguyen, S. Gupta, S. Rana, and S. Venkatesh, “Exploration enhanced expected improvement for bayesian optimization,” in Machine Learning and Knowledge Discov- ery in Databases, Cham, 2019, pp. 621–637.
M. T. M. Emmerich, A. H. Deutz, and J. W. Klinkenberg, “Hypervolume-based expected improvement: Monotonicity properties and exact computation,” in 2011 IEEE Congress of Evolutionary Computation (CEC), 2011, pp. 2147–2154.
D. Kim and B. Suh, “Enhancing vaes for collaborative fil- tering: flexible priors & gating mechanisms,” in Proceedings of the 13th ACM Conference on Recommender Systems, ser. RecSys ’19, Sep. 2019.
H. Steck, “Embarrassingly shallow autoencoders for sparse data,” in The World Wide Web Conference, ser. WWW ’19, May 2019.
I. Shenbin, A. Alekseev, E. Tutubalina, V. Malykh, and S. I. Nikolenko, “Recvae: A new variational autoencoder for top- n recommendations with implicit feedback,” in Proceedings of the 13th International Conference on Web Search and Data Mining, ser. WSDM ’20, New York, NY, USA, 2020, p. 528–536.
S. C. Han, T. Lim, S. Long, B. Burgstaller, and J. Poon, “Glocal-k: Global and local kernels for recommender sys- tems,” in Proceedings of the 30th ACM International Confer- ence on Information & Knowledge Management, ser. CIKM ’21, New York, NY, USA, 2021, p. 3063–3067.
Z. Chen and B. Wang, “How priors of initial hyperparame- ters affect gaussian process regression models,” Neurocom- puting, vol. 275, pp. 1702–1710, 2018.
Y. Liu, S. Wang, X. Li, and F. Sun, “A meta-adversarial framework for cross-domain cold-start recommendation,” Data Science and Engineering, vol. 9, no. 2, pp. 238–249, June 2024.
H. Xu, C. Li, Y. Zhang, L. Duan, I. W. Tsang, and J. Shao, “Metacar: Cross-domain meta-augmentation for content- aware recommendation,” IEEE Transactions on Knowledge and Data Engineering, vol. 35, no. 8, pp. 8199–8212, 2023.
M. Li, W. Que, Z. Geng, M. Li, Z. Kou, J. Chen, C. Guo, and B. Zhang, “Cold-start item recommendation for representa- tion learning based on heterogeneous information networks with fusion side information,” Future Generation Computer Systems, vol. 149, pp. 227–239, 2023.
P. Magron and C. Fe´votte, “Neural content-aware collabo- rative filtering for cold-start music recommendation,” Data Mining and Knowledge Discovery, vol. 36, no. 5, pp. 1971– 2005, September 2022.
N. Heidari, P. Moradi, and A. Koochari, “An attention-based deep learning method for solving the cold-start and sparsity issues of recommender systems,” Knowledge-Based Systems, vol. 256, p. 109835, 2022.
A. Sriram, H. Jun, S. Satheesh, and A. Coates, “Cold fusion: Training seq2seq models together with language models,” in Interspeech, 2017.
K. Song, W. Gao, L. Zhao, J. Lin, C. Sun, and X. Liu, “Cold- start aware deep memory network for multi-entity aspect- based sentiment analysis,” in Proceedings of the Twenty- Eighth International Joint Conference on Artificial Intelli- gence, IJCAI-19, 7 2019, pp. 5197–5203.
B. Hu, Z. Li, and W. Chao, “Data sparsity: A key dis- advantage of user-based collaborative filtering?” in Web Technologies and Applications, Berlin, Heidelberg, 2012, pp. 602–609.
M. Grcˇar, D. Mladenicˇ, B. Fortuna, and M. Grobelnik, “Data sparsity issues in the collaborative filtering framework,” in Advances in Web Mining and Web Usage Analysis, Berlin, Heidelberg, 2006, pp. 58–76.
F. Yin, Z. Wang, W. Tan, and W. Xiao, “Sparsity-tolerated al- gorithm with missing value recovering in user-based collab- orative filtering recommendation,” JOURNAL OF INFOR- MATION &COMPUTATIONAL SCIENCE, vol. 10, no. 15, pp. 4939–4948, 2013.
T. Head, M. Kumar, H. Nahrstaedt, G. Louppe, and I. Shcherbaty, “scikit-optimize/scikit-optimize (v0.9.0),” 2021, zenodo. https://doi.org/10.5281/zenodo.5565057.
N. Hug, “Surprise: A python library for recommender sys- tems,” Journal of Open Source Software, vol. 5, no. 52, p. 2174, 2020.
R. Garnett, gaussian processes, 2023, p. 15–44.
S. Rana, C. Li, S. Gupta, V. Nguyen, and S. Venkatesh, “High dimensional bayesian optimization with elastic gaus- sian process,” in Proceedings of the 34th International Conference on Machine Learning - Volume 70, ser. ICML’17, 2017, p. 2883–2891.
“Collaborative filtering,” https://spark.apache.org/docs/latest/ ml-collaborative-filtering.html, accessed: 2025-01-28.
H. Ha, S. Rana, S. Gupta, T. Nguyen, H. Tran-The, and S. Venkatesh, Bayesian optimization with unknown search space, Red Hook, NY, USA, 2019.
V. Nguyen, S. Gupta, S. Rane, C. Li, and S. Venkatesh, “Bayesian optimization in weakly specified search space,” in 2017 IEEE International Conference on Data Mining (ICDM), 2017, pp. 347–356.
M. Blum and M. A. Riedmiller, “Optimization of gaussian process hyperparameters using rprop,” in The European Symposium on Artificial Neural Networks, 2013.
R. Preuss and U. von Toussaint, “Parallel computation of Gaussian processes,” AIP Conference Proceedings, vol. 1853, no. 1, p. 050004, 06 2017.
N. T. Blog, “Netflix recommendations: Beyond the 5 stars,” Apr 2012, accessed: 2024-Sep-15.
B. Hallinan and T. Striphas, “Recommended for you: The netflix prize and the production of algorithmic culture,” New Media & Society, vol. 18, no. 1, pp. 117–137, 2016.
E. Campochiaro, R. Casatta, P. Cremonesi, and R. Turrin, “Do metrics make recommender algorithms?” in 2009 Inter- national Conference on Advanced Information Networking and Applications Workshops, 2009, pp. 648–653.
