This study addresses the limitations of traditional deep matrix factorization models in capturing prediction uncertainty and complex interactions between users and items in recommender systems. To overcome these issues, we introduce a new framework that uses Bayesian deep learning to measure prediction uncertainty, making the recommendations more reliable. We also add an attention mechanism to capture important relationships between the features of users and items. These relationships are further processed using a multi-layer perceptron, which offers a modern approach to compute the similarity score. Experiments on well-known datasets show that our model improves both the accuracy and reliability of predictions. These results show that combining Bayesian learning and attention mechanisms can help make recommender systems more effective and adaptable.