The increase in advanced types of malwares is one of the factors that are very challenging to traditional detection systems. In this paper, a new hybridized deep learning model is proposed, which combines Variational Autoencoder (VAE) and self-attention architecture and MobileNet with strong malware detection and classification. It is proposed that the research methodology will use the EMBER 2018 and EMBER 2024 datasets, with extensive preprocessing and normalization processes to improve the ability of features to be represented. Our dual-branch model learns complementary information the VAE-attention branch learns contextual dependencies of latent representations, whereas the MobileNet branch learns hierarchical visual patterns of malware representation in the form of byte representations. These attributes are combined with the help of interconnecting layers and trained with XGBoost to provide a final classification. As reported by experiment, binary classification and multi-class classification accuracy of 99.47 and 98.23 respectively are significantly better than the current state-of-the-art methods on EMBER 2024. The framework has a 3.2% improvement of detection rate with 42% false positives compared to traditional methods.