The aim of this study is to implement both experimental and numerical approaches to investigate the permanent deformation in asphalt paving mixtures modified with waste plastic (WP) and crumb rubber (CR). In this study, three types of asphalt cement were used: neat binder (VB) obtained from a local refinery, VB+3% WP, and VB+15% CR. Two aggregate gradations were employed for the preparation of asphalt concrete mixtures as surface and base courses. The paper plan included conducting a wheel tracking test to investigate the rutting behavior of the mixtures. The input data was proposed structure of pavement layers was built on dynamic modulus and wheel tracking results obtained at various temperatures. The study showed that WP- and CR-modified asphalt mixtures (base course) with a coarse aggregate gradation can provide greater resistance to rutting than the unmodified mix prepared with a fine and/or coarse aggregate gradation. ABAQUS software was applied for different dynamic modulus data of asphalt mixes, California Bearing Ratio (CBR) values of subgrade, temperatures, and tire pressures. There was a good relationship between the rut depth derived from the wheel tracking test and the ABAQUS results. The modified asphalt mixture prepared with aggregate gradation used in the base course and mixed with rubber-modified binder showed more rutting resistance than other modified asphalt mixtures. The novel rut depth model was set up using statistical analysis by analyzing 336 runs of the ABAQUS models.