In this paper, we introduce a new generalization of the Hermite matrix polynomials expansions of some relevant matrix functions appearing in the solution of differential systems. An explicit representation and an expansion of the matrix exponential in a series of these matrix polynomials are obtained. Properties of Hermite matrix polynomials such as the recurrence formula permit an efficient computations of matrix functions are established. A new expansion of the matrix exponential for a wide class of matrices in terms of Hermite matrix polynomials is proposed.