A new reaction force model of particle collision is built, which is based on the conservation of momentum and the conservation of energy. Combined with weakly compressible smoothed particle hydrodynamics and the artificial compressibility, the new model and a conventional reaction force model of Lennard-Jones is used to simulate the phenomenon of shear driven cavity and the flow around a square cylinder, respectively. For verifying the accuracy of the new model, the DNS method is also used to simulate the flow phenomenon. By comparing and analyzing the calculating results, it can be concluded that: The new reaction force model can effectively prevent particles from unphysically penetrating through the boundary, and the variation of velocity and tangential stress near the boundary can be relatively accurate calculated. The new model helps to enhance the calculation accuracy of smooth particle hydrodynamics (SPH) for the whole flow field, and it has relatively good stability.