The main reason for soft images in IR is split focus. Due to the differing wavelengths, sharp focus is not possible for both visible light and IR light with the same focus position. In many cases cutting visible light completely, such as with the IR spectrum OLPF, yields much sharper images.
If using artificial IR sources, particularly LEDs, be aware that the LED wavelength can also cause this issue. 850nm is a good wavelength for shooting IR on RED, as it matches the peak of exposure response. However, many other IR wavelengths are available, and if mixed together in a scene, can cause soft focus.