Spin Relaxation

In atoms, spontaneous emission is (claimed to be) due to the interaction of the continuum of vacuum modes with the atom that causes an irreversible decay of the atom( why irreversible ??). What about a system of spin half particles in a magnetic field. If initially in the higher energy state (anti-parallel to the field), the particles will eventually relax to the lower energy state. Why? What kind of vacuum does the particle need to interact with and relax ??
The only guess in my mind is a bath full of magnetic dipoles in all orientations and may be energies. Is this correct ??