Addressing the muscle soreness point, no, lactic acid does not cause delayed soreness. Wikipedia has a nice terse summary of why, backed up by this review (of multiple related studies).
The lactic acidosis of exercise has been a classic explanation of the biochemistry of acidosis for more than 80 years. [...]
This review presents clear evidence that there is no biochemical support for lactate production causing acidosis. Lactate production retards, not causes, acidosis. Similarly, there is a wealth of research evidence to show that acidosis is caused by reactions other than lactate production. [...]
It is only when the exercise intensity increases beyond steady state that there is a need for greater reliance on ATP regeneration from glycolysis and the phosphagen system. The ATP that is supplied from these non-mitochondrial sources and is eventually used to fuel muscle contraction increases proton release and causes the acidosis of intense exercise. [...] If muscle did not produce lactate, acidosis and muscle fatigue would occur more quickly and exercise performance would be severely impaired.
— Abstract from Robergs, R A, F Ghiasvand, and D Parker, Biochemistry of exercise-induced metabolic acidosis
The muscle soreness later, post-exercise, often felt the following day, cannot be caused by lactic acid as it is "flushed from the muscle cells within 40-60 minutes", though it is unclear if the blood stream carries it away or it is simply metabolized within the cell.
Cramping seems to be a more neurological symptom, when the peripheral nerves, rather than the CNS, generate a signal directing skeletal muscles to contract. I can't find any papers that discuss lactic acid (which should be at a higher concentration with the muscle cells rather than extracellular) interfering with neurons or their receptors, so I don't know on that point.