Pocket Timescape Books, 1981. Mass market paperback. Book number 3 in the Pocket series of original Star Trek novels. There were several printings through about 1990.
The Klingon ship Terror has recently murdered the innocent crew of a Vulcan science ship. The Enterprise is sent to meet this new threat, only to fall apart from within. Crew members throw immature temper tantrums. Orders are ignored. One by one, the crew are losing their minds.
Author Vardeman wrote another Trek novel, "Mutiny on the Enterprise" (1983).