I would think that there would have to be some logic to rejecting a story line. Continually voting no without being able to articulate why would be wide open for abuse. This game seems to be aimed at those who are willing to working together to create a good cooperative story. If you vote no without being able to put a good reason behind it, that should be reason for your votes to eventually start being discarded from the tally.
Also, if the people writing the story ignore the prime source, then it is fair game for other players.