As a general rule, if you add even harmonics to a sound then you can make it more pleasing, and if you use odd order harmonics you can make a sound more displeasing. So it might be worth using different types of subtle distortion to either add even or odd order harmonics to a voice depending on which direction you want the character to go in.
If you prefer not to use distortion to create harmonics then a much more subtle effect is to make a voice more nasal or sibilant, it is amazing how quickly annoying it becomes, then add some deliberate but discrete, plosive popping, and then for the positive character make the EQ rich and full, with a slight proximity effect and presence boost (2-4 kHz). Then compress slightly the positive character, and expand slightly the negative character. Strong, confident individuals usually have slightly more constant levels in their speech, irrational speakers vary their levels much more as they are often more dramatic.
Lastly if you want to push it further add some artificial lip smacks, they really do not sound nice, and again will push a character towards the negative end of spectrum, and by contrast the positive character will be perceived as even more positive.
All these effects can be overused, but if you keep them so that they are only just audible, they can create a voice you feel more comfortable listening to, and one that jars slightly.