There is a poignant story told of Rabbi Aryeh Levin. He was once attending prayer services alongside members of the Neturei Karta (a religious anti-Zionist group). When one of those followers noticed Rabbi Levin (who was an admirer and confidant of Rabbi Avraham Yitzchak Kook, an ideological opponent of the Neturei Karta), the man tore the tefillin off Rabbi Levin's head and told him to leave. Rabbi Levin humbly gathered his belongings and left - without saying a word. One of the people who witnessed the event ran after Rabbi Levin and told him how impressed everyone was at how he handled such a public humiliation. He told the man to go back and tell the congregants that he learned how to respond like that from Rabbi Kook.
The Chofetz Chaim quotes a midrash which states that every time a person refrains from engaging in forbidden speech, they merit a hidden light that no angel can fathom. And that midrash is talking about a single moment; one can only imagine what lies in store for a person who can hold back from speaking ill of others on a consistent basis. Obviously, we should strive for good behavior simply because it's the right thing to do. But if you need a little incentive, that's a pretty good one.
Remaining silent has divine origins. The Talmud (Gittin 56b) evokes the praise of God following the Second Temple's destruction as "mi kamocha ba'ilmim Hashem" - "Who is like You among the mute ones O God" (this is a play on words of the phrase "mi kamocha ba'eilim Hashem" - "Who is like You among the mighty ones, O God"). It is referencing how God remained silent in the face of the profane conduct of Titus. This is the Creator of the Universe we're talking about - and it's concerning the destruction of His home on earth - and yet He's willing to exercise restraint! All we have to do is let go of our egos a little bit and try in our own small way to emulate God's characteristics when someone happens to say or do something that personally offends us.
Making our displeasure known with those with whom we disagree is often not worth it. Do we really have to get in our two cents every time we come across some subjective difference with a fellow Jew? Even when someone has objectively hurt us in some way, we have to make sure that our subsequent behavior is constructive. Unless we're fairly certain that rebuking them would have a positive effect, it's best to maintain our composure and simply keep quiet. All we can do is make the best decision based on the information we have at the time. Sometimes we'll still make a poor choice despite thinking before speaking, but at least we'll be more cognizant of our behavior and less likely to hurt someone else.
We have to pick our fights in life. There are times when speaking up is extremely important. For example, when it comes to fighting evil - especially something as serious as radical Islamic terrorism - good people of all stripes must speak out and act. But in the micro realm of life, keeping quiet is usually the best way to go. For example, when it comes to relationships between individuals - especially people we see day in and day out - letting things go is often a wise choice. Routinely following this mode of behavior is a great way to keep the peace. We don't have to be passive; we just have to use some self-restraint.
Our generation greatly values people who speak their minds. In many respects, this has been a positive development. However, a better world will only be achieved when silence is valued just as much - if not more. As the proverbial saying goes, "speech is silver; silence is golden."