The first thing to remember is that society is not a single whole. So, it's already fragmented, so-to-speak, into groups and those are often differentiated by various ideologies. That's sort of the point of living in a free society---freedom to associate with those of your choosing, and of course to not associate with others.
This question often comes from people who promote ideas of multiculturalism, diversity, equity, and inclusion. The rationale is that "inclusive" societies are the most peaceful. Not only is this claim dubious, but even advocates for these ideas would not want to be forced to associate with groups with whom they have vehement disagreement. These might include fundamentalist religious groups, for example.
Education freedom isn't about ensuring society conforms to the ideas you've concluded are the best ideas. It's about continually critiquing the ideas that predominate our society in pursuit of the truth. Proverbs 18:17 is apt here: "The one who states his case first seems right, until the other comes and examines him" (ESV).
Not only do we need to be free to challenge ideas that might be wrong, but we must be free to understand why right ideas are right. We can't be confident in what we know as being right, or in our ability to defend the truth, if we have forced conformity to ideas. This only comes from the freedom to disagree, be wrong, and the freedom to arrive at the truth in our own ways in our own timing. In other words, the goal of a free society is not to prevent siloing into differentiated groups. It's about the freedom to leave those silos when you want to.
This question is from Faith Seeking Freedom: Updated & Expanded, available in paperback, PDF, and Kindle.
