In 2016, the comedian John Oliver established “Our Lady of Perpetual Exemption,” a bogus church that received tax-exempt status as a religious organization. In 2015, the NFL cheerfully announced their front office was voluntarily giving up its nonprofit tax-exempt status. The MLB central office did so in 2007, while the NHL has remained tax-exempt. Country clubs are nonprofits. Federal Credit Unions are nonprofits, as are charities that spend no dollars on aid. Organizations classified as 501(c)(4)s can participate in politics, so your tax dollars are indirectly funding at least some political activism you dislike. At one point the NYSE was tax-exempt. The Church of Scientology still is.

What all of these organization have in common is what the IRS expects of them. Section 501(c) of the Internal Revenue Code is the source of most, though not all, of the list of types of groups that are exempt from federal income tax on their earnings as to that type of business. (Unrelated Business Income, as it is called, is always taxed.) The mechanics are simple: if your organization meets a categorical standard, the money that organization earns is free from federal income tax. The income cannot inure to the benefit of any private stockholder or individual, something that was in the news recently, or, these days, a lifetime ago, when the Trump Foundation came under scrutiny for essentially paying itself.

Now, the IRS still runs this, so, your status frees you from taxation, but not paperwork. Exempt organizations have to keep detailed records and file comprehensive returns, called Form 990s. In fact, one possible reason the professional leagues have given up their nonprofit status is to avoid publicizing salaries. There are a ton of types of 501(c) organizations, but the main umbrella is 501(c)(3): formed for religious, charitable, scientific, literary, or educational purposes, testing for public safety, amateur sports competitions, or for the prevention of cruelty to children or animals. Because this list is long, it’s often shortened to “charities,” which is incomplete, but works because charity itself is such a flexible term. Notably, these organizations can also receive tax-deductible donations. So the benefit is stacked: the organization itself can operate without its earnings being taxed, and those who donate to it can also then take that money out of their income that is taxed. That’s what separates 501(c)(3)s, which includes churches, from country clubs and 501(c)(4) “social welfare” groups, which are allowed to lobby.

All of this backgrounds two points: first, that non-profit tax status is here to stay. Second, that there’s no way to filter out the duds.

Taxing nonprofits would be so unpopular no one has suggested it, for all the normal reasons regarding entrenched interests and the status quo, as well as the actual upsides of charitable work. But it’s good to think through what the reasons are for this chunk of the IRS code, especially given the wide variety of organizations that benefit. U.S. charity law offers “little guidance to nonprofits regarding how to provide charity.” James R. Hines Jr. et al., The Attack on Nonprofit Status: A Charitable Assessment, 108 Mich. L. Rev. 1179, 1181 (2010). Good organizations operate as nonprofits, but nonprofit status does not guarantee that an organization is a good one, despite everyone’s best intentions. Theoretically, “the official exempt designation from the IRS signifies to the public that the organization serves a purpose that Congress or the IRS has identified as socially beneficial.” Susannah Camic Tahk, Crossing the Tax Code’s For-Profit/Nonprofit Border, 118 Penn St. L. Rev 489, 491 (2014). That’s not clearly the case. Additionally, if the “neutral” here would be taxing all earnings, regardless of the putative aim of the organization, then tax-exempt status for nonprofits is just an efficient block grant. The government spends money on these types of organizations by leaving them alone.

There are three economic arguments that support keeping nonprofits untaxed: public goods, agency, and altruism. Anup Malani and Eric Posner, in arguing for a for-profit charity apparatus, address these. Public goods theory stands for the principle that people preference their marginal private benefit over marginal social benefits when making payment choices. So, tax-deductible donations, and tax-preferred status, means more money goes in for a good cause than would otherwise. Agency theory argues that nonprofits position themselves as experts in the field, and so can do more good—the same general idea, scaled, as why food banks prefer you to send them cash, not cans. They have leverage you don’t.  And the altruism theory is a sort of feedback loop: good people do good, so subsidizing that good results in more good going back into the system.

All of this collapses into one general point: as a capitalist society, we decide what we value in dollars. By taxing nonprofits, especially charities, differently, we signal their legitimacy.

Most of the arguments about tax status for charitable organizations and other nonprofits assume that they are good, and focus either on efficiencies, expansion, or category-specific reform. But when you value a category, tax-wise, you value every player in that category. And there are a lot of scams out there. There are tax-exempt organizations that do real harm. One problem is my list of those is not going to be the same as your list. Every argument ever had at the family dinner table, or in the editorial page of a newspaper, or on public transportation, or in the comments section, about some societal ill—some organization with tax-exempt status has actively worked to either perpetuate or stanch the flow of that harm, depending on your side. Any time a group of people gets together with an overarching purpose: this is where our sharpest disagreements develop, and those purposes define what gains tax-exempt status.

The justifications for permitting the particular treatment that 501(c)(3) organizations receive are similar to the ones behind free speech. John Oliver’s ridiculous church is tax-exempt, just like the Church of Scientology, because of the collective resistance to government line-drawing about what makes the right kind of religion. When the government is empowered to make normative judgements about which purposes are legitimate—what types of goals, what kinds of living—eventually, that power will be used to undermine something you value, or to perpetuate something you don’t. If this tax structure is valid qua tax structure, it’s valid in each instance. We live in a pluralistic society where different people believe in different things. This is good. It’s the whole point.