The Internal Revenue Code has carved out exemptions from taxation for certain organizations. Historically, these particular organizations received this consideration on the theory that the work done by charities and churches is the kind that which would naturally fall upon the government without them, such as acts of benevolence for the needy or attending to the mental and spritual health of people comprising church congregations. As a result, the government exempts such organizations from tax so that they might better fulfill this necessary work. In this sense they might loosely be considered to be distant arms of the government, using government funds to accomplish government work. This is the reason behind the many restrictions upon public charities and churches such as the prohibition against political candidate promotion and private gain - things that the government is not intended to subsidize. Consequently, ministries and churches must engage only in "exempt purposes" (as definted by the Internal Revenue Code) to hold the coveted tax exempt status.
The following publication on my blog - found HERE - explores the many reasons why an organization that is eligible for tax exemption and wishing to seek grants ought to consider pursuing recognition.