CHURCH-BASED PROJECTS LACK DATA ON RESULTS

PHILADELPHIA -- In his office at the University of Pennsylvania, Prof.
Byron R. Johnson has just shut off the electronic chirp on his computer
that announced every incoming e-mail message. It was chirping more than 80
times a day, joining the ringing telephone in contrapuntal distraction.

Mr. Johnson is suddenly in demand because he is among the few social
scientists who have tried to measure the influence of religion on social
problems. Less than a year ago, he joined the Center for Research on
Religion and Urban Civil Society, the institute started by his fellow
criminologist John J. DiIulio Jr. Not long after his arrival, though, Mr.
Johnson was left alone; Mr. DiIulio went to Washington to lead the new
White House Office on Faith-Based and Community Initiatives.

Now, with Congress holding hearings on the Bush plan on Tuesday, Mr.
Johnson is being deluged with requests for the research to support the
assertions made by President Bush and other politicians that religious
programs can transform the lives of drug addicts, criminals, welfare
recipients and troubled teenagers, and that it can do so for less money
than government programs.

The truth, Mr. Johnson and many other social scientists say, is that there
is little reliable research proving the effectiveness of religious
programs. They also add that there is scant evidence showing which
religious programs show the best results and how they stack up against
secular programs.

"From the left to the right, everyone assumes that faith-based programs
work," Mr. Johnson said. "Even the critics of DiIulio and his office
haven't denied that. We hear that and just sit back and laugh. In terms of
empirical evidence that they work, it's pretty much nonexistent.

"We've created an office out of anecdotes."

In the history of grand presidential initiatives, this would not be the
first to take the stage without a script. But this one is different. A body
of research is essential to the project's success for the simple reason
that it would be unconstitutional for the government to decide which
religious programs to finance based on theology or favoritism or
familiarity. President Bush and Mr. DiIulio have frequently said that a
record of effectiveness is the only viable measure.

No one denies that religious organizations and volunteers do indispensable
work caring for people in need. For example, Mr. Johnson's colleague, Ram
A. Cnaan, a professor of social work at the University of Pennsylvania,
found that more than 90 percent of Philadelphia's congregations provided
community services. There is also extensive research showing the benefits
of faith: religious people cope better with old age, sickness and hardship;
they are healthier; they drink less alcohol; they volunteer more.

Mr. Johnson pulls reports off his shelves with more evidence: religious
youths are less likely to use drugs, or be involved in crime. But these
studies, he and other researchers say, do not prove if someone can get more
help from a religious program than from a secular one.

Even large human service organizations like those affiliated with the
United Way are only now beginning to measure the effectiveness of their
work, said William H. Wubbenhorst, technical director for ORC Macro
International, a consulting firm in Maryland. For years, groups that did
keep records tracked only how many people they had served or how much time
they spent with clients.

"They'll hand you a three-page in-house report," Mr. Johnson said, "showing
that they reached 1,300 people that year. But what does 'reach' mean?
That's not going to cut it."

One program that has opened itself to scrutiny is Teen Challenge, which
treats nearly 3,000 drug and alcohol addicts annually in 150 centers around
the country. The group says the secret to its success is what it calls the
Jesus factor.

In 1995, Teen Challenge helped Aaron Todd Bicknese, a doctoral student at
Northwestern University, track down 59 people one to two years after they
had completed Teen Challenge's yearlong residential program. Mr. Bicknese
compared them with a similar group of addicts who had spent one or two
months in a hospital rehabilitation program.

The results were favorable to Teen Challenge, which posted a simplified
summary of the dissertation on its Web site concluding that it had an 86
percent success rate. In recent months, politicians and evangelical leaders
have used that figure to assert that religious programs are superior to
secular ones.

Mr. Bicknese found that Teen Challenge graduates reported returning to drug
use less often than the hospital program graduates, but not less than the
hospital program graduates who continued attending Alcoholics Anonymous
support groups (which some also consider to be religious because of their
reference to a "higher power").

He also found that Teen Challenge graduates were far more likely to be
employed (18 of the 59 worked at Teen Challenge itself, which relies on
former clients to run the program).

Social scientists have pointed out that the 86 percent success rate of Teen
Challenge is misleading. It does not count the people who dropped out
during the program. And like many religious and private charities, Teen
Challenge picks its clients.

Before they are accepted, most of the addicts have already been through
detoxification programs, said the Rev. John D. Castellani, president of
Teen Challenge International U.S.A. In the program's first four-month
phase, Mr. Castellani said, 25 to 30 percent drop out, and in the next
eight months, 10 percent more leave.

This raises questions for David Reingold, a researcher at the Indiana
University School of Public and Environmental Affairs. A study Mr. Reingold
has just completed of social services in Indiana found that religious
programs are more likely than their secular counterparts to limit the
clientele they serve. As a result, Mr. Reingold said, "It's an extreme
exaggeration to say that religious organizations are more effective."

Mr. Wubbenhorst said the lack of reliable studies should not derail the
president's plan. "I believe the initiative should be viewed in the spirit
of experimentation and innovation," he said.

Until now, studying religion's influence was "the kiss of death" for
research proposals, Mr. Johnson said.

As Mr. Cnaan explained, "The religious foundations said the benefits of
religion were too obvious, and the secular foundations wouldn't touch
religion."

That is likely to change. Now, Mr. Johnson said, "we're going to have a
chance to find out how effective faith-based groups are."

With that, he gathered up his papers and left for a meeting with a
foundation that is considering giving him a multimillion-dollar research grant.