For the past 20 years, I’ve been a program evaluator by trade, assessing nonprofit and governmental programs at the funder’s request.
Yet when I walk in the door of a newly-funded initiative, I pretty consistently notice a program that is not ready for evaluation. In fact, most are not even ready for implementation. Time and again I find muddled goals, stakeholders with vastly divergent expectations, and vague program strategies.
Most funders’ initial due-diligence processes are comprehensive in ensuring that financial, legal and organizational structures are in place.
But a thorough pre-program assessment examines a number of other factors that are just as critical to program success. And, they are usually easy to fix if detected early.
Deciding whether a program is feasible is all about asking the simple questions.
Is it logical? Does it have a track record? Is there a direct relationship between the strategy and the problem the grantee is proposing to solve?
All of this sounds basic, but these are common pitfalls. Very often a program’s strategy is only tangentially related to the problem. It may be that the strategy is easy to implement, but whether it actually has a chance of solving the problem, or at least solving it within a reasonable timeframe, is a different matter.
Another important consideration is context, which includes all the different things going on around a given program: political issues, cultural issues, leadership issues, the environment in which the program is housed. Any can become landmines if ignored upfront.
Once it’s clear that an initiative is a good idea on paper, a pre-program assessment should ask if it can actually be implemented.
Is there research, past history or empirical evidence to show that this program strategy is really going to work? Is there enough time and the right resources? Are the most qualified people on board? Is the teen-pregnancy prevention staff member the right person to implement a new initiative on substance abuse?
Many nonprofits have multiple funding streams. This particular initiative is going to have to compete with a lot of others. Does the grantee truly have enough energy and resources for one more?
Fidelity is another important yet subtle factor to consider. Is there sufficient dedication among stakeholders to implement this program? Or is it just something to bring in more funding and make the agency look good?
Though a program is both feasible and reasonable, it may still leave an evaluator scratching her head.
The final critical factor to consider is measurability. Will there be any way to demonstrate the effectiveness of the program model? Are there clear goals and objectives? Are systems in place to collect accurate and reliable data? Will anyone be able to detect success?
A better way
Too often when a new grant initiative is announced, short timelines and busy schedules force grantees to develop unrealistic proposals. Several months later when the congratulatory phone call comes in the hastily-written proposal serves as dogma by which the program is implemented.
The process of sitting down up front and revisiting what was put in the grant application is crucial and should involve all stakeholders: the grantmaker’s staff, grantee program directors and any others intimately involved in the project.
This can be a painful process. Most grantees want to shoot right out of the gate, and grantmakers usually expect this.
But grantmakers who think this way really do their grantees a disservice. Programs could be significantly more effective if grantees were allowed a little time upfront, maybe three to six months, to reflect on the initiative and make a few changes to ensure their proposed programs are reasonable, feasible, and measureable.
The Colorado Trust in Denver, for example, often offers three- to six-month planning grants of $5,000 to $15,000 before choosing a final grantee. With this small investment it’s easier to determine where to invest $500,000.
Another issue grantmakers can address upfront is their commitment to supporting a program’s evaluation findings – favorable or unfavorable. Grantees don’t want to publish unfavorable findings, especially if it means program cutbacks, staffing changes or a new direction. Sometimes grantees feel if their program didn’t work out, they won’t be eligible for additional funding.
But, according to many grantmakers, this is an unfounded fear. Grantmakers want to reward grantees for using the truth they uncover – if grantees learn from program failures and make changes, funding will be there for improved programs and new ones. Smart grantmakers know that giving grantees time and resources to critically assess programs before launching them and permission to design meaningful evaluations that get used are the biggest gifts any grantmaker can award a grantee.