Grant Experts Grilled an AI Grant Writer

Here's what it wrote

Last week, I asked folks on LinkedIn to suggest some typical grant proposal questions to ask Grantable, an AI-powered grant writing platform, to answer for me. (Full disclosure, I am the co-founder and CEO of Grantable) I wanted to see how the software would do, and to share the drafts it came up with for people to evaluate.

As background, Grantable uses OpenAI’s GPT3 large language model, the same technology behind the famous ChatGPT chatbot, to derive its writing abilities. To use the software, a grant writer uploads previously written grant proposals (or other writing about the grant applicant), which Grantable uses as context to answer new prompts. My account contains four prior investment and grant applications I’ve written and I was curious to see how it would perform.

Four prompts were suggested, I ran each through the system and posted screenshots of the results. I was nervous because I wasn’t sure I had given the software enough contextual material for it to generate decent answers, but I committed to post the outputs without editing them.

Here’s how it went.

Question 1: How will you ensure sustainability of this project?

The first question suggested was:

How will you ensure sustainability of this project?

To which Grantable replied:

To ensure the sustainability of this project, our team is committed to developing long-term strategies to maintain the project’s success over time. We will use our technology to improve the capacity of grantors and grantees to match more effectively, thus creating a more efficient allocation of resources. Additionally, our team will focus on creating long-term goals that prioritize equity and help generate profoundly positive outcomes in the world. We are confident that, through these strategies, our project can continue to have a positive and lasting impact on society.

The person who suggested this prompt was not very impressed, writing:

Sorry (not sorry), Philip Deng FAIL. 👎🏽 This is a canned response that does not outline a plan, name other sources of revenue, nor give me any indication of how the applicant is actually doing the work of creating financial or social sustainability, i.e. implementation. If I had a dollar for every time I edited out the phrase "...our team is committed to developing long-term strategies to maintain the project's success over time...", I'd donate it all to my local animal shelter and no dog would ever be lonely again

I’m in complete agreement with this commenter. This is how I used to answer when a teacher called on me in class and I hadn’t done my homework.

Grantable’s response was all style and no substance, in part because the documents I gave it as template material do not contain much about sustainability, specifically. I’ve written about how we will sustain our company, but in different terms than a nonprofit might discuss them, such as by describing our business model and customer acquisition strategy.

A key takeaway here is the importance of inputting good data as a prerequisite to receiving good outputs. This prompt was unspecific and I hadn’t given Grantable enough context to work with, and so the result was a bit like my ad-libbing back in middle school.

Here’s how I would edit the draft Grantable gave me.

To ensure the sustainability of this project company, our team is committed to developing long-term strategies to maintain the project’s success over time create the best possible grant writing software, to meet our customers where they are, and to deeply understand our cost and revenue drivers. We will Our long-term vision is to use our technology to improve the capacity of grantors and grantees to match more effectively, thus creating a more efficient and equitable allocation of resources grant funding around the world. Additionally, our team will focus on creating long-term goals that prioritize equity and help generate profoundly positive outcomes in the world. We are confident that, through these strategies, our project can continue to have a positive and lasting impact on society.

I made significant modifications to bring the answer more in line with what I know it should be, and I also discarded the last two sentences entirely. It took me about two minutes to make these changes.

Another important takeaway is, now that Grantable has this edited answer, the next time I encounter a question about sustainability, the software will have much better template material to create a much stronger draft.

Question 2: Where does this project fit within your organizational strategy and vision?

The second question suggested was:

Where does this project fit within your organizational strategy and vision?

Grantable’s reply was:

This project fits perfectly within our organizational strategy and vision of providing the best grant writing software solution on the market. Our strategy and vision are focused on providing a solution that will reduce the time and effort required for grant writing, allowing organizations to focus on other activities and apply for more grants. We believe that this project will help us achieve this goal by showcasing our software at major nonprofit technology conferences, building an attractive and well-supported affiliate program, and supporting limited engineering improvements. Ultimately, we believe that this project will help us to secure 50 or more enterprise contracts and/or affiliate partnerships by the end of 2023, which will ultimately help us reach our goal of providing the best grant writing software solution on the market.

This time the same commenter was fairly impressed by the quality of this answer. The biggest difference is this response communicates meaningful information about a goal and a sequence of activities leading to it. It has specific targets and is very descriptive.

Both the prompt and the template materials were much more suitable for the software to generate this response. This time, Grantable had done its homework and was more prepared for the teacher to call on it.

Question 3: What are the technical objectives in development of Grantable?

The third prompt was:

What are the technical objectives in development of Grantable?

Grantable wrote:

The technical objectives for the development of Grantable are two-fold. First, we need to create a premium SaaS tier that makes full use of generative AI writing technology by investing in AI language models and bringing in expertise to embed this technology effectively at the core of our platform. Second, we need to build our digital marketing channels and create curriculum to train people how to use Grantable to win more grant funding. To achieve the technical objectives for the development of Grantable, we need to allocate resources for AI language model development, software engineering, marketing and communications, and curriculum development.

“Decent, but needs work,” was the consensus opinion on the thread regarding the third response. There is some specific information, but the reply is not crisp or clear. A question seeking technical objectives should point directly at what is to be achieved, in what manner, and when.

Reading this answer reminded me these random sample questions were given to the system without context. There was no larger proposal within which to position or refine the narrative. If I was actually writing a grant proposal, I would know the story I wanted to tell and be able to both give better initial guidance to the software and make more effective refinements. This where a human guiding the direction of the narrative remains critically important.

Question 4: What qualitative metrics will you use to evaluate the outcomes of this project?

The last question was:

What qualitative metrics will you use to evaluate the outcomes of this project?

The reply was:

To evaluate the outcomes of this project, our qualitative metrics will include how much time our users are saving and how much increased grant funding they are securing. We will measure this success by embedding this project in our regular semi-annual planning process during which we write detailed narrative plans laying out goals for the upcoming six months. Additionally, we will gather a cohort of research participants and provide them with resources and training that set expectations for the duration of our study. Finally, we will conduct an after action analysis of this entire process, itself, to see how it can be improved before conducting additional cycles to measure our ongoing social impact.

Once again I can tell Grantable is stretching a bit to make it to the end of the response. The template material I gave it had a few bits and pieces related to metrics and measurement, and it does a reasonably good job of stringing them together smoothly. Based on this attempt, I believe, with more context around the prompt, I would be able to give the software better context at input to receive a more effective output.

Looking ahead

Given how little template material the system was given, the slight misalignment between nonprofit and for-profit vernacular, and the random and non-contextualized nature of the prompts, I am heartened by how well Grantable did.

A theme I’ve heard often from many grant writers I’ve spoken with over the years is grant writing can be isolating and lonely work. When I was a freelance grant consultant, I certainly experienced this. I’m seeing the potential to improve the qualitative experience of grant writing by making it a collaborative experience even for solo grant writers.

For people who work on grants, the first few proposals are where most of the creative process takes place. After that, you’re reheating leftovers. I’m hopeful that working alongside AI can feel like developing new recipes with a talented sous chef bringing you spoonfuls of narrative to taste and modify. “Let’s make this a bit punchier.” “More detail.” “A dash more data.”

Ok, enough with the food metaphors. I’m getting hungry!

---

Thanks for reading this edition of The Process. Please share and comment or you can email me at philip.deng@grantable.co

Philip Deng is the CEO of Grantable, a company building AI-powered grant writing software to empower mission-driven organizations to access grant funding they deserve.