I Asked GPT-3 to Write Some Chocolate Chip Cookie Recipes…and I Baked Them 🍪

I was recently granted access to OpenAI’s GPT-3 API. This system provides state-of-the-art natural language intelligence for a variety of applications including text generation, translation, summarization, code generation, classification, and more.

To read OpenAI’s GPT-3 post, visit: https://openai.com/blog/gpt-3-apps/.

Recipe Generation

One of the available GPT-3 apps is the Recipe Generator, which takes in a list of ingredients and it will generate the instructions based on those ingredients.

In this API, you have the option between two “Instruct” models: da Vinci and Curie, which are described as follows:

  • davinci-instruct-beta: Shorter and more naturally phrased prompts, complex intent, cause and effect.
  • curie-instruct-beta: Shorter and more naturally phrased prompts, language translation, complex classification, and sentiment.

Then, within each model there are a few parameters with which to tinker and may lead to varying results.

I had it generate 2 recipes and I baked them along with a real recipe as a control…you know, for science. 🤓

Better Homes’ Chocolate Chip Cookie Recipe

To start, I dug out my mother’s Better Homes cookbook from 1989 and found the chocolate chip cookie recipe.

This ingredients list for this recipe is quite simple:

  • ½ Cup — Shortening
  • ½ Cup — Butter
  • 2 ½ Cup — All-purpose Flour
  • 1 Cup — Brown Sugar
  • ½ Cup — Sugar
  • 2 Eggs
  • 1 Tsp. — Vanilla
  • ½ Tsp. — Baking Soda
  • 2 Cups — Semisweet Chocolate

Instructions:

  • In a mixing bowl, beat the shortening and butter with an electric mixer on medium to high speed for 30 seconds.
  • Add about half of the flour and the brown sugar, sugar, eggs, vanilla, and baking soda.
  • Beat mixture until thoroughly combined.
  • Beat in the remaining flour.
  • Stir in chocolate pieces.
  • Drop dough by rounded teaspoons 2 inches apart onto an ungreased cookie sheet.
  • Bake in a 350ºF oven for 8 to 10 minutes.
Original Better Homes Cookie

These cookies turned out nice and plump, no spreading out very much on the baking sheet. They are a bit dry and crumbly with a scone-like texture.

I then used this same ingredient list in the text two GPT-3-generated recipes. (Note: I halved the recipe because I didn’t need 3 full batches of cookies to eat.)

Cookie Generation 1: da Vinci Model

For the first attempt, I used the da Vinci model and left the randomness (called Temperature) and penalties low. This generated the following instructions:

  1. Preheat oven to 350ºF.
  2. In a large bowl, cream together shortening and butter.
  3. Add brown sugar and sugar and mix well.
  4. Add eggs and vanilla and mix well.
  5. Add flour, baking soda, and salt and mix well.
  6. Add chocolate chips and mix well.
  7. Drop by spoonfuls onto a cookie sheet.
  8. Bake for 10–12 minutes.

Apparently, it’s imperative that I mix this well.

da Vinci Cookie

Though the taste of these cookies is identical to the original recipe, they have a softer, chewier texture since they cooked at a lower temp for a longer amount of time.

Cookie Generation 2: Curie Model

For the second attempt, I changed the model to the Curie version and cranked up the randomness and penalties of the generation. The instructions were a bit more unusual:

  1. Preheat oven to 350ºF.
  2. Cream together shortening, butter, and sugars in a large bowl until light and fluffy.
  3. Whisk in eggs and vanilla.
  4. Whisk in flour, vanilla, and baking soda.
  5. Mix in chocolate cocoa power by hand
  6. Bake for 10–12 minutes or until lightly browned on top.

Note that this model interpreted “semisweet chocolate” as cocoa powder (and that I had to mix it in by hand).

Curie Cookie

These cookies worried me a bit. I thought they’d be terrible, but they’re actually great! The cocoa powder cuts the sweetness and would be a decent option for those of you who like dark chocolate. ,Since these baked for a similar amount of time as the da Vinci cookies, they also have a softer, chewy texture.

How’d it Do?

Honestly? Not bad. The original Better Homes’ recipe isn’t the best cookie recipe to begin with, so the da Vinci and Curie cookies were actually improvements on the original as far as texture goes.

The original recipe leaves you with cookies that are a bit more scone-like and crumbly whereas the GPT-3-generated ones bake at a lower temperature and for longer, yielding a softer, flatter, all-around better cookie. However, I suppose the Curie model’s recipe isn’t really a chocolate chip cookie, rather more of a dark chocolate cookie.

It was really interesting to see how similar many of the resulting instruction sets were despite my efforts to get some crazy responses. Almost all of the generated cookie recipes say to bake at 350ºF, but vary the time and verbs used in the mixing process (like whisk vs. stir vs. mix).

Given that GPT-3 is trained based on millions of examples from across the web, it’s not crazy to think that there are plenty of chocolate chip cookie recipes that it has seen. So, while these results aren’t surprising, I did have it try some more complicated recipes and the instructions were certainly believable.

What should I make next? Also, what other things could GPT-3 be used to generate?

Cloud AI and genomics guy and aspiring polymath. I do research in machine learning and bioinformatics and I sometimes write things here.