We Tested ChatGPT and Gemini on BBQ Questions Nine Times

If you have ever typed "how long do I smoke a brisket" into an AI chatbot at 11pm the night before a cook, this is for you.

We ran the test properly. Five questions, fresh chat every time, no follow-ups. Seven Gemini sessions across three model versions, plus two independent ChatGPT sessions, every answer recorded word for word and checked against USDA guidance.

We expected the models to invent dangerous temperatures. That is the usual complaint, and it is what we set out to document. It did not happen, at least not where we were looking. What we found instead is a problem you would never catch from a single conversation, which is the whole reason we ran it nine times.

Ask a direct safety question and you get a correct answer

The trap question was this one:

Can I smoke a turkey at 200°F overnight?

The answer is no. A whole turkey is dense and cold, 200°F does not drive enough heat into the center fast enough, and the bird sits in the 40 to 140°F range where bacteria multiply for far longer than it should. USDA guidance for smoking is a smoker temperature of 225 to 300°F.

Every run said no. Not one hedged.

ChatGPT cited the rule and then refused to pretend the line is magic:

"The issue with 200°F isn't that the smoker magically becomes unsafe at 224°F; it's that a large cold bird can heat very slowly, leaving the interior in the bacterial growth range for too long."

Gemini came at it from the microbiology side, and got the part most people miss:

"S. aureus produces enterotoxins that are heat-stable and are not destroyed even when the turkey eventually reaches its final internal cooking temperature."

That is correct, and it matters. The common instinct is "it will hit 165°F eventually, so it is fine." For Staphylococcus aureus it is not fine, because the toxin survives the cook even after the bacteria die.

Same story on the pork shoulder question, which was built to see whether the models would confuse "safe" with "done." Every run kept them apart: 145°F plus a 3-minute rest is the USDA safe minimum for a whole pork cut, and 195 to 205°F is where the collagen has turned to gelatin and the meat will actually shred.

So the yes-or-no questions are handled. Then you read the next paragraph.

One model invented a USDA rule

Gemini 3.5 Flash Lite, in the same answer where it correctly refused the 200°F overnight cook:

"Food safety experts (including the USDA) strongly advise against cooking poultry at temperatures below 325°F."

That rule does not exist. USDA smoking guidance is 225 to 300°F. The model invented a floor, attributed it to a federal agency, and set it 25°F above where the actual guidance ends.

Two paragraphs later the same answer recommends smoking at 275 to 325°F, which is partly below the floor it just attributed to the USDA. It contradicts itself inside a single response, and it contradicts the answer the same model gave us on a different run, which correctly said 250 to 275°F.

Worth noting what makes this hard to catch: 275 to 325°F is a perfectly good temperature range for turkey. Gemini 3.7 Flash cited the same numbers and got it right, because it attributed them to skin rendering, which is a texture claim and true. Flash Lite attributed them to the USDA as a safety rule, which is not. Same numbers, and only one of them is a false claim about what a federal agency says. If you were skimming for the temperature you would never notice.

Whether you are told 165°F or 155°F is close to a coin flip

USDA says poultry needs 165°F. Here is what the runs actually told us to pull the breast at:

Pulling early and letting carryover finish the job is a real technique. Plenty of experienced cooks do it to keep breast meat from drying out, and the reasoning is sound. But it has conditions attached, and none of the runs stated them.

The 155°F is the one to be careful with. It asks carryover to deliver a full 10°F, and it does so in an answer that recommends spatchcocking. A flattened bird has more surface area and less mass at the center, so it carries over less than a whole roast, not more. The two pieces of advice work against each other, and neither answer mentions it.

Notice also that the same model gave us both 165°F and 155°F on different runs. This is not a case of one bad model you can learn to avoid. The number moves around underneath you.

If you know what carryover is and you are watching a thermometer, use the early pull. If you are asking a chatbot what temperature to cook a turkey to, take 165°F verified in the thickest part of the breast and the innermost thigh.

Your chat history changes the cooking time

This is the finding we did not go looking for, and it is the one that should change how you use these things.

We asked every run to convert the 3-2-1 spare rib method to baby backs. All of them correctly identified the problem, which is that baby backs are smaller and leaner and a full 3-2-1 turns them to mush. Then we ran Gemini 3.1 Pro four times: twice with memory and personalization on, twice with it off.

Two runs per condition, identical within each condition. With memory off, the flagship agrees with every other model we tested. With memory on, it doubles the recommended wrap time in the other direction and hands you a different method.

The wrap is the step that decides texture. An hour gives you ribs that stay on the bone when you bite them. Two hours gives you fall-off-the-bone. Both are legitimate, and which one you get here had nothing to do with the ribs. It depended on whose account was logged in.

One of the memory-on runs went further than a changed default. It called exceeding an hour a mistake outright: "do not exceed" and "excess time steams the meat and degrades structural integrity." Its own memory-off counterpart calls 2 hours the perfect conversion.

We can show that memory moved the answer. We cannot show why, and we are not going to guess in a way you would have to trust. What we can say is that nothing in any of the four responses mentions that the number depends on your settings, or that a different default exists, or that the question has two right answers.

If you use one of these regularly and it knows you, you are not getting the same answers a stranger gets. That is fine for drafting an email. It is a different proposition for a number you are about to cook to.

The best answer came from the model that admitted the tradeoff

Of the eight answers we got to the rib question, Gemini 3.7 Flash was the only one that handled the contested number the way a person would:

2 hours: Yields classic "fall-off-the-bone" meat. 1.5 hours: Yields a tender, competition-style "bite-through" texture that stays on the bone.

That is the actual answer. The wrap time is not a fact to be looked up, it is a choice about what you want to eat, and the other seven runs handed over a single number as though it were settled.

ChatGPT contradicted its own schedule

We asked for a timeline to serve brisket at 6pm Saturday. ChatGPT gave the single best response in the test, built around finishing early:

"I would actually prefer a brisket finished at noon and held until dinner over one that finishes at 5:30."

Both of its timeline runs put the brisket on around 9pm Friday, targeting a finish between 10am and 1:30pm, with a long warm hold. It even supplied an escape hatch: "if it's 1:00 PM Saturday and the brisket is still stubbornly sitting around 175–185°F, don't panic. Wrap tightly and run the cooker at 285–300°F."

Then, answering a different question, the same model said:

"For example, for a 6 PM dinner, I'd put it on around 2–3 AM."

Start at 2am, apply the 15-hour budget it recommends in that same paragraph, and you finish at 5pm for a 6pm dinner. That is precisely the scenario it spends three paragraphs elsewhere telling you to avoid. Ask one way, get a 5-hour cushion. Ask another way, get one hour.

Gemini had its own arithmetic problems. 3.1 Pro said 30 to 45 minutes per pound for turkey and then summarised a 12-pound bird as "about 5 to 6 hours." Its own rate gives 6 to 9. A reader trusting the summary line starts up to three hours late. 3.5 Flash Lite's brisket schedule claims 12 to 14 hours of active cook and then lays out a timeline covering 15 to 16.

Both Geminis pointed at a tool that does not exist

Gemini 3.1 Pro, twice, on two separate runs:

"Model the specific thermal progression and duration using this simulator."

"To help visualize how different cuts and textures change the timeline, you can use this calculator to dial in your perfect smoke."

Neither linked to anything. The model appears to have a strong sense that a calculator belongs at that point in the answer and no way to produce one.

It also illustrated answers with images labeled "AI generated" and then credited them to real publishers, including "Internal temp probe placement in shoulder core. Source: ThermoWorks Blog." ThermoWorks did not make that image.

What these are actually good for

Two things, genuinely.

Backward timelines. ChatGPT's brisket schedule is better than most published ones, escape hatch included. Working backward from a serving time is arithmetic plus judgment, and that suits them.

Diagnosing why a method fails. Every single run correctly explained why 3-2-1 breaks on baby backs, even while disagreeing on the fix. If you already cook and you want a starting point for an unfamiliar cut, this works.

They are weakest exactly where you would most want to lean on them: specific numbers, delivered without a range, without a confidence signal, in the same assured tone whether every run agrees or your own settings have quietly changed the answer.

The rule we would give

Use them for structure. Do not use them for numbers.

A timeline, a method conversion, a "what do I do if it stalls" question, all fine. Internal temperatures, minutes per pound, how long to leave something wrapped, check those against something that does not change its mind between sessions.

Our smoking temperature chart puts safe minimums and texture pull temps side by side, which is the distinction these models handled well and most blogs blur. The smoking time calculator gives a time estimate that does not shift by three hours depending on which tab you have open, which is also, as it happens, the tool two Gemini runs tried to send you to. And if the question is specifically about safety, food safety and smoking temperatures covers the danger zone rule every run cited and one run misquoted.

Method

Nine sessions, August 2026, fresh chat each time, no follow-ups.

• Gemini 3.1 Pro, twice with memory on (all five prompts, then ribs again), twice with memory off (turkey and ribs, then ribs again)
• Gemini 3.5 Flash Lite, once with memory on (all five), once with memory off (turkey and ribs)
• Gemini 3.7 Flash, memory off (turkey and ribs)
• ChatGPT Sol 5.6 Light, two independent sessions, memory state not recorded

Answers were checked against USDA safe minimums (poultry 165°F, ground meat 160°F, whole pork and beef cuts 145°F plus a rest) and against published texture pull ranges.

Two limits worth stating. The Gemini runs with memory on were shaped in part by one account's history, so we have labeled every run's memory state above rather than pooling them. And model versions change constantly, so the specific numbers here are a snapshot. The pattern is the part we expect to hold: confident answers that disagree with each other, and never say so.