讓AI做對事情：引入確定性

Hacker News·3 個月前

本文探討如何透過引入確定性來提升AI的可靠性，並以一個大型語言模型（LLM）在日期計算上遇到困難的例子，說明如何透過客製化腳本來解決此問題。

Jessitron

Making AI Do Things Right: Introduce Determinism

Today, a quick example of introducing determinism when an AI struggles.

I have a simple instruction for it:

Run date to identify the current week. Check the calendar for the week to identify workdays, travel, and meetings

I tell it to run date because it’s terrible at knowing what day it is. In its CLAUDE.md, I gave it a command for checking the calendar.

Today is Monday. What does it say?

Calendar summary:

Ha ha I wish!

It’s doing it wrong! After yelling at my computer, I set about fixing this for the future.

Step 1: Ask why it’s wrong.

It can tell me what it did and notice what went wrong. In this case, I have a pretty good idea where to point it.

Me: what command did you use for gcalcli? because it is not workingClaude: Ah, I see the problem! I used:

But Monday is Jan 12, not Jan 13! So I missed today entirely. Let me fix that…

Yet again, the LLM proves bad at math. Especially dates.

Step 2: Introduce determinism

I want to fix this problem permanently. Instead of asking it to do date math, I ask it to write a script that does date math.

Me: ok, let’s make a script. Make a script for week-agenda, and put make itcalculate the start and end date as monday through friday. Then it callsgcalcli

Claude: Good idea!

It writes a script. I try the script. It left out Friday this time. I tell it so, and it fixes the script.

Next I tell it to change its own instructions. No more trying, only succeeding, from now on.

Me: Now update your skill to point to the scriptClaude: Good idea. Let me update the skill to use the script.

Finally, my favorite part: /clear. Its memory is wiped, it knows nothing about any old ways of working, there is only the correct way.

A bit of deterministic code goes a long way.

The weaknesses of AI can be obviated by its strengths. It’s bad at date math, but it’s good at writing code. A little direction from me, and it gets better at its job all the time. This is working with AI.