讓 AI 說其母語

Hacker News·4 個月前

本文探討了如何透過教授 AI 處理程式碼，特別是利用圖形轉換器和抽象語法樹，來顯著提升其推理能力，方法是保留結構並克服傳統分詞方法的限制。

Developer forever

Let AI Speak in Its Mother Tongue

4 hours ago

Why Teaching Code Improves Reasoning in AI

Tool use masks architectural limitation

This was quite visible in GPT5’s FrontierMath performance during its launch announcement:

There are two fundamental issues that cause this plateau:

Tokenization Fragmentation
Structure Flattening

Graph Transformer: Mind Map for AI

The answer: a three-part pipeline that preserves structure throughout.

Part 1: Parse User Prompts into Structured Graphs

Part 2: Parse Code as Abstract Syntax Trees

Part 3: Convert ACE prompt into code during inference

During inference, instead of predicting tokens sequentially, graph transformers predict nodes structurally. The model asks "what operation does sqrt take as input?" not "what word comes after 'sqrt'?" This reduces the accuracy loss that happens when rebuilding structure as hidden states. Parse tree makes the structure explicit and avoids loss:

Why This Solves the Accuracy Problem

Beyond Metadata: The Path to AGI

This is similar to how we move up the value chain from user to developer to language creator. As users, we use calculators (data tools) to solve specific problems like finding ladder distance. When we learn to code, we abstract these tasks into code (metadata), using programming languages like python. Once we are very good at coding, we dream about creating new language like python using parser generators like PEG (meta-metadata).

If we revise OpenAI’s levels framework using our meta framework, we get:

Before LLMs, we doubted if AI could ever create art or poetry that understands human taste. Yet here we are. Perhaps the same will happen for programming languages and theorems. After all, intelligence isn't just about solving problems. It's about discovering the hidden rules that create them. And maybe, just maybe, it will find the hidden rules across meta-metadata like pythagoren theorem, python, Go that we haven't discovered yet. And that will usher the age of discovery with AGI.

你的個人知識庫