ChatGPTがGodot 3のコードばかり出すのはなぜ?150件の回答で検証 #AI - Qiita
@Ziva(Engine Ziva)

Why Does ChatGPT Keep Outputting Godot 3 Code? Verified Across 150 Responses

  • Godot
  • ChatGPT
  • GDScript
  • AI
  • Game Development
0
Posted at 2026-11-11

https://ziva.sh/blogs/chatgpt-godot-3-code is the Japanese version of this article (the original English version was published by Ziva.sh on November 11, 2026).

The reason ChatGPT outputs Godot 3 code is that the model behind it is confusing the two versions.Godot 4.0 was released on March 1, 2023, and many Godot 3 classes, methods, and keywords were renamed or removed, so KinematicBody2D, export var, and yield can no longer be used. The current model may no longer make this mistake. In November 2026, we asked GPT-6 Luna, then OpenAI’s current model, ten beginner-level Godot questions in regular chat. We tested this by writing “Godot 4” in the prompt, omitting the version, and enabling web search. Across 150 responses, there were zero Godot 3 API errors. Instead, a different kind of problem occurred: a few parse errors and a few runtime-only bugs that the Godot parser accepts. Even so, if Godot 3 code still appears, use the current model, specify the Godot version, and verify the code in Godot.

Ziva is an AI agent for Godot developed by Ziva.sh, and Ziva.sh also conducted this test.

All responses were retrieved on November 10, 2026 (UTC), and the code was checked with Godot 4.7.2. GPT-6 Luna was invoked through OpenAI’s Codex endpoint while signed in with a ChatGPT account, rather than through the ChatGPT app. Behavior may differ in the ChatGPT app. We also ran the same prompts with Ziva using the same model. The results are as described below.

Where Do the Complaints Come From?

Only the first entry specifies an exact model version, and that model is GPT-4o. The models used in the other posts are unknown, and these results do not allow us to say anything about those models.

Testing methodology

Based on the forum thread, I wrote ten prompts for beginners. They cover a 2D platformer player, a third-person controller, spawning enemies with a timer, saving high scores, a Start button that switches scenes, fading out coins, placing tiles from code, an HP bar linked to an exported variable, random numbers, and a trap that disappears after two seconds. None of the prompts include the names of APIs that changed between Godot 3 and 4. Example: “I'm making my first 2D platformer in Godot 4. Can you write me a player script so my character can run left and right, jump, and fall with gravity?” (“I'm making my first 2D platformer in Godot 4. Can you write me a player script so my character can run left and right, jump, and fall with gravity?”)

In regular chat, each prompt was sent three times per setting. There was no project and no system prompt. The exception was the web-search setting, for which I added a short instruction to answer based on web search results. An AI agent read all the code blocks and then scanned the scripts against a checklist of Godot 3 conventions. This checklist was created from the official upgrade guide and the table mapping renamed APIs in Godot 4.7.2's own upgrade tool. Because the reading and scanning were performed by the same agent, the reading was not blinded to the scan results. All scripts also passed Godot 4.7.2's `--check-only`. All the runtime errors below were verified by running the code headlessly in Godot 4.7.2.

Results

Setting Godot 3 API errors Other actual errors
GPT-6 Luna regular chat, prompt includes “Godot 4,” reasoning set to default GPT-6 Luna regular chat: 0 out of 30 GPT-6 Luna regular chat: 3 responses (2 parse errors, 1 runtime error)
GPT-6 Luna regular chat, prompt includes “Godot 4,” reasoning set to low GPT-6 Luna regular chat, reasoning low: 0 out of 30 GPT-6 Luna regular chat, reasoning low: none
GPT-6 Luna regular chat, prompt includes only “Godot” GPT-6 Luna regular chat, no version: 0 out of 30 GPT-6 Luna regular chat, no version: 2 responses (1 runtime error, 1 instructed to connect a signal that does not exist on StaticBody2D)
GPT-6 Luna with web search, prompt includes “Godot 4” GPT-6 Luna with web search: 0 out of 30 GPT-6 Luna with web search: 2 responses (1 runtime error, 1 incorrect setup procedure)
GPT-6 Luna with web search, prompt includes only “Godot” GPT-6 Luna with web search, no version: 0 out of 30 GPT-6 Luna with web search, no version: 2 responses (both parse errors)
Ziva running on GPT-6 Luna, prompt in the same question format Ziva running on GPT-6 Luna: 0 out of 10 chats Ziva running on GPT-6 Luna: 2 chats (1 parse error in the code within the chat reply, 1 runtime error)
Ziva running on GPT-6 Luna, prompt instructing creation (“Add the script to my project.”) Ziva running on GPT-6 Luna: 0 out of 10 chats Ziva running on GPT-6 Luna: 1 chat (runtime error)
Ziva running on GPT-6 Luna, prompt instructing creation, “Godot” only Ziva running on GPT-6 Luna: 0 out of 10 chats Ziva running on GPT-6 Luna: 2 chats (1 runtime error, 1 instruction to break the script)

The code was written throughout in the style of Godot 4.@export, @onready, a CharacterBody2D that uses the velocity property and move_and_slide() without arguments, instantiate(), pressed.connect(...), create_tween(), change_scene_to_file(), TileMapLayer.set_cell(), randi_range(), await ... .timeout.

When the prompt included no version and no search was performed, 18 of the 30 regular chat responses stated that they were for Godot 4, while the rest did not mention a version. None were for Godot 3. When web search was enabled, every search performed by GPT-6 Luna began with site:docs.godotengine.org/en/stable. This was the model’s own choice. Pages from the Godot 3 and earlier documentation appeared in the search results for 23 of the 60 responses, but no response cited them, and none of those 23 responses contained Godot 3 code.

Zero cases are not proof that something will “never happen.” If all responses were independent, the result of 0 out of 150 would indicate an occurrence rate of less than approximately 2% at 95% confidence. In reality, responses to the same prompt are not independent, so the actual uncertainty is greater.

The problem that occurred instead

Seven of the 150 regular chat responses contained code that did not work in Godot 4.7.2. Four failed to parse, and three parsed but crashed when run. The other two gave incorrect setup instructions. Here are three examples.

1. Turning off an Area2D inside its own signal. The following line appeared in five locations across regular chat and Ziva for the coin and trap prompt.

func _on_body_entered_body: Node2D) -> void: collected = true monitoring = false

--check-only accepts this. When the player touches the coin, Godot 4.7.2 outputs the following, and monitoring remains true.

ERROR: Function blocked during in/out signal. Use set_deferred("monitoring", true/false).

Use set_deferred("monitoring", false), as in several other responses.

2. A setter that accesses a node that does not yet exist. One of the answers that used a web search wrote the following HP bar script.

@export var health: float = 100.0: set(value): health = clampf(value, 0.0, max_health) $ProgressBar.max_value = max_health $ProgressBar.value = health

This passes parsing. If you set Health to 50 in the Inspector and run the scene, the stored value is applied before the child ProgressBar is added. Therefore, the setter fails, and the bar displays 0 until health changes again in the code.

ERROR: Node not found: "ProgressBar" (relative to "Node"). SCRIPT ERROR: Invalid assignment of property or key 'max_value' with value of type 'float' on a base object of type 'null instance'.

Wrap the two $ProgressBar lines with if is_node_ready(): and update the bar once in _ready().

3. A type that Godot cannot infer. Four answers failed to parse, three of which were caused by an untyped loop variable in tile-placement code like the following.

for direction in [Vector2i.UP, Vector2i.DOWN, Vector2i.LEFT, Vector2i.RIGHT]: var neighbor := cell + direction
Parse Error: Cannot infer the type of "neighbor" variable because the value doesn't have a set type.

Using `for direction: Vector2i in [...]` fixes it. Unlike the first two, this is detected by `--check-only`.

What Godot's parser finds and what it doesn't

Since there was no answer using Godot 3 code, this list is the result of Ziva.sh personally verifying each piece of Godot 3 syntax one by one by running it in Godot 4.7.2.

Godot 4.7.2's `--check-only` rejects the following, so they are detected during the parse check.

  • export var、onready var、tool、setget、yield、remote func
  • button.connect("pressed", self, "_on_pressed"):「Invalid argument for "connect()" function: argument 2 should be "Callable" but is ...」(The argument to the `connect()` function is invalid. The second argument must be a `Callable`, but it is ...)
  • File.new()、Directory.new()、OS.get_ticks_msec()、rand_range()、update()、BUTTON_LEFT
  • `.empty()` on a typed `Array`
  • move_and_slide(velocity, Vector2.UP) and velocity = move_and_slide()
  • Classes such as `KinematicBody2D`, `Spatial`, `Position2D`, and `PoolStringArray`
  • The script’s own node’s rect_position and translation

The following pass Godot 4.7.2’s --check-only and fail only when the corresponding line is executed.

  • scene.instance(): “Invalid call. Nonexistent function 'instance' in base 'PackedScene'.” (Invalid call. PackedScene has no function named instance.)
  • get_tree().change_scene() and change_scene_to()
  • Tween.new() and the Tween node’s interpolate_property(): “Tween can't be created directly. Use create_tween() method.” (A Tween cannot be created directly. Use the create_tween() method.)
  • Replacing yield(get_tree(), "idle_frame") with await get_tree().idle_frame
  • TileMap’s set_cellv() and world_to_map()
  • rect_position and .empty() on untyped variables

Checking AI-written GDScript one line at a time

Parse and analyze a single script without opening the editor.

godot --headless --path /path/to/your/project --check-only --script res://player.gd

It outputs SCRIPT ERROR: Parse Error: ... along with the relevant line and exits with code 1. Scripts with no issues exit with code 0. Replace godot with the path to the executable for your Godot 4.

Check the items in the list that fail only at runtime by running the scene. In the editor, press F6 and read the Output and Debugger panels. In headless mode, use the following:

godot --headless --path /path/to/your/project --quit-after 120 res://main.tscn

--quit-after 120 stops after 120 frames. Look for the SCRIPT ERROR line in the output. In this run, the scene that calls change_scene() output “Invalid call. Nonexistent function 'change_scene' in base 'SceneTree'.” (Invalid call. SceneTree has no function named change_scene), and Godot still exited with code 0.

Even if Godot 3 code still appears

  • Use the current model. The complaints above that mention model names concern GPT-4o from 2024, and GPT-6 Luna did not write a single instance of Godot 3 code in this test. We did not run a test comparing older models side by side.
  • Tell it the exact version. Say something like “Godot 4.7.” No difference was observed with GPT-6 Luna, but the chat model cannot see the project.
  • Check the code in Godot. Check it using the two commands above before building on top of that code.
  • Give the agent a mapping table for the renamed APIs. The free, MIT-licensed godot-skills pack published by Ziva.sh includes the godot-4-api-pitfalls skill, which contains a conversion table for rewriting Godot 3 to 4 and a list of Godot 3 lines that pass --check-only but fail at runtime. It can be used with agents that load the Agent Skills format. For details, see Godot AI Skills (in English).

Ziva’s positioning

Ziva is an AI agent within the Godot editor developed by Ziva.sh. It is proprietary and offers a free Hobby plan. We ran the same 10 prompts in Ziva using the same GPT-6 Luna model. There was one chat per prompt, in three sets of 10. Ziva also had zero Godot 3 API errors.

  • When asked a question, Ziva usually replied in the chat. With the prompt above (“How do I ...? Please write the script.”, or “How do I ...? Please write the script.”), Ziva wrote a file in only 1 of the 10 chats and replied in the chat in the remaining 9. Ziva does not check code included in chat replies.
  • When instructed to create it, Ziva wrote the file and checked it. When the same prompt was reworded as an instruction (“Add the script to my project.”, or “Add the script to my project”), Ziva wrote a .gd file in all 20 chats. Ziva ran a check after every write, and all 21 writes in the chats where creation was requested returned “No problems in this file or its callers.” (“No problems in this file or its callers.”)
  • The check is a parse, not an execution. It is the same kind of check as --check-only, performed by Godot’s GDScript language server and by reloading the script in the editor. Two of the files written that way contained the monitoring = false bug from Example 1, but they still returned “No problems” (“No problems”).In a separate check, a script containing instance(), change_scene(), Tween.new(), and set_cellv() also returned “No problems.”

In other words, this test could not confirm a case where Ziva’s checks would find a Godot 3 error. There were no errors that should have been found, and these checks do not detect errors that occur only at runtime. Runtime-only errors can be found by running the scene and reading the output. Ziva can also run scenes, but scene execution was not included in this test.

Limitations

  • Only one model, GPT-6 Luna, was tested, and testing covered only one day. Older models, other providers, and the ChatGPT app were not tested.
  • OpenAI’s Codex endpoint was used for regular chat, and its search tool was used for web searches, with instructions to answer based on the web search results. Because the model searched only Godot’s documentation, old forum answers never appeared.
  • There were ten prompts written by Ziva.sh. In each regular-chat configuration, three answers were generated per prompt; in each Ziva set, there was one chat per prompt.
  • The Ziva prompts in the form “add the script to my project” were added after watching Ziva answer a question-style prompt in chat, so the two Ziva sets were not the planned comparison. The versionless set and the web-search set were also added after seeing the initial results. The contents of both were recorded before execution.
  • Ziva was run with the reasoning setting set to low, while regular chat was left at its default. The regular-chat set with low reasoning made up for this difference. Ziva tells the model the Godot version in its own instructions, so the version was also included in Ziva’s “Godot”-only set.
  • The code was read by an AI agent, not a human, and the same agent performed the checklist scan, so the code-reading task was not blind to the scan results.
  • In the Ziva run that was instructed to create the code, the language server port of the second editor was changed. Two Godot editors were open on the test machine; otherwise, Ziva’s check would query the other editor. This actually happened with one file written in the question-style set. It was reported as a Ziva bug.

Related Articles

Frequently Asked Questions

Why does ChatGPT keep outputting Godot 3 code even though I'm using Godot 4?

In Godot 4.0 (March 2023), many Godot 3 classes, methods, and keywords were renamed or removed, so models that confuse the two versions write code that Godot 4 does not accept. Complaints on forums do exist, and among those I found, the ones that specify a particular model version concern GPT-4o from October 2024. In a November 2026 test, across 150 regular-chat responses to 10 beginner-oriented Godot prompts, GPT-6 Luna had zero Godot 3 API errors. This included cases where the prompt specified Godot 4, omitted the version, or enabled web search. If Godot 3 code still appears, use the current model, tell it which version of Godot you're using, and verify the code in Godot.

Does writing “Godot 4” in the prompt help?

In this test, there was no measurable difference for GPT-6 Luna. There were zero Godot 3 API errors in 30 responses when the prompt specified Godot 4, and also zero in 30 responses when it said only Godot. When no version was given, 18 of the 30 responses stated that they were for Godot 4, and none were for Godot 3. Even so, there is no downside to specifying the version, and regular-chat models have no way to know which version of Godot you're using except by being told.

Does ChatGPT write Godot 3 code when web search is enabled?

In this test, it did not. GPT-6 Luna with web search enabled had zero Godot 3 API errors across 60 responses. All searches were restricted to Godot's documentation site. Pages from documentation for Godot 3 and earlier were returned in 23 of the 60 responses, but no response cited them. I used the search tool of OpenAI's Codex endpoint rather than the ChatGPT app, and forum and Reddit pages were never returned, so this test does not validate older forum answers.

How can I check whether AI-written GDScript is correct Godot 4 code?

In Godot 4, run `godot --headless --path /path/to/project --check-only --script res://player.gd`. If there are removed keywords such as `export`, `onready`, and `yield`; renamed classes such as `KinematicBody2D`; or an old-style `connect()`, it will print a Parse Error line and exit with code 1. Some Godot 3 code will pass this check and fail only when that line is executed (such as `instance()`, `change_scene()`, and `Tween.new()`), so run the scene as well and read the output.

What Godot 3 code can Godot 4 detect only at runtime?

In Godot 4.7.2, the following passes `--check-only` and fails only when that line is executed.PackedScene.instance(), get_tree().change_scene() and change_scene_to(), Tween.new() and interpolate_property on a Tween node, the idle_frame signal, TileMap’s set_cellv and world_to_map, and rect_position and .empty() on untyped variables.

If this was not Godot 3 code, what mistakes did GPT-6 Luna make?

Of 150 regular chat responses, 4 had parse errors (for example, var neighbor := cell + direction. Godot 4.7.2 cannot infer this type), and 3 had runtime-only bugs that passed the parser: one that sets monitoring = false inside an Area2D’s body_entered handler (Godot blocks this), and a health setter that accesses $ProgressBar before the node exists. The other 2 gave incorrect setup instructions.

Can Ziva prevent Godot 3 code?

This test cannot show that. Ziva, running the same GPT-6 Luna model, had zero Godot 3 API errors across 30 chats, so there was nothing for the checks to find. In the chats where it was instructed to create code, Ziva’s checks (Godot’s GDScript language server and the editor’s script reload) returned “No problems” on all 21 writes. Two of those scripts contained runtime-only bugs. This is because the checks parse and analyze without executing the scripts.

Original version of this article (English):https://ziva.sh/blogs/chatgpt-godot-3-code

0
0
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin