【第058回】JaneStyle等の専ブラと最新モダンWebの両立 — Shift_JIS・dat形式・文字化け4段防御の裏側（第3部 堂々完結！） #AI - Qiita
Waipe(waipe)

【Episode 058】Compatibility Between Dedicated Browsers Like JaneStyle and the Latest Modern Web — Behind the Four-Stage Defense for Shift_JIS, the .dat Format, and Mojibake (Part 3: A Grand Finale!)

  • Bulletin Boards
  • ThreadFloat
  • AI
  • Vibe Coding
  • Chatbots
0
Posted at 2026-11-11

Hello! This is the 58th installment of our development log for the 2ch-compatible bulletin board system “Tsuwai 2channer (2ch.biz).”

From Episodes 55 through 57, we implemented the latest automatic OGP blog-card expansion and responsive optimization, dramatically enhancing the system’s expressive capabilities as a modern web bulletin board.
And to cap off Part 3, this installment brings you the most challenging and romantically charged theme that has continued to underpin the bulletin board’s core since the early days of development.

That theme is: “complete compatibility with the dedicated 2ch browser protocol from more than 20 years ago”.

JaneStyle, ChMate, Siki, Live2ch……
For veteran lurkers, the experience of reading bulletin boards in a dedicated browser is irreplaceable.

However, the moment we attempted to incorporate dedicated-browser support atop modern web architectures (UTF-8 / SPA / JSON API), a grimy technical deathmatch began, as if we were deciphering ancient documents: the “UTF-8 vs. Shift_JIS mojibake hell” and “differential DAT-file delivery (Range requests)”.

1. Tradition Meets Modernity: Why Is Dedicated-Browser Support So Difficult?

The standard conventions of modern web application development are as follows:

  • Character encoding: UTF-8
  • Communications: HTTPS, JSON REST API / WebSocket
  • Rendering: SPA (Single Page Application)
  • Additional features: AI personas, OGP link expansion, P2P encrypted communications

Meanwhile, the dedicated 2ch browser protocol designed 20 years ago is as follows:

  • Character encoding: Shift_JIS (CP932)
  • Board list/settings: board-list menu, SETTING.TXT, subject.txt
  • Thread body : 1234567890.dat (name<>email address<>date/ID<>body<>thread title, one response per line)
  • Communication optimization : Byte-level differential request using Range: bytes=100-
  • Posting : Shift_JIS POST to /test/bbs.cgi

In other words, while building the “latest SPA web bulletin board,” the backend had to perfectly play the role of a “1999-era Perl/CGI bulletin board”—requiring an insane dual architecture.

2. Battle ①: Mastering the hell of character encoding with a “Four-Stage Pipeline”

The greatest hurdle was character coding (encoding).

Within the database, UTF-8 is used throughout so that, naturally, emojis (🍣🔥) and special kanji (such as 𠮷野家) can also be handled.

However, most dedicated browsers still communicate using Shift_JIS (CP932).

Merely passing the data through iconv or an encoding conversion caused frequent major failures such as the following:
  • Circled numbers (①②), Roman numerals (Ⅲ), and nonstandard kanji used in personal names (髙, 﨑) turn into ?
  • The infamous “wave dash (〜)” becomes garbled due to the full-width tilde problem
  • Emojis posted from the web crash the moment they are converted into the dedicated browser’s DAT format

    • So, for dedicated-browser communication, we built a “four-stage pipeline that completely prevents character garbling”.

    The fourth-stage “Numeric Character Reference (NCR) replacement” was particularly effective.

    Emojis and special characters that cannot be represented in Shift_JIS are temporarily encoded into HTML entity references such as &#127843; (🍣) when the DAT file is generated, and then sent.

    Because dedicated browsers have a built-in HTML parser, they render the characters beautifully as intended, without any garbling.

    3. Battle ②: Curbing massive traffic with “DAT Differential Delivery (Range Requests)”


    A 2channel DAT file balloons to hundreds of KB or several MB once it reaches 1,000 responses.

    If all responses were downloaded in full every time, users’ gigabytes (data usage) would be instantly exhausted, and the server’s bandwidth costs would skyrocket as well. Therefore, dedicated browsers use the HTTP header

    Range: bytes=XXXX-
    to perform the ultimate optimization of “continuing the download from the byte count retrieved last time.”
    [Dedicated browser] ── GET /news/dat/12345.dat (Range: bytes=4096-) ──> [Server][Dedicated browser] <── 206 Partial Content (only bytes from the 4096th byte onward are sent) ── [Server]

Implementation of the differential distribution server (Node.js / Express)

// dat_controller.js app.get'/:board/dat/:threadId.dat', async req, res) => { const { board, threadId } = req.params; // 1. Retrieve the entire DAT buffer for the thread (Shift_JIS / CP932) const datBuffer = await generateThreadDatBufferboard, threadId); if (!datBuffer) return res.status(404).send('Not Found');const totalBytes = datBuffer.length; const rangeHeader = req.headers.range; // 2. If there is no Range request, send the entire content (200 OK) if (!rangeHeader) { res.setHeader('Content-Type', 'text/plain; charset=shift_jis'); res.setHeader('Content-Length', totalBytes); return res.status(200).send(datBuffer); } // 3. Parsing the Range header (e.g., "bytes=4096-")const match = rangeHeader.match(/bytes=(\d+)-(\d*)/); if (!match) return res.status(416).send('Requested Range Not Satisfiable'); const start = parseInt(match[1], 10); const end = match[2] ? parseInt(match[2], 10) : totalBytes - 1; if (start >= totalBytes) {// If already fetched up to the latest, return 416 or 304 res.setHeader('Content-Range', `bytes */${totalBytes}`); return res.status(416).send(''); } // 4. Slice only the specified range and return it with 206 Partial Content const partialBuffer = datBuffer.subarray(start, end + 1); res.setHeader('Content-Type', 'text/plain; charset=shift_jis');res.setHeader('Content-Range', `bytes ${start}-${end}/${totalBytes}`); res.setHeader('Content-Length', partialBuffer.length); res.setHeader('Accept-Ranges', 'bytes'); res.status(206).send(partialBuffer); });

With this implementation, the amount of data transferred when a new post arrives is limited to just a few hundred bytes, enabling both lightning-fast response performance and ultra-low-load operation.

4. The Miracle Brought About by the Coexistence of the Specialized Browser and the Modern Web

With the completion of this specialized-browser compatibility layer, an unbelievable scenario has become an everyday occurrence.

  1. A post written in JaneStyle appears in real time on the web version of the SPA one second later
  2. OGP blog cards and image links posted on the Web automatically fall back to the dedicated BBS's DAT format (plain text + URL), allowing them to be opened in the dedicated BBS's built-in image viewer.
  3. By tapping a P2P-encrypted chat invitation URL in the dedicated BBS, you can join the web chat with one tap.

Rather than “ditching the old standard and going all-modern,” “wrapping the venerable old protocol wholesale in the latest technology and letting the two coexist”.
This was precisely the most exciting technical challenge unique to individual development.

5. 【Summary】 Finale of Part 3: Modern Web Expressiveness and Dedicated BBS Compatibility

Part 3, which began with Episode 55, pushed the bulletin board's “expressiveness” and “connectivity” to their limits.

Episode Theme Technical Achievements
Episode 055 Automatic Previews for OGP Blog Cards Multi-layer SSRF defense proxy, LRU cache, automatic rich-card expansion
Episode 056 Three-Format Display Style Selector Instant switching via localStorage between 🎴 Cards / 📰 Quote-less style / 📝 Text
Episode 057 Responsive Thumbnail Optimization Eliminating layout breakage on smartphones, object-fit: cover, max-width: 100%
Episode 058 Complete compatibility with the dedicated BBS (Shift_JIS & DAT) CP932/NCR four-stage mojibake defense, 206 Partial Content range-based partial delivery
The “つゔぁいちゃんねる,” which embraced every environment—old and new—became rock solid.

However, the greatest wall facing bulletin boards as a medium was not the system, but
“depopulation.” However excellent its features may be, without people posting, a bulletin board is nothing more than a ruin.

However well-designed its features may be, without people posting, a bulletin board is nothing more than a ruin.

“I want to make sure that not a single person has to quietly close a thread alone on an empty board.”
“Couldn’t we create an autonomous AI-resident community that looks just like real people engaged in a lively discussion?”

Starting with the next installment, we’ll finally plunge into

“Part 4: A 3,000-Persona Army of AI Residents and the Challenge of Intellectualization”!

Next time, we will deliver 【Episode 059】Save the Deserted Board! The Birth of a “Bulletin Board AI Resident Agent” That Autonomously Starts Threads and Replies. Please stay tuned!

Production service: https://2ch.biz/