Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,4 @@ lib
TODO.md
package-lock.json
.vscode
BASELINE.md
115 changes: 115 additions & 0 deletions BASELINE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
# Generated model migration baseline

Recorded on 2026-07-12 at commit
`df1e8dd0088f5d4b71790a54ee2fc1f2afe6cb0a` (`v3`), using Node.js v24.10.0
and npm 11.6.1.

`TODO.md` is locally present but ignored by the repository. There were no
tracked or unignored changes when this baseline was recorded. Generated `lib/`
and coverage output are also ignored.

## Checks

| Check | Result |
| --------------------------------- | ------------------------------------------------------------------ |
| `npm test` | Pass: 8 files, 48 tests passed, 8 todo |
| TypeScript | Pass |
| Coverage | 93.04% statements, 85.44% branches, 95.86% functions, 93.13% lines |
| `npm run models:verify` | Pass; models and evaluation are reproducible |
| `npm run validate -- --split=all` | Completes; 0/138 encoding and language predictions correct |
| `npm run benchmark` | 7,848.13 files/sec; 1.42 MiB/sec |
| `npm run build` | Pass |
| Built `lib/` | 341,661 bytes across files; 448 KiB allocated on disk |
| `npm pack --dry-run` | 42,359 bytes packed; 348,964 bytes unpacked; 42 entries |

The benchmark processed 13,800 calls and 2,622,400 bytes in 1,758.38 ms,
producing 100,200 matches. These figures are a local performance reference, not
a hard CI threshold; comparisons should use the same machine and runtime.

## Runtime corpus validation

The validation corpus currently contains only newly generated models that are
not registered with the runtime. The 0% result is therefore the expected
pre-migration baseline, not a model-evaluator failure. Every input still
received at least one prediction.

### By split

| Split | Files | Encoding correct | Language correct |
| ---------- | ------: | ---------------: | ---------------: |
| train | 92 | 0 (0.00%) | 0 (0.00%) |
| validation | 23 | 0 (0.00%) | 0 (0.00%) |
| test | 23 | 0 (0.00%) | 0 (0.00%) |
| **Total** | **138** | **0 (0.00%)** | **0 (0.00%)** |

### By expected encoding

| Expected encoding | Files | Encoding correct | Most common current prediction |
| ----------------- | ----: | ---------------: | ------------------------------ |
| CP850 | 18 | 0 | windows-1252 |
| CP852 | 6 | 0 | windows-1250 |
| CP949 | 6 | 0 | EUC-KR |
| IBM855 | 12 | 0 | windows-1252 |
| IBM866 | 6 | 0 | windows-1252 |
| ISO-8859-10 | 6 | 0 | ISO-8859-1 |
| ISO-8859-13 | 6 | 0 | windows-1257 |
| ISO-8859-14 | 6 | 0 | ISO-8859-1 |
| ISO-8859-15 | 18 | 0 | ISO-8859-1 |
| ISO-8859-16 | 6 | 0 | ISO-8859-2 |
| ISO-8859-3 | 6 | 0 | ISO-8859-1 |
| ISO-8859-4 | 6 | 0 | windows-1257 |
| KOI8-U | 6 | 0 | KOI8-R |
| macintosh | 18 | 0 | windows-1252 |
| x-mac-cyrillic | 12 | 0 | windows-1251 |

The most common predictions above are useful collision targets, but do not
describe every document. Notably, weak IBM855 input can also be classified as
Shift_JIS or windows-1255.

## Public naming decisions

Existing names remain exactly as declared by `EncodingName`; the migration must
not silently recase or rename them. The new SBCS canonical result names are:

```text
KOI8-U
IBM866
IBM855
macintosh
x-mac-cyrillic
ISO-8859-3
ISO-8859-4
ISO-8859-10
ISO-8859-13
ISO-8859-14
ISO-8859-15
ISO-8859-16
CP850
CP852
```

CP949 is reserved as the canonical future result name for the separate
multibyte follow-up.

The current public API does not accept encoding-name filters and does not expose
alias normalization. Consequently, the initial migration adds only these
canonical `EncodingName` values. Iconv/compiler input labels such as `CP866`,
`CP855`, `MACINTOSH`, and `MAC-CYRILLIC` are implementation aliases and must not
be returned by `detect` or `analyse`.

## Migration scope decision

All existing SBCS recognizers will move to generated models. The legacy byte
maps and n-gram tables in `src/encoding/sbcs.ts` will not remain as a parallel
model system. Existing encodings must first be added to the corpus/model inputs,
generated, and validated before their hard-coded tables are removed.

## Known pre-migration limitations

- Generated SBCS models are not consumed or registered by the detector.
- New encodings therefore lose to the closest existing recognizer.
- Confidence values are not yet calibrated between generated SBCS and existing
UTF, ISO-2022, or MBCS recognizers.
- The existing test suite has eight explicitly pending tests.
- CP949 is represented in the corpus but intentionally excluded from the SBCS
model compiler and requires separate MBCS work.
15 changes: 10 additions & 5 deletions corpus/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,9 @@ This directory contains a deterministic seed corpus for developing additional
encoding recognisers. The prose is synthetic and should not be treated as a
replacement for a large, independently sourced natural-language corpus.

`sources.json` stores UTF-8 originals. `manifest.json` maps languages to iconv
encodings. Generated byte samples and statistics live under `generated/`.
`sources.json` stores UTF-8 originals. Held-out material is kept separately in
`test-sources.json`. `manifest.json` maps languages to iconv encodings.
Generated byte samples and statistics live under `generated/`.

Build and verify the corpus with:

Expand All @@ -17,8 +18,9 @@ npm run models:evaluate
npm run models:verify
```

The build script generates the corpus and performs an exact iconv round trip for
every document. The check script independently rebuilds into a temporary
The build script generates the corpus and performs an iconv round trip for every
document, allowing canonically equivalent Unicode composition where an encoding
such as windows-1258 stores combining marks. The check script independently rebuilds into a temporary
directory and byte-compares the result with `generated/`. Four documents per
language are assigned to training, one to validation, and one independent
document to testing. Statistics are calculated from training documents only.
Expand All @@ -37,6 +39,9 @@ learns Laplace-smoothed high-byte distributions from the training split. The
evaluator uses byte likelihood to rank candidates whose trigram hit rates are
statistically competitive at a 95% margin. `models:evaluate` reports every
held-out test document, while `models:verify` checks reproducibility and fails if
any encoding or language result is incorrect. CP949 is excluded because its
any encoding or language result is incorrect. A different encoding label is
accepted only when both encodings decode the actual held-out bytes identically;
the report distinguishes these byte-equivalent results from exact matches.
CP949 is excluded because its
multibyte recogniser requires character frequency data rather than byte
trigrams.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/da/test/community-garden.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
F�lleshaven

Naboerne gjorde en tom g�rd til en gr�n have, b�rnene malede skilte, og familierne vander planterne hver uge.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/da/train/city-morning.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Morgen i byen

Byen v�gner langsomt, caf�erne �bner, og folk venter p� de f�rste busser.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/da/train/library.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Det stille bibliotek

L�sere finder b�ger om historie og videnskab, mens bibliotekaren hj�lper eleverne.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/da/train/market.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
L�rdagsmarked

B�nder s�lger frisk br�d, �bler, ost og honning p� den travle plads.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/da/train/workshop.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Digitalt v�rksted

Et lille hold bygger en museumsguide, unders�ger koden og skriver tydelige tekster.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/da/validation/river-trip.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Tur til floden

Vennerne tog et kort og varm te med p� en s�ndag og hvilede bagefter ved floden.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/de/test/community-garden.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Der Gemeinschaftsgarten

Die Nachbarn haben einen leeren Hof begr�nt und dort Kr�uter, Gem�se und junge B�ume gepflanzt. Die Sch�ler malten bunte Schilder, w�hrend �ltere Bewohner regelm��ig gie�en und Ger�te aufr�umen.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/de/train/city-morning.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Morgen in der Stadt

Die Stadt erwacht langsam. B�ckereien �ffnen ihre T�ren, Busse beginnen ihre Fahrt und viele Menschen lesen unterwegs die Nachrichten. �ber den D�chern wird der Himmel allm�hlich heller.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/de/train/library.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Die ruhige Bibliothek

In der kleinen Bibliothek rascheln leise die Seiten. Besucher suchen B�cher �ber Geschichte, Technik und ferne L�nder. Eine Mitarbeiterin bereitet eine Ausstellung f�r Sch�ler vor.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/de/train/market.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Markt am Samstag

Auf dem Platz bieten Bauern �pfel, K�se, Honig und frisches Brot an. Die Kunden vergleichen Gem�se, sprechen �ber die Ernte und tauschen einfache Rezepte aus.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/de/train/workshop.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Digitale Werkstatt

Ein kleines Team entwickelt einen digitalen Museumsf�hrer. Designer ordnen das Men�, Entwickler pr�fen den Code und Redakteure schreiben verst�ndliche Texte f�r G�ste.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/de/validation/river-trip.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Ausflug an den Fluss

Am Sonntag wandern Freunde zu einem ruhigen Fluss. Sie nehmen eine Karte, hei�en Tee und eine Kamera mit. Nach dem langen Weg sitzen sie am Ufer und h�ren den V�geln zu.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/en/test/community-garden.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
The community garden

Neighbours turned an empty courtyard into a garden with flowers, vegetables, young trees, and a small caf� table.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/en/train/city-morning.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Morning in the city

The city wakes slowly; caf�s open their doors and commuters read news beside a fa�ade.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/en/train/library.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
The quiet library

Readers browse books about history, science, and na�ve art while the librarian prepares a r�sum�.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/en/train/market.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Saturday market

Farmers sell bread, apples, jalape�os, and p�t� while neighbours exchange recipes.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/en/train/workshop.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Digital workshop

A small team designs the museum guide and discusses its r�le, code, and clear descriptions.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/en/validation/river-trip.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Journey to the river

Friends carried a map and a thermos, walked past the caf�, and rested by the river.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/es/test/community-garden.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
El jard�n comunitario

Los vecinos convirtieron un patio vac�o en un jard�n lleno de �rboles, hierbas y verduras. Los ni�os pintaron carteles de colores y cada s�bado varias familias riegan las plantas.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/es/train/city-morning.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
La ma�ana en la ciudad

La ciudad despierta poco a poco. Las cafeter�as abren sus puertas, los autobuses inician la ruta y los vecinos leen las noticias. El cielo se vuelve m�s claro sobre los tejados.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/es/train/library.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Una biblioteca tranquila

En la biblioteca del barrio se oyen p�ginas y pasos suaves. Los lectores buscan libros de historia, ciencia y viajes. La bibliotecaria prepara una exposici�n para los alumnos.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/es/train/market.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
El mercado del s�bado

El mercado llena la plaza de colores. Los agricultores venden manzanas, miel, queso y pan reci�n hecho. Los clientes comparan las verduras y comparten recetas de sus familias.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/es/train/workshop.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Taller digital

Un equipo peque�o crea el cat�logo digital del museo. Los dise�adores ordenan el men�, los programadores revisan el c�digo y los editores escriben textos claros para el p�blico.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/es/validation/river-trip.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Paseo junto al r�o

El domingo unos amigos caminan hacia un r�o tranquilo. Llevan un mapa, t� caliente y una c�mara. Despu�s del paseo descansan en la orilla y escuchan a los p�jaros.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/fr/test/community-garden.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Le jardin partag�

Les habitants ont transform� une cour oubli�e en jardin fleuri. Les enfants ont pr�par� des �tiquettes color�es, tandis que les voisins se r�unissent chaque semaine pour arroser les l�gumes et les jeunes arbres.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/fr/train/city-morning.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Le matin en ville

La ville se r�veille lentement. Les caf�s ouvrent leurs portes, les autobus commencent leur trajet et les passants consultent les nouvelles. Chacun pr�pare sa journ�e sous un ciel encore p�le.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/fr/train/library.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
La biblioth�que calme

Dans la biblioth�que du quartier, les pages tournent en silence. Les lecteurs cherchent des livres sur l'histoire, les sciences et les voyages. Une biblioth�caire pr�pare une exposition pour les �l�ves.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/fr/train/market.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Le march� du samedi

Le march� remplit la place de couleurs et de parfums. Les producteurs proposent des pommes, du miel, du fromage et du pain frais. Les clients discutent des saisons et �changent des recettes familiales.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/fr/train/workshop.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Un atelier num�rique

Une petite �quipe construit le catalogue num�rique du mus�e. Les graphistes organisent le menu, les d�veloppeurs v�rifient le code et les r�dacteurs pr�parent des textes clairs pour le public.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/fr/validation/river-trip.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Une promenade pr�s de la rivi�re

Dimanche, des amis partent vers une rivi�re tranquille. Ils emportent une carte, du th� chaud et un appareil photo. Apr�s une longue marche, ils se reposent sur la rive et �coutent les oiseaux.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/it/test/community-garden.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Il giardino comune

I vicini hanno trasformato un cortile vuoto in un giardino della citt�, i bambini hanno dipinto cartelli e le famiglie annaffiano le piante.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/it/train/city-morning.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Mattina in citt�

La citt� si sveglia lentamente, i caff� aprono e la gente aspetta i primi autobus.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/it/train/library.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
La biblioteca tranquilla

I lettori cercano pi� libri di storia, geografia e scienza mentre la bibliotecaria aiuta gli studenti pi� giovani.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/it/train/market.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Mercato del sabato

Gli agricoltori vendono pane fresco, caff�, mele, formaggio e miele nella piazza della citt�.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/it/train/workshop.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Laboratorio digitale

Una piccola squadra prepara la guida del museo, controlla il codice e scrive descrizioni chiare perch� siano utili a pi� persone.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/it/validation/river-trip.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Gita al fiume

Gli amici portano una mappa e t� caldo, camminano nel bosco e riposano sulla riva.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/nl/test/community-garden.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
De buurttuin

Buren veranderden een lege binnenplaats naast een caf� in een groene tuin, kinderen schilderden borden en gezinnen geven de planten water.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/nl/train/city-morning.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Ochtend in de stad

De stad ontwaakt langzaam, caf�s openen hun deuren en mensen wachten op de eerste bussen.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/nl/train/library.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
De stille bibliotheek

Lezers zoeken boeken over geschiedenis en wetenschap terwijl de bibliothecaris ge�nteresseerde leerlingen en ��n leraar helpt.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/nl/train/market.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Zaterdagmarkt

Boeren verkopen vers brood, appels, kaas en honing naast twee caf�s op het drukke plein.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/nl/train/workshop.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Digitale werkplaats

Een klein team maakt een museumgids, controleert de code en schrijft duidelijke idee�n over po�zie en teksten.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/nl/validation/river-trip.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Reis naar de rivier

Vrienden namen een kaart en warme thee mee en rustten na ��n lange wandeling bij de rivier.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/no/test/community-garden.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Felleshagen

Naboene gjorde en tom g�rd til en gr�nn hage, barna malte skilt og familiene vanner plantene hver uke.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/no/train/city-morning.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Morgen i byen

Byen v�kner langsomt, kafeene �pner d�rene og folk venter p� de f�rste bussene n�r bryggen og sj�en.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/no/train/library.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Det stille biblioteket

Lesere finner b�ker om historie, �rlige fortellinger og vitenskap mens bibliotekaren hjelper elevene.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/no/train/market.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
L�rdagsmarked

B�nder selger ferskt br�d, epler, ost og honning p� det travle torget.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/no/train/workshop.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Digitalt verksted

Et lite lag lager en museumsguide, unders�ker koden og skriver tydelige tekster.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/no/validation/river-trip.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Tur til elven

Vennene tok med kart og varm te p� s�ndag og hvilte senere ved elven.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/pt/test/community-garden.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
A horta comunit�ria

Os vizinhos transformaram um p�tio vazio em jardim, as crian�as pintaram placas e as fam�lias regam as plantas toda semana.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/pt/train/city-morning.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Manh� na cidade

A cidade acorda devagar, os caf�s abrem e as pessoas esperam os primeiros �nibus.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/pt/train/library.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
A biblioteca tranquila

Os leitores procuram livros de hist�ria e ci�ncia enquanto a bibliotec�ria ajuda os alunos.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/pt/train/market.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Mercado de s�bado

Os agricultores vendem p�o fresco, ma��s, queijo e mel na pra�a movimentada.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/pt/train/workshop.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Oficina digital

Uma pequena equipe prepara o guia do museu, verifica o c�digo e escreve descri��es claras.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/pt/validation/river-trip.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Viagem ao rio

Os amigos levaram um mapa e ch� quente, caminharam pela mata e descansaram junto ao rio.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/sv/test/community-garden.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Gemensam tr�dg�rd

Grannarna gjorde en tom g�rd till en gr�n tr�dg�rd, barnen m�lade skyltar och familjerna vattnar v�xterna varje vecka.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/sv/train/city-morning.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Morgon i staden

Staden vaknar l�ngsamt, kaf�erna �ppnar och m�nniskor v�ntar p� de f�rsta bussarna.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/sv/train/library.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Det tysta biblioteket

L�sare s�ker b�cker om historia och vetenskap medan bibliotekarien hj�lper eleverna.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/sv/train/market.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
L�rdagsmarknad

B�nder s�ljer f�rskt br�d, �pplen, ost och honung p� det livliga torget.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/sv/train/workshop.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Digital verkstad

Ett litet lag skapar en museiguide, granskar koden och skriver tydliga texter f�r bes�kare.
3 changes: 3 additions & 0 deletions corpus/generated/ISO-8859-1/sv/validation/river-trip.bin
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Utflykt till floden

V�nnerna tog med karta och varmt te och vilade sedan vid floden.
Loading