Big release: a Cython scoring kernel joins mypyc in compiled wheels, every model retrained on a deduplicated corpus, UTF-7 fixed in both directions, and a guarantee that detect() never returns an encoding that can't decode your complete input.
Performance
Compiled wheels now score bigram profiles through a small Cython kernel alongside mypyc, and the pair is 4.7x faster than the pure wheel on CPython 3.14. _kernel.py stays plain Python (PyPy and pure wheels run it interpreted, unchanged), and detection output is bit-identical. The kernel declares itself safe without the GIL, so free-threaded CPython scales instead of silently re-enabling the GIL on import: 3.14t runs the whole suite in ~340ms across 8 threads, the fastest configuration measured. Compiled builds now need both hooks: HATCH_BUILD_HOOK_ENABLE_MYPYC=true HATCH_BUILD_HOOK_ENABLE_CUSTOM=true.
Added support for CPython 3.15, including the free-threaded build. No code changes were needed.
Bug Fixes
detect() no longer returns an encoding that cannot decode the input it was given (#380, thanks @yarikoptic). When the whole input has been examined and the winner's only multi-byte evidence is an incomplete trailing sequence, the best candidate that decodes the input completely wins instead. Genuinely truncated data keeps its answer.
Fixed delimited ASCII data like |NAME,+LAY| misdetecting as UTF-7 (#371 follow-up, thanks @agreenburg). The whole buffer must now actually decode as UTF-7, and a lone shifted character must land in a plausible script range.
Signed UTF-7 no longer reads as ASCII: the BOM stage recognizes the four UTF-7 signature prefixes when the rest of the buffer decodes as UTF-7.
Fixed short apostrophe-heavy English being labeled Scottish Gaelic or Breton: a rare-language label on an input under 128 bytes now needs a 0.03 lead over the best mainstream language (ADR-0005).
Fixed Hungarian text losing to a Czech reading in confusion rescoring; tied pairs are compared only under language models both encodings have.
Fixed space-padded text matching a degenerate Serbian model at high confidence; statistical scoring now skips repeated-whitespace bigrams. This also fixes windows-1251 logs misdetecting as windows-1250 (#379).
Fixed EBCDIC text being invisible to the early pipeline stages, and the last two EBCDIC sibling misdetections.
Fixed training normalization gaps that starved ISO-8859-16 and the 26 pre-euro encodings at exactly their distinguishing bytes.
Improvements
Retrained every bigram model on a refreshed, deduplicated corpus with training provenance now recorded per model.
New ANSI-art model for cp437, trained on 16,621 text-mode art files from 16colo.rs.
Rare-language arbitration (ADR-0005): low-confidence statistical winners from languages with no documented legacy-encoding population yield to near-tied mainstream candidates.
Confusion-group resolution is context-aware: per-occurrence votes, word-shape demotions, art-model exemption.
Statistical dead heats no longer resolve by candidate enumeration order.
Training pipeline hardening after a cache-loss post-mortem.
This PR contains the following updates:
| Package | Update | Change |
|---|---|---|
| [chardet](https://github.com/chardet/chardet) ([changelog](https://chardet.readthedocs.io/en/latest/changelog.html)) | minor | `==7.5.1` → `==7.6.0` |
---
### Release Notes
<details>
<summary>chardet/chardet (chardet)</summary>
### [`v7.6.0`](https://github.com/chardet/chardet/releases/tag/7.6.0)
[Compare Source](https://github.com/chardet/chardet/compare/7.5.1...7.6.0)
Big release: a Cython scoring kernel joins mypyc in compiled wheels, every model retrained on a deduplicated corpus, UTF-7 fixed in both directions, and a guarantee that `detect()` never returns an encoding that can't decode your complete input.
#### Performance
- Compiled wheels now score bigram profiles through a small Cython kernel alongside mypyc, and the pair is 4.7x faster than the pure wheel on CPython 3.14. `_kernel.py` stays plain Python (PyPy and pure wheels run it interpreted, unchanged), and detection output is bit-identical. The kernel declares itself safe without the GIL, so free-threaded CPython scales instead of silently re-enabling the GIL on import: 3.14t runs the whole suite in \~340ms across 8 threads, the fastest configuration measured. Compiled builds now need both hooks: `HATCH_BUILD_HOOK_ENABLE_MYPYC=true HATCH_BUILD_HOOK_ENABLE_CUSTOM=true`.
- Added support for CPython 3.15, including the free-threaded build. No code changes were needed.
#### Bug Fixes
- `detect()` no longer returns an encoding that cannot decode the input it was given ([#​380](https://github.com/chardet/chardet/issues/380), thanks [@​yarikoptic](https://github.com/yarikoptic)). When the whole input has been examined and the winner's only multi-byte evidence is an incomplete trailing sequence, the best candidate that decodes the input completely wins instead. Genuinely truncated data keeps its answer.
- Fixed delimited ASCII data like `|NAME,+LAY|` misdetecting as UTF-7 ([#​371](https://github.com/chardet/chardet/issues/371) follow-up, thanks [@​agreenburg](https://github.com/agreenburg)). The whole buffer must now actually decode as UTF-7, and a lone shifted character must land in a plausible script range.
- Signed UTF-7 no longer reads as ASCII: the BOM stage recognizes the four UTF-7 signature prefixes when the rest of the buffer decodes as UTF-7.
- Fixed short apostrophe-heavy English being labeled Scottish Gaelic or Breton: a rare-language label on an input under 128 bytes now needs a 0.03 lead over the best mainstream language (ADR-0005).
- Fixed Hungarian text losing to a Czech reading in confusion rescoring; tied pairs are compared only under language models both encodings have.
- Fixed space-padded text matching a degenerate Serbian model at high confidence; statistical scoring now skips repeated-whitespace bigrams. This also fixes windows-1251 logs misdetecting as windows-1250 ([#​379](https://github.com/chardet/chardet/issues/379)).
- Fixed EBCDIC text being invisible to the early pipeline stages, and the last two EBCDIC sibling misdetections.
- Fixed training normalization gaps that starved ISO-8859-16 and the 26 pre-euro encodings at exactly their distinguishing bytes.
#### Improvements
- Retrained every bigram model on a refreshed, deduplicated corpus with training provenance now recorded per model.
- New ANSI-art model for cp437, trained on 16,621 text-mode art files from [16colo.rs](https://16colo.rs/).
- Rare-language arbitration (ADR-0005): low-confidence statistical winners from languages with no documented legacy-encoding population yield to near-tied mainstream candidates.
- Confusion-group resolution is context-aware: per-occurrence votes, word-shape demotions, art-model exemption.
- Statistical dead heats no longer resolve by candidate enumeration order.
- Training pipeline hardening after a cache-loss post-mortem.
**Full Changelog**: <https://github.com/chardet/chardet/compare/7.5.1...7.6.0>
</details>
---
### Configuration
📅 **Schedule**: (in timezone America/Edmonton)
- Branch creation
- At any time (no schedule defined)
- Automerge
- At any time (no schedule defined)
🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box
---
This PR has been generated by [Mend Renovate CLI](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4zMC4xIiwidXBkYXRlZEluVmVyIjoiNDQuMzAuMSIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOltdfQ==-->
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
This PR contains the following updates:
==7.5.1→==7.6.0Release Notes
chardet/chardet (chardet)
v7.6.0Compare Source
Big release: a Cython scoring kernel joins mypyc in compiled wheels, every model retrained on a deduplicated corpus, UTF-7 fixed in both directions, and a guarantee that
detect()never returns an encoding that can't decode your complete input.Performance
_kernel.pystays plain Python (PyPy and pure wheels run it interpreted, unchanged), and detection output is bit-identical. The kernel declares itself safe without the GIL, so free-threaded CPython scales instead of silently re-enabling the GIL on import: 3.14t runs the whole suite in ~340ms across 8 threads, the fastest configuration measured. Compiled builds now need both hooks:HATCH_BUILD_HOOK_ENABLE_MYPYC=true HATCH_BUILD_HOOK_ENABLE_CUSTOM=true.Bug Fixes
detect()no longer returns an encoding that cannot decode the input it was given (#380, thanks @yarikoptic). When the whole input has been examined and the winner's only multi-byte evidence is an incomplete trailing sequence, the best candidate that decodes the input completely wins instead. Genuinely truncated data keeps its answer.|NAME,+LAY|misdetecting as UTF-7 (#371 follow-up, thanks @agreenburg). The whole buffer must now actually decode as UTF-7, and a lone shifted character must land in a plausible script range.Improvements
Full Changelog: https://github.com/chardet/chardet/compare/7.5.1...7.6.0
Configuration
📅 Schedule: (in timezone America/Edmonton)
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
This PR has been generated by Mend Renovate CLI.
e23ba04e6fto5ac51b4024