Added codecs: dvorak, soundex, telegraph

Author: dhondta

README.md

@@ -446,8 +446,11 @@ This category also contains `ascii85`, `adobe`, `[x]btoa`, `zeromq` with the `ba
 
 - [X] `binhex`: BinHex 4.0 (`.hqx`), the binary-to-text encoding of the classic Mac OS
 - [X] `dna`: implements the 8 rules of DNA sequences (N belongs to [1,8])
+- [X] `dvorak`: text typed with the Dvorak keyboard layout instead of QWERTY
 - [X] `letter-indices`: encodes consonants and/or vowels with their corresponding indices
 - [X] `markdown`: unidirectional encoding from Markdown to HTML
+- [X] `soundex`: Soundex phonetic codes (decoding gives English words with the same codes)
+- [X] `telegraph`: Chinese telegraph code, with the codebooks of Mainland China and Taiwan
 - [X] `xx`: XXencode, a UUencode variant with an alphanumeric alphabet (`+`, `-`, digits and letters)
 - [X] `yenc`: Usenet binary-to-text encoding (raw, wrapped or framed with header, trailer and CRC32)
 

docs/pages/enc/others.md

@@ -48,6 +48,23 @@ CACTCGGTCGGCCATATGTTCGGCCATATGTTCGTCTGTTCACTCGCCCATACACT
 
 -----
 
+### Dvorak
+
+The [Dvorak Simplified Keyboard](https://en.wikipedia.org/wiki/Dvorak_keyboard_layout), patented in 1936, rearranges the letters and some punctuation of the QWERTY layout (US). Encoding replaces each character by the one of the same key with the Dvorak layout (as when typing with the QWERTY layout on a computer set to Dvorak), while decoding does the opposite. This [permutation of 35 keys](https://www.montefischer.com/2021/03/21/dvorak-group-action.html) (letters and the `-`, `=`, `[`, `]`, `;`, `'`, `,`, `.` and `/` keys, with their shifted characters) gets back to the text after 210 encodings.
+
+**Codec** | **Conversions** | **Aliases** | **Comment**
+:---: | :---: | --- | ---
+`dvorak` | text <-> text typed with the Dvorak layout | `qwerty-dvorak`, `qwerty_to_dvorak` | other characters kept as is
+
+```python
+>>> codext.encode("Hello, World!", "dvorak")
+'D.nnrw <rpne!'
+>>> codext.decode("D.nnrw <rpne!", "dvorak")
+'Hello, World!'
+```
+
+-----
+
 ### Letter indices
 
 This encodes consonants and/or vowels with their respective indices. This codec is case insensitive, strips white spaces and only applies to letters.
@@ -96,6 +113,46 @@ This is only for "encoding" (converting) Markdown to HTML.
 
 -----
 
+### Soundex
+
+[Soundex](https://www.archives.gov/research/census/soundex) indexes names by their sound, as used by the US National Archives: each word is written as its first letter followed by three digits for its next consonants (B, F, P and V give 1 ; C, G, J, K, Q, S, X and Z give 2 ; D and T give 3 ; L gives 4 ; M and N give 5 ; R gives 6), the vowels being dropped, the consecutive consonants with the same digit (including the first letter, or separated by H or W) giving a single digit, and the code being padded with zeros or truncated. Encoding replaces each word by its code, keeping the other characters. As Soundex is not reversible, decoding replaces each code by the most frequent word with this code from an English word list (9850 words, from the [lists of Ciphey](https://github.com/Ciphey/CipheyDists)), or, with the `-all` variant, by all these words in order of frequency.
+
+**Codec** | **Conversions** | **Aliases** | **Comment**
+:---: | :---: | --- | ---
+`soundex` | text <-> Soundex codes | | decoding gives the most frequent English word for each code ; codes without word handled with the errors handling
+`soundex-all` | text <-> Soundex codes | `soundex_all` | decoding gives all the English words for each code (e.g. `[world|worldwide|worlds]`)
+
+!!! warning "Decoding"
+
+    Decoding cannot give back the original words, only words with the same code (e.g. `Robert` gives `R163`, which gives `report`). Ciphey's own table of codes does not merge the first letter with the next consonant with the same digit (e.g. `cc` gives `C200` instead of `C000` and `Pfister` gives `P123` instead of `P236`), unlike the rules of the US National Archives followed by this codec.
+
+```python
+>>> codext.encode("Robert Rupert Pfister", "soundex")
+'R163 R163 P236'
+>>> codext.decode("R163 W643", "soundex")
+'report world'
+```
+
+-----
+
+### Telegraph
+
+The [Chinese telegraph code](https://en.wikipedia.org/wiki/Chinese_telegraph_code), designed in 1871, sends Chinese characters by telegraph as groups of four digits (from `0000` to `9999`), as given in a codebook. Mainland China (with the simplified characters) and Taiwan (with the traditional characters) use different codebooks, taken from the [Unihan database](https://www.unicode.org/reports/tr38/#kMainlandTelegraph) of Unicode. Encoding gives the groups of four digits of the characters, separated by spaces, while decoding accepts groups separated by whitespaces or not.
+
+**Codec** | **Conversions** | **Aliases** | **Comment**
+:---: | :---: | --- | ---
+`telegraph` | Chinese text <-> groups of 4 digits | `chinese-telegraph`, `telegraph-mainland` | codebook of Mainland China (7078 characters)
+`telegraph-taiwan` | Chinese text <-> groups of 4 digits | `telegraph-tw` | codebook of Taiwan (9024 characters)
+
+```python
+>>> codext.encode("中文信息", "telegraph")
+'0022 2429 0207 1873'
+>>> codext.decode("5618 1947 0948", "telegraph-taiwan")
+'蕭愛國'
+```
+
+-----
+
 ### XXencode
 
 XXencode works like UUencode, but with an alphabet made of "`+`", "`-`", digits and letters only. Each line encodes up to 45 bytes and starts with the character holding its number of bytes ; the lines are framed by a `begin` line and an `end` line.

src/codext/others/__init__.py

@@ -1,9 +1,12 @@
 # -*- coding: UTF-8 -*-
 from .binhex import *
 from .dna import *
+from .dvorak import *
 from .kbshift import *
 from .letters import *
 from .markdown import *
+from .soundex import *
+from .telegraph import *
 from .uuencode import *
 from .xxencode import *
 from .yenc import *

src/codext/others/dvorak.py

@@ -0,0 +1,45 @@
+# -*- coding: UTF-8 -*-
+"""Dvorak Codec - QWERTY to Dvorak keyboard layout content encoding.
+
+The Dvorak Simplified Keyboard, patented by August Dvorak and William Dealey in 1936, rearranges the letters and some
+ punctuation of the standard QWERTY layout (US) so that the most frequent letters are on the home row. Encoding replaces
+ each character by the one of the same key with the Dvorak layout (as when typing with the QWERTY layout on a computer
+ set to Dvorak), while decoding does the opposite (as when typing with the Dvorak layout on a computer set to QWERTY).
+ This permutation of 35 keys (letters and the "-", "=", "[", "]", ";", "'", ",", "." and "/" keys, with their shifted
+ characters) is made of cycles of 15, 14, 2, 2, 1 and 1 keys, hence it gets back to the text after 210 encodings.
+
+This codec:
+- en/decodes strings from str to str
+- en/decodes strings from bytes to bytes
+- decodes file content to str (read)
+- encodes file content from str to bytes (write)
+
+Reference: https://www.montefischer.com/2021/03/21/dvorak-group-action.html
+"""
+from ..__common__ import *
+
+
+__examples__ = {
+    'enc(dvorak|qwerty-dvorak|qwerty_to_dvorak)': {'Hello, World!': "D.nnrw <rpne!", 'dvorak': "ekrpat"},
+    'dec(dvorak)': {'D.nnrw <rpne!': "Hello, World!", 'Cbol.jyrp Mai': "Inspector Mag"},
+    'enc-dec(dvorak)': ["@random{256}", "The quick brown fox jumps over the lazy dog: \"-=[];',./_+{}:<>?\""],
+}
+__guess__ = ["dvorak"]
+
+
+# characters of the keys of the QWERTY layout and of the Dvorak layout, unshifted then shifted (US)
+QWERTY = "-=qwertyuiop[]asdfghjkl;'zxcvbnm,./" "_+QWERTYUIOP{}ASDFGHJKL:\"ZXCVBNM<>?"
+DVORAK = "[]',.pyfgcrl/=aoeuidhtns-;qjkxbmwvz" "{}\"<>PYFGCRL?+AOEUIDHTNS_:QJKXBMWVZ"
+_ENCODE, _DECODE = str.maketrans(QWERTY, DVORAK), str.maketrans(DVORAK, QWERTY)
+
+
+def dvorak_encode(text, errors="strict"):
+    return ensure_str(text).translate(_ENCODE), len(text)
+
+
+def dvorak_decode(text, errors="strict"):
+    return ensure_str(text).translate(_DECODE), len(text)
+
+
+add("dvorak", dvorak_encode, dvorak_decode, r"^(?:qwerty[-_](?:to[-_])?)?dvorak$", examples=__examples__,
+    guess=__guess__, entropy=lambda e: e, printables_rate=lambda pr: pr, transitive=True)