PlexSheep
/
wordle-analyzer
generated from PlexSheep/rs-base
1
0
Fork 0
Code Issues 11 Pull Requests 1 Actions Packages Projects Releases 1 Wiki Activity

feat(german): add german wordlist with data from SUBTLEX-DE
cargo devel CI / cargo CI (push) Has been cancelled Details

This commit is contained in:
Christoph J. Scherr 2024-07-25 16:31:42 +02:00
parent 82e0b49434
commit d651756265
3 changed files with 215174 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

190502
data/wordlists/german_SUBTLEX-DE_full.json Executable file
View File

File diff suppressed because it is too large Load Diff

24654
data/wordlists/german_SUBTLEX-DE_small.json Normal file
View File

File diff suppressed because it is too large Load Diff

18
scripts/discard_unimportant_words.py Normal file
View File

@ -0,0 +1,18 @@
import json
# Load the word frequency dictionary
with open('../data/wordlists/german_SUBTLEX-DE_full.json', 'r') as f:
word_freqs = json.load(f)
# Set a frequency threshold (e.g., 0.001)
freq_threshold = 0.000001
# Set a maximum word length (e.g., 10)
max_word_length = 10
# Filter out words with low frequency and long length
filtered_word_freqs = {word: freq for word, freq in word_freqs.items() if freq >= freq_threshold and len(word) <= max_word_length}
# Save the filtered word frequencies to a new JSON file
with open('../data/wordlists/german_SUBTLEX-DE_small.json', 'w') as f:
json.dump(filtered_word_freqs, f, indent=4)