PlexSheep
/
gawa
1
0
Fork 0
Code Issues 27 Pull Requests Actions Packages Projects 1 Releases Wiki Activity

toml metafile works

This commit is contained in:
Christoph J. Scherr 2023-10-02 09:49:04 +02:00
parent 3bf0dd42c7
commit a9a90e9bd4
8 changed files with 39 additions and 151 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
gawa/blog/admin.py
View File

@ -38,5 +38,5 @@ class BlogPostAdmin(admin.ModelAdmin):
return my_urls + urls
def sync_with_fs(self, request):
BlogPost.sync_all()
BlogPost.sync_with_fs()
return HttpResponseRedirect("../")

22
gawa/blog/data/articles/bash-arrays.toml Normal file
View File

@ -0,0 +1,22 @@
date = "2023-09-29"
keywords = ["bash"]
category = "Guide"
featured = true
public = true
thumbnail = "img/thumbnails/bash.png"
[lang.en]
title = "bash arrays"
subtitle = "how to work with bash arrays"
desc = """
bash scripting can be kind of weird.
This guide explains how they work without any fuzz.
"""
[lang.de]
title = "Bash Arrays"
subtitle = "Wie man mit Bash Arrays arbeitet"
desc = """
Bash Skripte sind manchmal etwas seltsam.
Hier wird erklärt, wie man mit Bash Arrays arbeitet.
"""

0
gawa/blog/data/articles/de-test.md → gawa/blog/data/articles/de-bash-arrays.md
View File

12
gawa/blog/data/articles/en-bash-arrays.md
View File

@ -1,15 +1,3 @@
---
Title: Bash Arrays
Subtitle: sub
Desc: Brief intro to Bash Arrays
Date: 2023-09-29
Keywords: bash
technology
Category: Test
Featured: True
Public: True
---
**NOTE**
This is a stolen article from [opensource.com](https://opensource.com/article/18/5/you-dont-know-bash-intro-bash-arrays)

8
gawa/blog/data/articles/en-empty.md
View File

@ -1,8 +0,0 @@
---
Title: empty
Subtitle: empty
Desc: empty
Date: 2023-09-29
Featured: True
Public: False
---

90
gawa/blog/data/articles/en-test.md
View File

@ -1,90 +0,0 @@
**NOTE**
This is a stolen article from [opensource.com](https://opensource.com/article/18/5/you-dont-know-bash-intro-bash-arrays)
about bash scripting. It's a good article and I've decided to use it to test my
markdown rendering.
# Bash scripting
[TOC]
## Wait, but why?
Writing about Bash is challenging because it's remarkably easy for an article
to devolve into a manual that focuses on syntax oddities. Rest assured,
however, the intent of this article is to avoid having you RTFM.
## A real (actually useful) example
To that end, let's consider a real-world scenario and how Bash can help:
You are leading a new effort at your company to evaluate and optimize the
runtime of your internal data pipeline. As a first step, you want to do a
parameter sweep to evaluate how well the pipeline makes use of threads. For
the sake of simplicity, we'll treat the pipeline as a compiled C++ black box
where the only parameter we can tweak is the number of threads reserved for
data processing: `./pipeline --threads 4.`
## The basics
The first thing we'll do is define an array containing the values of the
`--threads` parameter that we want to test:
```bash
allThreads=(1 2 4 8 16 32 64 128)
```
In this example, all the elements are numbers, but it need not be the
case—arrays in Bash can contain both numbers and strings, e.g., `myArray=(1
2 "three" 4 "five")` is a valid expression. And just as with any other Bash
variable, make sure to leave no spaces around the equal sign. Otherwise,
Bash will treat the variable name as a program to execute, and the `=` as its
first parameter!
Now that we've initialized the array, let's retrieve a few of its
elements. You'll notice that simply doing `echo $allThreads` will output only
the first element.
To understand why that is, let's take a step back and revisit how we usually
output variables in Bash. Consider the following scenario:
```bash
type="article" echo "Found 42 $type"
```
Say the variable $type is given to us as a singular noun and we want to add
an `s` at the end of our sentence. We can't simply add an s to `$type` since
that would turn it into a different variable, `$types`. And although we could
utilize code contortions such as `echo "Found 42 "$type"s"`, the best way
to solve this problem is to use curly braces: `echo "Found 42 ${type}s"`,
which allows us to tell Bash where the name of a variable starts and ends
(interestingly, this is the same syntax used in JavaScript/ES6 to inject
variables and expressions in template literals).
So as it turns out, although Bash variables don't generally require curly
brackets, they are required for arrays. In turn, this allows us to specify
the index to access, e.g., `echo ${allThreads[1]}` returns the second element
of the array. Not including brackets, e.g.,`echo $allThreads[1]`, leads Bash
to treat `[1]` as a string and output it as such.
Yes, Bash arrays have odd syntax, but at least they are zero-indexed, unlike
some other languages (I'm looking at you, R).[^1]
## Looping through arrays
Although in the examples above we used integer indices in our arrays, let's
consider two occasions when that won't be the case: First, if we wanted the
$i-th element of the array, where $i is a variable containing the index of
interest, we can retrieve that element using: echo ${allThreads[$i]}. Second,
to output all the elements of an array, we replace the numeric index with
the @ symbol (you can think of @ as standing for all):
```bash
type="article"
echo "Found 42 $type"
```
*[RTFM]: Read the Fucking Manual
*[HTML]: Hyper Text Markup Language
[^1]: Example Footnote

24
gawa/blog/data/articles/test.toml
View File

@ -1,24 +0,0 @@
date = "2023-09-29"
keywords = ["test", "bash"]
category = "Test"
featured = true
public = true
thumbnail = "img/thumbnails/bash.png"
[lang.en]
title = "title"
subtitle = "subtitle"
desc = """
long
multiline
desc
"""
[lang.de]
title = "Titel"
subtitle = "Subtitel"
desc = """
Lange,
mehrzeilige,
Beschreibung
"""

32
gawa/blog/models.py
View File

@ -130,9 +130,6 @@ class BlogPost(Searchable):
with open(f"{self.DATA_DIR}/{lang}-{self.slug}.md") as f_en:
MD.reset()
body: str = f_en.read()
# FIXME:
# these fields are loaded but wrong?
# de is in en, de is empty
match lang:
case "en":
self.title_en = data['lang'][lang]["title"]
@ -140,9 +137,9 @@ class BlogPost(Searchable):
self.desc_en = data['lang'][lang]["desc"]
self.body_en = MD.convert(body)
case "de":
self.title_en = data['lang'][lang]["title"]
self.subtitle_en = data['lang'][lang]["subtitle"]
self.desc_en = data['lang'][lang]["desc"]
self.title_de = data['lang'][lang]["title"]
self.subtitle_de = data['lang'][lang]["subtitle"]
self.desc_de = data['lang'][lang]["desc"]
self.body_de = MD.convert(body)
case _:
logger.error(
@ -153,7 +150,7 @@ class BlogPost(Searchable):
# NOTE: thumbnail is optional
if "thumbnail" in data:
self.thumbnail = data["thumbnail"]
# NOTE: thumbnail is optional
# NOTE: category is optional
if "category" in data:
try:
category: Category = Category.objects.get(
@ -163,26 +160,27 @@ class BlogPost(Searchable):
name=data['category'], slug=data['category'])
self.category = category
self.save()
logger.debug("keywords next")
for keyword in data["keywords"]:
try:
self.keywords.add(Keyword.objects.get(slug=keyword))
except Keyword.DoesNotExist:
self.keywords.create(
slug=keyword, text_en=keyword, text_de=keyword)
logger.debug(f"keywords of '{self}': {self.keywords}")
self.save()
def get_langs(self) -> dict[str, bool]:
"""
get available languages
"""
# TODO:
# make sure this is safe
# SECURITY:
# If someone could inject the langs field, arbitrary python code might
# run, Potentially ending in a critical RCE vulnerability
# If someone could inject the langs field, literal_eval will evaluate to
# any literal Python value. This should not be a concern.
# If we would use a regular eval here, an attacker who controls one of
# the `langs` of a BlogPost could run arbitrary Python code.
try:
# literal_eval is safe because it only parses literals, not any kind
# of python expression. If the given str is not a valid literal,
# ValueError will be raised
langs = ast.literal_eval(str(self.langs))
return langs
except ValueError as e:
@ -197,13 +195,13 @@ class BlogPost(Searchable):
self.langs = langs.__repr__()
@ classmethod
def sync_all(cls):
def sync_with_fs(cls):
"""
Sync all Blog Posts with the filesystem.
Caution: Will delete all Blog Posts
"""
# logger.name = logger.name + ".sync_all"
# logger.name = logger.name + ".sync_with_fs"
# delete all existing objects
BlogPost.objects.all().delete()
@ -221,6 +219,7 @@ class BlogPost(Searchable):
# finding lang and title
regex = r"^(en|de)-(.*)\.md"
# TODO: only find toml files
# filepath, language codes, slug
files = [[f, cls.DEFAULT_LANGS, ""] for f in files]
@ -242,7 +241,7 @@ class BlogPost(Searchable):
files.remove(file)
# PERF:
# Could possibly be done in one loop
# Optimize for single loop, should be doable and better design
# collapse diffrent versions
for file in files:
@ -261,6 +260,7 @@ class BlogPost(Searchable):
logger.error(
f"Could not combine BlogPosts for '{file[0]}': {e}")
# TODO: not needed when relying on toml files
try:
# deduplicate
_files = []