toml metafile works

gawa/blog/admin.py

@@ -38,5 +38,5 @@ class BlogPostAdmin(admin.ModelAdmin):
         return my_urls + urls
 
     def sync_with_fs(self, request):
-        BlogPost.sync_all()
+        BlogPost.sync_with_fs()
         return HttpResponseRedirect("../")

gawa/blog/data/articles/bash-arrays.toml

@@ -0,0 +1,22 @@
+date = "2023-09-29"
+keywords = ["bash"]
+category = "Guide"
+featured = true
+public = true
+thumbnail = "img/thumbnails/bash.png"
+
+[lang.en]
+title = "bash arrays"
+subtitle = "how to work with bash arrays"
+desc = """
+bash scripting can be kind of weird.
+This guide explains how they work without any fuzz.
+"""
+
+[lang.de]
+title = "Bash Arrays"
+subtitle = "Wie man mit Bash Arrays arbeitet"
+desc = """
+Bash Skripte sind manchmal etwas seltsam.
+Hier wird erklärt, wie man mit Bash Arrays arbeitet.
+"""

gawa/blog/data/articles/en-bash-arrays.md

@@ -1,15 +1,3 @@
----
-Title:      Bash Arrays
-Subtitle:   sub
-Desc:       Brief intro to Bash Arrays
-Date:       2023-09-29
-Keywords:   bash
-            technology
-Category:   Test
-Featured:   True
-Public:     True
----
-
 **NOTE**
 
 This is a stolen article from [opensource.com](https://opensource.com/article/18/5/you-dont-know-bash-intro-bash-arrays)

gawa/blog/data/articles/en-empty.md

@@ -1,8 +0,0 @@
----
-Title:      empty
-Subtitle:   empty
-Desc:       empty
-Date:       2023-09-29
-Featured:   True
-Public:     False
----

gawa/blog/data/articles/en-test.md

@@ -1,90 +0,0 @@
-**NOTE**
-
-This is a stolen article from [opensource.com](https://opensource.com/article/18/5/you-dont-know-bash-intro-bash-arrays)
-about bash scripting. It's a good article and I've decided to use it to test my
-markdown rendering.
-
-# Bash scripting
-
-[TOC]
-
-
-## Wait, but why?
-
-Writing about Bash is challenging because it's remarkably easy for an article
-to devolve into a manual that focuses on syntax oddities. Rest assured,
-however, the intent of this article is to avoid having you RTFM.
-
-## A real (actually useful) example
-
-To that end, let's consider a real-world scenario and how Bash can help:
-You are leading a new effort at your company to evaluate and optimize the
-runtime of your internal data pipeline. As a first step, you want to do a
-parameter sweep to evaluate how well the pipeline makes use of threads. For
-the sake of simplicity, we'll treat the pipeline as a compiled C++ black box
-where the only parameter we can tweak is the number of threads reserved for
-data processing: `./pipeline --threads 4.`
-
-## The basics
-
-The first thing we'll do is define an array containing the values of the
-`--threads` parameter that we want to test:
-
-```bash
-allThreads=(1 2 4 8 16 32 64 128)
-```
-
-In this example, all the elements are numbers, but it need not be the
-case—arrays in Bash can contain both numbers and strings, e.g., `myArray=(1
-2 "three" 4 "five")` is a valid expression. And just as with any other Bash
-variable, make sure to leave no spaces around the equal sign. Otherwise,
-Bash will treat the variable name as a program to execute, and the `=` as its
-first parameter!
-
-Now that we've initialized the array, let's retrieve a few of its
-elements. You'll notice that simply doing `echo $allThreads` will output only
-the first element.
-
-To understand why that is, let's take a step back and revisit how we usually
-output variables in Bash. Consider the following scenario:
-
-```bash
-type="article" echo "Found 42 $type"
-```
-
-Say the variable $type is given to us as a singular noun and we want to add
-an `s` at the end of our sentence. We can't simply add an s to `$type` since
-that would turn it into a different variable, `$types`. And although we could
-utilize code contortions such as `echo "Found 42 "$type"s"`, the best way
-to solve this problem is to use curly braces: `echo "Found 42 ${type}s"`,
-which allows us to tell Bash where the name of a variable starts and ends
-(interestingly, this is the same syntax used in JavaScript/ES6 to inject
-variables and expressions in template literals).
-
-So as it turns out, although Bash variables don't generally require curly
-brackets, they are required for arrays. In turn, this allows us to specify
-the index to access, e.g., `echo ${allThreads[1]}` returns the second element
-of the array. Not including brackets, e.g.,`echo $allThreads[1]`, leads Bash
-to treat `[1]` as a string and output it as such.
-
-Yes, Bash arrays have odd syntax, but at least they are zero-indexed, unlike
-some other languages (I'm looking at you, R).[^1]
-
-## Looping through arrays
-
-Although in the examples above we used integer indices in our arrays, let's
-consider two occasions when that won't be the case: First, if we wanted the
-$i-th element of the array, where $i is a variable containing the index of
-interest, we can retrieve that element using: echo ${allThreads[$i]}. Second,
-to output all the elements of an array, we replace the numeric index with
-the @ symbol (you can think of @ as standing for all):
-
-```bash
-type="article"
-echo "Found 42 $type"
-```
-
-*[RTFM]: Read the Fucking Manual
-*[HTML]: Hyper Text Markup Language
-
-[^1]: Example Footnote

gawa/blog/data/articles/test.toml

@@ -1,24 +0,0 @@
-date = "2023-09-29"
-keywords = ["test", "bash"]
-category = "Test"
-featured = true
-public = true
-thumbnail = "img/thumbnails/bash.png"
-
-[lang.en]
-title = "title"
-subtitle = "subtitle"
-desc = """
-long
-multiline
-desc
-"""
-
-[lang.de]
-title = "Titel"
-subtitle = "Subtitel"
-desc = """
-Lange,
-mehrzeilige,
-Beschreibung
-"""

gawa/blog/models.py

@@ -130,9 +130,6 @@ class BlogPost(Searchable):
             with open(f"{self.DATA_DIR}/{lang}-{self.slug}.md") as f_en:
                 MD.reset()
                 body: str = f_en.read()
-                # FIXME:
-                # these fields are loaded but wrong?
-                # de is in en, de is empty
                 match lang:
                     case "en":
                         self.title_en = data['lang'][lang]["title"]
@@ -140,9 +137,9 @@ class BlogPost(Searchable):
                         self.desc_en = data['lang'][lang]["desc"]
                         self.body_en = MD.convert(body)
                     case "de":
-                        self.title_en = data['lang'][lang]["title"]
+                        self.title_de = data['lang'][lang]["title"]
-                        self.subtitle_en = data['lang'][lang]["subtitle"]
+                        self.subtitle_de = data['lang'][lang]["subtitle"]
-                        self.desc_en = data['lang'][lang]["desc"]
+                        self.desc_de = data['lang'][lang]["desc"]
                         self.body_de = MD.convert(body)
                     case _:
                         logger.error(
@@ -153,7 +150,7 @@ class BlogPost(Searchable):
         # NOTE: thumbnail is optional
         if "thumbnail" in data:
             self.thumbnail = data["thumbnail"]
-        # NOTE: thumbnail is optional
+        # NOTE: category is optional
         if "category" in data:
             try:
                 category: Category = Category.objects.get(
@@ -163,26 +160,27 @@ class BlogPost(Searchable):
                     name=data['category'], slug=data['category'])
             self.category = category
         self.save()
-        logger.debug("keywords next")
         for keyword in data["keywords"]:
             try:
                 self.keywords.add(Keyword.objects.get(slug=keyword))
             except Keyword.DoesNotExist:
                 self.keywords.create(
                     slug=keyword, text_en=keyword, text_de=keyword)
-        logger.debug(f"keywords of '{self}': {self.keywords}")
         self.save()
 
     def get_langs(self) -> dict[str, bool]:
         """
         get available languages
         """
-        # TODO:
-        # make sure this is safe
         # SECURITY:
-        # If someone could inject the langs field, arbitrary python code might
+        # If someone could inject the langs field, literal_eval will evaluate to
-        # run, Potentially ending in a critical RCE vulnerability
+        # any literal Python value. This should not be a concern.
+        # If we would use a regular eval here, an attacker who controls one of
+        # the `langs` of a BlogPost could run arbitrary Python code.
         try:
+            # literal_eval is safe because it only parses literals, not any kind
+            # of python expression. If the given str is not a valid literal,
+            # ValueError will be raised
             langs = ast.literal_eval(str(self.langs))
             return langs
         except ValueError as e:
@@ -197,13 +195,13 @@ class BlogPost(Searchable):
         self.langs = langs.__repr__()
 
     @ classmethod
-    def sync_all(cls):
+    def sync_with_fs(cls):
         """
         Sync all Blog Posts with the filesystem.
 
         Caution: Will delete all Blog Posts
         """
-        # logger.name = logger.name + ".sync_all"
+        # logger.name = logger.name + ".sync_with_fs"
 
         # delete all existing objects
         BlogPost.objects.all().delete()
@@ -221,6 +219,7 @@ class BlogPost(Searchable):
 
         # finding lang and title
         regex = r"^(en|de)-(.*)\.md"
+        # TODO: only find toml files
 
         # filepath, language codes, slug
         files = [[f, cls.DEFAULT_LANGS, ""] for f in files]
@@ -242,7 +241,7 @@ class BlogPost(Searchable):
                 files.remove(file)
 
         # PERF:
-        # Could possibly be done in one loop
+        # Optimize for single loop, should be doable and better design
 
         # collapse diffrent versions
         for file in files:
@@ -261,6 +260,7 @@ class BlogPost(Searchable):
                 logger.error(
                     f"Could not combine BlogPosts for '{file[0]}': {e}")
 
+        # TODO: not needed when relying on toml files
         try:
             # deduplicate
             _files = []