toml metafile works
This commit is contained in:
parent
3bf0dd42c7
commit
a9a90e9bd4
|
@ -38,5 +38,5 @@ class BlogPostAdmin(admin.ModelAdmin):
|
|||
return my_urls + urls
|
||||
|
||||
def sync_with_fs(self, request):
|
||||
BlogPost.sync_all()
|
||||
BlogPost.sync_with_fs()
|
||||
return HttpResponseRedirect("../")
|
||||
|
|
|
@ -0,0 +1,22 @@
|
|||
date = "2023-09-29"
|
||||
keywords = ["bash"]
|
||||
category = "Guide"
|
||||
featured = true
|
||||
public = true
|
||||
thumbnail = "img/thumbnails/bash.png"
|
||||
|
||||
[lang.en]
|
||||
title = "bash arrays"
|
||||
subtitle = "how to work with bash arrays"
|
||||
desc = """
|
||||
bash scripting can be kind of weird.
|
||||
This guide explains how they work without any fuzz.
|
||||
"""
|
||||
|
||||
[lang.de]
|
||||
title = "Bash Arrays"
|
||||
subtitle = "Wie man mit Bash Arrays arbeitet"
|
||||
desc = """
|
||||
Bash Skripte sind manchmal etwas seltsam.
|
||||
Hier wird erklärt, wie man mit Bash Arrays arbeitet.
|
||||
"""
|
|
@ -1,15 +1,3 @@
|
|||
---
|
||||
Title: Bash Arrays
|
||||
Subtitle: sub
|
||||
Desc: Brief intro to Bash Arrays
|
||||
Date: 2023-09-29
|
||||
Keywords: bash
|
||||
technology
|
||||
Category: Test
|
||||
Featured: True
|
||||
Public: True
|
||||
---
|
||||
|
||||
**NOTE**
|
||||
|
||||
This is a stolen article from [opensource.com](https://opensource.com/article/18/5/you-dont-know-bash-intro-bash-arrays)
|
||||
|
|
|
@ -1,8 +0,0 @@
|
|||
---
|
||||
Title: empty
|
||||
Subtitle: empty
|
||||
Desc: empty
|
||||
Date: 2023-09-29
|
||||
Featured: True
|
||||
Public: False
|
||||
---
|
|
@ -1,90 +0,0 @@
|
|||
**NOTE**
|
||||
|
||||
This is a stolen article from [opensource.com](https://opensource.com/article/18/5/you-dont-know-bash-intro-bash-arrays)
|
||||
about bash scripting. It's a good article and I've decided to use it to test my
|
||||
markdown rendering.
|
||||
|
||||
# Bash scripting
|
||||
|
||||
[TOC]
|
||||
|
||||
|
||||
## Wait, but why?
|
||||
|
||||
Writing about Bash is challenging because it's remarkably easy for an article
|
||||
to devolve into a manual that focuses on syntax oddities. Rest assured,
|
||||
however, the intent of this article is to avoid having you RTFM.
|
||||
|
||||
## A real (actually useful) example
|
||||
|
||||
To that end, let's consider a real-world scenario and how Bash can help:
|
||||
You are leading a new effort at your company to evaluate and optimize the
|
||||
runtime of your internal data pipeline. As a first step, you want to do a
|
||||
parameter sweep to evaluate how well the pipeline makes use of threads. For
|
||||
the sake of simplicity, we'll treat the pipeline as a compiled C++ black box
|
||||
where the only parameter we can tweak is the number of threads reserved for
|
||||
data processing: `./pipeline --threads 4.`
|
||||
|
||||
## The basics
|
||||
|
||||
The first thing we'll do is define an array containing the values of the
|
||||
`--threads` parameter that we want to test:
|
||||
|
||||
```bash
|
||||
allThreads=(1 2 4 8 16 32 64 128)
|
||||
```
|
||||
|
||||
In this example, all the elements are numbers, but it need not be the
|
||||
case—arrays in Bash can contain both numbers and strings, e.g., `myArray=(1
|
||||
2 "three" 4 "five")` is a valid expression. And just as with any other Bash
|
||||
variable, make sure to leave no spaces around the equal sign. Otherwise,
|
||||
Bash will treat the variable name as a program to execute, and the `=` as its
|
||||
first parameter!
|
||||
|
||||
Now that we've initialized the array, let's retrieve a few of its
|
||||
elements. You'll notice that simply doing `echo $allThreads` will output only
|
||||
the first element.
|
||||
|
||||
To understand why that is, let's take a step back and revisit how we usually
|
||||
output variables in Bash. Consider the following scenario:
|
||||
|
||||
```bash
|
||||
type="article" echo "Found 42 $type"
|
||||
```
|
||||
|
||||
Say the variable $type is given to us as a singular noun and we want to add
|
||||
an `s` at the end of our sentence. We can't simply add an s to `$type` since
|
||||
that would turn it into a different variable, `$types`. And although we could
|
||||
utilize code contortions such as `echo "Found 42 "$type"s"`, the best way
|
||||
to solve this problem is to use curly braces: `echo "Found 42 ${type}s"`,
|
||||
which allows us to tell Bash where the name of a variable starts and ends
|
||||
(interestingly, this is the same syntax used in JavaScript/ES6 to inject
|
||||
variables and expressions in template literals).
|
||||
|
||||
So as it turns out, although Bash variables don't generally require curly
|
||||
brackets, they are required for arrays. In turn, this allows us to specify
|
||||
the index to access, e.g., `echo ${allThreads[1]}` returns the second element
|
||||
of the array. Not including brackets, e.g.,`echo $allThreads[1]`, leads Bash
|
||||
to treat `[1]` as a string and output it as such.
|
||||
|
||||
Yes, Bash arrays have odd syntax, but at least they are zero-indexed, unlike
|
||||
some other languages (I'm looking at you, R).[^1]
|
||||
|
||||
## Looping through arrays
|
||||
|
||||
Although in the examples above we used integer indices in our arrays, let's
|
||||
consider two occasions when that won't be the case: First, if we wanted the
|
||||
$i-th element of the array, where $i is a variable containing the index of
|
||||
interest, we can retrieve that element using: echo ${allThreads[$i]}. Second,
|
||||
to output all the elements of an array, we replace the numeric index with
|
||||
the @ symbol (you can think of @ as standing for all):
|
||||
|
||||
```bash
|
||||
type="article"
|
||||
echo "Found 42 $type"
|
||||
```
|
||||
|
||||
*[RTFM]: Read the Fucking Manual
|
||||
*[HTML]: Hyper Text Markup Language
|
||||
|
||||
[^1]: Example Footnote
|
|
@ -1,24 +0,0 @@
|
|||
date = "2023-09-29"
|
||||
keywords = ["test", "bash"]
|
||||
category = "Test"
|
||||
featured = true
|
||||
public = true
|
||||
thumbnail = "img/thumbnails/bash.png"
|
||||
|
||||
[lang.en]
|
||||
title = "title"
|
||||
subtitle = "subtitle"
|
||||
desc = """
|
||||
long
|
||||
multiline
|
||||
desc
|
||||
"""
|
||||
|
||||
[lang.de]
|
||||
title = "Titel"
|
||||
subtitle = "Subtitel"
|
||||
desc = """
|
||||
Lange,
|
||||
mehrzeilige,
|
||||
Beschreibung
|
||||
"""
|
|
@ -130,9 +130,6 @@ class BlogPost(Searchable):
|
|||
with open(f"{self.DATA_DIR}/{lang}-{self.slug}.md") as f_en:
|
||||
MD.reset()
|
||||
body: str = f_en.read()
|
||||
# FIXME:
|
||||
# these fields are loaded but wrong?
|
||||
# de is in en, de is empty
|
||||
match lang:
|
||||
case "en":
|
||||
self.title_en = data['lang'][lang]["title"]
|
||||
|
@ -140,9 +137,9 @@ class BlogPost(Searchable):
|
|||
self.desc_en = data['lang'][lang]["desc"]
|
||||
self.body_en = MD.convert(body)
|
||||
case "de":
|
||||
self.title_en = data['lang'][lang]["title"]
|
||||
self.subtitle_en = data['lang'][lang]["subtitle"]
|
||||
self.desc_en = data['lang'][lang]["desc"]
|
||||
self.title_de = data['lang'][lang]["title"]
|
||||
self.subtitle_de = data['lang'][lang]["subtitle"]
|
||||
self.desc_de = data['lang'][lang]["desc"]
|
||||
self.body_de = MD.convert(body)
|
||||
case _:
|
||||
logger.error(
|
||||
|
@ -153,7 +150,7 @@ class BlogPost(Searchable):
|
|||
# NOTE: thumbnail is optional
|
||||
if "thumbnail" in data:
|
||||
self.thumbnail = data["thumbnail"]
|
||||
# NOTE: thumbnail is optional
|
||||
# NOTE: category is optional
|
||||
if "category" in data:
|
||||
try:
|
||||
category: Category = Category.objects.get(
|
||||
|
@ -163,26 +160,27 @@ class BlogPost(Searchable):
|
|||
name=data['category'], slug=data['category'])
|
||||
self.category = category
|
||||
self.save()
|
||||
logger.debug("keywords next")
|
||||
for keyword in data["keywords"]:
|
||||
try:
|
||||
self.keywords.add(Keyword.objects.get(slug=keyword))
|
||||
except Keyword.DoesNotExist:
|
||||
self.keywords.create(
|
||||
slug=keyword, text_en=keyword, text_de=keyword)
|
||||
logger.debug(f"keywords of '{self}': {self.keywords}")
|
||||
self.save()
|
||||
|
||||
def get_langs(self) -> dict[str, bool]:
|
||||
"""
|
||||
get available languages
|
||||
"""
|
||||
# TODO:
|
||||
# make sure this is safe
|
||||
# SECURITY:
|
||||
# If someone could inject the langs field, arbitrary python code might
|
||||
# run, Potentially ending in a critical RCE vulnerability
|
||||
# If someone could inject the langs field, literal_eval will evaluate to
|
||||
# any literal Python value. This should not be a concern.
|
||||
# If we would use a regular eval here, an attacker who controls one of
|
||||
# the `langs` of a BlogPost could run arbitrary Python code.
|
||||
try:
|
||||
# literal_eval is safe because it only parses literals, not any kind
|
||||
# of python expression. If the given str is not a valid literal,
|
||||
# ValueError will be raised
|
||||
langs = ast.literal_eval(str(self.langs))
|
||||
return langs
|
||||
except ValueError as e:
|
||||
|
@ -197,13 +195,13 @@ class BlogPost(Searchable):
|
|||
self.langs = langs.__repr__()
|
||||
|
||||
@ classmethod
|
||||
def sync_all(cls):
|
||||
def sync_with_fs(cls):
|
||||
"""
|
||||
Sync all Blog Posts with the filesystem.
|
||||
|
||||
Caution: Will delete all Blog Posts
|
||||
"""
|
||||
# logger.name = logger.name + ".sync_all"
|
||||
# logger.name = logger.name + ".sync_with_fs"
|
||||
|
||||
# delete all existing objects
|
||||
BlogPost.objects.all().delete()
|
||||
|
@ -221,6 +219,7 @@ class BlogPost(Searchable):
|
|||
|
||||
# finding lang and title
|
||||
regex = r"^(en|de)-(.*)\.md"
|
||||
# TODO: only find toml files
|
||||
|
||||
# filepath, language codes, slug
|
||||
files = [[f, cls.DEFAULT_LANGS, ""] for f in files]
|
||||
|
@ -242,7 +241,7 @@ class BlogPost(Searchable):
|
|||
files.remove(file)
|
||||
|
||||
# PERF:
|
||||
# Could possibly be done in one loop
|
||||
# Optimize for single loop, should be doable and better design
|
||||
|
||||
# collapse diffrent versions
|
||||
for file in files:
|
||||
|
@ -261,6 +260,7 @@ class BlogPost(Searchable):
|
|||
logger.error(
|
||||
f"Could not combine BlogPosts for '{file[0]}': {e}")
|
||||
|
||||
# TODO: not needed when relying on toml files
|
||||
try:
|
||||
# deduplicate
|
||||
_files = []
|
||||
|
|
Loading…
Reference in New Issue