blog-markdown #31
|
@ -0,0 +1,102 @@
|
||||||
|
---
|
||||||
|
Title: Bash Arrays
|
||||||
|
Subtitle: sub
|
||||||
|
Desc: Brief intro to Bash Arrays
|
||||||
|
Date: 2023-09-29
|
||||||
|
Keywords: bash
|
||||||
|
technology
|
||||||
|
Category: Test
|
||||||
|
Featured: True
|
||||||
|
Public: True
|
||||||
|
---
|
||||||
|
|
||||||
|
**NOTE**
|
||||||
|
|
||||||
|
This is a stolen article from [opensource.com](https://opensource.com/article/18/5/you-dont-know-bash-intro-bash-arrays)
|
||||||
|
about bash scripting. It's a good article and I've decided to use it to test my
|
||||||
|
markdown rendering.
|
||||||
|
|
||||||
|
# Bash scripting
|
||||||
|
|
||||||
|
[TOC]
|
||||||
|
|
||||||
|
|
||||||
|
## Wait, but why?
|
||||||
|
|
||||||
|
Writing about Bash is challenging because it's remarkably easy for an article
|
||||||
|
to devolve into a manual that focuses on syntax oddities. Rest assured,
|
||||||
|
however, the intent of this article is to avoid having you RTFM.
|
||||||
|
|
||||||
|
## A real (actually useful) example
|
||||||
|
|
||||||
|
To that end, let's consider a real-world scenario and how Bash can help:
|
||||||
|
You are leading a new effort at your company to evaluate and optimize the
|
||||||
|
runtime of your internal data pipeline. As a first step, you want to do a
|
||||||
|
parameter sweep to evaluate how well the pipeline makes use of threads. For
|
||||||
|
the sake of simplicity, we'll treat the pipeline as a compiled C++ black box
|
||||||
|
where the only parameter we can tweak is the number of threads reserved for
|
||||||
|
data processing: `./pipeline --threads 4.`
|
||||||
|
|
||||||
|
## The basics
|
||||||
|
|
||||||
|
The first thing we'll do is define an array containing the values of the
|
||||||
|
`--threads` parameter that we want to test:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
allThreads=(1 2 4 8 16 32 64 128)
|
||||||
|
```
|
||||||
|
|
||||||
|
In this example, all the elements are numbers, but it need not be the
|
||||||
|
case—arrays in Bash can contain both numbers and strings, e.g., `myArray=(1
|
||||||
|
2 "three" 4 "five")` is a valid expression. And just as with any other Bash
|
||||||
|
variable, make sure to leave no spaces around the equal sign. Otherwise,
|
||||||
|
Bash will treat the variable name as a program to execute, and the `=` as its
|
||||||
|
first parameter!
|
||||||
|
|
||||||
|
Now that we've initialized the array, let's retrieve a few of its
|
||||||
|
elements. You'll notice that simply doing `echo $allThreads` will output only
|
||||||
|
the first element.
|
||||||
|
|
||||||
|
To understand why that is, let's take a step back and revisit how we usually
|
||||||
|
output variables in Bash. Consider the following scenario:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
type="article" echo "Found 42 $type"
|
||||||
|
```
|
||||||
|
|
||||||
|
Say the variable $type is given to us as a singular noun and we want to add
|
||||||
|
an `s` at the end of our sentence. We can't simply add an s to `$type` since
|
||||||
|
that would turn it into a different variable, `$types`. And although we could
|
||||||
|
utilize code contortions such as `echo "Found 42 "$type"s"`, the best way
|
||||||
|
to solve this problem is to use curly braces: `echo "Found 42 ${type}s"`,
|
||||||
|
which allows us to tell Bash where the name of a variable starts and ends
|
||||||
|
(interestingly, this is the same syntax used in JavaScript/ES6 to inject
|
||||||
|
variables and expressions in template literals).
|
||||||
|
|
||||||
|
So as it turns out, although Bash variables don't generally require curly
|
||||||
|
brackets, they are required for arrays. In turn, this allows us to specify
|
||||||
|
the index to access, e.g., `echo ${allThreads[1]}` returns the second element
|
||||||
|
of the array. Not including brackets, e.g.,`echo $allThreads[1]`, leads Bash
|
||||||
|
to treat `[1]` as a string and output it as such.
|
||||||
|
|
||||||
|
Yes, Bash arrays have odd syntax, but at least they are zero-indexed, unlike
|
||||||
|
some other languages (I'm looking at you, R).[^1]
|
||||||
|
|
||||||
|
## Looping through arrays
|
||||||
|
|
||||||
|
Although in the examples above we used integer indices in our arrays, let's
|
||||||
|
consider two occasions when that won't be the case: First, if we wanted the
|
||||||
|
$i-th element of the array, where $i is a variable containing the index of
|
||||||
|
interest, we can retrieve that element using: echo ${allThreads[$i]}. Second,
|
||||||
|
to output all the elements of an array, we replace the numeric index with
|
||||||
|
the @ symbol (you can think of @ as standing for all):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
type="article"
|
||||||
|
echo "Found 42 $type"
|
||||||
|
```
|
||||||
|
|
||||||
|
*[RTFM]: Read the Fucking Manual
|
||||||
|
*[HTML]: Hyper Text Markup Language
|
||||||
|
|
||||||
|
[^1]: Example Footnote
|
|
@ -1,86 +1,102 @@
|
||||||
---
|
---
|
||||||
Title: test
|
Title: Bash Arrays
|
||||||
Subtitle: sub
|
Subtitle: sub
|
||||||
Desc: brief desc
|
Desc: Brief intro to Bash Arrays
|
||||||
Date: 2023-09-27
|
Date: 2023-09-29
|
||||||
Keywords: main
|
Keywords: bash
|
||||||
foo
|
technology
|
||||||
bar
|
|
||||||
qux
|
|
||||||
Category: Test
|
Category: Test
|
||||||
Featured: True
|
Featured: True
|
||||||
Public: True
|
Public: True
|
||||||
---
|
---
|
||||||
|
|
||||||
# Test article
|
**NOTE**
|
||||||
----------------------------------------
|
|
||||||
This is a crazy testy
|
This is a stolen article from [opensource.com](https://opensource.com/article/18/5/you-dont-know-bash-intro-bash-arrays)
|
||||||
|
about bash scripting. It's a good article and I've decided to use it to test my
|
||||||
|
markdown rendering.
|
||||||
|
|
||||||
|
# Bash scripting
|
||||||
|
|
||||||
### toc?
|
|
||||||
[TOC]
|
[TOC]
|
||||||
|
|
||||||
## foo
|
|
||||||
|
|
||||||
HTML
|
## Wait, but why?
|
||||||
|
|
||||||
lipsum[^1]
|
Writing about Bash is challenging because it's remarkably easy for an article
|
||||||
|
to devolve into a manual that focuses on syntax oddities. Rest assured,
|
||||||
|
however, the intent of this article is to avoid having you RTFM.
|
||||||
|
|
||||||
<div>INLINE DIV<b>bold</b></div>
|
## A real (actually useful) example
|
||||||
|
|
||||||
### bar bar
|
To that end, let's consider a real-world scenario and how Bash can help:
|
||||||
|
You are leading a new effort at your company to evaluate and optimize the
|
||||||
|
runtime of your internal data pipeline. As a first step, you want to do a
|
||||||
|
parameter sweep to evaluate how well the pipeline makes use of threads. For
|
||||||
|
the sake of simplicity, we'll treat the pipeline as a compiled C++ black box
|
||||||
|
where the only parameter we can tweak is the number of threads reserved for
|
||||||
|
data processing: `./pipeline --threads 4.`
|
||||||
|
|
||||||
| important | table |
|
## The basics
|
||||||
|-----------|-------|
|
|
||||||
| value | 2 |
|
|
||||||
| 3 | 4 |
|
|
||||||
| 5 | 2 |
|
|
||||||
|
|
||||||
## foo
|
The first thing we'll do is define an array containing the values of the
|
||||||
|
`--threads` parameter that we want to test:
|
||||||
|
|
||||||
#### big code
|
```bash
|
||||||
|
allThreads=(1 2 4 8 16 32 64 128)
|
||||||
```py
|
|
||||||
from django.db import models
|
|
||||||
from django.utils.translation import gettext as _
|
|
||||||
from start.models import Searchable
|
|
||||||
|
|
||||||
import logging
|
|
||||||
logger = logging.getLogger(__name__)
|
|
||||||
|
|
||||||
import markdown
|
|
||||||
ARTICLE_DIR = "/app/blog/data/articles"
|
|
||||||
EXTENSIONS = [
|
|
||||||
"extra",
|
|
||||||
"admonition",
|
|
||||||
"codehilite",
|
|
||||||
"toc"
|
|
||||||
]
|
|
||||||
|
|
||||||
class Category(models.Model):
|
|
||||||
"""
|
|
||||||
A category of blog posts
|
|
||||||
|
|
||||||
Name not translated because it would make i18n in urls and Searchables specifically a pain.
|
|
||||||
Maybe some day it would be cool if these were Searchable
|
|
||||||
"""
|
|
||||||
name= models.CharField(max_length=50)
|
|
||||||
slug = models.SlugField()
|
|
||||||
|
|
||||||
class Meta:
|
|
||||||
verbose_name = _("Category")
|
|
||||||
verbose_name_plural = _("Categories")
|
|
||||||
|
|
||||||
def __str__(self):
|
|
||||||
return f"{{<{self.__class__.__name__}>\"{self.name}\"}}"
|
|
||||||
```
|
```
|
||||||
|
|
||||||
### a
|
In this example, all the elements are numbers, but it need not be the
|
||||||
bar
|
case—arrays in Bash can contain both numbers and strings, e.g., `myArray=(1
|
||||||
#### a
|
2 "three" 4 "five")` is a valid expression. And just as with any other Bash
|
||||||
qux
|
variable, make sure to leave no spaces around the equal sign. Otherwise,
|
||||||
###### a
|
Bash will treat the variable name as a program to execute, and the `=` as its
|
||||||
baau
|
first parameter!
|
||||||
|
|
||||||
|
Now that we've initialized the array, let's retrieve a few of its
|
||||||
|
elements. You'll notice that simply doing `echo $allThreads` will output only
|
||||||
|
the first element.
|
||||||
|
|
||||||
|
To understand why that is, let's take a step back and revisit how we usually
|
||||||
|
output variables in Bash. Consider the following scenario:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
type="article" echo "Found 42 $type"
|
||||||
|
```
|
||||||
|
|
||||||
|
Say the variable $type is given to us as a singular noun and we want to add
|
||||||
|
an `s` at the end of our sentence. We can't simply add an s to `$type` since
|
||||||
|
that would turn it into a different variable, `$types`. And although we could
|
||||||
|
utilize code contortions such as `echo "Found 42 "$type"s"`, the best way
|
||||||
|
to solve this problem is to use curly braces: `echo "Found 42 ${type}s"`,
|
||||||
|
which allows us to tell Bash where the name of a variable starts and ends
|
||||||
|
(interestingly, this is the same syntax used in JavaScript/ES6 to inject
|
||||||
|
variables and expressions in template literals).
|
||||||
|
|
||||||
|
So as it turns out, although Bash variables don't generally require curly
|
||||||
|
brackets, they are required for arrays. In turn, this allows us to specify
|
||||||
|
the index to access, e.g., `echo ${allThreads[1]}` returns the second element
|
||||||
|
of the array. Not including brackets, e.g.,`echo $allThreads[1]`, leads Bash
|
||||||
|
to treat `[1]` as a string and output it as such.
|
||||||
|
|
||||||
|
Yes, Bash arrays have odd syntax, but at least they are zero-indexed, unlike
|
||||||
|
some other languages (I'm looking at you, R).[^1]
|
||||||
|
|
||||||
|
## Looping through arrays
|
||||||
|
|
||||||
|
Although in the examples above we used integer indices in our arrays, let's
|
||||||
|
consider two occasions when that won't be the case: First, if we wanted the
|
||||||
|
$i-th element of the array, where $i is a variable containing the index of
|
||||||
|
interest, we can retrieve that element using: echo ${allThreads[$i]}. Second,
|
||||||
|
to output all the elements of an array, we replace the numeric index with
|
||||||
|
the @ symbol (you can think of @ as standing for all):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
type="article"
|
||||||
|
echo "Found 42 $type"
|
||||||
|
```
|
||||||
|
|
||||||
|
*[RTFM]: Read the Fucking Manual
|
||||||
*[HTML]: Hyper Text Markup Language
|
*[HTML]: Hyper Text Markup Language
|
||||||
|
|
||||||
[^1]: foooootnote
|
[^1]: Example Footnote
|
||||||
|
|
|
@ -74,56 +74,58 @@ class BlogPost(Searchable):
|
||||||
"""
|
"""
|
||||||
logger.info(f"regenerating article from markdown for: {self}")
|
logger.info(f"regenerating article from markdown for: {self}")
|
||||||
try:
|
try:
|
||||||
f_en = open(f"{ARTICLE_DIR}/en-{self.slug}.md")
|
with open(f"{ARTICLE_DIR}/en-{self.slug}.md") as f_en:
|
||||||
|
|
||||||
body_en: str = f_en.read()
|
body_en: str = f_en.read()
|
||||||
|
|
||||||
html_en: str = MD.convert(body_en)
|
html_en: str = MD.convert(body_en)
|
||||||
try:
|
try:
|
||||||
meta_en = MD.Meta
|
meta_en = MD.Meta
|
||||||
self.title_en = meta_en["title"][0]
|
self.title_en = meta_en["title"][0]
|
||||||
self.subtitle_en = meta_en["subtitle"][0]
|
self.subtitle_en = meta_en["subtitle"][0]
|
||||||
self.desc_en = meta_en["desc"][0]
|
self.desc_en = meta_en["desc"][0]
|
||||||
# TODO: parse date from markdown
|
# TODO: parse date from markdown
|
||||||
self.featured = meta_en["featured"][0] == "True"
|
self.featured = meta_en["featured"][0] == "True"
|
||||||
self.public = meta_en["public"][0] == "True"
|
self.public = meta_en["public"][0] == "True"
|
||||||
# TODO: parse keywords from markdown
|
# TODO: parse keywords from markdown
|
||||||
# TODO: parse category from markdown
|
# TODO: parse category from markdown
|
||||||
|
|
||||||
# if keyword or category do not exist, create them
|
# if keyword or category do not exist, create them
|
||||||
# I suppose
|
# I suppose
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.warning(f"could not generate metadata {self.slug} from markdown: {e}")
|
logger.warning(f"could not generate metadata {self.slug} from markdown: {e}")
|
||||||
|
|
||||||
self.body_en = html_en
|
self.body_en = ""
|
||||||
|
self.body_en = html_en
|
||||||
except FileNotFoundError as e:
|
except FileNotFoundError as e:
|
||||||
# TODO: mark as untranslated
|
# TODO: mark as untranslated
|
||||||
pass
|
pass
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.warning(f"could not generate article {self.slug} from markdown: {e}")
|
logger.warning(f"could not generate article {self.slug} from markdown: {e}")
|
||||||
try:
|
try:
|
||||||
f_de = open(f"{ARTICLE_DIR}/de-{self.slug}.md")
|
with open(f"{ARTICLE_DIR}/de-{self.slug}.md") as f_de:
|
||||||
|
|
||||||
body_de: str = f_de.read()
|
body_de: str = f_de.read()
|
||||||
|
|
||||||
html_de: str = MD.convert(body_de)
|
html_de: str = MD.convert(body_de)
|
||||||
try:
|
try:
|
||||||
meta_de = MD.Meta
|
meta_de = MD.Meta
|
||||||
self.title_de = meta_de["title"][0]
|
self.title_de = meta_de["title"][0]
|
||||||
self.subtitle_de = meta_de["subtitle"][0]
|
self.subtitle_de = meta_de["subtitle"][0]
|
||||||
self.desc_de = meta_de["desc"][0]
|
self.desc_de = meta_de["desc"][0]
|
||||||
# TODO: parse date from markdown
|
# TODO: parse date from markdown
|
||||||
self.featured = meta_de["featured"][0] == "True"
|
self.featured = meta_de["featured"][0] == "True"
|
||||||
self.public = meta_de["public"][0] == "True"
|
self.public = meta_de["public"][0] == "True"
|
||||||
# TODO: parse keywords from markdown
|
# TODO: parse keywords from markdown
|
||||||
# TODO: parse category from markdown
|
# TODO: parse category from markdown
|
||||||
|
|
||||||
# if keyword or category do not exist, create them
|
# if keyword or category do not exist, create them
|
||||||
# I suppose
|
# I suppose
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.warning(f"could not generate metadata {self.slug} from markdown: {e}")
|
logger.warning(f"could not generate metadata {self.slug} from markdown: {e}")
|
||||||
|
|
||||||
self.body_de = html_de
|
self.body_de = ""
|
||||||
|
self.body_de = html_de
|
||||||
except FileNotFoundError as e:
|
except FileNotFoundError as e:
|
||||||
# TODO: mark as untranslated
|
# TODO: mark as untranslated
|
||||||
pass
|
pass
|
||||||
|
|
Loading…
Reference in New Issue