diff options
Diffstat (limited to 'content/gemtext/gemfeed/2021-05-15-buetow.org.sh-one-bash-script-to-rule-it-all.draft.gmi')
| m--------- | content/gemtext | 6 | ||||
| -rw-r--r-- | content/gemtext/gemfeed/2021-05-15-buetow.org.sh-one-bash-script-to-rule-it-all.draft.gmi | 183 |
2 files changed, 6 insertions, 183 deletions
diff --git a/content/gemtext b/content/gemtext new file mode 160000 +Subproject faa2cd392016af9b76a4f40eeb12003785ced15 diff --git a/content/gemtext/gemfeed/2021-05-15-buetow.org.sh-one-bash-script-to-rule-it-all.draft.gmi b/content/gemtext/gemfeed/2021-05-15-buetow.org.sh-one-bash-script-to-rule-it-all.draft.gmi deleted file mode 100644 index 858478aa..00000000 --- a/content/gemtext/gemfeed/2021-05-15-buetow.org.sh-one-bash-script-to-rule-it-all.draft.gmi +++ /dev/null @@ -1,183 +0,0 @@ -# buetow.org.sh - One Bash script to rule it all - -> TODO: ADD WRITTEN BY AND CREATED AT BLABLA - -You might have read my previous blog post about entering the Geminispace. - -=> ./2021-04-24-welcome-to-the-geminispace Welcome to the Geminispace - -## Motivation - -Another benefit of using Gemini is that the Gemtext markup language is very easy to parse. As my site is dual hosted (Gemini+HTTP) I could in theory just write a shell script to deal with the conversion from Gemtext to HTML and not to rely on any external tools here. - -So I did exactly that, I wrote a Bash script which does the following: - -- Converts all Gemtext (*.gmi) files to HTML files -- Generates a Gemtext atom.xml feed for my blog posts -- Generates a HTML atom.xml feed of my blog posts - -I could have done all of that with a more powerful language than Bash (such as Perl, Ruby, Go...), but I didn't. The purpose of this exercise was to challenge what I can do with a "simple" Bash script and also to learn new things. - -``` - o .,<>., o - |\/\/\/\/| - '========' - (_ SSSSSSs - )a'`SSSSSs - /_ SSSSSS - .=## SSSSS - .#### SSSSs - ###::::SSSSS - .;:::""""SSS - .:;:' . . \\ - .::/ ' .'| - .::( . | - :::) \ - /\( / - /) ( | - .' \ . ./ / - _-' |\ . | - _..--.. . /"---\ | ` | . | - -=====================,' _ \=(*#(7.#####() | `/_.. , ( - _.-''``';'-''-) ,. \ ' '+/// | .'/ \ ``-.) \ - ,' _.- (( `-' `._\ `` \_/_.' ) /`-._ ) | - ,'\ ,' _.'.`:-. \.-' / <_L )" | - _/ `._,' ,')`; `-'`' | L / / - / `. ,' ,|_/ / \ ( <_-' \ - \ / `./ ' / /,' \ /|` `. | - )\ /`._ ,'`._.-\ |) \' - / `.' )-'.-,' )__) |\ `| - : /`. `.._(--.`':`':/ \ ) \ \ - |::::\ ,'/::;-)) / ( )`. | - ||::::: . .::': :`-( |/ . | - ||::::| . :| |==[]=: . - \ - |||:::| : || : | | /\ ` | - ___ ___ '|;:::| | |' \=[]=| / \ \ -| /_ ||``|||::::: | ; | | | \_.'\_ `-. -: \_``[]--[]|::::'\_;' )-'..`._ .-'\``:: ` . \ - \___.>`''-.||:.__,' SSt |_______`> <_____:::. . . \ _/ - `+a:f:......jrei''' -``` - -## W3C validator says all good -# -All generated HTML and Atom files pass the W3C validation. It is crazy that generating the Atom feed with valid XHTML content body for each blog posts was the most difficult part to implement in Bash. These formats are the reason why I decided to use Gemini as the primary protocol in the first place. However, Ironically I spent a couple of hours to get the XHTML and web Atoom feed working. To be fair, the Atom feed also works with Gemini. - -## Meta files for atom feed generation - -## Not without sed and grep and cut - -Soon I realised that I didn't want to go without a bit of grep and sed and cut. Regular expression matchings and simple string substitution tasks can be done in pure Bash but in my own opinion grep+sed are more powerful and easier to use (as I am used to these anyway). I managed not to use any AWK though. - -### Grepping - -I could use Bash's built-in regular expression matching engine here, but I am used to the grep pattern syntax, that's why I decided to do it this way: -``` -if grep -E -q "$IMAGE_PATTERN" <<< "$link"; then - html::img "$link" "$descr" - return -fi -``` - -### Sed-ing - -Sed comes in very handy for things like fixing HTML block text by replacing the lower than "<" and larger than ">" symbols with their corresponding HTML codes with one single command : - -``` -TODO: UPDATE SNIPPET echo "$line" | sed 's|<|\<|g; s|>|\>|g' -``` - -Sed is also useful in the following example, where the script checks whether the newly generated Atom feed file has changed compared to the previous version or not: - -``` -if ! diff -u <(sed 3d "$atom_file.tmp") <(sed 3d "$atom_file"); then - ... -else - ... -fi -``` - -### Cut-ing - -## Bash Modules for better structure - -I separated the script into different section; you could call them modules. For example, all functions dealing with the Atom feed are prefixed with atomfeed::, all functions dealing with HTML are prefixed with html:: and so on. - -As of writing this the script has the following modules and module functions: - -``` -TODO: UPDATE SNIPPET -❯ grep '::.* ()' buetow.org.sh -assert::equals () { -atom::meta () { -atom::generate () { -html::paragraph () { -html::heading () { -html::quote () { -html::img () { -html::link () { -html::gemini2html () { -html::generate () { -html::test () { -main::help () { -``` - -## Declaring all variables - -Many Bash scripts out in the wild don't have their variables declared, which leads to bad surprises as the default behaviour is that an undeclared variable is automatically a global variable once in use. So the best practise is to always declare a variable with one of the keywords "delcare", "readonly" or "local". - -Whole numbers can also have the option "-i", e.g. "declare -i num=52" and read only variables can be either declared via "readonly" or "rdeclare -r" or "local -r". Function local variables can also be declared with the "local" keyword. - -This is an example from the Atom module, where all variables are local to the function. I also make use of the "assign-then-shift"-pattern which goes like this: "local -r var1=$1; shift; local -r var2=$1; shift". The idea is that you only use "$1" to assign function arguments to named (better readable) local function variables. You will never have to bother about "$2" or above. That's is very useful when you constantly refactor your code and remove or add function arguments. It's something what I picked up from a colleague (a purely Bash wizard) some time ago: - -``` -atomfeed::meta () { - local -r now="$1"; shift - local -r gmi_file_path="$1"; shift - ... -} -``` - -## Unit tests - -Especially the Gemtext to HTML conversion part is an excellent use case for unit testing. There are unit tests for various Gemtext to HTML conversions (e.g. A header, paragraph, link, quote ...). My small unit test framework only consists of the test::assert() function. - -Forces to think creatively and to keep features fairly simple (good things) - -## De-facto templates - -## It's a static website generator - -Generate statically on my laptop and commit all statically generated files to fit. Can also preview locally. - -A lot of bash tricks - -## Config file - -## Learnings from ShellCheck - -ShellSheck: Not happy with all recommentations but most, e.g. read -r, quotes, etc. - -### While-read loops - -Specify -r - -### Warnings about variables not quoted - -### if cmd; then - -## The result(s) - -### Gemtext via Gemini protocol - -=> gemini://buetow.org gemini://buetow.org - The original Gemini capsule -=> gemini://buetow.org/gemfeed/ gemini://buetow.org/gemfeed/ - The Gemfeed -=> gemini://buetow.org/gemfeed/atom.xml gemini://buetow.org/gemfeed/atom.xml - The Atom feed - -### XHTML via HTTP protocol - -=> https://buetow.org https://buetow.org - The original Gemini capsule -=> https://buetow.org/gemfeed/ https://buetow.org/gemfeed/ - The Gemfeed -=> https://buetow.org/gemfeed/atom.xml https://buetow.org/gemfeed/atom.xml - The Atom feed - -TODO: ADD GO BACK LINK |