diff options
| author | Paul Buetow <git@mx.buetow.org> | 2021-06-05 19:10:40 +0100 |
|---|---|---|
| committer | Paul Buetow <git@mx.buetow.org> | 2021-06-05 19:10:40 +0100 |
| commit | d6876775097ff175e3c2f590687eebcf6302c983 (patch) | |
| tree | fd124e0ab99bf0b84ed7f281d956704f207cacf5 | |
| parent | 5f3dc597fea303030361e17ad25598529681d496 (diff) | |
Publish blog post about Gemtexter
| -rw-r--r-- | gemfeed/2021-04-24-welcome-to-the-geminispace.html | 4 | ||||
| -rw-r--r-- | gemfeed/2021-05-15-buetow.org.sh-one-bash-script-to-rule-it-all.draft.html | 233 | ||||
| -rw-r--r-- | gemfeed/2021-06-05-gemtexter-one-bash-script-to-rule-it-all.html | 224 | ||||
| -rw-r--r-- | gemfeed/2021-06-05-gemtexter-one-bash-script-to-rule-it-all/blog-engine.jpg | bin | 0 -> 41291 bytes | |||
| -rw-r--r-- | gemfeed/atom.xml | 144 | ||||
| -rw-r--r-- | gemfeed/index.html | 3 | ||||
| -rw-r--r-- | index.html | 6 |
7 files changed, 376 insertions, 238 deletions
diff --git a/gemfeed/2021-04-24-welcome-to-the-geminispace.html b/gemfeed/2021-04-24-welcome-to-the-geminispace.html index f4e0d194..2149f448 100644 --- a/gemfeed/2021-04-24-welcome-to-the-geminispace.html +++ b/gemfeed/2021-04-24-welcome-to-the-geminispace.html @@ -86,7 +86,9 @@ h2, h3 { <p>Why is there a need for a new protocol? As the modern web is a superset of Gemini, can't we use simple HTML 1.0 instead? That's a good and valid question. It is not a technical problem but a human problem. We tend to abuse the features once they are available. You can ensure that things stay efficient and straightforward as long as you are using the Gemini protocol. On the other hand, you can't force every website on the modern web to only create plain and straightforward-looking HTML pages.</p> <h2>My own Gemini capsule</h2> <p>As it is effortless to set up and maintain your own Gemini capsule (Gemini server + content composed via the Gemtext markup language), I decided to create my own. What I like about Gemini is that I can use my favourite text editor and get typing. I don't need to worry about the style and design of the presence, and I also don't have to test anything in ten different web browsers. I can only focus on the content! As a matter of fact, I am using the Vim editor + its spellchecker + auto word completion functionality to write this. </p> -<h2>Advantages summarised</h2> +<p>This site was generated with Gemtexter. You can read more about it here:</p> +<a class="textlink" href="./2021-06-05-gemtexter-one-bash-script-to-rule-it-all.html">Gemtexter - One Bash script to rule it all</a><br /> +<h2>Gemini advantages summarised</h2> <ul> <li>Supports an alternative to the modern bloated web</li> <li>Easy to operate and easy to write content</li> diff --git a/gemfeed/2021-05-15-buetow.org.sh-one-bash-script-to-rule-it-all.draft.html b/gemfeed/2021-05-15-buetow.org.sh-one-bash-script-to-rule-it-all.draft.html deleted file mode 100644 index 78efccef..00000000 --- a/gemfeed/2021-05-15-buetow.org.sh-one-bash-script-to-rule-it-all.draft.html +++ /dev/null @@ -1,233 +0,0 @@ -<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> -<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"> -<head> -<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> -<title>buetow.org.sh - One Bash script to rule it all</title> -<link rel="shortcut icon" type="image/gif" href="/favicon.ico" /> -<style type="text/css"> -body { - margin: auto; - max-width: 900px; - background-color: #FFFFEF; - border: 1px dashed #880000; - border-radius: 8px; - padding: 5px; -} -img { - display:block; - max-width: 80%; -} -a.textlink:before { - content: " ⇒ "; - padding-left: 2px; -} -a.textlink { - text-decoration: none; - color: #FF0000; -} -a.textlink:hover { - text-decoration: underline; -} -i { - color: #48AAAD; -} -pre { - background-color: #F1F8E9; - border: 1px dashed #BB0000; - border-radius: 8px; - padding: 5px; - font-family: "Lucida Console", "Courier New", monospace; -} -h1 { - text-align: center; - color: #880000; -} -h2, h3 { - color: #BB0000; -} -</style> -</head> -<body> -<h1>buetow.org.sh - One Bash script to rule it all</h1> -<p class="quote"><i>TODO: ADD WRITTEN BY AND CREATED AT BLABLA</i></p> -<p>You might have read my previous blog post about entering the Geminispace.</p> -<a class="textlink" href="./2021-04-24-welcome-to-the-geminispace">Welcome to the Geminispace</a><br /> -<h2>Motivation</h2> -<p>Another benefit of using Gemini is that the Gemtext markup language is very easy to parse. As my site is dual hosted (Gemini+HTTP) I could in theory just write a shell script to deal with the conversion from Gemtext to HTML and not to rely on any external tools here.</p> -<p>So I did exactly that, I wrote a Bash script which does the following:</p> -<p>- Converts all Gemtext (*.gmi) files to HTML files</p> -<p>- Generates a Gemtext atom.xml feed for my blog posts</p> -<p>- Generates a HTML atom.xml feed of my blog posts</p> -<p>I could have done all of that with a more powerful language than Bash (such as Perl, Ruby, Go...), but I didn't. The purpose of this exercise was to challenge what I can do with a "simple" Bash script and also to learn new things.</p> -<pre> - o .,<>., o - |\/\/\/\/| - '========' - (_ SSSSSSs - )a'`SSSSSs - /_ SSSSSS - .=## SSSSS - .#### SSSSs - ###::::SSSSS - .;:::""""SSS - .:;:' . . \\ - .::/ ' .'| - .::( . | - :::) \ - /\( / - /) ( | - .' \ . ./ / - _-' |\ . | - _..--.. . /"---\ | ` | . | - -=====================,' _ \=(*#(7.#####() | `/_.. , ( - _.-''``';'-''-) ,. \ ' '+/// | .'/ \ ``-.) \ - ,' _.- (( `-' `._\ `` \_/_.' ) /`-._ ) | - ,'\ ,' _.'.`:-. \.-' / <_L )" | - _/ `._,' ,')`; `-'`' | L / / - / `. ,' ,|_/ / \ ( <_-' \ - \ / `./ ' / /,' \ /|` `. | - )\ /`._ ,'`._.-\ |) \' - / `.' )-'.-,' )__) |\ `| - : /`. `.._(--.`':`':/ \ ) \ \ - |::::\ ,'/::;-)) / ( )`. | - ||::::: . .::': :`-( |/ . | - ||::::| . :| |==[]=: . - \ - |||:::| : || : | | /\ ` | - ___ ___ '|;:::| | |' \=[]=| / \ \ -| /_ ||``|||::::: | ; | | | \_.'\_ `-. -: \_``[]--[]|::::'\_;' )-'..`._ .-'\``:: ` . \ - \___.>`''-.||:.__,' SSt |_______`> <_____:::. . . \ _/ - `+a:f:......jrei''' -</pre> -<h2>W3C validator says all good</h2> -<p>#</p> -<p>All generated HTML and Atom files pass the W3C validation. It is crazy that generating the Atom feed with valid XHTML content body for each blog posts was the most difficult part to implement in Bash. These formats are the reason why I decided to use Gemini as the primary protocol in the first place. However, Ironically I spent a couple of hours to get the XHTML and web Atoom feed working. To be fair, the Atom feed also works with Gemini.</p> -<h2>Meta files for atom feed generation</h2> -<h2>Not without sed and grep and cut</h2> -<p>Soon I realised that I didn't want to go without a bit of grep and sed and cut. Regular expression matchings and simple string substitution tasks can be done in pure Bash but in my own opinion grep+sed are more powerful and easier to use (as I am used to these anyway). I managed not to use any AWK though.</p> -<h3>Grepping</h3> -<p>I could use Bash's built-in regular expression matching engine here, but I am used to the grep pattern syntax, that's why I decided to do it this way:</p> -<pre> -if grep -E -q "$IMAGE_PATTERN" <<< "$link"; then - html::img "$link" "$descr" - return -fi -</pre> -<h3>Sed-ing</h3> -<p>Sed comes in very handy for things like fixing HTML block text by replacing the lower than "<" and larger than ">" symbols with their corresponding HTML codes with one single command :</p> -<pre> -TODO: UPDATE SNIPPET echo "$line" | sed 's|<|\&lt;|g; s|>|\&gt;|g' -</pre> -<p>Sed is also useful in the following example, where the script checks whether the newly generated Atom feed file has changed compared to the previous version or not:</p> -<pre> -if ! diff -u <(sed 3d "$atom_file.tmp") <(sed 3d "$atom_file"); then - ... -else - ... -fi -</pre> -<h3>Cut-ing</h3> -<h2>Bash Modules for better structure</h2> -<p>I separated the script into different section; you could call them modules. For example, all functions dealing with the Atom feed are prefixed with atomfeed::, all functions dealing with HTML are prefixed with html:: and so on.</p> -<p>As of writing this the script has the following modules and module functions:</p> -<pre> -TODO: UPDATE SNIPPET -❯ grep '::.* ()' buetow.org.sh -assert::equals () { -atom::meta () { -atom::generate () { -html::paragraph () { -html::heading () { -html::quote () { -html::img () { -html::link () { -html::gemini2html () { -html::generate () { -html::test () { -main::help () { -</pre> -<h2>Declaring all variables</h2> -<p>Many Bash scripts out in the wild don't have their variables declared, which leads to bad surprises as the default behaviour is that an undeclared variable is automatically a global variable once in use. So the best practise is to always declare a variable with one of the keywords "delcare", "readonly" or "local".</p> -<p>Whole numbers can also have the option "-i", e.g. "declare -i num=52" and read only variables can be either declared via "readonly" or "rdeclare -r" or "local -r". Function local variables can also be declared with the "local" keyword. </p> -<p>This is an example from the Atom module, where all variables are local to the function. I also make use of the "assign-then-shift"-pattern which goes like this: "local -r var1=$1; shift; local -r var2=$1; shift". The idea is that you only use "$1" to assign function arguments to named (better readable) local function variables. You will never have to bother about "$2" or above. That's is very useful when you constantly refactor your code and remove or add function arguments. It's something what I picked up from a colleague (a purely Bash wizard) some time ago:</p> -<pre> -atomfeed::meta () { - local -r now="$1"; shift - local -r gmi_file_path="$1"; shift - ... -} -</pre> -<h2>Unit tests</h2> -<p>Especially the Gemtext to HTML conversion part is an excellent use case for unit testing. There are unit tests for various Gemtext to HTML conversions (e.g. A header, paragraph, link, quote ...). My small unit test framework only consists of the test::assert() function.</p> -<p>Forces to think creatively and to keep features fairly simple (good things)</p> -<h2>De-facto templates</h2> -<h2>It's a static website generator</h2> -<p>Generate statically on my laptop and commit all statically generated files to fit. Can also preview locally.</p> -<p>A lot of bash tricks</p> -<h2>Config file</h2> -<h2>Learnings from ShellCheck</h2> -<p>ShellSheck: Not happy with all recommentations but most, e.g. read -r, quotes, etc.</p> -<h3>While-read loops</h3> -<p>Specify -r</p> -<h3>Warnings about variables not quoted</h3> -<h3>if cmd; then</h3> -<h2>The result(s)</h2> -<h3>Gemtext via Gemini protocol</h3> -<a class="textlink" href="gemini://buetow.org">gemini://buetow.org - The original Gemini capsule </a><br /> -<a class="textlink" href="gemini://buetow.org/gemfeed/">gemini://buetow.org/gemfeed/ - The Gemfeed</a><br /> -<a class="textlink" href="gemini://buetow.org/gemfeed/atom.xml">gemini://buetow.org/gemfeed/atom.xml - The Atom feed</a><br /> -<h3>XHTML via HTTP protocol</h3> -<a class="textlink" href="https://buetow.org">https://buetow.org - The original Gemini capsule </a><br /> -<a class="textlink" href="https://buetow.org/gemfeed/">https://buetow.org/gemfeed/ - The Gemfeed</a><br /> -<a class="textlink" href="https://buetow.org/gemfeed/atom.xml">https://buetow.org/gemfeed/atom.xml - The Atom feed</a><br /> -<p>TODO: ADD GO BACK LINK</p> -<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> -<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"> -<head> -<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> -<title>buetow.org.sh - One Bash script to rule it all</title> -<link rel="shortcut icon" type="image/gif" href="/favicon.ico" /> -<style type="text/css"> -body { - margin: auto; - max-width: 900px; - background-color: #FFFFEF; - border: 1px dashed #880000; - border-radius: 8px; - padding: 5px; -} -img { - display:block; - max-width: 80%; -} -a.textlink:before { - content: " ⇒ "; - padding-left: 2px; -} -a.textlink { - text-decoration: none; - color: #FF0000; -} -a.textlink:hover { - text-decoration: underline; -} -i { - color: #48AAAD; -} -pre { - background-color: #F1F8E9; - border: 1px dashed #BB0000; - border-radius: 8px; - padding: 5px; - font-family: "Lucida Console", "Courier New", monospace; -} -h1 { - text-align: center; - color: #880000; -} -h2, h3 { - color: #BB0000; -} -</style> -</head> -<body> diff --git a/gemfeed/2021-06-05-gemtexter-one-bash-script-to-rule-it-all.html b/gemfeed/2021-06-05-gemtexter-one-bash-script-to-rule-it-all.html new file mode 100644 index 00000000..c65d4d24 --- /dev/null +++ b/gemfeed/2021-06-05-gemtexter-one-bash-script-to-rule-it-all.html @@ -0,0 +1,224 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> +<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"> +<head> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<title>Gemtexter - One Bash script to rule it all</title> +<link rel="shortcut icon" type="image/gif" href="/favicon.ico" /> +<style type="text/css"> +body { + margin: auto; + max-width: 900px; + background-color: #FFFFEF; + border: 1px dashed #880000; + border-radius: 8px; + padding: 5px; +} +img { + display:block; + max-width: 80%; +} +a.textlink:before { + content: " ⇒ "; + padding-left: 2px; +} +a.textlink { + text-decoration: none; + color: #FF0000; +} +a.textlink:hover { + text-decoration: underline; +} +i { + color: #48AAAD; +} +pre { + background-color: #F1F8E9; + border: 1px dashed #BB0000; + border-radius: 8px; + padding: 5px; + font-family: "Lucida Console", "Courier New", monospace; +} +h1 { + text-align: center; + color: #880000; +} +h2, h3 { + color: #BB0000; +} +</style> +</head> +<body> +<h1>Gemtexter - One Bash script to rule it all</h1> +<pre> + o .,<>., o + |\/\/\/\/| + '========' + (_ SSSSSSs + )a'`SSSSSs + /_ SSSSSS + .=## SSSSS + .#### SSSSs + ###::::SSSSS + .;:::""""SSS + .:;:' . . \\ + .::/ ' .'| + .::( . | + :::) \ + /\( / + /) ( | + .' \ . ./ / + _-' |\ . | + _..--.. . /"---\ | ` | . | + -=====================,' _ \=(*#(7.#####() | `/_.. , ( + _.-''``';'-''-) ,. \ ' '+/// | .'/ \ ``-.) \ + ,' _.- (( `-' `._\ `` \_/_.' ) /`-._ ) | + ,'\ ,' _.'.`:-. \.-' / <_L )" | + _/ `._,' ,')`; `-'`' | L / / + / `. ,' ,|_/ / \ ( <_-' \ + \ / `./ ' / /,' \ /|` `. | + )\ /`._ ,'`._.-\ |) \' + / `.' )-'.-,' )__) |\ `| + : /`. `.._(--.`':`':/ \ ) \ \ + |::::\ ,'/::;-)) / ( )`. | + ||::::: . .::': :`-( |/ . | + ||::::| . :| |==[]=: . - \ + |||:::| : || : | | /\ ` | + ___ ___ '|;:::| | |' \=[]=| / \ \ +| /_ ||``|||::::: | ; | | | \_.'\_ `-. +: \_``[]--[]|::::'\_;' )-'..`._ .-'\``:: ` . \ + \___.>`''-.||:.__,' SSt |_______`> <_____:::. . . \ _/ + `+a:f:......jrei''' +</pre> +<p class="quote"><i>Written by Paul Buetow 2021-06-05</i></p> +<p>You might have read my previous blog post about entering the Geminispace, where I pointed out the benefits of having and maintaining an internet presence there. This whole site (the blog and all other pages) is composed in the Gemtext markup language. </p> +<a class="textlink" href="./2021-04-24-welcome-to-the-geminispace.html">Welcome to the Geminispace</a><br /> +<p>This comes with the benefit that I can write content in my favourite text editor (Vim). </p> +<h2>Motivation</h2> +<p>Another benefit of using Gemini is that the Gemtext markup language is easy to parse. As my site is dual-hosted (Gemini+HTTP), I could, in theory, just write a shell script to deal with the conversion from Gemtext to HTML; there is no need for a full-featured programming language here. I have done a lot of Bash in the past, but I am also often revisiting old tools and techniques for refreshing and keeping the knowledge up to date here.</p> +<i>Motivational comic strip:</i><a href="./2021-06-05-gemtexter-one-bash-script-to-rule-it-all/blog-engine.jpg"><img alt="Motivational comic strip" title="Motivational comic strip" src="./2021-06-05-gemtexter-one-bash-script-to-rule-it-all/blog-engine.jpg" /></a><br /> +<p>I have exactly done that - I wrote a Bash script, named gemtexter, for that. Have a look at GitHub:</p> +<a class="textlink" href="https://github.com/snonux/gemtexter">https://github.com/snonux/gemtexter</a><br /> +<p>In short, Gemtexter is a static site generator and blogging engine that uses Gemtext as its input format.</p> +<h2>Output formats</h2> +<p>Gemtexter takes the Gemntext Markup files as the input and generates the following outputs from it (you find examples for each of these output formats on the Gemtexter GitHub page):</p> +<ul> +<li>HTML files for my website</li> +<li>Markdown files for a GitHub page</li> +<li>A Gemtext Atom feed for my blog posts</li> +<li>A Gemfeed for my blog posts (a particular feed format commonly used in Geminispace. The Gemfeed can be used as an alternative to the Atom feed).</li> +<li>An HTML Atom feed of my blog posts</li> +</ul> +<p>I could have done all of that with a more robust language than Bash (such as Perl, Ruby, Go...), but I didn't. The purpose of this exercise was to challenge what I can do with a "simple" Bash script and learn new things.</p> +<h2>Taking it as far as I should, but no further</h2> +<p>The Bash is suitable very well for small scripts and ad-hoc automation on the command line. But it is for sure not a robust programming language. Writing this blog post, Gemtexter is nearing 1000 lines of code, which is actually a pretty large Bash script.</p> +<h3>Modularization </h3> +<p>I modularized the code so that each core functionality has its own file in ./lib. All the modules are included (sourced) from the main Gemtexter script. For example, there is one module for HTML generation, one for Markdown generation, and so on. </p> +<pre> +paul in uranus in gemtexter on 🌱 main +❯ wc -l gemtexter lib/* + 117 gemtexter + 59 lib/assert.source.sh + 128 lib/atomfeed.source.sh + 64 lib/gemfeed.source.sh + 161 lib/generate.source.sh + 50 lib/git.source.sh + 162 lib/html.source.sh + 30 lib/log.source.sh + 63 lib/md.source.sh + 834 total +</pre> +<p>This way, the script could grow far beyond 1000 lines of code and still be maintainable. With more features, execution speed may slowly become a problem, though. I already notice that Gemtexter doesn't produce results instantly but after a few seconds of runtime. That's not a problem yet, though. </p> +<h3>Bash best practises and ShellCheck</h3> +<p>While working on Gemtexter, I also had a look at the Google Shell Style Guide and wrote a blog post on that:</p> +<a class="textlink" href="./2021-05-16-personal-bash-coding-style-guide.html">Personal bash coding style guide</a><br /> +<p>I followed all these best practices, and in my opinion, the result is a pretty maintainable Bash script (given that you are fluent with all the sed and grep commands I used).</p> +<p>ShellCheck, a shell script analysis tool written in Haskell, is run on Gemtexter together with its unit tests, ensuring that all code is still acceptable. I am pretty impressed with what ShellCheck found. </p> +<p>It, for example, detected "some_command | while read var; do ...; done" loops and hinted that these create a new subprocess for the while part. The result is that all variable modifications taking place in the while-subprocess won't reflect the primary Bash process. ShellSheck then recommended rewriting the loop so that no subprocess is spawned as "while read -r var; do ...; done < <(some_command)". ShellCheck also pointed out to add a "-r" to the read; otherwise, there could be an issue with backspaces in the loop data.</p> +<p>Furthermore, ShellCheck recommended many more improvements. Declaration of unused variables and missing variable and string quotations were the most common ones. ShellSheck immensely helped to improve the robustness of the script.</p> +<a class="textlink" href="https://shellcheck.net">https://shellcheck.net</a><br /> +<h3>Unit testing</h3> +<p>There is a basic unit test module in ./lib/assert.source.sh, which is used for unit testing various functions of Gemtexter. I found this to be very beneficial for cross-platform development. For example, I noticed that some unit tests failed on macOS while everything still worked fine on my Fedora Linux laptop. </p> +<p>After digging a bit, I noticed that I had to install the GNU versions of the sed and grep commands on macOS and a newer version of the Bash to make all unit tests pass and it all work.</p> +<p>It has been proven quite helpful to have unit tests in place for the HTML part already when working on the Markdown generator part. To test the Markdown part, I copied the HTML unit tests and changed the expected outcome in the assertions. This way, I could implement the Markdown generator in a test-driven way (writing the test first and afterwards the implementation).</p> +<h3>HTML unit test example</h3> +<pre> +line='=> http://example.org Description of the link' +assert::equals "$(generate::make_link html "$line")" \ + '<a class="textlink" href="http://example.org">Description of the link</a><br />' + +</pre> +<h3>Markdown unit test example</h3> +<pre> +line='=> http://example.org Description of the link' +assert::equals "$(generate::make_link md "$line")" \ + '[Description of the link](http://example.org) ' +</pre> +<h2>Handcrafted HTML styles</h2> +<p>I had a look at some ready off the shelf CSS styles, but they all seemed too bloated. There is a whole industry selling CSS styles on the interweb. I preferred an effortless and minimalist style for the HTML site. So I handcrafted the Cascading Style Sheets manually with love and included them in the HTML header template. </p> +<p>For now, I have to re-generate all HTML files whenever the CSS changes. That should not be an issue now, but I might move the CSS into a separate file one day.</p> +<p>It's worth mentioning that all generated HTML files and Atom feeds pass the W3C validation tests.</p> +<p> </p> +<h2>Configurability</h2> +<p>In case someone else than me wants to use Gemtexter for his own site, it is pretty much configurable. It is possible to specify your own configuration file and your own HTML templates. Have a look at the GitHub page for examples.</p> +<h2>Future features</h2> +<p>I could think of the following features added to a future version of gemtexter:</p> +<ul> +<li>Templating of Gemtext files so that the .gmi files are generated from .gmi.tpl files. The template engine could do such things as an automatic table of contents and sitemap generation. It could also include the output of inlined shell code, e.g. a fortune quote. </li> +<li>Add support for more output formats, such as Groff, PDF, plain text, Gopher, etc.</li> +<li>External CSS file for HTML.</li> +</ul> +<h2>Conclusion</h2> +<p>It was quite a lot of fun writing Gemtexter. It's a relatively small project, but given that I worked on that in my spare time once in a while, it kept me busy for several weeks. </p> +<p>I finally revamped my personal internet site and started to blog again. I wanted the result to be exactly how it is now: A slightly retro-inspired internet site built for fun with unconventional tools. </p> +<p>E-Mail me your thoughts at comments@mx.buetow.org!</p> +<a class="textlink" href="../">Go back to the main site</a><br /> +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> +<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"> +<head> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<title>Gemtexter - One Bash script to rule it all</title> +<link rel="shortcut icon" type="image/gif" href="/favicon.ico" /> +<style type="text/css"> +body { + margin: auto; + max-width: 900px; + background-color: #FFFFEF; + border: 1px dashed #880000; + border-radius: 8px; + padding: 5px; +} +img { + display:block; + max-width: 80%; +} +a.textlink:before { + content: " ⇒ "; + padding-left: 2px; +} +a.textlink { + text-decoration: none; + color: #FF0000; +} +a.textlink:hover { + text-decoration: underline; +} +i { + color: #48AAAD; +} +pre { + background-color: #F1F8E9; + border: 1px dashed #BB0000; + border-radius: 8px; + padding: 5px; + font-family: "Lucida Console", "Courier New", monospace; +} +h1 { + text-align: center; + color: #880000; +} +h2, h3 { + color: #BB0000; +} +</style> +</head> +<body> diff --git a/gemfeed/2021-06-05-gemtexter-one-bash-script-to-rule-it-all/blog-engine.jpg b/gemfeed/2021-06-05-gemtexter-one-bash-script-to-rule-it-all/blog-engine.jpg Binary files differnew file mode 100644 index 00000000..844bc9fc --- /dev/null +++ b/gemfeed/2021-06-05-gemtexter-one-bash-script-to-rule-it-all/blog-engine.jpg diff --git a/gemfeed/atom.xml b/gemfeed/atom.xml index 0c40c208..715e61c5 100644 --- a/gemfeed/atom.xml +++ b/gemfeed/atom.xml @@ -1,12 +1,150 @@ <?xml version="1.0" encoding="utf-8"?> <feed xmlns="http://www.w3.org/2005/Atom"> - <updated>2021-06-05T10:13:46+01:00</updated> + <updated>2021-06-05T19:10:21+01:00</updated> <title>buetow.org feed</title> <subtitle>Having fun with computers!</subtitle> <link href="https://buetow.org/gemfeed/atom.xml" rel="self" /> <link href="https://buetow.org/" /> <id>https://buetow.org/</id> <entry> + <title>Gemtexter - One Bash script to rule it all</title> + <link href="https://buetow.org/gemfeed/2021-06-05-gemtexter-one-bash-script-to-rule-it-all.html" /> + <id>https://buetow.org/gemfeed/2021-06-05-gemtexter-one-bash-script-to-rule-it-all.html</id> + <updated>2021-06-05T19:03:32+01:00</updated> + <author> + <name>Paul Buetow</name> + <email>comments@mx.buetow.org</email> + </author> + <summary>You might have read my previous blog post about entering the Geminispace, where I pointed out the benefits of having and maintaining an internet presence there. This whole site (the blog and all other pages) is composed in the Gemtext markup language. . .....to read on please visit my site.</summary> + <content type="xhtml"> + <div xmlns="http://www.w3.org/1999/xhtml"> + <h1>Gemtexter - One Bash script to rule it all</h1> +<pre> + o .,<>., o + |\/\/\/\/| + '========' + (_ SSSSSSs + )a'`SSSSSs + /_ SSSSSS + .=## SSSSS + .#### SSSSs + ###::::SSSSS + .;:::""""SSS + .:;:' . . \\ + .::/ ' .'| + .::( . | + :::) \ + /\( / + /) ( | + .' \ . ./ / + _-' |\ . | + _..--.. . /"---\ | ` | . | + -=====================,' _ \=(*#(7.#####() | `/_.. , ( + _.-''``';'-''-) ,. \ ' '+/// | .'/ \ ``-.) \ + ,' _.- (( `-' `._\ `` \_/_.' ) /`-._ ) | + ,'\ ,' _.'.`:-. \.-' / <_L )" | + _/ `._,' ,')`; `-'`' | L / / + / `. ,' ,|_/ / \ ( <_-' \ + \ / `./ ' / /,' \ /|` `. | + )\ /`._ ,'`._.-\ |) \' + / `.' )-'.-,' )__) |\ `| + : /`. `.._(--.`':`':/ \ ) \ \ + |::::\ ,'/::;-)) / ( )`. | + ||::::: . .::': :`-( |/ . | + ||::::| . :| |==[]=: . - \ + |||:::| : || : | | /\ ` | + ___ ___ '|;:::| | |' \=[]=| / \ \ +| /_ ||``|||::::: | ; | | | \_.'\_ `-. +: \_``[]--[]|::::'\_;' )-'..`._ .-'\``:: ` . \ + \___.>`''-.||:.__,' SSt |_______`> <_____:::. . . \ _/ + `+a:f:......jrei''' +</pre> +<p class="quote"><i>Written by Paul Buetow 2021-06-05</i></p> +<p>You might have read my previous blog post about entering the Geminispace, where I pointed out the benefits of having and maintaining an internet presence there. This whole site (the blog and all other pages) is composed in the Gemtext markup language. </p> +<a class="textlink" href="https://buetow.org/gemfeed/2021-04-24-welcome-to-the-geminispace.html">Welcome to the Geminispace</a><br /> +<p>This comes with the benefit that I can write content in my favourite text editor (Vim). </p> +<h2>Motivation</h2> +<p>Another benefit of using Gemini is that the Gemtext markup language is easy to parse. As my site is dual-hosted (Gemini+HTTP), I could, in theory, just write a shell script to deal with the conversion from Gemtext to HTML; there is no need for a full-featured programming language here. I have done a lot of Bash in the past, but I am also often revisiting old tools and techniques for refreshing and keeping the knowledge up to date here.</p> +<i>Motivational comic strip:</i><a href="https://buetow.org/gemfeed/2021-06-05-gemtexter-one-bash-script-to-rule-it-all/blog-engine.jpg"><img alt="Motivational comic strip" title="Motivational comic strip" src="https://buetow.org/gemfeed/2021-06-05-gemtexter-one-bash-script-to-rule-it-all/blog-engine.jpg" /></a><br /> +<p>I have exactly done that - I wrote a Bash script, named gemtexter, for that. Have a look at GitHub:</p> +<a class="textlink" href="https://github.com/snonux/gemtexter">https://github.com/snonux/gemtexter</a><br /> +<p>In short, Gemtexter is a static site generator and blogging engine that uses Gemtext as its input format.</p> +<h2>Output formats</h2> +<p>Gemtexter takes the Gemntext Markup files as the input and generates the following outputs from it (you find examples for each of these output formats on the Gemtexter GitHub page):</p> +<ul> +<li>HTML files for my website</li> +<li>Markdown files for a GitHub page</li> +<li>A Gemtext Atom feed for my blog posts</li> +<li>A Gemfeed for my blog posts (a particular feed format commonly used in Geminispace. The Gemfeed can be used as an alternative to the Atom feed).</li> +<li>An HTML Atom feed of my blog posts</li> +</ul> +<p>I could have done all of that with a more robust language than Bash (such as Perl, Ruby, Go...), but I didn't. The purpose of this exercise was to challenge what I can do with a "simple" Bash script and learn new things.</p> +<h2>Taking it as far as I should, but no further</h2> +<p>The Bash is suitable very well for small scripts and ad-hoc automation on the command line. But it is for sure not a robust programming language. Writing this blog post, Gemtexter is nearing 1000 lines of code, which is actually a pretty large Bash script.</p> +<h3>Modularization </h3> +<p>I modularized the code so that each core functionality has its own file in ./lib. All the modules are included (sourced) from the main Gemtexter script. For example, there is one module for HTML generation, one for Markdown generation, and so on. </p> +<pre> +paul in uranus in gemtexter on 🌱 main +❯ wc -l gemtexter lib/* + 117 gemtexter + 59 lib/assert.source.sh + 128 lib/atomfeed.source.sh + 64 lib/gemfeed.source.sh + 161 lib/generate.source.sh + 50 lib/git.source.sh + 162 lib/html.source.sh + 30 lib/log.source.sh + 63 lib/md.source.sh + 834 total +</pre> +<p>This way, the script could grow far beyond 1000 lines of code and still be maintainable. With more features, execution speed may slowly become a problem, though. I already notice that Gemtexter doesn't produce results instantly but after a few seconds of runtime. That's not a problem yet, though. </p> +<h3>Bash best practises and ShellCheck</h3> +<p>While working on Gemtexter, I also had a look at the Google Shell Style Guide and wrote a blog post on that:</p> +<a class="textlink" href="https://buetow.org/gemfeed/2021-05-16-personal-bash-coding-style-guide.html">Personal bash coding style guide</a><br /> +<p>I followed all these best practices, and in my opinion, the result is a pretty maintainable Bash script (given that you are fluent with all the sed and grep commands I used).</p> +<p>ShellCheck, a shell script analysis tool written in Haskell, is run on Gemtexter together with its unit tests, ensuring that all code is still acceptable. I am pretty impressed with what ShellCheck found. </p> +<p>It, for example, detected "some_command | while read var; do ...; done" loops and hinted that these create a new subprocess for the while part. The result is that all variable modifications taking place in the while-subprocess won't reflect the primary Bash process. ShellSheck then recommended rewriting the loop so that no subprocess is spawned as "while read -r var; do ...; done < <(some_command)". ShellCheck also pointed out to add a "-r" to the read; otherwise, there could be an issue with backspaces in the loop data.</p> +<p>Furthermore, ShellCheck recommended many more improvements. Declaration of unused variables and missing variable and string quotations were the most common ones. ShellSheck immensely helped to improve the robustness of the script.</p> +<a class="textlink" href="https://shellcheck.net">https://shellcheck.net</a><br /> +<h3>Unit testing</h3> +<p>There is a basic unit test module in ./lib/assert.source.sh, which is used for unit testing various functions of Gemtexter. I found this to be very beneficial for cross-platform development. For example, I noticed that some unit tests failed on macOS while everything still worked fine on my Fedora Linux laptop. </p> +<p>After digging a bit, I noticed that I had to install the GNU versions of the sed and grep commands on macOS and a newer version of the Bash to make all unit tests pass and it all work.</p> +<p>It has been proven quite helpful to have unit tests in place for the HTML part already when working on the Markdown generator part. To test the Markdown part, I copied the HTML unit tests and changed the expected outcome in the assertions. This way, I could implement the Markdown generator in a test-driven way (writing the test first and afterwards the implementation).</p> +<h3>HTML unit test example</h3> +<pre> +line='=> http://example.org Description of the link' +assert::equals "$(generate::make_link html "$line")" \ + '<a class="textlink" href="http://example.org">Description of the link</a><br />' + +</pre> +<h3>Markdown unit test example</h3> +<pre> +line='=> http://example.org Description of the link' +assert::equals "$(generate::make_link md "$line")" \ + '[Description of the link](http://example.org) ' +</pre> +<h2>Handcrafted HTML styles</h2> +<p>I had a look at some ready off the shelf CSS styles, but they all seemed too bloated. There is a whole industry selling CSS styles on the interweb. I preferred an effortless and minimalist style for the HTML site. So I handcrafted the Cascading Style Sheets manually with love and included them in the HTML header template. </p> +<p>For now, I have to re-generate all HTML files whenever the CSS changes. That should not be an issue now, but I might move the CSS into a separate file one day.</p> +<p>It's worth mentioning that all generated HTML files and Atom feeds pass the W3C validation tests.</p> +<p> </p> +<h2>Configurability</h2> +<p>In case someone else than me wants to use Gemtexter for his own site, it is pretty much configurable. It is possible to specify your own configuration file and your own HTML templates. Have a look at the GitHub page for examples.</p> +<h2>Future features</h2> +<p>I could think of the following features added to a future version of gemtexter:</p> +<ul> +<li>Templating of Gemtext files so that the .html files are generated from .html.tpl files. The template engine could do such things as an automatic table of contents and sitemap generation. It could also include the output of inlined shell code, e.g. a fortune quote. </li> +<li>Add support for more output formats, such as Groff, PDF, plain text, Gopher, etc.</li> +<li>External CSS file for HTML.</li> +</ul> +<h2>Conclusion</h2> +<p>It was quite a lot of fun writing Gemtexter. It's a relatively small project, but given that I worked on that in my spare time once in a while, it kept me busy for several weeks. </p> +<p>I finally revamped my personal internet site and started to blog again. I wanted the result to be exactly how it is now: A slightly retro-inspired internet site built for fun with unconventional tools. </p> +<p>E-Mail me your thoughts at comments@mx.buetow.org!</p> + </div> + </content> + </entry> + <entry> <title>Personal Bash coding style guide</title> <link href="https://buetow.org/gemfeed/2021-05-16-personal-bash-coding-style-guide.html" /> <id>https://buetow.org/gemfeed/2021-05-16-personal-bash-coding-style-guide.html</id> @@ -364,7 +502,9 @@ fi <p>Why is there a need for a new protocol? As the modern web is a superset of Gemini, can't we use simple HTML 1.0 instead? That's a good and valid question. It is not a technical problem but a human problem. We tend to abuse the features once they are available. You can ensure that things stay efficient and straightforward as long as you are using the Gemini protocol. On the other hand, you can't force every website on the modern web to only create plain and straightforward-looking HTML pages.</p> <h2>My own Gemini capsule</h2> <p>As it is effortless to set up and maintain your own Gemini capsule (Gemini server + content composed via the Gemtext markup language), I decided to create my own. What I like about Gemini is that I can use my favourite text editor and get typing. I don't need to worry about the style and design of the presence, and I also don't have to test anything in ten different web browsers. I can only focus on the content! As a matter of fact, I am using the Vim editor + its spellchecker + auto word completion functionality to write this. </p> -<h2>Advantages summarised</h2> +<p>This site was generated with Gemtexter. You can read more about it here:</p> +<a class="textlink" href="https://buetow.org/gemfeed/2021-06-05-gemtexter-one-bash-script-to-rule-it-all.html">Gemtexter - One Bash script to rule it all</a><br /> +<h2>Gemini advantages summarised</h2> <ul> <li>Supports an alternative to the modern bloated web</li> <li>Easy to operate and easy to write content</li> diff --git a/gemfeed/index.html b/gemfeed/index.html index 6171d83a..44abbd38 100644 --- a/gemfeed/index.html +++ b/gemfeed/index.html @@ -50,8 +50,9 @@ h2, h3 { <body> <h1>buetow.org's Gemfeed</h1> <h2>Having fun with computers!</h2> +<a class="textlink" href="./2021-06-05-gemtexter-one-bash-script-to-rule-it-all.html">2021-06-05 (1198 words) - Gemtexter - One Bash script to rule it all</a><br /> <a class="textlink" href="./2021-05-16-personal-bash-coding-style-guide.html">2021-05-16 (1717 words) - Personal Bash coding style guide</a><br /> -<a class="textlink" href="./2021-04-24-welcome-to-the-geminispace.html">2021-04-24 (0772 words) - Welcome to the Geminispace</a><br /> +<a class="textlink" href="./2021-04-24-welcome-to-the-geminispace.html">2021-04-24 (0797 words) - Welcome to the Geminispace</a><br /> <a class="textlink" href="./2021-04-22-dtail-the-distributed-log-tail-program.html">2021-04-22 (2117 words) - DTail - The distributed log tail program</a><br /> <a class="textlink" href="./2018-06-01-realistic-load-testing-with-ioriot-for-linux.html">2018-06-01 (2171 words) - Realistic load testing with I/O Riot for Linux</a><br /> <a class="textlink" href="./2016-11-20-methods-in-c.html">2016-11-20 (0314 words) - Methods in C</a><br /> @@ -86,8 +86,12 @@ h2, h3 { <a class="textlink" href="./gemfeed/index.html">Subscribe to this blog's Gemfeed</a><br /> <h3>Posts</h3> <p>I have switched blog software multiple times. I might be backfilling some of the older articles here. So please don't wonder when suddenly old posts appear here.</p> +<p><<<<<<< HEAD</p> +<p>=======</p> +<p>>>>>>>> 5a55ee3 (Publish blog post about Gemtexter)</p> +<a class="textlink" href="./gemfeed/2021-06-05-gemtexter-one-bash-script-to-rule-it-all.html">2021-06-05 (1198 words) - Gemtexter - One Bash script to rule it all</a><br /> <a class="textlink" href="./gemfeed/2021-05-16-personal-bash-coding-style-guide.html">2021-05-16 (1717 words) - Personal Bash coding style guide</a><br /> -<a class="textlink" href="./gemfeed/2021-04-24-welcome-to-the-geminispace.html">2021-04-24 (0772 words) - Welcome to the Geminispace</a><br /> +<a class="textlink" href="./gemfeed/2021-04-24-welcome-to-the-geminispace.html">2021-04-24 (0797 words) - Welcome to the Geminispace</a><br /> <a class="textlink" href="./gemfeed/2021-04-22-dtail-the-distributed-log-tail-program.html">2021-04-22 (2117 words) - DTail - The distributed log tail program</a><br /> <a class="textlink" href="./gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux.html">2018-06-01 (2171 words) - Realistic load testing with I/O Riot for Linux</a><br /> <a class="textlink" href="./gemfeed/2016-11-20-methods-in-c.html">2016-11-20 (0314 words) - Methods in C</a><br /> |
