diff options
| author | Paul Buetow <paul@buetow.org> | 2021-05-14 09:36:49 +0100 |
|---|---|---|
| committer | Paul Buetow <git@mx.buetow.org> | 2021-05-21 05:11:05 +0100 |
| commit | 48912c9200d86bdcd0d03d7f8beb6c57632e2fdb (patch) | |
| tree | 88bbd9489292880241165242871d2373b9730338 | |
| parent | 02c49e983d478fad99db8658180828fea0ab242e (diff) | |
restyle and also publish ioriot
45 files changed, 472 insertions, 137 deletions
diff --git a/content/gemtext/gemfeed/2008-06-26-perl-poetry.gmi b/content/gemtext/gemfeed/2008-06-26-perl-poetry.gmi index 1ba53967..0414a23f 100644 --- a/content/gemtext/gemfeed/2008-06-26-perl-poetry.gmi +++ b/content/gemtext/gemfeed/2008-06-26-perl-poetry.gmi @@ -1,7 +1,3 @@ -> Written by Paul Buetow 2008-06-26, last updated 2021-05-04 - -=> ../ Go back to the main site - # Perl Poetry ``` @@ -27,6 +23,8 @@ _~~|~/_|_|__/|~~~~~~~ | / ~~~~~ | | ~~~~~~~~ (__) (____) ``` +> Written by Paul Buetow 2008-06-26, last updated 2021-05-04 + Here are some Perl Poems I wrote. They don't do anything useful when you run them, but they don't produce a compiler error either. They only exist for fun and demonstrate what you can do with Perl syntax. Wikipedia: "Perl poetry is the practice of writing poems that can be compiled as legal Perl code, for example the piece known as Black Perl. Perl poetry is made possible by the large number of English words that are used in the Perl language. New poems are regularly submitted to the community at PerlMonks." @@ -164,3 +162,5 @@ Did you like what you saw? Have a look at Github to see my other poems too: => https://github.com/snonux/perl-poetry E-Mail me your thoughts at comments@mx.buetow.org! + +=> ../ Go back to the main site diff --git a/content/gemtext/gemfeed/2010-05-09-the-fype-programming-language.gmi b/content/gemtext/gemfeed/2010-05-09-the-fype-programming-language.gmi index 3b008457..dc28ef6c 100644 --- a/content/gemtext/gemfeed/2010-05-09-the-fype-programming-language.gmi +++ b/content/gemtext/gemfeed/2010-05-09-the-fype-programming-language.gmi @@ -1,6 +1,4 @@ -> Written by Paul Buetow 2010-05-09, last updated 2021-05-05 - -=> ../ Go back to the main site +# The Fype Programming Language ``` ____ _ __ @@ -9,10 +7,9 @@ _ / /| _| |_| | |_) | __/ | |_| | __/ (_| | | | |_| _| |_| | (_)_/ |_| \__, | .__/ \___| \__, |\___|\__,_|_| |_(_)_| \__, | |___/|_| |___/ |___/ - ``` -# The Fype Programming Language +> Written by Paul Buetow 2010-05-09, last updated 2021-05-05 Fype is an interpreted programming language created by me for learning and fun. The interpreter is written in C. It has been tested on FreeBSD and NetBSD and may also work on other Unix like operating systems such as Linux based ones. To be honest, besides learning and fun there is really no other use case of why Fype actually exists as many other programming languages are much faster and more powerful. @@ -509,3 +506,5 @@ You can find all of this on the GitHub page. There is also an "examples" folders => https://github.com/snonux/fype E-Mail me your thoughts at comments@mx.buetow.org! + +=> ../ Go back to the main site diff --git a/content/gemtext/gemfeed/2011-05-07-perl-daemon-service-framework.gmi b/content/gemtext/gemfeed/2011-05-07-perl-daemon-service-framework.gmi index 958a4842..d13ff600 100644 --- a/content/gemtext/gemfeed/2011-05-07-perl-daemon-service-framework.gmi +++ b/content/gemtext/gemfeed/2011-05-07-perl-daemon-service-framework.gmi @@ -1,7 +1,3 @@ -> Written by Paul Buetow 2011-05-07, last updated 2021-05-07 - -=> ../ Go back to the main site - # Perl Daemon (Service Framework) ``` @@ -11,6 +7,8 @@ //\ //\\ //\ //\\ //\ //\\jrei ``` +> Written by Paul Buetow 2011-05-07, last updated 2021-05-07 + PerlDaemon is a minimal daemon for Linux and other Unix like operating systems programmed in Perl. It is a minimal but pretty functional and fairly generic service framework. This means that it does not do anything useful other than providing a framework for starting, stopping, configuring and logging. In order to do something useful, a module (written in Perl) must be provided. ## Features @@ -161,3 +159,5 @@ You can find PerlDaemon (including the examples) at: => https://github.com/snonux/perldaemon E-Mail me your thoughts at comments@mx.buetow.org! + +=> ../ Go back to the main site diff --git a/content/gemtext/gemfeed/2016-04-03-offsite-backup-with-zfs.gmi b/content/gemtext/gemfeed/2016-04-03-offsite-backup-with-zfs.gmi index 314bea84..cbde3ce6 100644 --- a/content/gemtext/gemfeed/2016-04-03-offsite-backup-with-zfs.gmi +++ b/content/gemtext/gemfeed/2016-04-03-offsite-backup-with-zfs.gmi @@ -1,7 +1,3 @@ -> Written by Paul Buetow 2016-04-03 - -=> ../ Go back to the main site - # Offsite backup with ZFS ``` @@ -17,6 +13,8 @@ \____||__|_____|__| ``` +> Written by Paul Buetow 2016-04-03 + ## Please don't lose all my pictures again! When it comes to data storage and potential data loss I am a paranoid person. It is not just due to my job but also due to a personal experience I encountered over 10 years ago: A single drive failure and loss of all my data (pictures, music, ....). @@ -40,3 +38,5 @@ The solution is adding another USB drive (2TB) with an encryption container (GEL I am thinking of buying a second 2TB USB drive and to set it up the same way as the first one. So I could alternate the backups. One drive would be at the secret location, and the other drive would be at home. And these drives would swap location after each cycle. This would give some security about the failure of that drive and I would have to go to the secret location only once (swapping the drives) instead of twice (picking that drive up in order to update the data + bringing it back to the secret location). E-Mail me your thoughts at comments@mx.buetow.org! + +=> ../ Go back to the main site diff --git a/content/gemtext/gemfeed/2016-04-16-offsite-backup-with-zfs-part2.gmi b/content/gemtext/gemfeed/2016-04-16-offsite-backup-with-zfs-part2.gmi index b91e7727..beb1ab9f 100644 --- a/content/gemtext/gemfeed/2016-04-16-offsite-backup-with-zfs-part2.gmi +++ b/content/gemtext/gemfeed/2016-04-16-offsite-backup-with-zfs-part2.gmi @@ -1,7 +1,3 @@ -> Written by Paul Buetow 2016-04-16 - -=> ../ Go back to the main site - # Offsite backup with ZFS (Part 2) ``` @@ -19,6 +15,8 @@ \____||__|_____|__| ``` +> Written by Paul Buetow 2016-04-16 + => ./2016-04-03-offsite-backup-with-zfs.gmi Read the first part before reading any furter here... I enhanced the procedure a bit. From now on I am having two external 2TB USB hard drives. Both are setup exactly the same way. To decrease the probability that they will not fail at about the same time both drives are of different brands. One drive is kept at the secret location. The other one is kept at home right next to my HP MicroServer. @@ -28,3 +26,5 @@ Whenever I am updating offsite backup, I am doing it to the drive which is kept Furthermore, I added scrubbing (*zpool scrub...*) to the script. It ensures that the file system is consistent and that there are no bad blocks on the disk and the file system. To increase the reliability I also run a *zfs set copies=2 zroot*. That setting is also synchronized to the offsite ZFS pool. ZFS stores every data block to disk twice now. Yes, it consumes twice as much disk space but it makes it better fault tolerant against hardware errors (e.g. only individual disk sectors going bad). E-Mail me your thoughts at comments@mx.buetow.org! + +=> ../ Go back to the main site diff --git a/content/gemtext/gemfeed/2016-05-22-spinning-up-my-own-authoritative-dns-servers.gmi b/content/gemtext/gemfeed/2016-05-22-spinning-up-my-own-authoritative-dns-servers.gmi index 1d481fb3..1be6fa74 100644 --- a/content/gemtext/gemfeed/2016-05-22-spinning-up-my-own-authoritative-dns-servers.gmi +++ b/content/gemtext/gemfeed/2016-05-22-spinning-up-my-own-authoritative-dns-servers.gmi @@ -1,9 +1,7 @@ -> Written by Paul Buetow 2016-05-22 - -=> ../ Go back to the main site - # Spinning up my own authoritative DNS servers +> Written by Paul Buetow 2016-05-22 + ## Background Finally, I had time to deploy my own authoritative DNS servers (master and slave) for my domains "buetow.org" and "buetow.zone". My domain name provider is Schlund Technologies. They allow their customers to manually edit the DNS records (BIND files). And they also give you the opportunity to set your own authoritative DNS servers for your domains. From now, I am making use of that option. @@ -237,3 +235,5 @@ Whenever I have to change a DNS entry all have to do is: That's much more comfortable now than manually clicking at some web UIs at Schlund Technologies. E-Mail me your thoughts at comments@mx.buetow.org! + +=> ../ Go back to the main site diff --git a/content/gemtext/gemfeed/2016-11-20-methods-in-c.gmi b/content/gemtext/gemfeed/2016-11-20-methods-in-c.gmi index 0d47a3fe..d2035252 100644 --- a/content/gemtext/gemfeed/2016-11-20-methods-in-c.gmi +++ b/content/gemtext/gemfeed/2016-11-20-methods-in-c.gmi @@ -1,9 +1,7 @@ -> Written by Paul Buetow 2016-11-20 - -=> ../ Go back to the main site - # Methods in C +> Written by Paul Buetow 2016-11-20 + You can do some sort of object oriented programming in the C Programming Language. However, that is very limited. But also very easy and straight forward to use. ## Example @@ -84,3 +82,5 @@ How to overcome this? You need to take it further... If you want to take it further type "Object-Oriented Programming with ANSI-C" into your favorite internet search engine, you will find some crazy stuff. Some go as far as writing a C preprocessor in AWK, which takes some object oriented pseudo-C and transforms it to plain C so that the C compiler can compile it to machine code. This is actually similar to how the C++ language had its origins. E-Mail me your thoughts at comments@mx.buetow.org! + +=> ../ Go back to the main site diff --git a/content/gemtext/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux.draft.gmi b/content/gemtext/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux.gmi index 09fa911a..7fd95643 100644 --- a/content/gemtext/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux.draft.gmi +++ b/content/gemtext/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux.gmi @@ -1,7 +1,3 @@ - Written by Paul Buetow 2018-06-01, last updated 2021-05-08 - -=> ../ Go back to the main site - # Realistic load testing with I/O Riot for Linux ``` @@ -15,6 +11,8 @@ jgs\__/'---'\__/ ``` +> Written by Paul Buetow 2018-06-01, last updated 2021-05-08 + ## Foreword This text first was published in the german IT-Administrator computer Magazine. 3 years have passed since and I decided to publish it on my blog too. @@ -23,7 +21,7 @@ This text first was published in the german IT-Administrator computer Magazine. I havn't worked on I/O Riot for some time now, but all what is written here is still valid. I am still using I/O Riot to debug I/O issues and pattern once in a while, so by all means the tool is not obsolete yet. The tool even helped to resolve a major production incident at work involving I/O. -I am eagerly looking forward to revamp I/O Riot so that it uses the new BPF Linux capabilities instead of Systemtap. Also, when I wrote I/O Riot initially, I didn't have any experience with the Go programming language yet and therefore I wrote it in C. Once it gets revamped I might consider using Go instead of C as it would spare me from many segmentation faults and headaches during development ;-). I might also just stick to C for plain performance reasons and just refactor the code dealing with concurrency. +I am eagerly looking forward to revamp I/O Riot so that it uses the new BPF Linux capabilities instead of plain old Systemtap (or alternatively: Newer versions of Systemtap can also use BPF as the backend I have learned). Also, when I wrote I/O Riot initially, I didn't have any experience with the Go programming language yet and therefore I wrote it in C. Once it gets revamped I might consider using Go instead of C as it would spare me from many segmentation faults and headaches during development ;-). I might also just stick to C for plain performance reasons and just refactor the code dealing with concurrency. Pleace notice that some of the screenshots show the command "ioreplay" instead of "ioriot". That's because the name has changed after taking those. @@ -189,3 +187,5 @@ For example, the open syscall opens a file and returns the responsible file desc => https://en.wikipedia.org/wiki/Memory-mapped_I/O Memory mapped I/O E-Mail me your thoughts at comments@mx.buetow.org! + +=> ../ Go back to the main site diff --git a/content/gemtext/gemfeed/2021-04-22-dtail-the-distributed-log-tail-program.gmi b/content/gemtext/gemfeed/2021-04-22-dtail-the-distributed-log-tail-program.gmi index 15fcd899..f177d12f 100644 --- a/content/gemtext/gemfeed/2021-04-22-dtail-the-distributed-log-tail-program.gmi +++ b/content/gemtext/gemfeed/2021-04-22-dtail-the-distributed-log-tail-program.gmi @@ -1,9 +1,7 @@ -> Written by Paul Buetow 2021-04-22, last updated 2021-04-26 - -=> ../ Go back to the main site - # DTail - The distributed log tail program +> Written by Paul Buetow 2021-04-22, last updated 2021-04-26 + => ./2021-04-22-dtail-the-distributed-log-tail-program/dtail-logo.png DTail logo image This article first appeared at the Mimecast Engineering Blog but I made it available here in my personal Gemini capsule too. @@ -106,3 +104,5 @@ Mimecast highly encourages you to have a look at DTail and submit an issue for a => https://dtail.dev E-Mail me your thoughts at comments@mx.buetow.org! + +=> ../ Go back to the main site diff --git a/content/gemtext/gemfeed/2021-04-24-welcome-to-the-geminispace.gmi b/content/gemtext/gemfeed/2021-04-24-welcome-to-the-geminispace.gmi index 8d4fc0c2..e01dc297 100644 --- a/content/gemtext/gemfeed/2021-04-24-welcome-to-the-geminispace.gmi +++ b/content/gemtext/gemfeed/2021-04-24-welcome-to-the-geminispace.gmi @@ -1,10 +1,7 @@ -> Written by Paul Buetow 2021-04-24, last updated 2021-04-30 -> ASCII Art by Andy Hood - -=> ../ Go back to the main site - # Welcome to the Geminispace +> Written by Paul Buetow 2021-04-24, last updated 2021-04-30, ASCII Art by Andy Hood + Have you reached this article already via Gemini? You need a special client for that, web browsers such as Firefox, Chrome, Safari etc. don't support the Gemini protocol. The Gemini address of this site (or the address of this capsule as people say in Geminispace) is: => gemini://buetow.org @@ -76,3 +73,4 @@ Check out one of the following links for more information about Gemini. For exam E-Mail me your thoughts at comments@mx.buetow.org! +=> ../ Go back to the main site diff --git a/content/gemtext/gemfeed/2021-05-15-buetow.org.sh-One-Bash-script-to-rule-it-all.draft.gmi b/content/gemtext/gemfeed/2021-05-15-buetow.org.sh-One-Bash-script-to-rule-it-all.draft.gmi index be65b8b9..858478aa 100644 --- a/content/gemtext/gemfeed/2021-05-15-buetow.org.sh-One-Bash-script-to-rule-it-all.draft.gmi +++ b/content/gemtext/gemfeed/2021-05-15-buetow.org.sh-One-Bash-script-to-rule-it-all.draft.gmi @@ -1,9 +1,7 @@ -ADD WRITTEN BY AND CREATED AT BLABLA - -ADD GO BACK LINK - # buetow.org.sh - One Bash script to rule it all +> TODO: ADD WRITTEN BY AND CREATED AT BLABLA + You might have read my previous blog post about entering the Geminispace. => ./2021-04-24-welcome-to-the-geminispace Welcome to the Geminispace @@ -181,3 +179,5 @@ Specify -r => https://buetow.org https://buetow.org - The original Gemini capsule => https://buetow.org/gemfeed/ https://buetow.org/gemfeed/ - The Gemfeed => https://buetow.org/gemfeed/atom.xml https://buetow.org/gemfeed/atom.xml - The Atom feed + +TODO: ADD GO BACK LINK diff --git a/content/gemtext/gemfeed/atom.xml b/content/gemtext/gemfeed/atom.xml index 6fe9b6a5..152cf019 100644 --- a/content/gemtext/gemfeed/atom.xml +++ b/content/gemtext/gemfeed/atom.xml @@ -1,6 +1,6 @@ <?xml version="1.0" encoding="utf-8"?> <feed xmlns="http://www.w3.org/2005/Atom"> - <updated>2021-05-10T10:15:48+01:00</updated> + <updated>2021-05-14T09:34:34+01:00</updated> <title>buetow.org feed</title> <subtitle>Having fun with computers!</subtitle> <link href="gemini://buetow.org/gemfeed/atom.xml" rel="self" /> @@ -19,6 +19,7 @@ <content type="xhtml"> <div xmlns="http://www.w3.org/1999/xhtml"> <h1>Welcome to the Geminispace</h1> +<p class="quote"><i>Written by Paul Buetow 2021-04-24, last updated 2021-04-30, ASCII Art by Andy Hood</i></p> <p>Have you reached this article already via Gemini? You need a special client for that, web browsers such as Firefox, Chrome, Safari etc. don't support the Gemini protocol. The Gemini address of this site (or the address of this capsule as people say in Geminispace) is:</p> <a class="textlink" href="gemini://buetow.org">gemini://buetow.org</a><br /> <p>If you however still use HTTP then you are just surfing the fallback HTML version of this capsule. In that case I suggest reading on what this is all about :-).</p> @@ -69,6 +70,7 @@ <a class="textlink" href="gemini://gemini.circumlunar.space">gemini://gemini.circumlunar.space</a><br /> <a class="textlink" href="https://gemini.circumlunar.space">https://gemini.circumlunar.space</a><br /> <p>E-Mail me your thoughts at comments@mx.buetow.org!</p> +<a class="textlink" href="../">Go back to the main site</a><br /> </div> </content> </entry> @@ -85,6 +87,7 @@ <content type="xhtml"> <div xmlns="http://www.w3.org/1999/xhtml"> <h1>DTail - The distributed log tail program</h1> +<p class="quote"><i>Written by Paul Buetow 2021-04-22, last updated 2021-04-26</i></p> <i>DTail logo image:</i><a href="https://buetow.org/gemfeed/2021-04-22-dtail-the-distributed-log-tail-program/dtail-logo.png"><img alt="DTail logo image" title="DTail logo image" src="https://buetow.org/gemfeed/2021-04-22-dtail-the-distributed-log-tail-program/dtail-logo.png" /></a><br /> <p>This article first appeared at the Mimecast Engineering Blog but I made it available here in my personal Gemini capsule too.</p> <a class="textlink" href="https://medium.com/mimecast-engineering/dtail-the-distributed-log-tail-program-79b8087904bb">Original Mimecast Engineering Blog post at Medium</a><br /> @@ -149,6 +152,147 @@ dtail –servers serverlist.txt –files ‘/var/log/*.log’ –regex ‘(?i:er <p>Mimecast highly encourages you to have a look at DTail and submit an issue for any features you would like to see. Have you found a bug? Maybe you just have a question or comment? If you want to go a step further: We would also love to see pull requests for any features or improvements. Either way, if in doubt just contact us via the DTail GitHub page.</p> <a class="textlink" href="https://dtail.dev">https://dtail.dev</a><br /> <p>E-Mail me your thoughts at comments@mx.buetow.org!</p> +<a class="textlink" href="../">Go back to the main site</a><br /> + </div> + </content> + </entry> + <entry> + <title>The article</title> + <link href="gemini://buetow.org/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux.gmi" /> + <id>gemini://buetow.org/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux.gmi</id> + <updated>2018-06-01T09:30:27+01:00</updated> + <author> + <name>Paul Buetow</name> + <email>comments@mx.buetow.org</email> + </author> + <summary>This text first was published in the german IT-Administrator computer Magazine. 3 years have passed since and I decided to publish it on my blog too. . .....to read on please visit my site.</summary> + <content type="xhtml"> + <div xmlns="http://www.w3.org/1999/xhtml"> + <h1>Realistic load testing with I/O Riot for Linux</h1> +<pre> + .---. + / \ + \.@-@./ + /`\_/`\ + // _ \\ + | \ )|_ + /`\_`> <_/ \ +jgs\__/'---'\__/ +</pre> +<p class="quote"><i>Written by Paul Buetow 2018-06-01, last updated 2021-05-08</i></p> +<h2>Foreword</h2> +<p>This text first was published in the german IT-Administrator computer Magazine. 3 years have passed since and I decided to publish it on my blog too. </p> +<a class="textlink" href="https://www.admin-magazin.de/Das-Heft/2018/06/Realistische-Lasttests-mit-I-O-Riot">https://www.admin-magazin.de/Das-Heft/2018/06/Realistische-Lasttests-mit-I-O-Riot</a><br /> +<p>I havn't worked on I/O Riot for some time now, but all what is written here is still valid. I am still using I/O Riot to debug I/O issues and pattern once in a while, so by all means the tool is not obsolete yet. The tool even helped to resolve a major production incident at work involving I/O.</p> +<p>I am eagerly looking forward to revamp I/O Riot so that it uses the new BPF Linux capabilities instead of Systemtap. Also, when I wrote I/O Riot initially, I didn't have any experience with the Go programming language yet and therefore I wrote it in C. Once it gets revamped I might consider using Go instead of C as it would spare me from many segmentation faults and headaches during development ;-). I might also just stick to C for plain performance reasons and just refactor the code dealing with concurrency.</p> +<p>Pleace notice that some of the screenshots show the command "ioreplay" instead of "ioriot". That's because the name has changed after taking those.</p> +<h1>The article</h1> +<p>With I/O Riot IT administrators can load test and optimize the I/O subsystem of Linux-based operating systems. The tool makes it possible to record I/O patterns and replay them at a later time as often as desired. This means bottlenecks can be reproduced and eradicated. </p> +<p>When storing huge amounts of data, such as more than 200 billion archived emails at Mimecast, it's not only the available storage capacity that matters, but also the data throughput and latency. At the same time, operating costs must be kept as low as possible. The more systems involved, the more important it is to optimize the hardware, the operating system and the applications running on it.</p> +<h2>Background: Existing Techniques</h2> +<p>Conventional I/O benchmarking: Administrators usually use open source benchmarking tools like IOZone and bonnie++. Available database systems such as Redis and MySQL come with their own benchmarking tools. The common problem with these tools is that they work with prescribed artificial I/O patterns. Although this can test both sequential and randomized data access, the patterns do not correspond to what can be found on production systems.</p> +<p>Testing by load test environment: Another option is to use a separate load test environment in which, as far as possible, a production environment with all its dependencies is simulated. However, an environment consisting of many microservices is very complex. Microservices are usually managed by different teams, which means extra coordination effort for each load test. Another challenge is to generate the load as authentically as possible so that the patterns correspond to a productive environment. Such a load test environment can only handle as many requests as its weakest link can handle. For example, load generators send many read and write requests to a frontend microservice, whereby the frontend forwards the requests to a backend microservice responsible for storing the data. If the frontend service does not process the requests efficiently enough, the backend service is not well utilized in the first place. As a rule, all microservices are clustered across many servers, which makes everything even more complicated. Under all these conditions it is very difficult to test I/O of separate backend systems. Moreover, for many small and medium-sized companies, a separate load test environment would not be feasible for cost reasons.</p> +<p>Testing in the production environment: For these reasons, benchmarks are often carried out in the production environment. In order to derive value from this such tests are especially performed during peak hours when systems are under high load. However, testing on production systems is associated with risks and can lead to failure or loss of data without adequate protection.</p> +<h2>Benchmarking the Email Cloud at Mimecast</h2> +<p>For email archiving, Mimecast uses an internally developed microservice, which is operated directly on Linux-based storage systems. A storage cluster is divided into several replication volumes. Data is always replicated three times across two secure data centers. Customer data is automatically allocated to one or more volumes, depending on throughput, so that all volumes are automatically assigned the same load. Customer data is archived on conventional, but inexpensive hard disks with several terabytes of storage capacity each. I/O benchmarking proved difficult for all the reasons mentioned above. Furthermore, there are no ready-made tools for this purpose in the case of self-developed software. The service operates on many block devices simultaneously, which can make the RAID controller a bottleneck. None of the freely available benchmarking tools can test several block devices at the same time without extra effort. In addition, emails typically consist of many small files. Randomized access to many small files is particularly inefficient. In addition to many software adaptations, the hardware and operating system must also be optimized.</p> +<p>Mimecast encourages employees to be innovative and pursue their own ideas in the form of an internal competition, Pet Project. The goal of the pet project I/O Riot was to simplify OS and hardware level I/O benchmarking. The first prototype of I/O Riot was awarded an internal roadmap prize in the spring of 2017. A few months later, I/O Riot was used to reduce write latency in the storage clusters by about 50%. The improvement was first verified by I/O replay on a test system and then successively applied to all storage systems. I/O Riot was also used to resolve a production incident related to disk I/O load.</p> +<h2>Using I/O Riot</h2> +<p>First, all I/O events are logged to a file on a production system with I/O Riot. It is then copied to a test system where all events are replayed in the same way. The crucial point here is that you can reproduce I/O patterns as they are found on a production system as often as you like on a test system. This results in the possibility of optimizing the set screws on the system after each run.</p> +<h3>Installation</h3> +<p>I/O Riot was tested under CentOS 7.2 x86_64. For compiling, the GNU C compiler and Systemtap including kernel debug information are required. Other Linux distributions are theoretically compatible but untested. First of all, you should update the systems involved as follows:</p> +<pre> +% sudo yum update +</pre> +<p>If the kernel is updated, please restart the system. The installation would be done without a restart but this would complicate the installation. The installed kernel version should always correspond to the currently running kernel. You can then install I/O Riot as follows:</p> +<pre> +% sudo yum install gcc git systemtap yum-utils kernel-devel-$(uname -r) +% sudo debuginfo-install kernel-$(uname -r) +% git clone https://github.com/mimecast/ioriot +% cd ioriot +% make +% sudo make install +% export PATH=$PATH:/opt/ioriot/bin +</pre> +<p>Note: It is not best practice to install any compilers on production systems. For further information please have a look at the enclosed README.md.</p> +<h3>Recording of I/O events</h3> +<p>All I/O events are kernel related. If a process wants to perform an I/O operation, such as opening a file, it must inform the kernel of this by a system call (short syscall). I/O Riot relies on the Systemtap tool to record I/O syscalls. Systemtap, available for all popular Linux distributions, helps you to take a look at the running kernel in productive environments, which makes it predestined to monitor all I/O-relevant Linux syscalls and log them to a file. Other tools, such as strace, are not an alternative because they slow down the system too much.</p> +<p>During recording, ioriot acts as a wrapper and executes all relevant Systemtap commands for you. Use the following command to log all events to io.capture:</p> +<pre> +% sudo ioriot -c io.capture +</pre> +<i>Screenshot I/O recording:</i><a href="https://buetow.org/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux/figure1-ioriot-io-recording.png"><img alt="Screenshot I/O recording" title="Screenshot I/O recording" src="https://buetow.org/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux/figure1-ioriot-io-recording.png" /></a><br /> +<p>A Ctrl-C (SIGINT) stops recording prematurely. Otherwise, ioriot terminates itself automatically after 1 hour. Depending on the system load, the output file can grow to several gigabytes. Only metadata is logged, not the read and written data itself. When replaying later, only random data is used. Under certain circumstances, Systemtap may omit some system calls and issue warnings. This is to ensure that Systemtap does not consume too many resources.</p> +<h3>Test preparation</h3> +<p>Then copy io.capture to a test system. The log also contains all accesses to the pseudo file systems devfs, sysfs and procfs. This makes little sense, which is why you must first generate a cleaned and playable version io.replay from io.capture as follows:</p> +<pre> +% sudo ioriot -c io.capture -r io.replay -u $USER -n TESTNAME +</pre> +<p>The parameter -n allows you to assign a freely selectable test name. An arbitrary system user under which the test is to be played is specified via paramater -u.</p> +<h3>Test Initialization</h3> +<p>The test will most likely want to access existing files. These are files the test wants to read but does not create by itself. The existence of these must be ensured before the test. You can do this as follows:</p> +<pre> +% sudo ioriot -i io.replay +</pre> +<p>To avoid any damage to the running system, ioreplay only works in special directories. The tool creates a separate subdirectory for each file system mount point (e.g. /, /usr/local, /store/00,...) (here: /.ioriot/TESTNAME, /usr/local/.ioriot/TESTNAME, /store/00/.ioriot/TESTNAME,...). By default, the working directory of ioriot is /usr/local/ioriot/TESTNAME.</p> +<i>Screenshot test preparation:</i><a href="https://buetow.org/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux/figure2-ioriot-test-preparation.png"><img alt="Screenshot test preparation" title="Screenshot test preparation" src="https://buetow.org/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux/figure2-ioriot-test-preparation.png" /></a><br /> +<p>You must re-initialize the environment before each run. Data from previous tests will be moved to a trash directory automatically, which can be finally deleted with "sudo ioriot -P".</p> +<h3>Replay</h3> +<p>After initialization, you can replay the log with -r. You can use -R to initiate both test initialization and replay in a single command and -S can be used to specify a file in which statistics are written after the test run.</p> +<p>You can also influence the playback speed: "-s 0" is interpreted as "Playback as fast as possible" and is the default setting. With "-s 1" all operations are performed at original speed. "-s 2" would double the playback speed and "-s 0.5" would halve it.</p> +<i>Screenshot replaying I/O:</i><a href="https://buetow.org/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux/figure3-ioriot-replay.png"><img alt="Screenshot replaying I/O" title="Screenshot replaying I/O" src="https://buetow.org/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux/figure3-ioriot-replay.png" /></a><br /> +<p>As an initial test, for example, you could compare the two Linux I/O schedulers CFQ and Deadline and check which scheduler the test runs the fastest. They run the test separately for each scheduler. The following shell loop iterates through all attached block devices of the system and changes their I/O scheduler to the one specified in variable $new_scheduler (in this case either cfq or deadline). Subsequently, all I/O events from the io.replay protocol are played back. At the end, an output file with statistics is generated:</p> +<pre> +% new_scheduler=cfq +% for scheduler in /sys/block/*/queue/scheduler; do + echo $new_scheduler | sudo tee $scheduler +done +% sudo ioriot -R io.replay -S cfq.txt +% new_scheduler=deadline +% for scheduler in /sys/block/*/queue/scheduler; do + echo $new_scheduler | sudo tee $scheduler +done +% sudo ioriot -R io.replay -S deadline.txt +</pre> +<p>According to the results, the test could run 940 seconds faster with Deadline Scheduler:</p> +<pre> +% cat cfq.txt +Num workers: 4 +hreads per worker: 128 +otal threads: 512 +Highest loadavg: 259.29 +Performed ioops: 218624596 +Average ioops/s: 101544.17 +Time ahead: 1452s +Total time: 2153.00s +% cat deadline.txt +Num workers: 4 +Threads per worker: 128 +Total threads: 512 +Highest loadavg: 342.45 +Performed ioops: 218624596 +Average ioops/s: 180234.62 +Time ahead: 2392s +Total time: 1213.00s +</pre> +<p>In any case, you should also set up a time series database, such as Graphite, where the I/O throughput can be plotted. Figures 4 and 5 show the read and write access times of both tests. The break-in makes it clear when the CFQ test ended and the deadline test was started. The reading latency of both tests is similar. Write latency is dramatically improved using the Deadline Scheduler.</p> +<i>Graphite visualization of the mean read access times in ms with CFQ and Deadline Scheduler.:</i><a href="https://buetow.org/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux/figure4-ioriot-read-latency.png"><img alt="Graphite visualization of the mean read access times in ms with CFQ and Deadline Scheduler." title="Graphite visualization of the mean read access times in ms with CFQ and Deadline Scheduler." src="https://buetow.org/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux/figure4-ioriot-read-latency.png" /></a><br /> +<i>Graphite visualization of the average write access times in ms with CFQ and Deadline Scheduler.:</i><a href="https://buetow.org/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux/figure5-ioriot-write-latency.png"><img alt="Graphite visualization of the average write access times in ms with CFQ and Deadline Scheduler." title="Graphite visualization of the average write access times in ms with CFQ and Deadline Scheduler." src="https://buetow.org/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux/figure5-ioriot-write-latency.png" /></a><br /> +<p>You should also take a look at the iostat tool. The iostat screenshot shows the output of iostat -x 10 during a test run. As you can see, a block device is fully loaded with 99% utilization, while all other block devices still have sufficient buffer. This could be an indication of poor data distribution in the storage system and is worth pursuing. It is not uncommon for I/O Riot to reveal software problems.</p> +<i>Output of iostat. The block device sdy seems to be almost fully utilized by 99%.:</i><a href="https://buetow.org/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux/figure6-iostat.png"><img alt="Output of iostat. The block device sdy seems to be almost fully utilized by 99%." title="Output of iostat. The block device sdy seems to be almost fully utilized by 99%." src="https://buetow.org/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux/figure6-iostat.png" /></a><br /> +<h2>I/O Riot is Open Source</h2> +<p>The tool has already proven to be very useful and will continue to be actively developed as time and priority permits. Mimecast intends to be an ongoing contributor to Open Source. You can find I/O Riot at:</p> +<a class="textlink" href="https://github.com/mimecast/ioriot">https://github.com/mimecast/ioriot</a><br /> +<h2>Systemtap</h2> +<p>Systemtap is a tool for the instrumentation of the Linux kernel. The tool provides an AWK-like programming language. Programs written in it are compiled from Systemtap to C- and then into a dynamically loadable kernel module. Loaded into the kernel, the program has access to Linux internals. A Systemtap program written for I/O Riot monitors when, with which parameters, at which time, and from which process I/O syscalls take place and their return values.</p> +<p>For example, the open syscall opens a file and returns the responsible file descriptor. The read and write syscalls can operate on a file descriptor and return the number of read or written bytes. The close syscall closes a given file descriptor. I/O Riot comes with a ready-made Systemtap program, which you have already compiled into a kernel module and installed to /opt/ioriot. In addition to open, read and close, it logs many other I/O-relevant calls.</p> +<a class="textlink" href="https://sourceware.org/systemtap/">https://sourceware.org/systemtap/</a><br /> +<h2>More refereces</h2> +<a class="textlink" href="http://www.iozone.org/">IOZone</a><br /> +<a class="textlink" href="https://www.coker.com.au/bonnie++/">Bonnie++</a><br /> +<a class="textlink" href="https://graphiteapp.org">Graphite</a><br /> +<a class="textlink" href="https://en.wikipedia.org/wiki/Memory-mapped_I/O">Memory mapped I/O</a><br /> +<p>E-Mail me your thoughts at comments@mx.buetow.org!</p> +<a class="textlink" href="../">Go back to the main site</a><br /> </div> </content> </entry> @@ -165,6 +309,7 @@ dtail –servers serverlist.txt –files ‘/var/log/*.log’ –regex ‘(?i:er <content type="xhtml"> <div xmlns="http://www.w3.org/1999/xhtml"> <h1>Methods in C</h1> +<p class="quote"><i>Written by Paul Buetow 2016-11-20</i></p> <p>You can do some sort of object oriented programming in the C Programming Language. However, that is very limited. But also very easy and straight forward to use.</p> <h2>Example</h2> <p>Lets have a look at the following sample program. Basically all you have to do is to add a function pointer such as "calculate" to the definition of struct "something_s". Later, during the struct initialization, assign a function address to that function pointer:</p> @@ -228,6 +373,7 @@ mult.calculate(mult,a,b)); <h2>Taking it further</h2> <p>If you want to take it further type "Object-Oriented Programming with ANSI-C" into your favorite internet search engine, you will find some crazy stuff. Some go as far as writing a C preprocessor in AWK, which takes some object oriented pseudo-C and transforms it to plain C so that the C compiler can compile it to machine code. This is actually similar to how the C++ language had its origins.</p> <p>E-Mail me your thoughts at comments@mx.buetow.org!</p> +<a class="textlink" href="../">Go back to the main site</a><br /> </div> </content> </entry> @@ -244,6 +390,7 @@ mult.calculate(mult,a,b)); <content type="xhtml"> <div xmlns="http://www.w3.org/1999/xhtml"> <h1>Spinning up my own authoritative DNS servers</h1> +<p class="quote"><i>Written by Paul Buetow 2016-05-22</i></p> <h2>Background</h2> <p>Finally, I had time to deploy my own authoritative DNS servers (master and slave) for my domains "buetow.org" and "buetow.zone". My domain name provider is Schlund Technologies. They allow their customers to manually edit the DNS records (BIND files). And they also give you the opportunity to set your own authoritative DNS servers for your domains. From now, I am making use of that option.</p> <a class="textlink" href="http://www.schlundtech.de">Schlund Technologies</a><br /> @@ -452,6 +599,7 @@ apply Service "dig6" { </ul> <p>That's much more comfortable now than manually clicking at some web UIs at Schlund Technologies.</p> <p>E-Mail me your thoughts at comments@mx.buetow.org!</p> +<a class="textlink" href="../">Go back to the main site</a><br /> </div> </content> </entry> @@ -482,11 +630,13 @@ apply Service "dig6" { | || | | | \____||__|_____|__| </pre> +<p class="quote"><i>Written by Paul Buetow 2016-04-16</i></p> <a class="textlink" href="https://buetow.org/gemfeed/2016-04-03-offsite-backup-with-zfs.html">Read the first part before reading any furter here...</a><br /> <p>I enhanced the procedure a bit. From now on I am having two external 2TB USB hard drives. Both are setup exactly the same way. To decrease the probability that they will not fail at about the same time both drives are of different brands. One drive is kept at the secret location. The other one is kept at home right next to my HP MicroServer.</p> <p>Whenever I am updating offsite backup, I am doing it to the drive which is kept locally. Afterwards I bring it to the secret location and swap the drives and bring the other one back home. This ensures that I will always have an offiste backup available at a different location than my home - even while updating one copy of it.</p> <p>Furthermore, I added scrubbing (*zpool scrub...*) to the script. It ensures that the file system is consistent and that there are no bad blocks on the disk and the file system. To increase the reliability I also run a *zfs set copies=2 zroot*. That setting is also synchronized to the offsite ZFS pool. ZFS stores every data block to disk twice now. Yes, it consumes twice as much disk space but it makes it better fault tolerant against hardware errors (e.g. only individual disk sectors going bad). </p> <p>E-Mail me your thoughts at comments@mx.buetow.org!</p> +<a class="textlink" href="../">Go back to the main site</a><br /> </div> </content> </entry> @@ -515,6 +665,7 @@ apply Service "dig6" { | || | | | \____||__|_____|__| </pre> +<p class="quote"><i>Written by Paul Buetow 2016-04-03</i></p> <h2>Please don't lose all my pictures again!</h2> <p>When it comes to data storage and potential data loss I am a paranoid person. It is not just due to my job but also due to a personal experience I encountered over 10 years ago: A single drive failure and loss of all my data (pictures, music, ....).</p> <p>A little about my personal infrastructure: I am running my own (mostly FreeBSD based) root servers (across several countries: Two in Germany, one in Canada, one in Bulgaria) which store all my online data (E-Mail and my Git repositories). I am syncing incremental (and encrypted) ZFS snapshots between these servers forth and back so either data could be recovered from the other server.</p> @@ -527,6 +678,7 @@ apply Service "dig6" { <h2>Walking one round less</h2> <p>I am thinking of buying a second 2TB USB drive and to set it up the same way as the first one. So I could alternate the backups. One drive would be at the secret location, and the other drive would be at home. And these drives would swap location after each cycle. This would give some security about the failure of that drive and I would have to go to the secret location only once (swapping the drives) instead of twice (picking that drive up in order to update the data + bringing it back to the secret location).</p> <p>E-Mail me your thoughts at comments@mx.buetow.org!</p> +<a class="textlink" href="../">Go back to the main site</a><br /> </div> </content> </entry> @@ -549,6 +701,7 @@ apply Service "dig6" { \, /-( /'-,\, /-( /'-, \, /-( / //\ //\\ //\ //\\ //\ //\\jrei </pre> +<p class="quote"><i>Written by Paul Buetow 2011-05-07, last updated 2021-05-07</i></p> <p>PerlDaemon is a minimal daemon for Linux and other Unix like operating systems programmed in Perl. It is a minimal but pretty functional and fairly generic service framework. This means that it does not do anything useful other than providing a framework for starting, stopping, configuring and logging. In order to do something useful, a module (written in Perl) must be provided.</p> <h2>Features</h2> <p>PerlDaemon supports:</p> @@ -672,6 +825,7 @@ sub do ($) { <p>You can find PerlDaemon (including the examples) at:</p> <a class="textlink" href="https://github.com/snonux/perldaemon">https://github.com/snonux/perldaemon</a><br /> <p>E-Mail me your thoughts at comments@mx.buetow.org!</p> +<a class="textlink" href="../">Go back to the main site</a><br /> </div> </content> </entry> @@ -688,6 +842,15 @@ sub do ($) { <content type="xhtml"> <div xmlns="http://www.w3.org/1999/xhtml"> <h1>The Fype Programming Language</h1> +<pre> + ____ _ __ + / / _|_ _ _ __ ___ _ _ ___ __ _| |__ / _|_ _ + / / |_| | | | '_ \ / _ \ | | | |/ _ \/ _` | '_ \ | |_| | | | + _ / /| _| |_| | |_) | __/ | |_| | __/ (_| | | | |_| _| |_| | +(_)_/ |_| \__, | .__/ \___| \__, |\___|\__,_|_| |_(_)_| \__, | + |___/|_| |___/ |___/ +</pre> +<p class="quote"><i>Written by Paul Buetow 2010-05-09, last updated 2021-05-05</i></p> <p>Fype is an interpreted programming language created by me for learning and fun. The interpreter is written in C. It has been tested on FreeBSD and NetBSD and may also work on other Unix like operating systems such as Linux based ones. To be honest, besides learning and fun there is really no other use case of why Fype actually exists as many other programming languages are much faster and more powerful.</p> <p>The Fype syntax is very simple and is using a maximum look ahead of 1 and a very easy top down parsing mechanism. Fype is parsing and interpreting its code simultaneously. This means, that syntax errors are only detected during program runtime. </p> <p>Fype is a recursive acronym and means "Fype is For Your Program Execution" or "Fype is Free Yak Programmed for ELF". You could also say "It's not a hype - it's Fype!".</p> @@ -1078,6 +1241,7 @@ BB <p>You can find all of this on the GitHub page. There is also an "examples" folders containing some Fype scripts!</p> <a class="textlink" href="https://github.com/snonux/fype">https://github.com/snonux/fype</a><br /> <p>E-Mail me your thoughts at comments@mx.buetow.org!</p> +<a class="textlink" href="../">Go back to the main site</a><br /> </div> </content> </entry> @@ -1116,6 +1280,7 @@ _~~|~/_|_|__/|~~~~~~~ | / ~~~~~ | | ~~~~~~~~ ~ ~ ~~~ _|| (_/ (___)_| |Nov291999 (__) (____) </pre> +<p class="quote"><i>Written by Paul Buetow 2008-06-26, last updated 2021-05-04</i></p> <p>Here are some Perl Poems I wrote. They don't do anything useful when you run them, but they don't produce a compiler error either. They only exist for fun and demonstrate what you can do with Perl syntax.</p> <p>Wikipedia: "Perl poetry is the practice of writing poems that can be compiled as legal Perl code, for example the piece known as Black Perl. Perl poetry is made possible by the large number of English words that are used in the Perl language. New poems are regularly submitted to the community at PerlMonks."</p> <a class="textlink" href="https://en.wikipedia.org/wiki/Perl">https://en.wikipedia.org/wiki/Perl</a><br /> @@ -1241,6 +1406,7 @@ This is perl, v5.8.8 built for i386-freebsd-64int <p>Did you like what you saw? Have a look at Github to see my other poems too:</p> <a class="textlink" href="https://github.com/snonux/perl-poetry">https://github.com/snonux/perl-poetry</a><br /> <p>E-Mail me your thoughts at comments@mx.buetow.org!</p> +<a class="textlink" href="../">Go back to the main site</a><br /> </div> </content> </entry> diff --git a/content/gemtext/gemfeed/index.gmi b/content/gemtext/gemfeed/index.gmi index c4272d76..6b35f04a 100644 --- a/content/gemtext/gemfeed/index.gmi +++ b/content/gemtext/gemfeed/index.gmi @@ -4,6 +4,7 @@ => ./2021-04-24-welcome-to-the-geminispace.gmi 2021-04-24 - Welcome to the Geminispace => ./2021-04-22-dtail-the-distributed-log-tail-program.gmi 2021-04-22 - DTail - The distributed log tail program +=> ./2018-06-01-realistic-load-testing-with-ioriot-for-linux.gmi 2018-06-01 - Realistic load testing with I/O Riot for Linux => ./2016-11-20-methods-in-c.gmi 2016-11-20 - Methods in C => ./2016-05-22-spinning-up-my-own-authoritative-dns-servers.gmi 2016-05-22 - Spinning up my own authoritative DNS servers => ./2016-04-16-offsite-backup-with-zfs-part2.gmi 2016-04-16 - Offsite backup with ZFS (Part 2) diff --git a/content/gemtext/index.gmi b/content/gemtext/index.gmi index ae8ce60f..3020e633 100644 --- a/content/gemtext/index.gmi +++ b/content/gemtext/index.gmi @@ -54,6 +54,7 @@ I have switched blog software multiple times. I might be back filling some of th => ./gemfeed/2021-04-24-welcome-to-the-geminispace.gmi 2021-04-24 - Welcome to the Geminispace => ./gemfeed/2021-04-22-dtail-the-distributed-log-tail-program.gmi 2021-04-22 - DTail - The distributed log tail program +=> ./gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux.gmi 2018-06-01 - Realistic load testing with I/O Riot for Linux => ./gemfeed/2016-11-20-methods-in-c.gmi 2016-11-20 - Methods in C => ./gemfeed/2016-05-22-spinning-up-my-own-authoritative-dns-servers.gmi 2016-05-22 - Spinning up my own authoritative DNS servers => ./gemfeed/2016-04-16-offsite-backup-with-zfs-part2.gmi 2016-04-16 - Offsite backup with ZFS (Part 2) diff --git a/content/html/contact-information.html b/content/html/contact-information.html index 08052fe2..658642be 100644 --- a/content/html/contact-information.html +++ b/content/html/contact-information.html @@ -15,7 +15,7 @@ body { } img { display:block; - max-width: 90%; + max-width: 80%; } p.quote:before { content: " | "; diff --git a/content/html/gemfeed/2008-06-26-perl-poetry.html b/content/html/gemfeed/2008-06-26-perl-poetry.html index c5bb38a9..160a4936 100644 --- a/content/html/gemfeed/2008-06-26-perl-poetry.html +++ b/content/html/gemfeed/2008-06-26-perl-poetry.html @@ -15,7 +15,7 @@ body { } img { display:block; - max-width: 90%; + max-width: 80%; } p.quote:before { content: " | "; @@ -48,8 +48,6 @@ h2, h3 { </style> </head> <body> -<p class="quote"><i>Written by Paul Buetow 2008-06-26, last updated 2021-05-04</i></p> -<a class="textlink" href="../">Go back to the main site</a><br /> <h1>Perl Poetry</h1> <pre> '\|/' * @@ -73,6 +71,7 @@ _~~|~/_|_|__/|~~~~~~~ | / ~~~~~ | | ~~~~~~~~ ~ ~ ~~~ _|| (_/ (___)_| |Nov291999 (__) (____) </pre> +<p class="quote"><i>Written by Paul Buetow 2008-06-26, last updated 2021-05-04</i></p> <p>Here are some Perl Poems I wrote. They don't do anything useful when you run them, but they don't produce a compiler error either. They only exist for fun and demonstrate what you can do with Perl syntax.</p> <p>Wikipedia: "Perl poetry is the practice of writing poems that can be compiled as legal Perl code, for example the piece known as Black Perl. Perl poetry is made possible by the large number of English words that are used in the Perl language. New poems are regularly submitted to the community at PerlMonks."</p> <a class="textlink" href="https://en.wikipedia.org/wiki/Perl">https://en.wikipedia.org/wiki/Perl</a><br /> @@ -198,5 +197,6 @@ This is perl, v5.8.8 built for i386-freebsd-64int <p>Did you like what you saw? Have a look at Github to see my other poems too:</p> <a class="textlink" href="https://github.com/snonux/perl-poetry">https://github.com/snonux/perl-poetry</a><br /> <p>E-Mail me your thoughts at comments@mx.buetow.org!</p> +<a class="textlink" href="../">Go back to the main site</a><br /> </body> </html> diff --git a/content/html/gemfeed/2010-05-09-the-fype-programming-language.html b/content/html/gemfeed/2010-05-09-the-fype-programming-language.html index b654644c..1088bf2a 100644 --- a/content/html/gemfeed/2010-05-09-the-fype-programming-language.html +++ b/content/html/gemfeed/2010-05-09-the-fype-programming-language.html @@ -15,7 +15,7 @@ body { } img { display:block; - max-width: 90%; + max-width: 80%; } p.quote:before { content: " | "; @@ -48,8 +48,7 @@ h2, h3 { </style> </head> <body> -<p class="quote"><i>Written by Paul Buetow 2010-05-09, last updated 2021-05-05</i></p> -<a class="textlink" href="../">Go back to the main site</a><br /> +<h1>The Fype Programming Language</h1> <pre> ____ _ __ / / _|_ _ _ __ ___ _ _ ___ __ _| |__ / _|_ _ @@ -57,9 +56,8 @@ h2, h3 { _ / /| _| |_| | |_) | __/ | |_| | __/ (_| | | | |_| _| |_| | (_)_/ |_| \__, | .__/ \___| \__, |\___|\__,_|_| |_(_)_| \__, | |___/|_| |___/ |___/ - </pre> -<h1>The Fype Programming Language</h1> +<p class="quote"><i>Written by Paul Buetow 2010-05-09, last updated 2021-05-05</i></p> <p>Fype is an interpreted programming language created by me for learning and fun. The interpreter is written in C. It has been tested on FreeBSD and NetBSD and may also work on other Unix like operating systems such as Linux based ones. To be honest, besides learning and fun there is really no other use case of why Fype actually exists as many other programming languages are much faster and more powerful.</p> <p>The Fype syntax is very simple and is using a maximum look ahead of 1 and a very easy top down parsing mechanism. Fype is parsing and interpreting its code simultaneously. This means, that syntax errors are only detected during program runtime. </p> <p>Fype is a recursive acronym and means "Fype is For Your Program Execution" or "Fype is Free Yak Programmed for ELF". You could also say "It's not a hype - it's Fype!".</p> @@ -450,5 +448,6 @@ BB <p>You can find all of this on the GitHub page. There is also an "examples" folders containing some Fype scripts!</p> <a class="textlink" href="https://github.com/snonux/fype">https://github.com/snonux/fype</a><br /> <p>E-Mail me your thoughts at comments@mx.buetow.org!</p> +<a class="textlink" href="../">Go back to the main site</a><br /> </body> </html> diff --git a/content/html/gemfeed/2011-05-07-perl-daemon-service-framework.html b/content/html/gemfeed/2011-05-07-perl-daemon-service-framework.html index 3e7ef78b..847cab0f 100644 --- a/content/html/gemfeed/2011-05-07-perl-daemon-service-framework.html +++ b/content/html/gemfeed/2011-05-07-perl-daemon-service-framework.html @@ -15,7 +15,7 @@ body { } img { display:block; - max-width: 90%; + max-width: 80%; } p.quote:before { content: " | "; @@ -48,8 +48,6 @@ h2, h3 { </style> </head> <body> -<p class="quote"><i>Written by Paul Buetow 2011-05-07, last updated 2021-05-07</i></p> -<a class="textlink" href="../">Go back to the main site</a><br /> <h1>Perl Daemon (Service Framework)</h1> <pre> a'! _,,_ a'! _,,_ a'! _,,_ @@ -57,6 +55,7 @@ h2, h3 { \, /-( /'-,\, /-( /'-, \, /-( / //\ //\\ //\ //\\ //\ //\\jrei </pre> +<p class="quote"><i>Written by Paul Buetow 2011-05-07, last updated 2021-05-07</i></p> <p>PerlDaemon is a minimal daemon for Linux and other Unix like operating systems programmed in Perl. It is a minimal but pretty functional and fairly generic service framework. This means that it does not do anything useful other than providing a framework for starting, stopping, configuring and logging. In order to do something useful, a module (written in Perl) must be provided.</p> <h2>Features</h2> <p>PerlDaemon supports:</p> @@ -180,5 +179,6 @@ sub do ($) { <p>You can find PerlDaemon (including the examples) at:</p> <a class="textlink" href="https://github.com/snonux/perldaemon">https://github.com/snonux/perldaemon</a><br /> <p>E-Mail me your thoughts at comments@mx.buetow.org!</p> +<a class="textlink" href="../">Go back to the main site</a><br /> </body> </html> diff --git a/content/html/gemfeed/2016-04-03-offsite-backup-with-zfs.html b/content/html/gemfeed/2016-04-03-offsite-backup-with-zfs.html index 1cfb9f2d..8eb59bd0 100644 --- a/content/html/gemfeed/2016-04-03-offsite-backup-with-zfs.html +++ b/content/html/gemfeed/2016-04-03-offsite-backup-with-zfs.html @@ -15,7 +15,7 @@ body { } img { display:block; - max-width: 90%; + max-width: 80%; } p.quote:before { content: " | "; @@ -48,8 +48,6 @@ h2, h3 { </style> </head> <body> -<p class="quote"><i>Written by Paul Buetow 2016-04-03</i></p> -<a class="textlink" href="../">Go back to the main site</a><br /> <h1>Offsite backup with ZFS</h1> <pre> ________________ @@ -63,6 +61,7 @@ h2, h3 { | || | | | \____||__|_____|__| </pre> +<p class="quote"><i>Written by Paul Buetow 2016-04-03</i></p> <h2>Please don't lose all my pictures again!</h2> <p>When it comes to data storage and potential data loss I am a paranoid person. It is not just due to my job but also due to a personal experience I encountered over 10 years ago: A single drive failure and loss of all my data (pictures, music, ....).</p> <p>A little about my personal infrastructure: I am running my own (mostly FreeBSD based) root servers (across several countries: Two in Germany, one in Canada, one in Bulgaria) which store all my online data (E-Mail and my Git repositories). I am syncing incremental (and encrypted) ZFS snapshots between these servers forth and back so either data could be recovered from the other server.</p> @@ -75,5 +74,6 @@ h2, h3 { <h2>Walking one round less</h2> <p>I am thinking of buying a second 2TB USB drive and to set it up the same way as the first one. So I could alternate the backups. One drive would be at the secret location, and the other drive would be at home. And these drives would swap location after each cycle. This would give some security about the failure of that drive and I would have to go to the secret location only once (swapping the drives) instead of twice (picking that drive up in order to update the data + bringing it back to the secret location).</p> <p>E-Mail me your thoughts at comments@mx.buetow.org!</p> +<a class="textlink" href="../">Go back to the main site</a><br /> </body> </html> diff --git a/content/html/gemfeed/2016-04-16-offsite-backup-with-zfs-part2.html b/content/html/gemfeed/2016-04-16-offsite-backup-with-zfs-part2.html index 3e6d0206..0939c297 100644 --- a/content/html/gemfeed/2016-04-16-offsite-backup-with-zfs-part2.html +++ b/content/html/gemfeed/2016-04-16-offsite-backup-with-zfs-part2.html @@ -15,7 +15,7 @@ body { } img { display:block; - max-width: 90%; + max-width: 80%; } p.quote:before { content: " | "; @@ -48,8 +48,6 @@ h2, h3 { </style> </head> <body> -<p class="quote"><i>Written by Paul Buetow 2016-04-16</i></p> -<a class="textlink" href="../">Go back to the main site</a><br /> <h1>Offsite backup with ZFS (Part 2)</h1> <pre> ________________ @@ -65,10 +63,12 @@ h2, h3 { | || | | | \____||__|_____|__| </pre> +<p class="quote"><i>Written by Paul Buetow 2016-04-16</i></p> <a class="textlink" href="./2016-04-03-offsite-backup-with-zfs.html">Read the first part before reading any furter here...</a><br /> <p>I enhanced the procedure a bit. From now on I am having two external 2TB USB hard drives. Both are setup exactly the same way. To decrease the probability that they will not fail at about the same time both drives are of different brands. One drive is kept at the secret location. The other one is kept at home right next to my HP MicroServer.</p> <p>Whenever I am updating offsite backup, I am doing it to the drive which is kept locally. Afterwards I bring it to the secret location and swap the drives and bring the other one back home. This ensures that I will always have an offiste backup available at a different location than my home - even while updating one copy of it.</p> <p>Furthermore, I added scrubbing (*zpool scrub...*) to the script. It ensures that the file system is consistent and that there are no bad blocks on the disk and the file system. To increase the reliability I also run a *zfs set copies=2 zroot*. That setting is also synchronized to the offsite ZFS pool. ZFS stores every data block to disk twice now. Yes, it consumes twice as much disk space but it makes it better fault tolerant against hardware errors (e.g. only individual disk sectors going bad). </p> <p>E-Mail me your thoughts at comments@mx.buetow.org!</p> +<a class="textlink" href="../">Go back to the main site</a><br /> </body> </html> diff --git a/content/html/gemfeed/2016-05-22-spinning-up-my-own-authoritative-dns-servers.html b/content/html/gemfeed/2016-05-22-spinning-up-my-own-authoritative-dns-servers.html index 04f1c58f..eed4561b 100644 --- a/content/html/gemfeed/2016-05-22-spinning-up-my-own-authoritative-dns-servers.html +++ b/content/html/gemfeed/2016-05-22-spinning-up-my-own-authoritative-dns-servers.html @@ -15,7 +15,7 @@ body { } img { display:block; - max-width: 90%; + max-width: 80%; } p.quote:before { content: " | "; @@ -48,9 +48,8 @@ h2, h3 { </style> </head> <body> -<p class="quote"><i>Written by Paul Buetow 2016-05-22</i></p> -<a class="textlink" href="../">Go back to the main site</a><br /> <h1>Spinning up my own authoritative DNS servers</h1> +<p class="quote"><i>Written by Paul Buetow 2016-05-22</i></p> <h2>Background</h2> <p>Finally, I had time to deploy my own authoritative DNS servers (master and slave) for my domains "buetow.org" and "buetow.zone". My domain name provider is Schlund Technologies. They allow their customers to manually edit the DNS records (BIND files). And they also give you the opportunity to set your own authoritative DNS servers for your domains. From now, I am making use of that option.</p> <a class="textlink" href="http://www.schlundtech.de">Schlund Technologies</a><br /> @@ -259,5 +258,6 @@ apply Service "dig6" { </ul> <p>That's much more comfortable now than manually clicking at some web UIs at Schlund Technologies.</p> <p>E-Mail me your thoughts at comments@mx.buetow.org!</p> +<a class="textlink" href="../">Go back to the main site</a><br /> </body> </html> diff --git a/content/html/gemfeed/2016-11-20-methods-in-c.html b/content/html/gemfeed/2016-11-20-methods-in-c.html index c2ae11e4..544ad28f 100644 --- a/content/html/gemfeed/2016-11-20-methods-in-c.html +++ b/content/html/gemfeed/2016-11-20-methods-in-c.html @@ -15,7 +15,7 @@ body { } img { display:block; - max-width: 90%; + max-width: 80%; } p.quote:before { content: " | "; @@ -48,9 +48,8 @@ h2, h3 { </style> </head> <body> -<p class="quote"><i>Written by Paul Buetow 2016-11-20</i></p> -<a class="textlink" href="../">Go back to the main site</a><br /> <h1>Methods in C</h1> +<p class="quote"><i>Written by Paul Buetow 2016-11-20</i></p> <p>You can do some sort of object oriented programming in the C Programming Language. However, that is very limited. But also very easy and straight forward to use.</p> <h2>Example</h2> <p>Lets have a look at the following sample program. Basically all you have to do is to add a function pointer such as "calculate" to the definition of struct "something_s". Later, during the struct initialization, assign a function address to that function pointer:</p> @@ -114,5 +113,6 @@ mult.calculate(mult,a,b)); <h2>Taking it further</h2> <p>If you want to take it further type "Object-Oriented Programming with ANSI-C" into your favorite internet search engine, you will find some crazy stuff. Some go as far as writing a C preprocessor in AWK, which takes some object oriented pseudo-C and transforms it to plain C so that the C compiler can compile it to machine code. This is actually similar to how the C++ language had its origins.</p> <p>E-Mail me your thoughts at comments@mx.buetow.org!</p> +<a class="textlink" href="../">Go back to the main site</a><br /> </body> </html> diff --git a/content/html/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux.draft.html b/content/html/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux.draft.html index 8e5db6f3..47075196 100644 --- a/content/html/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux.draft.html +++ b/content/html/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux.draft.html @@ -15,7 +15,7 @@ body { } img { display:block; - max-width: 90%; + max-width: 80%; } p.quote:before { content: " | "; @@ -48,9 +48,7 @@ h2, h3 { </style> </head> <body> -<p> Written by Paul Buetow 2018-06-01, last updated 2021-05-08</p> -<a class="textlink" href="../">Go back to the main site</a><br /> -<h1>Realistic load testing with I/O Riot for Linux</h1> +<p># Realistic load testing with I/O Riot for Linux</p> <pre> .---. / \ @@ -61,6 +59,7 @@ h2, h3 { /`\_`> <_/ \ jgs\__/'---'\__/ </pre> +<p class="quote"><i>Written by Paul Buetow 2018-06-01, last updated 2021-05-08</i></p> <h2>Foreword</h2> <p>This text first was published in the german IT-Administrator computer Magazine. 3 years have passed since and I decided to publish it on my blog too. </p> <a class="textlink" href="https://www.admin-magazin.de/Das-Heft/2018/06/Realistische-Lasttests-mit-I-O-Riot">https://www.admin-magazin.de/Das-Heft/2018/06/Realistische-Lasttests-mit-I-O-Riot</a><br /> @@ -173,5 +172,6 @@ Total time: 1213.00s <a class="textlink" href="https://graphiteapp.org">Graphite</a><br /> <a class="textlink" href="https://en.wikipedia.org/wiki/Memory-mapped_I/O">Memory mapped I/O</a><br /> <p>E-Mail me your thoughts at comments@mx.buetow.org!</p> +<a class="textlink" href="../">Go back to the main site</a><br /> </body> </html> diff --git a/content/html/gemfeed/2021-04-22-dtail-the-distributed-log-tail-program.html b/content/html/gemfeed/2021-04-22-dtail-the-distributed-log-tail-program.html index 56db435e..9e1e6e16 100644 --- a/content/html/gemfeed/2021-04-22-dtail-the-distributed-log-tail-program.html +++ b/content/html/gemfeed/2021-04-22-dtail-the-distributed-log-tail-program.html @@ -15,7 +15,7 @@ body { } img { display:block; - max-width: 90%; + max-width: 80%; } p.quote:before { content: " | "; @@ -48,9 +48,8 @@ h2, h3 { </style> </head> <body> -<p class="quote"><i>Written by Paul Buetow 2021-04-22, last updated 2021-04-26</i></p> -<a class="textlink" href="../">Go back to the main site</a><br /> <h1>DTail - The distributed log tail program</h1> +<p class="quote"><i>Written by Paul Buetow 2021-04-22, last updated 2021-04-26</i></p> <i>DTail logo image:</i><a href="./2021-04-22-dtail-the-distributed-log-tail-program/dtail-logo.png"><img alt="DTail logo image" title="DTail logo image" src="./2021-04-22-dtail-the-distributed-log-tail-program/dtail-logo.png" /></a><br /> <p>This article first appeared at the Mimecast Engineering Blog but I made it available here in my personal Gemini capsule too.</p> <a class="textlink" href="https://medium.com/mimecast-engineering/dtail-the-distributed-log-tail-program-79b8087904bb">Original Mimecast Engineering Blog post at Medium</a><br /> @@ -115,5 +114,6 @@ dtail –servers serverlist.txt –files ‘/var/log/*.log’ –regex ‘(?i:er <p>Mimecast highly encourages you to have a look at DTail and submit an issue for any features you would like to see. Have you found a bug? Maybe you just have a question or comment? If you want to go a step further: We would also love to see pull requests for any features or improvements. Either way, if in doubt just contact us via the DTail GitHub page.</p> <a class="textlink" href="https://dtail.dev">https://dtail.dev</a><br /> <p>E-Mail me your thoughts at comments@mx.buetow.org!</p> +<a class="textlink" href="../">Go back to the main site</a><br /> </body> </html> diff --git a/content/html/gemfeed/2021-04-24-welcome-to-the-geminispace.html b/content/html/gemfeed/2021-04-24-welcome-to-the-geminispace.html index e522d1ed..2a30510d 100644 --- a/content/html/gemfeed/2021-04-24-welcome-to-the-geminispace.html +++ b/content/html/gemfeed/2021-04-24-welcome-to-the-geminispace.html @@ -15,7 +15,7 @@ body { } img { display:block; - max-width: 90%; + max-width: 80%; } p.quote:before { content: " | "; @@ -48,10 +48,8 @@ h2, h3 { </style> </head> <body> -<p class="quote"><i>Written by Paul Buetow 2021-04-24, last updated 2021-04-30</i></p> -<p class="quote"><i>ASCII Art by Andy Hood</i></p> -<a class="textlink" href="../">Go back to the main site</a><br /> <h1>Welcome to the Geminispace</h1> +<p class="quote"><i>Written by Paul Buetow 2021-04-24, last updated 2021-04-30, ASCII Art by Andy Hood</i></p> <p>Have you reached this article already via Gemini? You need a special client for that, web browsers such as Firefox, Chrome, Safari etc. don't support the Gemini protocol. The Gemini address of this site (or the address of this capsule as people say in Geminispace) is:</p> <a class="textlink" href="gemini://buetow.org">gemini://buetow.org</a><br /> <p>If you however still use HTTP then you are just surfing the fallback HTML version of this capsule. In that case I suggest reading on what this is all about :-).</p> @@ -102,5 +100,6 @@ h2, h3 { <a class="textlink" href="gemini://gemini.circumlunar.space">gemini://gemini.circumlunar.space</a><br /> <a class="textlink" href="https://gemini.circumlunar.space">https://gemini.circumlunar.space</a><br /> <p>E-Mail me your thoughts at comments@mx.buetow.org!</p> +<a class="textlink" href="../">Go back to the main site</a><br /> </body> </html> diff --git a/content/html/gemfeed/2021-05-15-buetow.org.sh-One-Bash-script-to-rule-it-all.draft.html b/content/html/gemfeed/2021-05-15-buetow.org.sh-One-Bash-script-to-rule-it-all.draft.html index 32272e4d..e1f53ec8 100644 --- a/content/html/gemfeed/2021-05-15-buetow.org.sh-One-Bash-script-to-rule-it-all.draft.html +++ b/content/html/gemfeed/2021-05-15-buetow.org.sh-One-Bash-script-to-rule-it-all.draft.html @@ -15,7 +15,7 @@ body { } img { display:block; - max-width: 90%; + max-width: 80%; } p.quote:before { content: " | "; @@ -48,9 +48,8 @@ h2, h3 { </style> </head> <body> -<p>ADD WRITTEN BY AND CREATED AT BLABLA</p> -<p>ADD GO BACK LINK</p> <h1>buetow.org.sh - One Bash script to rule it all</h1> +<p class="quote"><i>TODO: ADD WRITTEN BY AND CREATED AT BLABLA</i></p> <p>You might have read my previous blog post about entering the Geminispace.</p> <a class="textlink" href="./2021-04-24-welcome-to-the-geminispace">Welcome to the Geminispace</a><br /> <h2>Motivation</h2> @@ -181,5 +180,6 @@ atomfeed::meta () { <a class="textlink" href="https://buetow.org">https://buetow.org - The original Gemini capsule </a><br /> <a class="textlink" href="https://buetow.org/gemfeed/">https://buetow.org/gemfeed/ - The Gemfeed</a><br /> <a class="textlink" href="https://buetow.org/gemfeed/atom.xml">https://buetow.org/gemfeed/atom.xml - The Atom feed</a><br /> +<p>TODO: ADD GO BACK LINK</p> </body> </html> diff --git a/content/html/gemfeed/atom.xml b/content/html/gemfeed/atom.xml index 514f6a47..8f66a786 100644 --- a/content/html/gemfeed/atom.xml +++ b/content/html/gemfeed/atom.xml @@ -1,6 +1,6 @@ <?xml version="1.0" encoding="utf-8"?> <feed xmlns="http://www.w3.org/2005/Atom"> - <updated>2021-05-10T10:15:48+01:00</updated> + <updated>2021-05-14T09:34:34+01:00</updated> <title>buetow.org feed</title> <subtitle>Having fun with computers!</subtitle> <link href="https://buetow.org/gemfeed/atom.xml" rel="self" /> @@ -19,6 +19,7 @@ <content type="xhtml"> <div xmlns="http://www.w3.org/1999/xhtml"> <h1>Welcome to the Geminispace</h1> +<p class="quote"><i>Written by Paul Buetow 2021-04-24, last updated 2021-04-30, ASCII Art by Andy Hood</i></p> <p>Have you reached this article already via Gemini? You need a special client for that, web browsers such as Firefox, Chrome, Safari etc. don't support the Gemini protocol. The Gemini address of this site (or the address of this capsule as people say in Geminispace) is:</p> <a class="textlink" href="https://buetow.org">https://buetow.org</a><br /> <p>If you however still use HTTP then you are just surfing the fallback HTML version of this capsule. In that case I suggest reading on what this is all about :-).</p> @@ -69,6 +70,7 @@ <a class="textlink" href="https://gemini.circumlunar.space">https://gemini.circumlunar.space</a><br /> <a class="textlink" href="https://gemini.circumlunar.space">https://gemini.circumlunar.space</a><br /> <p>E-Mail me your thoughts at comments@mx.buetow.org!</p> +<a class="textlink" href="../">Go back to the main site</a><br /> </div> </content> </entry> @@ -85,6 +87,7 @@ <content type="xhtml"> <div xmlns="http://www.w3.org/1999/xhtml"> <h1>DTail - The distributed log tail program</h1> +<p class="quote"><i>Written by Paul Buetow 2021-04-22, last updated 2021-04-26</i></p> <i>DTail logo image:</i><a href="https://buetow.org/gemfeed/2021-04-22-dtail-the-distributed-log-tail-program/dtail-logo.png"><img alt="DTail logo image" title="DTail logo image" src="https://buetow.org/gemfeed/2021-04-22-dtail-the-distributed-log-tail-program/dtail-logo.png" /></a><br /> <p>This article first appeared at the Mimecast Engineering Blog but I made it available here in my personal Gemini capsule too.</p> <a class="textlink" href="https://medium.com/mimecast-engineering/dtail-the-distributed-log-tail-program-79b8087904bb">Original Mimecast Engineering Blog post at Medium</a><br /> @@ -149,6 +152,147 @@ dtail –servers serverlist.txt –files ‘/var/log/*.log’ –regex ‘(?i:er <p>Mimecast highly encourages you to have a look at DTail and submit an issue for any features you would like to see. Have you found a bug? Maybe you just have a question or comment? If you want to go a step further: We would also love to see pull requests for any features or improvements. Either way, if in doubt just contact us via the DTail GitHub page.</p> <a class="textlink" href="https://dtail.dev">https://dtail.dev</a><br /> <p>E-Mail me your thoughts at comments@mx.buetow.org!</p> +<a class="textlink" href="../">Go back to the main site</a><br /> + </div> + </content> + </entry> + <entry> + <title>The article</title> + <link href="https://buetow.org/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux.html" /> + <id>https://buetow.org/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux.html</id> + <updated>2018-06-01T09:30:27+01:00</updated> + <author> + <name>Paul Buetow</name> + <email>comments@mx.buetow.org</email> + </author> + <summary>This text first was published in the german IT-Administrator computer Magazine. 3 years have passed since and I decided to publish it on my blog too. . .....to read on please visit my site.</summary> + <content type="xhtml"> + <div xmlns="http://www.w3.org/1999/xhtml"> + <h1>Realistic load testing with I/O Riot for Linux</h1> +<pre> + .---. + / \ + \.@-@./ + /`\_/`\ + // _ \\ + | \ )|_ + /`\_`> <_/ \ +jgs\__/'---'\__/ +</pre> +<p class="quote"><i>Written by Paul Buetow 2018-06-01, last updated 2021-05-08</i></p> +<h2>Foreword</h2> +<p>This text first was published in the german IT-Administrator computer Magazine. 3 years have passed since and I decided to publish it on my blog too. </p> +<a class="textlink" href="https://www.admin-magazin.de/Das-Heft/2018/06/Realistische-Lasttests-mit-I-O-Riot">https://www.admin-magazin.de/Das-Heft/2018/06/Realistische-Lasttests-mit-I-O-Riot</a><br /> +<p>I havn't worked on I/O Riot for some time now, but all what is written here is still valid. I am still using I/O Riot to debug I/O issues and pattern once in a while, so by all means the tool is not obsolete yet. The tool even helped to resolve a major production incident at work involving I/O.</p> +<p>I am eagerly looking forward to revamp I/O Riot so that it uses the new BPF Linux capabilities instead of Systemtap. Also, when I wrote I/O Riot initially, I didn't have any experience with the Go programming language yet and therefore I wrote it in C. Once it gets revamped I might consider using Go instead of C as it would spare me from many segmentation faults and headaches during development ;-). I might also just stick to C for plain performance reasons and just refactor the code dealing with concurrency.</p> +<p>Pleace notice that some of the screenshots show the command "ioreplay" instead of "ioriot". That's because the name has changed after taking those.</p> +<h1>The article</h1> +<p>With I/O Riot IT administrators can load test and optimize the I/O subsystem of Linux-based operating systems. The tool makes it possible to record I/O patterns and replay them at a later time as often as desired. This means bottlenecks can be reproduced and eradicated. </p> +<p>When storing huge amounts of data, such as more than 200 billion archived emails at Mimecast, it's not only the available storage capacity that matters, but also the data throughput and latency. At the same time, operating costs must be kept as low as possible. The more systems involved, the more important it is to optimize the hardware, the operating system and the applications running on it.</p> +<h2>Background: Existing Techniques</h2> +<p>Conventional I/O benchmarking: Administrators usually use open source benchmarking tools like IOZone and bonnie++. Available database systems such as Redis and MySQL come with their own benchmarking tools. The common problem with these tools is that they work with prescribed artificial I/O patterns. Although this can test both sequential and randomized data access, the patterns do not correspond to what can be found on production systems.</p> +<p>Testing by load test environment: Another option is to use a separate load test environment in which, as far as possible, a production environment with all its dependencies is simulated. However, an environment consisting of many microservices is very complex. Microservices are usually managed by different teams, which means extra coordination effort for each load test. Another challenge is to generate the load as authentically as possible so that the patterns correspond to a productive environment. Such a load test environment can only handle as many requests as its weakest link can handle. For example, load generators send many read and write requests to a frontend microservice, whereby the frontend forwards the requests to a backend microservice responsible for storing the data. If the frontend service does not process the requests efficiently enough, the backend service is not well utilized in the first place. As a rule, all microservices are clustered across many servers, which makes everything even more complicated. Under all these conditions it is very difficult to test I/O of separate backend systems. Moreover, for many small and medium-sized companies, a separate load test environment would not be feasible for cost reasons.</p> +<p>Testing in the production environment: For these reasons, benchmarks are often carried out in the production environment. In order to derive value from this such tests are especially performed during peak hours when systems are under high load. However, testing on production systems is associated with risks and can lead to failure or loss of data without adequate protection.</p> +<h2>Benchmarking the Email Cloud at Mimecast</h2> +<p>For email archiving, Mimecast uses an internally developed microservice, which is operated directly on Linux-based storage systems. A storage cluster is divided into several replication volumes. Data is always replicated three times across two secure data centers. Customer data is automatically allocated to one or more volumes, depending on throughput, so that all volumes are automatically assigned the same load. Customer data is archived on conventional, but inexpensive hard disks with several terabytes of storage capacity each. I/O benchmarking proved difficult for all the reasons mentioned above. Furthermore, there are no ready-made tools for this purpose in the case of self-developed software. The service operates on many block devices simultaneously, which can make the RAID controller a bottleneck. None of the freely available benchmarking tools can test several block devices at the same time without extra effort. In addition, emails typically consist of many small files. Randomized access to many small files is particularly inefficient. In addition to many software adaptations, the hardware and operating system must also be optimized.</p> +<p>Mimecast encourages employees to be innovative and pursue their own ideas in the form of an internal competition, Pet Project. The goal of the pet project I/O Riot was to simplify OS and hardware level I/O benchmarking. The first prototype of I/O Riot was awarded an internal roadmap prize in the spring of 2017. A few months later, I/O Riot was used to reduce write latency in the storage clusters by about 50%. The improvement was first verified by I/O replay on a test system and then successively applied to all storage systems. I/O Riot was also used to resolve a production incident related to disk I/O load.</p> +<h2>Using I/O Riot</h2> +<p>First, all I/O events are logged to a file on a production system with I/O Riot. It is then copied to a test system where all events are replayed in the same way. The crucial point here is that you can reproduce I/O patterns as they are found on a production system as often as you like on a test system. This results in the possibility of optimizing the set screws on the system after each run.</p> +<h3>Installation</h3> +<p>I/O Riot was tested under CentOS 7.2 x86_64. For compiling, the GNU C compiler and Systemtap including kernel debug information are required. Other Linux distributions are theoretically compatible but untested. First of all, you should update the systems involved as follows:</p> +<pre> +% sudo yum update +</pre> +<p>If the kernel is updated, please restart the system. The installation would be done without a restart but this would complicate the installation. The installed kernel version should always correspond to the currently running kernel. You can then install I/O Riot as follows:</p> +<pre> +% sudo yum install gcc git systemtap yum-utils kernel-devel-$(uname -r) +% sudo debuginfo-install kernel-$(uname -r) +% git clone https://github.com/mimecast/ioriot +% cd ioriot +% make +% sudo make install +% export PATH=$PATH:/opt/ioriot/bin +</pre> +<p>Note: It is not best practice to install any compilers on production systems. For further information please have a look at the enclosed README.md.</p> +<h3>Recording of I/O events</h3> +<p>All I/O events are kernel related. If a process wants to perform an I/O operation, such as opening a file, it must inform the kernel of this by a system call (short syscall). I/O Riot relies on the Systemtap tool to record I/O syscalls. Systemtap, available for all popular Linux distributions, helps you to take a look at the running kernel in productive environments, which makes it predestined to monitor all I/O-relevant Linux syscalls and log them to a file. Other tools, such as strace, are not an alternative because they slow down the system too much.</p> +<p>During recording, ioriot acts as a wrapper and executes all relevant Systemtap commands for you. Use the following command to log all events to io.capture:</p> +<pre> +% sudo ioriot -c io.capture +</pre> +<i>Screenshot I/O recording:</i><a href="https://buetow.org/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux/figure1-ioriot-io-recording.png"><img alt="Screenshot I/O recording" title="Screenshot I/O recording" src="https://buetow.org/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux/figure1-ioriot-io-recording.png" /></a><br /> +<p>A Ctrl-C (SIGINT) stops recording prematurely. Otherwise, ioriot terminates itself automatically after 1 hour. Depending on the system load, the output file can grow to several gigabytes. Only metadata is logged, not the read and written data itself. When replaying later, only random data is used. Under certain circumstances, Systemtap may omit some system calls and issue warnings. This is to ensure that Systemtap does not consume too many resources.</p> +<h3>Test preparation</h3> +<p>Then copy io.capture to a test system. The log also contains all accesses to the pseudo file systems devfs, sysfs and procfs. This makes little sense, which is why you must first generate a cleaned and playable version io.replay from io.capture as follows:</p> +<pre> +% sudo ioriot -c io.capture -r io.replay -u $USER -n TESTNAME +</pre> +<p>The parameter -n allows you to assign a freely selectable test name. An arbitrary system user under which the test is to be played is specified via paramater -u.</p> +<h3>Test Initialization</h3> +<p>The test will most likely want to access existing files. These are files the test wants to read but does not create by itself. The existence of these must be ensured before the test. You can do this as follows:</p> +<pre> +% sudo ioriot -i io.replay +</pre> +<p>To avoid any damage to the running system, ioreplay only works in special directories. The tool creates a separate subdirectory for each file system mount point (e.g. /, /usr/local, /store/00,...) (here: /.ioriot/TESTNAME, /usr/local/.ioriot/TESTNAME, /store/00/.ioriot/TESTNAME,...). By default, the working directory of ioriot is /usr/local/ioriot/TESTNAME.</p> +<i>Screenshot test preparation:</i><a href="https://buetow.org/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux/figure2-ioriot-test-preparation.png"><img alt="Screenshot test preparation" title="Screenshot test preparation" src="https://buetow.org/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux/figure2-ioriot-test-preparation.png" /></a><br /> +<p>You must re-initialize the environment before each run. Data from previous tests will be moved to a trash directory automatically, which can be finally deleted with "sudo ioriot -P".</p> +<h3>Replay</h3> +<p>After initialization, you can replay the log with -r. You can use -R to initiate both test initialization and replay in a single command and -S can be used to specify a file in which statistics are written after the test run.</p> +<p>You can also influence the playback speed: "-s 0" is interpreted as "Playback as fast as possible" and is the default setting. With "-s 1" all operations are performed at original speed. "-s 2" would double the playback speed and "-s 0.5" would halve it.</p> +<i>Screenshot replaying I/O:</i><a href="https://buetow.org/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux/figure3-ioriot-replay.png"><img alt="Screenshot replaying I/O" title="Screenshot replaying I/O" src="https://buetow.org/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux/figure3-ioriot-replay.png" /></a><br /> +<p>As an initial test, for example, you could compare the two Linux I/O schedulers CFQ and Deadline and check which scheduler the test runs the fastest. They run the test separately for each scheduler. The following shell loop iterates through all attached block devices of the system and changes their I/O scheduler to the one specified in variable $new_scheduler (in this case either cfq or deadline). Subsequently, all I/O events from the io.replay protocol are played back. At the end, an output file with statistics is generated:</p> +<pre> +% new_scheduler=cfq +% for scheduler in /sys/block/*/queue/scheduler; do + echo $new_scheduler | sudo tee $scheduler +done +% sudo ioriot -R io.replay -S cfq.txt +% new_scheduler=deadline +% for scheduler in /sys/block/*/queue/scheduler; do + echo $new_scheduler | sudo tee $scheduler +done +% sudo ioriot -R io.replay -S deadline.txt +</pre> +<p>According to the results, the test could run 940 seconds faster with Deadline Scheduler:</p> +<pre> +% cat cfq.txt +Num workers: 4 +hreads per worker: 128 +otal threads: 512 +Highest loadavg: 259.29 +Performed ioops: 218624596 +Average ioops/s: 101544.17 +Time ahead: 1452s +Total time: 2153.00s +% cat deadline.txt +Num workers: 4 +Threads per worker: 128 +Total threads: 512 +Highest loadavg: 342.45 +Performed ioops: 218624596 +Average ioops/s: 180234.62 +Time ahead: 2392s +Total time: 1213.00s +</pre> +<p>In any case, you should also set up a time series database, such as Graphite, where the I/O throughput can be plotted. Figures 4 and 5 show the read and write access times of both tests. The break-in makes it clear when the CFQ test ended and the deadline test was started. The reading latency of both tests is similar. Write latency is dramatically improved using the Deadline Scheduler.</p> +<i>Graphite visualization of the mean read access times in ms with CFQ and Deadline Scheduler.:</i><a href="https://buetow.org/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux/figure4-ioriot-read-latency.png"><img alt="Graphite visualization of the mean read access times in ms with CFQ and Deadline Scheduler." title="Graphite visualization of the mean read access times in ms with CFQ and Deadline Scheduler." src="https://buetow.org/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux/figure4-ioriot-read-latency.png" /></a><br /> +<i>Graphite visualization of the average write access times in ms with CFQ and Deadline Scheduler.:</i><a href="https://buetow.org/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux/figure5-ioriot-write-latency.png"><img alt="Graphite visualization of the average write access times in ms with CFQ and Deadline Scheduler." title="Graphite visualization of the average write access times in ms with CFQ and Deadline Scheduler." src="https://buetow.org/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux/figure5-ioriot-write-latency.png" /></a><br /> +<p>You should also take a look at the iostat tool. The iostat screenshot shows the output of iostat -x 10 during a test run. As you can see, a block device is fully loaded with 99% utilization, while all other block devices still have sufficient buffer. This could be an indication of poor data distribution in the storage system and is worth pursuing. It is not uncommon for I/O Riot to reveal software problems.</p> +<i>Output of iostat. The block device sdy seems to be almost fully utilized by 99%.:</i><a href="https://buetow.org/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux/figure6-iostat.png"><img alt="Output of iostat. The block device sdy seems to be almost fully utilized by 99%." title="Output of iostat. The block device sdy seems to be almost fully utilized by 99%." src="https://buetow.org/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux/figure6-iostat.png" /></a><br /> +<h2>I/O Riot is Open Source</h2> +<p>The tool has already proven to be very useful and will continue to be actively developed as time and priority permits. Mimecast intends to be an ongoing contributor to Open Source. You can find I/O Riot at:</p> +<a class="textlink" href="https://github.com/mimecast/ioriot">https://github.com/mimecast/ioriot</a><br /> +<h2>Systemtap</h2> +<p>Systemtap is a tool for the instrumentation of the Linux kernel. The tool provides an AWK-like programming language. Programs written in it are compiled from Systemtap to C- and then into a dynamically loadable kernel module. Loaded into the kernel, the program has access to Linux internals. A Systemtap program written for I/O Riot monitors when, with which parameters, at which time, and from which process I/O syscalls take place and their return values.</p> +<p>For example, the open syscall opens a file and returns the responsible file descriptor. The read and write syscalls can operate on a file descriptor and return the number of read or written bytes. The close syscall closes a given file descriptor. I/O Riot comes with a ready-made Systemtap program, which you have already compiled into a kernel module and installed to /opt/ioriot. In addition to open, read and close, it logs many other I/O-relevant calls.</p> +<a class="textlink" href="https://sourceware.org/systemtap/">https://sourceware.org/systemtap/</a><br /> +<h2>More refereces</h2> +<a class="textlink" href="http://www.iozone.org/">IOZone</a><br /> +<a class="textlink" href="https://www.coker.com.au/bonnie++/">Bonnie++</a><br /> +<a class="textlink" href="https://graphiteapp.org">Graphite</a><br /> +<a class="textlink" href="https://en.wikipedia.org/wiki/Memory-mapped_I/O">Memory mapped I/O</a><br /> +<p>E-Mail me your thoughts at comments@mx.buetow.org!</p> +<a class="textlink" href="../">Go back to the main site</a><br /> </div> </content> </entry> @@ -165,6 +309,7 @@ dtail –servers serverlist.txt –files ‘/var/log/*.log’ –regex ‘(?i:er <content type="xhtml"> <div xmlns="http://www.w3.org/1999/xhtml"> <h1>Methods in C</h1> +<p class="quote"><i>Written by Paul Buetow 2016-11-20</i></p> <p>You can do some sort of object oriented programming in the C Programming Language. However, that is very limited. But also very easy and straight forward to use.</p> <h2>Example</h2> <p>Lets have a look at the following sample program. Basically all you have to do is to add a function pointer such as "calculate" to the definition of struct "something_s". Later, during the struct initialization, assign a function address to that function pointer:</p> @@ -228,6 +373,7 @@ mult.calculate(mult,a,b)); <h2>Taking it further</h2> <p>If you want to take it further type "Object-Oriented Programming with ANSI-C" into your favorite internet search engine, you will find some crazy stuff. Some go as far as writing a C preprocessor in AWK, which takes some object oriented pseudo-C and transforms it to plain C so that the C compiler can compile it to machine code. This is actually similar to how the C++ language had its origins.</p> <p>E-Mail me your thoughts at comments@mx.buetow.org!</p> +<a class="textlink" href="../">Go back to the main site</a><br /> </div> </content> </entry> @@ -244,6 +390,7 @@ mult.calculate(mult,a,b)); <content type="xhtml"> <div xmlns="http://www.w3.org/1999/xhtml"> <h1>Spinning up my own authoritative DNS servers</h1> +<p class="quote"><i>Written by Paul Buetow 2016-05-22</i></p> <h2>Background</h2> <p>Finally, I had time to deploy my own authoritative DNS servers (master and slave) for my domains "buetow.org" and "buetow.zone". My domain name provider is Schlund Technologies. They allow their customers to manually edit the DNS records (BIND files). And they also give you the opportunity to set your own authoritative DNS servers for your domains. From now, I am making use of that option.</p> <a class="textlink" href="http://www.schlundtech.de">Schlund Technologies</a><br /> @@ -452,6 +599,7 @@ apply Service "dig6" { </ul> <p>That's much more comfortable now than manually clicking at some web UIs at Schlund Technologies.</p> <p>E-Mail me your thoughts at comments@mx.buetow.org!</p> +<a class="textlink" href="../">Go back to the main site</a><br /> </div> </content> </entry> @@ -482,11 +630,13 @@ apply Service "dig6" { | || | | | \____||__|_____|__| </pre> +<p class="quote"><i>Written by Paul Buetow 2016-04-16</i></p> <a class="textlink" href="https://buetow.org/gemfeed/2016-04-03-offsite-backup-with-zfs.html">Read the first part before reading any furter here...</a><br /> <p>I enhanced the procedure a bit. From now on I am having two external 2TB USB hard drives. Both are setup exactly the same way. To decrease the probability that they will not fail at about the same time both drives are of different brands. One drive is kept at the secret location. The other one is kept at home right next to my HP MicroServer.</p> <p>Whenever I am updating offsite backup, I am doing it to the drive which is kept locally. Afterwards I bring it to the secret location and swap the drives and bring the other one back home. This ensures that I will always have an offiste backup available at a different location than my home - even while updating one copy of it.</p> <p>Furthermore, I added scrubbing (*zpool scrub...*) to the script. It ensures that the file system is consistent and that there are no bad blocks on the disk and the file system. To increase the reliability I also run a *zfs set copies=2 zroot*. That setting is also synchronized to the offsite ZFS pool. ZFS stores every data block to disk twice now. Yes, it consumes twice as much disk space but it makes it better fault tolerant against hardware errors (e.g. only individual disk sectors going bad). </p> <p>E-Mail me your thoughts at comments@mx.buetow.org!</p> +<a class="textlink" href="../">Go back to the main site</a><br /> </div> </content> </entry> @@ -515,6 +665,7 @@ apply Service "dig6" { | || | | | \____||__|_____|__| </pre> +<p class="quote"><i>Written by Paul Buetow 2016-04-03</i></p> <h2>Please don't lose all my pictures again!</h2> <p>When it comes to data storage and potential data loss I am a paranoid person. It is not just due to my job but also due to a personal experience I encountered over 10 years ago: A single drive failure and loss of all my data (pictures, music, ....).</p> <p>A little about my personal infrastructure: I am running my own (mostly FreeBSD based) root servers (across several countries: Two in Germany, one in Canada, one in Bulgaria) which store all my online data (E-Mail and my Git repositories). I am syncing incremental (and encrypted) ZFS snapshots between these servers forth and back so either data could be recovered from the other server.</p> @@ -527,6 +678,7 @@ apply Service "dig6" { <h2>Walking one round less</h2> <p>I am thinking of buying a second 2TB USB drive and to set it up the same way as the first one. So I could alternate the backups. One drive would be at the secret location, and the other drive would be at home. And these drives would swap location after each cycle. This would give some security about the failure of that drive and I would have to go to the secret location only once (swapping the drives) instead of twice (picking that drive up in order to update the data + bringing it back to the secret location).</p> <p>E-Mail me your thoughts at comments@mx.buetow.org!</p> +<a class="textlink" href="../">Go back to the main site</a><br /> </div> </content> </entry> @@ -549,6 +701,7 @@ apply Service "dig6" { \, /-( /'-,\, /-( /'-, \, /-( / //\ //\\ //\ //\\ //\ //\\jrei </pre> +<p class="quote"><i>Written by Paul Buetow 2011-05-07, last updated 2021-05-07</i></p> <p>PerlDaemon is a minimal daemon for Linux and other Unix like operating systems programmed in Perl. It is a minimal but pretty functional and fairly generic service framework. This means that it does not do anything useful other than providing a framework for starting, stopping, configuring and logging. In order to do something useful, a module (written in Perl) must be provided.</p> <h2>Features</h2> <p>PerlDaemon supports:</p> @@ -672,6 +825,7 @@ sub do ($) { <p>You can find PerlDaemon (including the examples) at:</p> <a class="textlink" href="https://github.com/snonux/perldaemon">https://github.com/snonux/perldaemon</a><br /> <p>E-Mail me your thoughts at comments@mx.buetow.org!</p> +<a class="textlink" href="../">Go back to the main site</a><br /> </div> </content> </entry> @@ -688,6 +842,15 @@ sub do ($) { <content type="xhtml"> <div xmlns="http://www.w3.org/1999/xhtml"> <h1>The Fype Programming Language</h1> +<pre> + ____ _ __ + / / _|_ _ _ __ ___ _ _ ___ __ _| |__ / _|_ _ + / / |_| | | | '_ \ / _ \ | | | |/ _ \/ _` | '_ \ | |_| | | | + _ / /| _| |_| | |_) | __/ | |_| | __/ (_| | | | |_| _| |_| | +(_)_/ |_| \__, | .__/ \___| \__, |\___|\__,_|_| |_(_)_| \__, | + |___/|_| |___/ |___/ +</pre> +<p class="quote"><i>Written by Paul Buetow 2010-05-09, last updated 2021-05-05</i></p> <p>Fype is an interpreted programming language created by me for learning and fun. The interpreter is written in C. It has been tested on FreeBSD and NetBSD and may also work on other Unix like operating systems such as Linux based ones. To be honest, besides learning and fun there is really no other use case of why Fype actually exists as many other programming languages are much faster and more powerful.</p> <p>The Fype syntax is very simple and is using a maximum look ahead of 1 and a very easy top down parsing mechanism. Fype is parsing and interpreting its code simultaneously. This means, that syntax errors are only detected during program runtime. </p> <p>Fype is a recursive acronym and means "Fype is For Your Program Execution" or "Fype is Free Yak Programmed for ELF". You could also say "It's not a hype - it's Fype!".</p> @@ -1078,6 +1241,7 @@ BB <p>You can find all of this on the GitHub page. There is also an "examples" folders containing some Fype scripts!</p> <a class="textlink" href="https://github.com/snonux/fype">https://github.com/snonux/fype</a><br /> <p>E-Mail me your thoughts at comments@mx.buetow.org!</p> +<a class="textlink" href="../">Go back to the main site</a><br /> </div> </content> </entry> @@ -1116,6 +1280,7 @@ _~~|~/_|_|__/|~~~~~~~ | / ~~~~~ | | ~~~~~~~~ ~ ~ ~~~ _|| (_/ (___)_| |Nov291999 (__) (____) </pre> +<p class="quote"><i>Written by Paul Buetow 2008-06-26, last updated 2021-05-04</i></p> <p>Here are some Perl Poems I wrote. They don't do anything useful when you run them, but they don't produce a compiler error either. They only exist for fun and demonstrate what you can do with Perl syntax.</p> <p>Wikipedia: "Perl poetry is the practice of writing poems that can be compiled as legal Perl code, for example the piece known as Black Perl. Perl poetry is made possible by the large number of English words that are used in the Perl language. New poems are regularly submitted to the community at PerlMonks."</p> <a class="textlink" href="https://en.wikipedia.org/wiki/Perl">https://en.wikipedia.org/wiki/Perl</a><br /> @@ -1241,6 +1406,7 @@ This is perl, v5.8.8 built for i386-freebsd-64int <p>Did you like what you saw? Have a look at Github to see my other poems too:</p> <a class="textlink" href="https://github.com/snonux/perl-poetry">https://github.com/snonux/perl-poetry</a><br /> <p>E-Mail me your thoughts at comments@mx.buetow.org!</p> +<a class="textlink" href="../">Go back to the main site</a><br /> </div> </content> </entry> diff --git a/content/html/gemfeed/index.html b/content/html/gemfeed/index.html index 38a1cd41..6a06d90e 100644 --- a/content/html/gemfeed/index.html +++ b/content/html/gemfeed/index.html @@ -15,7 +15,7 @@ body { } img { display:block; - max-width: 90%; + max-width: 80%; } p.quote:before { content: " | "; @@ -52,6 +52,7 @@ h2, h3 { <h2>Having fun with computers!</h2> <a class="textlink" href="./2021-04-24-welcome-to-the-geminispace.html">2021-04-24 - Welcome to the Geminispace</a><br /> <a class="textlink" href="./2021-04-22-dtail-the-distributed-log-tail-program.html">2021-04-22 - DTail - The distributed log tail program</a><br /> +<a class="textlink" href="./2018-06-01-realistic-load-testing-with-ioriot-for-linux.html">2018-06-01 - Realistic load testing with I/O Riot for Linux</a><br /> <a class="textlink" href="./2016-11-20-methods-in-c.html">2016-11-20 - Methods in C</a><br /> <a class="textlink" href="./2016-05-22-spinning-up-my-own-authoritative-dns-servers.html">2016-05-22 - Spinning up my own authoritative DNS servers</a><br /> <a class="textlink" href="./2016-04-16-offsite-backup-with-zfs-part2.html">2016-04-16 - Offsite backup with ZFS (Part 2)</a><br /> diff --git a/content/html/index.html b/content/html/index.html index 0022ebc6..6f5b5dfd 100644 --- a/content/html/index.html +++ b/content/html/index.html @@ -15,7 +15,7 @@ body { } img { display:block; - max-width: 90%; + max-width: 80%; } p.quote:before { content: " | "; @@ -88,6 +88,7 @@ h2, h3 { <p>I have switched blog software multiple times. I might be back filling some of the older articles here. So please don't wonder when suddenly very old posts appear here.</p> <a class="textlink" href="./gemfeed/2021-04-24-welcome-to-the-geminispace.html">2021-04-24 - Welcome to the Geminispace</a><br /> <a class="textlink" href="./gemfeed/2021-04-22-dtail-the-distributed-log-tail-program.html">2021-04-22 - DTail - The distributed log tail program</a><br /> +<a class="textlink" href="./gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux.html">2018-06-01 - Realistic load testing with I/O Riot for Linux</a><br /> <a class="textlink" href="./gemfeed/2016-11-20-methods-in-c.html">2016-11-20 - Methods in C</a><br /> <a class="textlink" href="./gemfeed/2016-05-22-spinning-up-my-own-authoritative-dns-servers.html">2016-05-22 - Spinning up my own authoritative DNS servers</a><br /> <a class="textlink" href="./gemfeed/2016-04-16-offsite-backup-with-zfs-part2.html">2016-04-16 - Offsite backup with ZFS (Part 2)</a><br /> diff --git a/content/html/resources.html b/content/html/resources.html index 392d28dd..b021c733 100644 --- a/content/html/resources.html +++ b/content/html/resources.html @@ -15,7 +15,7 @@ body { } img { display:block; - max-width: 90%; + max-width: 80%; } p.quote:before { content: " | "; diff --git a/content/md/gemfeed/2008-06-26-perl-poetry.md b/content/md/gemfeed/2008-06-26-perl-poetry.md index 3bd22ece..83376f01 100644 --- a/content/md/gemfeed/2008-06-26-perl-poetry.md +++ b/content/md/gemfeed/2008-06-26-perl-poetry.md @@ -1,7 +1,3 @@ -> Written by Paul Buetow 2008-06-26, last updated 2021-05-04 - -[Go back to the main site](../) - # Perl Poetry ``` @@ -27,6 +23,8 @@ _~~|~/_|_|__/|~~~~~~~ | / ~~~~~ | | ~~~~~~~~ (__) (____) ``` +> Written by Paul Buetow 2008-06-26, last updated 2021-05-04 + Here are some Perl Poems I wrote. They don't do anything useful when you run them, but they don't produce a compiler error either. They only exist for fun and demonstrate what you can do with Perl syntax. Wikipedia: "Perl poetry is the practice of writing poems that can be compiled as legal Perl code, for example the piece known as Black Perl. Perl poetry is made possible by the large number of English words that are used in the Perl language. New poems are regularly submitted to the community at PerlMonks." @@ -164,3 +162,5 @@ Did you like what you saw? Have a look at Github to see my other poems too: [https://github.com/snonux/perl-poetry](https://github.com/snonux/perl-poetry) E-Mail me your thoughts at comments@mx.buetow.org! + +[Go back to the main site](../) diff --git a/content/md/gemfeed/2010-05-09-the-fype-programming-language.md b/content/md/gemfeed/2010-05-09-the-fype-programming-language.md index e7ff7c5c..05844b4d 100644 --- a/content/md/gemfeed/2010-05-09-the-fype-programming-language.md +++ b/content/md/gemfeed/2010-05-09-the-fype-programming-language.md @@ -1,6 +1,4 @@ -> Written by Paul Buetow 2010-05-09, last updated 2021-05-05 - -[Go back to the main site](../) +# The Fype Programming Language ``` ____ _ __ @@ -9,10 +7,9 @@ _ / /| _| |_| | |_) | __/ | |_| | __/ (_| | | | |_| _| |_| | (_)_/ |_| \__, | .__/ \___| \__, |\___|\__,_|_| |_(_)_| \__, | |___/|_| |___/ |___/ - ``` -# The Fype Programming Language +> Written by Paul Buetow 2010-05-09, last updated 2021-05-05 Fype is an interpreted programming language created by me for learning and fun. The interpreter is written in C. It has been tested on FreeBSD and NetBSD and may also work on other Unix like operating systems such as Linux based ones. To be honest, besides learning and fun there is really no other use case of why Fype actually exists as many other programming languages are much faster and more powerful. @@ -509,3 +506,5 @@ You can find all of this on the GitHub page. There is also an "examples" folders [https://github.com/snonux/fype](https://github.com/snonux/fype) E-Mail me your thoughts at comments@mx.buetow.org! + +[Go back to the main site](../) diff --git a/content/md/gemfeed/2011-05-07-perl-daemon-service-framework.md b/content/md/gemfeed/2011-05-07-perl-daemon-service-framework.md index 285c8ec2..e23c2b8c 100644 --- a/content/md/gemfeed/2011-05-07-perl-daemon-service-framework.md +++ b/content/md/gemfeed/2011-05-07-perl-daemon-service-framework.md @@ -1,7 +1,3 @@ -> Written by Paul Buetow 2011-05-07, last updated 2021-05-07 - -[Go back to the main site](../) - # Perl Daemon (Service Framework) ``` @@ -11,6 +7,8 @@ //\ //\\ //\ //\\ //\ //\\jrei ``` +> Written by Paul Buetow 2011-05-07, last updated 2021-05-07 + PerlDaemon is a minimal daemon for Linux and other Unix like operating systems programmed in Perl. It is a minimal but pretty functional and fairly generic service framework. This means that it does not do anything useful other than providing a framework for starting, stopping, configuring and logging. In order to do something useful, a module (written in Perl) must be provided. ## Features @@ -161,3 +159,5 @@ You can find PerlDaemon (including the examples) at: [https://github.com/snonux/perldaemon](https://github.com/snonux/perldaemon) E-Mail me your thoughts at comments@mx.buetow.org! + +[Go back to the main site](../) diff --git a/content/md/gemfeed/2016-04-03-offsite-backup-with-zfs.md b/content/md/gemfeed/2016-04-03-offsite-backup-with-zfs.md index dc22a41b..8e0fdfcf 100644 --- a/content/md/gemfeed/2016-04-03-offsite-backup-with-zfs.md +++ b/content/md/gemfeed/2016-04-03-offsite-backup-with-zfs.md @@ -1,7 +1,3 @@ -> Written by Paul Buetow 2016-04-03 - -[Go back to the main site](../) - # Offsite backup with ZFS ``` @@ -17,6 +13,8 @@ \____||__|_____|__| ``` +> Written by Paul Buetow 2016-04-03 + ## Please don't lose all my pictures again! When it comes to data storage and potential data loss I am a paranoid person. It is not just due to my job but also due to a personal experience I encountered over 10 years ago: A single drive failure and loss of all my data (pictures, music, ....). @@ -40,3 +38,5 @@ The solution is adding another USB drive (2TB) with an encryption container (GEL I am thinking of buying a second 2TB USB drive and to set it up the same way as the first one. So I could alternate the backups. One drive would be at the secret location, and the other drive would be at home. And these drives would swap location after each cycle. This would give some security about the failure of that drive and I would have to go to the secret location only once (swapping the drives) instead of twice (picking that drive up in order to update the data + bringing it back to the secret location). E-Mail me your thoughts at comments@mx.buetow.org! + +[Go back to the main site](../) diff --git a/content/md/gemfeed/2016-04-16-offsite-backup-with-zfs-part2.md b/content/md/gemfeed/2016-04-16-offsite-backup-with-zfs-part2.md index bac8680c..72445954 100644 --- a/content/md/gemfeed/2016-04-16-offsite-backup-with-zfs-part2.md +++ b/content/md/gemfeed/2016-04-16-offsite-backup-with-zfs-part2.md @@ -1,7 +1,3 @@ -> Written by Paul Buetow 2016-04-16 - -[Go back to the main site](../) - # Offsite backup with ZFS (Part 2) ``` @@ -19,6 +15,8 @@ \____||__|_____|__| ``` +> Written by Paul Buetow 2016-04-16 + [Read the first part before reading any furter here...](./2016-04-03-offsite-backup-with-zfs.md) I enhanced the procedure a bit. From now on I am having two external 2TB USB hard drives. Both are setup exactly the same way. To decrease the probability that they will not fail at about the same time both drives are of different brands. One drive is kept at the secret location. The other one is kept at home right next to my HP MicroServer. @@ -28,3 +26,5 @@ Whenever I am updating offsite backup, I am doing it to the drive which is kept Furthermore, I added scrubbing (*zpool scrub...*) to the script. It ensures that the file system is consistent and that there are no bad blocks on the disk and the file system. To increase the reliability I also run a *zfs set copies=2 zroot*. That setting is also synchronized to the offsite ZFS pool. ZFS stores every data block to disk twice now. Yes, it consumes twice as much disk space but it makes it better fault tolerant against hardware errors (e.g. only individual disk sectors going bad). E-Mail me your thoughts at comments@mx.buetow.org! + +[Go back to the main site](../) diff --git a/content/md/gemfeed/2016-05-22-spinning-up-my-own-authoritative-dns-servers.md b/content/md/gemfeed/2016-05-22-spinning-up-my-own-authoritative-dns-servers.md index 04c85fe5..21237eae 100644 --- a/content/md/gemfeed/2016-05-22-spinning-up-my-own-authoritative-dns-servers.md +++ b/content/md/gemfeed/2016-05-22-spinning-up-my-own-authoritative-dns-servers.md @@ -1,9 +1,7 @@ -> Written by Paul Buetow 2016-05-22 - -[Go back to the main site](../) - # Spinning up my own authoritative DNS servers +> Written by Paul Buetow 2016-05-22 + ## Background Finally, I had time to deploy my own authoritative DNS servers (master and slave) for my domains "buetow.org" and "buetow.zone". My domain name provider is Schlund Technologies. They allow their customers to manually edit the DNS records (BIND files). And they also give you the opportunity to set your own authoritative DNS servers for your domains. From now, I am making use of that option. @@ -237,3 +235,5 @@ Whenever I have to change a DNS entry all have to do is: That's much more comfortable now than manually clicking at some web UIs at Schlund Technologies. E-Mail me your thoughts at comments@mx.buetow.org! + +[Go back to the main site](../) diff --git a/content/md/gemfeed/2016-11-20-methods-in-c.md b/content/md/gemfeed/2016-11-20-methods-in-c.md index 45ee2bc4..9906dbd5 100644 --- a/content/md/gemfeed/2016-11-20-methods-in-c.md +++ b/content/md/gemfeed/2016-11-20-methods-in-c.md @@ -1,9 +1,7 @@ -> Written by Paul Buetow 2016-11-20 - -[Go back to the main site](../) - # Methods in C +> Written by Paul Buetow 2016-11-20 + You can do some sort of object oriented programming in the C Programming Language. However, that is very limited. But also very easy and straight forward to use. ## Example @@ -84,3 +82,5 @@ How to overcome this? You need to take it further... If you want to take it further type "Object-Oriented Programming with ANSI-C" into your favorite internet search engine, you will find some crazy stuff. Some go as far as writing a C preprocessor in AWK, which takes some object oriented pseudo-C and transforms it to plain C so that the C compiler can compile it to machine code. This is actually similar to how the C++ language had its origins. E-Mail me your thoughts at comments@mx.buetow.org! + +[Go back to the main site](../) diff --git a/content/md/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux.draft.md b/content/md/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux.md index 83412e92..5bb5086f 100644 --- a/content/md/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux.draft.md +++ b/content/md/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux.md @@ -1,7 +1,3 @@ - Written by Paul Buetow 2018-06-01, last updated 2021-05-08 - -[Go back to the main site](../) - # Realistic load testing with I/O Riot for Linux ``` @@ -15,6 +11,8 @@ jgs\__/'---'\__/ ``` +> Written by Paul Buetow 2018-06-01, last updated 2021-05-08 + ## Foreword This text first was published in the german IT-Administrator computer Magazine. 3 years have passed since and I decided to publish it on my blog too. @@ -189,3 +187,5 @@ For example, the open syscall opens a file and returns the responsible file desc [Memory mapped I/O](https://en.wikipedia.org/wiki/Memory-mapped_I/O) E-Mail me your thoughts at comments@mx.buetow.org! + +[Go back to the main site](../) diff --git a/content/md/gemfeed/2021-04-22-dtail-the-distributed-log-tail-program.md b/content/md/gemfeed/2021-04-22-dtail-the-distributed-log-tail-program.md index 5722a047..5899c7de 100644 --- a/content/md/gemfeed/2021-04-22-dtail-the-distributed-log-tail-program.md +++ b/content/md/gemfeed/2021-04-22-dtail-the-distributed-log-tail-program.md @@ -1,9 +1,7 @@ -> Written by Paul Buetow 2021-04-22, last updated 2021-04-26 - -[Go back to the main site](../) - # DTail - The distributed log tail program +> Written by Paul Buetow 2021-04-22, last updated 2021-04-26 + [](./2021-04-22-dtail-the-distributed-log-tail-program/dtail-logo.png) This article first appeared at the Mimecast Engineering Blog but I made it available here in my personal Gemini capsule too. @@ -106,3 +104,5 @@ Mimecast highly encourages you to have a look at DTail and submit an issue for a [https://dtail.dev](https://dtail.dev) E-Mail me your thoughts at comments@mx.buetow.org! + +[Go back to the main site](../) diff --git a/content/md/gemfeed/2021-04-24-welcome-to-the-geminispace.md b/content/md/gemfeed/2021-04-24-welcome-to-the-geminispace.md index 609f83a3..fcc77ab5 100644 --- a/content/md/gemfeed/2021-04-24-welcome-to-the-geminispace.md +++ b/content/md/gemfeed/2021-04-24-welcome-to-the-geminispace.md @@ -1,10 +1,7 @@ -> Written by Paul Buetow 2021-04-24, last updated 2021-04-30 -> ASCII Art by Andy Hood - -[Go back to the main site](../) - # Welcome to the Geminispace +> Written by Paul Buetow 2021-04-24, last updated 2021-04-30, ASCII Art by Andy Hood + Have you reached this article already via Gemini? You need a special client for that, web browsers such as Firefox, Chrome, Safari etc. don't support the Gemini protocol. The Gemini address of this site (or the address of this capsule as people say in Geminispace) is: [gemini://buetow.org](gemini://buetow.org) @@ -76,3 +73,4 @@ Check out one of the following links for more information about Gemini. For exam E-Mail me your thoughts at comments@mx.buetow.org! +[Go back to the main site](../) diff --git a/content/md/gemfeed/2021-05-15-buetow.org.sh-One-Bash-script-to-rule-it-all.draft.md b/content/md/gemfeed/2021-05-15-buetow.org.sh-One-Bash-script-to-rule-it-all.draft.md index 08ef2cde..e463d52d 100644 --- a/content/md/gemfeed/2021-05-15-buetow.org.sh-One-Bash-script-to-rule-it-all.draft.md +++ b/content/md/gemfeed/2021-05-15-buetow.org.sh-One-Bash-script-to-rule-it-all.draft.md @@ -1,9 +1,7 @@ -ADD WRITTEN BY AND CREATED AT BLABLA - -ADD GO BACK LINK - # buetow.org.sh - One Bash script to rule it all +> TODO: ADD WRITTEN BY AND CREATED AT BLABLA + You might have read my previous blog post about entering the Geminispace. [Welcome to the Geminispace](./2021-04-24-welcome-to-the-geminispace) @@ -181,3 +179,5 @@ Specify -r [https://buetow.org - The original Gemini capsule ](https://buetow.org) [https://buetow.org/gemfeed/ - The Gemfeed](https://buetow.org/gemfeed/) [https://buetow.org/gemfeed/atom.xml - The Atom feed](https://buetow.org/gemfeed/atom.xml) + +TODO: ADD GO BACK LINK diff --git a/content/md/gemfeed/index.md b/content/md/gemfeed/index.md index 0f3f15aa..841eba09 100644 --- a/content/md/gemfeed/index.md +++ b/content/md/gemfeed/index.md @@ -4,6 +4,7 @@ [2021-04-24 - Welcome to the Geminispace](./2021-04-24-welcome-to-the-geminispace.md) [2021-04-22 - DTail - The distributed log tail program](./2021-04-22-dtail-the-distributed-log-tail-program.md) +[2018-06-01 - Realistic load testing with I/O Riot for Linux](./2018-06-01-realistic-load-testing-with-ioriot-for-linux.md) [2016-11-20 - Methods in C](./2016-11-20-methods-in-c.md) [2016-05-22 - Spinning up my own authoritative DNS servers](./2016-05-22-spinning-up-my-own-authoritative-dns-servers.md) [2016-04-16 - Offsite backup with ZFS (Part 2)](./2016-04-16-offsite-backup-with-zfs-part2.md) diff --git a/content/md/index.md b/content/md/index.md index 17d157bc..3bac82ba 100644 --- a/content/md/index.md +++ b/content/md/index.md @@ -54,6 +54,7 @@ I have switched blog software multiple times. I might be back filling some of th [2021-04-24 - Welcome to the Geminispace](./gemfeed/2021-04-24-welcome-to-the-geminispace.md) [2021-04-22 - DTail - The distributed log tail program](./gemfeed/2021-04-22-dtail-the-distributed-log-tail-program.md) +[2018-06-01 - Realistic load testing with I/O Riot for Linux](./gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux.md) [2016-11-20 - Methods in C](./gemfeed/2016-11-20-methods-in-c.md) [2016-05-22 - Spinning up my own authoritative DNS servers](./gemfeed/2016-05-22-spinning-up-my-own-authoritative-dns-servers.md) [2016-04-16 - Offsite backup with ZFS (Part 2)](./gemfeed/2016-04-16-offsite-backup-with-zfs-part2.md) diff --git a/content/meta/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux.meta b/content/meta/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux.meta new file mode 100644 index 00000000..e053c3bb --- /dev/null +++ b/content/meta/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux.meta @@ -0,0 +1,5 @@ +local meta_date="2018-06-01T09:30:27+01:00" +local meta_author="Paul Buetow" +local meta_email="comments@mx.buetow.org" +local meta_title="The article" +local meta_summary="This text first was published in the german IT-Administrator computer Magazine. 3 years have passed since and I decided to publish it on my blog too. . .....to read on please visit my site." diff --git a/header.html.part b/header.html.part index 375cf6fc..91f0d338 100644 --- a/header.html.part +++ b/header.html.part @@ -15,7 +15,7 @@ body { } img { display:block; - max-width: 90%; + max-width: 80%; } p.quote:before { content: " | "; |
