summaryrefslogtreecommitdiff
path: root/content/html/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux.html
diff options
context:
space:
mode:
authorPaul Buetow <git@mx.buetow.org>2021-05-17 21:02:55 +0100
committerPaul Buetow <git@mx.buetow.org>2021-05-21 05:11:05 +0100
commit1079f927a27db9d194c8e25eb3a188396fdf8eab (patch)
tree624f19114f664d23f651cce07e0a59c564c1798a /content/html/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux.html
parent66891ce50ed59d19f54ee3765b295dda6f806bf5 (diff)
refactor code
Diffstat (limited to 'content/html/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux.html')
-rw-r--r--content/html/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux.html4
1 files changed, 2 insertions, 2 deletions
diff --git a/content/html/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux.html b/content/html/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux.html
index 53ef5543..c201997f 100644
--- a/content/html/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux.html
+++ b/content/html/gemfeed/2018-06-01-realistic-load-testing-with-ioriot-for-linux.html
@@ -67,7 +67,7 @@ jgs\__/'---'\__/
<h2>Foreword</h2>
<p>This text first was published in the german IT-Administrator computer Magazine. 3 years have passed since and I decided to publish it on my blog too. </p>
<a class="textlink" href="https://www.admin-magazin.de/Das-Heft/2018/06/Realistische-Lasttests-mit-I-O-Riot">https://www.admin-magazin.de/Das-Heft/2018/06/Realistische-Lasttests-mit-I-O-Riot</a><br />
-<p>I havn't worked on I/O Riot for some time now, but all what is written here is still valid. I am still using I/O Riot to debug I/O issues and pattern once in a while, so by all means the tool is not obsolete yet. The tool even helped to resolve a major production incident at work involving I/O.</p>
+<p>I havn't worked on I/O Riot for some time now, but all what is written here is still valid. I am still using I/O Riot to debug I/O issues and pattern once in a while, so by all means the tool is not obsolete yet. The tool even helped to resolve a major production incident at work caused by disk I/O.</p>
<p>I am eagerly looking forward to revamp I/O Riot so that it uses the new BPF Linux capabilities instead of plain old Systemtap (or alternatively: Newer versions of Systemtap can also use BPF as the backend I have learned). Also, when I wrote I/O Riot initially, I didn't have any experience with the Go programming language yet and therefore I wrote it in C. Once it gets revamped I might consider using Go instead of C as it would spare me from many segmentation faults and headaches during development ;-). I might also just stick to C for plain performance reasons and just refactor the code dealing with concurrency.</p>
<p>Pleace notice that some of the screenshots show the command "ioreplay" instead of "ioriot". That's because the name has changed after taking those.</p>
<h1>The article</h1>
@@ -79,7 +79,7 @@ jgs\__/'---'\__/
<p>Testing in the production environment: For these reasons, benchmarks are often carried out in the production environment. In order to derive value from this such tests are especially performed during peak hours when systems are under high load. However, testing on production systems is associated with risks and can lead to failure or loss of data without adequate protection.</p>
<h2>Benchmarking the Email Cloud at Mimecast</h2>
<p>For email archiving, Mimecast uses an internally developed microservice, which is operated directly on Linux-based storage systems. A storage cluster is divided into several replication volumes. Data is always replicated three times across two secure data centers. Customer data is automatically allocated to one or more volumes, depending on throughput, so that all volumes are automatically assigned the same load. Customer data is archived on conventional, but inexpensive hard disks with several terabytes of storage capacity each. I/O benchmarking proved difficult for all the reasons mentioned above. Furthermore, there are no ready-made tools for this purpose in the case of self-developed software. The service operates on many block devices simultaneously, which can make the RAID controller a bottleneck. None of the freely available benchmarking tools can test several block devices at the same time without extra effort. In addition, emails typically consist of many small files. Randomized access to many small files is particularly inefficient. In addition to many software adaptations, the hardware and operating system must also be optimized.</p>
-<p>Mimecast encourages employees to be innovative and pursue their own ideas in the form of an internal competition, Pet Project. The goal of the pet project I/O Riot was to simplify OS and hardware level I/O benchmarking. The first prototype of I/O Riot was awarded an internal roadmap prize in the spring of 2017. A few months later, I/O Riot was used to reduce write latency in the storage clusters by about 50%. The improvement was first verified by I/O replay on a test system and then successively applied to all storage systems. I/O Riot was also used to resolve a production incident related to disk I/O load.</p>
+<p>Mimecast encourages employees to be innovative and pursue their own ideas in the form of an internal competition, Pet Project. The goal of the pet project I/O Riot was to simplify OS and hardware level I/O benchmarking. The first prototype of I/O Riot was awarded an internal roadmap prize in the spring of 2017. A few months later, I/O Riot was used to reduce write latency in the storage clusters by about 50%. The improvement was first verified by I/O replay on a test system and then successively applied to all storage systems. I/O Riot was also used to resolve a production incident caused by disk I/O load.</p>
<h2>Using I/O Riot</h2>
<p>First, all I/O events are logged to a file on a production system with I/O Riot. It is then copied to a test system where all events are replayed in the same way. The crucial point here is that you can reproduce I/O patterns as they are found on a production system as often as you like on a test system. This results in the possibility of optimizing the set screws on the system after each run.</p>
<h3>Installation</h3>