Noteflakes

Threads vs Fibers - Can't We Be Friends?

Fri, 19 Dec 2025 00:00:00 GMT

In the last few weeks I’ve been writing here about my work on UringMachine, a Ruby gem for doing I/O with io_uring. Before I talk about my work this week, I’d like to address something that is important to me.

Maybe, We’ll See…

I usually share my blog posts in a few different places on the internet, and sometime people write their reactions. Last week, on Reddit, someone didn’t like the title (of all things… 🙂) I gave my last blog post, and made the following remark:

When a project or people aren’t open on what their tech is good for, and what it’s not good for, it really drives me away…

To which I replied:

I try to be as open as possible, and the thing is I don’t yet know what this is good for, but I’m excited about the possibilities.

To which that person retorted:

I find it surprising one would get involved with such project without having an idea of the result to expect. The pros and cons of fibers and async are pretty well known.

Well, starting this project I did have some vague idea of what will happen if we combined Ruby fibers with io_uring. At the same time, I wanted to be able to show some concrete results (as in, benchmarks), and in that sense I don’t think anyone could claim they could know the expected result in advance.

But also, in a more general sense, I believe it would have been presumtuous, even arrogant for me to claim I already knew everything there was to know about io_uring, Ruby fibers and how the two would work together. Yes, there’s a general agreement that fibers are good for I/O-bound work and less useful for CPU-bound work, but that is just a general concept.

Beyond that, there are a lot of questions to answer: Where are the limits? Can an I/O-bound workload become CPU-bound, or vice versa? What about mixed workloads? What are the shortcomings of the Fiber Scheduler interface? Are there other things we can improve about Ruby’s I/O layer? And finally, would this work lead to new solutions for apps written in Ruby? I don’t know the answer to these questions, but that’s what I want to find out!

In that regard, I really appreciate the remarks that Matz made at the last Euruko conference in Viana do Castelo. He talked about the spirit of openess and experimentation and inquisitiveness: is this project worth anything, or is it just a big waste of time? Who knows? I guess we’ll see…

So, Back to UringMachine

This week I continued work on benchmarking UringMachine against other concurrency solutions. I also spent more time testing the functioning of the UringMachine fiber scheduler. I noticed that when using an IO with buffered writes (which is the default in Ruby,) when the IO’s write buffer is flushed, the fiber scheduler’s #io_write hook would not get invoked. Instead, Ruby would invoke the #blocking_operation_wait hook, which run a blocking write system call on a worker thread. This, of course, is far from being satisfactory, as io_uring lets us run file I/O asynchronously. So I submitted a fix which has been already been merged into the Ruby 4.0 release.

Measuring how fast UringMachine goes led to some work on the UringMachine low-level API implementation. For example, I added methods for doing vectorized write and send (io_uring provides support for vectorized send as of kernel version 6.17). So, UM#writev and UM#sendv let you write or send multiple buffers at once. I still need to measure how those methods stack up against the other ways of writing tro a socket, but from some experiments I did this does seem promising.

Between UringMachine and the Ruby GVL

While I was working on the UringMachine benchmarks, it occurred to me that the UM implementation of I/O operations has one substantial shortcoming: while UringMachine does a great job of switching between fibers that are runnable, every once in a while it still has to enter the kernel in order to process submission entries, and completion entries.

Let’s back up a bit: the way UringMachine works is that when a fiber calls one of the UringMachine I/O methods, UringMachine prepares a submission queue entry (SQE), and switches execution to the next fiber in the runqueue. When the runqueue is exhausted, UringMachine calls, depending on the context, io_uring_submit or io_uring_wait_for_cqe in order to tell the kernel to process pending entries in the submission queues, and also to wait for at least one completion entry (CQE) to become available.

Those functions to interact with io_uring (from liburing, which provides a nicer interface to the io_uring kernel API), are actually wrappers to the io_uring_enter system call, and therefore are potentially blocking. In order to play nice with Ruby, UringMachine will release the Ruby GVL while this system call is in progress, and will reacquire the Ruby GVL when the system call returns.

What this means is that with UringMachine, when there’s no more CPU-bound work to do, and all fibers are waiting on some I/O operation to complete, UringMachine will “stop the world” and wait for one or more I/O operations to complete. It follows then that in a single-thread setup, the GVL will be periodically unused for a certain amount of time, depending on the I/O work being done. Naturally, the exact amount of time the GVL is available will be reduced as the I/O load increases, but it will still be there, and therefore the GVL as a resource will never be 100% saturated.

So, if we ran multiple instances of UringMachine on separate threads, we might be able to better saturate the GVL and thus achieve higher throughput for our I/O workload. This, at least was my theory.

It should be noted that up until now, when discussing the different solutions for concurrency in Ruby, we’ve been talking about either threads or fibers. The basic approach says: you can either run your workload on multiple threads, where you’ll enjoy some parallelism when any thread does I/O work; or you can run your workload on a single thread using multiple fibers, which lets you run a very large number of concurrent I/O operations and amortize some of the cost of talking to the kernel over multiple operations (preferably using io_uring).

But in fact those two approaches have important shortcomings: with threads you’re limited in terms of how many I/O operations you can do concurrently, and you pay a heavy price on GVL contention (that is, you’ll eventually see worse performance as you increase the number of threads); with fibers, your CPU-bound code is penalized because you need to periodically “stop the world” and talk to the kernel in order to process your I/O.

But what would happen if we actually used multiple threads, and on each thread we ran multiple fibers? What if it wasn’t an either / or situation, but rather threads and fibers?

What’s Faster than Fibers? Threads + Fibers!

So eager to test my theory, I rolled my sleeves and added some code to the UringMachine benchmarks. I wanted to test how UringMachine performs when the workload is split across 2 or more threads. Here’s the code that drives the implementation:

def run_um_x2
  threads  = 2.times.map do
    Thread.new do
      machine = UM.new(4096)
      fibers = []
      fds = []
      do_um_x(2, machine, fibers, fds)
      machine.await_fibers(fibers)
      fds.each { machine.close(it) }
    end
  end
  threads.each(&:join)
end

And here’s the do_um_x code for the bm_io_pipe benchmark:

def do_um_x(div, machine, fibers, fds)
  (GROUPS/div).times do
    r, w = UM.pipe
    fibers << machine.spin do
      ITERATIONS.times { machine.write(w, DATA) }
      machine.close_async(w)
    end
    fibers << machine.spin do
      ITERATIONS.times { machine.read(r, +'', SIZE) }
      machine.close_async(r)
    end
  end
end

Basically, it does the same thing, but since it’s invoked for each thread, we divide the number of “groups” by the number of threads, so if we have say 2 threads and 48 groups in total, on each thread we’ll start 24 of them. Here are the results:

                    user     system      total        real
Threads         4.638775   5.723567  10.362342 (  9.306601)
Async uring     2.197082   1.101110   3.298192 (  3.298313)
Async uring x2  2.471700   1.186602   3.658302 (  3.654717)
UM FS           1.167294   0.668348   1.835642 (  1.835746)
UM FS x2        1.169006   0.726825   1.895831 (  1.208773)
UM              0.430376   0.666198   1.096574 (  1.096809)
UM x2           0.463280   0.708560   1.171840 (  0.622890)
UM x4           0.589586   0.995353   1.584939 (  0.795669)
UM x8           0.889519   1.210246   2.099765 (  1.251179)

If we just look at the pure UM implementations, we see the 2 threads version providing a significant speedup over the single thread version (0.62s vs. 1.10s, or ~1.77 times faster). But we also see that as we increase the thread count we got dimishing returns, as the 4 thread version is slower than the 2 thread version, and the 8 thread version is even slower than the single thread version.

Looking at the fiber scheduler implementations, we see the Async fiber scheduler actually performing worse with 2 threads, and the UM fiber scheduler performing somewhat better (1.21s vs. 1.84s, about 1.52 times faster).

Running the bm_io_socketpair benchmark will yield a somewhat different result:

                    user     system      total        real
Threads         3.107412   4.375018   7.482430 (  5.518199)
Async uring     1.168644   2.116141   3.284785 (  3.285159)
Async uring x2  2.115308   4.267043   6.382351 (  4.233598)
UM FS           1.002312   1.910953   2.913265 (  2.913601)
UM FS x2        2.077869   4.041456   6.119325 (  3.984509)
UM              0.319183   1.618411   1.937594 (  1.937892)
UM x2           0.367900   1.700580   2.068480 (  1.114170)
UM x4           0.431765   1.881608   2.313373 (  0.768043)
UM x8           0.497737   2.286719   2.784456 (  0.831759)

Here we see the UringMachine fiber scheduler also performing worse in a multithreaded setup. This might be due to the fact that the fiber scheduler interface does not have hooks for socket operations such as send and recv, and it only participates in the actual I/O operations via its io_wait hook, which is invoked to check for readiness before Ruby performs the actual socket I/O.

Looking at the pure UringMachine implementation, we also see a performance increase as we increase the number of threads, but eventually we’ll see a performance degradation as we go past 4 threads.

So, this technique of starting up multiple threads, each with its UringMachine instance and multiple concurrent fibers, definitely has value, but the actual performance increase we’ll get, and the ideal number of threads we start, is highly dependant on the actual workload.

I wanted to go a bit beyond those synthetic benchmarks and see how UM does under a more realistic scenario - that of a web server. So I sketched a very simple web server that listens for incoming connections, parses HTTP requests, and sends response headers followed by a response body. I then measured how it did using rewrk with the following command:

$ rewrk -c 256 -t 4 -d 10s -h http://127.0.0.1:1234

Thread count	reqs/s	latency-avg	latency-stdev	latency-min	latency-max
1	103917	2.46ms	0.70ms	1.30ms	39.00ms
2	184753	1.38ms	0.47ms	0.05ms	39.60ms
4	216203	1.18ms	0.46ms	0.04ms	42.49ms
6	208420	1.23ms	0.48ms	0.04ms	35.80ms
8	205969	1.24ms	0.53ms	0.05ms	50.64ms

(The benchmark source code - it’s a just a rough sketch! - is here.)

So, putting aside the fact this sketch of a web server doesn’t do any CPU-bound work, like rendering templates, for example, we can see that we can more than double the throughput if we spread the workload over multiple threads. What I also find interesting is that the latency numbers actually improve as we go beyond a single thread. But, as we saw in the benchmarks above, we finally hit diminishing returns as we continue to increase the number of threads.

The Sidecar Thread Scenario

Another possibility that occured to me as I thought about combining threads and fibers, is a sidecar setup, where we have one thread that runs the fibers, and an auxiliary, or sidecar, thread that calls the kernel on behalf of the primary thread.

In this setup, we have a single UringMachine instance running on a primary thread, handling the workload in multiple fibers, and it’s basically CPU-bound. The UringMachine instance starts an auxiliary thread that runs in a loop, invoking the io_uring_enter system call repeatedly in order to submit new I/O operations, and processing completions as they arrive.

It remains to be seen how this will perform, and hopefully I’ll be able to report on this next week.

Better Buffer Management in UringMachine

My work on vectorized write/send this week also ties in with the future work I intend to do on buffer management. To make a long story short, io_uring provides the possibility to register buffer rings where the application can create multiple buffer rings and then dynamically add buffers to those buffer rings for them to be used for sending or receiving to sockets.

UringMachine already supports setting up buffer rings, but in a somewhat primitive form: to use a buffer ring you need to first call UM#setup_buffer_ring, which takes a buffer size and a buffer count and returns a buffer group id. You can then use this buffer group to receive messages from a socket:

# setup a buffer ring 1024 buffers with a size of 64KB each:
bgid = machine.setup_buffer_ring(1 << 16, 1024)

# receive using the buffer ring:
machine.recv_each(fd, bgid, flags) { |buffer| process_msg(buffer) }

What’s nice about this is that you can setup a single buffer ring and use it to receive on multiple sockets at once. The problem with the current implementation is that there’s no way to know if the buffer ring is close to being exhausted, and thus you risk getting a ENOBUFS error.

Also, while you can also send messages with buffer rings (using the UM#send_bundle method), it seems to me that this has limited utility since you cannot use a single buffer ring to send on multiple sockets at once. That’s why I did the work on vectorized write/send, which performs similarly to UM#send_bundle.

So my plan is to provide an API to make the buffer management completely transparent, so you’ll be able to setup a buffer pool (for reading/receiving only), which will automatically manage buffer groups. It will also automatically add and reuse buffers as needed. It will probably look something like this:

machine = UM.new
# setup a buffer pool that allocates 64KB buffers
buffer_pool = UM::BufferPool.new(machine, buffer_size: 1 << 16)
...
machine.recv_each(fd, buffer_pool) { process_message(it) }

The idea is to minimize buffer allocations by reusing buffers as much as possible. In more recent kernels, io_uring can optimize buffer use by being able to partially use buffers, so even if we allocate large buffers, they will be incrementally consumed by io_uring.

Conclusion

In just five days, we’ll all get a very nice Christmas gift - Ruby 4.0! If you want to get a sense of all the good stuff that’s arriving, just take a look at the activity on the ruby/ruby repo. There’s so many bug fixes and so many improvements on there, it’s really mind-boggling!

What I intend to do next week, between spending time with the family and seeing friends, is to continue to explore the combination of threads and fibers, and to start working on my idea for a fiber pool for automatically managing buffers for reading/receiving.

Beyond that, beginning in January I plan to start working on OpenSSL integration in UringMachine. I already have some ideas on how to do this, but we’ll discuss them when the time arrives.

Merry Christmas!

OSS Friday Update - Fibers are the Future of Ruby

Fri, 12 Dec 2025 00:00:00 GMT

In the last few days I’ve managed to finalize work on the UringMachine fiber scheduler. Beyond making sure the fiber scheduler is feature complete, that is, it implements all the different Fiber Scheduler hooks and their expected behaviour. To make sure of this, I also spent a couple of days writing test cases, not only of the fiber scheduler, but also of UM’s low-level API.

Beyond the tests, I wrote a series of benchmarks to have an idea of how UringMachine compares to other concurrency solutions:

You can consult the full results here. I’ll refrain from making overly generalized statements about what these benchmark results mean, but I think they demonstrate the promise of working with fibers to create concurrent Ruby apps.

So, as these benchmarks show, the Fiber Scheduler can bring significant benefits to concurrent Ruby apps, with minimal changes to the code (basically, instead of Thread.new you’ll use Fiber.schedule). The fact that the scheduler does the I/O transparently behind the scenes and integrates with the rest of the Ruby ecosystem feels almost like magic.

So I think this really validates the approach of Samuel Williams in designing how the fiber scheduler interfaces with the rest of the Ruby runtime. And the fact that the web server he authored, Falcon, is now used in production at Shopify, is an even stronger validation!

Here’s a detailed report of my work this last week:

Samuel has fixed the issue with the hanging #pwrite (it turns out the the #io_pwrite hook was being invoked with the GVL released.)
Added support for SQPOLL mode when setting up a UringMachine instance. It’s not clear to me what are the performance implications of that, but I’ll try to make some time to check this against TP2, a UringMachine-based web server I’m currently using in a bunch of projects.
started looking at getting #io_close to work, and found out that Samuel has already done the work, that is the code was already there, but was commented out. Samuel explained that it was impossible to get it to work due to the complexity of the implementation of IO#close, and indeed when I tried it myself I saw that in fact it was just not possible the way the IO state is managed when an IO is closed. I then had the idea that maybe we could pass the underlying fd instead of the IO object itself to the #io_close hook. The only issue is that this breaks the convention where the different io_xxx hooks take an io as their first argument. Nevertheless, I suggested this idea to Samuel and gladly he accepted when he saw this is the only we can make this hook work. Samuel then proceeded to prepare a PR and merge it.
Added the #io_close hook to the UringMachine fiber scheduler, as well as a #yield hook for dealing with thread interrupts in response to another PR by Samuel. I also added missing docs for the different methods in the fiber scheduler.
Spent a lot of time writing lots of tests for the fiber scheduler. I tried to cover the entire IO API - both class- and instance methods. I also wrote some “integration” tests - different scenarios not unlike those in the benchmarks, which exercise the different hooks in the fiber scheduler.
Added some new APIs to help with testing: UM#await_fibers is a method for waiting for one or more fibers to terminate. Unlike UM#join, it doesn’t return the return values of the given fibers, it just waits for them to terminate. Another new API is UM.socketpair, which is like Socket.socketpair except it returns raw fd’s.
Fixed some small issues in the UM fiber scheduler and in the UM low-level API implementation.
Added and streamlined metrics that indicate the following:
- The ring size
- Total number of ops
- Total number of fiber switches
- Total number of waits for CQEs
- Current number of pending ops
- Current number of unsubmitted ops
- Current size of runqueue
- Current number of transient ops
- Current number of free ops
I also added some basic time measurements:
- Total CPU time
- Total time spent waiting for CQEs
These are off by default, but can be enabled by calling UM#profile(true). I’d like to do a lot more with profiling, like measuring the CPU time spent on each fiber, but I’m a bit apprehensive of the performance costs involved, as getting the CLOCK_THREAD_CPUTIME_ID clock is relatively slow, and then managing this for each fiber means getting and setting a couple of instance variables, which can really slow things down. On top of that, I’m not that sure this is really needed.

What’s Next for UringMachine

One of the ideas I discussed with Samuel is to add support for registered buffers that integrates with the IO::Buffer class. While UringMachine already has support for buffer rings, it uses a custom implementation of buffers. So I might start by converting this to use IO::Buffer instead.
I’d also like to do a bit more work on performance tuning the UringMachine low-level API, specifically to be able to control the maximum number of fiber context switches before doing I/O work, i.e. submitting ops and checking for completions.
Beyond that, I also want to spend some time documenting the UringMachine API, as it is sorely lacking, and I’d like for other people to be able to play with it.

OSS Friday Update - The Shape of Ruby I/O to Come

Fri, 05 Dec 2025 00:00:00 GMT

I’m currently doing grant work for the Japanese Ruby Association on UringMachine, a new Ruby gem that provides a low-level API for working with io_uring. As part of my work I’ll be providing weekly updates on this website. Here’s what I did this week:

Last week I wrote about the work I did under the guidance of Samuel Williams to improve the behavior of fiber schedulers when forking. After some discussing the issues around forking with Samuel, we decided that the best course of action would be to remove the fiber scheduler after a fork. Samuel did work around cleaning up schedulers in threads that terminate on fork, and I submitted a PR for removing the scheduler from the active thread on fork, as well as resetting the fiber to blocking mode. This is my first contribution to Ruby core!
I Continued implementing the missing fiber scheduler hooks: #fiber_interrupt, #address_resolve, #timeout_after. For the most part, they were simple to implement. I probably spent most of my time figuring out how to test these, rather than implementing them. Most of the hooks involve just a few lines of code, with many of them consisting of a single line of code, calling into the relevant UringMachine low-level API.
Implemented the #io_select hook, which involved implementing a low-level UM#select method. This method took some effort to implement, since it needs to handle an arbitrary number of file descriptors to check for readiness. We need to create a separate SQE for each fd we want to poll. When one or more CQEs arrive for polled fd’s, we also need to cancel all poll operations that have not completed.

Since in many cases, IO.select is called with just a single IO, I also added a special-case implementation of UM#select that specifically handles a single fd.
Implemented a worker pool for performing blocking operations in the scheduler. Up until now, each scheduler started their own worker thread for performing blocking operations for use in the #blocking_operation_wait hook. The new implementation uses a worker thread pool shared by all schedulers, with a worker count limited to CPU count. Workers are started when needed.

I also added an optional entries argument to set the SQE and CQE buffer sizes when starting a new UringMachine instance. The default size is 4096 SQE entries (liburing by default makes the CQE buffer size double that of the SQE buffer). The blocking operations worker threads specify a value of 4 since they only use their UringMachine instance for popping jobs off the job queue and pushing the blocking operation result back to the scheduler.
Added support for file_offset argument in UM#read and UM#write in preparation for implementing the #io_pread and #io_pwrite hooks. The UM#write_async API, which permits writing to a file descriptor without waiting for the operation to complete, got support for specifying length and file_offset arguments as well. In addition, UM#write and UM#write_async got short-circuit logic for writes with a length of 0.
Added support for specifying buffer offset in #io_read and #io_write hooks, and support for timeout in #block, #io_read and #io_write hooks.
I found and fixed a problem with how futex_wake was done in the low-level UringMachine code handling mutexes and queues. This fixed a deadlock in the scheduler background worker pool where clients of the pool where not properly woken after the submitted operation was done.
I finished work on the #io_pread and #io_pwrite hooks. Unfortunately, the test for #io_pwrite consistently hangs (not in IO#pwrite itself, rather on closing the file.) With Samuel’s help, hopefully we’ll find a solution…
With those two last hooks, the fiber scheduler implementation is now feature complete!

Why is The Fiber Scheduler Important?

I think there is some misunderstanding around the Ruby fiber scheduler interface. This is the only Ruby API that does not have a built-in implementation in Ruby itself, but rather requires an external library or gem. The question has been raised lately on Reddit, why doesn’t Ruby include an “official” implementation of the fiber scheduler?

I guess Samuel is really the person to ask this, but personally I would say this is really about experimentation, and seeing how far we can take the idea of a pluggable I/O implementation. Also, the open-ended design of this interface means that we can use a low-level API such as UringMachine to implement it.

What’s Coming Next Week?

Now that the fiber scheduler is feature complete, I’m looking to make it as robust as possible. For this, I intend to add a lot of tests. Right now, the fiber scheduler has 25 tests with 77 assertions, in about 560LOC (the fiber scheduler itself is at around 220LOC). To me this is not enough, so next week I’m going to add tests for the following:

IO - tests for all IO instance methods.
working with queues: multiple concurrent readers / writers.
net/http test: ad-hoc HTTP/1.1 server + Net::HTTP client.
sockets: echo server + many clients.

In conjunction with all those tests, I’ll also start working on benchmarks for measuring the performance of the UringMachine low-level API against the UringMachine fiber scheduler and against the “normal” thread-based Ruby APIs.

In addition, I’m working on a pull request for adding an #io_close hook to the fiber scheduler interface in Ruby. Samuel already did some preparation for this, so I hope I can finish this in time for it to be merged in time for the release of Ruby 4.0.

I intend to release UringMachine 1.0 on Christmas, to mark the release of Ruby 4.0.

What About Papercraft?

This week I also managed to take the time to reflect on what I want to do next in Papercraft. I already wrote here about wanting to implement template inlining for Papercraft. I also wanted to rework how the compiled code is generated. I imagined a kind of DSL for code generation, but I didn’t really know what such a DSL would look like.

Then, a few days ago, the idea hit me. I’ve already played with this idea a last year, when I wrote Sirop, a sister gem to Papercraft that does a big part of the work of converting code into AST’s and vice versa. Here’s what I put in the readme:

Future directions: implement a macro expander with support for quote/unquote:

trace_macro = Sirop.macro do |ast| source = Sirop.to_source(ast) quote do result = unquote(ast) puts “The result of #{source} is: #{result}” result end end

def add(x, y) trace(x + y) end

Sirop.expand_macros(method(:add), trace: trace_macro)

The example is trivial and contrived, but I suddenly understand how such an interface could be used to actually generate code in Papercraft. I wrote up an issue for this, and hopefully I’ll have some time to work on this in January.

OSS Friday Update - The Fiber Scheduler is Taking Shape

Fri, 28 Nov 2025 00:00:00 GMT

This week I made substantial progress on the UringMachine fiber scheduler implementation, and also learned quite a bit about the inner workings of the Ruby I/O layer. Following is my weekly report:

I added some benchmarks measuring how the UringMachine mutex performs against the stock Ruby Mutex class. It turns out the UM#synchronize was much slower than core Ruby Mutex#synchronize. This was because the UM version was always performing a futex wake before returning, even if no fiber was waiting to lock the mutex. I rectified this by adding a num_waiters field to struct um_mutex, which indicates the number of fibers currently waiting to lock the mutex, and avoiding calling um_futex_wake if it’s 0.
I also noticed that the UM::Mutex and UM::Queue classes were marked as RUBY_TYPED_EMBEDDABLE, which means the underlying struct um_mutex and struct um_queue were subject to moving. Obviously, you cannot just move a futex var while the kernel is potentially waiting on it to change. I fixed this by removing the RUBY_TYPED_EMBEDDABLE flag.
Added support for IO::Buffer in all low-level I/O APIs, which also means the fiber scheduler doesn’t need to convert from IO::Buffer to strings in order to invoke the UringMachine API.
Added a custom UM::Error exception class raised on bad arguments or other API misuse. I’ve also added a UM::Stream::RESPError exception class to be instantiated on RESP errors. (commit 72a597d9f47d36b42977efa0f6ceb2e73a072bdf)
I explored the fiber scheduler behaviour after forking. A fork done from a thread where a scheduler was set will result in a main thread with the same scheduler instantance. For the scheduler to work correctly after a fork, its state must be reset. This is because sharing the same io_uring instance between parent and child processes is not possible (https://github.com/axboe/liburing/issues/612), and also because the child process keeps only the fiber from which the fork was made as its main fiber (the other fibers are lost).
On Samuel’s suggestions, I’ve submitted a PR for adding a Fiber::Scheduler#process_fork hook that is automatically invoked after a fork. This is in continuation to the #post_fork method. I still have a lot to learn about working with the Ruby core code, but I’m really excited about the possibility of this PR (and the previous one as well) getting merged in time for the Ruby 4.0 release.
Added two new low-level APIs for waiting on processes, instead of UM#waitpid, using the io_uring version of waitid. The vanilla version UM#waitid returns an array containing the terminated process pid, exit status and code. The UM#waitid_status method returns a Process::Status with the pid and exit status. This method is present only if the rb_process_status_new function is available (see above).
Implemented FiberScheduler#process_wait hook using #waitid_status.

For the sake of completeness, I also added UM.pidfd_open and UM.pidfd_send_signal for working with PID. A simple example:

child_pid = fork { ... }
fd = UM.pidfd_open(child_pid)
...
UM.pidfd_send_signal(fd, UM::SIGUSR1)
...
pid2, status = machine.waitid(P_PIDFD, fd, UM::WEXITED)

Wrote a whole bunch of tests for UM::FiberScheduler: socket I/O, file I/O, mutex, queue, waiting for threads. In the process I discovered a lots of things that can be improved in the way Ruby invokes the fiber scheduler.

Things I Learned This Week

As I dive deeper into integrating UringMachine with the Fiber::Scheduler interface, I’m discovering all the little details about how Ruby does I/O. As I wrote last week, Ruby treats files differently than other IO types, such as sockets and pipes:

For regular files, Ruby assumes file I/O can never be non-blocking (or async), and thus invokes the #blocking_operation_wait hook in order to perform the I/O in a separate thread. With io_uring, of course, file I/O is asynchronous.
For sockets there are no specialized hooks, like #socket_send etc. Instead, Ruby makes the socket fd’s non-blocking and invokes #io_wait to wait for the socket to be ready when performing a send or recv.

I find it interesting how io_uring breaks a lot of assumptions about how I/O should be done. Basically, with io_uring you can treat all fd’s as blocking (i.e. without the O_NONBLOCK control flag), and you can use io_uring to perform asynchrnous I/O on them, files included!

It remains to be seen if in the future the Ruby I/O implementation could be simplified to take full advantage of io_uring. Right now, the way things are done in the core Ruby IO classes leaves a lot of performance opportunities on the table. So, while the UringMachine fiber scheduler implementation will help in integrating UringMachine with the rest of the Ruby ecosystem, to really do high-performance I/O, one would still need to use UringMachine’s low-level API.

What’s Coming Next Week

Next week I hope to finish the fiber scheduler implementation by adding the last few things that are missing: handling of timeout, the #io_pread and io_pwrite hooks, and a few more minor features, as well as a lot more testing.

I also plan to start benchmarking UringMachine and compare the performance of its low-level API, the UringMachine fiber scheduler, and the regular thread-based concurrent I/O.

I also have some ideas for improvements to the UringMachine low-level implementation, which hopefully I’ll be able to report on next week.

If you appreciate my OSS work, please consider becoming a sponsor.

OSS Friday Update

Fri, 21 Nov 2025 00:00:00 GMT

Note: while my schedule is quite hectic these last few weeks, I’ve taken the decision to dedicate at least one day per week for developing open-source tools, and henceforth I plan to post an update on my progress in this regard every Friday evening. Here’s the first update:

UringMachine Grant Work

As I wrote here previously, a few weeks ago I learned I’ve been selected as one of the recipients of a grant from the Ruby Association in Japan, for working on UringMachine, a new gem that brings low-level io_uring I/O to Ruby. For this project, I’ve been paired with a terrific mentor - Samuel Williams - who is the authority on all things related to Ruby fibers. We’ve had a talk about the project and discussed the different things that I’ll be able to work on. I’m really glad to be doing this project under his guidance.

UringMachine implements a quite low-level API for working with I/O. You basically work with raw file descriptors, you can spin up fibers for doing multiple things concurrently, and there are low-level classes for mutexes and queues (based on the io_uring implementation of the futex API). Incidentally, I find it really cool that futexes can be used with io_uring to synchronize fibers, with very low overhead.

The problem with this, of course, is that this API is useless when you want to use the standard Ruby I/O classes, or any third-party library that relies on those standard classes.

This is where the Ruby fiber scheduler comes into the picture. Early on in my work on UringMachine, it occurred to me that the Fiber::Scheduler added to Ruby by Samuel is a perfect way to integrate such a low-level API with the Ruby I/O layer and the entire Ruby ecosystem. An implementation of Fiber::Scheduler for UringMachine would use the different scheduler hooks to punt work to the low-level UringMachine API.

So this week I finally got around to making some progress on the UringMachine fiber scheduler, and there’s finally a basic working version that can do basic I/O, as well as some other stuff like sleeping, waiting on blocking operations (such as locking a mutex or waiting on a queue), and otherwise managing the life cycle of a scheduler.

This is also a learning process. The Ruby IO class implementation is really complex: the io.c file itself is about 10K LOCs! I’m still figuring out the mechanics of the fiber scheduler as I go, and lots of things are still unclear, but I’m taking it one step at a time, and when I hit a snag I just try to take the problem apart and try to understand what’s going on. But now that I have moved from a rough sketch to something that works and has some tests, I intend to continue working on it by adding more and more tests and TDD’ing my way to an implementation that is both complete (feature-wise) and robust.

Here are some of the things I’ve learned while working on the fiber scheduler:

When you call Kernel.puts, the trailing newline character is actually written separately (i.e. with a separate write operation), which can lead to unexpected output if for example you have multiple fibers writing to STDOUT at the same time. To prevent this, Ruby seems to use a mutex (per IO instance) to synchronize writes to the same IO.
There are inconsistencies in how different kinds of IO objects are handled, with regards to blocking/non-blocking operation (O_NONBLOCK):
- Files and standard I/O are blocking.
- Pipes are non-blocking.
- Sockets are non-blocking.
- OpenSSL sockets are non-blocking.
The problem is that for io_uring to function properly, the fds passed to it should always be in blocking mode. To rectify this, I’ve added code to the fiber scheduler implementation that makes sure the IO instance is blocking:
```
def io_write(io, buffer, length, offset)
  reset_nonblock(io)
  @machine.write(io.fileno, buffer.get_string)
rescue Errno::EINTR
  retry
end

def reset_nonblock(io)
  return if @ios.key?(io)
        
  @ios[io] = true
  UM.io_set_nonblock(io, false)  
end
```
A phenomenon I’ve observed is that in some situations of multiple fibers doing I/O, some of those I/O operations would raise an EINTR, which should mean the I/O operation was interrupted because of a signal sent to the process. This is weird! I’m still not sure where this is coming from, certainly something I’ll ask Samuel about.
There’s some interesting stuff going on when calling IO#close. Apparently there’s a mutex involved, and I noticed two scheduler hooks are being called: #blocking_operation_wait which means a blocking operation that should be ran on a separate thread, and #block, which means a mutex is being locked. I still need to figure out what is going on there and why it is so complex. FWIW, UringMachine has a #close_async method which, as its name suggests, submits a close operation, but does not wait for it to complete.

Improving and extending the fiber scheduler interface

One of the things I’ve discussed with Samuel is the possibility of extending the fiber scheduler interface by adding more hooks, for example a hook for closing an IO (from what I saw there’s already some preparation for that in the Ruby runtime), or a hook for doing a splice. We’ve also discussed working with pidfd_open to prevent race conditions when waiting on child processes. I think there’s still a lot of cool stuff that can be done by bringing low-level I/O functionality to Ruby.

I’ve also suggested to Samuel to use the relatively recent io_uring_prep_waitid API to wait for child processes, and more specifically to do this in Samuel’s own io-event gem, which provides a low-level cross-platform API For building async programs in Ruby. With the io_uring version of waitid, there’s no need to use pidfd_open (in order to poll for readiness when the relevant process terminates). Instead, we use the io_uring interface to directly wait for the process to terminate. Upon termination, the operation completes and we get back the pid and status of the terminated process. This is also has the added advantage that you can wait for any child process, or any child process in the process group, which means better compatibility with the Process.wait and associated methods.

One problem is that the fiber scheduler process_wait hook is supposed to return an instance of Process::Status. This is a core Ruby class, but you cannot create instances of it. So, if we use io_uring to directly wait for a child process to terminate, we also need a way to instantiate a Process::Status object with the information we get back from io_uring. I’ve submitted a PR that hopefully will be merged before the release of Ruby 4.0. I’ve also submitted a PR to io-event with the relevant changes.

Going forward

So here’s where the UringMachine project is currently at:

The fiber scheduler implementation.
The fiber scheduler tests.
My grant development journal.

If you appreciate my OSS work, please consider sponsoring me.

My Consulting Work

Apart from my open-source work, I’m also doing consulting work for. Here’s some of the things I’m currently working on for my clients:

Transitioning a substantial PostgreSQL database (~4.5TB of data) from RDS to EC2. This is done strictly for the sake of reducing costs. My client should see a reduction of about 1000USD/month.
Provisioning of machines for the RealiteQ web platform to be used for industrial facilities in India.
Exploring the integration of AI tools for analyzing the performance of equipment such as water pumps for water treatment facilities. I’m still quite sceptical about LLM’s being the right approach for this. ML algorithms might be a better fit. Maybe, we’ll see…

You Win Some, You Lose Some: on Papercraft and more

Tue, 11 Nov 2025 00:00:00 GMT

In the last few weeks I’ve been busy with a few different projects, and I guess I’m not the only freelancer who has trouble finding their work-life balance. That is, in the last few weeks life has been hitting me in the face, and it was a bit overwhelming. But I still did manage to make some progress on my work, and thought I might share it here.

Papercraft Update

Since releasing Papercraft 3.0, I’ve been busy preparing a talk on Papercraft for Paris.rb. To me this was a major undertaking, since I’ve near-zero experience doing conference talks, and for me this was a test of my writing abilities, as well as my talking abilities. More on that in a moment.

I also managed to release a few more versions of Papercraft, which is now at version 3.2.0. Here are the major changes since version 3.0:

Fixed compilation of ternary operator expressions, so stuff like x ? h1('foo') : h2('bar') will compile correctly.
Added an optional Proc API, so you could do stuff like template.html(...) instead of calling Papercraft.html(template, ...). To use this API you need to first require "papercraft/proc". This essentially restores the pre-3.0 API, but as an opt-in.

Added Tilt integration, so you could use Papercraft with Tilt. Tilt is a generic interface for using different Ruby template engines. Here’s an example of how to use it:

require 'tilt/papercraft'

t = Tilt['papercraft'].new {
  "
    h1 locals[:a]
    p locals[:b]
    render block if block
  "
}

t.render(Object.new, a: 'foo', b: 'bar') {
  hr
}
#=> "<h1>foo</h1><p>bar</p><hr>"

The Paris.rb Talk

It actually took me a few weeks of writing and rewritng to finally arrive at a talk that I found both interesting and to the point. I wanted to talk about Papercraft and the functional style in Ruby, but without falling into the trap of discussing all of the theoretical stuff around functional programming. I mean, there’s already a lot of that on YouTube, and I also find it kind of tedious. I really prefer to stick to practical stuff, like what can we learn from functional programming that makes us into better programmers.

So, when the day came I took the train to Paris, had a couple hours to burn so I just sat in a café and rehearsed the text over and over. I thought I had it. But when the time finally came to give the talk later that evening, something happened, something that hasn’t happened to me in a while - I had a panic on stage. Well, I didn’t start screaming or anything, but as I started talking I suddenly felt like I had no air. I tried to calm myself and breathe but it was like my body was tightening into a coil, I felt like I was drowning. Somehow, I pulled through, I just went through the text and the slides as best I could, and as I got to the end, I had calmed down enough to be able to be present and responsive to the people in the audience. There were some questions from the audience, and I was calm enough to be able to answer, but inside I just felt crushed and beaten.

This, I mean stage fright, is something I’ve been struggling with over the years, but after the positive experience of my lightning talk at Euruko I thought I finally made some progress on this front. This experience was so discouraging that afterwards I felt completely empty and without energy. I still want to do more public speaking, and I think I have interesting stuff to share, but every such negative experience just adds to my predicament. Well, I guess I still have more work to do, you win some, you lose some…

By the way, the Euruko talks are now available on YouTube. You can find my Papercraft lightning talk here. The slides for the Paris.rb talk are here.

UringMachine Grant Work

I’ve written before about UringMachine, a Ruby gem low-level I/O using io_uring. I’m pleased to announce that I’m the recipient of a grant for working on UringMachine from the Ruby Association in Japan.

In this project, I’ll work on three things:

Developing a FiberScheduler implementation for UringMachine, in order to be able to allow its use in any fiber-based Ruby application.
Bringing SSL/TLS capabilities to UringMachine, in order to allow building high-performance clients and servers using encrypted connections.
Bringing more io_uring features to UringMachine, such as writev, splice, fsync, fadvise etc.

I’ll also take the time to work on documentation, benchmarks, and correctness of the implementation.

I’ll write here regularly about my progress. The grant also requires me publish a progress report around December, and then a final report on my work in March.

A New Client

I’ve just received a new commission for creating a blog website. The client is a person I love deeply - my daughter Noa. She has some very specific requirements regarding the design, functionality, and privacy concerns of the blog, so for me this is a challenge to see how far I can take my personal web framework, and how far I can push the new ideas and techniques I’ve been developing for my work.

As I develop and discover solutions for the different problems this new project presents, I’ll try to generalize them and fold them into Syntropy, so I’ll be able to use them for other projects as well.

Papercraft 3.0 Released

Mon, 20 Oct 2025 00:00:00 GMT

I have just released Papercraft version 3.0. This release includes a new API for rendering templates, improved XML support and an improved API for the Papercraft::Template wrapper class. Below is a discussion of the changes in this version, as well as what’s coming in the near future.

A New Rendering API

Papercraft 2.0 was all about embracing lambdas as the basic building block for HTML templates. Papercraft 2.0 introduced automatic compilation of Papercraft templates into an optimized form that provides best-in-class performance. The two most important operations on templates were #render and #apply:

# Papercraft 2.0:
Greet = ->(name) { h1 "Hello, #{name}!" }
Greet.render("world") #=> "<h1>Hello, world!</h1>"

# alternatively
GreetWorld = Greet.apply("world")
GreetWorld.render #=> "<h1>Hello, world!</h1>"

While this API is certainly very elegant and convenient, there was legitimate concern among potential users that Papercraft was in effect extending the core Proc class, with generic name methods that are specific to Papercraft templates, while Procs are in fact used everywhere in a given Ruby codebase, and not only for templates (everytime you call a method with a block, that block is in fact a Proc instance).

So, after giving it some thought I’ve decided to change the Papercraft API such that the act of rendering a template or applying arguments to a template will be done with singelton methods on the Papercraft module:

# Papercraft 3.0:
Greet = ->(name) { h1 "Hello, #{name}!" }
Papercraft.render(Greet, "world") #=> "<h1>Hello, world!</h1>"

# alternatively
GreetWorld = Papercraft.apply(Greet, "world")
Papercraft.render(GreetWorld) #=> "<h1>Hello, world!</h1>"

I think this change is a big step forward for Papercraft. It demarcates an important distinction between writing templates in the form of lambdas using the Papercraft DSL, and the actual rendering (or application) of said templates, which is done at the edges of the program, when those templates are actually used.

This change further embraces the functional style in Ruby, a style of programming I’ve been gravitating towards in the last few years, with an emphasis on explicitness and conciseness. I like that Papercraft.render and Papercraft.apply are simply functions that take templates (and optional arguments) as input and return a string as output.

Improved XML Support

While Papercraft 2.0 was concerned exclusively with HTML, I’ve decided to bring back support for rendering XML, even if only for the sake of being able to render RSS feeds. Version 3.0 introduces improved support for rendering XML. You can now render XML templates by calling Papercraft.xml:

template = ->(items) {
  articles {
    items.each {
      item it
    }
  }
}
Papercraft.xml(template, ['foo', 'bar'])

Papercraft 3.0 also adds support for rendering self-closing XML tags, for elements with no inner text or child nodes:

Papercraft.xml { item(ref: "foo") }
#=> "<item ref=\"foo\"/>"

A Streamlined Papercraft::Template Class

A few months ago, Papercraft version 2.4 introduced a wrapper class for templates called Papercraft::Template. The use case for this class was to be able to distinguish between template Procs and non-template Procs. With the new rendering API introduced in version 3.0, the Papercraft::Template class has also undergone some changes, with its interface streamlined and simplified:

Greet = Papercraft::Template.new { |name| h1 "Hello, #{name}!" }
Greet.render("world")
#=> "<h1>Hello, world!</h1>"

You can also use this class to render XML templates, by passing mode: :xml to Template.new:

Papercraft::Template.new(mode: :xml) { ... }

People that are have been using Papercraft since before version 2.0 API may prefer to use the Papercraft::Template class, which is in many ways similar to the original Papercraft API.

Coming Soon: Support for Inlining

When rendering complex HTML, and as your application grows, there’s a natural tendency to prefer to put separate parts of the markup in separate templates, which are then composed together. Papercraft makes this very easy to do, whether in the form of layouts, derived layouts, components or partials. But while the quality of your code is improved, this may come at a significant cost to rendering performance.

This problem is not unique to Papercraft. Any templating solution, be it ERB or Phlex is going to suffer from the same problem. ERB is especially susceptible to this.

One of the ideas I’ve been exploring since the introduction of automatic template compilation in Papercraft 2.0, was automatic inlining of sub-templates. Consider the following example:

Card = ->(title:, text:) {
  card {
    h1 title
    p text
  }
}
Deck = ->(items) {
  deck {
    items.each {
      Card(**item)
    }
  }
}

Currently, Papercraft will optimize Card and Deck separately, and the compiled Deck template while call the compiled Card template for each item:

# compiled code (edited for legibility)
->(__buffer__, items) {
  __buffer__.<<("<deck>")
  items.each {
    Card.__papercraft_compiled_proc.(__buffer__, **item)
  }
  __buffer__.<<("</deck>")
  __buffer__
}

If Papercraft were capable of inlining sub-templates, the compiled Deck template would have looked something like the following:

->(__buffer__, items) {
  __buffer__.<<("<deck>")
  items.each {
    __buffer__.<<("<card><h1>")
      .<<(ERB::Escape.html_escape((item[:title])))
      .<<("</h1><p>")
      .<<(ERB::Escape.html_escape((item[:text])))
      .<<("</p></card>");
  }
  __buffer__.<<("</deck>")
  __buffer__
}

It’s AST’s All the Way Down!

I’ve spent months thinking about this problem and had no clear idea of how this could be implemented. A quick recap: when Papercraft compiles a template it does it in three steps: first it loads the source code for the template and parses it using Prism, then the AST is mutated to convert tag method calls to custom nodes, and finally the mutated AST is converted back to optimized source code that is eval’d to produce the compiled template proc.

At first I presumed that inlining should be done at the last step, when converting the mutated AST to source code. But I had no clear idea on how to do this. Then, yesterday, I was making some notes for a talk I’m preparing about Papercraft and functional programming in Ruby, and I had a Eureka moment when I realized this could be solved by mutating and combining ASTs!

If we look at each template in terms of an AST, instead of its source code, the solution becomes clear: whenever we encounter a CallNode with the tag Card, we can simply replace this node with the AST of the corresponding template.

There’s some work to be done around translating arguments between the original call and the actual arguments used by the inlined AST, but this is certainly doable. In addition, this technique can be applied not only to rendering components, but also to the composition of layouts using render_yield, or any usage of Papercraft.apply.

I’m really excited to be implementing this feature, and making Papercraft the best and fastest HTML templating engine for Ruby. In the meanwhile, feel free to explore Papercraft and start using it in your app.

Papercraft update: IRB Support, Bug Fixes, More Speed

Sun, 12 Oct 2025 00:00:00 GMT

This week I was away on a little trip to Paris to attend a Paris.rb meetup and meet some friends, so I was less productive, but still got some stuff done, and still managed to do some work on Papercraft. Here’s what’s changed:

Using Papercraft in IRB

Up until now, perhaps the biggest limitation of Papercraft was that you couldn’t use it in an IRB session. That was because Papercraft always compiles your templates, and for that it needs access to the templates’ source code. But if you’re defining a template in IRB, where is that source code?

Then, while taking the train to Paris, it occurred to me that maybe IRB keeps the lines of code you input into it somewhere, and maybe it would be possible to access those lines of code. It took a bit of digging, but finally I’ve found it.

As a result, you can now define ad-hoc Papercraft templates right in your IRB session:

sharon@nf1:~$ irb -rpapercraft
irb(main):001> ->{ h1 "Hello, IRB!" }.render
=> "<h1>Hello, IRB!</h1>"
irb(main):002>

It’s fun to be able to explore Papercraft templates in IRB, enjoy!

Some Bug Fixes

I’ve also had the time to deal with some edge cases I discovered while using Papercraft:

Fix compilation of an empty template:
```
-> {}.render #=> ""
```

Raise error on a void element with child nodes or inner text:

-> { input 'foo' }.render #=> !!! Papercraft::Error
-> { hr { span } }.render #=> !!! Papercraft::Error

Fix apply parameter handling when called with a block:

a = proc { |*a, **b| body { render_yield(*a, **b) } }
b = a.apply(:foo, p: 42) { |*c, **d| article { render_yield(*c, **d) } }
b.render(:bar, q: 43) { |*a, **b|
  h1 a.first, class: b[:p]
  h2 a.last, class: b[:q]
}
#=> "<body><article><h1 class=\"42\">foo</h1><h2 class=\"43\">bar</h2></article></body>"

HTML Escaping Gets a Speed Boost

Last but not least, I was looking at the source code to ERB::Escape.html_escape, which is the method Papercraft uses to escape all HTML content (in order to prevent HTML injection). I figured the implementation could be improved, and opened a PR with the proposed change:

The existing html_escape implementation always allocates buffer space (6 times the length of the input string), even when the input string does not contain any character that needs to be escaped.

This PR modifies the implementation of optimized_escape_html to not pre-allocate an output buffer, but instead allocate it on the first occurence of a character that needs escaping. In addition, instead of copying non-escaped characters one by one to the output buffer, continuous non-escaped segments of characters are copied using memcpy.

A synthetic benchmark employing the input strings used in the test_html_escape method in test/test_erb.rb shows the modified implementation to be about 35% faster than the original…

The PR was merged and a new version of ERB is already released. And the best part - this speed up is available to the entire Ruby ecosystem, not just Papercraft. Everybody that uses ERB (or its html_escape method) will benefit from this!

Hanami on Papercraft

Sun, 05 Oct 2025 00:00:00 GMT

Lately I’ve been really excited about Papercraft and the possibilities it brings to developing web apps with Ruby. Frankly, the more I use it, the more I see how simple and joyful it can be to write beautiful HTML templates in plain Ruby.

Now that the Papercraft website is up, I’d like to concentrate on making it easier for everyone to use Papercraft in their apps, whatever their web framework. So this is exactly what I set out to do this weekend. First on my list: Hanami, an established Ruby web framework with a substantial following.

Since I never used Hanami, I decided to follow the Getting Started guide and then started to peek under the hood to see how I could replace the ERB templates with Papercraft ones.

After a few hours and a quite a bit of fiddling, I had a working proof of concept. I then proceeded to extract the code into a new gem I’m releasing today called hanami-papercraft.

To use it, do the following:

1. Add hanami-papercraft

In your Gemfile, add the following line:

gem "hanami-papercraft"

Then run bundle install to update your dependencies.

2. Set your app’s basic view class

In app/view.rb, change the View classes superclass to Hanami::PapercraftView:

# app/view.rb

module Bookshelf
  class View < Hanami::PapercraftView
  end
end

3. Use a Papercraft layout template

Replace the app’s layout template stored in app/templates/layouts/app.html.erb with a file named app/templates/layouts/app.papercraft.rb:

# app/templates/layouts/app.papercraft.rb

->(config:, context:, **props) {
  html(lang: "en") {
    head {
      meta charset: "UTF-8"
      meta name: "viewport", content: "width=device-width, initial-scale=1.0"
      title "Bookshelf"
      favicon_tag
      stylesheet_tag(context, "app")
    }
    body {
      render_children(config:, context:, **props)
      javascript_tag(context, "app")
    }
  }
}

4. Use Papercraft view templates

You can now start writing your view templates with Papercraft, e.g.:

# app/templates/books/index.papercraft.rb

->(context:, books:, **props) {
  h1 "Books"

  ul {
    books.each do |book|
      Kernel.p book
      li book[:title]
    end
  }
}

Passing Template Parameters

While theoretically you have access to the view class in your templates (through self), you should use explicit arguments in your templates, as shown in the examples above. The PapercraftView class always passes template parameters as keyword arguments to the layout and the view templates.

In the view template above, the books keyword argument is defined because the view class exposes such a parameter:

# app/views/books/index.rb

module Bookshelf
  module Views
    module Books
      class Index < Bookshelf::View
        expose :books do
          [
            {title: "Test Driven Development"},
            {title: "Practical Object-Oriented Design in Ruby"}
          ]
        end
      end
    end
  end
end

That’s it for now. There’s probably a lot of stuff that won’t work. If you run into any problems, please let me know. I’ll gladly accept contributions in the form of bug reports or PR’s. Just head on over to the hanami-papercraft repo…

Papercraft Update: New Version, New Website

Fri, 03 Oct 2025 00:00:00 GMT

I’ve been working quite a bit on Papercraft these last few weeks. Yesterday I released Papercraft version 2.16, and here are some of the notable changes introduced since the last update:

Emit DOCTYPE for html tag by default. Before this change, you needed to use the html5 tag to include the DOCTYPE at the top of the generated markup. Now you can just use html. This is important since this way you avoid quirks mode.
Do not content of style and script tags. This makes it easier to write inline CSS and Javascript.
Add Papercraft.markdown_doc convenience method which returns a Kramdown::Document instance for further processing of Markdown content.
Add support for rendering of namespaced components, so you can now do stuff like Foo::Bar('baz') right in your templates.

New Papercraft Website

I’ve also been working on a website for Papercraft and it’s finally online. Check it out:

papercraft.noteflakes.com

Like the noteflakes.com website, which you’re currently reading, the Papercraft website is made using Syntropy. All of the documentation pages are written using Markdown. Let’s look at some examples of how Papercraft is used on its own website:

The Default Layout

Here is the content of the default layout (source code):

export template { |page_title: nil, **props|
  html {
    head {
      title(page_title ? "Papercraft - #{page_title}" : 'Papercraft - Functional HTML Templating for Ruby')
      meta charset: 'utf-8'
      meta name: 'viewport', content: 'width=device-width, initial-scale=1.0'
      link rel: 'stylesheet', type: 'text/css', href: '/assets/style.css'
    }
    body {
      render_children(**props)
    }
    auto_refresh_watch!
  }
}

It’s all pretty standard except for that export at the top, which means that this file is loaded by Syntropy as a Syntropy module (more on that later). There’s also the auto_refresh_watch! directive, which is a Syntropy extension that permits refreshing the page automatically whenever the source code changes in development mode.

The Docs Layout

The documentation page layout (source code) is a bit more involved, bit basically it is derived from the default layout using apply:

DefaultLayout = import '_layout/default'
Pages = import '_pages'

export DefaultLayout.apply { |entry:, pages:, href:, **props|
  header { ... }
  main {
    ...
    sidebar { ... }
    article {
      content {
        h1 entry[:title]
        raw entry[:html]
        nav {
          if entry[:prev]
            a(href: entry[:prev][:href]) {
              p "Previous page"
              h2 entry[:prev][:title]
            }
          else
            span
          end
          if entry[:next]
            a(href: entry[:next][:href]) {
              p "Next page"
              h2 entry[:next][:title]
            }
          else
            span
          end
        }
      }
    }
  }
}

I’ve omitted the header and the side bar for the sake of brevity, so let’s look at the article element which contains the actual content of the page. There’s the title, there’s the pre-rendered HTML rendered from the Markdown content (but you can just as well use the markdown method to render it in-place), and then the nav element holds links to the previous and next pages. You can see how the logic flows naturally along the HTML content expressed with plain Ruby.

Here’s the code that renders the documentation pages (source code):

Layout = import '_layout/docs'
Pages = import '_pages'

export ->(req) {
  href = req.path
  entry = Pages[href]
  if entry
    html = Layout.render(page_title: entry[:title], pages: Pages, href:, entry:)
    req.respond(html, 'Content-Type' => Qeweney::MimeTypes[:html])
  else
    raise Syntropy::Error.not_found
  end
}

The page entry is retrieved from the Pages collection, and then passed to the layout template, along with some other metadata.

The Landing Page

The landing page (a.k.a. the index page) also uses the apply method to fill the default layout with content. Here’s a part of it (source code):

Pages = import '_pages'
Layout = import '_layout/default'

export Layout.apply {
  main {
    single {
      hero {
        logo {
          img src: "/assets/papercraft.png"
        }
        h1 {
          span "Papercraft"
        }
        h2 "Functional HTML Templating for Ruby"
        snippet {
          markdown <<~MD
            ```ruby
            -> {
              h1 "Hello from Papercraft!"
            }.render
            ```
          MD
        }
        links {
          a "Documentation", href: Pages.default_href
          a "Source Code", href: "https://github.com/digital-fabric/papercraft", target: "_blank"
        }
      }
      hr
      ...
    }
  }
}

As you can see, we can mix HTML and Markdown content freely. Another thing that may stick out is the fact that I (almost) don’t use any CSS classes. I prefer using semantic tag names, which not only makes the templates much more readable, but also makes the generated HTML much smaller in size, which helps in creating a snappy user experience.

Refactoring Opportunities

For the Papercraft site, since there are basically just two kinds of layouts, with little in common (except for the outer HTML envelope), I didn’t really feel there was a need to create components. But this possibility always exists. For example, let’s look at another snippet from the landing page:

...
features {
  a(href: "/docs/01-introduction/01-overview") {
    markdown <<~MD
      #### Easy to write & read
      
      Write your HTML templates in plain Ruby. Use beautiful
      syntax for generating HTML.
    MD
  }

  a(href: "/docs/03-template-composition/01-component-templates") {
    markdown <<~MD
      #### Layouts & Components
      
      Compose and reuse your templates for layouts,
      components and partials.
    MD
  }
  ...
}

There are a total of six “featurettes” like that on the landing page, so supposing we wanted to create a featurette component, it might look like this:

Featurette = ->(href:, title:, text:) {
  a(href:) {
    h4 title
    p text
  }
}

And then the landing page markup would look as follows:

...
features {
  Featurette(
    href: "/docs/01-introduction/01-overview",
    title: "Easy to write & read",
    text: "
      Write your HTML templates in plain Ruby. Use beautiful
      syntax for generating HTML.
    "
  )

  Featurette(
    href: "/docs/03-template-composition/01-component-templates",
    title: "Layouts & Components",
    text: "
      Compose and reuse your templates for layouts,
      components and partials.
    "
  )
}

I think from the point of view of effort vs gain it’s not so interesting to do this, but this is certainly a possibility, and just goes to show how easy it is to compose and reuse templates in Papercraft.

Integration with Other APIs

Another thing that occurred to me while working on the Papercraft website is that in fact a lot of the difficulties or issues surrounding the integration of a templating library with existing frameworks or tools just disappear with Papercraft. There’s no boilerplate code, no ceremony around setting up state or context objects. Your templates become pure functions that take some parameters as input, and give you back HTML code, ready to serve. And, did I mention it’s really fast?

Please feel free to test-drive Papercraft in your projects. Head on over to the Papercraft website, and enjoy!

Words Can Hurt: A Plea to the Ruby Community

Sat, 27 Sep 2025 00:00:00 GMT

I’ve been watching the recent drama within the Ruby community slowly devolve in the last few days into name-calling and virtue-signalling, and frankly just plain silliness. I won’t repeat here the details of the disagreement, and I won’t link to any posts written about what’s happened.

It is clear to me that some of this has to do with business interests of the different parties involved, some of this has to do with political views, and some of this apparently also has to with a clash of personalities. But what really troubles me is not the details of the disagreements themselves, however strongly each of us may feel about them, but rather how people have come to treat each other over these disagreements.

While some people have been very vocal about what’s happening, I believe many people have been keeping quiet not because they don’t care, or because they don’t have an opinion about the issue at hand, but because there’s a growing sense of violence - verbal violence, and people are afraid if they express themselves they’ll become targets for this violence. In fact, I’m pretty sure when this article is published I’ll become a target too. But I need to speak up.

Words Can Hurt

A lot of people have reacted very strongly to DHH’s recent post about London, which I’ll get to in a moment, but I’d like to address an earlier post of his, which is in my eyes more relevant to the present moment. In “Words are not violence”, David talks about the murder of Charlie Kirk, and how some people have justified this horrific act of violence as a response to Charlie’s words:

What I cannot come to terms with, though, is the modern equation of words with violence. The growing sense of permission that if the disagreement runs deep enough, then violence is a justified answer to settle it. That sounds so obvious that we shouldn’t need to state it in a civil society, but clearly it is not.

Needless to say I agree with David that an argument or disagreement can never serve as an excuse for physical violence. But I cannot agree that words cannot be violent, they can, and unfortunately violent speech is becoming more and more present in public discourse.

For the sake of discussion let’s define verbal violence as any speech that you would not wish on people dear to you, yourself included - name calling, hurtful remarks, etc.

Verbal violence can be just as hurtful as physical violence - calling someone a Nazi, a facist, a loony, stupid, retarded, and even worse is violent. Verbal abuse can also in some cases be illegal. You can think it’s quite harmless to call someone names, but when done repeatedly it can incite other people to physical violence (as in the case of the late Charlie Kirk, or MLK, or Gandhi), or even cause people to commit suicide.

We All Should Strive for Compassion

Look, I believe that human kind is naturally kind and good. We all need to be loved and respected, we all want dignity, we all want to have a peaceful, meaningful life. No one wants to be despised or called names, so why do we do this to each other? We should treat each other with respect and compassion.

I’ve read David’s London post and was hurt. I’m an immigrant, so I know how hard it is to be one, how hard it is to start a new life in a country where you sometimes don’t even know the language, don’t have any network of friends and family to rely on, where you don’t know the social ettiquete, where you don’t know how to deal with the government, where your legal situation may be precarious.

I know that immigration is a complex subject, I know that immigration sometimes has negative effects on the so-called “native” population (which itself may be composed of 2nd- or 3rd generation immigrants). I know that sometimes immigration engenders crime. But then again, I also know those people, some of whom have have been through hell, some of whom have had their lives broken, all need dignity. Nobody wants to spend entire days out on the street in idleness. They all want to have a dignified, productive life.

There’s of course a lot more to be said about immigration, this can’t be summarized in the space of two paragraphs. But what I want to say is that we should all give each other the benefit of the doubt, and assume that our interlocutor is naturally good. When I meet a stranger, I want to be able to treat them with compassion, not only for their sake, but also for my sake. And if you’re reading this and you’re still bitter and cynical about my words, here are some quotes for you from greater and smarter people than me:

We must learn to live together as brothers or perish together as fools. Every individual must learn this, and every nation must learn this. - Martin Luther King

Do to others as you would have them do to you. - Jesus

You shall not hate your brother in your heart. - Leviticus 19, 17

David, if you’re reading this, I don’t know you personally, but I’ve been a fan of your work for many years. I still am. Like many others, I started programming in Ruby because of you. I think you’re a true visionary, and I think in many ways you’ve revolutionized how web apps are built. I really admire your work. But your words have hurt me, and they have hurt others.

I don’t hate you, I’m not even mad at you, I’m just sad that a person in your position would not think their words through before saying them out loud. I know you are opinionated, and I believe you mean well, but please consider that even if you have strong convictions about political and societal matters, how you express them can have a lot of impact on the community, and on people who look up to you.

How should an immigrant who uses Ruby on Rails feel when you basically accuse them of causing crime and of ruining London? How can they read your message as anything but racist when you reduce them to “a statistic as evident as day when you walk the streets of London now”? (How can it be evident if it were not for their skin color, or their clothes?) Should people of color feel welcome at all at RailsWorld? Should they feel welcome at 37signals?

I know you were hurt by people calling you a Nazi, because you wrote about it. I would be hurt too. Nothing that you said can justify that, nothing that you said can justify any violent act towards you. But that is exactly the point, words can hurt, words can be violent!

So I ask you, please think about your words, and think about your message. As BDFL of Rails (and you do this marvelously well when it’s about technology), you have a responsibility to the community. You are responsible for making everybody feel welcome and included. I really hope my words find their way to your heart.

A Plea to the Ruby Community

And to the Ruby community in general, I’d like to say this: please stop the name-calling, please stop vilifying people. There’s a big difference between saying a person is expressing racist views (which can be debated rationally) and saying a person is a “Nazi” (which doesn’t allow any kind of debate). Name-calling will not stop racism, name-calling will not rectify the situation, and reducing another person to a label such as “Nazi”, “facist” etc is to me just as bad as racist speech. Being called names doesn’t give you the right to do the same. For the sake of us all, let’s return to a calm and respectful discourse, if not in society at large, then at least in the Ruby community.

I’m also really hopeful that the people behind the recent debacle around Ruby Central find the courage in their hearts to come together and work on a solution that will be acceptable to everybody involved, and that will bring relief to the community. It is incumbent upon all of us to do our part to arrive at a peaceful resolution.

It is not for you to finish the work, but neither are you at liberty to neglect it. - Rabbi Tarfon, Pirkey Avot

That’s it, I will not speak publicly any further on this subject. If you want to talk to me privately, please feel free to contact me. I leave you with the following words from another great visionary:

Carefully watch your thoughts, for they become your words. Manage and watch your words, for they will become your actions. Consider and judge your actions, for they have become your habits. Acknowledge and watch your habits, for they shall become your values. Understand and embrace your values, for they become your destiny. - Mahatma Gandhi

My Thoughts on Euruko

Tue, 23 Sep 2025 00:00:00 GMT

I’ve just got back home from Euruko last night. The conference ended on Friday, but I decided to stay two more nights in Portugal and visit Porto. In between walking all over the city, eating great food and enjoying the dancing and music making in the street, I’ve also had time to think about all the wonderful people I met at the conference (and even some I’ve met and talked to by chance on the streets of Viana do Castelo and Porto), and the incredible experiences I’ve had at Euruko.

First, I’d like to express my deep appreciation for the organizers, headed by Henrique. This was my first ever programming conference that I go to, so I had no idea how it was going to go. But it was obvious from the first moment, taking the shuttle kindly offered by the conference to take us from the airport to Viana, how much the conference organizers cared about making everybody feel welcome. It really felt like Henrique and the team went out of their way to tend all of our needs, from the great food and drinks available at all times, to sharing the wonderful cultural heritage of Viana with us all.

I was also really impressed with the level of production. The branding work was magnificent and was present everywhere down to the smallest details. It was also much bigger than I expected. I imagined there would be about 200 people, but I was told there were actually almost 600 of us. I’ve met a lot of people I knew online, and even more people I didn’t know at all, and I was really grateful for the more personal encounters I’ve had. Talking about code is great, but talking about life and sharing our common humanity is even better!

I’ve also noticed there were a lot more women than I imagined, and a lot more older people (i.e. people of my age), and both of these observations made me very happy. I’m a strong believer in diversity!

An Auspicious Start

The conference kicked off with a keynote by Matz, the legendary creator of Ruby, who 30-some years later still sets the tone for the Ruby community, putting the accent on developer happiness and inspiring the saying “Matz is nice so we are nice”. At the beginning of his talk Matz said he was going to talk about “my favorite things”. He was reminiscing about how he got started with computing and about the fact that it took him about 10 years from wanting to invent his own programming language to actually gaining the knowledge and the experience to be able to do it. Good things come to those who wait!

Matz has positioned the creation of Ruby at the intersection of three “favorite things”: human recognition, programming and language, and he continued by talking about the different types of joy that he got from Ruby: the joy of creation, the joy of design and the joy of power, and made a remark that for him joy was more important than performance, hence the importance of “developer happiness”.

He talked about how now he was less involved in the implementation details of Ruby, but with the recent work on ZJIT (Ruby’s next JIT engine), more people can now get involved in improving Ruby’s performance. Matz continued by telling the chinese parable of the old man who lost his horse, sharing with all of us an approach of openness and inquisitiveness, of refraining from pre-judging what happens to us but rather saying “maybe, we’ll see”. He finished his talk by saying that our greatest treasure is the community.

The Euruko Talks

Following Matz is a tough act for anybody, but the following talk was just fantastic: Marco Roth talked about his recent work on HERB and ReactionView which can really revolutionize the Ruby on Rails view layer, and brings some important innovations to developing web UI’s using Ruby. His work is a great inspiration to me personally, and I guess also to a lot of other people in the community.

What really blew my mind was that Marco has in fact built a parser for ERB templates that builds an AST of the template, which then permits not only compilation, but also all kinds of other operations such as linting, annotation, analysis and perhaps in the future the addition of reactivity!

I found it really interesting that at more or less the same time, there are three different people (Marco, myself and Joel Drapper, the author of Phlex) having the same idea of parsing templates into ASTs! But this is how things are sometimes, maybe this has less to do with our respective talents but rather with a certain Zeitgeist that arises out of the emergence of tools such as Prism.

I have been somewhat less attentive to the later talks, and have also missed some of them because I wanted to talk to people, not only about technical stuff, but also get to know people on a more personal level. One talk though that really stood out to me was given by Rémy Hannequin, who talked about the value of niche OSS projects, and his own project Astronoby which does astronomical calculations. I guess it really struck a chord with me because my projects also tend to be very niche. In fact, one of my goals in coming to Euruko was to share with the Ruby community my own projects and maybe that way they’ll become a bit less niche!

My Lightning Talk

So, arriving in Viana for the conference I already had an idea of what I wanted to talk about, and had prepared some slides, which I submitted to the conference organizers as a lightning talk (i.e. a talk that takes only 5 minutes). My proposal was accepted and I was able to talk on stage in front of a lot of people about Papercraft, which was a pretty scary experience, but I think it went OK - I stuck to the script I prepared and was done within the allocated time.

Luckily for me, the day before my talk I met Szymon Fiedler, who not only has a wonderfully warm and open personality, he also gave me some tips on how to prepare myself for my lightning talk, which proved really helpful, since I mentioned to him that I had terrible stage fright.

So later that evening I just did what he said and practiced my script again and again in front of the mirror, until I knew the text by heart and could take some liberties with it, and felt more or less fluent with it while staying within the 5 minute allocated to my talk. So, while I was a bit shaky (I felt my voice tremble,) I came off the stage not only with a sense of relief, but also of accomplishment and a taste for more of it, because at the end of the day, like all OSS developers I’m driven by the passion of creating something cool and beautiful that I want to share with others.

The PicoRuby Workshop

Immediately after giving my lightning talk I scrambled off to a side room to join the workshop on PicoRuby, which is an implementation of Ruby for microcontrollers. Each of the participants got a little kit of electronic bits (including a Raspberry Pi Pico 2 W microcontroller) and a bunch of instructions on how to connect to it and get an IRB session running on it.

It took some fiddling and some help from Hitoshi, our workshop guide, but once I had an IRB session running on the microcontroller, it was only a question of hooking up the various components to the breadboard and having fun with it. So, the first thing to do was to connect a LED component and make it blink. I was so excited to make this work, and it reminded me of the joy I felt as a kid when I started programming and making stuff work on my first computer (it was a TI-99/4A). We got to keep the kit, and I know one day I’ll find something useful to do with it.

AI at Euruko

One important aspect of Euruko was the place of AI in it. There were I think four different talks about AI tooling, and Matz even made mention of the fact that he was now using Claude.code, which was a bit surprising to me. But I guess even more surprising to me was hearing people expressing skepticism about AI. I thought I was the only one!

So I was quite happy to listen to other people express apprehension at using LLM’s for programming, not so much for the political/social implications of it, but just for the practical (or rather, impractical) aspects of it. But I’m still not completely sure these tools don’t have any merit, and am curious what value they can bring to software development.

Out of the different AI tools presented at the conference, and which each had their own approach to integration with Ruby programming workflows, the one that was the most interesting to me was RubyLLM by Carmine Paolino. To me it hits the right chord, in that it provides a uniform interface to interacting with AI agents, but doesn’t make too many assumptions on what you want to do with it. To me this really hits the sweet spot of abstraction - and also in a very Ruby-esque way!

The End of Euruko

As the end of the conference drew nearer, I was less and less interested in following the talks and frankly was a bit absent-minded. The afternoon after the workshops (and the lightning talks) was composed of two talks about AI tools (which personally didn’t really resonate with me), but I was really interested in the closing keynote, given by Eileen Uchitelle of Shopify, which was about problems with the “mythical modular monolith”, or really problems with code organization, especially in big companies.

Now, throughout the conference I felt like a very small fish swimming along all those big fish, not only because I’m a nobody in the community and here I am talking to all those people that are all over GitHub and HN and YouTube etc., but also because I’m a one-man company, and the conference was filled with people from very big organizations, such as Shopify.

The fact that I’m not using Rails also made me feel somewhat marginalized, but I quickly found out that I wasn’t the only one. I was talking to Chris Hasinski and he suggested that I talk to Matz, because Matz doesn’t use Rails, and apparently in Japan they all don’t use Rails! Chris told me that actually at RubyKaigi (the Japanese Ruby conference) it’s taboo to talk about Rails, which I found absolutely hilarious.

This gave me an excuse to talk to Matz, so I jumped at the opportunity to present myself to him. He was very kind, and indeed he told me he does not use Rails, and that I should come to RubyKaigi and present my projects. Maybe I will!

Going back to the closing keynote, what struck me was the difference in tone to the opening keynote by Matz. Matz was talking about openness, the importance of “human recognition”, embracing the unknown, and the joy of programming. But Eileen’s talk was very dense (lots of text, I had some trouble following), and the impression I got was that she was laying down all those rules for what works, what doesn’t.

The talk was also delivered in somewhat negative, absolute terms (“this doesn’t work”, “that doesn’t work”), and the injunctions were also somewhat heavy-handed: improve developer education, indoctrinate developers, etc. While she did stress how all code-organization problems are really human problems, there was no compassion there, just a bunch of do’s and dont’s, and while she mentioned developer happiness, I was wondering how can there be happiness without compassion?

So to me it was a bit of a letdown, because it made the conference end on a somewhat negative, cold tone. Maybe this talk would have fared better as an internal talk at Shopify, or maybe in front of other big orgs that use Rails. To me it was more of an example of how I don’t want to do software development, and how I don’t want to deal with human problems in a work situation.

This experience also ties in to the drama currently going on in the Ruby community, which Joel shared with all of us on the morning of the second day of the conference. I was never a user of Rails (except for a very short time when it first came out like 20 years ago), then again I really admired DHH for his vision. But having read some of his recent writings I was quite alarmed that a person with such views should hold such a prominent position in the Ruby community, especially if that community cherishes diversity and inclusivity.

While there are some signs that DHH’s toxic views have played a role in the current crisis enveloping Ruby Central and the Rubygems system, what’s even more bothering to me is the heavy-handed approach we are seeing from Ruby Central, and through them, Shopify (and perhaps other big actors in the Ruby community). While I’m a long time Ruby developer, I’ve only been marginally involved in the community, so to me a lot of this is new. But what is clear to me is that what I don’t want in my life, and what I don’t want to see in the community, is big corporations running the show and deciding for the rest of us what we can or can’t do, or how we do it.

This is the time to show that Ruby is not just Rails, and that there is a Ruby ecosystem outside of Rails (or off the rails if you will). At Euruko I met some of the people who are actively working on this problem, be it developing alternatives to bundler, or new and exciting alternative web frameworks for Ruby.

The strength of an ecosystem is in its diversity. I’m already seeing this in the recent work on HTML templating that I mentioned above, where different solutions to the same problem arise at the same time and cross-fertilize. This can also happen with other parts of the Ruby ecosystem. The more participation we have of small players, the more resilient our community will be. The current crisis teaches us that to prevent abuse of power in our community we need to double down on creating and participating. I definitely intend to do my part to see this happen.

Last But Not Least

I think a special mention should be made about the incredible cultural heritage that the Euruko organizers have shared with us all. It was such a joy to hear and see the richness of Portuguese culture that was on display in the opening and closing ceremonies! And also to taste it: I loved everything I tasted (and I tried to eat mostly where the locals eat), and a glass (or two) of porto just makes you happy!

Hopefully, I’ll come back many times more to this beautiful country and this beautiful people. See you at the next Ruby conference!

P2 is the New Papercraft

Fri, 12 Sep 2025 00:00:00 GMT

In the last few months I’ve been busy working on a few different open-source projects. Some of those projects are still at a preliminary stage, and I’m taking my time in continuing to develop them, use them ( on this website for example), but one project I’ve devoting a lot of attention to is P2, a Ruby gem for writing HTML templates in plain Ruby, which I’ve been writing about lately.

P2 has started as an exploration of how to make Papercraft faster. Papercraft, about which I’ve also written, was centered on the idea that HTML templates should be fun to read and write, and fun to compose in various ways, following the idea of promoting developer happiness, a central tenet in the Ruby community.

While I absolutely adore making tools that are fun to use, I also very much like maximizing performance in the different projects I develop. And benchmarks I did showed me that Papercraft’s performance was somewhat lacking. That is why at a certain point I started looking at how to improve Papercraft’s performance, and I started to explore the idea of doing some kind of transformation (or compilation) of templates in order to make rendering them faster. While I had a general idea of how to do this (by parsing the template’s source code, then transforming the AST, and finally converting it back to source code), and have made sketches that showed this could work, I was not sure about how to integrate all this work into Papercraft.

Fast-forward to a few months ago, after some discussions with Joel Drapper, who’s the author of Phlex and who also is interested in using the same techniques to compile Phlex templates to make them faster, I’ve decided to see if I could look again at this problem, but without the baggage of an already-existing codebase.

As an aside, this idea of re-examining old ideas, old codebases, and old assumptions, is something I’ve been doing more and more of lately. Basically, I try to take a certain functionality which is already implemented in a project, and try to see if I can rethink it, and see if I can arrive at something that is simpler, has less lines of code, works faster, more robust, and has less dependencies.

So that’s how P2 started - it was a reimagining of a HTML-generation Ruby DSL that is always compiled. After a few months of work, I got P2 to where I wanted it to be in terms of performance, namely it is now as fast as ERB, and this is because finally the “compiled” HTML generation source code is almost identical between P2, ERB and ERubi.

So now that P2 is done more or less, it’s time to fold this work back into Papercraft, and concentrate on further improving the developer experience. There’s some work I’ve already done on being able to inject HTML attributes into the rendered HTML, in order to allow the implementation of frontend template debugging tools, such as those recently shown by Marco Roth in ReactionView. And since finally, the Papercraft/P2 templates go through a process of conversion and transformation of AST’s, other future directions Marco’s talking about, such as reactive templates, are also possible.

Now it’s time to turn my attention to my other projects, which I’ll write about in the coming months. Meanwhile, please feel free to take Papercraft for a ride. It’s really fun to use.

How I Made Ruby Faster than Ruby

Mon, 18 Aug 2025 00:00:00 GMT

Note: P2 is now folded back into Papercraft. The P2 repository will eventually be archived.

If you’re a Ruby programmer, you most probably will be familiar ERB templates and the distinctive syntax where you mix normal HTML with snippets of Ruby for embedding dynamic values in HTML.

I wrote recently about P2, a new HTML templating library for Ruby, where HTML is expressed using plain Ruby. Now this is nothing new or even unique. There’s a lot of other Ruby gems that allow you to do that: Phlex, (my own) Papercraft and Ruby2html come to mind.

What is different about P2 is that the template source code is always compiled into an efficient Ruby code that generates the HTML. In other words, the code you write inside a P2 template is actually never run, it just serves as a description of what you actually want to do.

While there have been some previous attempts to use this technique for speeding up template generation, namely Phlex and Papercraft, to the best of my knowledge P2 is the first Ruby gem that actually employs this technique exclusively.

In this post I’ll discuss how I took P2’s template generation performance from “OK” to “Great”. Along the way I was helped by Jean Boussier, a.k.a. byroot who not only showed me how far P2 still has to go in terms of performance, but also gave me some possible directions to explore.

How P2 Templates Work

Here’s a brief explanation of how P2 compiles template code. In P2, HTML templates are expressed as Ruby Procs, for example:

->(title:) {
  html {
    body {
      h1 title
    }
  }
}.render(title: 'Hello from P2') # "<html><body><h1>Hello from P2</h1></body></html>"

Calling the #render method will automatically compile and run the generated code, which will look something like the following:

->(__buffer__, title:) {
  __buffer__ << "<html><body><h1>"
  __buffer__ << ERB::Escape.html_escape((title).to_s)
  __buffer__ << "</h1></body></html>"
  __buffer__
}

As you can see, while the original source code is made of nested blocks, the generated code takes an additional __buffer__ parameter and pushes snippets of HTML into it. Any dynamic value is pushed separately after being properly escaped.

Let’s quickly go over how this code transformation is achieved. First, P2 locates the source file where the template is defined, and parses the template’s source code (using a little gem I wrote called Sirop) into a Prism AST. Here’s a part of the AST for the above example, showing the call to body with the nested h1 (with non-relevant parts removed):

@ CallNode (location: (6,4)-(8,5))
├── receiver: ∅
├── name: :body
├── arguments: ∅
└── block:
    @ BlockNode (location: (6,9)-(8,5))
    ├── locals: []
    ├── parameters: ∅
    └── body:
        @ StatementsNode (location: (7,6)-(7,14))
        └── body: (length: 1)
            └── @ CallNode (location: (7,6)-(7,14))
                ├── receiver: ∅
                ├── name: :h1
                ├── arguments:
                │   @ ArgumentsNode (location: (7,9)-(7,14))
                │   └── arguments: (length: 1)
                │       └── @ LocalVariableReadNode (location: (7,9)-(7,14))
                │           ├── name: :title
                │           └── depth: 2
                └── block: ∅

(You can look at the AST for any proc by calling Sirop.to_ast(my_proc) or my_proc.ast.)

Now if we look at the above DSL we can see that the calls to html, body and h1 are represented as nodes of type CallNode, and those nodes have the receiver set to nil (because there’s no receiver), and that the HTML tag name is stored in name. So the first step in transforming the code is to translate each CallNode into a custom node type that could later be used to generate snippets of HTML that will be added to the HTML buffer. The translation is performed by the TagTranslator class, which looks for specific patterns and when a pattern is matched, replaces the given node with a custom node. Let’s look at TagTranslator#visit_call_node:

class TagTranslator < Prism::MutationCompiler
  ...

  def visit_call_node(node, dont_translate: false)
    return super(node) if dont_translate

    match_builtin(node) ||
    match_extension(node) ||
    match_const_tag(node) ||
    match_block_call(node) ||
    match_tag(node) ||
    super(node)
  end

  ...
end

A Prism::MutationCompiler is a class that returns a modified AST based on the return value of each #visit_xxx method. So #visit_call_node, as its name suggests, visits nodes of type CallNode and the return value is used for mutating the AST. If we look at the #match_tag method, we’ll see how the call node is transformed:

def match_tag(node)
  return if node.receiver

  TagNode.new(node, self)
end

So what happens is that for normal HTML tags, the #match_tag method will return a custom TagNode. Once the entire AST is traversed, we we’ll have a mutated AST where all relevant calls have been translated into instances of TagNode (there are other custom node classes that correspond to other parts of the P2 DSL).

The next step is to transform the mutated AST back to source. The heavy lifting is done by the Sirop gem, with the Sourcifier class, which allows us to transform a given AST to Ruby source code. But the Sirop sourcifier doesn’t know anything about those custom P2 node types, such as TagNode, so we need to help it a bit. We do this by subclassing it, and adding some code for dealing with all those custom nodes:

def visit_tag_node(node)
  tag = node.tag
  is_void = is_void_element?(tag)

  # emit open tag
  emit_html(node.tag_location, format_html_tag_open(tag, node.attributes))
  return if is_void

  # emit nested block
  case node.block
  when Prism::BlockNode
    visit(node.block.body)
  when Prism::BlockArgumentNode
    flush_html_parts!
    adjust_whitespace(node.block)
    emit("; #{format_code(node.block.expression)}.compiled_proc.(__buffer__)")
  end

  # emit inner text
  if node.inner_text
    if is_static_node?(node.inner_text)
      emit_html(node.location, ERB::Escape.html_escape(format_literal(node.inner_text)))
    else
      to_s = is_string_type_node?(node.inner_text) ? '' : '.to_s'
      emit_html(node.location, interpolated("ERB::Escape.html_escape((#{format_code(node.inner_text)})#{to_s})"))
    end
  end

  # emit close tag
  emit_html(node.location, format_html_tag_close(tag))
end

When HTML is emitted, the corresponding code is not generated immediately. Instead, each piece of HTML is pushed into an array of pending HTML parts. When the time comes to flush the pending HTML parts and generate code for them, we concatenate all static strings together into a single buffer push, while each dynamic part is escaped and pushed separately.

The P2 compiler does similar work for dealing with other parts of the P2 DSL, such as template composition, deferred execution, extension tags etc. In addition there’s quite a bit of work around generating a source map that maps lines from the compiled code to lines in the original source code. When an exception is raised while generating a template, P2 uses these source maps to translate the exception’s backtrace such that it will point to the original source code.

So How Can We Make Ruby Faster than Ruby?

Now that we have an idea of how P2 works, let’s look at how I’ve taken P2 performance from OK to great. When I first released P2, I was quite content with its performance, since it was significantly faster than Papercraft, and the benchmark I wrote compared it against ERB. But I haven’t taken into account the fact that I know so little about ERB, and especially about getting the best performance out of ERB templates.

Luckily, right after first publishing the repository, I got a nice PR from byroot that showed that P2 was not so fast as I thought. While the discussion above shows how P2 generates code now, at the time it was generating code that was not the best. Here’s how the code P2 generated at the time looked (for the same template example shown above):

->(__buffer__, title:) do
  __buffer__ << "<html><body><h1>#{CGI.escape_html((title).to_s)}</h1></body></html>"
  __buffer__
rescue => e
  P2.translate_backtrace(e)
  raise e
end

Now there are a few things in the above code that prevent it from being as fast as compiled ERB (using the ERB or the ERubi gems):

Pushing an interpolated string to the buffer is slower than pushing each part separately. This is kind of obvious when you take into account the fact with an interpolated string, you first need to create a string that receives the static and dynamic parts of the interpolated string, and then push that string to __buffer__.
The rescue clause adds some overhead. This can be especially expensive when you have nested templates. If each partial has its own rescue block, that can quickly add up to a significant overhead.
As byroot pointed out, when the generated code is evaled into a Proc, literal strings will be default not be frozen, which adds allocation overhead and GC pressure.
As mrinterweb pointed out in a separate PR, escaping HTML with the ERB::Escape.html_escape is faster than CGI.escape_html (just a couple percentage points but still…)

So taking all this advice into account, I’ve rewritten the compiler code to do the following:

Separate HTML code generation, such that static HTML strings are contatenated and pushed to the HTML buffer once, and any dynamic parts are pushed separately.
Remove the rescue clause and instead do the backtrace translation only once in Proc#render.
Add the # frozen_string_literal: true magic comment at the top of the compiled code, so all static HTML content is made of frozen strings, which reduces allocation and GC pressure. BTW, when are we going to get frozen literal strings by default?
Switch from using CGI.escape_html to ERB::Escape.html_escape.

When byroot made his PR, the benchmark looked like this:

ruby 3.4.2 (2025-02-15 revision d2930f8e7a) +YJIT +PRISM [arm64-darwin24]
Warming up --------------------------------------
                 erb    31.381k i/100ms
                  p2    65.312k i/100ms
               erubi   179.937k i/100ms
Calculating -------------------------------------
                 erb    314.436k (± 1.3%) i/s    (3.18 μs/i) -      1.600M in   5.090675s
                  p2    669.849k (± 1.1%) i/s    (1.49 μs/i) -      3.396M in   5.070806s
               erubi      1.869M (± 2.3%) i/s  (535.01 ns/i) -      9.357M in   5.008683s

Comparison:
                 erb:   314436.3 i/s
               erubi:  1869118.6 i/s - 5.94x  faster
                  p2:   669849.2 i/s - 2.13x  faster

Which showed the P2 still had a lot to improve, as it was almost 3 times slower than ERubi. (I later also found out how to make ERB compile its templates, its compiled performance is more or less the same as compiled ERubi.) After the changes I’ve implemented here are the updated benchmark results:

ruby 3.4.5 (2025-07-16 revision 20cda200d3) +YJIT +PRISM [x86_64-linux]
Warming up --------------------------------------
                  p2   128.815k i/100ms
          papercraft    17.480k i/100ms
               phlex    15.620k i/100ms
                 erb   159.678k i/100ms
               erubi   154.085k i/100ms
Calculating -------------------------------------
                  p2      1.454M (± 2.4%) i/s  (687.59 ns/i) -      7.342M in   5.051705s
          papercraft    173.686k (± 2.7%) i/s    (5.76 μs/i) -    874.000k in   5.035996s
               phlex    155.211k (± 2.5%) i/s    (6.44 μs/i) -    781.000k in   5.035369s
                 erb      1.567M (± 4.2%) i/s  (637.97 ns/i) -      7.824M in   5.000791s
               erubi      1.498M (± 4.2%) i/s  (667.45 ns/i) -      7.550M in   5.048427s

Comparison:
                  p2:  1454360.2 i/s
                 erb:  1567482.7 i/s - 1.08x  faster
               erubi:  1498238.4 i/s - same-ish: difference falls within error
          papercraft:   173686.1 i/s - 8.37x  slower
               phlex:   155211.0 i/s - 9.37x  slower

The benchmark shows that P2 is now on par with ERB and ERubi in terms of the performance of compiled templates (and basically, the generated code for all three is more or less identical.) I’ve also added Papercraft and Phlex to show the difference compilation makes, especially since P2 is really an offshoot of Papercraft, and the DSL in P2 and Papercraft is almost identical. (Phlex has also seen some work on template compilation, but I don’t know how far advanced this is.)

As you can see, the compiled approach can be about 10X as fast as the non-compiled approach. Of course, there’s the usual caveat about benchmarks: it’s a very simple template with just two partials and not a lot of dynamic parts, but this is indicative of the kind of performance you can expect from P2. As far as I know, P2 is the first Ruby HTML-generation DSL that offers the same performance as compiled ERB/ERubi.

Conclusion

What I find most interesting about the changes I’ve made to code generation in P2, is that the currently compiled code is more than twice as fast as it was when P2 first came out, which just goes to show than in fact Ruby is not slow, it is actually quite fast, you just need to know how to write fast code! (And I guess this is true for any programming language.)

Hopefully, the Ruby-to-Ruby compilation technique discussed above would be adpoted for other uses, and for more DSL’s. I already have some ideas rolling around in my head…

P2 - a Functional HTML Templating Engine for Ruby

Thu, 07 Aug 2025 00:00:00 GMT

Note: P2 is now folded back into Papercraft. The P2 repository will eventually be archived.

I’ve just released P2, a new HTML templating engine for Ruby. P2 builds on the work I did in Papercraft, but takes things quite a bit farther: templates are expressed as plain procs and are automatically compiled in order to make it fast. How fast?

Here are the results of benchmarking Papercraft against ERB:

sharon@nf1:~/repo/papercraft$ ruby --yjit examples/perf.rb
Warming up --------------------------------------
          papercraft    10.187k i/100ms
                 erb    14.435k i/100ms
Calculating -------------------------------------
          papercraft    106.983k (± 4.2%) i/s -    539.911k in   5.057203s
                 erb    155.081k (± 2.4%) i/s -    779.490k in   5.029535s

Comparison:
                 erb:   155080.9 i/s
          papercraft:   106983.3 i/s - 1.45x  slower

And here are the results of benchmarking P2 against ERB, with the same template:

sharon@nf1:~/repo/p2$ ruby --yjit examples/perf.rb
Warming up --------------------------------------
                  p2    27.859k i/100ms
                 erb    14.935k i/100ms
Calculating -------------------------------------
                  p2    309.524k (± 3.4%) i/s -      1.560M in   5.047232s
                 erb    154.619k (± 2.5%) i/s -    776.620k in   5.026242s

Comparison:
                  p2:   309523.6 i/s
                 erb:   154619.0 i/s - 2.00x  slower

So, while Papercraft was about 30% slower than ERB, P2 is about twice as fast! How does it do that?

Writing HTML with a Ruby DSL

Of course, Papercraft and P2 are not the only Ruby gems that allow expressing HTML using plain Ruby. Here’s a simple example of a template that will work in both Papercraft and P2:

template = ->(title) {
  div {
    h1 title
  }
}

template.render('Hello') #=> "<div><h1>Hello</h1></div>"

This approach to writing HTML templates is very appealing (to me at least). I find that it is easy to write, easy to read, and also very easy to embed dynamic values in the produced HTML. But, this approach does have a cost: all those curly brackets - they denote blocks, and calling methods with blocks is kind of slow.

It’s not like ERB is super fast, but ERB templates are normally compiled to code that just stuffs strings and interpolated values into a buffer. Surely we can do the same with the above template. An ideal compiled version of the above template would look like the following:

buffer << "<div><h1>#{CGI.escape_html(title)}</h1></div>"

… which looks so deceptively simple. But how can we transform the original template, with all its curly brackets, into the single line of code above?

Parsing Ruby Code with Prism

We first need to parse our template code, so we’ll be able to transform it. We do this with Prism, Ruby’s new parser gem. Prism takes a piece of source code, and returns an AST (abstract syntax tree) that we can then manipulate:

Prism.parse("-> { h1 'Hello, world!' }").value
#=> @ ProgramNode ... (a whole page's worth of AST nodes)

However, some stuff is still missing:

We want to be able to compile Procs on the fly. How can we dynamically get the source code for each proc we want to compile?
We also need to be able to convert ASTs back into source code. How do we do this?

To answer these two questions I have created another gem which I call Sirop. Sirop takes a Proc or a Method object, finds its source code, and then uses Prism to return its AST:

require 'sirop'

template = -> { h1 'Hello, world!' }
Sirop.to_ast(template) #=> @LambdaNode ...

Sirop also allows us to get the source of the given template:

Sirop.to_source(template) #=> "-> { h1 'Hello, world!' }"

One important limitation that Sirop has is that it can only work on Procs (or methods) defined in a file. In other words, it will not work on procs defined in an IRB session. The same limitation holds for P2 as well.

Sirop provides a Sourcifier class that does the conversion of an AST back into source code. P2 uses the Sourcifier class and overrides its stock behaviour in order to be able to convert calls such as h1 'Hello, world!' into strings that are emitted into a buffer.

How P2 Compiles Templates

The template compilation process is done using the following steps:

Convert the given template Proc into an AST (using Sirop)
Transform the AST - replacing each instance of Prism::CallNode with a P2::TagNode.
Convert the transformed AST into source code, converting any TagNode into the appropriate HTML.

The last step is more involved than the first two, and needs to take into account buffering (for example, taking multiple tag calls and coalescing them into a single string that’s pushed into the buffer), escaping of HTML content, and dealing with other features that P2 provides, such as deferred rendering, template composition etc.

Dealing with Exception Backtraces

Another consideration to take into account when compiling templates is that any exception raised while rendering the template will show an incorrect backtrace. P2 generates a source map mapping line numbers in the compiled proc to line numbers in the original proc source code. This allows us to manipulate the backtrace for any exception raised while rendering, such that any entry occurring in the compiled proc will point to the corresponding line in the original template source code:

require 'p2'

template = -> {
  h1 'foo'
  raise 'bar'
}
template.render #=> throws an exception...

And the backtrace will show the line where the exception was raised, except that the code that actually ran is completely different!

test.rb:5:in 'block (2 levels) in <main>': bar (RuntimeError)
	from /home/sharon/.rbenv/versions/3.4.5/lib/ruby/gems/3.4.0/gems/p2-2.0.1/lib/p2/proc_ext.rb:31:in 'Proc#render'
	from test.rb:11:in '<main>'

Some Future Directions

The automatic compilation of Ruby DSLs can bring other benefits beyond just performance improvements. We can really extend the syntax in order to make it easier to express HTML. Here are some ideas I am currently exploring for extending the syntax for expressing HTML tags and attributes:

# syntax for specifying id's:
h1[:foo] 'bar' #=> <h1 id="foo">bar</h1>

# syntax for specifying class:
h1.foo 'bar' #=> <h1 class="foo"></h1>

# syntax for specifying arbitrary attributes:
a.href('/about') 'About' #=> <a href="/about">About</a>

Another direction I’m exploring is being able to extend the DSL by expressing extensions as Procs, and having P2 automatically inline them:

P2.extend(
  ulist: ->(list) {
    ul {
      list.each {
        li it
      }
    }
  }
)

Currently, P2 is perfectly able to emit procs, but the code looks like this:

# source
->(items) {
  div { emit ulist(items) }
}

# compiled template
->(__buffer__, items) {
  __buffer__ << "<div>#{P2.render_emit_call(ulist(items))}</div>"
  ...
}

With proc inlining, it would look like this:

->(__buffer__, items) {
  __buffer__ << "<div><ul>"
  items.each {
    __buffer__ << "<li>#{CGI.escape_html(it.to_s)}</li>"
  }
  __buffer__ << "</ul></div>"
  ...
}

That’s it for now. If you find P2 useful, please let me know, and feel free to contribute issues or PR’s here: https://github.com/digital-fabric/p2

Introducing UringMachine

Sat, 28 Jun 2025 00:00:00 GMT

UringMachine is a new Ruby gem for performing concurrent I/O using io_uring and Ruby fibers. In UringMachine I’ve implemented all the lessons I’ve learned from my previous attempts at bringing io_uring to Ruby: Polyphony, a comprehensive gem providing io_uring functionality, structured concurrency, and monkey-patching for the Ruby standard library; and IOU, a low level async API for using io_uring from Ruby.

Here are some of those lessons:

Monkey-patching

I’ve defended monkey-patching before, but my stance now is more nuanced. Frankly, it’s a big pain to maintain, because the Ruby stdlib is evolving, and in some cases it’s really an excercise in futility. So, UringMachine doesn’t really touch the standard IO classes, it just doesn’t use them. UringMachine works with plain file descriptors:

require 'uringmachine'

machine = UM.new
fd = machine.open('foo.bar', UM::O_RDONLY)
buf = +''
ret = machine.read(fd, buf, 1 << 16)
puts "read #{ret} bytes: #{buf.inspect}"
machine.close(fd)

Now, at first sight this might look like a really clumsy way to do I/O in Ruby, but there’s more than meets the eye in the above example. A second example will make things a bit clearer:

filenames = ['foo.bar', 'bar.baz', 'baz.foo']
content_map = {}

MAX_READ_LEN = 65536

# A neat abstraction to do what IO.read(fn) does
def read_file(machine, fn)
  fd = machine.open(fn, UM::O_RDONLY)
  buf = +''
  loop do
    # the -1 means append to buffer
    ret = machine.read(fd, buf, MAX_READ_LEN, -1)
    return buf if ret < MAX_READ_LEN
  end
ensure
  machine.close(fd) if fd
end

fibers = filenames.map do |fn|
  machine.spin do
    content_map[fn] = read_file(fn)
  rescue SystemCallError
    puts "Failed to read #{fn}"
  end
end
machine.join(*fibers)
p content_map: content_map

So yes it’s much lower level than IO’s #read and #write methods, but what we gain here is easy launching of concurrent (a.k.a. asynchronous) operations. In the above example, three files are read concurrently in three separate fibers by calling #spin (just like in Polyphony). Each fiber opens the given filename, reads the contents until EOF and then puts the content in a content map. Meanwhile the main fiber calls #join in order to wait for the three spun fibers to terminate.

The “Everything but the Kitchen Sink” Approach

Polyphony was a library that was trying to do a lot of things, such as structured concurrency, on-the-fly file compression (using pipes), HTTP parsing and chunked transfer encdoing, and an actor API with message passing.

The result was a big gem (~2.7KLOC of Ruby, plus ~6KLOC of C) with a very big surface area, lots of classes, lots of money-patches. In comparison, UringMachine is currently at about 200LOC of Ruby, plus ~2.7KLOC of C.

UringMachine does not try to do everything that’s related to concurrency. Instead, it focuses on providing the basics, which you can then use to build out the abstractions you want, such as message passing between fibers, structured concurrency etc.

A Simpler, More Robust Implementation

Polyphony has started as a wrapper around libev, and has evolved with time to provide an alternative backend that uses io_uring instead of libev. UringMachine was built from the beginning as a Linux-only, io_uring-only library. Its design is also in many ways simpler than that of the Polyphony backend.

I still haven’t run benchmarks comparing UringMachine to Polyphony, but from using UringMachine for running a webserver, it looks about the same, which means fast.

A New Ecosystem

Polyphony is but one of a bunch of Ruby gems I’ve been writing for fun and profit. Some of those tools were written just to explore what’s possible in Ruby, and some of those are being actively used in production on my clients’ websites.

But UringMachine does not offer the same (theoretical) level of integration with the Ruby ecosystem as Polyphony. It just does I/O its own way. It does not interfere with other gems or the way Ruby does I/O, but any I/O that does not go through UringMachine means that the parts that do use UringMachine for I/O and concurrency may be waiting for stock Ruby I/O operations to complete before being able to complete their own I/O operations.

Thus, to do anything useful, UringMachine will need to be accompanied by other gems that create an ecosystem around it.

TP2 - a New Webserver for Ruby

TP2 is a new webserver based on UringMachine. It is the continuation of my work on Tipi. TP2 also has a much simpler design than Tipi (and a much smaller codebase). TP2 uses UringMachine for managing I/O and concurrency.

Syntropy - a New Web Framework for Ruby

A third part of the new ecosystem is Syntropy, a new web framework I’ve been working on. It, too, is a continuation of the work I’ve been doing in Impression.

Like Impression, Syntropy is a filesystem-based router. The routes of the app are defined by the directory and file structure of your app. So for example, HTTP requests to /foo/bar will be routed to site/foo/bar.rb. Syntropy can also serve static files and markdown files.

Dynamic routes are served by Ruby files that are called modules. Each module exports an request handler, which can be in the form of a custom class, a Papercraft template, or a simple Proc:

# The world's simplest possible Syntropy module:
export ->(req) { req.respond('Hello, world!') }

Other features in Syntropy:

Define middleware with _hook.rb modules
Define custom error handlers with _error.rb modules
Support for working with collections of markdown files, e.g. a list of blog articles.
Support for clean URLs (i.e. without the file extension) for HTML, markdown and module files.
Development mode with automatic route reloading on file change.

Future directions for Syntropy include:

Tools for deploying your web app to any Linux server using Docker Compose
Tools for creating a skeleton web app.
A data modeling layer based on SQLite databases, for use in: caching, logging, web analytics and more.
The ability to integrate applets in a webapp, so for example an applet can include subroutes defined in a separate directory (such as a different gem’s directory), for ease of integration.

Future Work on UringMachine

UringMachine is still missing some features. Here are some of the things I intend to add in the near future:

Support for more io_uring ops, such as splice, sendto, recvfrom
Support for polling
Some abstractions for easily reading and writing entire files.
Support for SSL I/O.

Conclusion

UringMachine, TP2 and Syntropy represent a new direction in my work. My aim is to build tools that fit my way of writing software, and that are tailor-made for my workflow and my preferences. As such, I’m trying to avoid feature creep, and instead focus on just adding (or changing) the stuff I need in order to execute for my clients.

Hopefully, someone else may find these projects interesting enough to use them, but it’s really more about scratching my own itch. The proof is right here on this website, which is made with the above tools.

Exploring unix pipes with Ruby and Polyphony

Mon, 04 Apr 2022 00:00:00 GMT

Most of you are probably familiar with the concept of pipes on Unix-like OSes. We mostly encounter pipes on the command line, where we can use them to use one command’s output as another’s input. But pipes can also be used programmtically. On Linux specifically, pipes let us move data between two arbitrary file descriptors, without the data ever being seen by our user-space program, and in some cases without it even being copied at all. This is done using the splice() system call, which we’ll explore in more detail below.

In the last few weeks I’ve been working on adding some abstractions to Polyphony that make it easier to work with pipes and to use splice() to move data between arbitrary file descriptors (which Rubyists know as IO instances.)

I have also added a few data compression/decompression APIs that can significantly improve performance when working with compressed data, and remove the allocation overhead involved when working with the zlib APIs in Ruby’s standard library. But let’s start by discussing splice() and what it allows us to do.

How to use splice with Polyphony

Let’s start with the basics: the splice system call (available only on Linux) allows us to move data between two arbitrary file descriptors, without reading it from the source fd, and then writing it to the target fd, which necessitates moving data back and forth between the kernel and our user space program. If you’ve read the above-linked man page for splice(), you’ll have noticed that splice() needs at least one of the given file descriptors to refer to a pipe. As Linus Torvalds himself explains:

The pipe is just the standard in-kernel buffer between two arbitrary points. Think of it as a scatter-gather list with a wait-queue. That’s what a pipe is. Trying to get rid of the pipe totally misses the whole point of splice()…

…it’s what allows you to do other things with splice that are simply impossible to do with sendfile. Notably, splice allows very naturally the “readv/writev” scatter-gather behaviour of mixing streams. If you’re a web-server, with splice you can do
    write(pipefd, header, header_len);
    splice(file, pipefd, file_len);
    splice(pipefd, socket, total_len);

The splice() lets us move data between, say, a file and a socket, for example in an HTTP server, without that data ever being moved back and forth between our user space program and the kernel. The kernel handles the moving of data directly, and in some cases without any copying at all - which means less CPU time, less memory usage and less memory allocations.

In addition, the use of a pipe also takes care of back pressure. I’ll let Linus explain:

The reason you want to have a pipe in the middle is that you have to handle partial moves some way. And the pipe being the buffer really does allow that, and also handles the case of “what happens when we received more data than we could write”…

In particular, what happens when you try to connect two streaming devices, but the destination stops accepting data? You cannot put the received data “back” into the streaming source any way - so if you actually want to be able to handle error recovery, you have to get access to the source buffers.

So, to sum up, if we want to directly move data between two arbitrary fd’s, we need to:

Create a pipe, with a read fd and a write fd, which actually encapsulates a kernel buffer.
Use splice() to move data from the source fd to the pipe.
Use splice() to move data from the pipe to the target fd.

Operations 2 and 3 would repeat until the source fd reaches end-of-file (EOF), and ideally would run concurrently. Let’s examine how we can do that using Polyphony.

Polyphony introduces the IO.splice method, which has the following signature:

IO.splice(src, dest, len)

This method splices a total of len bytes from src to dest, which are both IO instances. If len is negative, IO.splice will keep splicing (with chunks of up to -len bytes) until EOF is encountered. What’s important to note about this method is that it will block the current fiber if the source is not readable or the destination is not writable. And of course, as stated above, at least one of the IOs involved needs to refer to a pipe.

def move_data_between_ios(source, target)
  r, w = IO.pipe
  f1 = spin do
    IO.splice(source, w, -8192)
    w.close
  end
  f2 = spin do
    IO.splice(r, target, -8192)
  end
  Fiber.await(f1, f2)
end

The move_data_between_ios method above starts by creating a pipe, then spins up a fiber (f1) which splices data continuously from the source IO to the pipe (until EOF is encountered), then closes the write end of the pipe. A second fiber (f2) splices data from the pipe to the target IO, until the pipe has reached EOF (hence the importance of the call to w.close in the first fiber).

So actually, there’s a quite lot happening here: we split the work between two fibers: one for moving data from the source to the pipe, another for moving data from the pipe to the target. What’s interesting is that these two concurrent splice operations are dependent on the speed of both the source and the target, with the pipe providing a buffer (by default holding up to 64KB of data) that can somewhat dampen any “stuttering” of either the source or the target.

Finally, the performance of this double splice operation will be limited by the slowest of the two. In other words, the above solution theoretically provides the best possible behaviour in terms of responding to back pressure. Under ideal conditions, where data is always available for reading from the source, and the target always accepts more data to write, we pay a negligible price for passing data first from the source to a kernel buffer, then from the buffer to the target (the kernel performs splicing by incrementing reference counts, in general there’s no copying of data involved). But when we have a read-constrained source, or a write-constrained target, our kernel buffer will allow us to minimize the time wasted waiting for data to become readable or writable.

And the best part: we don’t need to manage a user-space buffer in our program! We could be streaming GBs of data without worrying about allocating Ruby strings and stressing the Ruby GC. And we didn’t need to copy data from the kernel to our program and back.

You will have noticed, however, that there are some drawbacks to this approach: we need to create a pipe, and we also need to create two fibers in which to run the two splice operations concurrently. This overhead means that this approach to moving data between two file descriptors is probably worth it only under certain circumstances, for example above a certain quantity of data, but most importantly when dealing with potentially slow clients, or when latency varies wildly. Later on in this article I’ll discuss some new APIs I have introduced meant to reduce the overhead involved in performing concurrent splicing.

Some use cases for splicing

Let’s now look at a few examples that show how splice() can be used in a variety of situations. The most obvious place to start is, for me, an echo server. Since the echo server just sends back whatever you send to it, it seems logical to use splice to implement it. Let’s look at how a “normal” echo server will be implemented using Polyphony:

require 'polyphony'

def handle_client(conn)
  spin do
    while (data = conn.read(8192))
      conn.write(data)
    end
  end
end

server = TCPServer.open('127.0.0.1', 1234)
puts 'Echoing on port 1234...'
while (conn = server.accept)
  handle_client(conn)
end

As you can see, the #handle_client method simply reads from the client, then writes the data back to the client. So the data itself does not interest us, we don’t do any processing on it, but we still need to create a string every time we read from the connection, then write this string back.

Echo server using splice

Using splice, however, frees us from having to deal with allocating Ruby strings:

def handle_client(conn)
  spin do
    r, w = IO.pipe
    spin do
      IO.splice(conn, w, -8192)
      w.close
    end
    IO.splice(r, conn, -8192)
  end
end

This version of #handle_client using splice looks almost identical to the move_data_between_ios method discussed above. The only difference is that we slightly reduce the overhead involved in setting up the concurrent operations by spinning up only one fiber in addition to the fiber dedicated to the client. We pass -8192 as the len argument to IO.splice, which means that the splice operation will be repeated until EOF is encountered, in chunks of 8KB.

TCP proxy

Another use case for splice that comes to mind is a TCP proxy. Our TCP proxy will accept incoming connections, will create a connection to some destination TCP server, and will pass data in both directions between the two connections.

Here we need to setup two pipes and additional fibers in order to perform the splicing in both directions:

require 'polyphony'

DESTINATION = ['127.0.0.1', 1234]

def handle_client(conn)
  spin do
    dest = TCPSocket.new(*DESTINATION)

    r1, w1 = IO.pipe
    spin { IO.splice(conn, w1, -8192); w1.close }
    spin { IO.splice(r1, dest, -8192) }

    r2, w2 = IO.pipe
    spin { IO.splice(dest, w2, -8192); w2.close }
    IO.splice(r2, conn, -8192)
  rescue Errno::ECONNRESET
    # ignore
  ensure
    dest.close rescue nil
  end
end

puts "Serving TCP proxy on port 4321..."
TCPServer.new('127.0.0.1', 4321).accept_loop { |c| handle_client(c) }

For each incoming connection to our TCP proxy (serving on port 4321), a new connection to the target (on port 1234) will be established, and then data will be moved in both directions. Any data sent by the client will be spliced to the destination connection, and any data received from the destination will be spliced back to the client.

Note how we now need to create two pipes, one for each direction, and we need to run four splice operations at once. As you can see, the overhead involved is already substantial - we need to setup at least three more fibers to run our splice operations, and we need to create two pipes with two IO instances for each pipe. Surely we can do better…

A better pipe abstraction

Let’s start with pipes. Currently the Ruby core IO API provides the IO.pipe method, which returns two IO instances - one for the read end of the pipe, and one for the write end of the pipe. This way of working with pipes has two problems: first, as discussed above, whenever we create a pipe we actually need to allocate and setup two IO instances; and second, we need to name those IO instances in a way that will signify their usage. Note how in the above TCP proxy program I just gave generic names to the pipe ends:

r1, w1 = IO.pipe
r2, w2 = IO.pipe

Those names are not very descriptive and it’s easy to get them mixed up when passing them to IO.splice. What if we introduced a Pipe class that encapsulated both ends of the pipe? We can then give each pipe a name that will mean something:

client_to_dest_pipe = Polyphony.pipe
dest_to_client_pipe = Polyphony.pipe

The Polyphony.pipe method, introduced to Polyphony a few days ago does just that. It offers the same API as a normal IO object, but any read operation will be performed on the read end of the pipe, and any write operation will be performed on the write end. This means that Polyphony::Pipe instances can be used just like normal IO instances, and can also be passed to IO.splice.

So if we were to use Polyphony.pipe instead of IO.pipe, here’s how the handle_client method would look like:

def handle_client(conn)
  spin do
    pipe = Polyphony.pipe
    spin do
      IO.splice(conn, pipe, -8192)
      pipe.close
    end
    IO.splice(pipe, conn, -8192)
  end
end

The difference is subtle, but notice how this code looks a lot cleaner, and we have only a single variable acting as our pipe. We have saved on object allocations, and we have also made our code more readable and easier to understand. The TCP proxy example now looks like this:

def handle_client(conn)
  spin do
    dest = TCPSocket.new(*DESTINATION)

    client_to_dest = Polyphony.pipe
    spin { IO.splice(conn, client_to_dest, -8192); client_to_dest.close }
    spin { IO.splice(client_to_dest, dest, -8192) }

    dest_to_client = Polyphony.pipe
    spin { IO.splice(dest, dest_to_client, -8192); dest_to_client.close }
    IO.splice(dest_to_client, conn, -8192)
  rescue Errno::ECONNRESET
    # ignore
  ensure
    dest.close rescue nil
  end
end

Reducing fiber usage for splicing

The second issue we had, where we needed two splice operations to happen at the same time, was that this necessitates two separate fibers to be running concurrently. This is due to Polyphony’s design, where each fiber can only perform one blocking operation at a time.

But since the Polyphony backend is perfectly capable of launching multiple I/O operations at once, it occurred to me that an abstraction that sets up a pipe, then performs two concurrent splice operations could be immensely handy! Here’s the basic idea I came up with, implemented on the io_uring backend (the actual implementation is here):

def backend_double_splice_to_eof(src, dest)
  r, w = create_pipe
  total = 0

  ctx_src = prep_splice_op_ctx(src, w)
  ctx_dest = prep_splice_op_ctx(r, dest)
  submit_deferred_sqes

  while true
    resume_value = backend_await
    if interrupted(resume_value)
      raise_if_error(resume_value)
      return value
    end

    if ctx_src&.completed?
      ctx_src.release
      if ctx_src.result == 0
        w.close
        ctx_src = nil
      else
        ctx_src = prep_splice_op_ctx(src, w)
      end
    end

    if ctx_dest&.completed?
      ctx_dest.release
      break if ctx_dest.result == 0

      total += ctx_dest.result
      ctx_dest = prep_splice_op_ctx(r, dest)
    end
    submit_deferred_sqes
  end

  total
ensure
  r.close
  w.close
end

Let’s examine what’s going on above.

We start by creating a pipe and resetting a variable for counting the total bytes transferred.
We then setup two operation contexts, one for each splice (source to pipe, pipe to destination), and submit the corresponding SQEs.
We then start a loop:
- We yield control to the next runnable fiber by calling backend_await. Our fiber will be resumed once any of the two splice operations completes.
- We perform checks to see if the operation has been interrupted. If so, we either raise an error in case the resume value is an exception, or simply return the value.
- We check if the splice moving data from the source to the pipe has completed.
  - If the result of the operation is 0, that means we have hit EOF and we can stop splicing from the source, and we can close the write-end of the pipe (in order to signal an EOF to the other splice operation.)
  - Otherwise, we create a new, identical splice operation in order to continue moving data from the source to the pipe.
- We check if the splice moving data from the pipe to the destination has completed.
  - If the result of the operation is 0, that means we have hit EOF and we can break out of the loop.
  - Otherwise, we add the bytes transferred to our total bytes counter, and then create a new, identical splice operation to continue moving data from the pipe to the destination.
We return the total bytes spliced, and make sure the pipe is closed.

So now that we have our IO.double_splice_to_eof method, let’s see how our echo server now looks:

def handle_client(conn)
  spin do
    IO.double_splice(conn, conn, -8192)
  end
end

We have basically eliminated all of the code dealing with setting up a pipe and performing two splices concurrently. And we still keep the same cancellation behaviour (with Polyphony we can cancel any blocking operation at any moment).

Let’s now see how this new method can be used in the TCP proxy we looked at above:

def handle_client(conn)
  spin do
    dest = TCPSocket.new(*DESTINATION)

    spin { IO.double_splice(conn, dest, -8192) }
    IO.double_splice(dest, conn, -8192)
  rescue Errno::ECONNRESET
    # ignore
  ensure
    conn.close rescue nil
    dest.close rescue nil
  end
end

Here we still need to spin up a second fiber so we could move data in both directions at the same time, but we have greatly simplified our code, and minimized the overhead involved. Here again, the actual data transferred between the two sockets stays in the kernel, we just tell the kernel how to move it.

If we run both our TCP proxy and our echo server, we can easily test that they both work correctly. Let’s start by checking the echo server:

$ ruby examples/pipes/echo_server.rb &
$ echo "foobar" | nc -N localhost 1234
foobar

We can then run our TCP proxy, which will connect to our echo server:

$ ruby examples/pipes/tcp_proxy.rb &
$ echo "foobar" | nc -N localhost 4321
foobar

Pipes and compression

Let’s take a short detour from splice and look at some other new APIs recently introduced in Polyphony, all related to data compression. The Ruby stdlib includes the zlib gem, which provides a Ruby binding for zlib - a popular library for compressing and decompressing data, and it works just fine, but I wanted to create an API that will allow developers to compress data directly between two file descriptors, without having to copy it to and from Ruby strings.

The new compression/decompression APIs are:

IO.deflate(src, dest)
IO.inflate(src, dest)
IO.gzip(src, dest, info = nil)
IO.gunzip(src, dest, info = nil)

These four methods all take a source and a destination, which would normally be IO instances, but they also accept Ruby strings as either a source or a destination. IO.gzip takes an optional hash that can be used to set gzip meta data, with the following keys:

:orig_name - original file name
:mtime - the file’s time stamp
:comment - comment

IO.gunzip can also take an optional hash that will be populated with the same gzip meta data read from the source file descriptor, with the same keys as IO.gzip. Here’s a simple Ruby script that takes uncompressed data on STDIN, gzips it and outputs the compressed data to STDOUT:

require 'polyphony'

IO.gzip(STDIN, STDOUT)

We can very easily test that our script works correctly:

$ echo "foobar" | ruby gzip.rb | gunzip
foobar

The new compression/decompression APIs introduced in Polyphony have been designed to minimize allocations when compressing or decompressing data between two file descriptors (or IO instances.) To achieve this, the new compression/decompression methods use static stack-allocated buffers. In addition, the compression/decompression methods are tightly integrated with the different backend I/O methods, so no Ruby strings are involved (Polyphony’s backend APIs such as read, recv etc. can read and write data directly to/from raw buffers.)

Splicing and compression

Now, you might be already guessing at what I aim to explore here: can we combine pipes and those new compression APIs in order to create interesting I/O pipelines that minimize both allocations and copying of data?

Suppose we want to be able to send compressed data using some protocol not unlike HTTP/1.1 with chunked transfer encoding. What we need to be able to do is to write a header with the (hexadecimal) chunk length followed by \r\n, then splice the data, again followed by \r\n until we finally signal EOF by writing 0\r\n\r\n (an empty chunk) to the client. How can we integrate data compression into this workflow?

The answer is: Just introduce more pipes. Let’s look at the simplest case of doing chunked transfer using splice, without any compression:

MAX_CHUNK = 1 << 16

# src is the data source as an IO instance
# dest is the client's socket (or maybe an intermediary pipe)
def chunked_transfer(src, dest)
  pipe = Polyphony.pipe
  while true
    len = IO.splice(src, pipe, MAX_CHUNK)
    break if len == 0

    dest << "#{len}\r\n"
    IO.splice(pipe, dest, len)
    dest << "\r\n"
  end
  dest << "0\r\n\r\n"
end

The following diagram shows how the pipeline we created in #chunked_transfer:

+------+    +----+    +-----------+
|source| => |pipe| => |destination|
+------+    +----+    +-----------+

In essence, what splice allows us to do is to read a chunk of data from the source into a kernel buffer (the pipe), then use the result (the number of bytes read) to write a header to the destination, followed by a second splice operation moving data from the kernel buffer to the destination. So we get to control the moving of data between source and desination, without having to copy data back and forth between the kernel and our program.

Using the above method is simple:

def respond_from_io(conn, headers, io)
  conn << format_headers(headers)
  chunked_transfer(io, conn)
end

def respond_with_static_file(conn, req, headers = {})
  path = req.path
  if File.file?(path)
    File.open(path, 'r') { |f| respond_from_io(conn, headers, f) }
  end
end

Now if we also want to gzip the data before passing it to the client (which browsers and users love!) we need to introduce an intermediary pipe in conjunction with IO.gzip:

def respond_from_io_gzipped(conn, headers, io)
  gzipped = Polyphony.pipe
  spin { IO.gzip(io, gzipped); gzipped.close }
  respond_from_io(conn, headers, gzipped)
end

In the above method, we create an additional gzipped pipe, and spin up a separate fiber that will and zip all data from the given io to the gzipped pipe. We then call #respond_from_io, which will perform the chunked transfer from the gzipped pipe.

Here’s a diagram representing how data moves in the above pipeline:

         fiber 1              fiber 2
            +               +---------+
            |               |         |
+------+    v    +-------+  v +----+  v +-----------+
|source| (gzip)> |gzipped| => |pipe| => |destination|
+------+         +-------+    +----+    +-----------+

As you can see, with a pipe, an extra fiber, and a call to IO.gzip we added a substantial new feature to our fledgling pseudo-HTTP server, without having to change our internal interfaces.

So, we see a pattern begin to emerge here: we have different methods that take a source file descriptor (or IO instance) and a destination file descriptor, and we create a pipeline by running them concurrently on separate fibers. Back pressure is handled automatically (by virtue of using pipes as buffers), and we reduce CPU and memory usage to the strict minimum, all while still writing Ruby!

Performance

I don’t mean to disappoint you, but I still don’t have any numbers to show you regarding performance. In the last month I’ve been juggling work, gardening and family affairs, and I didn’t have enough time to do benchmarks. I’ll try to that in the comings weeks and publish the results in a subsequent article. Currently all I have is my intuition, which tells me there’s a lot to be gained from using the new APIs and ideas I’ve exposed in this article. If my intuition proves right, they will make their way to Tipi, the web server which I’ve been working on for a while now.

Future directions

A few weeks ago I wrote about some future directions I’ve been looking at, including a new gem for writing HTTP/2 clients and servers, as well as a new HTTP client library. I hope to be able to make some progress on these as well in the coming month (or two.) Let me know if you have any question about any of this stuff. If you care about my work, please consider becoming a sponsor on Github.

My Open Source Ruby Gems - February 2022 Report

Sat, 05 Mar 2022 00:00:00 GMT

I’m an independent software developer working mostly on Ruby apps. In the line of my work, I’ve developing various open-source projects spanning the gamut from low-level concurrency constructs to database and frontend concerns. If you care about my work, please consider becoming a sponsor on Github.

Here’s a summary of my open-source work in February:

Polyphony - fiber-based concurrency for Ruby

Polyphony got a lot of my attention in the last month, and I spent a lot of time working on mostly low-level details:

Added supported for IPv6 addresses.
Improved handling of process signals.
Improved behaviour of SSL sockets.
Improved error handling in forked processes.
Improved compatibility of drop-in fiber-aware Queue implementation with stock Ruby API.
Overhauled the tracing subsystem for tracing backend events. This feature lets developers track what’s happening at the backend level - scheduling and switching fibers, polling for I/O completion etc. This tracing subsystem will eventually be used to build an interface for monitoring the behaviour of fibers in live Polyphony-based apps, similar to the Erlang observer, or the new tokio-console for Rust.
Refactored, cleaned up and documented a whole lot of code.

The documentation effort is still ongoing, and hopefully I’ll be able to create a new website for the project in the coming months. My plan is to be able to put out a version 1.0 sometime in 2022.

Extralite - A fast Ruby gem for working with SQLite3 databases

Extralite has also been a principal focus of my attention in February. I worked on two big features:

Prepared statements - now users of Extralite can create prepared statements which can then be used multiple times for querying, with the same flexible API that lets you get results in a variety of ways (hash, array, single column, single row, or single value):
```
require 'extralite'

db = Extralite::Database.new('my.db')
stmt = db.prepare('select count(*) from order_items where order_id = :order_id')
stmt.query_single_value(order_id: 1)
stmt.query_single_value(order_id: 2)
```
Prepared statements can provide a nice performance boost (on top of Extralite’s already very fast performance) as they are compiled only once and can then be reused repeatedly without recompiling on each invocation.
A bundled version of Extralite now includes the latest version of SQLite (currently at version 3.38.0). This provides a double advantage: there’s no need to install the sqlite3 lib on your system, and you can use the latest features provided by SQLite. In order to use the bundled Extralite gem, just add gem 'extralite-bundle' instead of gem 'extralite' to your Gemfile. The API stays exactly the same.

Papercraft - composable templating for Ruby

Papercraft also got a lot of attention in the last month:

Added support for XML namespaced tags and attributes.
Added bundled SOAP extension for creating SOAP request/response bodies (contributed by @aemadrid).
Refactored, cleaned up and documented the entire code base.

Papercraft is used to generate the web page you’re currently reading: the page content is in [Markdown], the article layout is a Papercraft template, derived from the default layout. The RSS feed for this website is also implemented as a Papercraft template.

I’m looking for contributors willing to integrate Papercraft into Rails. Let me know if you’re interested!

Impression - a web framework (WIP)

Impression is a relatively recent project I put together for building the present website (code here). It’s still more of a rough sketch than anything serious, but it presents a novel way (I believe) to look at web apps.

Instead of the standard MVC pattern (and related patterns as well,) Impression is based around a single entity - the resource. A resource lives in a tree structure (rooted in a root resource). Different kinds of resources provide different functionalities. For example, there’s a resource for serving static files, there’s a resource for running a Rack app, there’s a resource for serving Jamstack-like websites, etc.

An Impression-based application is then simply a tree of resources. For example:


* /               => Impression::App
|
+-* /static       => Impression::FileTree
|
+-* /api
  |
  +-* /api/users  => Impression::RestfulAPI
  |
  +-* /api/orders => Impression::RestfulAPI

Each resource above is mounted at a specific location in the URL namespace. Incoming HTTP requests are first routed to the corresponding resource, which then handles the request and generates a response.

So the idea is to be able to build an app out of those different resource types, and for each resource to be designed to fit the specific functionality required. This design seems to me to be both simple and flexible.

The resource used for the present website is called Impression::App. It resembles a Jamstack app in that it renders any static assets in the given file directory, but it can also render markdown files with layouts, and dynamically load resource modules written in Ruby. For example, the Noteflakes website has a /ping endpoint, which simply responds with a pong. Here’s the source code for the endpoint:

# In this case, the endpoint is a basic resource with a custom response block.
export_default Impression.resource { |req|
  req.respond('pong')
}

That way, an Impression application can be composed of a bunch of dynamically loaded resources that are “mounted” according to their location in the app’s file tree. In the future, Impression apps will also be able to automatically reload resource files that have been updated. I hope to be able to write more about impression in due time.

H1P - HTTP/1 parser for Ruby

H1P got the ability to parse HTTP/1 responses as well as requests, so now you can also use it to implement HTTP clients. Here’s a basic example:

require 'socket'
require 'h1p'

conn = TCPSocket.new('ipinfo.io', 80)
parser = H1P::Parser.new(conn, :client)

# send request
conn << "GET / HTTP/1.1\r\nHost: ipinfo.io\r\n\r\n"

# read response
headers = parser.parse_headers
body = parser.read_body

conn.close
p headers
p body

Notice how the code above doesn’t need to deal with reading or buffering incoming data. It’s automatically handled by the H1P parser, and what you get in the end is a hash with the response headers, and of course the response body. It’s that simple! For more information, go to the H1P project page.

Perspectives for this month

In the month of March, I’ll be concentrating on the following areas:

Polyphony: I’ve written in the past about being able to express I/O operations in terms of a composition of steps. Lately I’ve been thinking about how that relates to being able to do data compression on the fly. One use case for that would be sending and receving gzip’ed or deflated HTTP responses.

As I always start with designing the API, I came up with this:
```
module ResponseExtensions
  def serve_io_gzipped(io)
    r, w = IO.pipe
    spin do
      io.gzip(w)
      w.close
    end
    serve_io(r)
  end
end
```
The idea here is to maximize performance and minimize allocations and GC pressure by doing as much of the work using pipes, which are basically kernel buffers. We do need to spin up a fiber but that’s the only cost, and thanks to Polyphony’s innovative design, we get both optimized I/O scheduling and automatic back-pressure.

I’ll dedicate a future article to how this mechanism once it is implemented.

H2P: This is a new gem for implementing HTTP/2 servers and clients. The idea is to provide the same kind blocking API design that underlies H1P (discussed above). Here’s an example of how an HTTP/2 server implementation might look like:

def handle_incoming_http2_connection(conn, &handler)
  h2p = H2P::Peer.new(conn, :server)
  h2p.each do |stream_id, headers|
    # each stream gets its own fiber
    handle_http2_stream(h2p, stream_id, headers, &handler)
  end
end

def handle_http2_stream(h2p, stream_id, headers, &handler)
  spin do
    stream_adapter = H2StreamAdapter.new(h2p, stream_id)
    req = Qeweney::Request.new(headers, stream_adapter)
    handler.(req)
  end
end

class H2StreamAdapter
  ...

  def respond(request, body, headers)
    @h2p.send_headers(@stream_id, headers, done: false)
    @h2p.send_data(@stream_id, body, done: true)
  end
end

Inuit - this is another new gem that I’m currently designing, meant to be a Polyphony-based HTTP client with the following features:
- Support for HTTP/1, HTTP/2, WebSocket.
- Support for HTTPS, WSS.
- Persistent connections, connection pooling.
- Support for sessions and cookies.

Please let me know if my work interests you. You can delve into my code here.

Papercraft - Composable Templating for Ruby

Fri, 04 Feb 2022 00:00:00 GMT

Papercraft is a new Ruby gem I’ve been working on, that provides a new way to render HTML, XML and JSON using plain Ruby. Here’s what it looks like:

require 'papercraft'

template = Papercraft.html {
  html5 {
    head {
      title 'Some title'
    }
    body {
      emit_yield
    }
  }
}

template.render('Page title') { h1 'Hello, world!' }
#=> "<html><head><title>Some title</title></head><body><h1>Hello, world!</h1></body></html>"

Of course, a Ruby DSL for constructing HTML, XML or JSON is nothing new. Here’s a list of previous projects that already do that:

All of the above offer some variation on the same theme: you construct HTML using nested Ruby blocks, e.g.:

html {
  body {
    div {
      p 'foo'
      p 'bar'
    }
  }
}

I’ve been a long time admirer of Ruby procs. To me, procs are perhaps the single most unique feature of Ruby, especially in their block form:

items.sort_by { |item| item.path }.each { |item| do_something_with(item) }

When you call a method with a block, you are in injecting a piece of your own code into some other code, and this lets you compose different pieces of code in a variety of ways. The way Ruby blocks are integrated into the language is perhaps the defining feature of Ruby, and to me it’s one of the main reasons Ruby makes developers happy.

I’ve been playing for a while now with all kinds of ideas for constructing DSLs using blocks. One project I created a while ago was Rubyoshka, a Ruby gem that lets you write HTML using plain Ruby syntax. The name was a nod to the Matryoshka, the Russian nesting doll.

I’ve recently came up with a way to make it even better, and have decided to give this library a new name, henceforth Papercraft. Papercraft is unique in how it embraces Ruby procs, and how it enables developers to express the different parts of an HTML page (or XML and JSON documents) using procs on one hand, and to compose those different parts in a variety of ways on the other hand.

Ruby procs as templates

The idea behind Papercraft is simple: a template is a Proc that can be rendered by executing it in the context of a special-purpose rendering object, using #instance_exec.

Ruby procs can take positional and/or named parameters, which lets us explicitly pass dynamic values to templates:

foobar = ->(foo, bar) {
  h1 foo
  p bar ? 'hi' : 'bye'
}

The above proc (strictly speaking it’s a lambda, which is a special kind of proc) takes two parameters, foo and bar, which affect its rendered output.

Ruby procs, being first-class objects, can also be composed and derived. For example, we can take the above foobar proc and use it in the context of a list template:

foobar_list = ->(list) {
  list.each { |i|
    emit foobar, i.foo, i.bar
  }
}

We could also create a derivative of foobar where foo is applied:

applied_foobar = ->(bar) { emit foobar, 'hi', bar }

(Note the use of #emit in the last two example, more on that below.)

So, once templates are expressed as procs, they can be composed, combined and transformed in a variety of ways, including all the ways proc objects can be manipulated. In fact, the primary template class in Papercraft, Papercraft::Template, is a subclass of Proc, which means it can be exchanged for a proc at any place.

Emitting HTML

Let’s start with the basics: Papercraft templates produce HTML by emitting tags. This is done primarily by making method calls with the tagname, followed by the tag content and any tag attributes. Here are some examples:

p 'foo' #=> <p>foo</p>
a 'bar', href: 'https://example.com/' #=> <a href="https://example.com">foo</a>
h1 'Title', id: 'title' #=> <h1 id="title">Title</h1>

Tags are nested by using blocks:

div(id: 'foo') {
  div(id: 'bar') {
    p 'hi'
  }
} #=> <div id="foo"><div id="bar"><p>hi</p></div></div>

In order to render text and have it escaped correctly, you can use the #text method:

p {
  span 'pre'
  text ' foo & bar '
  span 'post'
} #=> <p><span>pre</span> foo &amp; bar <span>post</span></p>

Other content (such as raw text, nested templates, markdown etc.) can be rendered by using the #emit method:

emit '<h1>hi</h1>' #=> <h1>hi</h1>

foo = -> { p 'foo' }
emit foo #=> <p>foo</p>

emit markdown('## Hi') #=> <h2 id="hi">Hi</h2>

(Papercraft includes built-in support for rendering Markdown. More on that below.)

Explicit template parameters

The most important difference between Papercraft and all of its predecessors is the fact that in Papercraft, any variables referenced in the template logic should be passed explicitly to the template when it is rendered:

greeter = Papercraft.html { |name:| h1 "Hello, #{name}!" }

greeter.render(name: 'world') #=> "<h1>Hello, world!</h1>"

In the above example, we create a template that takes a single named argument, name:. When we want to render the template, we need to supply the a name: parameter, which is then injected by the template code into the resulting HTML.

This way of injecting data into templates offers multiple advantages: the data flow is much clearer - since you’re not implicitly reyling on variables that just happen to be in your template’s binding, and debugging is easier - if you forget to provide the name parameter, Ruby will tell you!

Parameter application

Papercraft takes this idea even further by letting you create a derivative template by applying parameters to the source template, using the #apply method. You can select to do a full or partial application:

greeter = Papercraft.html { |greeting:, name:| h1 "#{greeting}, #{name}!" }
goodbyer = greeter.apply(greeting: 'Goodbye')

goodbyer.render(name: 'world') #=> "<h1>Goodbye, world!</h1>"

In the above example, we take our greeter and make it a bit more general - it now also takes a greeting: argument. We then create a derivative template by partially applying greeter with just the greeting: parameter filled in. We then render the goodbyer template, passing in the missing name: parameter.

Block application

The idea of application can be taken further with the use of applied blocks. Here’s how you do derivative layouts in Papercraft:

layout = Papercraft.html { |**props|
  html {
    head {
      title props[:title]
    }
    body {
      emit_yield **props
    }
  }
}

article_layout = layout.apply { |title:, markdown_content:|
  article {
    h1 title
    emit_markdown markdown_content
  }
}

article_layout.render(title: 'Foo', markdown_content: '## Bar')

In the above example, we first create a layout. This layout creates a generic HTML structure, with the body section containing a call to emit_yield, which expects a block to be injected. This can be done either using #apply or #render and passing a block. We then create a derivative layout that applies a block to be emitted inside the body section.

Notice how parameters passed to the layout template are explicitly passed along to the applied block (in the call to #emit_yield), and how they are destructured in the block given to layout.apply. Finally, we can render the article_layout to HTML by calling #render with the needed parameters.

Template composition

Using #emit_yield is not the only way to pass in arbitrary blocks to a template. In fact, you can pass any number of template blocks as parameters into your template, and then use #emit to emit them. Here’s another way layout templates can be created with Papercraft:

layout = Papercraft.html { |header:, content:, footer:|
  html {
    body {
      emit header
      emit content
      emit footer
    }
  }
}

layout.render(
  header:   -> { header { p 'some header' } },
  content:  -> { content { h1 'Some content' } },
  footer:   -> { footer { p 'some footer' } }
)

Higher order templates

We can create higher order templates by writing methods (or procs) that take a template as input and return a composed template as output, normally :

div_wrap = ->(templ) {
  div {
    emit templ
  }
}

layout = Papercraft.html { |content|
  body {
    emit div_wrap.call(content)
  }
}

layout.render(-> { h1 'hi' })
#=> "<body><div><h1>hi</h1></div></body>"

In the above example, we create a higher order template called div_wrap. It takes as an input a given template, and returns as its output a template wrapping the original template with a div element.

Rendering Markdown

Papercraft has built-in support for rendering Markdown. Markdown can be converted to HTML by calling the #markdown method:

markdown '## Hi' #=> "<h2 id=\"id\">Hi</h2>"

In HTML templates you can also use the #emit_markdown convenience method:

emit_markdown '## Hi' #=> <h2 id="hi">Hi</h2>

Papercraft uses Kramdown and rouge(https://github.com/rouge-ruby/rouge) to convert Markdown to HTML with support for syntax-highlighted code blocks. The default Markdown options can be overriden by passing them to #markdown or #emit_markdown:

# render markdown without heading ids
emit_markdown '## Hi', auto_ids: false #=> <h2>Hi</h2>

XML and JSON templates

Papercraft XML templates work just like HTML templates:

Papercraft.xml { |movies|
  movies.each { |m|
    movie {
      title     m.title
      year      m.year
      director  m.director
    }
  }
}

And here’s how a JSON template can be generated:

Papercraft.json { |movies|
  movies.each { |m|
    item {
      title     m.title
      year      m.year
      director  m.director
    }
  }
}

Extending Papercraft

Since the main goal of Papercraft is to allow developers to produce dynamic HTML, XML or JSON with the least amount of code, it also includes the possibility of creating extensions that provide a convenient API for creating complex HTML components. This might be particularly useful when using design frameworks such as Bootstrap or Tailwind, where some components demand quite complex markup. Here’s an example of how a Bootstrap extension might look like:

module BootstrapComponents
  ...

  def card(**props)
    div(class: 'card', **props) {
      div(class: 'card-body') {
        emit_yield **props
      }
    }
  end

  def card_title(title)
    h5 title, class: 'card-title'
  end

  ...
end

Papercraft.extension(bootstrap: BootstrapComponents)

my_card = Papercraft.html {
  bootstrap.card(style: 'width: 18rem') {
    bootstrap.card_title 'Card title'
    bootstrap.card_subtitle 'Card subtitle'
    bootstrap.card_text 'Some quick example text.'
    bootstrap.card_link '#', 'Card link'
    bootstrap.card_link '#', 'Another link'
    ...
  }
}

How Papercraft is Used by this website

Papercraft is used by this website, which is, just so you know, not a static website. While the source layout of this website largely resembles a Jamstack website, it is rendered dynamically by Impression, a web framework I’m currently developing for my own use, which in turn relies heavily on Papercraft for dealing with layouts.

So let’s explore how this website is constructed. The file for the present article is just a markdown file with some YAML front matter (permalink):

---
title: Papercraft - Composable Templating for Ruby
layout: article
---

Papercraft is a new Ruby gem I've been working on, that provides a new way to
render HTML, XML and JSON using plain Ruby. Here's what it looks like:

...

Here’s the layout used for rendering articles (permalink):

default = import './default'

export_default default.apply { |title:, date:, **props|
  article {
    h1 title
    h3 date.strftime('%d·%m·%Y'), class: 'date'
    emit_yield
  }
}

The article layout above imports the default layout (using Modulation), and uses #apply to create a derivative layout that adds an article element and expects a nested block to be injected into it.

And here is the default layout (permalink):

require 'papercraft'

export_default Papercraft.html { |**props|
  html5 {
    head {
      title(props[:title] ? "Noteflakes - #{props[:title]}" : "Noteflakes")
      meta charset: 'utf-8'
      meta name: 'viewport', content: 'width=device-width, initial-scale=1.0'
      style 'body { display: none }' # prevent FUOC
      link rel: 'icon', type: 'image/png', href: '/assets/nf-icon-black.png'
      link rel: 'stylesheet', type: 'text/css', href: '/assets/style.css'
      link rel: 'alternate', type: 'application/rss+xml', href: '/feeds/rss'
    }
    body {
      header {
        h1 {
          a(href: '/') {
            img src: '/assets/nf-icon-black.png'
            span 'noteflakes'
          }
        }
        ul {
          li 'by Sharon Rosner', class: 'byline'
          li { a 'archive', href: '/archive' }
          li { a 'about', href: '/about' }
          li { a 'RSS feed', href: '/feeds/rss' }
          li { a 'code', href: 'https://github.com/noteflakes', target: '_blank' }
        }
      }
      emit_yield **props
      footer {
        hr
        p {
          span 'Copyright © 2021 Sharon Rosner. This site runs on '
          a 'Impression', href: 'https://github.com/digital-fabric/impression'
          span ' and '
          a 'Tipi', href: 'https://github.com/digital-fabric/tipi'
          span '.'
        }
      }
    }
  }
}

Finally, Impression takes the layout referenced in the article’s front matter, and renders it by passing a block that renders the markdown:

def render_markdown_file(req, path_info)
  layout = get_layout(path_info[:layout])

  html = layout.render(request: req, resource: self, **path_info) {
    emit path_info[:html_content]
  }
  req.respond(html, 'Content-Type' => layout.mime_type)
end

Other templates on this website produce RSS and a JSON feed. Here’s the RSS template (permalink):

require 'papercraft'

export_default Papercraft.xml(mime_type: 'text/xml; charset=utf-8') { |resource:, **props|
  rss(version: '2.0', 'xmlns:atom' => 'http://www.w3.org/2005/Atom') {
    channel {
      title 'Noteflakes'
      link 'https://noteflakes.com/'
      description 'A website by Sharon Rosner'
      language 'en-us'
      pubDate Time.now.httpdate
      emit '<atom:link href="https://noteflakes.com/feeds/rss" rel="self" type="application/rss+xml" />'

      article_entries = resource.page_list('/articles').reverse

      article_entries.each { |e|
        item {
          title e[:title]
          link "https://noteflakes.com#{e[:url]}"
          guid "https://noteflakes.com#{e[:url]}"
          pubDate e[:date].to_time.httpdate
          description e[:html_content]
        }
      }
    }
  }
}

A note on Papercraft’s design

You will have noticed that Papercraft’s DSL looks quite terse, and that is because all of its API consists basically of unqualified method calls that look like html { body { h1 'foo' } }. This might look confusing to the young Ruby padawan, as they might ask “where do all these calls go, and how are they intercepted?”

The answer is that all template procs are evaulated in the context of a Papercraft renderer instance, which intercepts all calls and turns them into chunks of HTML/XML that are added to an internal buffer (the JSON renderer works in a slightly different manner).

On one hand, this allows us to write templates with a minimum of boilerplate, and have templates that look very clean. On the other hand, it does prevents us notably from using instance variables, e.g. @foo in our templates, and doesn’t let us use any methods in the scope of the receiver where we create our template. For example, the following code will fail to produce the desired result:

class Foo
  def to_html
    Papercraft.html { h1 bar }
  end

  def bar
    'bar'
  end
end

Foo.new.to_html #=> not what you might expect...

So, when writing Papercraft templates we need to follow a few rules:

Any variables or data referenced inside a template must be provided to the template as an explicit argument.
Any method calls that are not expected to emit HTML should be qualified, i.e. the receiver should be referenced explicitly, e.g.: receiver.foo.
No instance variables (or class variables, for that matter) should be used in Papercraft templates.

Conclusion

Papercraft is a new Ruby gem that lets you dynamically generate HTML, XML and JSON documents using plain Ruby. Papercraft templates use explicit parameter passing in order to “bind” template variables, and use application and composition to combine templates in a variety of ways. This website is the first to use Papercraft “in production”, and I hope other people will find it useful.

To learn more about Papercraft, checkout the API documentation. Contributions in the form of issues or pull requests will be gladly accepted on the Papercraft repository.

Extralite - a new Ruby gem for working with SQLite databases

Wed, 15 Dec 2021 00:00:00 GMT

In the last year I’ve been working a lot with SQLite databases. I started by using the popular sqlite3-ruby Ruby gem, but quickly noticed that for my usage there were a few things missing in the gem’s API. Being a tinkerer, and having had some experience writing C-extensions, I had a look at the SQLite C API and decided to try to write my own Ruby bindings for SQLite. Thus Extralite was born.

Extralite is an extra-lightweight (less than 460 lines of C-code) SQLite3 wrapper for Ruby. It provides a single class with a [minimal set of methods] for interacting with an SQLite3 database. Extralite provides the following improvements over the sqlite3-ruby gem:

Improved concurrency for multithreaded apps: the Ruby GVL is released while preparing SQL statements and while iterating over results.
Super fast - up to 13x faster than sqlite3-ruby.
Automatically execute SQL strings containing multiple semicolon-separated queries (handy for creating/modifying schemas).
Access data in a variety of ways: rows as hashes, rows as arrays, single row, single column, single value.

Concurrency

One of the most important limitations of the sqlite3-ruby gem is that it doesn’t release the GVL while running queries and fetching rows from the database. This means that if any query takes a significant amount of time to execute, other threads will also be blocked while the query is running.

There has been some discussion on the sqlite3-ruby repository as to why this is. Basically, since developers can define their own SQLite functions, aggregates and collations using Ruby, the gem needs to hold on to the GVL while running queries, since those might call back into Ruby code.

Extralite does release the GVL when running queries, which makes it much more friendly to multithreaded code. While Extralite is busy fetching a row, other threads can continue running. If your program has multiple threads accessing SQLite databases at the same time, you’ll get much better usage out of your multicore machine.

Performance

Preliminary benchmarks show Extralite to be significantly faster than sqlite3-ruby. The benchmark included in the Extralite repository creates a database with varying number of rows, then meausres the time it takes for sqlite3-ruby and Extralite to fetch those rows. The performance advantage becomes more pronounced as the number of rows is increased:

Row count	sqlite3-ruby	Extralite	Relative
10	56.48K rows/s	91.52K rows/s	1.62x
1K	256.3K rows/s	1758K rows/s	6.87x
100K	176.5K rows/s	2323.6K rows/s	13.17x

These results surprised me quite a bit, since Extralite does release the GVL on each fetched row, but I guess this is more than made up for by the fact that Extralite makes a minimum of allocations and offers a significantly smaller API surface area, compared with sqlite3-ruby.

Other features

Extralite provides a variety of ways to get query results: rows as hashes, rows as arrays, a single column, a single row, or a single value. While sqlite3-ruby has most of these (except for iterating over a single column), you need to set a mode (SQLite3::Database#results_as_hash) if you want to fetch rows as hashes.

Another important feature of Extralite is that it automatically executes SQL strings containing multiple SQL statements (separated with a semicolon.) In sqlite3-ruby you need to use a separate API (#execute_batch) in order to do that.

Other features, such as binding indexed and named parameters, getting the last inserted rowid, getting the number of changes made in the last query or loading extensions, are available in both gems.

What’s missing

Extralite is notably missing the ability to define custom functions, aggregates and collations using Ruby. If you rely on that feature, you’ll need to use sqlite3-ruby.

Usage with ORMs

Extralite includes an adapter for Sequel. If you wish to switch from sqlite3-ruby to Extralite, you can just add extralite to your Gemfile, and then change your database URLs to use the extralite schema instead of sqlite:

DB = Sequel.connect('extralite://my.db')

What about ActiveRecord? Well, I tried, but after spending a few hours looking at the ActiveRecord SQLite adapter code and trying to make sense out of that, all I got was cryptic error messages. I finally decided to abandon the effort. If you have experience writing ActiveRecord database adapters, I’ll greatly appreciate your contribution. Let me know on the Extralite repository.

Future directions

Extralite is pretty much feature-complete as far as I’m concerned, apart from missing an ActiveRecord adapter. I’m currently thinking about how to adapt the different abstractions I came up with while working with SQLite databases, but those will be published in a separate project.

In the meanwhile, if you have suggestions for improving Extralite, or wish to contribute, please let me know. I’ll gladly accept issues and PRs! The documentation for the Extralite gem can be found here.

Qeweney - a feature-rich HTTP request/response API for Ruby

Fri, 03 Dec 2021 00:00:00 GMT

As you may know, in the last few months I’ve been working on Tipi, a new web server for Ruby, with innovative features such as support for HTTP/2, automatic SSL certificates, and streaming request and response bodies. As part of the development process, I also had to deal with how to represent HTTP requests and responses internally.

Tipi being a modern web server, with emphasis on concurrency, performance, and streaming, I felt it was wrong to base it on the Rack interface. While Rack is today ubiquitous in the Ruby ecosystem—it underlies basically all Ruby web frameworks and all Ruby app servers—it has some important limitations, especially as regards being able to perform HTTP upgrades and the streaming of request and response bodies.

Instead, the interface I came up with is a single Request class, with an imperative API design. I have extracted this interface into a separate gem, for reasons that I’ll discuss below. I’m calling this interface Qeweney (pronounced “Q ‘n’ A”). As we’ll see, Qeweney can be used inside of a Rack app, and can also be used to drive a Rack app.

The Request interface

Qeweney exposes a single class, Request. An instance of Request is initialized with headers and an adapter. The adapter is the party actually responsible for the client connection. Tipi implements separate adapters for HTTP/1 and HTTP/2 connections. In the case of HTTP/2, the actual request adapter represents a single HTTP/2 stream, backed by an HTTP/2 connection adapter, as shown in the following diagram:

    +------+    +-------+                   +-------+    +-----+
    |Client|--->|HTTP/1 |------------------>|Request|--->|     |
    |      |<---|adapter|<------------------|       |<---|     |
    +------+    +-------+                   +-------+    |     |
                                                         |     |
    +------+    +-------+    +---------+    +-------+    |     |
    |Client|--->|       |--->|H2 stream|--->|Request|--->| Web |
    |      |<---|       |<---|adapter  |<---|       |<---| app |
    +------+    |HTTP/2 |    +---------+    +-------+    |     |
                |adapter|        ...                     |     |
                |       |    +---------+    +-------+    |     |
                |       |--->|H2 stream|--->|Request|--->|     |
                |       |----|adapter  |<---|       |<---|     |
                +-------+    +---------+    +-------+    +-----+

This design provides separation between the HTTP semantic layer (the representation of different parts of HTTP requests and responses,) and the HTTP transport layer, which changes according to the HTTP protocol version. In addition, this design facilitates the testing of HTTP requests and responses, for example with a mock request adapter. It also allows the implementation of other transport methods, such as HTTP/3, or other custom transport protocols for implementing custom HTTP proxies (I already have some ideas I’ll write about in the future.)

The Request API includes methods for inspecting the request headers, reading the request body, and finally responding to the request. The most important differentiator between the Qeweney API and the Rack interface is that with Qeweney the response is generated imperatively, by using one of the response APIs, while in Rack the response is the return value of an app function that takes a request as argument. Let’s compare the two approaches:

# Rack
app = ->(env) do
  [
    200,
    {'Content-Type' => 'text/plain'},
    ['Hello, world!']
  ]
}

# Qeweney
app = ->(req) do
  req.respond('Hello, world!', 'Content-Type' => 'text/plain')
end

As shown above, a Rack app is a Proc that takes a Hash as its env argument (containing information about the request and the server) and returns an Array consisting of the HTTP status code, response headers and the response body reprersented as an array.

In contrast, a Qeweney-based app takes a Qeweney::Request as its argument, and it is the app’s responsibility to respond to the request by explicitly calling one of the response methods, such as #respond, #send_headers, #send_chunk and #finish.

Request information

One of the most crucial design decisions I made early on with Tipi was to allow handling incoming requests without first reading and parsing the request body. In Tipi, as soon as all request headers have been received, the request is created and passed to the app. This allows an app to reject invalid or malicious requests before the request body is read. It also allows Ruby apps to apply backpressure, as they can control reading the request body.

With that in mind, let’s examine how the request information is actually represented in Qeweney::Request. As we saw, a request is instantiated by providing it with an adapter instance, and a hash containing the request headers. The header keys are represented by convention using lower case, and also include the following meta-headers:

:method - the HTTP method used (represented as an upper-case string.)
:path - the path specified in the HTTP request line (including query parameters.)
:scheme - the protocol scheme used, i.e. http or https.

It is important to note that these conventions concerning request headers should be adhered to by any request adapter. Finally those are the conventions that allow apps to behave identically for both HTTP/1 and HTTP/2 connections.

In order to facilitate the processing of request headers, Qeweney includes a module named RequestInfoMethods into the Request class. These method help us deal with such things as URL queries and cookies.

Here’s an example of how these methods can be used:

app = ->(req) do
  session = find_session(req.cookies['session-id'])
  validate_csrf_token(session, req.headers['x-csrf_token'])

  page_no = (req.query('page') || 1).to_i
  serve_page(req, page_no)
end

Request body

Once the app is ready to read the request body, it can use Request#read for reading the entire request body, or Request#next_chunk and Request#each_chunk to read the request body in chunks.

Here’s an example of how #each_chunk can be used to create a server that responds with the request body converted to upper case:

upper_caser = ->(req) do
  req.send_headers
  req.each_chunk { |c| req.send_chunk(c.upcase) }
  req.finish
end

In order to facilitate the handling of forms, Qeweney also includes the Request.parse_form_data method which can be used to handle form submission:

form_handler = ->(req) do
  form = Qeweney:::Request.parse_form_data(req.read, req.headers)
  submit_form(form)
  req.respond(nil, ':status' => Qeweney::Status::CREATED)
end

Responding

When it’s time to send out a response, Qeweney::Request provides the following methods:

#respond(body, headers) - send out a complete response
#send_headers(headers) - send out headers, without finishing the response
#send_chunk(chunk, done: false) - send a body chunk, possibly finishing the response
#finish() - finish the response (when not using respond)

These APIs allow the app to either send the body all at one, or as shown in the upper_caser example above, chunk by chunk. Note that Tipi’s HTTP/1 adapter will use chunked encoding by default, which lets apps send out streaming responses easily. By being able to send a response chunk by chunk, apps are able to generate large responses without having to buffer them in memory, thus lowering pressure on the Ruby GC, and the app’s memory footprint in general.

In a similar manner to the way Qeweney provides additional methods for processing request headers, it also provides response methods (defined in the Qeweney::ResponseMethods module) for generating responses more easily. Here are some of them:

# redirect to another URL
req.redirect(alternative_url)

# redirect to HTTPS
req.redirect_to_https

# serve a static file
req.serve_file(file_path)

# serve from an IO
req.serve_io(io)

# serve from a Rack app
req.serve_rack(app)

# Upgrade to arbitrary protocol
req.upgrade(protocol)

Now these methods may seem almost banal (and they mostly are,) but #serve_file method merits closer scrutiny, as it actually packs a lot of power behind its modest method signature. It sets cache headers, validates If-None-Match and If-Modified-Since headers included in the request, and responds with a 304 Not Modified status code if the client’s cache is valid. It also serves the file compressed according to the Accept-Encoding request header, using either the deflate or gzip algorithms.

Tipi further turbocharges the functionality provided by Qeweney using optimized, protocol-specific methods for serving arbitrary responses from an IO object (Request#serve_file uses #serve_io under the hood). Tipi’s HTTP/1 adapter is capable of sending requests from an IO by splicing them to the client’s socket connection, achieving up to 64% better throughput for large files.

Request routing

Qeweney also includes a basic API for routing requests without needing a web framework, or rolling your own. The Qeweney routing is largely inspired by Roda, and in fact uses the same mechanism (using catch/throw) behind the scenes. Here’s a simple demonstration of what Qeweney’s router is capable of:

app = ->(r) do
  r.route do
    r.on_root { r.redirect '/hello' }
    r.on('hello') do
      r.on_get('world') { r.respond 'Hello world' }
      r.on_get { r.respond 'Hello' }
      r.on_post do
        r.redirect '/'
      end
      r.default { r.respond nil, ':status' => 404 }
    end
  end
end

The Qeweney router is by no way comprehensive (it probably doesn’t have half the features of Roda,) nor does it have the best routing DSL, at least as far as I’m concerned, but it offers a solid starting point for implementing web apps on top of Qeweney, without needing to use any web framework.

Extending Qeweney

Qeweney’s API largely consists of a single class, Qeweney::Request, which deals with all aspects of processing HTTP requests and responses, from the point of view of the server. The different functionalities—request information, responding, and routing—are implemented in separate modules that are then included into the Request class.

This design choice means that you don’t need to deal with different objects and different classes, everything related to requests and responses available directly on the Request instance given to your web app. The downside is that the Request class has a big surface area with dozens of methods, and will most probably not satisfy OOP purists who demand no more than 10 methods per class, and a single area of responsibility for each class. YMMV.

The point I want to make here, is that to me, adding request/response functionality in this manner, by extending the Request class, makes a lot more sense, especially when you want to add middleware to your app, which brings us to:

Qeweney and Middleware

One of the defining features of Rack is the use of middleware - small pieces of specific HTTP functionality that you can plug into your application, without making any change to your app’s logic. This is due to Rack being a functional interface (you feed a request in, and you get a response out.) All you need to do to plug a middleware into your app is to call the middleware instead of your app, and have the middleware call your app before or after it has done its business. That way, a middleware can manipulate both the env parameter passed to your app, as well as the response your app has generated, before it is being returned to the app server.

In fact, you can put a whole bunch of middleware components in a pipe line, with each of them performing specific tasks before and after your app has dealt with the request.

As we saw above, Qeweney does not use a functional approach, but rather an imperative one. Once the app has sent its response (by calling methods on the given request,) there’s no way to change it post factum.

So how can we use middleware under such circumstances? We do not necessarily want to monkey-patch Qeweney::Request in order to provide specific functionality, such as logging, validating request headers or setting various response headers. In fact, it would be undesirable to change the behaviour of the class, when all we need is to apply custom behaviour for specific instances of the class. For example, we might want to apply CSRF protection only to some requests but not to others. We might want to measure response latency for some requests but not others.

Luckily, Ruby lets us do just that by letting us extend object instances with methods from arbitray modules. Let’s see how this technique can be applied to creating middleware in Qeweney:

module JSONContentTypeExtensions
  # overrides Qeweney::Request#respond
  def respond(body, headers = {})
    headers = headers.merge('Content-Type' => 'application/json')
    super(body, headers)
  end
end

def with_json_content_type(app)
  ->(req) do
    req.extend(JSONContentTypeExtensions)
    app.(req)
  end
end

payload = { 'message': 'Hello world!' }
app = ->(req) { req.respond(payload.to_json) }

# plug middleware in front of app
app = with_json_content_type(app)

The above example shows how a module modifying the stock behaviour of Qeweney::Request#respond can be used to extend a specific instance of the class. Note how we call super in the patched method. The with_json_content_type builds the middleware by taking an app and returning a modified Proc that first extends the request with the custom behaviour, then passes it on to the app.

Here’s another example that shows how we can implement a logging middleware:

def with_logger(app, log)
  m = Module.new do
    define_method(:respond) do |body, headers = {}|
      start = Time.now
      super(body, headers)
      elapsed = Time.now - start
      log.info "Got #{self.headers}, respond with #{headers.inspect} (#{elapsed}s)"
    end
  end
  ->(req) do
    app.(req.extend(m))
  end
end

This example is quite dense, so let’s analyze what it does. Since we want to be able to inject a log instance into the middleware, we need to dynamically override the #repsond method with a closure, in order to be able to access log from within the method. Once that has been done, with_logger returns a Proc that first extends the request with the dynamically defined module, then calls the app.

Finally, in order to make it easier to write middleware for Qeweney, it will eventually include a simple DSL for writing middleware. It will probably look something like the following:

with_logger = Qeweney.middleware do |log|
  respond do |body, headers = {}|
    start = Time.now
    super(body, headers)
    elapsed = Time.now - start
    logger.info "Responded with #{headers.inspect} (#{elapsed}s)"
  end
end

with_json_content_type = Qeweney.middleware do
  respond do |body, headers = {}|
    headers = headers.merge('Content-Type' => 'application/json')
    super(body, headers)
  end
end

payload = { 'message': 'Hello world!' }
app = ->(req) { req.respond(payload.to_json) }

# plug middleware
app = with_logger(with_json_content_type(app))

What about Rack?

Last, but not least, I’d like to discuss how Qeweney fits in with a world where practically all Ruby web apps are based on Rack. Qeweney can be used both as a driver for Rack apps (e.g. if you want to run your Rack app on Tipi,) and to run Qeweney apps on top of Rack servers such as Puma or Falcon.

In order to run your Rack app using Tipi, you don’t need to do anything. Just provide run Tipi with the path of your rack app, e.g.: tipi myapp.ru, and Tipi will take care of doing the translation.

If you prefer to develop your next web app using the Qeweney interface, it’s easy to convert it to a Rack app. Here’s how:

require 'qeweney'

my_qeweney_app = ->(req) do
  req.respond('Hello world!')
end

run Qeweney.rack(&my_qeweney_app)

Conclusion

This has been an introduction to Qeweney, a new HTTP request/reponse interface for Ruby web apps and servers. Its design was driven by the need for concurrency, performance, and streaming capabilities that are currrently lacking in Rack. While most developers using Tipi and the suite of tools I’m currently developing, will not interact directly with Qeweney, I wanted to give you an overview of its API and show some of its capabilities. If this work interests you, please let me know by contacting me. We can also hook up on GitHub.

Signal handling in concurrent apps with Ruby and Polyphony

Tue, 23 Nov 2021 00:00:00 GMT

In the last few weeks I’ve been writing about different aspects of Polyphony, a library for writing fiber-based concurrent apps in Ruby. Polyphony makes it easy for developers to use stock Ruby core and stdlib classes and APIs in a highly-concurrent environment in order to create scalable, high-performance apps.

In order for provide a solid developer experience, Polyphony reimplements different parts of the Ruby runtime functionality, which are adjusted so developers will see a consistent and reliable behaviour. In this article I’ll discuss how Polyphony implements signal handling. For the sake of brevity, I’ll assume the reader is familiar with POSIX signals and has some knowledge of how signals are handled in Ruby.

What happens when a signal is trapped

In order to get a clear picture of how signal traps work, here’s the relevant passage from the Linux sigreturn manpage:

If the Linux kernel determines that an unblocked signal is pending for a process, then, at the next transition back to user mode in that process (e.g., upon return from a system call or when the process is rescheduled onto the CPU), it creates a new frame on the user-space stack where it saves various pieces of process context (processor status word, registers, signal mask, and signal stack settings).

The kernel also arranges that, during the transition back to user mode, the signal handler is called, and that, upon return from the handler, control passes to a piece of user-space code commonly called the “signal trampoline”. The signal trampoline code in turn calls sigreturn().

… Using the information that was earlier saved on the user-space stack sigreturn() restores the process’s signal mask, switches stacks, and restores the process’s context (processor flags and registers, including the stack pointer and instruction pointer), so that the process resumes execution at the point where it was interrupted by the signal.

From the point of view of the developer, when a signal occurs, it’s as if control of your program has been momentarily hijacked and a special bit of code－the signal handler block－is executed. Once the block has finished running, control is returned to your program, which continues running normally as if nothing happened, unless the signal handler has raised an exception. In that case, the exception will be raised by the Ruby runtime in the signal trampoline.

By default, Ruby traps the INT and TERM signals by raising Interrupt and SystemExit exceptions, respectively. Other signals cause a SignalException to be raised. Normally, such exceptions will cause the program to exit, unless they are rescued. The following block will gracefully handle pressing ctrl-C in the terminal:

begin
  puts 'going to sleep...'
  sleep
rescue => Interrupt
  puts 'got Interrupt, waking up.'
end

In addition there are some operations that are not allowed in Ruby signal handlers, namely acquiring mutexes (which is needed for doing buffered I/O) and joining threads. If you need to do that in your program, you’ll need to implement your own mechanisms for handling signals asynchronously, that is, outside of the signal handler block. Sidekiq, for example, implements a mechanism for handling signals asynchronously by writing to a pipe.

However, when your program switches constantly between multiple fibers, a signal may occur in the context of any fiber, and if a signal exception is raised, it might be raised in the context of any fiber. This might lead to a situation where the exception terminates some worker fiber, and this may prevent the graceful handling of the signal.

Making signals work with structured concurrency

Polyphony’s implementation of structured concurrency assures developers that any exception occuring in any fiber will bubble up the fiber hierarchy if it is not rescued locally. How can we make signal exceptions work in an multi-fiber environment, and furthermore how can we make any signal handler work when we don’t know in what fiber the signal will occur?

Polyphony’s answer to that is actually quite simple: it runs the signal handler block on a new raw fiber (“raw” meaning it’s not bound by Polyphony’s rules of structured concurrency.) This out-of-band fiber is priority-scheduled by putting it at the head of the runqueue, causing it to be run immediately once the currently running fiber has yielded control. Any uncaught exception raised in the signal handler will be propagated to the main fiber, which will also be priority-scheduled:

module FiberControlClassMethods
  def schedule_priority_oob_fiber(&block)
    # Setup raw fiber
    oob_fiber = Fiber.new do
      Fiber.current.setup_raw
      block.call
    rescue Exception => e
      # Transfer uncaught exception to the main fiber by scheduling with the
      # exception as the resume value.
      Thread.current.schedule_and_wakeup(Thread.main.main_fiber, e)
    end
    # Thread#schedule_and_wakeup schedules the fiber at the head of the runqueue.
    Thread.current.schedule_and_wakeup(oob_fiber, nil)
  end
end

The Thread#schedule_and_wakeup method schedules the given fiber with priority:

VALUE Thread_fiber_schedule_and_wakeup(VALUE self, VALUE fiber, VALUE resume_obj) {
  if (fiber != Qnil) {
    Thread_schedule_fiber_with_priority(self, fiber, resume_obj);
  }

  if (Backend_wakeup(rb_ivar_get(self, ID_ivar_backend)) == Qnil) {
    Thread_switch_fiber(self);
  }

  return self;
}

Backend_wakeup will return true if the Polyphony backend is currently polling for completions (or for events if using libev instead of io_uring). If that was the case, there’s no need to take any further action, the signal handling fiber will be switched to as soon as the polling is done (that is, right after the signal trap has returned). If not, we call Thread_switch_fiber which will immediately switch to the signal handling fiber.

In this manner, signal handling becomes asynchronous, but any received signal is handled as soon as possible, without interfering with the work of any fiber in your Ruby process. As a consequence, there’s no more limits on what you can do in the signal handler block provided to Kernel#trap.

Finally, in order to transparently handle the setting of signal traps, Polyphony monkey-patches Kernel#trap:

module Kernel
  # The actual code for this method is a bit more involved
  def trap(sig, &block)
    orig_trap(sig) do
      Fiber.schedule_priority_oob_fiber(&block)
    end
  end
end

Signal handling patterns

There are some useful patterns you can employ when dealing with signals. In general, Polyphony shies away from callbacks. Almost the entire Polyphony API is exclusively synchronous and blocking. But when installing a signal trap, you actually provide a callback that will be called asynchronously sometime in the future. With Polyphony, it’s easy to turn this into a blocking API.

def await_signal(sig)
  this_fiber = Fiber.current
  trap(sig) { this_fiber.schedule }
  suspend
end

# We do work in a separate fiber
spin { do_some_work }

# On the main fiber we wait for a signal
await_signal('INT')
puts 'Got INT, quitting.'

If the signal handling logic is something that can happen multiple times, we can use fiber messaging to serialize the receipt of signals:

worker = spin { do_some_work }
signal_watcher = spin_loop do
  sig = receive
  case sig
  when 'USR1'
    worker.restart
  when 'TERM'
    worker.stop
    worker.await
  when 'INT'
    exit!
  end
end

%w{USR1 TERM INT}
  .each { |sig| trap(sig) { signal_watcher << sig } }

Conclusion

In this article we have explored how POSIX signals can be handled in a safe and consistent manner in a fiber-based concurrent environment such as Polyphony. Other interesting aspects of the behaviour of multi-fiber Ruby programs, such as forking and exception handling, will be addressed in future articles. Please feel free to contact me if you have any questions about this article or Polyphony in general.

Real-world Concurrency with Ruby and Polyphony: a Telnet Chat App

Sat, 13 Nov 2021 00:00:00 GMT

Recently there has been a lot of renewed interest in fibers as the building blocks for writing concurrent Ruby apps. Most of the articles written lately (my own included) have tried to explain what are fibers, how they can be used for writing concurrent apps, and how fiber scheduling works. While this obviously is great, I feel there’s also a need for developers to get a feel for how a real-world fiber-based app looks, and how writing such an app differs from using, say, EventMachine or some other Ruby library providing a different concurrency model.

In this article I’ll walk you through implementing a bare-bones Telnet chat app using Polyphony. Along the way, I’ll demonstrate how Polyphony lets us write concurrent programs in a natural, idiomatic style, and show how fiber messaging, one of Polyphony’s unique features, allows us to design a concurrent app as a collection of simple, autonomous entities, each having a single responsibility.

The source code for the chat app is available as a gist.

Designing our chat app

The chat app we’re going to implement today will have the following requirements:

Telnet-based (that is, using plain text over TCP sockets.)
Support any number of concurrent users.
Support any number of rooms.
Each user can be in a single room at a time.
Rooms are ephemeral, so no need to keep a history of messages.
Users are also ephemeral, no need to keep a user’s state.
User actions are: join a room, leave a room, send a message to a room.
When a user sends a message to a room, all users in the same room get it.
A user joins or leaves a room by sending :enter <room> or :leave, respectively.

Now that we have our requirements, let’s concentrate on the design of our program: what are the different moving parts and how do they connect? One of the biggest advantages of using fibers is that, fibers being so cheap to create (in terms of computer resources,) we can implement any entity in our program as a fiber. If we take the problem of a chat app, we have rooms, we have users, and we have TCP connections. As I’ll show below, each of these can be modeled as an independent fiber.

Fiber messaging

In order for all those different fibers to communicate with each other, we can use fiber messaging, a feature that is unique to Polyphony, and is greatly inspired by message-passing in Erlang, which essentially permits Erlang processes to behave as concurrent actors.

In Polyphony, each fiber has a mailbox, and can receive messages by calling Kernel#receive. A message can be any Ruby object. To send a message to a fiber, we call Fiber#send or Fiber#<<. Receiving a message is a blocking operation, if the fiber’s mailbox is empty, the call to #receive will block until a message is sent to the fiber. The call to #send, however, is not blocking (except if the fiber’s mailbox is capped and filled to capacity. By default fiber mailboxes are not capped.)

Here’s a simple example to show how fiber messaging works:

require 'polyphony'

receiver = Fiber.current
spin do
  sleep 1
  receiver << "hello"
end

puts "Waiting for message..."
message = receive
puts "Got #{message.inspect}"

In the above example, we spin up a fiber that will sleep for 1 second, then send a message to the main fiber, which meanwhile waits for a message to be received. This apparently simple mechanism for asynchronous communication between fibers has profound implications for how we can structure our concurrent programs. Since fibers can behave as actors (just like Erlang processes,) they can basically have the same capabilities as custom Ruby objects. Think about it: when we call a method on a Ruby object, we basically send it a message. If fibers can send and receive messages, we can use them instead of plain Ruby objects. And just like custom Ruby objects which hold state (stored in instance variables,) fibers can hold state in local variables.

In order to see how a fiber can hold state and receive “method calls”, let’s take the simple example of a calculator with a memory. Our calculator can do arythmetic operations on the last retained value. Here’s how we’ll implement such a calculator using a normal Ruby class definition:

class Calculator
  def initialize
    @value = 0
  end

  def add(x)
    @value += x
  end

  def mul(x)
    @value *= x
  end
end

calculator = Calculator.new
calculator.add(3) #=> 3
calculator.mul(2) #=> 6

Now let’s see how we can do the same thing with a fiber:

require 'polyphony'

calculator = spin do
  value = 0
  loop do
    peer, op, x = receive
    case op
    when :add
      value += x
      peer << value
    when :mul
      value *= x
      peer << value
    end
  end
end

calculator << [Fiber.current, :add, 3]
receive #=> 3
calculator << [Fiber.current, :mul, 2]
receive #=> 6

The calculator fiber loops infinitely, waiting for messages to be received in its mailbox. Each message, having been destructured, is processed by updating the state and sending the updated state to the peer fiber which originated the message. Notice that in the fiber-based version, in order to get the result of the arythmetic operation, we need to provide the calculator fiber with the current fiber, to which it will send the result of the operation. In effect, our calculator fiber can be said to be a sort of server: it receives requests, handles them, and sends back a reply.

This might seem like a a much more complicated way of doing things, but in fact look at the stuff we don’t need to worry about: we don’t need to define a custom class, and our state is safely stored as a local variable and cannot be accessed or tampered with from the outside. Finally, since our calculator fiber is doing one thing at a time we are basically guaranteed to not have any race conditions when making “calls” to our calculator. Compare this to the “normal” implementation above, which will fail miserably once we try to call methods from multiple threads at once.

If we want to make the fiber’s interface a bit more like what we’re used to with our normal Ruby method calls, we can wrap our calculator fiber implementation with something akin to Erlang’s GenServer (generic server) behavior, as shown in the Polyphony repository. Our fiber-based calculator would then look something like this:

module Calculator
  module_function

  def initial_state
    0
  end

  def add(state, value)
    state += value
    # The first value is the return value, the second is the mutated state. In
    # our case, they are the same.
    [state, state]
  end

  def mul(state, value)
    state *= value
    [state, state]
  end
end

# start server with initial state
calculator = GenServer.start(Calculator)
calculator.add(3) #=> 3
calculator.mul(2) #=> 6

One important detail to understand about fiber messaging is that like with any API, the actual messages sent and received between fibers (which, if you recall, can be any Ruby object) need to be well defined. An abstraction such as the GenServer example shown above, can help with making those interfaces more convenient to use, but it is in no way obligatory. We can get by explicitly sending and receiving fiber messages.

Using fibers to encapsulate state - and fiber messaging to communicate between fibers - has an additional ramification: it guides the developer towards a more functional style of programming (the example above is a case in point.) You stop thinking in classes and objects, and think more in terms of methods and message passing. While Ruby is pretty good at doing both, in the last few years I’ve been personally gravitating towards a more functional programming style, and Polyphony does facilitate moving in that direction.

But let’s go back to our chat app. We’d like to implement the different entities in our program as fibers, and make them interact using fiber messaging. As noted above, if we want to use fiber messaging, we’ll need to have defined the different messages that are going to be sent between the different fibers, in other words the different interfaces those fibers will have. Before starting to write our implementation, let’s first define those.

Defining fiber interfaces

As we said, we have three kinds of entities: Telnet session, user, and room. Let’s figure out the responsibilities of each entity, and how those entities interact:

A room has zero or more users in it.
A user can enter a room, and can leave a room.
Once a user enters a room, she’ll receive any messages broadcast by other users in the same room.
When a Telnet session starts (the client connection is accepted,) a user is created.
When a Telnet session ends (the socket is closed,) the corresponding user leaves the room she’s currently in, and then terminates.

Let’s now define the shape of the different messages our chat entities should be able to handle. A room fiber needs to be able to handle the following events:

A user entered the room: [:enter, name, fiber]
A user left the room: [:leave, name, fiber]
A user wrote something: [:message, message]

A user fiber should handle the following events:

A line was received from the corresponding socket: [:input, message]
A message was broadcast in the user’s room: [:message, message]

The Telnet session fiber does not need to handle incoming messages, as its job is only to wait for lines of text to arrive on the socket, and send them to the corresponding user. The distinction between session and user is important, since those two entities have different responsibilities. The user fiber implements the business logic from the point of view of the user, dealing with notifications coming either from the room or the Telnet session. The Telnet session deals exclusively with receiving data on the corresponding TCP socket.

Now that we have defined the interactions and messages sent between the different parts of our app, let’s start writing code!

The Telnet session

We start writing our code with a straightforward implementation of a TCP server:

server = spin do
  server_socket = TCPServer.new('0.0.0.0', 1234)
  server_socket.accept_loop do |s|
    spin { handle_session(s) }
  end
end

We start by spinning up a server fiber that will run the TCP server. The server fiber creates a TCPServer instance for accepting connections. The #accept_loop method runs an infinite loop, waiting for connections to be accepted. For each accepted connection, we spin a separate fiber, calling #handle_session with the accepted connection. Let’s look at how #handle_session is implemented:

def handle_session(socket)
  socket << 'Please enter your name: '
  name = socket.gets.chomp
  socket.puts "Hello, #{name}!"
  user_fiber = spin { run_user(name, socket) }
  while (line = socket.gets.chomp)
    user_fiber << [:input, line]
  end
ensure
  user_fiber << [:close]
end

We start by asking the user for their name, then setup a fiber for the user, calling #run_user. Finally, we run a loop waiting for lines to arrive on our socket, and send each line to the user fiber.

The user fiber

Our user fiber will run a loop, waiting for and processing incoming messages:

def run_user(name, socket)
  current_room = nil
  loop do
    event, message = receive
    case event
    when :close
      break
    when :input
      case message
      when /\:enter\s+(.+)/
        leave_room(current_room, name) if current_room
        current_room = enter_room($1, name)
      when ':leave'
        leave_room(current_room, name) if current_room
      else
        say(current_room, name, message)
      end
    when :message
      socket.puts message
    end
  end
ensure
  leave_room(current_room, name) if current_room
end

We destructure incoming messages (received as an Array of the form [event, message]), then take the correct action according to the message received. Here are the rest of the user’s business logic, which consist of sending messages to the room the user has entered or left:

def leave_room(room_fiber, name)
  room_fiber << [:leave, name, Fiber.current]
end

def enter_room(room_name, name)
  room_fiber = find_room(room_name)
  room_fiber << [:enter, name, Fiber.current]
  room_fiber
end

def say(room_fiber, name, message)
  room_fiber << [:say, name, message]
end

The room

Finally, we get to the room entity, which manages a list of users and takes care of broadcasting messages received from individual users in the room. Let’s start with the #find_room method, which is used by users to find the fiber for the room they want to enter:

@room_fibers = {}
@main_fiber = Fiber.current

def find_room(room_name)
  @room_fibers[room_name] ||= @main_fiber.spin { run_room(room_name) }
end

Since #find_room is called in the context of the user fiber, we need to be careful about how we spin up the room fiber. We want our room fiber to not be limited to the lifetime of the user fiber (which will terminate when the user’s Telnet session closes,) and that means we cannot spin it directly from the user fiber. Instead, we spin it from the main fiber. Notice that the user fiber itself is spun from the Telnet session fiber, but since the user fiber should not outlive its Telnet session that is just fine.

(In a future article I’ll show a better way to manage fibers by organizing them into supervision trees, but for the sake of the present discussion the above solution is good enough).

Lets continue with the room implementation:

def run_room(room_name)
  @users = {}
  loop do
    event, *args = receive
    case event
    when :leave
      name, fiber = args
      @users.delete(args[1])
      broadcast(@users.keys, "#{args[0]} has left the room.")
      break if @users.empty?
    when :enter
      @users[args[1]] = true
      broadcast(@users.keys, "#{args[0]} has entered the room.")
    when :say
      broadcast(@users.keys, "#{args[0]}: #{args[1]}")
    end
  end
ensure
  @room_fibers.delete(room_name)
end

def broadcast(fibers, message)
  fibers.each { |f| f << [:message, message] }
end

The room fiber is very similar to the user fiber, in that it runs a loop waiting for events to be received. The different events are processed by updating the list of users and broadcasting the corresponding messages to all users.

Tying it all together

Now that we have implemented the different parts of the application, all that’s left is for the main fiber to wait for the server fiber to terminate (which will never arrive). We do that by calling Fiber#await:

server.await

Now that our program is complete, let’s run it (we can run two separate Telnet sessions from separate terminal windows):

sharon@nf1:~$ Telnet localhost 1234
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Please enter your name: sharon
Hello, sharon!
:enter foo
sharon has entered the room.
hi
sharon: hi
sylvain has entered the room.
sylvain: hi there!
hello
sharon: hello
...

Conclusion

We now have a fully functioning bare-bones chat app able to handle hundreds or even thousands of concurrent users, implemented in about 85 lines of code, and including a total of 8 methods: 1 for Telnet sessions, 4 for users, 3 for rooms. Our code is compact, easy to understand, and does not include any class definitions.

Furthermore, any state we need to keep track of (the current room for the user, and the list of users for each room) is conveniently held as local variables inside the relevant methods. As discussed above, we could have encapsulated our different entities (namely, users and rooms) as GenServer interfaces, but I’ll leave that as an exercise to the reader.

Also, note how fluid and idiomatic our code looks. Spinning up fibers takes no effort, and neither does fiber messaging. We just sprinkle our code with a bunch of spin receive and fiber << message and everything works concurrently.

There’s a lot to be said for designing concurrent programs as collections of autonomous actors, interacting using messages. Programming in this way requires a shift in how we think about the different entities in our program, and in how we get them to interact. I’ll continue exploring this subject in more detail in future articles.

You can find the complete code to the chat app here. Please feel free to contact me if you have any questions about this article or Polyphony in general.

About that monkey-patching business...

Thu, 04 Nov 2021 00:00:00 GMT

A few days ago, a comment was made on the internet about Polyphony, an open source project of mine, mentioning the fact that Polyphony patches some core Ruby APIs. The context was the difference between Polyphony and Async (another fiber-based concurrency gem for Ruby):

Last time I checked, polyphony monkey patched Ruby core methods to do its work :-/. This has deterred from learning more about polyphony… On the other hand, async gem has always had incredibly clean code.

I’m sure Bruno Sutic, the author of the above comment, was writing in good faith, but his comment implies that Polyphony’s code is “dirty”. It also implies that monkey-patching is somehow illegitimate. While normally I don’t get too excited about people calling my code names, I do take pride in my work, and I feel a rebuttal is in order.

Moreover, the mere fact that Polyphony employs a (somewhat) controversial technique to do its thing should not deter people like Bruno from examining it. I’m sure all of us would benefit from approaching other people’s code with an open and inquisitive mind.

In this article I’ll explain in detail the strategy I chose for developing Polyphony and the role monkey-patching plays within it. I’ll also discuss the potential problems raised by the practice of monkey-patching, and how they can be minimized.

What is monkey-patching?

But first, for those that are confused about what monkey-patching actually is, here’s the Wikipedia entry (go ahead, read it!) For the sake of the present discussion, I’ll define monkey-patching as the practice of changing or extending the behavior of pre-existing Ruby classes or modules by overriding their instance or class methods. This can be done in a variety of ways, depending on what you want to achieve. The most obvious way would be to open the class, then redefine some methods:

class String
  def to_i
    42
  end
end

'123'.to_i #=> 42

You can also put your patched methods in a separate module, then prepend it to the target class (that way it will take precedence over existing method definitions):

module StringPatches
  def to_i
    42
  end
end

String.prepend StringPatches

'123'.to_i #=> 42

If you need to target specific object instances, you can patch their singleton class:

s = '123'
class << s
  def to_i
    42
  end
end

s.to_i #=> 42

You can also limit the monkey-patching to a single file, class, or module, by using refinements:

module StringPatches
  refine String do
    def to_i
      42
    end
  end
end

using StringPatches # activate refinement

'123'.to_i #=> 42

So monkey-patching can be done in a variety of ways in Ruby, depending on how specific you want the patched behaviour to be: from the level of a single object instance, through specific scopes, all the way to patching classes globally.

It’s also worth noting that there are other techniques that could be used instead of monkey-patching: subclassing is ubiquitous in Ruby, and can even work for extending core Ruby classes. Rails’s HashWithIndifferentAccess is a case in point. I could probably come up with a bunch of other alternatives, but I’ll leave it at that. The point is, it really depends on the circumstances.

Is monkey-patching inherently bad?

I’m sure many people have written before about monkey-patching and whether it’s good or bad for you, but in my most humble opinion, there’s no right or wrong when it comes to programming. Monkey-patching is just a technique that has its place like everything else under the heavens.

Of course, monkey-patching can lead to problems - it can cause compatibility issues and strange bugs, for example when your monkey-patching gem is combined with other gems. It can break behaviour across different versions of Ruby, or in conjunction with specific versions of specific dependencies. It can cause all kinds of havoc. But it can also provide a very elegant solution in specific circumstances, and can be amazingly effective.

When is monkey-patching useful?

Monkey-patching is useful when you need to alter or extend the way pre-existing classes behave. Ruby’s open nature lets you change almost everything about Ruby, even core classes such as Array or String can be modified (as shown in the above examples.) Why would we want to do this? Here are some cases where monkey-patching can be useful:

Ensuring compatibility between different versions of Ruby. This is especially useful when you need to backport some new method introduced in a later version of Ruby to an earlier version of Ruby. This is commonly called “polyfill” (there’s a whole bunch of them on rubygems.org.)
“Fixing” some gem to work with your code. Suppose you have encountered a bug in some gem your project depends on. In some cases, despite everybody’s best intentions, fixes to those problems can sometimes take months find their way into a new version. In those cases, a monkey-patch can solve the problem immediately, even if only temporarily, the new version containing the fix is put out by the gem’s author.

Debugging an application’s behaviour by overriding methods and adding tracing, for example:

require 'socket'

class TCPServer
  alias_method :orig_initialize, :initialize
  def initialize(hostname, port)
    puts "Connecting to #{hostname}:#{port}"
    orig_initialize(hostname, port)
    puts "Connected to #{hostname}:#{port}"
  end
end

Otherwise extending or replacing behaviours provided by the Ruby core or stdlib, or by Ruby gems. For example, the oj gem, which provides fast JSON processing, has a compatibility mode that effectively patches the json gem to provided as part of Ruby’s stdlib. This feature lets Ruby apps take advantage of faster JSON processing without any change to their code.

It’s important to note that the advantage monkey-patching provides over other techniques, such as subclassing, is that those patches are in fact going to impact all the other dependencies of your app. In the case of the oj gem, any other dependencies that make use of the JSON API are also going to show improved performance!

Designing Polyphony

When I first started working on Polyphony, I didn’t know where it would take me. Polyphony began as an experiment in designing an API for writing concurrent Ruby programs. My starting point was the nio4r gem, which implements an event loop based on libev. I really liked what nio4r was able to do, and wanted to experiment with different concurrency models, so I took its C-extension code and start fiddling with it. I went through a whole bunch of different designs: callbacks, promises, futures, async/await, and finally fibers.

As Polyphony slowly took form, the following principles manifested themselves:

Polyphony should extend the Ruby runtime and feel like an integral part of it.
Polyphony’s API should allow expressing concurrent operations in a concise manner, with a minimum of abstractions or boilerplate.
Polyphony should allow developers to continue working with core and stdlib classes and APIs such as IO, Socket and Net::HTTP.

In order to be able to apply the above principles to Polyphony’s design, I needed a way to make Ruby’s core classes, especially those having to do with I/O, usable under Polyphony’s concurrency model. The only solution that would have allowed me to do that was monkey-patching all those classes, including the IO class, the different Socket classes, even the OpenSSL classes dealing with I/O. Without monkey-patching, Polyphony as it currently is would have been impossible to implement!

Polyphony and Ruby’s new fiber scheduler interface

At this point people might ask: what about using the new Fiber::SchedulerInterface introduced in Ruby 3.0? Presumably, with the Fiber::SchedulerInterface I would be able to keep the same design based on the same principles, without resorting to monkey-patching Ruby core classes. That’s because the fiber scheduler is baked right into Ruby’s core.

I have long thought about this problem, and have always come to the same conclusion: if I were to base Polyphony on the Fiber::SchedulerInterface, it would have limited what Polyphony could do. In fact, some of the features Polyphony currently offers would have been impossible to achieve:

I want Polyphony to work on older versions of Ruby. In fact, one of my original constraints for developing Polyphony was to have it work on Ruby >= 2.6 (I started work in Polyphony in August 2018.)
Polyphony’s design is highly-integrated - from the io_uring- or libev-based backend through the fiber-scheduling code, all the way to the developer-facing APIs for spinning up fibers and controlling them. Polyphony’s io_uring backend in particular offers unique capabilities, such as chaining of I/O operations, which might have been much more difficult to achieve had it been based on the fiber scheduler interface.
The Fiber::SchedulerInterface itself is still in a state of flux, and is still missing hooks for socket operations (according to Samuel Williams, the developer behind the fiber scheduler interface, the read and write hooks are considered experimental at the moment.)
The Fiber::SchedulerInterface will not magically bring fiber-awareness to all Ruby gems, especially not those implemented as C-extensions. Take for example the pg gem, which has recently added support for fiber schedulers. Compare that with Polyphony’s monkey-patching approach, which is much more minimal. Another example is Polyphony’s patch for the redis gem.

Even gems that do not rely on C-extensions might be problematic. Such is the case with ActiveRecord, which does connection pooling per thread and is thus apparently incompatible with both Async and Polyphony. Here again, it seems to me that monkey-patching might be the more effective solution, and perhaps also simpler to implement, at least in the short term. That’s how Polyphony implements fiber-aware connection pooling for Sequel (thanks wjordan!)

Integrating fiber scheduling into Ruby’s core is not a trivial undertaking (and I applaud Samuel for his resolve and determination in the face of substantial pushback.) The problem is not only technological - making fiber scheduling work with the complex code of Ruby’s IO core - but also getting other Ruby core developers and gem authors to understand the merits of this effort, and finally to put out new fiber-aware versions of their code.

As the fiber scheduler interface matures, I guess I will have to reconsider my position regarding Polyphony. One interesting suggestion was to implement Polyphony as a fiber scheduler for Ruby >= 3.0, and as a “polyfill” for earlier Ruby versions.

What about compatibility?

Monkey-patching does introduce the problem of compatibility, and this should be taken seriously. Polyphony aims to reduce compatibility issues in two ways. firstly, Polyphony aims to mimic the same behaviour as much as possible across all monkey-patched APIs from the point of view of the application. Secondly, Polyphony aims to monkey-patch mostly stable APIs that have little chance of changing between versions.

This approach is not without problems. For example, the changes to irb introduced in Ruby 2.7 have broken Polyphony’s patch, and there’s an outstanding issue for it (I’ll get to it eventually.)

Polyphony also provides, as described above, mokey-patches for third-party gems, such as pg, redis and others. Those are are bundled as part of Polyphony, but in the futre might be extracted to separate gems, in order to be able to respond more quickly to local issues that arise in integrating those gems with Polyphony.

I’d also like to note that I do not expect people to just add Polypony to their Gemfile and start spinning up fibers all over the place. In fact, using Polyphony is to me such a radical shift from previous approaches to Ruby concurrency that I find it improbable that one day it will simply work with any Ruby on Rails app. Using Polyphony to its full potential will require much more careful consideration on the part of developers using it.

I’d also like to add that my goal is not for Polyphony to become the solution for fiber-based concurrency in Ruby. It’s just a project that I find useful for my own work and I feel could be useful for others as well. There’s nothing wrong with having multiple solutions to the same problem. On the contrary, I find it beneficial and stimulating to have competing projects based on different approaches.

So what does Polyphony patch?

Polyphony replaces whole parts of the Ruby core API with fiber-aware code that provides the same functionality, but integrated with Polyphony’s code. I took great care to make method signatures are the same and behave identically as much as possible.

It’s worth noting that running Ruby programs with multiple fibers present challenges that go beyond merely reading and writing to IO instances: there’s all kinds of subtleties around forking, signal handling, waiting for child processes and thread control. Much of the monkey-patching that Polyphony performs is around that.

Here’s a (probably incomplete) list of APIs monkey-patched by Polyphony:

IO - all read/write instance methods and all read/write class methods
Socket/TCPSocket/TCPServer et al - all I/O functionality including accept and connect
OpenSSL::SSL::SSLSocket/OpenSSL::SSL::SSLSocket - all read/write/accept/connect methods
Kernel - methods such as sleep, ` (backtick) , system, trap
Process#detach
Timeout#timeout
Thread#new and Thread#join

Polyphony also provides monkey-patches for gems such as pg, redis, mysql2 and sequel.

Conclusion

Polyphony uses monkey-patching extensively because it’s the best way to achieving the goals I set to myself in developing it. Yes, monkey-patching has its disadvantages, but it also has advantages (as I showed above). Finally, I believe Polyphony should be rather judged by what it can do, and by the value it provides to developers.

Explaining Ruby Fibers

Wed, 20 Oct 2021 00:00:00 GMT

Fibers have long been a neglected corner of the Ruby core API. Introduced in Ruby version 1.9 as a coroutine abstraction, fibers have never really seemed to realize their promise of lightweight concurrency, and remain relatively little explored. In spite of attempts to employ them for achieving concurrency, most notably em-synchrony, fibers have not caught on. Hopefully, with the advent of the FiberScheduler interface introduced in Ruby 3.0, and libraries such as Async and Polyphony, this situation will change and fibers will become better known and understood, and Ruby developers will be able to use them to their full potential.

My aim in this article is to explain how fibers work from the point of view of a concurrent Ruby application written using Polyphony. I’ll give an overview of fibers as concurrency constructs, and discuss how Polyphony harnesses Ruby fibers in order to provide an idiomatic and performant solution for writing highly-concurrent Ruby apps. For the sake of simplicity, I’ll omit some details, which will be mentioned in footnotes towards the end of this article.

What’s a Fiber?

Most developers (hopefully) know about threads, but just what are fibers? Linguistically we can already tell they have some relation to threads, but what is the nature of that relation? Is it one of aggregation (as in “a thread is made of one or more fibers”,) or is it one of similitude (as in “a fiber is sort of like a thread but lighter/smaller/whatever”?) As we shall see, it’s a little bit of both.

A fiber is simply an independent execution context that can be paused and resumed programmatically. We can think of fibers as story lines in a book or a movie: there are multiple happenings involving different persons at different places all occurring at the same time, but we can only follow a single story line at a time: the one we’re currently reading or watching.

This is one of the most important insights I had personally when I started working with Ruby fibers: there’s always a currently active fiber. In fact, even if you don’t use fibers and don’t do any fiber-related work, your code is still running in the context of the main fiber for the current thread, which is created automatically by the Ruby runtime for each thread.

So a fiber is an execution context, and in that regard it’s kind of like a thread: it keeps track of a sequence of operations, and as such it has its own stack and instruction pointer. Where fibers differ from threads is that they are not managed by the operating system, but instead are paused, resumed, and switched between by the userspace program itself. This programmatic way to switch between different execution contexts is also called “cooperative multitasking”, in contrast to the way threads are being switched by the operating system, called “preemptive multitasking”.

Switching between Fibers

Here’s a short example of how switching between fibers is done¹. In this example we’re doing the fiber switching manually:

require 'fiber'

@f1 = Fiber.new do
  puts 'Hi from f1'
  @f2.transfer
  puts 'Hi again from f1'
end

@f2 = Fiber.new do
  puts 'Hi from f2'
  @f1.transfer
end

puts 'Hi from main fiber'
@f1.transfer
puts 'Bye from main fiber'

In the above example, we create two fibers. The main fiber then transfers control (“switches”) to fiber @f1, which then transfers control to @f2, which then returns control to @f1. When @f1 has finished running, control is returned automatically to the main fiber, and the program terminates. The program’s output will be:

Hi from main fiber
Hi from f1
Hi from f2
Hi again from f1
Bye from main fiber

One peculiarity of the way switching between fibers is done, is that the context switch is always initiated from the currently active fiber. In other words, the currently active fiber must voluntarily yield control to another fiber. This is also the reason why this model of concurrency is called “cooperative concurrency.”

Another important detail to remember is that a fiber created using Fiber.new always starts in a suspended state. It will not run unless you first switch to it using Fiber#transfer.

Controlling fiber state via the context switch

What I find really interesting about the design of Ruby fibers is that the act of pausing a fiber, then resuming it later, is seen as a normal method call from the point of view of the fiber being resumed: we make a call to Fiber#transfer, and that call will return only when the fiber who made that call is itself resumed using a reciprocal call to Fiber#transfer. The return value will be the value given as an argument to the reciprocal call, as is demonstrated in the following example:

ping = Fiber.new do |peer|
  loop do
    msg = peer.transfer('ping')
    puts msg
  end
end

pong = Fiber.new do |msg|
  loop do
    puts msg
    msg = ping.transfer('pong')
  end
end

ping.transfer(pong)

The block provided to Fiber.new can take a block argument which will be set to the value given to Fiber#transfer the first time the fiber is being resumed. We use this to set ping’s peer, and then use Fiber#transfer to pass messages between the ping and pong fibers.

This characteristic of Fiber#transfer has profound implications: it means that we can control a fiber’s state using the value we pass to it as we resume it. Here’s an example for how this could be done:

main = Fiber.current
f = Fiber.new do |msg|
  count = 0
  loop do
    case msg
    when :reset
      count = 0
    when :increment
      count += 1
    end
    puts "count = #{count}"
    msg = main.transfer
  end
end

3.times { |i| f.transfer(:increment) } # count = 3
f.transfer(:reset)                     # count = 0
3.times { |i| f.transfer(:increment) } # count = 3

In the above example, the fiber f gets its state updated each time it is resumed. The main fiber can control f’s state by passing a special value when calling Fiber#transfer. As long as we have in place a well-defined convention for how the transferred value is interpreted by fibers in a given program, we can implement arbitrarily complex semantics for controlling our fibers’ state and life-cycle. Later on in this article we’ll see how Polyphony couples this mechanism with return values from blocking operations, as well as exceptions that permit us to cancel any long-running operation at any time.

Switching fibers on long-running operations

Now that we have a feel for how fibers can be paused and resumed, we can discuss how this can be used in conjunction with blocking operations. For the sake of our discussion, a blocking operation is any operation that potentially needs to wait for some external event to occur, such as a socket becoming readable, or waiting for a timer to elapse. We want to be able to have multiple concurrent tasks, each proceeding at its own pace, and each yielding control whenever it needs to wait for some external event to occur.

In order to demonstrate how this might work, let’s imagine a program where we have two fibers: one waits for data to arrive on STDIN, then echoes it; and a second fiber that prints the time once a second:

@echo_printer = Fiber.new do
  loop do
    if STDIN.readable?
      msg = STDIN.readpartial(1024)
      puts msg
    end
    @time_printer.transfer
  end
end

@time_printer = Fiber.new do
  timer = Timer.new(1)
  loop do
    if timer.elapsed?
      puts "Time: #{Time.now}"
      timer.reset
    end
    @echo_printer.transfer
  end
end

In each of the above fibers, we have have a condition that tells us if the fiber can proceed: for @echo_printer the condition is STDIN.readable?; for @time_printer it’s timer.elapsed?. What’s notable though about this example that the switching between fibers is done explicitly, and that each fiber needs to check a condition continually. Of course, this is not ideal, since the two fibers will just pass control between them until one of the conditions is met and actual work is done. If you run such a program, you’ll see one of your CPU cores saturated. But the main insight to draw here is that each fiber can yield control to another fiber if it cannot go on doing actual work.

Automatic fiber switching using an event reactor

Let’s see if we can avoid endlessly checking for readiness conditions, by using an event reactor - a piece of software that lets you subscribe to specific events, most importantly I/O readiness, and timers. In this case we’ll be using ever, a tiny Ruby gem that I wrote a few months ago, implementing an event reactor for Ruby apps based on libev. Here’s our rewritten example:

require 'ever'
require 'fiber'

evloop = Ever::Loop.new
@reactor = Fiber.current

@echo_printer = Fiber.new do
  loop do
    msg = STDIN.readpartial(1024)
    puts msg
    @reactor.transfer
  end
end

@time_printer = Fiber.new do
  loop do
    puts "Time: #{Time.now}"
    @reactor.transfer
  end
end

# register interest in events
evloop.watch_io(@echo_printer, STDIN, false, false)
evloop.watch_timer(@time_printer, 1, 1)

# run loop
evloop.each { |fiber| fiber.transfer }

As you can see, our fibers no longer need to check for readiness. The reactor, after registering interest in the respective event for each fiber, runs the event loop, and when an event becomes available, the corresponding fiber is resumed. Once resumed, each fiber does its work, then yields control back to the reactor.

This is already a great improvement, since our program does not need to endlessly check for readiness, and the code for each of the fibers doing actual work looks almost normal. The only sign we have of anything “weird” going on is that each fiber needs to yield control back to the reactor by calling @reactor.transfer.

If we look more closly at the above program, we can describe what’s actually happening as follows:

@echo_printer = Fiber.new do
  loop do
    msg = STDIN.readpartial(1024)
    puts msg

    wait_for_events if queued_events.empty?
    queued_events.each { |e| e.fiber.transfer }
  end
end

@time_printer = Fiber.new do
  loop do
    puts "Time: #{Time.now}"

    wait_for_events if queued_events.empty?
    queued_events.each { |e| e.fiber.transfer }
  end
end

...

For each of our fiber, at any point where the fiber needs to wait, we first look at our list of queued events. If there are none, we wait for events to occur. Finally we proceed to handle those events by transferring control to each respective fiber. This needs to be done for each blocking or long-running operation: reading from a file, reading from a socket, writing to a socket, waiting for a time period to elapsed, waiting for a process to terminate, etc. What if we had a tool that could automate this for us? This is where Polyphony enters the stage.

Using Polyphony for fiber-based concurrency

Polyphony automates all the different aspects of fiber switching, which boils down to knowing which fiber should be running at any given moment. Let’s see how Polyphony solves the problem of fiber switching by using it to rewrite the example above:

require 'polyphony'

echo_printer = spin do
  loop do
    msg = STDIN.read
    puts msg
  end
end

time_printer = spin do
  loop do
    sleep 1
    puts "Time: #{Time.now}"
  end
end

Fiber.await(echo_printer, time_printer)

As our rewritten example shows, we got completely rid of calls to Fiber#transfer. The fiber switching is handled automatically and implicitly by Polyphony². Each fiber is an autonomous infinite loop made of a sequence of operations that’s simple to write and simple to read. All the details of knowing when STDIN is ready or when a second has elapsed, and which fiber to run at any moment, are conveniently taken care of by Polyphony.

Even more importantly, Polyphony offers a fluent, idiomatic API that mostly gets out of your way and feels like an integral part of the Ruby core API. Polyphony introduces the #spin global method, which creates a new fiber and schedules it for running as soon as possible (remember: fibers start their life in a suspended state.) The call to Fiber.await means that the main fiber, from which the two other fibers were spun, will wait for those two fibers to terminate. Because both echo_printer and time_printer are infinite loops, the main fiber will wait forever.

Now that we saw how Polyphony handles behind the scenes all the details of switching between fibers, let’s examine how Polyphony does that.

The fiber switching dance

A fiber will spend its life in one of three states: :running, :waiting or :runnable³. As discussed above, only a single fiber can be :running at a given moment (for each thread). When a fiber needs to perform a blocking operation, such as reading from a file descriptor, it makes a call to the Polyphony backend associated with its thread, which performs the actual I/O operation. The backend will submit the I/O operation to the OS (using the io_uring interface⁴,) and will switch to the first fiber pulled from the runqueue, which will run until it too needs to perform a blocking operation, at which point another fiber switch will occur to the next fiber pulled from the runqueue.

Meanwhile, our original fiber has been put in the :waiting state. Eventually, the runqueue will be exhausted, which means that all fibers are waiting for some event to occur. At this point, the Polyphony backend will check whether any of the currently ongoing I/O operation have completed. For each completed operation, the corresponding fiber is “scheduled” by putting it on the runqueue, and the fiber transitions to the :runnable state. The runnable state means that the operation the fiber was waiting for has been completed (or cancelled), and the fiber can be resumed.

Once all completed I/O operations have been processed, the backend performs a fiber switch to the first fiber available on the runqueue, the fiber transitions back to the :running state, and the whole fiber-switching dance recommences.

What’s notable about this way of scheduling concurrent tasks is that Polyphony does not really have an event loop that wraps around your code. Your code does not run inside of a loop. Instead, whenever your code needs to perform some blocking operation, the Polyphony backend starts the operation, then switches to the next fiber that is :runnable, or ready to run.

The runqueue

The runqueue is simply a FIFO queue that contains all the currently runnable fibers, that is fibers that can be resumed. Let’s take our last example and examine how the contents of the runqueue changes as the program executes:

runqueue #=> []

echo_printer = spin { ... }
runqueue #=> [echo_printer]

time_printer = spin { ... }
runqueue #=> [echo_printer, time_printer]

# at this point the two fibers have been put on the runqueue and will be resumed
# once the current (main) fiber yields control:
Fiber.await(echo_printer, time_printer)

# while the main fiber awaits, the two fibers are resumed. echo_printer will
# wait for STDIN to become readable. time_printer will wait for 1 second to
# elapse.

# The runqueue is empty.
runqueue #=> []

# Since there's no runnable fiber left, the Polyphony backend will wait for
# io_uring to generate at least one completion entry. A second has elapsed, and
# time_printer's completion has arrived. The fiber becomes runnable and is put
# back on the runqueue.
runqueue #=> [time_printer]

# Polyphony pulls the fiber from the runqueue and switches to it. The time is
# printed, and time_printer goes back to sleeping for 1 second. The runqueue is
# empty again:
runqueue #=> []

# The Polyphony backend waits again for completions to occur. The user types a
# line and hits RETURN. The completion for echo_printer is received, and
# echo_printer is put back on the runqueue:
runqueue #=> [echo_printer]

# Polyphony pulls the fiber from the runqueue and switches to it.
runqueue #=> []

...

While the runqueue was represented above as a simple array of runnable fibers, its design is actually much more sophisticated than that. First of all, the runqueue is implemented as a ring buffer in order to achieve optimal performance. The use of a ring buffer algorithm results in predictable performance characteristics for both adding and removing of entries from the runqueue. When adding an entry to a runqueue that’s already full, the underlying ring buffer is reallocated to twice its previous size. For long running apps, the runqueue size will eventually stabilize around a value that reflects the maximum number of currently runnable fibers in the process.

In addition, each entry in the runqueue contains not only the fiber, but also the value with which it will be resumed. If you recall, earlier we talked about the fact that we can control a fiber’s state whenever we resume it by passing it a value using Fiber#transfer. The fiber will receive this value as the return value of its own previous call to Fiber#transfer.

In Polyphony, each time a fiber is resumed, the return value is checked to see if it’s an exception. In case of an exception, it will be raised in the context of the resumed fiber. Here’s an excerpt from Polyphony’s io_uring backend that implements the global #sleep method:

VALUE Backend_sleep(VALUE self, VALUE duration) {
  Backend_t *backend;
  GetBackend(self, backend);

  VALUE resume_value = Qnil;
  io_uring_backend_submit_timeout_and_await(backend, NUM2DBL(duration), &resume_value);
  RAISE_IF_EXCEPTION(resume_value);
  RB_GC_GUARD(resume_value);
  return resume_value;
}

In the code above, we first submit an iouring timeout entry, then yield control (by calling Fiber#transfer on the next runnable fiber) and await its completion. When the call to io_uring_backend_submit_timeout_and_await returns, our fiber has alreadey been resumed, with resume_value holding the value returned from our call to Fiber#transfer. We use RAISE_IF_EXCEPTION to check if resume_value is an exception, and raise it in case it is.

It’s also important to note the resume values stored alongside fibers in the runqueue can be used in effect to control a fiber’s state, just like in bare bones calls to Fiber#transfer. This can be done using the Polyphony-provided Fiber#schedule method, which puts the fiber on the runqueue, along with the provided resume value:

lazy = spin do
  loop do
    msg = suspend
    puts "Got #{msg}"
  end
end

every(1) { lazy.schedule('O hi!') }

In the example above, our lazy fiber suspends itself (using #suspend, which puts it in a :waiting state), and the main fiber schedules it once every second along with a message. The lazy fiber receives the message as the return value of the call to #suspend. One important difference between Fiber#schedule and Fiber#transfer is that Fiber#schedule does not perform a context switch. It simply puts the fiber on the runqueue along with its resume value. The fiber will be resumed as soon as all previous runnable fibers have been resumed and have yielded control.

Yielding control

As explained above, blocking operations involve submitting the operation to the io_uring interface, and then yielding (or #transferring) control to the next runnable fiber. A useful metaphor here is the relay race: only a single person runs at any given time (she holds the baton,) and eventually the runner will pass the baton to the next person who’s ready to run. Let’s examine what happens during this “passing of the baton” in a little more detail.

In effect, once the I/O operation has been submitted, the Polyphony backend calls the backend_base_switch_fiber function, which is responsible for this little ceremony, which consists of the following steps:

Shift the first runqueue entry from the runqueue.
If the entry is not nil:
- Check if it’s time to do a non-blocking poll (in order to prevent starvation of I/O completions. See discussion below.)
- proceed to step 4.
Otherwise:
- Perform a blocking poll.
- Go back to step 1.
Transfer control to the entry’s fiber with the entry’s resume value using Fiber#transfer.

All this happens in the context of the fiber that yields control, until a context switch is performed in step 4. To reuse our “relay race” metaphor, each time the current runner wishes to pass the baton to the next one, it’s as if she has a little gadget in her hand that holds the baton, performs all kinds of checks, finds out who the next runner is, and finally hands it over to the next runner.

Polling for I/O completions

When there are no runnable fibers left, Polyphony polls for at least one io_uring completion to arrive. For each received completion, the corresponding fiber is scheduled by putting it on the runqueue. Once the polling is done, the first fiber is pulled off the runqueue and is then resumed.

As we saw, polling for completions is only perofrmed done when the runqueue is empty. But what about situations where the runqueue is never empty? Consider the following example:

@f1 = spin { loop { @f2.schedule; puts 'f1' } }
@f2 = spin { loop { @f1.schedule; puts 'f2' } }

Fiber.await(@f1, @f2)

Each of the fibers above will run in an infinite loop, scheduling its peer and then printing a message. As shown above, the call to #puts, being an I/O operation, causes the Polyphony backend to submit the write operation to the io_uring interface, and then perform a context switch to the next runnable fiber. In order for the call to #puts to return, the Polyphony backend needs to poll for completions from the io_uring interface. But, since the runqueue is never empty (both fibers are scheduling each other, effectively adding each other to the runqueue,) the runqueue will never be empty!

In order to be able to deal with such circumstances, and prevent the “starvation” of completion processing, the Polyphony backend periodically performs a non-blocking check for any received completions. This mechanism assures that even in situations where our application becomes CPU-bound (since there’s always some fiber running!) we’ll continue to process io_uring completions, and our entire process will continue to behave normally.

Implications for performance

The asynchronous nature of the io_uring interface has some interesting implications for the performance of Polyphony’s io_uring backend. As mentioned above, the submission of SQE’s is deferred, and performed either when a certain number of submissions have acumulated, or before polling for completions.

In Polyphony, runnable fibers are always prioritized, and polling for events is done mostly when there are no runnable fibers. Theoretically, this might have a negative impact on latency, but I have seen the io_uring backend achieve more than twice the throughput achieved by the libev backend. Tipi, the Polyphony-based web server I’m currently developing, boasts a request rate of more than 138K requests/second at the time of this writing.

In fact, Polyphony provides multiple advantages over other concurrency solutions: The number of I/O-related syscalls is minimized (by using the io_uring interface.) In addition, the use Fiber#transfer to transfer control directly to the next runnable fiber halves the number of context-switches when compared to using Fiber#resume/Fiber.yield.

Most importantly, by prioritizing runnable fibers over processing of I/O completions (with an anti-starvation mechanism as described above,) Polyphony lets a Ruby app switch easily between I/O-bound work and CPU-bound work. For example, when a Polyphony-based web server receives 1000 connections in the space of 1ms, it needs to perform a lot of work, setting up a fiber, a parser and associated state for each connection. The design of Polyphony’s scheduling system allows the web server to do this burst of hard work while deferring any I/O work for later submission. When this CPU-bound burst has completed, the web server fires all of its pending I/O submissions at once, and can proceed to check for completions.

Cancellation

Now that we have a general idea of how Polyphony performs fiber switching, let’s examine how cancellation works in Polyphony. We want to be able to cancel any ongoing operation at any time. This can be done for any reason: a timeout has elapsed, an exception has occurred in a related fiber, or business logic that requires cancelling some specific operation in specific circumstances. The mechanism for doing this is simple, as mentioned above: we use an exception as the resume value that will be transferred to the fiber when the context switch occurs.

Here’s a simple example that shows how cancellation looks from the app’s point of view:

def gets_with_timeout(io, timeout)
  move_on_after(timeout) { io.gets }
end

The #move_on_after global API sets up a timer, then runs the given block. If the timer has elapsed before the block has finished executing, the fiber is scheduled with a Polyphony::MoveOn exception, any ongoing I/O operation is cancelled by the backend, the exception is caught by #move_on_after and an optional cancellation value is returned.

If we want to generate an exception on timeout, we can instead use the #cancel_after API, which will schedule the fiber with a Polyphony::Cancel exception, and the exception will have to be caught by the app code:

def gets_with_timeout(io, timeout)
  cancel_after(timeout) { io.gets }
rescue Polyphony::Cancel
  "gets was cancelled!"
end

More ways to control fiber execution

In addition to the various APIs discussed above, here’s a list of some of the various APIs used for controlling fiber execution:

#suspend - transition from :running to :waiting state. A fiber that was suspended will not be resumed until it is manually scheduled.
#snooze - transition from :running to :runnable state. The fiber is put on the runqueue, and will eventually get its turn again. This method is useful when a fiber performs a long-running CPU-bound operation and needs to periodically let other fibers have their turn.
Fiber#schedule - transition from :waiting to :runnable state, with an optional resume value.
Fiber#terminate - terminate a fiber.

Conclusion

Polyphony has been built to harness the full power of Ruby fibers, and provide a solid and joyful experience for those wishing to write highly-concurrent Ruby apps. There are many subtleties to designing a robust, well-behaving fiber-based concurrent environment. For example, there’s the problem of forking from arbitrary fibers, or the problem of correctly handling signals. Polyphony aims to take care of all these details, in order to be able to handle a broad range of applications and requirements.

I hope this article has helped clear up some of the mystery and misconceptions about Ruby fibers. Please let me know if you have specific questions about fibers in general or Polyphony in particular, and feel free to browse Polyphony’s source code. Contributions will be gladly accepted!

Footnotes

1. In this article we’ll confine ourselves to using the Fiber#transfer API for fiber switching, which is better suited for usage with symmetric coroutines. Using the Fiber.yield/Fiber#resume API implies an asymmetric usage which is better suited for fibers used as generators or iterators. Sadly, most articles dealing with Ruby fibers only discuss the latter, and make no mention of the former. Please note that in order to use Fiber#transfer we first need to require 'fiber'.

2. I’ve mentioned above the Fiber Scheduler interface introduced in Ruby 3.0, based on the work of Samuel Williams. This new feature, baked into the Ruby core, has roughly the same capabilities as Polyphony when it comes to automatically switching between fibers based on I/O readiness. As its name suggests, this is just a well-defined interface. In order to be able to employ it in your Ruby app, you’ll need to use an actual fiber scheduler. At the moment, the following fiber schedulers are available, in different states of production-readiness: evt, event, and my own libev-scheduler.

3. A fourth state, :dead is used for fibers that have terminated. A fiber’s state can be interrogated using Fiber#state.

4. Polyphony also includes an alternative backend used on non-Linux OSes or older Linux kernels. Both backends have the same capabilities. In this article we’ll discuss only the io_uring backend.

Embracing Infinite Loops with Ruby and Polyphony

Thu, 14 Oct 2021 00:00:00 GMT

In this article I’ll discuss the use of infinite loops as a major construct when writing concurrent apps in Ruby using Polyphony. I’ll show how infinite loops differ from normal, finite ones; how they can be used to express long-running tasks in a concurrent environment; and how they can be stopped.

Polyphony is a library for writing highly concurrent Ruby apps. Polyphony harnesses Ruby fibers and a powerful io_uring-based I/O runtime to provide a solid foundation for building high-performance concurrent Ruby apps.

In the last few months I’ve been slowly transitioning from working on Polyphony-designing APIs and adding functionality-to using to develop actual applications, some of them open source, and others closed source production apps for my clients.

In the process of actually using Polyphony as the basis for writing concurrent apps, I’ve discovered some patterns that I’d like to share. It’s really fascinating how the design of an API can impact the patterns that emerge in the application code. Take for example loops.

Developers that are used to asynchronous APIs will probably find the idea of writing loops in your app code anathema to asynchronous design: there’s only one loop - the main event loop - and it is that loop which drives your code. You just provide callbacks to be called at the right moment.

By contrast, with Polyphony the app code is written in a sequential style, and it is the app code that is in control. There is no event loop. Instead, you can create any number of fibers, all executing concurrently, with each of those fibers proceeding independently of the others.

But loops come into play when you want to launch autonomous long-running tasks, like listening for incoming connections on a TCP socket, pulling items from a queue and processing them, or periodically running some background task. Infinite loops are what makes it possible to “fire-and-forget” those concurrent processes.

Loops are everywhere!

Loops are one of the most useful ways to control execution. Loops are used anywhere you need to repeat an operation, and can be expressed in a variety of ways, from the lowly GOTO, through plain for and while loops, all the way to Ruby’s elegant #each and related methods, which take a block and apply it to items from some iterable object. While those don’t necessarily look like loops, they are, in fact, loops:

# this is a loop
while (item = queue.shift)
  item.process
end

# this is also a loop
queue.each { |i| i.process }

Infinite loops

Inifinite loops are loops that run indefinitely. A loop can be inadvertently infinite if the loop logic is faulty, but loops can also be infinite by design. Infinite loops are made for running autonomous, long-lived tasks that can run any number of iterations, and are not meant to be stopped conditionally. Here are some examples:

# Accept incoming connections:
loop do
  socket = server.accept
  handle_client_connection(socket)
end

# Process items from a queue:
loop do
  item = queue.shift
  process(item)
end

As the example above shows, Ruby provides the very useful #loop method which lets us express infinite loops in a clear and concise manner. Looking at the code we can immediately tell that we’re dealing with an infinite loop.

What’s important to note about infinite loops is that they can include a mechanism for breaking out of the loop if a certain condition is met. In fact, sometimes the distinction between a finite loop and an infinite one is not that clear.

Take for example a loop for handling an HTTP client connection. It needs to run for the life time of the connection, which can last for any duration and for any number of HTTP requests. In this case, this might look like an infinite loop, but it will include a conditional break:

# using h1p for parsing HTTP/1
def handle_client_connection(socket)
  parser = H1P::Parser.new(socket)
  loop do
    headers = parser.parse_headers # returns nil when socket is closed
    break unless headers

    body = parser.read_body
    handle_request(headers, body)
  end
end

Another way to express the same logic, which makes it look like a normal finite loop, is like this:

def handle_client_connection(socket)
  parser = H1P::Parser.new
  while (headers = parser.parse_headers)
    body = parser.read_body
    handle_request(headers, body)
  end
end

Concurrent infinite loops

What’s interesting about infinite loops is that once they start, theoretically they will go on forever! In Polyphony you can start any number of infinite loops, each running in its own fiber. Polyphony does the hard work of switching between all those fibers, letting each fiber proceed at its own pace once the operation it was waiting for has completed: reading from or writing to a socket, waiting for an item to become available on a queue, etc. To do this, we use the #spin global method provided by Polyphony, which spins up new fibers:

item_processor = spin do
  loop do
    item = item_queue.shift
    process(item)
  end
end

http_server = spin do
  server = TCPServer.new('0.0.0.0', 1234)
  loop do
    socket = server.accept
    # each client runs in its own fiber
    spin { handle_http_client(socket) }
  end
end

Fiber.await(item_processor, http_server)

In the above example, we start a fiber for processing items from a queue, and along side it an HTTP server. Each of those is implemented using an infinite loop running on a separate fiber. Finally, the main fiber waits for those two fibers to terminate. While the main fiber waits, Polyphony takes care of running the item processor and the HTTP server, with each fiber proceeding at its own pace as items are pushed into the queue, and as incoming HTTP connections are being accepted.

Interrupting an infinite loop

As we saw above, starting an inifinite loop on a separate fiber is really easy, but how do you interrupt one? Polyphony provides us with some tools for interrupting fibers at any time. We can do that by scheduling the specific fiber with an exception, which might be a normal exception, or one of the special exceptions that Polyphony provides for controlling fibers.

In order to stop a fiber running an infinite loop, we can call Fiber#stop:

item_processor = spin do
  loop do
    item = item_queue.shift
    process(item)
  end
end

# tell the item_processor to stop
item_processor.stop

# then wait for it to terminate
item_processor.await

Under the hood, Fiber#stop schedules the fiber with a Polyphony::MoveOn exception, which means that the fiber should just terminate at the earliest occasion, without the exception bubbling further up the fiber hierarchy.

As the example above shows, telling a fiber to stop does not mean it will do so immediately. We also need to properly wait for it to terminate, which we do by calling item_processor.await or Fiber.await(item_processor). As discussed above, stopping a fiber is done by scheduling it with a special exception that tells it to terminate. The terminated fiber will then proceed to terminate any child fibers it has, and perform other cleanup. This also means that you can use normal ensure blocks in order to perform cleanup. Let’s rewrite our item processor to process items by sending them in JSON format over a TCP connection:

item_processor = spin do
  soket = TCPSocket.new(PROCESSOR_HOSTNAME, PROCESSOR_PORT)
  loop do
    item = item_queue.shift
    socket.puts item.to_json
  end
ensure
  socket.close
end

item_processor.await
# the socket is now guaranteed to be closed

When the item_processor fiber is terminated, the ensure block guarantees that the socket used for sending items is closed before the call to item_processor.await returns.

More ways to stop a fiber

In addition to the Fiber#stop method, Polyphony has other APIs that can be used to stop a fiber in a variety of ways, including by raising an exception in the fiber’s context, and gracefully terminating a fiber. Termnating a fiber with an exception is done using Fiber#raise. This is especially useful when you need to implement your own error states:

my_fiber.raise(FooError, 'bar')

A graceful termination can be done using Fiber#terminate which takes an optional boolean flag. This requires a bit more logic in the fiber itself:

item_processor = spin do
  soket = TCPSocket.new(PROCESSOR_HOSTNAME, PROCESSOR_PORT)
  loop do
    item = item_queue.shift
    socket.puts item.to_json
  end
ensure
  if Fiber.current.graceful_shutdown?
    move_on_after(10) do
      wait_for_inflight_items
    end
  end
  socket.close
end

# terminate gracefully
item_processor.terminate(true)

In the example above, we added logic in the ensure block that waits up to 10 seconds for all inflight items to be processed, then proceeds with closing the TCP socket.

(We’ll take a closer look at exception handling and fiber termination in a future article.)

Polyphony 💛 Loops

Polyphony not only makes it easy to start and stop concurrent infinite loops, but it also further embraces loops by providing a bunch of loop APIs, including:

#spin_loop - used for spinning up fibers that just run a loop. That way you can be even more concise when expressing infinite loops:
```
item_processor = spin_loop { process(queue.shift) }
```
IO#read_loop/IO#recv_loop - used for reading from an IO instance (and provides even better performance, since it reads in a tighter loop):
```
connection.read_loop { |data| process(data) }
```

IO#feed_loop - used for feeding data read from an IO instance into a parser. Take for example a connection that uses MessagePack to pass messages:

require 'msgpack'

unpacker = MessagePack::Unpacker.new
# #feed_loop takes a receiver and the method to call on each chunk of data
connection.feed_loop(unpacker, :feed_each) { |msg| process(msg) }
end

In the future Polyphony will include even more #xxx_loop APIs that will provide more concise ways to express loops along with better performance.

Polyphony is just plain Ruby

Looking at all the above examples, you will have noticed how the Polyphony API really looks baked into the Ruby language, as if it was part of the Ruby core. One of my principal design goals was to minimize boilerplate code when expressing concurrent operations. There’s no instantiating of special objects, no weird mechanisms for controlling fibers or rescuing exceptions. It just looks like plain Ruby! This makes it easier to both write and read concurrent code.

Conclusion

In this article I’ve showed you how infinite loops can be used to express long-running concurrent tasks using Polyphony. Polyphony provides all the tools needed for controlling the execution of concurrent fibers. For more information about Polyphony you can go to the Polyphony website. You can also browse the examples in the Polyphony repository.

A Compositional Approach to Optimizing the Performance of Ruby Apps

Tue, 05 Oct 2021 00:00:00 GMT

Ruby has long been derided as a slow programming language. While this accusation has some element of truth to it, successive Ruby versions, released yearly, have made great strides in improving Ruby’s performance characteristics.

In addition to all the iterative performance improvements - Ruby 2.6 and 2.7 were especially impressive in terms of performance gains - recent versions have introduced bigger features aimed at improving performance, namely: a JIT compiler, the Ractor API for achieving parallelism, and the Fiber scheduler interface aimed at improving concurrency for I/O bound applications.

While those three big developments have yet to prove themselves in real-life Ruby apps, they represent great opportunities for improving the performance of Ruby-based apps. The next few years will tell if any of those new technologies will deliver on its promise for Ruby developers.

While Ruby developers (especially those working in and around Ruby on Rails) are still looking for the holy grail of Ruby fastness, I’d like to explore a different way of thinking about developing Ruby apps (and gems) that can be employed to achieve optimal performance.

Ruby Makes Developers Happy, Until it Comes to Performance…

Why do I love programming in Ruby? First of all, because it’s optimized for developer happiness. Ruby is famous for allowing you to express your ideas in any number of ways. You can write functional programs, or go all in on Java-like enterprise OOP patterns. You can build DSLs, and you can replace or extend whole chunks of the Ruby core functionality by redefining core class methods. And metaprogramming in Ruby lets you do nuclear stuff!

All this comes, of course, with a price tag in the form of reduced performance when compared to other, less magical, programming languages, such as Go, C or C++. But experienced Ruby developers will have already learned how to get the most “bang for the buck” out of Ruby, by carefully designing their code so as to minimize object allocations (and subsequent GC cycles), and picking core Ruby and Stdlib APIs that provide better performance.

Another significant way Ruby developers have been dealing with problematic performance is by using Ruby C-extensions, which implement specific functionalities in native compiled code that can be invoked from plain Ruby code.

Compositional Programming in Ruby

It has occurred to me during my work on Polyphony, a concurrency library for Ruby, that a C-extension API can be designed in such a way as to provide a small, task-specific execution layer for small programs composed of multiple steps that can be exspressed as data structures using plain Ruby objects. Let me explain using an example.

Let’s say we are implementing an HTTP server, and we would like to implement sending a large response using chunked encoding. Here’s how we can do this in Ruby:

def send_chunked_encoding(data, chunk_size)
  idx = 0
  len = data.bytesize
  while idx < len
    chunk = data[idx...(idx += chunk_size)]
    @socket << "#{chunk.bytesize.to_s(16)}\r\n#{chunk}\r\n"
  end
  # send empty chunk
  @socket << "0\r\n\r\n"
end

This is pretty short and sweet, but look how we’re allocating a string for each chunk and doing index arythmetic in Ruby. This kind of code surely could be made more efficient by reimplementing it as a C-extension. But if we already go to the trouble of writing a C-extension, we might want to generalize this approach, so we might be able to implement sending chunked data over other protocols as well.

What if we could come up with a method implemented in C, that takes a description of what we’re trying to do? Suppose we have a method with the following interface:

def send_data_in_chunks(
    data,
    chunk_size,
    chunk_head,
    chunk_tail
  )
end

We could then implement HTTP/1 chunked encoding by doing the following:

def send_chunked_encoding(data, chunk_size)
  @socket.send_data_in_chunks(
    data,
    chunk_size,
    ->(len) { "#{len.to_s(16)}\r\n" }, # chunk size + CRLF
    "\r\n"                             # trailing CRLF
  )
end

If the #send_data_in_chunks method is implemented in C, this means that Ruby code is not involved at all in the actual sending of the data. The C-extension code is responsible for looping and writing the data to the socket, and the Ruby code just provides instructions for what to send before and after each chunk.

Polyphony’s chunked splicing API

The above approach is actually how static file responses are generated in Tipi, the web server for Ruby I’m currently developing. One of Tipi’s distinguishing features is that it can send large files without ever loading them into memory, by using Polyphony’s Backend#splice_chunks API (Polyphony emulates splicing on non-Linux OSes). Here’s an excerpt from Tipi’s HTTP/1 adapter code:

def respond_from_io(request, io, headers, chunk_size = 2**14)
  formatted_headers = format_headers(headers, true, true)
  request.tx_incr(formatted_headers.bytesize)

  Thread.current.backend.splice_chunks(
    io,
    @conn,
    formatted_headers,
    "0\r\n\r\n",
    ->(len) { "#{len.to_s(16)}\r\n" },
    "\r\n",
    chunk_size
  )
end

The Backend#splice_chunks method is slightly more sophisticated than the previous example, as it also takes a string to send before all chunks (here it’s the HTTP headers), and a string to send after all chunks (the empty chunk string "0\r\n\r\n"). My non-scientific benchmarks have shown speed gains of up to 64% for multi-megabyte HTTP responses!

The main idea behind the #splice_chunks API is that the application provides a plan, or a program for what to do, and the underlying system “runs” that program.

Chaining multiple I/O operations in a single Ruby method call

A similar approach was also used to implement chaining of multiple I/O operations, a feature particularly useful when running on recent Linux kernels with io_uring (Polyphony automatically uses io_uring starting from Linux version 5.6.) Here again, the same idea is employed - the application provides a “program” expressed using plain Ruby objects. Here’s how chunked transfer encoding can be implemented using Backend#chain (when splicing a single chunk from an IO instance):

def send_chunk_from_io(io, chunk_size)
  r, w = IO.pipe
  len = w.splice(io, chunk_size)
  if len > 0
    Thread.current.backend.chain(
      [:write, @conn, "#{len.to_s(16)}\r\n"],
      [:splice, r, @conn, len],
      [:write, @conn, "\r\n"]
    )
  else
    @conn.write("0\r\n\r\n")
  end
  len
end

Let’s take a closer look at the call to #chain:

Thread.current.backend.chain(
  [:write, @conn, "#{len.to_s(16)}\r\n"],
  [:splice, r, @conn, len],
  [:write, @conn, "\r\n"]
)

The Backend#chain API takes one or more Ruby arrays each an I/O operation. The currently supported operations are :write, :send and :splice. For each operation we provide the operation type followed by its arguments. The most interesting aspect of this API is that it allows us to reap the full benefits of using io_uring, as the given operations are linked so that they will be performed by the kernel one after the other without the Ruby code ever being involved! The #chain method will return control to the Ruby layer once all operations have been performed by the kernel.

Designing Compositional APIs

This approach to API design might be called compositional APIs - the idea here is that the API provides a way to compose multiple tasks or operations by describing them using native data structures.

Interestingly enough, io_uring itself takes this approach: you describe I/O operations using SQEs (submission queue entries), which are nothing more than C data structures conforming to a standard interface. In addition, as mentioned above, with io_uring you can chain multiple operations to be performed one after another.

Future plans for io_uring include making it possible to submit eBPF programs for running arbitrary eBPF code kernel side. That way, we might be able to implement chunked encoding in eBPF code, and submit it to the kernel using io_uring.

A More General Approach to Chaining I/O operations

It has recently occurred to me that the compositional approach to designing APIs can be further enhanced and generalized, for example by providing the ability to express flow control. Here’s how the chunk splicing functionality might be expressed using such an API:

def respond_from_io(request, io, headers, chunk_size = 2**14)
  formatted_headers = format_headers(headers, true, true)
  r, w = IO.pipe

  Thread.backend.submit(
    [:write, @conn, formatted_headers],
    [:loop,
      [:splice, io, w, chunk_size],
      [:break_if_ret_eq, 0],
      [:store_ret, :len], # store the return code in @len
      [:write, @conn, ->(ret) { "#{ret.to_s(16)}\r\n" }],
      [:splice, r, @conn, :len], # use stored @len value
      [:write, @conn, "\r\n"]
    ],
    [:write, @conn, "0\r\n\r\n"]
  )
end

Now there are clearly a few problems here: this kind of API can quickly run into the problem of Turing-completeness - will developers be able to express any kind of program using this API? Where are the boundaries and how do we define them?

Also, how can we avoid having to allocate all those arrays every time we call the #respond_from_io method? All those allocations can put more pressure on the Ruby GC, and themselves can be costly in terms of performance. And that proc we provide - it’s still Ruby code that needs to be called for every iteration of the loop. That too can be costly to performance.

The answers to all those questions are still not clear to me, but one solution I thought about was to provide a “library” of operation types that is a bit higher-level than a simple write or splice. For example, we can come up with an operation to write the chunk header, which can look something like this:

Thread.backend.submit(
  [:write, @conn, formatted_headers],
  [:loop,
    [:splice, io, w, chunk_size],
    [:break_if_ret_eq, 0],
    [:store_ret, :len],
    [:write_cte_chunk_size, @conn, :len],
    [:splice, r, @conn, :len],
    [:write, @conn, "\r\n"]
  ],
  [:write, @conn, "0\r\n\r\n"]
)

Adding IO References

Another improvement we can make is to provide a way to reference io instances and dynamic strings from our respond_from_io “program” using indexes. This will allow us to avoid allocating all those arrays on each invocation:

# program references:
# 0 - headers
# 1 - io
# 2 - @conn
# 3 - pipe r
# 4 - pipe w
# 5 - chunk_size
RESPOND_FROM_IO_PROGRAM = [
  [:write, 2, 0],
  [:loop,
    [:splice, 1, 4, 5],
    [:break_if_ret_eq, 0],
    [:store_ret, :len],
    [:write_cte_chunk_size, 2, :len],
    [:splice, 3, 2, :len],
    [:write, 2, "\r\n"]
  ],
  [:write, 2, "0\r\n\r\n"]
]

def respond_from_io(request, io, headers, chunk_size = 2**14)
  formatted_headers = format_headers(headers, true, true)
  r, w = IO.pipe
  Thread.backend.submit(RESPOND_FROM_IO_PROGRAM, formatted_headers, io, @conn, r, w)
end

Creating IO Programs Using a DSL

Eventually, we could provide a way for developers to express IO programs with a DSL, instead of with arrays. We could then also use symbols for representing IO indexes:

RESPOND_FROM_IO_PROGRAM = Polyphony.io_program(
  :headers, :io, :conn, :pipe_r, :pipe_w, :chunk_size
) do
  write :conn, :headers
  io_loop do
    splice :io, :pipe_w, :chunk_size
    break_if_ret_eq 0
    store_ret :len
    write_cte_chunk_size :conn, :len
    splice :pipe_r, :conn, :len
    write :conn, "\r\n"
  end
  write :conn, "0\r\n\r\n"
end

Does this look better? I’m not sure. Anyways, there are some rough edges here that will need to be smoothed out for this approach to work.

Implementing a Protocol Parser Using the Compositional Approach

It as occurred to me that this kind of approach, expressing a “program” using plain Ruby objects, to be executed by a C-extension, could also be applied to protocol parsing. I’ve recently released a blocking HTTP/1 parser for Ruby, called h1p, implemented as a Ruby C extension, and I had some ideas about how this could be done.

We introduce a IO#parse method that accepts a program for parsing characters. The program expressed includes a set of steps, each one reading consecutive characters from the IO instance:

# for each part of the line we can express the valid range of lengths,
REQUEST_LINE_RULES = [
  [:read, { delimiter: ' ', length: 1..40, invalid: ["\r", "\n"], consume_delimiter: true }],
  [:consume_whitespace],
  [:read, { delimiter: ' ', length: 1..2048, invalid: ["\r", "\n"], consume_delimiter: true }],
  [:consume_whitespace],
  [:read_to_eol, { consume_eol: true, length: 6..8 } ]
]

HEADER_RULES = [
  [:read_or_eol, { delimiter: ':', length: 1..128, consume_delimiter: true }],
  [:return_if_nil],
  [:consume_whitespace],
  [:read_to_eol, { consume_eol: true, length: 1..2048, consume_delimiter: true }]
]

def parse_http1_headers
  (method, request_path, protocol) = @conn.parse(REQUEST_LINE_RULES)
  headers = {
    ':method' => method,
    ':path' => request_path,
    ':protocol' => protocol
  }

  while true
    (key, value) = @conn.parse(HEADER_RULES)
    return headers if !key

    headers[key.downcase] = value
  end
end

Here too, we can imagine being able to express these parsing rules using a DSL:

REQUEST_LINE_RULES = Polyphony.parse_program do
  read delimiter: ' ', length: 1..40, invalid: ["\r", "\n"], consume_delimiter: true
  consume_whitespace
  read delimiter: ' ', length: 1..2048, invalid: ["\r", "\n"], consume_delimiter: true
  consume_whitespace
  read_to_eol consume_eol: true, length: 6..8
end

It remains to be seen where are the limits to what we can achieve with this approach: can we really express everything that we need in order to parse any conceivable protocol. In addition, it is not clear whether this kind of solution provides performance benefits.

Summary

In this article I have presented an approach to optimizing the performance of Ruby apps by separating the program into two layers: a top layer that written in Ruby, expressing low-level operations using Ruby data structures; and an implementation layer written in C for executing those operations in an optimized manner. This approach is particularly interesting when dealing with long running or complex operations: sending an HTTP response with chunked encoding, parsing incoming data, running I/O operations in loops etc.

As I have mentioned above, this this is similar to that employed by io_uring on Linux. The idea is the same: we express (I/O) operations using data structures, then offload the execution to an lower-level optimized layer - in io_uring’s case it’s the kernel, in Ruby’s case it’s a C-extension.

It seems to me that

This is definitely an avenue I intend on further exploring, and I invite other Ruby developers to join me in this exploration. While we wait for all those exciting Ruby developments I mentioned at the beginning of this article to materialize (the new YJIT effort from Shopify looks especially promising), we can investigate other approaches that take advantage of Ruby’s expressivity while relying on native C code to execute lower level code.

How I Write Code: Pen & Paper

Thu, 02 Sep 2021 00:00:00 GMT

I am a self taught programmer. I first started programming as a kid, and have never bothered to formally study this discipline. To me, programming is first of all a pleasant creative pursuit. It’s a bit like putting together and taking apart all kinds of machines, except these machines happen to be virtual constructions, and you get to keep your hands clean (well, mostly…) Incidentally, this pleasant activity is also how I support my family.

But even though I love programming, I try not to sit in front of a computer screen too much. I do not find staring at a screen all day beneficial, not for my physical health, nor for my mental health. In the last few years, I’ve started a habit of sketching my programming ideas using pen and paper. I’m not talking here about todo lists, or making diagrams. I’m talking about actually writing code using pen and paper. Let me explain.

Why pen & paper

A lot has been written about the advantages of handwriting vs typing. I will not enumerate all of them here, but I will tell you that since I started this practice I find I’m more productive, and I seem to produce better-quality code.

The mere fact that I can concentrate on a single problem without any distractions is already a big advantage, and it seems to me that through writing pages upon pages with a pen, scratching bad ideas, rewriting small bits of code, I gain (as if by magic) a deeper understanding of my code.

What about debugging? What about testing?

When you write code on paper, you have no way to see if your code works. Maybe someday I’ll be able to handwrite code on an e-Ink device and then run it. Until that day, all that I have is my ideas, my intuition, my knowledge, and my “mental runtime” - a mental thought process that simulates some kind of computer runtime that goes through the code, evaluating it.

Of course, my mental process is not perfect. It will let slip through all kinds of bugs, things I’m not smart enough to detect before actually feeding the code to a computer. That’s why I try to concentrate on one small problem at a time. Nothing bigger than a page or two, and mostly short snippets that implement a specific algorithm.

Iterate, iterate, iterate

So once I start exploring a problem space with pen & paper, I just iterate on it until I feel I’ve found the best solution to the problem. Once I’ve achieved that, I can finally open my laptop and type the code into my favorite editor.

Sometimes it works like magic, I just type in the code and everything works. Sometimes it needs some additional effort in editing and elaborating. But still, even if my handwritten code turned out to be only partially correct, or needed some reworking for it to fit in with other components, I feel like spending time sketching code and reflecting on it before turning to the computer has given me a deeper understanding of the code.

A recent example

Here’s an example of coding I did completely with pen & paper: a few days ago I published a new open-source project called Ever - a libev-based event reactor for Ruby. The entire design was done with pen & paper. I did it over three evenings, taking a couple hours each evening to work through the design and different aspects of the implementation.

Finally, when I felt the design was solid and that I had resolved all the issues I could see, I opened my laptop, created a new Github repository, typed in the code, added some tests and wrote the README. It was probably all done in 4 or 5 hours of concentrated work.

At the end of the process, in addition to being able to create a small Ruby gem in a single day without too much fussing around looking for solutions to problems encountered during development, I feel like I have a deeper knowledge of the code, a deeper understanding of the implications of different choices, and a greater appreciation for writing the least amount of code.

When I hold a pen in my hand and set it to paper, I find it much easier to be “in the zone”, and I feel much closer to the ideas I’m exploring. Somehow, I never get this feeling of connectedness when laying my hands on a computer keyboard. It also empowers me to know that I don’t need a computer in order to create code. I’m not dependent in my creative pursuits on a machine that sometimes seems to disturb my creative process rather than facilitate it.

Steve Jobs once talked about computers being like a bicycle for our minds. To me it feels like a lot of times we expend lots of energy on bikeshedding rather than actually riding our bikes. And maybe we should also get off our bikes every once in a while, and just walk.

What's new in Polyphony and Tipi - August 2021 edition

Thu, 26 Aug 2021 00:00:00 GMT

The summer is drawing to an end, and with it I bring another edition of Polyphony (and Tipi) news, this time on my own website, where I’ll be publishing periodically from now on.

Polyphony is a library for writing highly concurrent Ruby apps. Polyphony harnesses Ruby fibers and a powerful io_uring-based I/O runtime to provide a solid foundation for building high-performance concurrent Ruby apps.

Tipi is a new Polyphony-based web server for Ruby. Tipi provides out of the box support for HTTP/1, HTTP/2, and WebSocket. Tipi also provides SSL termination (support for HTTPS) with automatic certificate provisioning and automatic ALPN protocol selection.

From counterpoint to composition

In the last month I’ve been doing a lot more work on Tipi than on Polyphony, and most of my work on Polyphony has been just fixing bugs. For me this is a major milestone, as I’m transitioning from working on the low-level stuff, to an actual application that can do something useful. To me this feels a bit like transitioning from writing counterpoint exercises to composing an actual piece of music.

The Polyphony API is maturing nicely and I hope to be able to make a 1.0 release in the coming weeks. As for Tipi, there’s still a lot of work to do in order for it to be ready for public release, and I’ll discuss that towards the end of this post.

Changes in Polyphony

Support for splicing on non-Linux platforms

The libev backend now supports all splicing APIs by emulating the Linux splice system call. I’ve already written about splicing and the amazing things that can be done with these APIs. So now they can be used on cross-platform, even if the performance gains are only achievable on Linux.

Fiber supervision

One of the major advantages of using Polyphony as the basis for concurrent programs is that it implements structured concurrency, a programming paradigm that makes it easier to control fiber execution in a highly-concurrent environment. Just imagine writing a program that performs thousands of long-running tasks concurrently. How do you manage that complexity? How do you deal with failures? How can you control any of those concurrent tasks?

Polyphony deals with this problem by adhering to three principles:

Fibers are arranged in a hierarchy: a fiber is considered the child of the fiber from which it was spun.
A fiber’s lifetime is limited to that of its immediate parent. In other words, a fiber is guaranteed to terminate before its parent does.
Any uncaught exception raised in a fiber will “bubble up” to its immediate parent, and potentially all the way up to the main fiber (which will cause the program to terminate with an exception, if not handled.)

Here’s an example to demonstrate these three principles in action:

# Kernel#spin starts a new fiber
@controller = spin do
  @worker = spin do
    loop do
      # Each fiber has a mailbox for receiving messages
      peer, op, x, y = receive
      result = x.send(op, y)
      # The result is sent back to the "client"
      peer << result
    end
  end
  # The controller fiber will block until the worker is done (but notice that
  # the worker runs an infinite loop.)
  @worker.await
end

def calc(op, x, y)
  # Send the job to the worker fiber...
  @worker << [Fiber.current, op, x, y]
  # ... and wait for the result
  receive
end

# wait for the controller to terminate
@controller.await

In the above example, we spin a controller fiber, which then spins a worker fiber. This creates the following hierarchy:

main
 |
 +- controller
      |
      +-- worker

Now we can just call #calc to perform calculations inside the worker:

# from the main fiber
calc(:+, 2, 3) #=> 5

# or from another fiber:
f = spin { calc(:**, 2, 3) }
f.await #=> 8

But notice what happens when we send an operation that results in an exception:

calc(:+, 2, nil)

Traceback (most recent call last):
        5: from examples/core/calc.rb:7:in `<main>'
        4: from examples/core/calc.rb:8:in `block in <main>'
        3: from examples/core/calc.rb:9:in `block (2 levels) in <main>'
        2: from examples/core/calc.rb:9:in `loop'
        1: from examples/core/calc.rb:12:in `block (3 levels) in <main>'
examples/core/calc.rb:12:in `+': nil can't be coerced into Integer (TypeError)

Actually, the exception that was raised inside of the worker fiber, has bubbled up to the controller fiber. The controller, which was busy waiting for the worker to terminate, has re-raised the exception, which bubbled up to the main fiber. The main fiber, which was waiting for the controller to terminate, has re-raised the exception and has finally exited with an error message and a back trace (you can find the full example here).

The fact that unrescued exceptions bubble up through the fiber hierarchy allow us to control the lifetime of child fibers. Here’s one way we can deal with uncaught exceptions in the worker fiber:

@controller = spin do
  @worker = spin { ... }
  @worker.await
rescue => e
  puts "Uncaught exception in worker: #{e}. Restarting..."
  # Yes, in Polyphony fibers can be restarted!
  @worker.restart
end

Since the controller fiber can intercept any unrescued exception that occurred in its child, we add a rescue block, report the error and then restart the fiber.

Another possibility would be to handle the error at the level of the main fiber, or maybe to handle it locally if it’s only about trivial errors, and let more serious exceptions bubble up - it really depends upon the circumstances. The point is that Polyphony allows us to control the lifetime of any fiber anywhere in the fiber hierarchy with a small set of tools that builds on the rubustness of Ruby exceptions: putting rescue and ensure blocks in the right places will already do 99% of the work for us.

But what if we want to automate the handling of errors? What if we just want things to continue working without us needing to manually write rescue blocks? Enter fiber supervision.

Inspired by Erlang

The new fiber supervision mechanism in Polyphony is greatly inspired by Erlang supervision trees. While Erlang processes are not organised hierarchically, Erlang provides a supervisor behaviour that allows expressing hierarchical dependencies between processes.

While a lot of the functionality of Erlang supervision trees is already included in Polyphony by virtue of the structured concurrency paradigm, the Erlang supervisor behaviour allows automating the handling of error conditions. This is what I set to solve in the new fiber supervision API.

The new Kernel#supervise method can be used to supervise one or more fibers. By default, it does nothing more than just waiting for all supervised fibers to terminate. But it can also be used to automatically restart fibers once they have terminated, or restart them only when an exception occurred, or to perform other work when a fiber is terminated (for example, writing to a log file).

Going back to our example, here’s how we can use the controller fiber to supervise the worker fiber:

@controller = spin do
  @worker = spin { ... }
  supervise(@worker, restart: :always)
end

The call to Kernel#supervise tells the controller fiber to monitor the worker fiber and to restart it always once it terminates, ignoring any exceptions. Alternatively, we can tell the controller to restart the worker only when an exception occurs:

supervise(@worker, restart: :on_error)

We can also define a custom behavior by passing a block that will be called when the worker fiber terminates:

supervise(@worker) do |fiber, result|
  log_exception(result) if result.is_a?(Exception)
  fiber.restart
end

Staying in the loop: the advantages of fiber supervision

In my work on Polyphony and on Tipi I have discovered a few programming patterns that I find very interesting:

When using Polyphony you write a lot of loops. A good bunch of those are infinite loops! Take for example the worker fiber above.
When developing a concurrent app using Polyphony, any uncaught exception might cause the entire process to terminate, since Polyphony never allows exceptions to be lost.

If we look at Tipi, a Polyphony app that can be used to serve HTTP/S on multiple ports, we’ll have a separate fiber listening for incoming connections on each port. When a connection is accepted, we spin a new fiber in order to handle the new connection concurrently:

http_listener = spin do
  while (conn = http_server.accept)
    spin { handle_client(conn) }
  end
end

https_listener = spin do
  while (conn = https_server.accept)
    spin { handle_client(conn) }
  end
end

Since the client handling fibers are spun from the listener fibers (either http_listener or https_listener), they are considered the children of those fibers. If any exception is raised in a client handling fiber and is not rescued, it will bubble up to the listener fiber and will cause it to terminate with the exception.

In addition, the listeners themselves might raise exception when accepting connections - these can be system call errors, I/O errors, OpenSSL errors (for the HTTPS listener) etc. We’d like an easy way to catch these errors. One way would be to just do this with a rescue block:

...

https_listener = spin do
  loop do
    conn = https_server.accept
    spin { handle_client(conn)
  rescue => e
    puts "HTTPS accept error: #{e.inspect}"
  end
end

This is a possibility, but we need to do it manually for each fiber, and we risk adding a lot of rescue blocks (some of them can even be for a specific class of exception) everywhere, an error-prone methodology that can prove problematic if overdone.

Instead, we can use the Kernel#supervise API provided by Polyphony to make sure our infinite loops (i.e. our listener fibers) continue running, even when an exception occurs. Thus we can embrace the Erlang moto: “Let it crash.” We let it crash, and then we restart it. Here’s how we can employ this using fiber superivision:

http_listener = spin(:http) { ... }
https_listener = spin(:https) { ... }
# If specific fibers are not specified, #supervise will supervise all of the
# current fiber's children.
supervise do |fiber, result|
  if result.is_a?(Exception)
    puts "Fiber #{fiber.tag} terminated with exception: #{result}. Restarting..."
    fiber.restart
  end
end

In this way we ensure that any uncaught exception from one of the listeners or their children will not slip through and stop the server from functioning. Any listener that has stopped because of an exception will just be restarted. And applying this to our controller example above:

@controller = spin do
  @worker = spin do
    loop do
      peer, op, x, y = receive
      result = x.send(op, y)
      peer << result
    end
  end
  supervise(@worker, restart: :always)
end

def calc(op, x, y)
  # Send the job to the worker fiber...
  @worker << [Fiber.current, op, x, y]
  # ... and wait for the result
  receive
end

supervise(@controller, restart: :always)

Bug fixes and other changes

Here’s a list of other, smaller changes and fixes in Polyphony:

Prevent possible segfault in the io_uring backend by GC marking read/write buffers when cancelling an I/O operation. When cancelling an IO operation, the Ruby process might have already moved on while the kernel is still accessing the associated buffers before finally cancelling the operation. In order to prevent a possible segfault in case a GC cycle kicks in immediately after cancellation, I have introduced a mechanism to GC mark the buffers used until the operation has been cancelled in the kernel as well.
Improve fiber monitoring: in preparation for work on fiber supervision, the Fiber#monitor implementation has undergone a lot of simplification and made much more robust.
Fiber#attach was renamed to Fiber#attach_to.
Fixed linking of operations in Backend#chain.
Fixed missing default value in #readpartial for socket classes.
Fixed removing child fiber from parent’s children list when terminated.
Reset backend runqueue and other state after forking.
Fix compilation on Ruby 3.0.

Changes in Tipi

The Tipi server is progressing nicely. I’ve been running it in production over the last few months, and while it’s still a long way from providing a stable, easy-to-use API for other developers, in terms of features and functionality it’s already got 90% of the features expected from a modern web server: support for HTTP/1 and HTTP/2, SSL termination, support for WebSocket and streaming responses, support for serving static files and of course running Rack apps. Tipi is also able to dynamically provision SSL certificates using an ACME provider (such as Let’s Encrypt), though this feature is still work in progress.

Following is a summary of the big changes in Tipi this month.

H1P - a new HTTP/1 parser

I’ve hinted before about writing an HTTP/1 parser made for Tipi. Well the work is more or less done, and I’ve released the parser as a separate project called H1P. What sets this parser apart is the fact that it is completely blocking. While other parsers (at least the ones I know of) provide a callback-based API, where you register callbacks for different events, and then feed the parser with data and wait for those callbacks to be invoked, by contrast H1P provides a blocking API that’s much easier to use:

conn = server.accept
parser = H1P::Parser.new(conn)
headers = parser.parse_headers
body = parser.read_body
handle_request(headers, body)

Yes, that’s it (for the most part). And, the beauty of this parser is that you don’t even need Polyphony in order to use it. In fact you can use it in a “normal” threaded server (spawning a thread for each connection), and you can use it in conjunction with the new fiber scheduler introduced in Ruby 3.0.

The H1P parser is implemented in less than 750 lines of C, has zero dependencies and supports chunked encoding and LF/CRLF line breaks, has hard limits on token length for minimizing server abuse, and is transport agnostic - you can have it read from any source, even sources that are not IO objects:

data = ['GET ', '/foo', " HTTP/1.1\r\n", "\r\n"]
parser = H1P::Parser.new { data.shift }

parser.parse_headers
#=> { ':method' => 'GET', ':path' => '/foo', ... }

I intend to keep on working on H1P, notably on the following:

Add conformance and security tests.
Add ability to parse HTTP/1 responses (for implementing HTTP clients.)
Add ability to splice the request body into an arbitrary fd (Polyphony-specific.)
Improve performance. In a synthetic benchmark, H1P is ~15% slower than the callback-based http_parser.rb gem, which uses the old node.js HTTP/1 parser. In actual use in Tipi, I’ve seen the throughput actually improve with H1P, and I think this is attributable to the fact that when converting from a callback-based API to a blocking API (as I do in Tipi) there’s quite a bit of overhead involved in buffering the requests, and all the more so when needing to deal with HTTP pipelining.

In addition to that, I also plan to implement a similar H2P project for handling HTTP/2 connections.

Automatic SSL certificate provisioning

If there’s one feature that can be a game changer for a Ruby web server, it’s automatic SSL certificate provisioning. Tipi already does SSL termination, and that makes it possible to use Tipi without any load balancer or reverse proxy in front of it, since it can deal with incoming HTTPS connections all by itself. But automatic SSL certificates take this autonomy to the next level: you don’t even have to provide a certificate for Tipi to use. Tipi will just take care of it all by itself, by dynamically provisioning a certificate from an ACME provider, such as Let’s Encrypt or ZeroSSL.

Imagine not having to set up Nginx, Apache or Caddy as a reverse proxy in order to run your web app. You just run Tipi (preferably with port-forwarding, so you don’t need to deal with binding to privileged ports) and point it at your Rack app. This is what I’m aiming to achieve in the near future.

So automatic certificates already work in Tipi. In fact, this very website, which I’ve put together a few weekends ago, already uses automatic certificates. While it works, there’s still a lot of details to take care of: testing, handling of failures, adding more ACME providers, and finally coming up with a simple API for configuring automatic certificates.

Other changes

In addition to the above big new features, I’ve also worked on the following:

Fixed upgrading to HTTP/2 with a request body.
Start work on CLI interface. Nothing to announce for the moment, but what I’m aiming for is a command line tool that can be used to serve static files in any directory, serve a rack app, or a custom app using Tipi’s API, with automatic certificates.

What’s next for the Polyphony ecosystem?

In the last few years I’ve been creating a set of Ruby gems that I call Digital Fabric, with the moto: “Software for a better world.” I believe in empowering small developers to build lightweight, autonomous digital systems to solve specific needs. The Digital Fabric suite already includes tools for working with SQLite databases, creating HTML templates, and managing dependencies, in addition to Polyphony and Tipi.

I’m a long-time Ruby programmer, and to date my most substantial contribution to the Ruby community is Sequel, of which I’m the original author. The same spirit that guided me in creating Sequel is the one that’s currently guiding me in working on the Digital Fabric suite of tools: create simple and powerfull APIs that make developers happy and that feel like natural extensions of the Ruby programming language. I believe Polyphony and Tipi have the potential to unleash a new wave of creativity in the Ruby community!

Here’s some of the things I intend to work on in the near future:

Further work on the H1P parser, as discussed above.
Start work on H2P - a library for dealing with HTTP/2 connections.
Continue work on automatic certificates in Tipi.
Prepare Polyphony for a 1.0 release.
Setup a websites for both Polyphony and Tipi.
Continue work on Impression, a new experimental web framework, which incidentially I’m using for this site. It’s just a sketch for the moment, but I have a bunch of ideas that that I’d like to test, but it’s still too early to tell if it will turn into a real project.
And finally, prepare Tipi for a first public release once the CLI tool is good enough.

What's new in Polyphony - July 2021 edition

Tue, 27 Jul 2021 00:00:00 GMT

Following last month’s update, here’s an update on the latest changes to Polyphony:

Redesigned tracing system
New methods for changing fiber ownership
Support for appending to buffers when reading
Improved backend statistics
Improved control over reading with #read and #readpartial

Redesigned tracing system

In previous versions, Polyphony included extensions to the core Ruby TracePoint API, so that events such as switching fibers, scheduling fibers, polling for I/O completions, could be traced using the same TracePoint API that’s used for tracing method calls, variable access etc. In Polyphony 0.59 the tracing system was completely overhauled and separated from TracePoint. Polyphony backend events can now be traced by calling Backend#trace_proc:

Thread.backend.trace_proc = proc { |*event| p event }

Events are fed to the tracing proc as plain Ruby arrays, where the first member signifies the type of event. Currently the following events are emitted:

[:spin, fiber]
[:schedule, fiber, resume_value, is_priority]
[:block, current_fiber]
[:unblock, fiber, resume_value]
[:enter_poll, current_fiber]
[:leave_poll, current_fiber]

New methods for changing fiber ownership

Polyphony follows the structured concurrency paradigm, where the lifetime of each fiber is limited to that of the fiber from which it was spun. This mechanism, also called a parent-child relationship, permits developers to spin up thousands of fibers in a structured and controlled manner. Version 0.60 introduces two new methods which allow you to change the parent of a fiber.

Fiber#detach sets the fiber’s parent to the main fiber. This method could be useful if you need a fiber to outlive its parent:

parent = spin do
  child = spin do
    do_something
  end
  child.detach
  child.parent #=> Fiber.main
end

parent.await
# parent is dead, but child is still alive!

Fiber#attach lets you set the fiber’s parent to any other fiber, which might be useful if you start a fiber in some context but then need it to be limited by the lifetime of another fiber:

worker_parent = spin { sleep }

fiber_maker = spin_loop do
  job = receive
  worker = spin { perform(job) }
  worker.attach(worker_parent)
end

# at some point we want to stop all workers
worker_parent.terminate # boom, all workers are terminated

Support for appending to buffers when reading

Up until now, the backend read/recv APIs allowed you to provide a buffer and read into it, replacing any content it may hold. A major change introduced in version 0.60 allows reading to any position in the provided buffer, including appending to the buffer. The Backend#read and Backend#recv methods now accept a buffer_pos argument:

# append to a buffer
i, o = IO.pipe

spin do
  o << 'foo'
  o << 'bar'
  o << 'baz'
end

buffer = +''
# buffer_pos is the last argument. -1 denotes the end of the buffer
Polyphony.backend_read(i, buffer, 3, false, -1)
buffer #=> 'foo'
Polyphony.backend_read(i, buffer, 3, false, -1)
buffer #=> 'foobar'
Polyphony.backend_read(i, buffer, 3, false, -1)
buffer #=> 'foobarbaz'

This addition may seem minor but what it allows us to do, beyond not needing to concatenate strings, is to write parsers that are competely blocking. I’m currently writing a custom HTTP/1 parser for Tipi that’s based on this unique feature and which promises to significantly improve both throughput and memory usage. (I’ll discuss this new parser in detail in another post.)

Improved backend statistics

Polyphony version 0.61 has introduced streamlined and more comprehensive backend statistics, now accessible using Backend#stats. The statistics are returned as a hash with the following keys:

:runqueue_size: the size of the run queue
:runqueue_length: the number of fibers currently in the runqueue
:runqueue_max_length: the max number of fibers in the runqueue since the last call to #stats
:op_count: the number of backend operations since the last call to #stats
:switch_count: the number of fiber switches since the last call to #stats
:poll_count: the number of times the backend polled for completions since the last call to #stats
:pending_ops: number of operations currently pending

Improved control over reading with `#read` and `#readpartial`

Finally, Polyphony versions 0.63 and 0.64 have added optional arguments to IO#read and IO#readpartial in order to allow developers to have more flexibility and to use the new “append to buffer” feature discussed above. Here are the updated signatures for those methods (they apply also to all socket classes):

IO#read(len = nil, buf = nil, buf_pos = 0)
IO#readpartial(len, str = +'', buffer_pos = 0, raise_on_eof = true)

Note the raise_on_eof argument, which can be used to control whether #readpartial raises an EOFError when an EOF is encountered.

What’s next for Polyphony

As I wrote above, I’m currently developing a custom HTTP/1 parser for Tipi, which already has promising performance characteristics, reduces dependencies and is completely synchronous (i.e. no callbacks are involved). I hope to be able to switch Tipi to using the new parser in the coming weeks and having it battle tasted in one of my production projects, and then to continue to write a HTTP/2 parser with a similar design.

What's new in Polyphony - June 2021 edition

Fri, 25 Jun 2021 00:00:00 GMT

Polyphony 0.58 has just been released. Here’s a summary and discussion of the latest changes and improvements:

Improved functionality for OpenSSL sockets and servers.
Fixes to the Mutex class.
A redesigned event anti-starvation algorithm.
A new API for splicing to/from pipes.
A new API for chaining multiple I/O operations.
New APIs for performing GC and other arbitrary work when the process is idle.

Improved functionality for OpenSSL sockets and servers

Following the work I’ve lately been doing on the Tipi server (you can read more about that towards the end of this post), I’ve made significant improvements to working with OpenSSL encrypted servers and client sockets. Here are some of the changes:

Add SSLSocket#recv_loop method (#54).
Add SSLServer#accept_loop method.
Override SSLSocket#peeraddr to support the same arity as Socket#peeraddr (#55) (this is an inconsistency at the level of the Ruby stdlib.)

Fixes to the `Mutex` class

Following a bug report from @primeapple, trying to use Polyphony in a Rails project (!), some missing methods were added to the Polyphony implementation of the Mutex class: #owned? and #locked?.

It is still too early to tell if Polyphony can be used to drive a Rails app, and frankly I am not actively trying to make it happen, but I’d love to receive more feedback on how Polyphony interacts with different parts of the Rails ecosystem.

A redesigned event anti-starvation algorithm

Polyphony is at its core an efficient fiber scheduler that eschews the traditional non-blocking design that wraps the entire program in one big event loop. Instead, when a fiber needs to perform some long-running operation, it simply passes control to the next fiber on the run queue. If the run queue is empty, the backend is polled for completion of events or operations. This is a blocking call that will return only when one or more events are available for the backend to process. Each completed event or operation will cause the corresponding fiber to be scheduled, and to be eventually resumed.

Under certain circumstances, though, if the runqueue is never empty (because fibers are kept being scheduled regardless of I/O events), this will prevent Polyphony from polling for events, leading to event starvation.

In order to prevent this from happening, Polyphony includes a mechanism for periodically performing a non-blocking poll, which assures the processing of ready events, even under such conditions. For the libev backend, this is done by calling ev_run(backend->ev_loop, EVRUN_NOWAIT), which will only process ready events without waiting. For the io_uring backend this done by simply processing the available CQEs without issuing a io_uring_enter system call.

Now, until Polyphony version 0.54, determining when to use this mechanism was problematic, and was based on false assumptions. In Polyphony 0.55 the algorithm for determining when to make a non-blocking poll was redesigned, and is now based on counting the number of times fibers have been switched, as well as keeping a high water mark for the number of fibers in the run queue:

A switch counter is incremented every time a fiber is pulled off the run queue.
The switch counter is reset when the run queue is empty.
A high water mark is updated every time it’s exceeded by the run queue size (when adding fibers to the run queue).
The non-blocking poll is performed when both the high water mark and the switch count reach certain thresholds (currently set at 128 and 64 respectively).

If this tickles your interest, you can have a look at the code.

A new API for splicing to/from pipes

The Linux kernel includes a relatively little known system call called splice, which lets developers move data from one file descriptor to another (for example, from a file to a socket) without needing to copy data back and forth between userspace and the kernel, a costly operation, and in some cases even without copying data inside the kernel itself, by using pipes (which act as kernel buffers). To learn more about splice and what it’s good for, read this explanation by Linus Torvalds himself.

Starting from Polyphony 0.53, I’ve been gradually adding support for splicing to and from I/Os on both the libev and io_uring backends. The following APIs were added:

Backend#splice(src, dest, maxlen)
Backend#splice_to_eof(src, dest, chunk_size)

In addition, the corresponding methods have been added to the IO class:

IO#splice(src, maxlen) - returns the number of bytes written
IO#splice_to_eof(src, chunk_size = 8192) - returns the number bytes written

So we know that to splice we need to use a pipe, either for the source or the destination or for both, but how do we use it in practice? Suppose we want to write the content of a file to a socket. Here’s one way we can do this with splice:

def write_file_to_socket(socket, path)
  r, w = IO.pipe
  File.open(path, 'r') do |f|
    spin do
      w.splice_to_eof(f)
    ensure
      w.close
    end
    socket.splice_to_eof(r)
  end
end

In the above example we create a pipe, and then we spin up a separate fiber that will splice from the file to the pipe, while the current fiber splices from the pipe to the socket. This technique can be used for files of arbitrary size (even GBs), without loading the file content into Ruby strings and putting pressure on the Ruby garbage collector. On top of this, we do this concurrently and with automatic backpressure (i.e. our socket will not get inondated with MBs of data.)

While the splice system call is only available on Linux, the libev backend includes fake implementations of Backend#splice and Backend#splice_to_eof done with plain read and write calls.

In addition to the above new methods, Polyphony 0.57 also introduces the Backend#splice_chunks method, which can be used for splicing chunks to some arbitrary destination IO instance, interespersed with writing plain Ruby strings to it. The use case arose while working on the Tipi web server, and trying to optimize serving static files on the web without loading the file content in Ruby strings. The Tipi HTTP/1.1 adapter tries whenver possible to use chunked encoding. In HTTP/1.1 for each chunk there should be a header including the chunk size, followed by the chunk itself, and finally a \r\n delimiter. In order to abstract away the creation of a pipe (for use with splicing) and the looping etc, I introduced the following method:

Backend#splice_chunks(src, dest, prefix, postfix, chunk_prefix, chunk_postfix, chunk_size)

… with the following arguments:

src - source IO instance
dest - destination IO instance
prefix - prefix to write to destination before splicing from the source
postfix - postfix to write to destination after splicing is done
chunk_prefix - the prefix to write before each chunk (a string or a proc)
chunk_postfix - the postfix to write after each chunk (a string or a proc)
chunk_size - the maximum chunk size

The chunk prefix and postfix can be a Proc that accepts the length of the current chunk, and returns a string to be written to the destination. Here’s how this new API is used in Tipi to serve big files:

# Edited for brevity
def respond_from_io(request, io, headers, chunk_size = 2**14)
  formatted_headers = format_headers(headers)
  Thread.current.backend.splice_chunks(
    io,
    @conn,
    # prefix: HTTP headers
    formatted_headers,
    # postfix: empty chunk denotes end of response
    "0\r\n\r\n",
    # dynamic chunk prefix with the chunk length
    ->(len) { "#{len.to_s(16)}\r\n" },
    # chunk delimiter
    "\r\n",
    chunk_size
  )
end

As the example demonstrates, this allows sending chunks from arbitrary IO instances (be it files or sockets or STDIO or pipes) without any of the data passing through the Ruby runtime, and the API is concise but also allows lots of flexibility. We can imagine using this API to send HTTP/2 data frames without much difficulty.

While the libev backend is more or less straightforward - doing splicing and writing sequentially one after the other, the io_uring backend implementation benefits from being able to issue multiple ordered I/O operations at once using the IOSQE_IO_LINK flag. This allows us to further minimize the number of system calls we make.

But what of the performance implications? Does using this technique result in any noticable improvements to performance? It’s still too early to tell how using this technique will affect the numbers in a real-world situation, but preliminary benchmarks for serving static files with Tipi show a marked improvement for files bigger than 1MB:

File size	Normal - req/s	Spliced - req/s	Change
1KB	8300	7568	-8%
64KB	7256	5702	-21%
1MB	730	768	+5%
4MB	130	189	+45%
16MB	28	46	+64%
64M	9	12	+33%
256MB	2	3	+50%

This benchmark was done using the io_uring backend, using wrk with the stock settings, i.e. 2 threads and 10 concurrent connections, on an lowly EC2 t3.xlarge machine.

A new API for chaining multiple I/O operations

Another big new feature introduced in Polyphony 0.55 is chaining of multiple ordered I/O operations using the Backend#chain API, which allows developers to specify multiple (outgoing) I/O operations in a single call to the backend, in order to minimize the overhead involved in going back and forth between the fiber issuing the I/O operations and the backend.

While Polyphony can already write multiple strings to the same file descriptor with a single method call (using writev), this new API allows developers to perform multiple I/O operations on different file descriptors in a single method call.

Here as well, the io_uring backend can reap additional performance benefits by issuing multiple ordered I/O operations using a single system call, without having to wakeup the fiber after each I/O operation, in a similar fashion to the Backend#splice_chunks API we just discussed.

The Backend#chain method takes one or more operation specifications expressed using plain arrays. Here’s a simplified version of Backend#splice_chunks implemented using the #chain API:

def splice_chunks_in_ruby(src, dest, prefix, postfix, chunk_size)
  r, w = IO.pipe
  while true
    len = w.splice(src, chunk_size)
    break if len == 0

    chain(
      [:write, dest, prefix],
      [:splice, r, dest, len],
      [:write, dest, postfix]
    )
  end
end

The following operation specifications are currently supported:

[:write, destination, data]
[:send, destination, data, flags] - for sockets only
[:splice, source, destination, len]

New APIs for performing GC and other arbitrary work when idle

When running web servers in production, I’d like not only to maximize the server’s throughput (expressed in requests per second), but also minimize latency. And when we talk about latency we also need to talk about percentiles. One of the things that can really hurt those 99th percentile latency numbers in Ruby web servers is the fact that the Ruby runtime needs to perform garbage collection from time to time, and normally this garbage collection event is both slow (costing tens of milliseconds or even more), and can come at any time, including while processing an incoming request.

In order to prevent garbage collection from happening while your server is busy preparing a response, a technique called out-of-band GC, or out-of-band processing, consists of disabling the garbage collector, and manually running a GC cycle when the server is otherwise idle (i.e. not busy serving requests.)

Polyphony 0.58 introduces new APIs that allow you to perform garbage collection or run any code only when the process is otherwise idle (i.e. when no fibers are scheduled.) Here are the new APIs:

Thread#idle_gc_period= - sets the period (in seconds) for performing GC when idle.
Thread#on_idle { ... } - installs a block to be executed whenever the system is idle.

Here’s how you can set automatic GC when idle:

# Here we set the minimum interval between consecutive GC's done only when the
# thread is otherwise idle to 60 seconds:
Thread.current.idle_gc_period = 60

GC.disable

You can also run an arbitrary block of code when idle by passing a block to Thread#on_idle:

Thread.current.on_idle do
  do_something_really_unimportant
end

What’s next for Polyphony?

Polyphony has been in development for almost three years now, and its API is slowly stabilizing. I’d like to be able to release version 1.0 in a few month but I still have some work left before we arrive there, including:

Support for UDP sockets.
Support for IPv6.
Redesign the fiber supervision API.

As for actual applications using Polyphony, I am continuing work on the Tipi web server, which is already used in production (in a closed-source product for one of my clients), and which already knows how to do HTTP/1.1, HTTP/2, Websockets and SSL termination.

I am currently working on two really exciting things:

Add automatic SSL certificate provisioning to Tipi, using the ACME protocol with providers such as Let’s Encrypt. This will allow developers to run Tipi as an all-in-one app- and web server, without needing to use a reverse proxy such as Nginx or Caddy at all!
Implement a modern HTTP client for Ruby based on Polyphony, with features such as persistent connections by default, automatic support for HTTP/2 connections, sessions, caching and persistent cookies to name but a few.