Sat 30 November 2024

🌫️ Sane C++ November 24

SC

Welcome to the update post for November 2024!
Main focus of the month is the addition of Async Streams!

📄 Sane C++ Libraries Documentation
💻 Sane C++ Libraries Repository
📹 Sane Coding Youtube channel
💬 Sane Coding Discord

SC::AsyncStreams#

Pushing it forward

The SC::Async library has been extended with the addition of SC::AsyncStreams!
Async Streams are largely inspired by node.js Streams, a very powerful tool to process large amounts of data in parallel.
The basic idea about an async stream is to create a Source / Sink abstraction (also called Readable and Writable) and process small buffers of data at time.
The state machine that coordinates this interaction handles data buffering and more importantly handles also back-pressure, that means:

  • Pausing the readable stream when a connected writable stream cannot process data fast enough
  • Resuming the readable stream when a connected writable stream is finally able to receive more data
By implementing streams on top of async operations it's possible to run many of them concurrently very efficiently.
When properly implemented for example an async pipeline can concurrently read from disk, write to a socket while compressing data.
For now only Readable / Writable File and Socket streams have been implemented, but Async Transform Streams (for compression) will be next!
Most notable differences with node.js streams are for now:
  • No allocation (designed to work inside user-provided list of buffers)
  • No object mode
  • Fixed Layout to create data pipelines (AsyncPipeline)
  • onData support only (no readable event)
Async Streams are for now in 🟥 Draft state. It's also possible that its API will evolve a little bit to be less verbose and there is also lack of nice examples, aside from the tests.
It's better waiting for it to become stable before doing anything significant with it.
Some changes and fixes in the SC::Async library have been made necessary to support Async Streams.

This is the list of related commits:

Additional fixes#

Fix fix fix

And just like every update, a bunch of fixes and improvements to all libraries have been committed. Nothing specific stands out this update, but for completeness this is the list of related commits: