SC::AsyncStreams
Most of the work has been focused on shaping the SC::AsyncStreams library.
Async streams got their own folder and the AsyncPipeline API has been improved so that it accept transform streams.
Transform streams are used to concurrently modify data read from a source, before it's being written to one or multiple destinations.
The sample use case has been compression through ZLIB library.
In order to keep Sane C++ Libraries free of build time dependencies, zlib is dynamically loaded on all major operating systems.
A nice trick has been figuring out that the .NET CLR on Windows ships zlib hidden in the clrcompression.dll, avoiding the need to deploy it.
C/C++ compression tip: Avoid deploying zlib binary with your executable, it's already included in the OS. macOS ▶️ libz.dylib Linux ▶️ libz.so.1 (on normal distros) Windows ▶️ clrcompression.dll (.NET CLR) exports: - deflate - deflateEnd - inflate - inflateEnd - deflateInit2_ - inflateInit2_ Enjoy!😏
— Stefano Cristiano (@pagghiu.bsky.social) December 17, 2024 at 10:14 PM
It seems that the fixed pool initial design constraint, that forbids any dynamic allocation inside Async Streams is still working. Some more work will need to be carried on to really prove if it works on a significant number of use cases!
Two Videos have been recorded while implementing the most significant portions of SC::AsyncTransformStream
Detailed List of commits:
- Move Async Streams to a dedicated folder
- Improve documentation
- Use threaded blocking IO for file operations in test
- Add an async zlib transform stream
- Replace Pipe with a Span of AsyncWritableStream
- Add method to remove all listeners bound to a specific class instance
- Fix pause handling inside AsyncRequestReadableStream
- Allow extending writable ending state until needed
- Add unpipe to remove listeners added by pipe
- Add CircularQueue::pushFront
- Make AsyncPipeline end writable sinks when readable source ends
- Add errors listener directly on the AsyncPipeline in the test
- Add ZLibAPI and ZLibStream
- Add Event::removeListener member function overload
- Implement write resuming and pushing a buffer to the top of the write queue
- Add auto-close descriptor for request streams
- Add AsyncTransformStream and a simple synchronous zlib stream
- Add AsyncPipeline::pipe to validate requested pipeline
SC::Async
SC::Async gets its usual set of monthly minimal improvements.
The most significant one is change in API when using thread pools, so that they can be set once and re-used over multiple start of the same request.
Detailed List of commits:
- Require setting thread-pool before start for AsyncLoopWork
- Allow setting request thread pool and task before start
- Avoid leaking link to next element in ThreadSafeLinkedList
- Set AsyncRequest to State::Free after stop()
SC::Foundation
SC::AsyncFoundation gets a few new classes to represent read-only and read/write strings without needing to include the entire SC::String library.
- Fix Span::reinterpret_as_array_of const correctness
- Add method to check if a Function is bound to a specific class instance
- Add SpanStringView and SpanString
- Add Span::sliceFromStartUntil and fix Span::equals
- Fix Span::reinterpret_as_array_of
Refactoring
Changes in SC::AsyncFoundation have been carried on to reduce header bloat and inter-dependencies between libraries.
More specifically both File and Socket library got this treatment, so that Socket doesn't depend on SC::String library at all anymore, and SC::File now has a StringView-free header (FileDescriptor.h) that can be included without bringing any StringView dependency