Async File System Operations
A major new feature this month is the unified AsyncFileSystemOperation
that provides async file system operations across all platforms. This replaces the previous AsyncFileClose
and AsyncSocketClose
with a more comprehensive and consistent API.
The new operations include:
- open/close (File)
- read/write (File)
- copyFile/copyDirectory (File / Directory)
- rename (File / Directory)
- removeFile/removeEmptyDirectory: (File / Directory)
Eventually AsyncFileRead
/ AsyncFileWrite
will be renamed to something like AsyncPipeRead
/ AsyncPipeWrite
, losing their threading support needed for buffered files, now that dedicated fs read / write operation exist.
Under the hood, this leverages platform-specific async APIs:
io_uring
on LinuxIOCP
(I/O Completion Ports) on Windows.
The abstraction handles the complexity of these very different APIs while providing a consistent interface.
Here's an example (extracted from the test suite) on how it's meant to be used:
Detailed list of commits:
- Async: Add AsyncFileSystemOperation
- Async: Add "open" async file system operation
- Async: Add "close" async file system operation
- Async: Add "read" async file system operation
- Async: Add "write" async file system operation
- Async: Add "copyFile" async file system operation
- Async: Add "rename" async file system operation
- Async: Add "removeFile" async file system operation
- Async: Add "removeDirectory" async file system operation
- Async: Add "copyDirectory" async file system operation
Async UDP Socket Operations
New async operations for UDP sockets have been added: sendTo
and receiveFrom
for unconnected sockets.
This is particularly useful for UDP networking scenarios where you need to send/receive from multiple endpoints.
The implementation handles the platform differences elegantly:
- Linux: Uses
io_uring
withIORING_OP_SENDMSG
andIORING_OP_RECVMSG
- Windows: Uses
IOCP
withWSASendTo
andWSARecvFrom
Here's how the async sendTo operation looks in practice:
And the corresponding receiveFrom operation:
Memory Library Split
One important change this mont is the split of memory management components from the Foundation library into a dedicated Memory library.
The goal has been isolating Segment
, Buffer
, Globals
, and VirtualMemory
- all the components that perform dynamic allocation.
This change brings much clearer dependency boundaries. Now if something depends only on Foundation, you can be certain it doesn't perform any dynamic allocation. This is particularly important for embedded systems, low-level libraries, or any code that needs to avoid heap allocation.
The split involved moving 46 files and updating countless include statements across the codebase. While this was a slighltly distrupting refactoring effort, it makes the library architecture much more predictable and easier to reason about.
FileSystemIterator and FileSystemWatcher Dependency Cleanup
This month saw significant work on cleaning up the dependency graph, which demonstrates the commitment to the Sane C++ Libraries principles.
FileSystemWatcher
and FileSystemIterator
have been cleaned up to remove unnecessary dependencies on Strings
and Containers
, making the dependency graph cleaner and more predictable.
As part of this cleanup, FileSystemIterator
now requires you to pass in a fixed span of FileSystemIterator::FolderState
that the iterator will use, avoiding any internal allocation.
Also FileSystemWatcher
got the no allocation treatment with the only exceptionon Linux where it's still doing a small (single) allocation holding paths for watching sub-directories (in case of recursive watch).
One could tecnically request the span of memory to use to the caller but maybe it's not great to bend the API for all other backends just for this one specific Linux difference. This topic will be re-visited in the future eventually maybe providing some hooks to remove also this very last allocation.
This kind of architectural hygiene is crucial for maintaining the "bloat free" and "easy to integrate" principles - when libraries have minimal, clear dependencies, they're much easier to understand, test, and integrate into existing projects!
- FileSystemWatcher: Remove dependency from Strings, Containers and FileSystemIterator
- FileSystemIterator: Remove dependency from Strings and Containers
Foundation
A new SC::StringViewData
class has been added to Foundation
and it can be used in place of SC::StringView
when parsing is not needed.
It's a non-owning view over a string and the most common use case is to pass it in and out of OS API as is for file system paths. As it holds also the encoding, some libraries will convert the string in some internal buffer before passing it to OS api.
Socket
Added methods to shutdown sockets and get string representations of SocketIPAddress.
- Socket: Add method to shutdown sockets
- Socket: Add method to return SocketIPAddress string representation