Reducing Internal Dependencies

The majority of the work this month focused on breaking internal dependencies between libraries.

The goal is for Sane C++ Libraries to be a collection of libraries that work well together, rather than a monolithic framework. Users should be able to use a single library in isolation without being forced to adopt the entire ecosystem.

Sane C++ Libraries is not a project that forces itself to be the central heart of your software project (even if it would probably be a good idea! 😎).

Balancing library isolation with the ergonomics of using multiple libraries together is challenging, but significant progress has been made.

Here is the resulting graph of all internal dependencies between libraries:

Key takeaways:

• Only libraries that depend on Memory require dynamic memory allocation (currently Containers and Http).
• The entire lower row of libraries are essentially standalone, depending only on Foundation.
• Foundation is a minimal collection of headers designed to avoid using the Standard C++ Library and including compiler-supplied headers in the public interface.

In many cases, all you need to get started is the Foundation single-file library plus any other library from the lower row!

Detailed list of commits:

• ContainersSerialization: Remove Serialization dependency on Containers
• FileSystem: Remove FileSystem dependency on File
• FileSystem: Remove FileSystem dependency on Time
• Plugin: Remove Plugin dependency on Containers
• Plugin: Remove Plugin dependency on Memory
• Plugin: Remove Plugin dependency on Threading
• SerializationText: Remove SerializationText dependency on Memory
• Strings: Move String and SmallString to Memory
• Strings: Remove Path dependency on String
• Strings: Remove StringBuilder dependency on Buffer and require finalize (breaking change)
• Strings: Remove StringBuilder dependency on String
• Strings: Remove StringFormat dependency on Buffer and String
• Testing: Remove Testing dependency on Memory
• Testing: Remove Testing dependency on Strings

Dependencies Support

To achieve the goal of reducing internal dependencies, it was necessary to break many internal sub-dependencies.

The most challenging task was moving String and SmallString from the Strings library to the Memory library.

Most String usages were related to file and path manipulation, which is now handled by the fixed-size and native-encoding-aware StringPath.

While this may seem counter-intuitive, it makes perfect sense. The Strings library should focus on string manipulation, and std::string-like classes don't belong there as they require allocator support.

In the future, alternatives to std::string-like allocation will be provided. For now, anyone needing it can use String and the Global allocator support from Memory.

This change caused a large refactoring across the entire project, which may have introduced some bugs (mainly due to re-working null-termination).

The most significant breaking change is the new requirement to call StringBuilder::finalize to ensure the underlying string container or buffer is null-terminated.

Please report any misbehavior or bugs you may find.

Detailed list of commits:

• Foundation: De-virtualize IGrowableBuffer::getDirectAccess
• Foundation: Export IGrowableBuffer and derived classes
• Foundation: Implement GrowableBuffer
• Foundation: Move StringView::compare to StringSpan
• Memory: Avoid losing data when tryGrowTo causes reallocation
• Memory: Make GrowableBuffer final
• Plugin: Null terminate error messages when compiling or linking
• SerializationText: Default specialization handles non-struct type
• SerializationText: Use StringSpan in SerializationText
• Strings: Allow custom encoding in construction of StringView from iterators
• Strings: Avoid allocation in Path::normalize when preserving UNC prefix
• Strings: Avoid reading past end pointers during UTF decoding
• Strings: Do not append null-terminator in StringBuilder / StringFormat
• Strings: Get rid of dynamic memory allocations from Console
• Strings: Make Console conversion buffer optional
• Strings: Make GrowableBuffer final
• Strings: Make output the first parameter of Path::relativeFromTo
• Strings: Make StringPath::path private
• Strings: Move null-termination helpers from StringConverter to String and StringBuilder
• Strings: Remove unused member functions from StringConverter
• Strings: Simplify StringConverter API and break dependency from Buffer
• Strings: Use StringSpan instead of StringView for StringFormat specifiers
• Testing: Use StringPath for library root / application root / executable paths

Dependencies Tooling

Internal dependencies are now automatically computed with a Python script that also generates SVGs for documentation and a small interactive visualization.

The interactive visualization can highlight the required downstream dependencies for a given set of libraries.

This is useful because it's now entirely automatic, ensuring the documentation is always in sync with the current state of internal dependencies.

Finally, the CI will fail if a PR introduces a dependency that doesn't adhere to proper dependency hygiene!

Detailed list of commits:

• Dependencies: Compute minimal dependencies for each library
• Dependencies: Generate a dependencies graph .dot file
• Dependencies: Generate Dependencies SVG for each library
• Dependencies: Generate the interactive dependencies graph
• Dependencies: Automatically check for accidental dependencies addition

Build

The SC::Build build generator has been transitioned into an application rather than a library.

The main driver for this change is its significant number of internal dependencies, which makes little sense to reuse as a library.

A few fixes have also been made.

Detailed list of commits:

• Build: Make Build just a tool rather than a library
• Build: Fix incorrect generation of Xcode project
• Build: Only define rules relevant to requested targets in Makefiles

GDB Pretty Printer

A pretty printer is an extension for a debugger that displays data structures in a more human-readable format. For example, it can decode a string with the proper UTF encoding or show the elements of a container rather than its raw implementation details. This is very useful when debugging and is often taken for granted when using the C++ standard library.

For a project that doesn't use the Standard Library, creating pretty printers for its users is almost a necessity.

The project already had LLDB and MSVC pretty printers (debug visualizers), with GDB being the only one missing. A new GDB debug visualizer has been added, so be sure to give it a try if GDB is your debugger of choice!

Detailed list of commits:

• DebugVisualizer: Add a GDB debug visualizer

Contributions

This month, the project received some really useful contributions from Francesco Cozzuto (cozis)!

This was also an opportunity to fix some issues in the CI for contributors that were not properly set up.

Detailed list of commits:

• Async: Use epoll_pwait instead of epoll_pwait2
• Build: Add "DebugValgrind" build for SCTest
• FileSystem: Add Operations::getCurrentDirectory
• FileSystem: Run clang-format
• CI: Allow running the CI jobs on every branch of the repo
• CI: Trigger "Documentation and Code Coverage" workflow on pull requests

Http

The SC::Http module has received some improvements to its URL parser, mainly by extending its test coverage.

However, this is just a small step. SC::Http is still in a 🟥 Draft state and will require substantial work before it can even be considered 🟨 MVP.

Detailed list of commits:

• Http: Add UTF8 tests for HttpURL Parser
• Http: Improve URL parser to handle case sensitivity, IPV6 and invalid ports
• Http: Improve URL Parser to test more edge cases

Miscellaneous

As with every month, here is a bunch of random fixes!

• Everywhere: Support running test suite under XCTest
• File: Check for errors during stdin / stdout / stderr handles duplication
• Process: Modify stdout / stderr Span to return how many bytes have been read