Build

Table of Contents

• Features
• Status
• Description
• Videos
• Blog
• Architecture
• Roadmap

🟨 Minimal build system where builds are described in C++

Build uses C++ to imperatively describe a sequence of build operations.

Features

• Describe builds in a .cpp file
• Integrates with SC-Package from Tools to download dependencies
• Generate XCode 14+ Projects
  • Generate app bundle icon and storyboard
  • Link locally defined LLDB debug visualizer
• Generate Visual Studio 2022 Projects
  • Link locally defined .natvis debug visualizer
• Generate Makefile for Linux and Apple (macOS / iOS) targets
  • Generates compile_commands.json for VSCode

Status

🟨 MVP
Build is currently used to generate test and example projects of this repository.
It has a limited set of features but it can be used to describe simple projects that have no dependencies between them.
Features will be added as needed.

Description

Build C++ files (named by convention SC-build.cpp) are compiled the fly and they generate project files for existing build systems.
In the future the plan is to allow also the system to run standalone directly invoking compilers to produce libraries and executables. Projects are generated by invoking ./SC.sh build or SC.bat build.

Note

Check the Tools page for more details on SC.sh build.

This is for example the Tools/SC-build.cpp file for the SCTest test suite and the SCExample:

// Copyright (c) Stefano Cristiano
// SPDX-License-Identifier: MIT
#include "SC-build.h"
#include "Libraries/Strings/StringBuilder.h"
#include "SC-package.h"
 
namespace SC
{
namespace Tools
{
[[nodiscard]] Result installSokol(const Build::Directories& directories, Package& package)
{
    Download download;
    download.packagesCacheDirectory   = directories.packagesCacheDirectory;
    download.packagesInstallDirectory = directories.packagesInstallDirectory;
 
    download.packageName    = "sokol";
    download.packageVersion = "7b5cfa7";
    download.url            = "https://github.com/floooh/sokol.git";
    download.isGitClone     = true;
    download.createLink     = false;
    package.packageBaseName = "sokol";
 
    CustomFunctions functions;
    functions.testFunction = &verifyGitCommitHash;
 
    SC_TRY(packageInstall(download, package, functions));
    return Result(true);
}
 
[[nodiscard]] Result installDearImGui(const Build::Directories& directories, Package& package)
{
    Download download;
    download.packagesCacheDirectory   = directories.packagesCacheDirectory;
    download.packagesInstallDirectory = directories.packagesInstallDirectory;
 
    download.packageName    = "dear-imgui";
    download.packageVersion = "00ad3c6";
    download.url            = "https://github.com/ocornut/imgui.git";
    download.isGitClone     = true;
    download.createLink     = false;
    package.packageBaseName = "dear-imgui";
 
    CustomFunctions functions;
    functions.testFunction = &verifyGitCommitHash;
 
    SC_TRY(packageInstall(download, package, functions));
    return Result(true);
}
 
} // namespace Tools
namespace Build
{
SC_COMPILER_WARNING_PUSH_UNUSED_RESULT; // Doing some optimistic coding here, ignoring all failures
 
void addSaneCppLibraries(Project& project, const Parameters& parameters)
{
    // Files
    project.addDirectory("Bindings/c", "**.cpp");              // add all cpp support files for c bindings
    project.addDirectory("Bindings/c", "**.c");                // add all c bindings
    project.addDirectory("Bindings/c", "**.h");                // add all c bindings
    project.addDirectory("Libraries", "**.cpp");               // recursively add all cpp files
    project.addDirectory("Libraries", "**.h");                 // recursively add all header files
    project.addDirectory("Libraries", "**.inl");               // recursively add all inline files
    project.addDirectory("LibrariesExtra", "**.h");            // recursively add all header files
    project.addDirectory("LibrariesExtra", "**.cpp");          // recursively add all cpp files
    project.addDirectory("Support/DebugVisualizers", "*.cpp"); // add debug visualizers
 
    // Libraries to link
    if (parameters.platform == Platform::Apple)
    {
        project.link.addFrameworks({"CoreFoundation", "CoreServices"});
    }
 
    if (parameters.platform != Platform::Windows)
    {
        project.link.addLibraries({"dl", "pthread"});
    }
 
    // Debug visualization helpers
    if (parameters.generator == Generator::VisualStudio2022)
    {
        project.addDirectory("Support/DebugVisualizers/MSVC", "*.natvis");
    }
    else
    {
        project.addDirectory("Support/DebugVisualizers/LLDB", "*");
    }
}
 
static constexpr StringView TEST_PROJECT_NAME = "SCTest";
 
Result buildTestProject(const Parameters& parameters, Project& project)
{
    project = {TargetType::Executable, TEST_PROJECT_NAME};
 
    // All relative paths are evaluated from this project root directory.
    project.setRootDirectory(parameters.directories.libraryDirectory.view());
 
    // Project Configurations
    project.addPresetConfiguration(Configuration::Preset::Debug, parameters);
    project.addPresetConfiguration(Configuration::Preset::Release, parameters);
    project.addPresetConfiguration(Configuration::Preset::DebugCoverage, parameters);
 
    // Defines
    // $(PROJECT_ROOT) expands to Project::setRootDirectory expressed relative to $(PROJECT_DIR)
    project.compile.addDefines({"SC_LIBRARY_PATH=$(PROJECT_ROOT)", "SC_COMPILER_ENABLE_CONFIG=1"});
 
    // Includes
    project.compile.addIncludes({
        ".",            // Libraries path (for PluginTest)
        "Tests/SCTest", // SCConfig.h path (enabled by SC_COMPILER_ENABLE_CONFIG == 1)
    });
 
    addSaneCppLibraries(project, parameters);
    project.addDirectory("Tests/SCTest", "*.cpp"); // add all .cpp from SCTest directory
    project.addDirectory("Tests/SCTest", "*.h");   // add all .h from SCTest directory
    project.addDirectory("Tools", "SC-*.cpp");     // add all tools
    project.addDirectory("Tools", "*.h");          // add tools headers
    project.addDirectory("Tools", "*Test.cpp");    // add tools tests
    return Result(true);
}
 
static constexpr StringView EXAMPLE_PROJECT_NAME = "SCExample";
 
Result buildExampleProject(const Parameters& parameters, Project& project)
{
    project = {TargetType::Executable, EXAMPLE_PROJECT_NAME};
 
    // All relative paths are evaluated from this project root directory.
    project.setRootDirectory(parameters.directories.libraryDirectory.view());
 
    // Project icon (currently used only by Xcode backend)
    project.iconPath = "Documentation/Doxygen/SC.svg";
 
    // Install dependencies
    Tools::Package sokol;
    SC_TRY(Tools::installSokol(parameters.directories, sokol));
    Tools::Package dearImGui;
    SC_TRY(Tools::installDearImGui(parameters.directories, dearImGui));
 
    // Add includes
    project.compile.addIncludes({".", sokol.packageLocalDirectory.view(), dearImGui.packageLocalDirectory.view()});
 
    // Project Configurations
    project.addPresetConfiguration(Configuration::Preset::Debug, parameters);
    project.addPresetConfiguration(Configuration::Preset::Release, parameters);
    project.addPresetConfiguration(Configuration::Preset::DebugCoverage, parameters);
 
    addSaneCppLibraries(project, parameters);            // add all SC Libraries
    project.removeFiles("Bindings/c", "*");              // remove all bindings
    project.removeFiles("Libraries", "**Test.cpp");      // remove all tests
    project.removeFiles("LibrariesExtra", "**Test.cpp"); // remove all tests
    project.removeFiles("Support", "**Test.cpp");        // remove all tests
 
    project.addDirectory(dearImGui.packageLocalDirectory.view(), "*.cpp");
    project.addDirectory(sokol.packageLocalDirectory.view(), "*.h");
    String imguiRelative, imguiDefine;
    SC_TRY(Path::relativeFromTo(project.rootDirectory.view(), dearImGui.packageLocalDirectory.view(), imguiRelative,
                                Path::AsNative));
    SC_TRY(StringBuilder(imguiDefine).format("SC_IMGUI_PATH=$(PROJECT_ROOT)/{}", imguiRelative));
    project.compile.addDefines({"SC_LIBRARY_PATH=$(PROJECT_ROOT)", imguiDefine.view()});
    project.link.set<Link::guiApplication>(true);
    if (parameters.platform == Platform::Apple)
    {
        project.addDirectory("Examples/SCExample", "*.m"); // add all .m from SCExample directory
        project.link.addFrameworks({"Metal", "MetalKit", "QuartzCore"});
        project.link.addFrameworks({"Cocoa"}, PlatformApple::macOS);
        project.link.addFrameworks({"UIKit", "Foundation"}, PlatformApple::iOS);
    }
    else
    {
        project.addDirectory("Examples/SCExample", "*.c"); // add all .c from SCExample directory
        if (parameters.platform == Platform::Linux)
        {
            project.link.addLibraries({"GL", "EGL", "X11", "Xi", "Xcursor"});
        }
    }
    if (parameters.platform == Platform::Windows)
    {
        project.compile.addDefines({"IMGUI_API=__declspec( dllexport )"});
    }
    else
    {
        project.compile.addDefines({"IMGUI_API=__attribute__((visibility(\"default\")))"});
    }
    project.addDirectory("Examples/SCExample", "**.h");   // add all .h from SCExample directory recursively
    project.addDirectory("Examples/SCExample", "**.cpp"); // add all .cpp from SCExample directory recursively
    return Result(true);
}
 
Result configure(Definition& definition, const Parameters& parameters)
{
    Workspace workspace = {"SCTest"};
    SC_TRY(workspace.projects.resize(2));
    SC_TRY(buildTestProject(parameters, workspace.projects[0]));
    SC_TRY(buildExampleProject(parameters, workspace.projects[1]));
    definition.workspaces.push_back(move(workspace));
    return Result(true);
}
SC_COMPILER_WARNING_POP;
 
Result executeAction(const Action& action) { return Build::Action::execute(action, configure, TEST_PROJECT_NAME); }
} // namespace Build
} // namespace SC

The abstraction is described by the following (top-down) hierarchy:

Class	Description
SC::Build::Definition	Top level build description holding all Workspace objects.
SC::Build::Workspace	Groups multiple Project together with shared compile and link flags.
SC::Build::Project	Groups multiple Configuration and source files with their compile and link flags.
SC::Build::Configuration	Groups SC::Build::CompileFlags and SC::Build::LinkFlags for a given SC::Build::Architecture.

Some additional types allow describing detailed properties of the build:

Class	Description
SC::Build::Platform	Build Platform (Operating System)
SC::Build::Architecture	Build Architecture (Processor / Instruction set)
SC::Build::Generator	Build system generator (Xcode / Visual Studio)
SC::Build::Optimization	Optimization level (Debug / Release)
SC::Build::Compile	Compilation switches (include paths, preprocessor defines, etc.)
SC::Build::CompileFlags	Map of SC::Build::Compile flags (include paths, preprocessor defines etc.)
SC::Build::Link	Linking switches (library paths, LTO etc.)
SC::Build::LinkFlags	Map of SC::Build::Link flags (library paths, LTO switch etc.)

Videos

This is the list of videos that have been recorded showing some of the internal thoughts that have been going into this library:

• Ep.02 - Creating a Makefile
• Ep.03 - Add Makefile backend to SC::Build
• Ep.04 - Start Linux Porting
• Ep.05 - Build Everything on Linux

Blog

Some relevant blog posts are:

• March 2024 Update
• May 2024 Update
• June 2024 Update
• July 2024 Update

Architecture

• User adds a SC-build.cpp files to his library
• SC-build.cpp describes the SC::Build::Definition using imperative C++ code
  • Define at least one SC::Build::Workspace
  • Define at least one SC::Build::Project
  • Define at least one SC::Build::Configuration
• SC-build.cpp is compiled to an executable with a single clang or cl.exe invocation
• The unity build file SC.cpp is linked together in the same invocation
  • Provides all needed Platform abstraction features to enumerate files and directories, launch processes etc.
• The generated executable is run as specified in SC::Build::Parameters
• SC::Build::Parameters specifies a combination of:
  • SC::Build::Platform Build Platform (Operating System)
  • SC::Build::Configuration Groups SC::Build::CompileFlags and SC::Build::LinkFlags for a given SC::Build::Architecture.
  • SC::Build::Generator Build system generator (Xcode / Visual Studio)
• SC-build.cpp calls SC::Build::ConfigurePresets::generateAllPlatforms or manually invokes SC::Build::Definition::generate
• Debugging the build script means just debugging a regular C++ executable

Note

Check the Tools page for more details on SC.sh build.

So far the entire build configuration is created in C++ but each invocation with a different set of "build parameters" it's building a data structure that is free of conditionals, as they've been evaluated by the imperative code. Such "post-configure" build settings could be serialized to JSON (or using binary Serialization) or to any other declarative format if needed.

Roadmap

🟩 Usable Features:

• Remove any left-over hardcoded setting
• Describe more project types (dynamic libraries, static libraries etc.)
• Describe dependencies between targets
• Allow nested builds (where the root script should compile child scripts found down the path)

🟦 Complete Features:

• ninja backend
• Self-hosted backend (no need for make, ninja, xcodebuild or msbuild)
• Generate ready to debug projects for the build program itself
• Describe very complex builds (like LLVM)

💡 Unplanned Features:

• Compile scripts to WASM so that they can't run arbitrary code