Dependencies

Dependencies: AsyncStreams
All dependencies: Async, AsyncStreams, File, FileSystem, Foundation, Socket, Threading

Dependency Graph

Features

HTTP 1.1 Parser
HTTP 1.1 Server

Status

🟥 Draft
In current state the library is able to host simple static website but it cannot be used for any internet facing application.
Additionally its API will be changing heavily as it's undergoing major re-design to make it fully allocation free and extend it to support more of the HTTP 1.1 standard.

Description

The HTTP parser is an incremental parser, that will emit events as soon as a valid element has been successfully parsed. This allows handling incomplete responses without needing holding it entirely in memory.

The HTTP server is for now just a basic implementations and it's missing many important features. It's however capable of serving files through http while staying inside a few user provided fixed buffers, without allocating any kind of dynamic memory.

The HTTP client has been for now moved to the tests as it needs significant re-work to become useful.

Videos

This is the list of videos that have been recorded showing some of the internal thoughts that have been going into this library:

Ep.27 - C++ Async Http Web Server

Blog

Some relevant blog posts are:

HttpAsyncServer

Async Http Server.

This class handles a fully asynchronous http server staying inside 5 fixed memory regions passed during init.

Usage:

Use SC::HttpAsyncServer::onRequest callback to intercept requests from http clients
Write to SC::HttpResponse using the writable stream obtained from HttpConnection::response.getWritableStream()
Alternatively use SC::HttpAsyncFileServer to statically serve files

See also: SC::HttpAsyncFileServer, SC::HttpConnectionsPool

    constexpr int MAX_CONNECTIONS = 3;        // Max number of concurrent http connections
    constexpr int REQUEST_SLICES  = 2;        // Number of slices of the request buffer for each connection
    constexpr int REQUEST_SIZE    = 1 * 1024; // How many bytes are allocated to stream data for each connection
    constexpr int HEADER_SIZE     = 8 * 1024; // How many bytes are dedicated to hold request and response headers
 
    // The size of the header and request memory, and length of read/write queues are fixed here, but user can
    // set any arbitrary size for such queues doing the same being done in HttpAsyncConnection constructor.
    using HttpConnectionType = HttpAsyncConnection<REQUEST_SLICES, REQUEST_SLICES, HEADER_SIZE, REQUEST_SIZE>;
 
    // 1. Memory to hold all http connections (single array for simplicity).
    // WebServerExample (SCExample) shows how to leverage virtual memory, to handle dynamic number of clients
    HttpConnectionType connections[MAX_CONNECTIONS];
 
    // Initialize and start the http server
    HttpAsyncServer httpServer;
    SC_TEST_EXPECT(httpServer.init(Span<HttpConnectionType>(connections)));
    SC_TEST_EXPECT(httpServer.start(eventLoop, "127.0.0.1", 6152));
 
    struct ServerContext
    {
        int numRequests;
    } serverContext = {0};
 
    // Handle the request and answer accordingly
    httpServer.onRequest = [this, &serverContext](HttpConnection& client)
    {
        HttpRequest&  request  = client.request;
        HttpResponse& response = client.response;
        if (request.getParser().method != HttpParser::Method::HttpGET)
        {
            SC_TEST_EXPECT(response.startResponse(405));
            SC_TEST_EXPECT(response.sendHeaders());
            SC_TEST_EXPECT(response.end());
            return;
        }
        if (request.getURL() != "/index.html" and request.getURL() != "/")
        {
            SC_TEST_EXPECT(response.startResponse(404));
            SC_TEST_EXPECT(response.sendHeaders());
            SC_TEST_EXPECT(response.end());
            return;
        }
        serverContext.numRequests++;
        SC_TEST_EXPECT(response.startResponse(200));
        SC_TEST_EXPECT(response.addHeader("Connection", "Closed"));
        SC_TEST_EXPECT(response.addHeader("Content-Type", "text/html"));
        SC_TEST_EXPECT(response.addHeader("Server", "SC"));
        SC_TEST_EXPECT(response.addHeader("Date", "Mon, 27 Aug 2023 16:37:00 GMT"));
        SC_TEST_EXPECT(response.addHeader("Last-Modified", "Wed, 27 Aug 2023 16:37:00 GMT"));
        const char sampleHtml[] = "<html>\r\n"
                                  "<body bgcolor=\"#000000\" text=\"#ffffff\">\r\n"
                                  "<h1>This is a title {}!</h1>\r\n"
                                  "We must start from somewhere\r\n"
                                  "</body>\r\n"
                                  "</html>\r\n";
 
        // Create a "user provided" dynamically allocated string, to show this is possible
        String content;
        SC_TEST_EXPECT(StringBuilder::format(content, sampleHtml, serverContext.numRequests));
        SmallString<16> contentLength;
        SC_TEST_EXPECT(StringBuilder::format(contentLength, "{}", content.view().sizeInBytes()));
        SC_TEST_EXPECT(response.addHeader("Content-Length", contentLength.view()));
        SC_TEST_EXPECT(response.sendHeaders());
        // Note that the system takes ownership of the dynamically allocated user provided string
        // through type erasure and it will invoke its destructor after th write operation will finish,
        // freeing user memory as expected.
        // This write operation succeeds because EXTRA_SLICES allocates one more slot buffer exactly
        // to hold this user provided buffer, that is not part of the "re-usable" buffers created
        // at the beginning of this sample.
        SC_TEST_EXPECT(response.getWritableStream().write(move(content)));
        SC_TEST_EXPECT(response.end());
    };
 

HttpAsyncFileServer

Http file server statically serves files from a directory.

This class registers the onRequest callback provided by HttpAsyncServer to serves files from a given directory.

    constexpr int MAX_CONNECTIONS = 16;       // Max number of concurrent http connections
    constexpr int REQUEST_SLICES  = 2;        // Number of slices of the request buffer for each connection
    constexpr int REQUEST_SIZE    = 1 * 1024; // How many bytes are allocated to stream data for each connection
    constexpr int HEADER_SIZE     = 8 * 1024; // How many bytes are dedicated to hold request and response headers
    constexpr int NUM_FS_THREADS  = 4;        // Number of threads in the thread pool for async file stream operations
 
    // This class is fixing buffer sizes at compile time for simplicity but it's possible to size them at runtime
    using HttpConnectionType = HttpAsyncConnection<REQUEST_SLICES, REQUEST_SLICES, HEADER_SIZE, REQUEST_SIZE>;
 
    // 1. Memory to hold all http connections (single array for simplicity).
    // WebServerExample (SCExample) shows how to leverage virtual memory, to handle dynamic number of clients
    HttpConnectionType connections[MAX_CONNECTIONS];
 
    // 2. Memory used by the async file streams started by file server.
    HttpAsyncFileServer::StreamQueue<REQUEST_SLICES> streams[MAX_CONNECTIONS];
 
    // Initialize and start the http and the file server
    HttpAsyncServer     httpServer;
    HttpAsyncFileServer fileServer;
 
    ThreadPool threadPool;
    if (eventLoop.needsThreadPoolForFileOperations()) // no thread pool needed for io_uring
    {
        SC_TEST_EXPECT(threadPool.create(NUM_FS_THREADS));
    }
    SC_TEST_EXPECT(httpServer.init(Span<HttpConnectionType>(connections)));
    SC_TEST_EXPECT(httpServer.start(eventLoop, "127.0.0.1", 8090));
    SC_TEST_EXPECT(fileServer.init(threadPool, eventLoop, webServerFolder));
 
    // Forward all http requests to the file server in order to serve files
    httpServer.onRequest = [&](HttpConnection& connection)
    { SC_ASSERT_RELEASE(fileServer.serveFile(streams[connection.getConnectionID().getIndex()], connection)); };

Examples

SCExample features the WebServerExample sample showing how to use SC::HttpAsyncFileServer and SC::HttpAsyncServer
Unit tests show how to use SC::HttpAsyncFileServer and SC::HttpAsyncServer

Roadmap

🟨 MVP

Implement Stream based client and agent for socket re-use and connection persistence
Support mostly used HTTP verbs / methods
HTTP 1.1 Chunked Encoding

🟩 Usable Features:

Implement (Web Streams API)
Connection Upgrade
Multipart encoding

🟦 Complete Features:

HTTPS
Support all HTTP verbs / methods

💡 Unplanned Features:

Http 2.0
Http 3.0

Statistics

Type	Lines Of Code	Comments	Sum
Headers	282	216	498
Sources	1530	283	1813
Sum	1812	499	2311

Table of Contents