////
Copyright (c) 2024 The C++ Alliance, Inc. (https://cppalliance.org)

Distributed under the Boost Software License, Version 1.0. (See accompanying
file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)

Official repository: https://github.com/boostorg/website-v2-docs
////
= System Components
:navtitle: System

Developing a system component for an operating system in pass:[C++] involves a wide range of low-level tasks. The relevant Boost libraries will largely depend on the specifics of your project. Some operating systems may not support all of the features of these libraries, and for low-level tasks, it may be more appropriate to use system APIs directly. For higher-level system operations, or cross-platform tasks, Boost libraries have a lot to offer.

[square]
* <<Libraries>>
* <<Sample System File and Error Handling>>
* <<Support Configuration Settings>>
* <<Time the System Operations>>
* <<Handle Atomic Operations and Racing Threads>>
* <<Next Steps>>
* <<See Also>>

== Libraries

Here are some Boost libraries that are useful in building system components:

[circle]
* boost:filesystem[] : This library provides a portable way of querying and manipulating paths, files, and directories. It can be very helpful for system-level tasks that need to interact with the file system.

* boost:program_options[] : This library allows program options to be defined, with types and default values, and their values to be retrieved from the command line, from config files, and programmatically.

* boost:system[] : This library provides simple, light-weight error_code objects that encapsulate system-specific "error codes", distinct from pass:[C++] exceptions.

* boost:chrono[] : This library provides a set of handy features for measuring time, which might be useful for system-level tasks that need to measure or manipulate time.

* boost:asio[] : This library provides a consistent asynchronous model using a modern pass:[C++] approach for network and low-level I/O programming. This might be useful for network-related components or any component that interacts with hardware.

* boost:interprocess[] : This library provides a way of sharing memory and communicating between processes. It's useful for creating shared memory regions, handling inter-process communication, managing shared objects, and synchronizing processes.

* boost:thread[] : This library provides a portable interface for multithreading. It includes features for creating and managing threads, mutexes, condition variables, and futures.

* boost:fiber[] : A fiber is a lightweight thread of execution. boost:fiber[] provides a framework for creating and managing fibers, which can be useful in some system-level programming tasks.

* boost:container[] : This provides advanced data structures beyond the ones provided by the pass:[C++] standard library, which may be useful in certain scenarios.

* boost:process[] : This library allows you to create child processes, setup their environment and provides means to communicate with them asynchronously through various streams.

Note:: The code in this tutorial was written and tested using Microsoft Visual Studio (Visual C++ 2022, Console App project) with Boost version 1.88.0.

== Sample System File and Error Handling

Two core features of most systems are in file handling and robust error reporting. For a simpler sample we'll create an app that uses boost:filesystem[] to manipulate files and directories, and boost:system[] to capture and display specific errors.

The following sample creates a directory (`example_directory`) if it does not exist, writes a file (`example_file.txt.`) to the directory, reads from the file, and handles system-specific errors. Finally, it cleans up by deleting the file and directory, still handling any errors.

For examples of networking and threading code, refer to xref:task-networking.adoc[] and xref:task-parallel-computation.adoc[].

[source,cpp]
----
#include <boost/filesystem.hpp>
#include <boost/system/error_code.hpp>
#include <iostream>
#include <fstream>

namespace fs = boost::filesystem;
namespace sys = boost::system;

//  Check and report system-specific errors
void handle_error(const sys::error_code& ec, const std::string& action) {
    if (ec) {
        std::cerr << "Error while " << action << ": " << ec.message()
            << " (Code: " << ec.value() << ")\n";
    }
    else
    {
        std::cout << "All OK while " << action << '\n';
    }
}

int main() {
    fs::path dir = "example_directory";
    fs::path file = dir / "example_file.txt";
    sys::error_code ec;

    // Create directory if it doesn't exist
    if (!fs::exists(dir)) {
        fs::create_directory(dir, ec);
        handle_error(ec, "creating directory");
    }

    // Write to the file
    {
        std::ofstream ofs(file.string());
        if (!ofs) {
            std::cerr << "Failed to open file for writing!\n";
            return 1;
        }
        ofs << "Hello, Boost.Filesystem!\n";
    }

    // Read from the file
    {
        std::ifstream ifs(file.string());
        if (!ifs) {
            std::cerr << "Failed to open file for reading!\n";
            return 1;
        }
        std::string content;
        std::getline(ifs, content);
        std::cout << "File content: " << content << '\n';
    }

    // Remove file
    fs::remove(file, ec);
    handle_error(ec, "removing file");

    // Remove directory
    fs::remove(dir, ec);
    handle_error(ec, "removing directory");

    return 0;
}

----

Note:: boost:filesystem[] ensures directory and file management is platform-independent.

Tip:: The first time you run this code, comment out the code to remove the file and directory at the end. After running the code, locate `example_directory`, then you can both verify the content of `example_file.txt` and record the parent directory of `example_directory` - which you will need in later examples.

== Support Configuration Settings

We are now going to include boost:program_options[] to allow configuration settings via command-line arguments and configuration files.

The code now allows users to specify directory and file names, reads settings from a `config.ini` file, and uses default values when one is not specified or located.

[source,cpp]
----
#include <boost/filesystem.hpp>
#include <boost/system/error_code.hpp>
#include <boost/program_options.hpp>
#include <iostream>
#include <fstream>

namespace fs = boost::filesystem;
namespace sys = boost::system;
namespace po = boost::program_options;

//  Check and report system-specific errors
void handle_error(const sys::error_code& ec, const std::string& action) {
    if (ec) {
        std::cerr << "Error while " << action << ": " << ec.message()
            << " (Code: " << ec.value() << ")\n";
    }
    else
    {
        std::cout << "All OK while " << action << '\n';
    }
}

int main(int argc, char* argv[]) {
    // Default configuration values
    std::string dir = "default_directory";
    std::string filename = "default_file.txt";
    std::string config_file = "config.ini";

    // Define command-line options
    po::options_description desc("Allowed options");
    desc.add_options()
        ("help,h", "Show help message")
        ("dir,d", po::value<std::string>(&dir), "Directory name")
        ("file,f", po::value<std::string>(&filename), "File name")
        ("config,c", po::value<std::string>(&config_file)->default_value("config.ini"), "Configuration file");

    // Parse command-line options
    po::variables_map vm;
    po::store(po::parse_command_line(argc, argv, desc), vm);
    po::notify(vm);

    if (vm.count("help")) {
        std::cout << desc << std::endl;
        return 0;
    }

    // Read options from configuration file (if available)
    std::ifstream ifs(config_file);
    if (ifs) {
        po::store(po::parse_config_file(ifs, desc), vm);
        po::notify(vm);
    }

    fs::path directory(dir);
    fs::path file = directory / filename;
    sys::error_code ec;

    // Create directory if it doesn't exist
    if (!fs::exists(directory)) {
        fs::create_directory(directory, ec);
        handle_error(ec, "creating directory");
    }

    // Write to the file
    {
        std::ofstream ofs(file.string());
        if (!ofs) {
            std::cerr << "Failed to open file for writing!\n";
            return 1;
        }
        ofs << "Hello, Boost.Program_Options and Boost.Filesystem!\n";
    }

    // Read from the file
    {
        std::ifstream ifs(file.string());
        if (!ifs) {
            std::cerr << "Failed to open file for reading!\n";
            return 1;
        }
        std::string content;
        std::getline(ifs, content);
        std::cout << "File content: " << content << '\n';
    }

    // Remove file
    fs::remove(file, ec);
    handle_error(ec, "removing file");

    // Remove directory
    fs::remove(directory, ec);
    handle_error(ec, "removing directory");

    return 0;
}

----

The command line options accepted by the sample are:

[width="100%",cols="1,3",options="header",stripes=even,frame=none]
|===
| Option | Description
| `--dir` or `-d`| Specify the directory.
| `--file` or `-f` | Specify the filename.
| `--config` or `-c`  | Specify the configuration file.
| `--help` or `-h` | Display available options.
|===

The following is an example `config.ini` file, create it and store it to the parent directory you recorded in the previous example.

[source,text]
----
dir = my_directory
file = my_file.txt

----

The following command lines show how to run with defaults, run with options specified manually, and then run with a config file:

[source,text]
----
./program

./program --dir=my_data --file=data.txt

./program --config=my_config.ini

----

== Time the System Operations

It might be important to record the time taken for system operations, both in testing and in the operation of a system app. So, let's integrate boost:chrono[] to measure the time taken for key filesystem operations, such as creating directories, writing to files, reading files, and deleting files.

image::system-timing.png[]

[source,cpp]
----
#include <boost/filesystem.hpp>
#include <boost/system/error_code.hpp>
#include <boost/program_options.hpp>
#include <boost/chrono.hpp>
#include <iostream>
#include <fstream>

namespace fs = boost::filesystem;
namespace sys = boost::system;
namespace po = boost::program_options;
namespace chrono = boost::chrono;

//  Check and report system-specific errors
void handle_error(const sys::error_code& ec, const std::string& action) {
    if (ec) {
        std::cerr << "Error while " << action << ": " << ec.message()
                  << " (Code: " << ec.value() << ")\n";
    }
    else
    {
        std::cout << "All OK while " << action << '\n';
    }
}

int main(int argc, char* argv[]) {

    // Default configuration values
    std::string dir = "default_directory";
    std::string filename = "default_file.txt";
    std::string config_file = "config.ini";

    // Define command-line options
    po::options_description desc("Allowed options");
    desc.add_options()
        ("help,h", "Show help message")
        ("dir,d", po::value<std::string>(&dir), "Directory name")
        ("file,f", po::value<std::string>(&filename), "File name")
        ("config,c", po::value<std::string>(&config_file)->default_value("config.ini"), "Configuration file");

    // Parse command-line options
    po::variables_map vm;
    po::store(po::parse_command_line(argc, argv, desc), vm);
    po::notify(vm);

    if (vm.count("help")) {
        std::cout << desc << std::endl;
        return 0;
    }

    // Read options from configuration file (if available)
    std::ifstream ifs_config(config_file);
    if (ifs_config) {
        po::store(po::parse_config_file(ifs_config, desc), vm);
        po::notify(vm);
    }

    fs::path directory(dir);
    fs::path file = directory / filename;
    sys::error_code ec;

    // Measure time for directory creation
    auto start = chrono::steady_clock::now();
    if (!fs::exists(directory)) {
        fs::create_directory(directory, ec);
        handle_error(ec, "creating directory");
    }
    auto end = chrono::steady_clock::now();
    std::cout << "Directory creation took: " 
              << chrono::duration_cast<chrono::microseconds>(end - start).count() 
              << " microseconds\n";

    // Measure time for writing to file
    start = chrono::steady_clock::now();
    {
        std::ofstream ofs(file.string());
        if (!ofs) {
            std::cerr << "Failed to open file for writing!\n";
            return 1;
        }
        ofs << "Hello, Boost.Program_Options, Boost.Filesystem, and Boost.Chrono!\n";
    }
    end = chrono::steady_clock::now();
    std::cout << "File writing took: " 
              << chrono::duration_cast<chrono::microseconds>(end - start).count() 
              << " microseconds\n";

    // Measure time for reading from file
    start = chrono::steady_clock::now();
    {
        std::ifstream ifs(file.string());
        if (!ifs) {
            std::cerr << "Failed to open file for reading!\n";
            return 1;
        }
        std::string content;
        std::getline(ifs, content);
        std::cout << "File content: " << content << '\n';
    }
    end = chrono::steady_clock::now();
    std::cout << "File reading took: " 
              << chrono::duration_cast<chrono::microseconds>(end - start).count() 
              << " microseconds\n";

    // Measure time for file deletion
    start = chrono::steady_clock::now();
    fs::remove(file, ec);
    handle_error(ec, "removing file");
    end = chrono::steady_clock::now();
    std::cout << "File deletion took: " 
              << chrono::duration_cast<chrono::microseconds>(end - start).count() 
              << " microseconds\n";

    // Measure time for directory deletion
    start = chrono::steady_clock::now();
    fs::remove(directory, ec);
    handle_error(ec, "removing directory");
    end = chrono::steady_clock::now();
    std::cout << "Directory deletion took: " 
              << chrono::duration_cast<chrono::microseconds>(end - start).count() 
              << " microseconds\n";

    return 0;
}

----

The following is example output from running the sample:

[source,text]
----
All OK while creating directory
Directory creation took: 459 microseconds
File writing took: 631 microseconds
File content: Hello, Boost.Program_Options, Boost.Filesystem, and Boost.Chrono!
File reading took: 1385 microseconds
All OK while removing file
File deletion took: 339 microseconds
All OK while removing directory
Directory deletion took: 386 microseconds

----

Adding timing features to your system operations will help you maintain more robust and performance-aware code, so as code is updated you will have built in the checks and balances so that if something goes awry - you will be able to capture and correct it early in the development cycle.

== Handle Atomic Operations and Racing Threads

In systems programming, having _atomic_ variables can often be useful. A normal increment of a counter has to read a current value, add 1, then write the result back. If threads are accessing the variable at the same time, increments can be missed. The solution is for these three operations to be treated as one - an atomic (that is, indivisible) operation - and boost:atomic[] provides this feature.

This first example shows multiple threads (managed by boost:thread[]) safely incrementing an atomic counter:

[source,cpp]
----
#include <boost/thread.hpp>
#include <boost/atomic.hpp>
#include <iostream>

boost::atomic<int> counter(0);

void worker(int id)
{
    for (int i = 0; i < 5; ++i)
    {
        int value = ++counter;

        std::cout
            << "Thread "
            << id
            << " incremented counter to "
            << value
            << "\n";

        boost::this_thread::sleep_for(
            boost::chrono::milliseconds(100));
    }
}

int main()
{
    boost::thread t1([] { worker(1); });
    boost::thread t2([] { worker(2); });

    t1.join();
    t2.join();
}
----

Run the program several times:

[source,text]
----
Thread 1 incremented counter to 1
Thread 2 incremented counter to 2
Thread 1 incremented counter to 3
Thread 2 incremented counter to 4
Thread Thread 2 incremented counter to 6
1 incremented counter to 5
Thread 1 incremented counter to 7
Thread 2 incremented counter to 8
Thread 2 incremented counter to 9
Thread 1 incremented counter to 10
----

You will notice that sometimes the text output interleaves "Thread Thread 2 incremented counter to 6 incremented counter to 5", which can look weird. The problem is not the atomic counter. The counter is working correctly. The problem is that `std::cout` is shared state too.

The simplest solution to this is to protect output with a boost:thread[] mutex:


[source,cpp]
----
#include <boost/thread.hpp>
#include <boost/atomic.hpp>
#include <iostream>

boost::atomic<int> counter(0);
boost::mutex cout_mutex;

void worker(int id)
{
    for (int i = 0; i < 5; i++)
    {
        int value = ++counter;

        {
            boost::lock_guard<boost::mutex> lock(cout_mutex);

            std::cout
                << "Thread "
                << id
                << " incremented counter to "
                << value
                << "\n";
        }
    }
}


int main()
{
    boost::thread t1([] { worker(1); });
    boost::thread t2([] { worker(2); });

    t1.join();
    t2.join();
}
----

Run this program several times, and notice there will be no interleaving:

[source,text]
----
Thread 2 incremented counter to 1
Thread 1 incremented counter to 2
Thread 2 incremented counter to 3
Thread 2 incremented counter to 5
Thread 2 incremented counter to 6
Thread 2 incremented counter to 7
Thread 1 incremented counter to 4
Thread 1 incremented counter to 8
Thread 1 incremented counter to 9
Thread 1 incremented counter to 10
----

You will notice though that the increments are out of order. 

The important point here is that _threads race naturally_, the increment order is nondeterministic. Best not to write code that assumes otherwise.

A possible issue with the above code though is that a thread's output (`cout<<`) can block other threads working. To remove this burden, the following example does not write directly to `std::cout`, each worker creates a log message that goes into a boost:lockfree[] managed lock-free queue. A dedicated logger thread then removes messages and prints them safely:

[source,cpp]
----
#include <boost/thread.hpp>
#include <boost/atomic.hpp>
#include <boost/lockfree/queue.hpp>
#include <iostream>

// --------------------------------------------------
// Shared atomic counter
// --------------------------------------------------

boost::atomic<int> counter(0);

// --------------------------------------------------
// Log message object
// (queue stores pointers, not std::string directly)
// --------------------------------------------------

struct LogMessage
{
    std::string text;

    LogMessage(const std::string& s)
        : text(s)
    {
    }
};

// Lock-free queue for messages
boost::lockfree::queue<LogMessage*> log_queue(128);

// Signal when workers are finished
boost::atomic<bool> done(false);

// --------------------------------------------------
// Worker threads
// --------------------------------------------------

void worker(int id)
{
    for (int i = 0; i < 15; i++)
    {
        int value = ++counter;

        std::string msg =
            "Thread " +
            std::to_string(id) +
            " incremented counter to " +
            std::to_string(value);

        // Allocate message and push to queue
        while (!log_queue.push(new LogMessage(msg)))
        {
            // queue full, retry
        }

        boost::this_thread::sleep_for(
            boost::chrono::milliseconds(100));
    }
}

// --------------------------------------------------
// Logger thread
// --------------------------------------------------

void logger()
{
    while (!done || !log_queue.empty())
    {
        LogMessage* msg = nullptr;

        if (log_queue.pop(msg))
        {
            std::cout << msg->text << "\n";

            delete msg;
        }
        else
        {
            boost::this_thread::sleep_for(
                boost::chrono::milliseconds(10));
        }
    }
}

// --------------------------------------------------
// Main
// --------------------------------------------------

int main()
{
    boost::thread log_thread(logger);

    boost::thread t1([] { worker(1); });
    boost::thread t2([] { worker(2); });
    boost::thread t3([] { worker(3); });

    t1.join();
    t2.join();
    t3.join();

    done = true;

    log_thread.join();

    std::cout
        << "Final counter value: "
        << counter.load()
        << "\n";
}
----

Notice that the `log_thread.join();` does not interfere with the other threads. Also notice the counter output is nondeterministic by nature (blank lines added for clarity):

[source,text]
----
Thread 2 incremented counter to 1
Thread 3 incremented counter to 2
Thread 1 incremented counter to 3
Thread 3 incremented counter to 4
Thread 2 incremented counter to 5
Thread 1 incremented counter to 6

Thread 1 incremented counter to 8
Thread 2 incremented counter to 7

Thread 3 incremented counter to 9
Thread 2 incremented counter to 10
Thread 1 incremented counter to 11
Thread 3 incremented counter to 12
Thread 1 incremented counter to 14
Thread 3 incremented counter to 13
Thread 2 incremented counter to 15
Final counter value: 15
----

== Next Steps

The onus on a systems program is to work reliably, fast, and, critically, with minimal impact on resources.

The line `new LogMessage(msg)` in the last sample typically would _not_ be used in a production system, because of its frequent heap allocations. For an industrial strength production system, consider using an object pool, or fixed-size buffers, or a ring buffer to handle message logging.

== See Also

* https://www.boost.org/doc/libs/latest/libs/libraries.htm#Memory[Category: Memory]
* https://www.boost.org/doc/libs/latest/libs/libraries.htm#Miscellaneous[Category: Miscellaneous]
* https://www.boost.org/doc/libs/latest/libs/libraries.htm#System[Category: System]
