Skip to content
Open
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
fa6808a
main model logs by reference
mgovers Aug 27, 2025
87057e3
implement threaded logger
mgovers Aug 28, 2025
9962299
logger seems to work
mgovers Sep 1, 2025
8462283
Merge remote-tracking branch 'origin/main' into feature/dependency-in…
mgovers Sep 1, 2025
1cec998
minor
mgovers Sep 1, 2025
0fe5288
Update tests/cpp_unit_tests/test_job_dispatch.cpp
mgovers Sep 2, 2025
85280c2
Update power_grid_model_c/power_grid_model/include/power_grid_model/c…
mgovers Sep 2, 2025
b60c807
Update power_grid_model_c/power_grid_model/include/power_grid_model/j…
mgovers Sep 2, 2025
53f80f4
cleanup
mgovers Sep 2, 2025
45bbcbb
Apply suggestion from @mgovers
mgovers Sep 2, 2025
0cfed9e
remove unused parameter
mgovers Sep 2, 2025
7496e27
more unused captures
mgovers Sep 2, 2025
9086696
clang-tidy
mgovers Sep 2, 2025
b0156a2
Merge branch 'main' into feature/dependency-inversion-main-model-impl…
mgovers Sep 2, 2025
3d99233
fix msvc
mgovers Sep 2, 2025
28d06aa
fix msvc
mgovers Sep 2, 2025
6fab6c1
fix clang-cl... it never ends
mgovers Sep 2, 2025
060d3f1
fix gcc + clang-cl tests
mgovers Sep 2, 2025
9ad7b8b
fix dangling reference
mgovers Sep 2, 2025
d348ff7
resolve comments
mgovers Sep 2, 2025
1a20b38
more resolve comments
mgovers Sep 2, 2025
ec8490b
more resolve comments
mgovers Sep 2, 2025
93dff85
attempt to fix macos
mgovers Sep 3, 2025
3bb015e
Merge branch 'main' into feature/dependency-inversion-main-model-impl…
mgovers Sep 4, 2025
8026cea
merge_into as member function
mgovers Sep 4, 2025
2c7341c
Merge branch 'main' into feature/dependency-inversion-main-model-impl…
mgovers Sep 4, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,13 @@

#pragma once

#include "logging_impl.hpp"
#include "multi_threaded_logging.hpp"

#include <map>

namespace power_grid_model {
namespace common::logging {
class CalculationInfo final : public Logger {
class CalculationInfo : public Logger {
using Data = std::map<LogEvent, double>;

public:
Expand Down Expand Up @@ -86,7 +86,24 @@ class CalculationInfo final : public Logger {
Report report() const { return data_; }
void clear() { data_.clear(); }
};

inline Logger& merge_into(Logger& destination, CalculationInfo const& source) {
for (const auto& [tag, value] : source.report()) {
destination.log(tag, value);
}
return destination;
}

class MultiThreadedCalculationInfo : public MultiThreadedLoggerImpl<CalculationInfo> {
Copy link
Member

@nitbharambe nitbharambe Sep 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The SubThreadLogger being CalculationInfo was kind of confusing to me before review.
It is now a SubThreadLogger which is a "map".

Lets throw all notions of earlier CalculationInfo out? Or atleast rename te calculation_info in places like MathSolver.
We do that in a later PR or now?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

math solver was already completely made agnostic of the calculation info

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the only locations that IMO should know of calculation info are the objects/functions that manage a calculation info in their own scope, that is: the benchmark and the C API (main_model.hpp for now)

public:
using MultiThreadedLoggerImpl<CalculationInfo>::MultiThreadedLoggerImpl;
using Report = CalculationInfo::Report;

Report report() const { return get().report(); }
void clear() { get().clear(); }
};
} // namespace common::logging

using common::logging::CalculationInfo;
using common::logging::MultiThreadedCalculationInfo;
} // namespace power_grid_model
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@

#include <cmath>
#include <complex>
#include <concepts>
#include <cstddef>
#include <cstdint>
#include <limits>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,10 @@ class Logger {
Logger& operator=(Logger const&) = default;
};

struct MultiThreadedLogger : public Logger {
virtual std::unique_ptr<Logger> create_child() = 0;
};

} // namespace common::logging

using common::logging::LogEvent;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,4 +26,6 @@ class NoLogger : public Logger {
}
};

inline Logger& merge_into(Logger& destination, NoLogger const& /*source*/) { return destination; }

} // namespace power_grid_model::common::logging
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
// SPDX-FileCopyrightText: Contributors to the Power Grid Model project <[email protected]>
//
// SPDX-License-Identifier: MPL-2.0

#pragma once

#include "logging_impl.hpp"

#include <mutex>

namespace power_grid_model::common::logging {

template <std::derived_from<Logger> LoggerType>
requires requires(LoggerType& destination, LoggerType const& source) {
{ merge_into(destination, source) };
}
class MultiThreadedLoggerImpl : public MultiThreadedLogger {
using SubThreadLogger = LoggerType;

public:
class ThreadLogger : public SubThreadLogger {
public:
ThreadLogger(MultiThreadedLoggerImpl& parent) : parent_{&parent} {}
ThreadLogger(ThreadLogger const&) = default;
ThreadLogger& operator=(ThreadLogger const&) = default;
ThreadLogger(ThreadLogger&&) noexcept = default;
ThreadLogger& operator=(ThreadLogger&&) noexcept = default;
~ThreadLogger() override { sync(); }
void sync() const { parent_->sync(*this); }

private:
MultiThreadedLoggerImpl* parent_;
};

std::unique_ptr<Logger> create_child() override { return std::make_unique<ThreadLogger>(*this); }
LoggerType& get() { return log_; }
LoggerType const& get() const { return log_; }

void log(LogEvent tag) override { log_.log(tag); }
void log(LogEvent tag, std::string_view message) override { log_.log(tag, message); }
void log(LogEvent tag, double value) override { log_.log(tag, value); }
void log(LogEvent tag, Idx value) override { log_.log(tag, value); }

private:
friend class ThreadLogger;

LoggerType log_;
std::mutex mutex_;

void sync(ThreadLogger const& logger) {
std::lock_guard const lock{mutex_};
merge_into(log_, static_cast<SubThreadLogger const&>(logger));
}
};

using NoMultiThreadedLogger = MultiThreadedLoggerImpl<NoLogger>;

} // namespace power_grid_model::common::logging
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
#include "../common/matrix_utils.hpp"
#include "../common/three_phase_tensor.hpp"

#include <iostream>
#include <numeric>

namespace power_grid_model {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@
#include "job_interface.hpp"
#include "main_model_fwd.hpp"

#include "main_core/calculation_info.hpp"
#include "main_core/update.hpp"

namespace power_grid_model {
Expand All @@ -23,15 +22,19 @@ class JobAdapter<MainModel, ComponentList<ComponentType...>>
public:
JobAdapter(std::reference_wrapper<MainModel> model_reference,
std::reference_wrapper<MainModelOptions const> options)
: model_reference_{model_reference}, options_{options} {}
: model_reference_{model_reference}, options_{options} {
reset_logger_impl();
}
JobAdapter(JobAdapter const& other)
: model_copy_{std::make_unique<MainModel>(other.model_reference_.get())},
model_reference_{std::ref(*model_copy_)},
options_{std::ref(other.options_)},
components_to_update_{other.components_to_update_},
update_independence_{other.update_independence_},
independence_flags_{other.independence_flags_},
all_scenarios_sequence_{other.all_scenarios_sequence_} {}
all_scenarios_sequence_{other.all_scenarios_sequence_} {
reset_logger_impl();
}
JobAdapter& operator=(JobAdapter const& other) {
if (this != &other) {
model_copy_ = std::make_unique<MainModel>(other.model_reference_.get());
Expand All @@ -41,6 +44,8 @@ class JobAdapter<MainModel, ComponentList<ComponentType...>>
update_independence_ = other.update_independence_;
independence_flags_ = other.independence_flags_;
all_scenarios_sequence_ = other.all_scenarios_sequence_;

reset_logger_impl();
}
return *this;
}
Expand All @@ -51,7 +56,9 @@ class JobAdapter<MainModel, ComponentList<ComponentType...>>
components_to_update_{std::move(other.components_to_update_)},
update_independence_{std::move(other.update_independence_)},
independence_flags_{std::move(other.independence_flags_)},
all_scenarios_sequence_{std::move(other.all_scenarios_sequence_)} {}
all_scenarios_sequence_{std::move(other.all_scenarios_sequence_)} {
reset_logger_impl();
}
JobAdapter& operator=(JobAdapter&& other) noexcept {
if (this != &other) {
model_copy_ = std::move(other.model_copy_);
Expand All @@ -61,10 +68,15 @@ class JobAdapter<MainModel, ComponentList<ComponentType...>>
update_independence_ = std::move(other.update_independence_);
independence_flags_ = std::move(other.independence_flags_);
all_scenarios_sequence_ = std::move(other.all_scenarios_sequence_);

reset_logger_impl();
}
return *this;
}
~JobAdapter() { model_copy_.reset(); }
~JobAdapter() {
reset_logger_impl();
model_copy_.reset();
}

private:
// Grant the CRTP base (JobInterface<JobAdapter>) access to
Expand All @@ -83,7 +95,7 @@ class JobAdapter<MainModel, ComponentList<ComponentType...>>
// current_scenario_sequence_cache_ is calculated per scenario, so it is excluded from the constructors.
main_core::utils::SequenceIdx<ComponentType...> current_scenario_sequence_cache_{};

std::mutex calculation_info_mutex_;
Logger* log_{nullptr};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does job adapter also need to store it?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i've thought about his for a bit and this relates to the registering/unregistering discussion from elsewhere in this PR (specifically the one in #1107)

i've described the fact that the registering/unregistering is quite tricky in itself. an alternative could be to use model.calculate(..., log_). Now we can also do a hybrid situation here: the adapter could be the boundary between registering/unregistering (to the job adapter). The model itself could be model.calculate(..., log_). Then, the adapter still contains a logger, but the model no longer does.

thoughts?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Experiment in #1115


void calculate_impl(MutableDataset const& result_data, Idx scenario_idx) const {
MainModel::calculator(options_.get(), model_reference_.get(), result_data.get_individual_scenario(scenario_idx),
Expand Down Expand Up @@ -135,14 +147,6 @@ class JobAdapter<MainModel, ComponentList<ComponentType...>>
std::ranges::for_each(current_scenario_sequence_cache_, [](auto& comp_seq_idx) { comp_seq_idx.clear(); });
}

CalculationInfo get_calculation_info_impl() const { return model_reference_.get().calculation_info(); }
void reset_calculation_info_impl() { model_reference_.get().reset_calculation_info(); }

void thread_safe_add_calculation_info_impl(CalculationInfo const& info) {
std::lock_guard const lock{calculation_info_mutex_};
model_reference_.get().merge_calculation_info(info);
}

auto get_current_scenario_sequence_view_() const {
return main_core::utils::run_functor_with_all_types_return_array<ComponentType...>([this]<typename CT>() {
constexpr auto comp_idx = main_core::utils::index_of_component<CT, ComponentType...>;
Expand All @@ -152,5 +156,14 @@ class JobAdapter<MainModel, ComponentList<ComponentType...>>
return std::span<Idx2D const>{std::get<comp_idx>(current_scenario_sequence_cache_)};
});
}

void reset_logger_impl() {
log_ = nullptr;
model_reference_.get().reset_logger();
}
void set_logger_impl(Logger& log) {
log_ = &log;
model_reference_.get().set_logger(*log_);
}
Comment on lines +159 to +167
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This tells me now than MainModelImpl owns the logger? Is that what we want? Or maybe it just owns a pointer, but then should we still call this via main model instead of via whoeover owns the logger data?

Edit: I see that main model just holds a pointer to each logger, which the adapter does too, so we can just redirect on each thread... This seems fine, and can potentially help us clean the pattern used here in the adapter of having a reference wrapper and a copy of the model at the same time. However, even though this makes it clean having "copies" in two places, could that bite us in the butt later? Should we keep a pointer to the logger only here for instance, and call it via the adaptor in main model impl? Probably not, but maybe something worth thinking about for a couple of minutes.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if you like, we can either go with the log in every function call OR we can create a logger view, which takes a logger but returns NoLogger by default if it's not set

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if either convince me, but I don't have any alternative idea either. So we can try the view and see how it looks like (puns intended 😆)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's do it in a separate PR

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this comment can be answered as well by #1115, right?

};
} // namespace power_grid_model
Loading
Loading