Skip to content

Allow Electricitymaps paid API key #220

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
btry wants to merge 2 commits into
base: develop
Choose a base branch
from
Open

Allow Electricitymaps paid API key #220

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
db3e3b3
fix(Datasource\CarbonIntensity\ElectricityMaps\Client): allow paid AP…
btry Mar 30, 2026
89a1d6f
test: update tests
btry Apr 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 14 additions & 14 deletions src/CarbonIntensity.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -124,15 +124,16 @@
/**
* get carbon intensity dates for a source and a zone
*
* @param string $zone_name Zone to examinate
* @param string $source_name Source to examinate
* @param array $source_zone Source_Zone to examinate
* @return array
*/
private function getKnownDatesQuery(string $zone_name, string $source_name)
private function getKnownDatesQuery(array $source_zone)
{
$intensity_table = CarbonIntensity::getTable();
$source_table = Source::getTable();
$zone_table = Zone::getTable();
$source_fk = getForeignKeyFieldForItemType(Source::class);
$zone_fk = getForeignKeyFieldForItemType(Zone::class);
return [
'SELECT' => [$intensity_table => ['id', 'date']],
'FROM' => $intensity_table,
Expand All @@ -151,25 +152,24 @@
],
],
'WHERE' => [
Source::getTableField('name') => $source_name,
Zone::getTableField('name') => $zone_name,
$source_fk => $source_zone[$source_fk],
$zone_fk => $source_zone[$zone_fk],
],
];
}

/**
* Get the last known date of carbon emissiosn
*
* @param string $zone_name Zone to examinate
* @param string $source_name Source to examinate
* @param array $source_zone Source_Zone to examinate
* @return DateTimeImmutable
*/
public function getLastKnownDate(string $zone_name, string $source_name): ?DateTimeImmutable
public function getLastKnownDate(array $source_zone): ?DateTimeImmutable
{
/** @var DBmysql $DB */
global $DB;

$request = $this->getKnownDatesQuery($zone_name, $source_name);
$request = $this->getKnownDatesQuery($source_zone);
$request['ORDER'] = CarbonIntensity::getTableField('date') . ' DESC';
$request['LIMIT'] = '1';
$result = $DB->request($request)->current();
Expand All @@ -191,7 +191,7 @@
/** @var DBmysql $DB */
global $DB;

$request = $this->getKnownDatesQuery($zone_name, $source_name);

Check failure on line 194 in src/CarbonIntensity.php

View workflow job for this annotation

GitHub Actions / GLPI 11.0.x - php:8.5 - mariadb:11.8 / Continuous integration 

Parameter #1 $source_zone of method GlpiPlugin\Carbon\CarbonIntensity::getKnownDatesQuery() expects array, string given.

Check failure on line 194 in src/CarbonIntensity.php

View workflow job for this annotation

GitHub Actions / GLPI 11.0.x - php:8.5 - mariadb:11.8 / Continuous integration 

Method GlpiPlugin\Carbon\CarbonIntensity::getKnownDatesQuery() invoked with 2 parameters, 1 required.

Check failure on line 194 in src/CarbonIntensity.php

View workflow job for this annotation

GitHub Actions / GLPI 11.0.x - php:8.2 - mariadb:10.6 / Continuous integration 

Parameter #1 $source_zone of method GlpiPlugin\Carbon\CarbonIntensity::getKnownDatesQuery() expects array, string given.

Check failure on line 194 in src/CarbonIntensity.php

View workflow job for this annotation

GitHub Actions / GLPI 11.0.x - php:8.2 - mariadb:10.6 / Continuous integration 

Method GlpiPlugin\Carbon\CarbonIntensity::getKnownDatesQuery() invoked with 2 parameters, 1 required.
$request['ORDER'] = CarbonIntensity::getTableField('date') . ' ASC';
$request['LIMIT'] = '1';
$result = $DB->request($request)->current();
Expand All @@ -206,12 +206,12 @@
* Download data for a single zone
*
* @param ClientInterface $data_source
* @param string $zone_name zone name
* @param array $source_zone source_zone row from database
* @param int $limit maximum count of items to process
* @param ProgressBar $progress_bar progress bar to update (CLI mode only)
* @return int count of item downloaded
*/
public function downloadOneZone(ClientInterface $data_source, string $zone_name, int $limit = 0, ?ProgressBar $progress_bar = null): int
public function downloadOneZone(ClientInterface $data_source, array $source_zone, int $limit = 0, ?ProgressBar $progress_bar = null): int
{
$start_date = $this->getDownloadStartDate();

Expand All @@ -221,14 +221,14 @@
$source = new Source();
$source->getFromDBByCrit(['name' => $data_source->getSourceName()]);
$zone = new Zone();
$zone->getFromDBByCrit(['name' => $zone_name]);
$zone->getFromDBByCrit(['id' => $source_zone[Zone::getForeignKeyField()]]);
$gaps = $this->findGaps($source->getID(), $zone->getID(), $start_date);
if (count($gaps) === 0) {
// Log a notice specifying the source and the zone
trigger_error(sprintf(
"No gap to fill for source %s and zone %s between %s and %s",
$data_source->getSourceName(),
$zone_name,
$zone->fields['name'],
$start_date->format('Y-m-d'),
'now'
), E_USER_WARNING);
Expand All @@ -249,7 +249,7 @@
foreach ($gaps as $gap) {
$gap_start = DateTimeImmutable::createFromFormat('Y-m-d H:i:s', $gap['start']);
$gap_end = DateTimeImmutable::createFromFormat('Y-m-d H:i:s', $gap['end']);
$count = $data_source->fullDownload($zone_name, $gap_start, $gap_end, $this, $limit, $progress_bar);
$count = $data_source->fullDownload($source_zone, $gap_start, $gap_end, $this, $limit, $progress_bar);
$total_count += $count;
if ($total_count >= $limit) {
return $total_count;
Expand Down
11 changes: 5 additions & 6 deletions src/CronTask.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -273,18 +273,17 @@ public static function downloadCarbonIntensityFromSource(GlpiCronTask $task, Cli
$task->addVolume($done_count);
}

$zones = $data_source->getZones(['is_download_enabled' => 1]);
if (count($zones) === 0) {
$sources_zones = $data_source->getSourceZones(['is_download_enabled' => 1]);
if (count($sources_zones) === 0) {
trigger_error(__('No zone to download', 'carbon'), E_USER_WARNING);
return 0;
}

$limit_per_zone = max(1, floor(($remaining) / count($zones)));
$limit_per_zone = max(1, floor(($remaining) / count($sources_zones)));
$count = 0;
foreach ($zones as $zone) {
$zone_name = $zone['name'];
foreach ($sources_zones as $source_zone) {
try {
$added = $intensity->downloadOneZone($data_source, $zone_name, $limit_per_zone);
$added = $intensity->downloadOneZone($data_source, $source_zone, $limit_per_zone);
} catch (RuntimeException $e) {
trigger_error($e->getMessage(), E_USER_WARNING);
continue;
Expand Down
69 changes: 56 additions & 13 deletions src/DataSource/CarbonIntensity/AbstractClient.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,24 +61,24 @@ abstract protected function formatOutput(array $response, int $step): array;
* Download all data for a single day from the datasource
*
* @param DateTimeImmutable $day
* @param string $zone
* @param array $source_zone
* @return array
*
* @throws AbortException if an error requires to stop all subsequent fetches
*/
abstract public function fetchDay(DateTimeImmutable $day, string $zone): array;
abstract public function fetchDay(DateTimeImmutable $day, array $source_zone): array;

/**
* Download a range if data from the data source
*
* @param DateTimeImmutable $start
* @param DateTimeImmutable $stop
* @param string $zone
* @param array $source_zone
* @return array
*
* @throws AbortException if an error requires to stop all subsequent fetches
*/
abstract public function fetchRange(DateTimeImmutable $start, DateTimeImmutable $stop, string $zone): array;
abstract public function fetchRange(DateTimeImmutable $start, DateTimeImmutable $stop, array $source_zone): array;

public function disableCache()
{
Expand Down Expand Up @@ -139,7 +139,46 @@ public function getZones(array $crit = []): array
$zone_fk = Zone::getForeignKeyField();
$source_zone_table = Source_Zone::getTable();
$iterator = $DB->request([
'SELECT' => Zone::getTableField('name'),
'SELECT' => [
Zone::getTableField('name')
],
'FROM' => $zone_table,
'INNER JOIN' => [
$source_zone_table => [
'ON' => [
$zone_table => 'id',
$source_zone_table => $zone_fk,
],
],
$source_table => [
'ON' => [
$source_table => 'id',
$source_zone_table => $source_fk,
],
],
],
'WHERE' => [
Source::getTableField('name') => $this->getSourceName(),
] + $crit,
]);

return iterator_to_array($iterator);
}

public function getSourceZones(array $crit = []): array
{
/** @var DBmysql $DB */
global $DB;

$source_table = Source::getTable();
$source_fk = Source::getForeignKeyField();
$zone_table = Zone::getTable();
$zone_fk = Zone::getForeignKeyField();
$source_zone_table = Source_Zone::getTable();
$iterator = $DB->request([
'SELECT' => [
getTableForItemType(Source_Zone::class) . '.*',
],
'FROM' => $zone_table,
'INNER JOIN' => [
$source_zone_table => [
Expand All @@ -163,7 +202,7 @@ public function getZones(array $crit = []): array
return iterator_to_array($iterator);
}

public function fullDownload(string $zone, DateTimeImmutable $start_date, DateTimeImmutable $stop_date, CarbonIntensity $intensity, int $limit = 0, ?ProgressBar $progress_bar = null): int
public function fullDownload(array $source_zone, DateTimeImmutable $start_date, DateTimeImmutable $stop_date, CarbonIntensity $intensity, int $limit = 0, ?ProgressBar $progress_bar = null): int
{
if ($start_date >= $stop_date) {
return 0;
Expand Down Expand Up @@ -193,17 +232,19 @@ public function fullDownload(string $zone, DateTimeImmutable $start_date, DateTi
$data = $this->fetchRange(
DateTimeImmutable::createFromMutable($current_date),
DateTimeImmutable::createFromMutable($next_month),
$zone
$source_zone
);
} catch (AbortException $e) {
break;
}
if (count($data) > 0) {
$data = $this->formatOutput($data, $this->step);
if (!isset($data[$zone])) {
$source = Source::getById($source_zone[Source::getForeignKeyField()]);
$zone = Zone::getById($source_zone[Zone::getForeignKeyField()]);
if (!isset($data[$zone->fields['name']])) {
break;
}
$saved = $intensity->save($zone, $this->getSourceName(), $data[$zone]);
$saved = $intensity->save($zone->fields['name'], $source->fields['name'], $data[$zone]);
if ($progress_bar) {
$progress_bar->advance($saved);
}
Expand All @@ -222,23 +263,25 @@ public function fullDownload(string $zone, DateTimeImmutable $start_date, DateTi
return $saved > 0 ? $count : -$count;
}

public function incrementalDownload(string $zone, DateTimeImmutable $start_date, CarbonIntensity $intensity, int $limit = 0): int
public function incrementalDownload(array $source_zone, DateTimeImmutable $start_date, CarbonIntensity $intensity, int $limit = 0): int
{
$end_date = new DateTimeImmutable('now');

$count = 0;
$saved = 0;
foreach ($this->sliceDateRangeByDay($start_date, $end_date) as $slice) {
try {
$data = $this->fetchDay($slice, $zone);
$data = $this->fetchDay($slice, $source_zone);
} catch (AbortException $e) {
throw $e;
}
$data = $this->formatOutput($data, $this->step);
if (!isset($data[$zone])) {
$source = Source::getById($source_zone[Source::getForeignKeyField()]);
$zone = Zone::getById($source_zone[Zone::getForeignKeyField()]);
if (!isset($data[$zone->fields['name']])) {
continue;
}
$saved = $intensity->save($zone, $this->getSourceName(), $data[$zone]);
$saved = $intensity->save($zone->fields['name'], $source->fields['name'], $data[$zone]);
$count += abs($saved);
if ($limit > 0 && $count >= $limit) {
return $saved > 0 ? $count : -$count;
Expand Down
16 changes: 12 additions & 4 deletions src/DataSource/CarbonIntensity/ClientInterface.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,14 @@ public function getSupportedZones(): array;
*/
public function getZones(array $crit = []): array;

/**
* get relations between the source and the zones
*
* @param array $crit
* @return array
*/
public function getSourceZones(array $crit = []): array;

public function getMaxIncrementalAge(): DateTimeImmutable;

/**
Expand All @@ -175,24 +183,24 @@ public function getDataInterval(): string;
* If the returned count is negative, then something went wrong
* and the absolute value of the count tells how many items were saved
*
* @param string $zone
* @param array $source_zone
* @param DateTimeImmutable $start_date date where the download must start
* @param DateTimeImmutable $stop_date date where the download must start
* @param CarbonIntensity $intensity Instance used to update the database
* @param int $limit
* @param ProgressBar $progress progress bar to update during the download (CLI)
* @return int count of successfully saved items
*/
public function fullDownload(string $zone, DateTimeImmutable $start_date, DateTimeImmutable $stop_date, CarbonIntensity $intensity, int $limit = 0, ?ProgressBar $progress = null): int;
public function fullDownload(array $source_zone, DateTimeImmutable $start_date, DateTimeImmutable $stop_date, CarbonIntensity $intensity, int $limit = 0, ?ProgressBar $progress = null): int;

/**
* Download recent carbon intensity data day by day
*
* @param string $zone zone to process
* @param array $source_zone
* @param DateTimeImmutable $start_date DAte where the downlos must begin
* @param CarbonIntensity $intensity Instance of CarbonIntensity to use to save data
* @param int $limit maximum count of items to process
* @return int count of processed items. Negative count on failure
*/
public function incrementalDownload(string $zone, DateTimeImmutable $start_date, CarbonIntensity $intensity, int $limit = 0): int;
public function incrementalDownload(array $source_zone, DateTimeImmutable $start_date, CarbonIntensity $intensity, int $limit = 0): int;
}
Loading
Loading