Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

tryAdd additional handlings #51

Merged
merged 6 commits into from
Mar 21, 2024
Merged

tryAdd additional handlings #51

merged 6 commits into from
Mar 21, 2024

Conversation

xmichele
Copy link
Contributor

@xmichele xmichele commented Mar 20, 2024 •
edited
Loading 

root@a5aa704a62a5:/app# Stars copy --harvest -conf=/usersettings.json -si CDS -rel -r 4 -v -o ./out /catalog.json
Stars/2.21.0
Loading Plugins
Plugin [AMAZON] injected
Plugin [ASF] injected
Plugin [CDS] injected
Plugin [CDS1] injected
Plugin [CDS2] injected
Plugin [CDS3] injected
Plugin [CDS4] injected
Plugin [CDS5] injected
Plugin [CREO] injected
Plugin [GOOGLE] injected
Plugin [MUNDI] injected
Plugin [ONDA] injected
Plugin [PLANETSCOPE] injected
Plugin [SOBLOO] injected
Plugin [USGS] injected
Plugin [ABAE] injected
Plugin [AIRBUS] injected
Plugin [ALOS2] injected
Plugin [BKA] injected
Plugin [BLACKSKY] injected
Plugin [CONAE] injected
Plugin [CSK] injected
Plugin [DIMAP] injected
Plugin [GAOFEN] injected
Plugin [GAOFEN-3] injected
Plugin [GDAL] injected
Plugin [GEOEYE] injected
Plugin [ICEYE] injected
Plugin [INPE] injected
Plugin [ISRO] injected
Plugin [KANOPUS-V] injected
Plugin [KOMPSAT-3] injected
Plugin [KOMPSAT-5] injected
Plugin [LANDSAT8] injected
Plugin [LANDSAT9] injected
Plugin [NEWSAT] injected
Plugin [PLANET] injected
Plugin [RCM] injected
Plugin [RESURSP] injected
Plugin [SAOCOM] injected
Plugin [SENTINEL-1] injected
Plugin [SENTINEL-2] injected
Plugin [TERRASAR-X] injected
Plugin [WORLDVIEW] injected
Trying to init a root catalog catalog on file:///app/out/
Root catalog catalog created at file:///app/out/catalog.json
Copy node output_stac from file:///catalog.json
The environment variables AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY/AWS_SESSION_TOKEN were not set with AWS credentials.
No configured AWS credentials. Defaulting to instance profile credentials (EC2)
Registering extension: atom
Registering extension: json
Registering extension: atom
Registering extension: json
Registering extension: atom
Registering extension: json
Registering extension: atom
Registering extension: json
Registering extension: atom
Registering extension: json
Registering extension: atom
Registering extension: json
Registering extension: atom
Registering extension: json
Registering extension: atom
Registering extension: json
Registering extension: atom
Registering extension: json
Registering extension: atom
Registering extension: json
Registering extension: atom
Registering extension: json
ONDA ODATA V2 API identified
Registering extension: atom
Registering extension: json
Registering extension: atom
Registering extension: json
[Copernicus OData search (https://catalogue.dataspace.copernicus.eu/odata/v1)] Searching for file:///catalog.json
Request parameters:
- $filter = Name eq 'output_stac'
- $top = 20
- $expand = Attributes
[Copernicus OData search (https://catalogue.dataspace.copernicus.eu/odata/v1)] --> no supply possible
Extracting asset file:///urn_ogc_def_EOP_CNSA_GF1PMS13271908001-product.tar.gz...
GF1_PMS1_W71.6_S33.2_20240205_L1A13271908001.jpg
GF1_PMS1_W71.6_S33.2_20240205_L1A13271908001-MSS1.dbf
GF1_PMS1_W71.6_S33.2_20240205_L1A13271908001-MSS1.jpg
GF1_PMS1_W71.6_S33.2_20240205_L1A13271908001-MSS1.rpb
GF1_PMS1_W71.6_S33.2_20240205_L1A13271908001-MSS1.rpb.aux.xml
GF1_PMS1_W71.6_S33.2_20240205_L1A13271908001-MSS1.rpb.gcp
GF1_PMS1_W71.6_S33.2_20240205_L1A13271908001-MSS1.shp
GF1_PMS1_W71.6_S33.2_20240205_L1A13271908001-MSS1.shx
GF1_PMS1_W71.6_S33.2_20240205_L1A13271908001-MSS1_thumb.jpg
GF1_PMS1_W71.6_S33.2_20240205_L1A13271908001-MSS1.tiff
GF1_PMS1_W71.6_S33.2_20240205_L1A13271908001-MSS1.xml
GF1_PMS1_W71.6_S33.2_20240205_L1A13271908001-PAN1.dbf
GF1_PMS1_W71.6_S33.2_20240205_L1A13271908001-PAN1.jpg
GF1_PMS1_W71.6_S33.2_20240205_L1A13271908001-PAN1.rpb
GF1_PMS1_W71.6_S33.2_20240205_L1A13271908001-PAN1.rpb.aux.xml
GF1_PMS1_W71.6_S33.2_20240205_L1A13271908001-PAN1.rpb.gcp
GF1_PMS1_W71.6_S33.2_20240205_L1A13271908001-PAN1.shp
GF1_PMS1_W71.6_S33.2_20240205_L1A13271908001-PAN1.shx
GF1_PMS1_W71.6_S33.2_20240205_L1A13271908001-PAN1_thumb.jpg
GF1_PMS1_W71.6_S33.2_20240205_L1A13271908001-PAN1.tiff
GF1_PMS1_W71.6_S33.2_20240205_L1A13271908001-PAN1.xml
GF1_PMS1_W71.6_S33.2_20240205_L1A13271908001_thumb.jpg
GF1_PMS1_W71.6_S33.2_20240205_L1A13271908001.xml
order.xml
Retrieving the metadata files in the product package
Metadata file is file:///app/out/GF1_PMS1_W71.6_S33.2_20240205_L1A13271908001-MSS1.xml
Deserializing metadata files
Metadata files deserialized. Starting metadata generation
Metadata file is file:///app/out/GF1_PMS1_W71.6_S33.2_20240205_L1A13271908001-PAN1.xml
Deserializing metadata files
Metadata files deserialized. Starting metadata generation
Metadata file is file:///app/out/GF1_PMS1_W71.6_S33.2_20240205_L1A13271908001.xml
Deserializing metadata files
Metadata files deserialized. Starting metadata generation
System.ArgumentException: An item with the same key has already been added. Key: file:size
   at System.Collections.Generic.Dictionary`2.TryInsert(TKey key, TValue value, InsertionBehavior behavior)
   at System.Collections.Generic.Dictionary`2.System.Collections.Generic.ICollection<System.Collections.Generic.KeyValuePair<TKey,TValue>>.Add(KeyValuePair`2 keyValuePair)
   at Stac.StacAccessorsHelpers.AddRange[T](ICollection`1 collection, IEnumerable`1 items)
   at Terradue.Stars.Data.Model.Metadata.Gaofen.GaofenMetadataExtractor.AddAssetAsync(StacItem stacItem, String satelliteId, IAsset asset, IAssetsContainer assetsContainer) in /source/src/Stars.Data/Model/Metadata/Gaofen1-2-4/GaofenMetadataExtractor.cs:line 268
   at Terradue.Stars.Data.Model.Metadata.Gaofen.GaofenMetadataExtractor.AddAssetsAsync(StacItem stacItem, String satelliteId, IAssetsContainer assetsContainer) in /source/src/Stars.Data/Model/Metadata/Gaofen1-2-4/GaofenMetadataExtractor.cs:line 192
   at Terradue.Stars.Data.Model.Metadata.Gaofen.GaofenMetadataExtractor.ExtractMetadata(IItem item, String suffix) in /source/src/Stars.Data/Model/Metadata/Gaofen1-2-4/GaofenMetadataExtractor.cs:line 88
   at Terradue.Stars.Data.Model.Metadata.MetadataExtraction.ProcessAsync(IResource resource, IDestination destination, CancellationToken ct, String suffix) in /source/src/Stars.Data/Model/Metadata/MetadataExtraction.cs:line 57
   at Terradue.Stars.Services.Processing.ProcessingService.ExtractMetadataAsync(StacItemNode itemNode, IDestination destination, StacStoreService storeService, CancellationToken ct) in /source/src/Stars.Services/Processing/ProcessingService.cs:line 83
   at Terradue.Stars.Console.Operations.CopyOperation.CopyNodeAsync(IResource node, IRouter router, Object state, CancellationToken ct) in /source/src/Stars.Console/Operations/CopyOperation.cs:line 308
   at Terradue.Stars.Services.Router.RouterService.RouteAsync(IResource route, Int32 recursivity, IRouter prevRouter, Object state, CancellationToken ct) in /source/src/Stars.Services/Router/RouterService.cs:line 78
   at Terradue.Stars.Console.Operations.CopyOperation.ExecuteAsync(CancellationToken ct) in /source/src/Stars.Console/Operations/CopyOperation.cs:line 371
   at Terradue.Stars.Console.Operations.BaseOperation.OnExecuteAsync(CancellationToken ct) in /source/src/Stars.Console/Operations/BaseOperation.cs:line 73
   at McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention.InvokeAsync(MethodInfo method, Object instance, Object[] arguments)
   at McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention.OnExecute(ConventionContext context, CancellationToken cancellationToken)
   at McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention.<>c__DisplayClass0_0.<<Apply>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.ExecuteAsync(String[] args, CancellationToken cancellationToken)
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.Execute(String[] args)
   at Terradue.Stars.Console.StarsApp.Main(String[] args) in /source/src/Stars.Console/Stars.cs:line 31

floeschau
floeschau reviewed Mar 21, 2024
View reviewed changes
src/Stars.Data/Model/Metadata/Gaofen1-2-4/GaofenMetadataExtractor.cs Outdated
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please change indentation after the newly introduced conditions, possibly with parenthesis

if ...
{
    return
}

floeschau
floeschau requested changes Mar 21, 2024
View reviewed changes
Copy link
Collaborator

@floeschau floeschau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See comment for src/Stars.Data/Model/Metadata/Gaofen1-2-4/GaofenMetadataExtractor.cs (regarding formatting).
Otherwise change is fine.

@codecov-commenter
Copy link

Codecov Report

Attention: Patch coverage is 34.14634% with 27 lines in your changes are missing coverage. Please review.

Project coverage is 10.86%. Comparing base (ce472ee) to head (3c6eae6).
Report is 1 commits behind head on develop.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           develop      #51      +/-   ##
===========================================
+ Coverage    10.84%   10.86%   +0.02%     
===========================================
  Files          240      240              
  Lines        35737    35726      -11     
  Branches      3150     3156       +6     
===========================================
+ Hits          3874     3880       +6     
+ Misses       31863    31846      -17
Flag Coverage Δ
unittests 10.86% <34.14%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
...el/Metadata/Gaofen1-2-4/GaofenMetadataExtractor.cs 12.69% <34.14%> (+1.08%) ⬆️

... and 1 file with indirect coverage changes

@floeschau floeschau merged commit e98f1f9 into develop Mar 21, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@floeschau floeschau floeschau approved these changes

Assignees

@floeschau floeschau

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@xmichele @codecov-commenter @floeschau