-
Notifications
You must be signed in to change notification settings - Fork 9
Processing Errors are now Stored in the ElasticSearch Index #48
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…nses and developers
…g, but no parent processing yet. Also no storage yet
…nd NOT optimized, but working
…ified in file itself, but in parent
…the whole parent hierarchy. Parents are only downloaded once, however, currently for every POM, not on-demand.
…least one version / attribute failed to resolve locally. However, if any parent is required the whole hierarchy will be downloaded! Fixed a bug in test shutdown.
…'error' in the 'delphi' index
Kudos, SonarCloud Quality Gate passed!
|
This was referenced Oct 21, 2021
Closed as this functionality is now part of the redesign proposed in #50 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reason for this PR
As decribed in #31, the Delphi Crawler does not keep track of any errors that may occur during processing of artifacts. If for example the JAR download fails for any reason, the respective artifact will not be stored in the ElasticSearch index, and there is no way of retrying the download at a later point in time.
Changes in this PR (changes relative to #47, needs to be merged first!)
The
MavenDiscoveryProcess
now accounts for different types of processing errors. These errors are caught and redirected to a dedicated actor, which stores them as a new mapping type (MavenProcessingError
) in the existing ElasticSearch index nameddelphi
. The initial database setup has been adapted so that the new mapping type is created on startup (for fresh DB instances only!). AMavenProcessingError
has four properties:type
: EitherPomDownloadFailed
,JarDownloadFailed
,PomParsingFailed
orHermesProcessingFailed
(It must be noted that, unlike in previous versions of the crawler, a failed JAR download is no longer an error if the POM file'spackaging
attribute is not set tojar
)occurred
: Datetime at which the error occurredidentifier
: Maven identifier referencing the artifact for which the error occurredmessage
: An error message that describes the error causeCloses #31.
Sample ElasticSearch Request
GET <elasticsearch>/delphi/error/_search