Add fix options for integrity issues #12495

priyanshu16095 · 2025-02-12T22:13:47Z

Fixes #11419

This PR introduces a functionality to fix integrity issues either individually or all at once by selecting a type.

Mandatory checks

I own the copyright of the code submitted and I licence it under the MIT license
Change in CHANGELOG.md described in a way that is understandable for the average user (if change is visible to the user)
Tests created for changes (if applicable)
Manually tested changed features in running JabRef (always required)
Screenshots added in PR description (for UI changes)
Checked developer's documentation: Is the information available and up to date? If not, I outlined it in this pull request.
Checked documentation: Is the information available and up to date? If not, I created an issue at https://github.com/JabRef/user-documentation/issues or, even better, I submitted a pull request to the documentation repository.

priyanshu16095 · 2025-02-12T22:14:47Z

Recording.2025-02-13.mp4

There is a problem with selecting messages using the Shift key. One has to click on a message first, as the focus moves to the entry editor. However, when clicking again with the mouse, all the selected entries are lost.

priyanshu16095 · 2025-02-12T22:15:38Z

Recording.2025-02-13.mp4

github-actions

JUnit tests are failing. In the area "Some checks were not successful", locate "Tests / Unit tests (pull_request)" and click on "Details". This brings you to the test output.

You can then run these tests in IntelliJ to reproduce the failing tests locally. We offer a quick test running howto in the section Final build system checks in our setup guide.

priyanshu16095 · 2025-02-16T19:44:57Z

Please review

priyanshu16095 · 2025-02-19T19:42:07Z

I have refactored the code a lot and used better practices.
I have used an enum as, without it I would have to use switch-case in every method.
Please guide me on whether I am moving in the right direction.

github-actions

JUnit tests are failing. In the area "Some checks were not successful", locate "Tests / Unit tests (pull_request)" and click on "Details". This brings you to the test output.
You can then run these tests in IntelliJ to reproduce the failing tests locally. We offer a quick test running howto in the section Final build system checks in our setup guide.

github-actions

JUnit tests are failing. In the area "Some checks were not successful", locate "Tests / Unit tests (pull_request)" and click on "Details". This brings you to the test output.
You can then run these tests in IntelliJ to reproduce the failing tests locally. We offer a quick test running howto in the section Final build system checks in our setup guide.

github-actions

JUnit tests are failing. In the area "Some checks were not successful", locate "Tests / Unit tests (pull_request)" and click on "Details". This brings you to the test output.
You can then run these tests in IntelliJ to reproduce the failing tests locally. We offer a quick test running howto in the section Final build system checks in our setup guide.

priyanshu16095 · 2025-02-20T18:00:46Z

Recording.2025-02-20.mp4

The feature is completed.
The fixes for the issue are left to be added.

src/main/java/org/jabref/logic/integrity/NoteChecker.java

koppor · 2025-03-19T20:57:17Z

After this PR is merged, one can reply at https://discourse.jabref.org/t/how-to-solve-quality-integrity-issues/1454/4?u=koppor

src/main/java/org/jabref/gui/integrity/IntegrityCheckDialog.java

koppor

The issue seems to be too hard.

No use of formatters at package org.jabref.logic.formatter.bibtexfields, but some hard-coded fixes.

Will takle longer to get this right.

koppor · 2025-03-19T21:25:33Z

src/main/java/org/jabref/gui/integrity/IntegrityCheckDialogViewModel.java

+        }
+    }
+
+    public void maskTitle(IntegrityMessage message) {


This code seems like AI generated.

I think, I pointed you to org.jabref.logic.cleanup.CleanupJob / org.jabref.logic.cleanup.Formatter

Maybe, we should start on working on a single integrity issue - discuss the archtiecture and then do a second one.

For pages the formatter to use is org.jabref.logic.formatter.bibtexfields.NormalizePagesFormatter

I am not sure why it feels AI-generated, it is basic string manipulation, as described in the issue.
Sure, I will use the describe classes.

Following puzzled me very much

updateField(message, StandardField.PAGES, "1-10");

Please tell me about the confusion, I am not clear about it.
I created this method because this code was being used in every method for updating fields.

Why I created this method?
Capital letters are not masked using curly brackets {} --> Basic string manipulation as mentioned in the issue, so to update the field I found this way
Map<Field, String> fields = new HashMap<>();
fields.put(field, newValue);
message.entry().setField(fields);
Then to impove the code reusablity, I created the method updateField.

From next time I will ask rather then pushing based on assumptions.

Very OK to push - we can then read code. This is much more easy.

I think, this task is really large. (note the "size: big" label).

Thus, it takes time. Will be a back and forth.

The clenaup jobs should do the cleaning, not manually. Look into how the cleanup dialog works.

(I currently cannot do deeper hints. I can try to draft the architecture next week if we cannot manage to move forward efficiently here)

koppor · 2025-03-19T23:57:09Z

It lists all the messages that are present in the table. I will improve this. I will identify issues that cannot be fixed automatically and remove them from the dropdown or AI can be used to fix these.

No need to add AI. Just use existing cleanup actions (see docs.jabref.org/finding-sorting-and-cleaning-entries/saveactions for a list)

Note that I linked the actions in the screenshot at #12495 (review) - and I hoped, you could find the code starting from that 😅

priyanshu16095 · 2025-03-20T00:05:28Z

In some of methods like formatPageNumberRange, replaceNonASCIICharacters, ensureValidEdition, as well as writing tests for them, I have used AI due to lack of understanding about the fields, I should rather ask or search on the web.

I have planned to ask for improvements about the fixes, which is why I have created fixes for only some issues to limit the usage of AI.

Everything else was planned by me, it took me so much iterations from using switch-case in every method to make use of enum, then solving decoupling issues. I used AI just for writing fix methods due to lack of knowledge.

subhramit · 2025-03-20T00:29:26Z

In some of methods like formatPageNumberRange, replaceNonASCIICharacters, ensureValidEdition, as well as writing tests for them, I have used AI due to lack of understanding about the fields, I should rather ask or search on the web.

I have planned to ask for improvements about the fixes, which is why I have created fixes for only some issues to limit the usage of AI.

Everything else was planned by me, it took me so much iterations from using switch-case in every method to make use of enum, then solving decoupling issues. I used AI just for writing fix methods due to lack of knowledge.

Hi, it's fine that you learnt what AI does when you use it without full understanding (issue with >95% PRs we see these days) - and thanks for being honest about it. Please be more careful next time, we are all here to answer questions.

Also, review AI code very well before using. Even if you didn't have idea about fields, the hardcoding of pages "1-10" in updateField(message, StandardField.PAGES, "1-10"); should have given you a hint that something is out of place (I am guessing that's what @koppor was referring to).

Edit: this was answer to your question in #12495 (comment):

Please tell me about the confusion, I am not clear about it.
I created this method because this code was being used in every method for updating fields.

trag-bot · 2025-03-31T17:59:24Z

src/main/java/org/jabref/logic/integrity/IntegrityIssue.java

+    public static Optional<IntegrityIssue> fromMessage(IntegrityMessage message) {
+        return Arrays.stream(IntegrityIssue.values())
+                     .filter(symbol -> symbol.getText().equals(message.message()))
+                     .findFirst();
+    }


The method fromMessage does not handle null input for the message parameter, which could lead to a NullPointerException.

src/main/java/org/jabref/gui/integrity/IntegrityCheckDialog.java

priyanshu16095 · 2025-03-31T19:08:10Z

Made the combo box display real-time values.

Recording.2025-04-01.mp4

src/main/java/org/jabref/gui/integrity/IntegrityCheckDialog.java

calixtus · 2025-04-03T18:59:04Z

src/main/java/org/jabref/gui/integrity/IntegrityCheckDialog.java

+        entryTypeCombo.getSelectionModel().selectFirst();
+    }
+
+    public void fix(IntegrityIssue issue, IntegrityMessage message) {


Fix seems a bit slang. "resolveIssue" maybe?

calixtus · 2025-04-03T18:59:57Z

src/main/java/org/jabref/gui/integrity/IntegrityCheckDialogViewModel.java

    }

    public ObservableList<IntegrityMessage> getMessages() {
        return messages;
    }
+
+    public void fix(IntegrityIssue issue, IntegrityMessage message) {
+        // fixes


calixtus · 2025-04-03T19:10:00Z

src/main/java/org/jabref/logic/integrity/ASCIICharacterChecker.java

@@ -21,7 +20,7 @@ public List<IntegrityMessage> check(BibEntry entry) {
        for (Map.Entry<Field, String> field : entry.getFieldMap().entrySet()) {
            boolean asciiOnly = CharMatcher.ascii().matchesAllOf(field.getValue());
            if (!asciiOnly) {
-                results.add(new IntegrityMessage(Localization.lang("Non-ASCII encoded character found"), entry,
+                results.add(new IntegrityMessage(IntegrityIssue.NON_ASCII_ENCODED_CHARACTER_FOUND.getText(), entry,


Goes a bit beyond the scope of this PR, but you could refactor the IntegrityMessage to accept just the symbol and to resolve the symbol to a string at the caller.

priyanshu16095 · 2025-04-03T19:13:59Z

Some changes I haven't commited such as removed the issuesListProperty, because I am working on it.

src/main/java/org/jabref/logic/integrity/DoiDuplicationChecker.java

src/main/java/org/jabref/logic/integrity/IntegrityMessage.java

trag-bot · 2025-04-10T13:04:24Z

src/main/java/org/jabref/gui/integrity/IntegrityCheckDialogViewModel.java

    }

    public ObservableList<IntegrityMessage> getMessages() {
        return messages;
    }
+
+    public void resolveIssue(IntegrityIssue issue, IntegrityMessage message) {


The method 'resolveIssue' is public and should not return null. It should use java.util.Optional to handle potential null values.

jabref-machine · 2025-04-10T13:12:33Z

Note that your PR will not be reviewed/accepted until you have gone through the mandatory checks in the description and marked each of them them exactly in the format of [x] (done), [ ] (not done yet) or [/] (not applicable).

Add fix options for integrity issues

9b11edf

This comment was marked as outdated.

Sign in to view

Fix the check

02d8e75

This comment was marked as outdated.

Sign in to view

Add a missing entry for Localization.lang

754b6ad

This comment was marked as outdated.

Sign in to view

priyanshu16095 added 2 commits February 13, 2025 05:06

Fix the check

696f904

Refactor code

4ab36a8

github-actions bot reviewed Feb 14, 2025

View reviewed changes

priyanshu16095 added 2 commits February 14, 2025 18:55

Fix the check

28de796

Add a CHANGELOG.md entry

70ae8eb

Refactor the code

77df6ea

github-actions bot reviewed Feb 19, 2025

View reviewed changes

priyanshu16095 added 2 commits February 20, 2025 16:47

Make use of IntegrityIssue enum

30c81ce

Add keys in Localization.lang

5b389da

github-actions bot reviewed Feb 20, 2025

View reviewed changes

priyanshu16095 added 2 commits February 20, 2025 22:10

Refactor the code

3a0d9a1

Remove use of enum

a04e06a

github-actions bot reviewed Feb 20, 2025

View reviewed changes

priyanshu16095 added 4 commits February 20, 2025 22:17

Remove use of enum from PageChecker class

8e9341a

Add fixAll method

235c74f

Add a check for fixAll and fixByType method

9839137

Fix a bug

5754680

priyanshu16095 added 2 commits February 23, 2025 01:09

Use dialogService.notify() to notify

8f7bdc2

Remove unused imports

f57719c

trag-bot bot reviewed Mar 17, 2025

View reviewed changes

src/main/java/org/jabref/logic/integrity/NoteChecker.java Show resolved Hide resolved

koppor added 2 commits March 19, 2025 22:15

Try to remove diagService parameter

8502e76

Fix indent

a717a45

trag-bot bot reviewed Mar 19, 2025

View reviewed changes

src/main/java/org/jabref/gui/integrity/IntegrityCheckDialog.java Show resolved Hide resolved

koppor requested changes Mar 19, 2025

View reviewed changes

priyanshu16095 marked this pull request as draft March 20, 2025 01:52

priyanshu16095 added 4 commits March 21, 2025 04:31

Remove fixes and tests

ee71b7f

Merge branch 'main' into integrityCheckActions

b979659

Remove integrityIssues.bib file

0656364

Ensure latest values in combo box

1181dda

trag-bot bot reviewed Mar 31, 2025

View reviewed changes

priyanshu16095 added 5 commits April 1, 2025 00:07

Make the combo box real time

c9b3df2

Fix the check

7973711

Fix the check again

31fecf8

Merge branch 'main' into integrityCheckActions

9c2a7eb

Remove updateItems() method

fcdeea2

trag-bot bot reviewed Mar 31, 2025

View reviewed changes

src/main/java/org/jabref/gui/integrity/IntegrityCheckDialog.java Show resolved Hide resolved

calixtus reviewed Apr 3, 2025

View reviewed changes

Add new constructor for IntegrityMessage

c60042f

trag-bot bot reviewed Apr 9, 2025

View reviewed changes

src/main/java/org/jabref/logic/integrity/DoiDuplicationChecker.java Show resolved Hide resolved

trag-bot bot reviewed Apr 9, 2025

View reviewed changes

src/main/java/org/jabref/logic/integrity/IntegrityMessage.java Show resolved Hide resolved

Rename variable

1f08cfa

trag-bot bot reviewed Apr 10, 2025

View reviewed changes

Uh oh!

Add fix options for integrity issues #12495

Are you sure you want to change the base?

Add fix options for integrity issues #12495

Uh oh!

Conversation

priyanshu16095 commented Feb 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Fixes #11419

Mandatory checks

Uh oh!

priyanshu16095 commented Feb 12, 2025

Uh oh!

priyanshu16095 commented Feb 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

priyanshu16095 commented Feb 16, 2025

Uh oh!

priyanshu16095 commented Feb 19, 2025

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

priyanshu16095 commented Feb 20, 2025

Uh oh!

Uh oh!

koppor commented Mar 19, 2025

Uh oh!

Uh oh!

koppor left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

koppor commented Mar 19, 2025

Uh oh!

priyanshu16095 commented Mar 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

subhramit commented Mar 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

trag-bot bot Mar 31, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

priyanshu16095 commented Mar 31, 2025

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

priyanshu16095 commented Apr 3, 2025

Uh oh!

priyanshu16095 commented Feb 12, 2025 •

edited

Loading

priyanshu16095 commented Feb 12, 2025 •

edited

Loading

priyanshu16095 commented Mar 20, 2025 •

edited

Loading

subhramit commented Mar 20, 2025 •

edited

Loading