Enhancement of Document Merging

[aditeyabaral]

Document merging can be enhanced in a few ways like

• Addition of neural coreferencing to retain more information while merging and comparing
• Better merging algorithms involving contextual analysis

[issue-label-bot]

Issue-Label Bot is automatically applying the label feature_request to this issue, with a confidence of 0.99. Please mark this comment with 👍 or 👎 to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

[sakares]

Hi @aditeyabaral,

I am interested in this issue but have no clue where to start.
Do you have any list of interesting algorithms/papers to try with? Thanks

[aditeyabaral]

Hey @sakares
You could look into topic modelling techniques to identify all sentences that belong to a topic and pick those sentences (after preserving their order). You could also try finding all the unique concepts after clustering the documents and picking sentences from each concept and combine them in order.

This should be done after using neural coreference to preprocess the text.

I am open to any other suggestions as well since I do not have a fixed solution to this issue :)

[sakares]

Thanks! I was thinking about Huggingface neural coreference as well. I will try this and would update you back.