Add entry manually in dependant database, automatically updating master

[Hugo-Heagren]

I'm working on an paper. I have a dependant database in the same folder as the paper itself, to make keeping track of the specific literature easier. I keep everything I read in my master database though, because I don't want my overall reading to get fractured over time -- you never know when something might be relevant to another project down the line.

So I spend most of my time in ebib looking at the dependant database. While searching for literature, I want to add what I find as quickly as possible. The most obvious way for me seems to be add in the normal way to the dependant database (not by pulling it in from a master, because the entry is new to both databases, but by typing in the values), and for ebib to update the master database accordingly. I can't see a way to do this at the moment without switching and doing the actual adding of the new entry in the master. I can edit the already existing entries in the dependant database, so presumably syncing information between the two is something ebib can handle.

I would be happy to have a look at implementing this if you think it would be a good idea. Perhaps (in anaolgy with the prefix argument for editting a field), adding a prefix argument to a in the dependant database could allow me to add an entry manually?

[joostkremers]

I agree, this would be a nice thing to have. It shouldn't be too difficult to implement, either. The new entry could be added to the main database first and then included in the dependent, where it can then be edited. I also agree that this would be a good use of the prefix argument.

If you find time to implement this, feel free to do so.

[dankessler]

This issue seems to be about using ebib tools to add a new entry to a dependent (and master) database simultaneously, but I'm curious about how to handle the case where an entry is literally manually added in the dependent database.

As an example, suppose that on my computer I have a main.bib file and I begin a project with some collaborators which uses dependent database project.bib which is configured to use my main.bib as its main database. At some point, a collaborator adds (using their own text editor or some other tool) a new entry to project.bib, which as a result is no longer a subset of my main.bib.

Is the proper remedy to merge project.bib into main.bib, taking care to handle situations where there are either duplicate keys or (worse!) semantically duplicate entries with distinct keys?

[joostkremers]

This issue seems to be about using ebib tools to add a new entry to a dependent (and master) database simultaneously, but I'm curious about how to handle the case where an entry is literally manually added in the dependent database.

As an example, suppose that on my computer I have a main.bib file and I begin a project with some collaborators which uses dependent database project.bib which is configured to use my main.bib as its main database. At some point, a collaborator adds (using their own text editor or some other tool) a new entry to project.bib, which as a result is no longer a subset of my main.bib.

Is the proper remedy to merge project.bib into main.bib, taking care to handle situations where there are either duplicate keys or (worse!) semantically duplicate entries with distinct keys?

Merging is indeed the way to go: you can merge a dependent database as if it's a normal database. Entries with duplicate keys are logged but not loaded twice, so that case is handled. The entries that your collaborator adds are merged into main.bib, restoring project.bib as a subset of main.bib. You should reload project.bib after merging, of course.

By default, entries are only identified by key, though, so if your collaborator adds an entry that is already in the database but uses a different key for it, it won't be recognised as duplicate. (parsebib does have an option parsebib-hashid-fields to calculate a hash value based on the contents of one or more fields, but it's not integrated into Ebib and the field values would need to be identical to detect duplicates anyway, so I think it's of little use.)

[Hugo-Heagren]

Merging is indeed the way to go

This might point to a way to solve both issues at once. If merging for out-of-sync dependent databases could be handled succesfully, then my original feature request would be pretty simple to implement. Adding a new entry (in ebib) to a dependent database would work in exactly the same way as adding to an independent one, but then to include it in both one would just merge. Perhaps the easiest way to do this would be to add an ebib-save-hook, and then have a merging function to it for dependent databases.

Then I could add a new entry to my dependent database and save the database (as in my standard workdlow) and have the upstream file updated automatically.

(There would be some vagueries to do with unsaved changes and merging in the upstream db. If the master db is open and has unsaved changes, then to get the dependent db's changes would requie a reload, which would overwrite them. So maybe in that case the master should be force-saved before the merge, and maybe force-reloaded after it, so that everything is obviously in sync? I for one would go so far as to implement this myself (with hooks and advice) if it wasn't part of the core, but it might not be for everyone)

[joostkremers]

Actually, I think for the case of adding an entry from within Ebib there is no need to use the merge functionality. All that's needed is to check whether the user is adding an entry to a dependent database, create the entry in the main database instead and then add it to the dependent. Most of that functionality already exists, all that's needed is a bit of glue code and perhaps some very simple refactoring (e.g., ebib-add-entry doesn't take an argument, perhaps that would have to be changed, or perhaps the non-interactive stuff it does needs to be moved to a separate function).

For the case where the dependent .bib file was changed on disk, most of the functionality needed to handle that also exists already. Probably all that's needed is to modify the behaviour of ebib-reload-current-database (or ebib--reload-database) when it's reloading a dependent database: instead of just reloading, it should merge the file on disk with the main database before reloading the dependent.

If I find some time, I'll look into this, but of course I'll also take a PR. :-)