Handling backups bug

[rukayaj]

Currently:

Summary: All create/update/delete operations on Tables are recorded based on a snapshot before executing some gpt-4 generated code, and a snapshot after.

1. We get a list of active Tables for an Agent/Task, and insert them into the initial System Message
2. We then keep track of inactive Tables by flagging them with stale_at or deleted_at
3. We also link Tables with subsequent Messages in the Agent conversation, which allows us to display new/changed Tables in the UI to the user after some work has been done on them.

If the GPT generated code:

• creates a new Table, we just make a MessageTableAssociation link
• deletes a Table, that Table is flagged with a deleted_at timestamp
• edits a Table, we keep a copy of the old, unchanged Table (flagged with a stale_at timestamp), and additionally make a new MessageTableAssociation link for the new Table

Bug discovered:

1. Agent "organize_tables" gets given TableA, TableB to work with in the initial System Message
2. Agent "organize_tables" merges TableA and TableB by doing e.g. TableA = [pd.merge something TableA + TableB joined]
3. The handle_backups method correctly sees that TableA has changed, but sees no change on TableB, which is also not flagged with deleted_at, as handle_backups has no way of realising TableB is no longer relevant
4. Agent "organize_tables" marks itself as complete
5. The next Agent starts, and it gets given TableA and TableB as active_tables for this Dataset

Note that we could also have TableC = pd.merge(TableA, TableB), and we'd have TableC, TableB and TableA all going to the next agent.

[rukayaj]

Michal's idea

setAgentTaskToComplete should take a list of active TableIDs as an argument

Downsides:

1. What if it forgets which TableIDs are actually active? Is it clever enough to do this?
2. Users will be able to see the code that it's generated in the message, but not the resultant output Table I guess?

What if we forced it to call a new DisplayMessageWithTableToUser method after function calls, and it has a table_ids list argument? It gets complicated again though.

[rukayaj]

We could also just modify the prompt and tell it to delete all Tables which are no longer relevant. This would be a simple and straightforward thing to try at least.

But: I think @MichalTorma and I both have the nagging feeling that this is a complicated problem which has already been solved by jupyter notebooks. What we really want is an interactive but guided jupyter notebook. We should think about whether there an easier way of doing this.