Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upsert Functionality for BigQuery Backend #9491

Closed
1 task done
hamishcraze opened this issue Jul 1, 2024 · 3 comments
Closed
1 task done

Upsert Functionality for BigQuery Backend #9491

hamishcraze opened this issue Jul 1, 2024 · 3 comments
Labels
feature Features or general enhancements

Comments

@hamishcraze
Copy link

hamishcraze commented Jul 1, 2024

Is your feature request related to a problem?

I am frustrated by the fact that there isn't a way to upsert data to bigquery using ibis.

What is the motivation behind your request?

I am trying to make an endpoint that I can send table data to. This endpoint determines if it needs to create the table or insert data into an existing table. However, the problem is that the records I send through occasionally gets updated. And, given the current functionality, I can only upload this as a new record. This inevitably leads to duplications and complications for downstream clients as I have to try to teach them "Oh yea, you can't rely on the data here because these transactions are going to be duplicated, no you didn't get 50 000 transactions in march, you got 3 highly edited ones"

Describe the solution you'd like

I'd like if there was a dedicated upsert function that I could say "Hey dude, here's the data, match using THIS column, please figure out the rest". As of right now I could only implement table specific solutions, which would defeat the point of trying to make the API endpoint generic.

What version of ibis are you running?

3.3

What backend(s) are you using, if any?

BigQuery

Code of Conduct

  • I agree to follow this project's Code of Conduct
@hamishcraze hamishcraze added the feature Features or general enhancements label Jul 1, 2024
@cpcloud
Copy link
Member

cpcloud commented Jul 3, 2024

Out of curiosity are you actually using Ibis 3.3? We wouldn't backport this feature, so you'd have to upgrade to a release to use it.

@cpcloud
Copy link
Member

cpcloud commented Jul 3, 2024

xref: #5391

@cpcloud
Copy link
Member

cpcloud commented Jul 3, 2024

Closing in favor of #5391.

@cpcloud cpcloud closed this as not planned Won't fix, can't repro, duplicate, stale Jul 3, 2024
@github-project-automation github-project-automation bot moved this from backlog to done in Ibis planning and roadmap Jul 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature Features or general enhancements
Projects
Archived in project
Development

No branches or pull requests

2 participants