Skip to content

Fetching interval data to arrow (and thus pandas) doesn't quite work using the REST API #950

@jimfulton

Description

@jimfulton

Given:

create table i (v interval)

insert into i (v) values (make_interval(1, 2, 3, 4, 5, 6))

Trying to fetch data as arrow:

>>> client.query("select * from i").to_arrow(create_bqstorage_client=False)
/home/jim/p/g/python-bigquery/google/cloud/bigquery/_pandas_helpers.py:244: UserWarning: Unable to determine type for field 'v'.
  warnings.warn("Unable to determine type for field '{}'.".format(bq_field.name))
pyarrow.Table
v: string

This is because BQ_TO_ARROW_SCALARS doesn't have an entry for INTERVAL.

I think we should stamp the array with an interval extension type, and convert timedelta when we covert to pandas.

Also #949

Metadata

Metadata

Assignees

Labels

api: bigqueryIssues related to the googleapis/python-bigquery API.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions