Describe the bug
I'm loading a polars DataFrame and one of the columns has negative and positive decimals. If there is a sign change, the driver throws a RuntimeError. I've tried to narrow it down, but the behavior is a little inconsistent.
Exception message: RuntimeError: Input string exceeds allowed column size at parameter index 0
Stack trace:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
df_to_sql(table_name="pytest", df=df, credential=cred)
~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\tsong\repo\pam\src\pam\sql\write_db.py", line 37, in df_to_sql
cursor.executemany(stmt, df.rows())
~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "C:\Users\tsong\repo\pam\.venv\Lib\site-packages\mssql_python\cursor.py", line 2338, in executemany
ret = ddbc_bindings.SQLExecuteMany(
self.hstmt, operation, columnwise_params, parameters_type, row_count, encoding_settings
)
At the moment, I work around this by casting all Decimal columns to Strings.
df = df.with_columns(pl.col(pl.Decimal).cast(pl.String).name.keep())
To reproduce
Include a complete code listing (or project/solution) that we can run to reproduce the issue.
Partial code listings, or multiple fragments of code, will slow down our response or cause us to push the issue back to you to provide code to reproduce the issue.
# I have a table named [pytest]
# CREATE TABLE [dbo].[pytest] ([col_1] [decimal](28, 14))
import polars as pl
import mssql_python as mssql
df = pl.DataFrame(
{
"col_1": [
Decimal("-0.1"),
Decimal("1.0"),
]
}
)
stmt = "INSERT INTO [pytest] ([col_1]) VALUES (?)"
connection_url = "Server=<SERVER_ADDRESS>;Database=<DATABASE>;Encrypt=yes;Authentication=ActiveDirectoryDefault;"
conn = mssql.connect(connection_url)
cursor = conn.cursor()
cursor.executemany(stmt, df.rows())
cursor.commit()
conn.close()
Expected behavior
Insert should be successful.
Further technical details
Python version: 3.14.4
SQL Server version: Azure SQL Managed Instance (locked to SQL Server 2022)
Operating system: Windows 11
Additional context
There's definitely something weird going on with how the driver is inserting decimals with executemany. The examples below all work normally:
# These dataframes all work
df = pl.DataFrame(
{
"col_1": [
Decimal("-1.1"),
Decimal("1.0"),
]
}
)
df = pl.DataFrame(
{
"col_1": [
Decimal("-0.1"),
Decimal("-1.0"),
]
}
)
df = pl.DataFrame(
{
"col_1": [
Decimal("0.1"),
Decimal("1.0"),
]
}
)
df = pl.DataFrame(
{
"col_1": [
Decimal("0.1"),
Decimal("-1.0"),
]
}
)
Weirdly enough, the order also affects the outcome:
# This one works
df = pl.DataFrame(
{
"col_1": [
Decimal("-0.1"),
Decimal("0.1"),
]
}
)
# But this one doesn't
df = pl.DataFrame(
{
"col_1": [
Decimal("0.1"),
Decimal("-0.1"),
]
}
)
Describe the bug
I'm loading a polars DataFrame and one of the columns has negative and positive decimals. If there is a sign change, the driver throws a RuntimeError. I've tried to narrow it down, but the behavior is a little inconsistent.
At the moment, I work around this by casting all Decimal columns to Strings.
To reproduce
Include a complete code listing (or project/solution) that we can run to reproduce the issue.
Partial code listings, or multiple fragments of code, will slow down our response or cause us to push the issue back to you to provide code to reproduce the issue.
Expected behavior
Insert should be successful.
Further technical details
Python version: 3.14.4
SQL Server version: Azure SQL Managed Instance (locked to SQL Server 2022)
Operating system: Windows 11
Additional context
There's definitely something weird going on with how the driver is inserting decimals with
executemany. The examples below all work normally:Weirdly enough, the order also affects the outcome: