Skip to content

Commit 851a3a3

Browse files
committed
Revert "Implement support for functions as FROM with columns clause support"
This reverts commit 05a31f2. Atom has this little button called "push" and just pushes to master, I wasn't even *on* master. oops
1 parent 05a31f2 commit 851a3a3

File tree

17 files changed

+75
-1667
lines changed

17 files changed

+75
-1667
lines changed

doc/build/core/functions.rst

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -19,10 +19,6 @@ unknown to SQLAlchemy, built-in or user defined. The section here only
1919
describes those functions where SQLAlchemy already knows what argument and
2020
return types are in use.
2121

22-
.. seealso::
23-
24-
:ref:`tutorial_functions` - in the :ref:`unified_tutorial`
25-
2622
.. automodule:: sqlalchemy.sql.functions
2723
:members:
2824
:undoc-members:

doc/build/tutorial/data.rst

Lines changed: 0 additions & 283 deletions
Original file line numberDiff line numberDiff line change
@@ -797,9 +797,6 @@ we call upon the name ``count()`` name::
797797
>>> print(count_fn)
798798
{opensql}count(user_account.id)
799799

800-
SQL functions are described in more detail later in this tutorial at
801-
:ref:`tutorial_functions`.
802-
803800
When using aggregate functions in SQL, the GROUP BY clause is essential in that
804801
it allows rows to be partitioned into groups where aggregate functions will
805802
be applied to each group individually. When requesting non-aggregated columns
@@ -1281,286 +1278,6 @@ clause:
12811278
[('patrick',)]
12821279
{opensql}ROLLBACK{stop}
12831280

1284-
.. _tutorial_functions:
1285-
1286-
Working with SQL Functions
1287-
^^^^^^^^^^^^^^^^^^^^^^^^^^
1288-
1289-
First introduced earlier in this section at :ref:`tutorial_group_by_w_aggregates`,
1290-
the :data:`_sql.func` object serves as a factory for creating new
1291-
:class:`_functions.Function` objects, which when used in a construct
1292-
like :func:`_sql.select`, produce a SQL function display, typically
1293-
consisting of a name, some parenthesis (although not always), and possibly
1294-
some arguments. Examples of typical SQL functions include:
1295-
1296-
* the ``count()`` function, an aggregate function which counts how many
1297-
rows are returned::
1298-
1299-
.. sourcecode:: pycon+sql
1300-
1301-
>>> stmt = select(func.count()).select_from(user_table)
1302-
>>> with engine.connect() as conn:
1303-
... result = conn.execute(stmt)
1304-
... print(result.all())
1305-
{opensql}BEGIN (implicit)
1306-
SELECT count(*) AS count_1
1307-
FROM user_account
1308-
[...] ()
1309-
[(3,)]
1310-
ROLLBACK
1311-
1312-
..
1313-
1314-
* the ``lower()`` function, a string function that converts a string to
1315-
lower case::
1316-
1317-
.. sourcecode:: pycon+sql
1318-
1319-
>>> stmt = select(func.lower("A String With Much UPPERCASE"))
1320-
>>> with engine.connect() as conn:
1321-
... print(conn.scalar(stmt))
1322-
{opensql}BEGIN (implicit)
1323-
SELECT lower(?) AS lower_1
1324-
[...] ('A String With Much UPPERCASE',)
1325-
a string with much uppercase
1326-
ROLLBACK
1327-
1328-
..
1329-
1330-
* the ``now()`` function, which provides for the current date
1331-
and time; as this is a common function, SQLAlhemy knows how to render this
1332-
differently for each backend, in the case of SQLite using the
1333-
CURRENT_TIMESTAMP function::
1334-
1335-
.. sourcecode:: pycon+sql
1336-
1337-
>>> stmt = select(func.now())
1338-
>>> with engine.connect() as conn:
1339-
... result = conn.execute(stmt)
1340-
... print(result.all())
1341-
{opensql}BEGIN (implicit)
1342-
SELECT CURRENT_TIMESTAMP AS now_1
1343-
[...] ()
1344-
[(datetime.datetime(...),)]
1345-
ROLLBACK
1346-
1347-
..
1348-
1349-
As most database backends feature dozens if not hundreds of different SQL
1350-
functions, the :data:`_sql.func` tries to be as liberal as possible in what
1351-
it accepts. Any name that is passed is automatically considered to be a
1352-
SQL function that will render in a generic way::
1353-
1354-
>>> print(select(func.some_crazy_function(user_table.c.name, 17)))
1355-
SELECT some_crazy_function(user_account.name, :some_crazy_function_2) AS some_crazy_function_1
1356-
FROM user_account
1357-
1358-
At the same time, a relatively small set of extremely common SQL functions
1359-
such as :class:`_functions.now`, :class:`_functions.max`, :class:`_functions.concat`
1360-
include pre-packaged versions of themselves which provide for proper typing
1361-
information as well as backend-specific SQL generation in some cases::
1362-
1363-
>>> from sqlalchemy.dialects import postgresql, oracle
1364-
>>> print(select(func.now()).compile(dialect=postgresql.dialect()))
1365-
SELECT now() AS now_1
1366-
>>> print(select(func.now()).compile(dialect=oracle.dialect()))
1367-
SELECT CURRENT_TIMESTAMP AS now_1 FROM DUAL
1368-
1369-
Functions Have Return Types
1370-
~~~~~~~~~~~~~~~~~~~~~~~~~~~
1371-
1372-
As functions are column expressions, they also have datatypes. These return
1373-
types are significant when making use of the function expression in the
1374-
context of a larger expression; that is, math operators will work better
1375-
when the datatype of the expression is something like :class:`_types.Integer`
1376-
or :class:`_types.Numeric`, JSON accessors in order to work need to be using
1377-
a type such as :class:`_types.JSON`.
1378-
1379-
The return type of the function may also be significant when executing a
1380-
statement and getting rows back, for those cases where SQLAlchemy has to
1381-
apply result-set processing. A prime example of this are date-related
1382-
functions on SQLite.
1383-
1384-
For common aggregate functions like :class:`_functions.count`,
1385-
:class:`_functions.max`, :class:`_functions.min` as well as date functions like
1386-
:class:`_functions.now`, string functions like :class:`_functions.concat`, the
1387-
return type is set up appropriately, sometimes based on usage. The
1388-
:class:`_functions.max` function and similar ones will set up the return type
1389-
based on the argument given::
1390-
1391-
>>> m1 = func.max(Column("some_int", Integer))
1392-
>>> m1.type
1393-
Integer()
1394-
1395-
>>> m2 = func.max(Column("some_str", String))
1396-
>>> m2.type
1397-
String()
1398-
1399-
Date and time functions typically return :class:`_types.DateTime` or
1400-
:class:`_types.Date`::
1401-
1402-
>>> func.now().type
1403-
DateTime()
1404-
>>> func.current_date().type
1405-
Date()
1406-
1407-
A known string function such as :class:`_ will know to return :class:`_types.String`::
1408-
1409-
>>> func.concat("x", "y").type
1410-
String()
1411-
1412-
However, for the vast majority of SQL functions, SQLAlchemy does not have them
1413-
explicitly present in it's very small list of known functions. For example,
1414-
while there is typically no issue using SQL functions ``func.lower()``
1415-
and ``func.upper()`` to convert the casing of strings, SQLAlchemy doesn't
1416-
actually know about these functions, so they have a "null" return type::
1417-
1418-
>>> func.upper("lowercase").type
1419-
NullType()
1420-
1421-
For simple functions like ``upper`` and ``lower``, the issue is not significant,
1422-
but when using a SQL function where the return type is important (most typically
1423-
involving PostgreSQL JSON or ARRAY functions, SQLite date-related functions),
1424-
we can pass the type using ``type_=<type>``, such as::
1425-
1426-
>>> from sqlalchemy import JSON
1427-
>>> json_expr = func.json_extract_path('{"key1":{"key2":99}}','key1', type_=JSON)
1428-
>>> stmt = select(json_expr["key2"])
1429-
>>> print(stmt.compile(dialect=postgresql.dialect()))
1430-
SELECT json_extract_path(%(json_extract_path_1)s, %(json_extract_path_2)s) -> %(json_extract_path_3)s AS anon_1
1431-
1432-
Above, the ``json_extract_path()`` function object would not be able to deliver
1433-
the expression ``json_expr["key2"]`` without knowing that its return type
1434-
should be using :class:`_types.JSON`.
1435-
1436-
Using Window Functions
1437-
~~~~~~~~~~~~~~~~~~~~~~
1438-
1439-
A window function is a special use of a SQL aggregate function which calculates
1440-
the aggregate value over the rows being returned in a group as the individual
1441-
result rows are processed. Whereas a function like ``MAX()`` will give you
1442-
the highest value of a column within a set of rows, using the same function
1443-
as a "window function" will given you the highest value for each row,
1444-
*as of that row*.
1445-
1446-
In SQL, window functions allow one to specify the rows over which the
1447-
function should be applied, a "partition" value which considers the window
1448-
over different sub-sets of rows, and an "order by" expression which importantly
1449-
indicates the order in which rows should be applied to the aggregate function.
1450-
1451-
All SQL functions include a method :meth:`_functions.Function.over` which
1452-
grants the window function, or "OVER", syntax. A common function used with
1453-
window functions is the ``row_number()`` function which simply counts
1454-
rows. We may partition this row count against user name to number the
1455-
email addresses of individual users::
1456-
1457-
.. sourcecode:: pycon+sql
1458-
1459-
>>> stmt = select(
1460-
... func.row_number().over(partition_by=user_table.c.name),
1461-
... user_table.c.name,
1462-
... address_table.c.email_address).select_from(user_table).join(address_table)
1463-
>>> with engine.connect() as conn:
1464-
... result = conn.execute(stmt)
1465-
... print(result.all())
1466-
{opensql} BEGIN (implicit)
1467-
SELECT row_number() OVER (PARTITION BY user_account.name) AS anon_1, user_account.name, address.email_address
1468-
FROM user_account JOIN address ON user_account.id = address.user_id
1469-
[...] ()
1470-
[(1, 'sandy', '[email protected]'), (2, 'sandy', '[email protected]'), (1, 'spongebob', '[email protected]')]
1471-
ROLLBACK
1472-
1473-
Within :meth:`_functions.Function.over`, the :paramref:`_functions.Function.over.order_by`
1474-
and :paramref:`_functions.Function.over.partition_by` set up the ORDER BY and
1475-
PARTITION BY options.
1476-
1477-
Special Modifiers WITHIN GROUP, FILTER
1478-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1479-
1480-
The "WITHIN GROUP" SQL syntax is used in conjunction with an "ordered set"
1481-
or a "hypothetical set" aggregate
1482-
function. Common "ordered set" functions include ``percentile_cont()``
1483-
and ``rank()``. SQLAlchemy includes built in implementations
1484-
:class:`_functions.rank`, :class:`_functions.dense_rank`,
1485-
:class:`_functions.mode`, :class:`_functions.percentile_cont` and
1486-
:class:`_functions.percentile_disc` which include a :meth:`_functions.FunctionElement.within_group`
1487-
method::
1488-
1489-
>>> print(
1490-
... func.unnest(
1491-
... func.percentile_disc([0.25,0.5,0.75,1]).within_group(user_table.c.name)
1492-
... )
1493-
... )
1494-
unnest(percentile_disc(:percentile_disc_1) WITHIN GROUP (ORDER BY user_account.name))
1495-
1496-
"FILTER" is supported by some backends to limit the range of an aggregate function to a
1497-
particular subset of rows compared to the total range of rows returned::
1498-
1499-
>>> stmt = select(
1500-
... func.count(address_table.c.email_address).filter(user_table.c.name == 'sandy'),
1501-
... func.count(address_table.c.email_address).filter(user_table.c.name == 'spongebob')
1502-
... ).select_from(user_table).join(address_table)
1503-
>>> with engine.connect() as conn:
1504-
... result = conn.execute(stmt)
1505-
... print(result.all())
1506-
{opensql}BEGIN (implicit)
1507-
SELECT count(address.email_address) FILTER (WHERE user_account.name = ?) AS anon_1,
1508-
count(address.email_address) FILTER (WHERE user_account.name = ?) AS anon_2
1509-
FROM user_account JOIN address ON user_account.id = address.user_id
1510-
[...] ('sandy', 'spongebob')
1511-
[(2, 1)]
1512-
ROLLBACK
1513-
1514-
1515-
Table-Valued Functions
1516-
~~~~~~~~~~~~~~~~~~~~~~~~~
1517-
1518-
Table-valued SQL functions support a scalar representation that contains
1519-
named sub-elements. Often used for JSON functions as well as functions like
1520-
``generate_series()``, the table-valued function is specified in the FROM
1521-
clause, and is then referred towards as a table, or sometimes even as a
1522-
column. Functions of this form are prominent within the PostgreSQL database
1523-
and are also supported by SQLite.
1524-
1525-
To provide support for these, SQLAlchemy provides the
1526-
:meth:`_functions.FunctionElement.alias` method, which will convert a
1527-
:func:`_sql.func` object into a FROM clause. From there, it can then
1528-
be referenced directly in the columns or WHERE clause of a SELECT using
1529-
the ``.c`` collection normally. When using the ``.c`` collection, the
1530-
names of columns desired should be passed to the original function using
1531-
the :paramref:`_functions.Function.table_valued` parameter::
1532-
1533-
>>> stmt = select(
1534-
... func.json_each(
1535-
... '["one", "two", "three"]',
1536-
... ).table_valued("value").alias()
1537-
... )
1538-
>>> with engine.connect() as conn:
1539-
... result = conn.execute(stmt)
1540-
... print(result.all())
1541-
BEGIN (implicit)
1542-
SELECT anon_1.value
1543-
FROM json_each(?) AS anon_1
1544-
[...] ('["one", "two", "three"]',)
1545-
[('one',), ('two',), ('three',)]
1546-
ROLLBACK
1547-
1548-
A scalar-valued function such as PostgreSQL's ``json_array_elements`` may
1549-
also be referred towards as itself in the columns clause, using the special
1550-
accessor ``.column``::
1551-
1552-
>>> fn = func.json_array_elements(
1553-
... '[{"c1": 1, "c2": 2}, {"c1": 5, "c2": 10}]',
1554-
... type_=JSON).alias("result_elem")
1555-
>>> stmt = select(fn.column['c1'], fn.column['c2'])
1556-
>>> print(stmt)
1557-
SELECT result_elem[:result_elem_1] AS anon_1, result_elem[:result_elem_2] AS anon_2
1558-
FROM json_array_elements(:json_array_elements_1) AS result_elem
1559-
1560-
Above, when we produce the ``json_array_elements()`` function, we provide
1561-
the :class:`_types.JSON` datatype as its return type, so that we may use
1562-
Python ``__getitem__()`` access on it, e.g. ``fn.column['c1']``.
1563-
15641281

15651282
.. rst-class:: core-header, orm-addin
15661283

lib/sqlalchemy/dialects/oracle/base.py

Lines changed: 0 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -924,19 +924,6 @@ def function_argspec(self, fn, **kw):
924924
else:
925925
return ""
926926

927-
def visit_function(self, func, **kw):
928-
text = super(OracleCompiler, self).visit_function(func, **kw)
929-
if kw.get("asfrom", False):
930-
text = "TABLE (%s)" % func
931-
return text
932-
933-
def visit_table_valued_column(self, element, **kw):
934-
text = super(OracleCompiler, self).visit_table_valued_column(
935-
element, **kw
936-
)
937-
text = "COLUMN_VALUE " + text
938-
return text
939-
940927
def default_from(self):
941928
"""Called when a ``SELECT`` statement has no froms,
942929
and no ``FROM`` clause is to be appended.

lib/sqlalchemy/sql/coercions.py

Lines changed: 0 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -431,20 +431,6 @@ def _literal_coercion(
431431
except exc.ArgumentError as err:
432432
self._raise_for_expected(element, err=err)
433433

434-
def _raise_for_expected(self, element, argname=None, resolved=None, **kw):
435-
if isinstance(element, roles.AnonymizedFromClauseRole):
436-
advice = (
437-
"To create a "
438-
"column expression from a FROM clause row "
439-
"as a whole, use the .record() method."
440-
)
441-
else:
442-
advice = None
443-
444-
return super(ExpressionElementImpl, self)._raise_for_expected(
445-
element, argname=argname, resolved=resolved, advice=advice, **kw
446-
)
447-
448434

449435
class BinaryElementImpl(ExpressionElementImpl, RoleImpl):
450436

@@ -611,13 +597,6 @@ class ColumnArgumentOrKeyImpl(_ReturnsStringKey, RoleImpl):
611597
__slots__ = ()
612598

613599

614-
class StrAsPlainColumnImpl(_CoerceLiterals, RoleImpl):
615-
__slots__ = ()
616-
617-
def _text_coercion(self, element, argname=None):
618-
return elements.ColumnClause(element)
619-
620-
621600
class ByOfImpl(_CoerceLiterals, _ColumnCoercions, RoleImpl, roles.ByOfRole):
622601

623602
__slots__ = ()

0 commit comments

Comments
 (0)