Skip to content

Commit 3a3602c

Browse files
committed
[GR-13481] Migrate to Truffle Libraries.
PullRequest: graalpython/446
2 parents e9902f6 + 210b797 commit 3a3602c

File tree

225 files changed

+13359
-11085
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

225 files changed

+13359
-11085
lines changed

CHANGELOG.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,18 @@
33
This changelog summarizes major changes between GraalVM versions of the Python
44
language runtime. The main focus is on user-observable behavior of the engine.
55

6+
## Version 1.0.0 RC15
7+
8+
* Migrate to Truffle libraries for interop
9+
* Support the buffer protocol for mmap
10+
* Support importing java classes using normal Python import syntax
11+
* Improve performance of literal dictionary creation when the first but not all keys are strings
12+
* Improve performance of getting the length of a string
13+
* Improve performance of accessing defaults, keyword-defaults, and code of a function
14+
* Fix getting file separator from the Truffle filesystem rather than the operating system
15+
* Fix constructing and calling methods with non-function callables
16+
* Fix execution of subprocesses with non-default python homes on JVM
17+
618
## Version 1.0.0 RC14
719

820
* Mark a subset of the Graal Python launcher options as "stable". All other options are subject to change and need to be unlocked explicitly on the commandline.

doc/INTEROP.md

Lines changed: 23 additions & 129 deletions
Original file line numberDiff line numberDiff line change
@@ -12,19 +12,17 @@ You can import a global value from the entire polyglot scope:
1212
```python
1313
imported_polyglot_global = polyglot.import_value("global_name")
1414
```
15-
This global should then work as expected, accessing items sends `READ` messages,
16-
calling methods sends `INVOKE` messages etc.
15+
16+
This global should then work as expected; accessing attributes assumes it reads
17+
from the `members` namespace; accessing items is supported both with strings and
18+
numbers; calling methods on the result tries to do a straight invoke and falls
19+
back to reading the member and trying to execute it.
1720

1821
You can evaluate some code in another language:
1922
```python
2023
polyglot.eval(string="1 + 1", language="ruby")
2124
```
2225

23-
This kind of `polyglot.eval` also works with the somewhat obscure mime-types:
24-
```python
25-
polyglot.eval(string="1 + 1", language="application/x-ruby")
26-
```
27-
2826
It also works with the path to a file:
2927
```python
3028
polyglot.eval(file="./my_ruby_file.rb", language="ruby")
@@ -38,7 +36,8 @@ polyglot.eval(file="./my_ruby_file.rb")
3836
To export something from Python to other Polyglot languages so they can import
3937
it:
4038
```python
41-
foo = object() polyglot.export_value(foo, "python_foo")
39+
foo = object()
40+
polyglot.export_value(foo, name="python_foo")
4241
```
4342

4443
The export function can be used as a decorator, in this case the function name
@@ -62,6 +61,21 @@ byteArray = myBigInt.toByteArray()
6261
print(list(byteArray)) # Java arrays can act like Python lists
6362
```
6463

64+
For packages under the `java` package, you can also use the normal Python import
65+
syntax:
66+
```python
67+
import java.util.ArrayList
68+
from java.util import ArrayList
69+
70+
# these are the same class
71+
java.util.ArrayList == ArrayList
72+
73+
al = ArrayList()
74+
al.add(1)
75+
al.add(12)
76+
print(al) # prints [1, 12]
77+
```
78+
6579
In addition to the `type` builtin method, the `java` module, exposes the following
6680
methods as well:
6781

@@ -81,124 +95,4 @@ print(java.is_symbol(my_list)) # prints False, my_list is not a Java host s
8195
print(java.is_object(ArrayList)) # prints True, symbols are also host objects
8296
print(java.is_function(my_list.add))# prints True, the add method of ArrayList
8397
print(java.instanceof(my_list, ArrayList)) # prints True
84-
```
85-
86-
87-
#### Python responses to Truffle interop messages
88-
89-
###### READ
90-
If the key is a String try using `__getattribute__` to read. If that fails or
91-
the key isn't a String, but there is both a `__len__` and a `__getitem__`
92-
method, call `__getitem__`.
93-
94-
Since the `KEYS` message returns an object's attributes, `READ` thus prefers the
95-
object attributes if the key is a String, and only falls back to `__getitem__`
96-
if the key is not a String or the object actually looks like a sequence.
97-
98-
Since disambiguation is hard here, to be explicit when using String keys, the
99-
key may be prefixed with `@` to force `__getattribute__` access or with `[` to
100-
force `__getitem__` access. If an item is accessed that starts with `@`, this
101-
will mean we first try to read it as an attribute without the `@`, only to fall
102-
back to reading it as an item. If the performance of this access is an issue,
103-
always prefixing may be advisable.
104-
105-
###### UNBOX
106-
* `str` => `java.lang.String`
107-
* `byte` => `java.lang.String`, assuming Java system encoding
108-
* `float` => `double`
109-
* `int` => `int` or `long`, if it fits, otherwise raises an interop exception
110-
111-
###### WRITE
112-
If the key is an attribute defined directly on the object (not inherited), use
113-
`__setattr__`. If the key is a String and there is a `keys`, `items`, `values`
114-
and a `__setitem__` method, we assume this is a Mapping and try to set the key
115-
using `__setitem__`. If the key is a String, but not all of `keys`, `items`, and
116-
`values` exists, we use `__setattr__`. Otherwise, we try to use `__setitem__` if
117-
that exists, or otherwise fall back to `__setattr__`.
118-
119-
Just as with `READ`, disambiguation is hard here, so to be explicit when using
120-
String keys, the key may be prefixed with `@` to force `__setattr__` access or
121-
with `[` to force `__setitem__` access.
122-
123-
###### REMOVE
124-
The remove message follows the same logic as the `WRITE` message, except with
125-
`__delattr__` and `__delitem__`. It returns true if the removal was successful.
126-
127-
###### EXECUTE
128-
Call the `__call__` method of the receiver with the provided arguments.
129-
130-
###### IS_EXECUTABLE
131-
Returns true if the receiver has inherited a `__call__` field.
132-
133-
###### IS_INSTANTIABLE
134-
Returns true only for python classes, i.e., type instances constructed through
135-
the `type` constructor.
136-
137-
###### INVOKE
138-
The equivalent of `receiver.methodname(arguments)`.
139-
140-
###### NEW
141-
Calls the constructor only if the receiver object is a Python class.
142-
143-
###### IS_NULL
144-
Returns true for None only.
145-
146-
###### HAS_SIZE
147-
According to the Truffle interop contract answering `true` to `HAS_SIZE` implies
148-
that indexed element access is available. However, we cannot fully guarantee
149-
this. We may answer `true` here when the object has both a `__len__` field and a
150-
`__getitem__` field. If the object's length is reported >0, we also try to read
151-
the item `0` and if that fails, we answer `false`. If the object reports it's
152-
empty, we cannot know if a read with an index will actually work, but we'll
153-
report `true`.
154-
155-
###### GET_SIZE
156-
Calls `__len__`. Just because `GET_SIZE` returns something positive does not
157-
mean that accessing the object using integers is possible. We have no way of
158-
knowing what the `__getitem__` method does with an integer argument. Use
159-
`HAS_SIZE` first for a more conservative check if indexed access is possible.
160-
161-
###### IS_BOXED
162-
Returns true for those values that can be unboxed using the `UNBOX` message.
163-
164-
###### KEY_INFO
165-
This will lookup the key using the Python MRO. It will check if it's readable
166-
and writable, and also check if it has side-effects based on wether it is an
167-
inherited descriptor (i.e., an object with `__get__`, `__set__`, and/or
168-
`__delete__`). If the owner of the key is mutable (the owner being the class the
169-
key is inherited from or the object itself) then `REMOVABLE` and `MODIFABLE` are
170-
true. If the object itself is mutable, `INSERTABLE` will also be true. Finally,
171-
if the attribute is a function or it is *not* a descriptor and has a `__call__`,
172-
we declare it `INOCABLE`. We don't do this for descriptors, because we would
173-
have to run the `__get__` method and this message should not have side-effects.
174-
175-
###### HAS_KEYS
176-
Always returns true.
177-
178-
###### KEYS
179-
This returns the all attributes of the receiver object that would usually be
180-
available through `__getattribute__`, i.e., both inherited and direct
181-
attributes.
182-
183-
If the object responds to `keys`, `values`, `items`, and `__getitem__`, we
184-
assume it is Mapping, and we present the String result of the `keys` method in
185-
combination with the attributes, prefixed with `[` if, and only if, the request
186-
asked for _internal_ keys also. The `KEYS` message requires the returned object
187-
to have only `java.lang.String` items, so inlo String keys are added to the
188-
result set. The `[` prefix ensures that in our handling of `READ` and `WRITE`
189-
messages we also treat them as mapping entries, not attributes.
190-
191-
It's still possible that none of the keys can be `READ`: the `READ` message uses
192-
Python semantics for lookup, which means that an inherited descriptor with a
193-
`__get__` method or the `__getitem__` method may still intercept actual access.
194-
195-
###### IS_POINTER
196-
Returns true if the object is a Python function defined in a Python C extension
197-
module and the function pointer is a native pointer.
198-
199-
###### AS_POINTER
200-
Returns the underlying C function pointer for the Python C extension module.
201-
202-
###### TO_NATIVE
203-
Returns the underlying TruffleObject for a Python C extension module function
204-
if that is a native pointer.
98+
```

graalpython/com.oracle.graal.python.test/src/com/oracle/graal/python/test/datatype/PRangeTests.java

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
/*
2-
* Copyright (c) 2017, 2018, Oracle and/or its affiliates.
2+
* Copyright (c) 2017, 2019, Oracle and/or its affiliates.
33
* Copyright (c) 2013, Regents of the University of California
44
*
55
* All rights reserved.
@@ -33,7 +33,7 @@
3333

3434
import com.oracle.graal.python.PythonLanguage;
3535
import com.oracle.graal.python.builtins.objects.range.PRange;
36-
import com.oracle.graal.python.nodes.control.GetIteratorNode;
36+
import com.oracle.graal.python.nodes.control.GetIteratorExpressionNode.GetIteratorNode;
3737
import com.oracle.graal.python.nodes.control.GetNextNode;
3838
import com.oracle.graal.python.nodes.object.IsBuiltinClassProfile;
3939
import com.oracle.graal.python.runtime.exception.PException;
@@ -67,7 +67,7 @@ public void doInsert(Node child) {
6767

6868
@Test
6969
public void loopWithOnlyStop() throws UnexpectedResultException {
70-
PRange range = PythonObjectFactory.create().createRange(10);
70+
PRange range = PythonObjectFactory.getUncached().createRange(10);
7171
int index = 0;
7272
TestRoot testRoot = new TestRoot(PythonLanguage.getCurrent());
7373
GetIteratorNode getIter = GetIteratorNode.create();
@@ -91,7 +91,7 @@ public void loopWithOnlyStop() throws UnexpectedResultException {
9191

9292
@Test
9393
public void loopWithStep() throws UnexpectedResultException {
94-
PRange range = PythonObjectFactory.create().createRange(0, 10, 2);
94+
PRange range = PythonObjectFactory.getUncached().createRange(0, 10, 2);
9595
int index = 0;
9696
TestRoot testRoot = new TestRoot(PythonLanguage.getCurrent());
9797
GetIteratorNode getIter = GetIteratorNode.create();
@@ -115,7 +115,7 @@ public void loopWithStep() throws UnexpectedResultException {
115115

116116
@Test
117117
public void getItem() {
118-
PRange range = PythonObjectFactory.create().createRange(10);
118+
PRange range = PythonObjectFactory.getUncached().createRange(10);
119119
assertEquals(3, range.getItemNormalized(3));
120120
}
121121

graalpython/com.oracle.graal.python.test/src/com/oracle/graal/python/test/grammar/TestParserTranslator.java

Lines changed: 27 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -107,6 +107,10 @@
107107
import com.oracle.graal.python.runtime.PythonContext;
108108
import com.oracle.graal.python.runtime.PythonParser.ParserMode;
109109
import com.oracle.graal.python.test.PythonTests;
110+
import com.oracle.truffle.api.RootCallTarget;
111+
import com.oracle.truffle.api.Truffle;
112+
import com.oracle.truffle.api.TruffleLanguage;
113+
import com.oracle.truffle.api.frame.VirtualFrame;
110114
import com.oracle.truffle.api.nodes.Node;
111115
import com.oracle.truffle.api.nodes.NodeUtil;
112116
import com.oracle.truffle.api.nodes.RootNode;
@@ -120,6 +124,28 @@ public TestParserTranslator() {
120124
context = PythonLanguage.getContextRef().get();
121125
}
122126

127+
private static class JUnitRootNode extends RootNode {
128+
129+
@Child private ExpressionNode body;
130+
131+
public JUnitRootNode(TruffleLanguage<?> language, ExpressionNode body) {
132+
super(language);
133+
this.body = body;
134+
}
135+
136+
@Override
137+
public Object execute(VirtualFrame frame) {
138+
return body.execute(frame);
139+
}
140+
141+
}
142+
143+
private Object runInRoot(ExpressionNode expr) {
144+
JUnitRootNode jUnitRootNode = new JUnitRootNode(context.getLanguage(), expr);
145+
RootCallTarget callTarget = Truffle.getRuntime().createCallTarget(jUnitRootNode);
146+
return callTarget.call();
147+
}
148+
123149
RootNode parse(String src) {
124150
Source source = Source.newBuilder(PythonLanguage.ID, src, "foo").build();
125151
return (RootNode) context.getCore().getParser().parse(ParserMode.File, context.getCore(), source, null);
@@ -175,8 +201,7 @@ <T> T parseAs(String src, Class<? extends T> klass) {
175201
}
176202

177203
Object literalAs(String src, Class<? extends PNode> klass) {
178-
ExpressionNode firstChild = (ExpressionNode) parseAs(src, klass);
179-
return firstChild.execute(null);
204+
return runInRoot((ExpressionNode) parseAs(src, klass));
180205
}
181206

182207
<T> T literalAs(String src, Class<? extends PNode> klass, Class<? extends T> rklass) {

0 commit comments

Comments
 (0)