Skip to content

PyJulia broken with Julia 1.5.2 Python 3.8? #425

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
opus111 opened this issue Sep 30, 2020 · 8 comments
Open

PyJulia broken with Julia 1.5.2 Python 3.8? #425

opus111 opened this issue Sep 30, 2020 · 8 comments

Comments

@opus111
Copy link

opus111 commented Sep 30, 2020 •
edited
Loading

Hello, I installed Julia 1.5.2 and tried to call it from Python. It failed on the "Julia( compiled_modules=False )"

To reproduce in Python call

>>> import julia
>>> julia.install()
>>> from julia.api import Julia
>>> j = Julia( compiled_modules=False )

Full log below

(ti) DRMB-P1K5HTD6:inquiries-proto peter.wolf$ python
Python 3.8.5 (default, Sep  4 2020, 02:22:02) 
[Clang 10.0.0 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import julia
>>> julia.install()
[ Info: Julia version info
Julia Version 1.5.1
Commit 697e782ab8 (2020-08-25 20:08 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin19.5.0)
  uname: Darwin 18.7.0 Darwin Kernel Version 18.7.0: Mon Apr 27 20:09:39 PDT 2020; root:xnu-4903.278.35~1/RELEASE_X86_64 x86_64 i386
  CPU: Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz: 
              speed         user         nice          sys         idle          irq
       #1  2900 MHz      50533 s          0 s      38657 s     114178 s          0 s
       #2  2900 MHz       8104 s          0 s       5792 s     189467 s          0 s
       #3  2900 MHz      43072 s          0 s      25683 s     134608 s          0 s
       #4  2900 MHz       7571 s          0 s       4409 s     191383 s          0 s
       #5  2900 MHz      40053 s          0 s      21347 s     141963 s          0 s
       #6  2900 MHz       7810 s          0 s       4184 s     191369 s          0 s
       #7  2900 MHz      37447 s          0 s      17718 s     148199 s          0 s
       #8  2900 MHz       8445 s          0 s       4099 s     190819 s          0 s
       
  Memory: 16.0 GB (786.87109375 MB free)
  Uptime: 20335.0 sec
  Load Avg:  2.4521484375  2.80029296875  3.0302734375
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, skylake)
Environment:
  JULIA_HOME = /Applications/Julia-1.5.app/Contents/Resources/julia
  TERM = xterm-256color
  PATH = /anaconda3/envs/ti/bin:/anaconda3/condabin:/Users/peter.wolf/.cargo/bin:/Applications/Julia-1.5.app/Contents/Resources/julia/bin:/Library/Java/JavaVirtualMachines/jdk1.8.0_172.jdk/Contents/Home/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
  JAVA_HOME = /Library/Java/JavaVirtualMachines/jdk1.8.0_172.jdk/Contents/Home
  XPC_FLAGS = 0x0
  HOME = /Users/peter.wolf
  JULIA_HOME = /Applications/Julia-1.5.app/Contents/Resources/julia
[ Info: Julia executable: /Applications/Julia-1.5.app/Contents/Resources/julia/bin/julia
[ Info: Trying to import PyCall...
┌ Warning: PyCall is already installed.  However, you may have trouble using
│ this Python executable because it is statically linked to libpython.
│ 
│ For more information, see:
│     https://pyjulia.readthedocs.io/en/latest/troubleshooting.html
│ 
│ Python executable:
│     /anaconda3/envs/ti/bin/python
│ Julia executable:
│     /Applications/Julia-1.5.app/Contents/Resources/julia/bin/julia
└ @ Main /anaconda3/envs/ti/lib/python3.8/site-packages/julia/install.jl:90
>>> from julia.api import Julia
>>> j = Julia( compiled_modules=False )
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/anaconda3/envs/ti/lib/python3.8/site-packages/julia/core.py", line 503, in __init__
    self._call(IMPORT_PYCALL)
  File "/anaconda3/envs/ti/lib/python3.8/site-packages/julia/core.py", line 538, in _call
    self.check_exception(src)
  File "/anaconda3/envs/ti/lib/python3.8/site-packages/julia/core.py", line 587, in check_exception
    raise JuliaError(u'Exception \'{}\' occurred while calling julia code:\n{}'
julia.core.JuliaError: Exception 'LoadError' occurred while calling julia code:
const PyCall = Base.require(Base.PkgId(Base.UUID("438e738f-606a-5dbb-bf0a-cddfbfd45ab0"), "PyCall"))
@opus111
Copy link
Author

opus111 commented Sep 30, 2020

For reference, here is the same code successfully running with Python 3.8 and Julia 1.4.2

>>> import julia
>>> julia.install()
[ Info: Julia version info
Julia Version 1.4.2
Commit 44fa15b150* (2020-05-23 18:35 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin18.7.0)
  uname: Darwin 18.7.0 Darwin Kernel Version 18.7.0: Mon Apr 27 20:09:39 PDT 2020; root:xnu-4903.278.35~1/RELEASE_X86_64 x86_64 i386
  CPU: Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz: 
              speed         user         nice          sys         idle          irq
       #1  2900 MHz      53665 s          0 s      40822 s     119461 s          0 s
       #2  2900 MHz       9145 s          0 s       6361 s     198436 s          0 s
       #3  2900 MHz      45900 s          0 s      27262 s     140781 s          0 s
       #4  2900 MHz       8596 s          0 s       4887 s     200459 s          0 s
       #5  2900 MHz      42743 s          0 s      22734 s     148465 s          0 s
       #6  2900 MHz       8893 s          0 s       4658 s     200392 s          0 s
       #7  2900 MHz      40007 s          0 s      18925 s     155010 s          0 s
       #8  2900 MHz       9648 s          0 s       4583 s     199711 s          0 s
       
  Memory: 16.0 GB (56.6640625 MB free)
  Uptime: 21393.0 sec
  Load Avg:  4.06494140625  3.58154296875  3.1845703125
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-8.0.1 (ORCJIT, skylake)
Environment:
  JULIA_HOME = /Applications/Julia-1.4.app/Contents/Resources/julia
  TERM = xterm-256color
  PATH = /anaconda3/bin:/anaconda3/condabin:/Users/peter.wolf/.cargo/bin:/Applications/Julia-1.4.app/Contents/Resources/julia/bin:/Library/Java/JavaVirtualMachines/jdk1.8.0_172.jdk/Contents/Home/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
  JAVA_HOME = /Library/Java/JavaVirtualMachines/jdk1.8.0_172.jdk/Contents/Home
  XPC_FLAGS = 0x0
  HOME = /Users/peter.wolf
  JULIA_HOME = /Applications/Julia-1.4.app/Contents/Resources/julia
[ Info: Julia executable: /Applications/Julia-1.4.app/Contents/Resources/julia/bin/julia
[ Info: Trying to import PyCall...
┌ Warning: PyCall is already installed.  However, you may have trouble using
│ this Python executable because it is statically linked to libpython.
│ 
│ For more information, see:
│     https://pyjulia.readthedocs.io/en/latest/troubleshooting.html
│ 
│ Python executable:
│     /anaconda3/bin/python
│ Julia executable:
│     /Applications/Julia-1.4.app/Contents/Resources/julia/bin/julia
└ @ Main /anaconda3/lib/python3.7/site-packages/julia/install.jl:74
>>> from julia.api import Julia
>>> j = Julia(compiled_modules=False)
>>>

@tkf
Copy link
Member

tkf commented Oct 1, 2020

This is likely the same as #424 and if so it would be solved by JuliaPy/PyCall.jl#841

@tkf
Copy link
Member

tkf commented Oct 1, 2020

Ref JuliaRegistries/General#22235

@danvip10
Copy link

I'm getting the same error when not running julia.install() before.
However, when I run julia.install() and then Julia(compiled_modules=False), I get a segmentation fault:

>>> from julia.api import Julia
>>> jl = Julia(compiled_modules=False)

signal (11): Segmentation fault: 11
in expression starting at none:0
PyUnicode_FromFormatV at /opt/anaconda3/bin/python (unknown line)
PyErr_Format at /opt/anaconda3/bin/python (unknown line)
_PyObject_FastCallKeywords at /opt/anaconda3/bin/python (unknown line)
call_function at /opt/anaconda3/bin/python (unknown line)
_PyEval_EvalFrameDefault at /opt/anaconda3/bin/python (unknown line)
_PyEval_EvalCodeWithName at /opt/anaconda3/bin/python (unknown line)
_PyFunction_FastCallKeywords at /opt/anaconda3/bin/python (unknown line)
call_function at /opt/anaconda3/bin/python (unknown line)
_PyEval_EvalFrameDefault at /opt/anaconda3/bin/python (unknown line)
function_code_fastcall at /opt/anaconda3/bin/python (unknown line)
call_function at /opt/anaconda3/bin/python (unknown line)
_PyEval_EvalFrameDefault at /opt/anaconda3/bin/python (unknown line)
_PyEval_EvalCodeWithName at /opt/anaconda3/bin/python (unknown line)
_PyFunction_FastCallDict at /opt/anaconda3/bin/python (unknown line)
slot_tp_init at /opt/anaconda3/bin/python (unknown line)
type_call at /opt/anaconda3/bin/python (unknown line)
_PyObject_FastCallKeywords at /opt/anaconda3/bin/python (unknown line)
call_function at /opt/anaconda3/bin/python (unknown line)
_PyEval_EvalFrameDefault at /opt/anaconda3/bin/python (unknown line)
_PyEval_EvalCodeWithName at /opt/anaconda3/bin/python (unknown line)
PyRun_InteractiveOneObjectEx at /opt/anaconda3/bin/python (unknown line)
PyRun_InteractiveLoopFlags at /opt/anaconda3/bin/python (unknown line)
PyRun_AnyFileExFlags at /opt/anaconda3/bin/python (unknown line)
pymain_main at /opt/anaconda3/bin/python (unknown line)
main at /opt/anaconda3/bin/python (unknown line)
Allocations: 19269207 (Pool: 19264329; Big: 4878); GC: 21
zsh: segmentation fault  python

@danvip10
Copy link

Is there any fix for this?

@danvip10
Copy link

danvip10 commented Oct 30, 2020 •
edited
Loading

I uninstalled and reinstalled both julia and PyCall, and although don't a segmentation error as before, I do get the same error message as @opus111 in the original post. Moreover, note that my python version is 3.7.6.

Could I fix this by updating any package through github?

@vogt31337
Copy link

Maybe I can help a bit. But I think not...
Using Julia 1.5.3 and python 3.8.6 together with conda (I know it's not supported, but with some symlinks it seems to work).

>>> julia.install()
[ Info: Julia version info
Julia Version 1.5.3
Commit 788b2c77c1 (2020-11-09 13:37 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  uname: Linux 3.10.0-1160.6.1.el7.x86_64 #1 SMP Tue Nov 17 13:59:11 UTC 2020 x86_64 x86_64
  CPU: Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz:
                 speed         user         nice          sys         idle          irq
       #1-24  1200 MHz    4679934 s       1521 s    1741261 s  2493517096 s          0 s

  Memory: 31.1365966796875 GB (2360.359375 MB free)
  Uptime: 1.042066e6 sec
  Load Avg:  0.4296875  0.15673828125  0.091796875
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, haswell)
Environment:
  TERM = xterm
  ANT_HOME = /opt/rocks
  LD_LIBRARY_PATH = /opt/openmpi/lib
  PATH = /home/user/julia-1.5.3/bin:/home/user/miniconda3/envs/ppjl/bin:/home/user/miniconda3/condabin:/opt/openmpi/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/pdsh/bin:/opt/rocks/bin:/opt/rocks/sbin:/home/user/bin
  JAVA_HOME = /usr/java/latest
  MODULEPATH = /usr/share/Modules/modulefiles:/etc/modulefiles
  HOME = /home/user
  MPIHOME = /opt/openmpi
  PYTHONPATH = :/home/user/pandapower:
  MODULESHOME = /home/user/miniconda3/envs/pp/Modules/3.2.10
[ Info: Julia executable: /home/user/julia-1.5.3/bin/julia
[ Info: Trying to import PyCall...
┌ Info: PyCall is already installed and compatible with Python executable.
│
│ PyCall:
│     python: /home/user/miniconda3/envs/ppjl/bin/python
│     libpython: /home/user/miniconda3/envs/ppjl/lib/libpython3.8.so.1.0
│ Python:
│     python: /home/user/miniconda3/envs/ppjl/bin/python
└     libpython: /home/user/miniconda3/envs/ppjl/lib/libpython3.8.so.1.0
>>> from julia import Base

signal (11): Segmentation fault
in expression starting at none:0
_Py_DECREF at /usr/local/src/conda/python-3.8.6/Include/object.h:470 [inlined]
GetResult at /usr/local/src/conda/python-3.8.6/Modules/_ctypes/callproc.c:944 [inlined]
_ctypes_callproc at /usr/local/src/conda/python-3.8.6/Modules/_ctypes/callproc.c:1257
PyCFuncPtr_call at /usr/local/src/conda/python-3.8.6/Modules/_ctypes/_ctypes.c:4201
list_sort_impl at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Objects/listobject.c:2453
compiler_addop_j at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Python/compile.c:1434
pymalloc_alloc at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Objects/obmalloc.c:1578
validate_stmt at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Python/ast.c:396 [inlined]
validate_stmts at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Python/ast.c:523
pymalloc_alloc at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Objects/obmalloc.c:1578
validate_stmt at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Python/ast.c:396 [inlined]
validate_stmts at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Python/ast.c:523
sre_ucs4_charset at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Modules/sre_lib.h:169
_Py_ClearFileSystemEncoding at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Python/preconfig.c:35
list_sort_impl at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Objects/listobject.c:2424
_codecs_utf_7_decode at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Modules/clinic/_codecsmodule.c.h:406
mult at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Python/dtoa.c:653
convertsimple at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Python/getargs.c:1019 [inlined]
convertitem at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Python/getargs.c:601 [inlined]
vgetargs1_impl at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Python/getargs.c:391
validate_expr at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Python/ast.c:249
pymalloc_alloc at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Objects/obmalloc.c:1578
validate_stmt at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Python/ast.c:396 [inlined]
validate_stmts at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Python/ast.c:523
pymalloc_alloc at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Objects/obmalloc.c:1578
validate_expr at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Python/ast.c:328
pymalloc_alloc at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Objects/obmalloc.c:1578
validate_expr at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Python/ast.c:328
pymalloc_alloc at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Objects/obmalloc.c:1578
validate_expr at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Python/ast.c:328
pymalloc_alloc at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Objects/obmalloc.c:1578
k_mul at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Objects/longobject.c:3455
missing_arguments at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Python/ceval.c:3908
new_interpreter at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Python/pylifecycle.c:1528 [inlined]
Py_NewInterpreter at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Python/pylifecycle.c:1565
deque_new at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Modules/_collectionsmodule.c:162
parse_internal_render_format_spec at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Python/formatter_unicode.c:182
_Py_ListComp at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Python/Python-ast.c:2072
sre_ucs4_charset at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Modules/sre_lib.h:169
fixstate at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Parser/acceler.c:106 [inlined]
fixdfa at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Parser/acceler.c:58 [inlined]
PyGrammar_AddAccelerators at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Parser/acceler.c:29
validate_expr at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Python/ast.c:328
sre_ucs4_charset at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Modules/sre_lib.h:169
fixstate at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Parser/acceler.c:106 [inlined]
fixdfa at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Parser/acceler.c:58 [inlined]
PyGrammar_AddAccelerators at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Parser/acceler.c:29
k_mul at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Objects/longobject.c:3455
missing_arguments at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Python/ceval.c:3908
_Py_FinishPendingCalls at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Python/ceval.c:607
SubString_new_object_or_empty at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Objects/stringlib/unicode_format.h:73
sre_ucs4_charset at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Modules/sre_lib.h:169
new_arena at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Objects/obmalloc.c:1235 [inlined]
pymalloc_alloc at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Objects/obmalloc.c:1494
PyEval_EvalCode at python (unknown line)
unknown function (ip: 0x560427c96ca2)
unknown function (ip: 0x560427cb08f2)
import_all_from at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Python/ceval.c:5246 [inlined]
_PyEval_EvalFrameDefault at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Python/ceval.c:3017
import_all_from at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Python/ceval.c:5255 [inlined]
_PyEval_EvalFrameDefault at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Python/ceval.c:3017
_PyEval_EvalFrameDefault at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Python/ceval.c:1993
_Py_DECREF at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Include/object.h:478 [inlined]
_PyEval_EvalFrameDefault at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Python/ceval.c:2914
Py_BytesMain at python (unknown line)
__libc_start_main at /lib64/libc.so.6 (unknown line)
PyInit_errno at /home/conda/feedstock_root/build_artifacts/python-split_1606502903469/work/Modules/errnomodule.c:210
Allocations: 3290498 (Pool: 3289132; Big: 1366); GC: 4
Segmentation fault

I tried this "test" multiple times: same result.

Then I tried your "code":

(ppjl) [user@pc ~]$ python
Python 3.8.6 | packaged by conda-forge | (default, Nov 27 2020, 19:31:52)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import julia
>>> from julia.api import Julia
>>> j = Julia(compiled_modules=False)
>>> from julia import Base
>>> Base.sind(90)
1.0

Et voilá it works. I don't know why, just started with Julia. But maybe this helps getting some insight in this problem. Btw. maybe now it's "fixed", and it runs, if you ignore compiled_modules. Maybe someone with more insight can provide help / details.

@nclarkjudd
Copy link

I am unable to replicate this issue.

Python 3.9.1 in a virtualenv, Julia 1.5.3 acquired through Fedora package manager, installed PyCall from Julia's REPL and then in IPython ran:

import julia
julia.install()
from julia import Base
Base.sind(90)

These commands completed successfully.

Note I did not use conda here (at least, not in purpose --- I gather that PyCall uses conda under the hood somewhere, based on the output it generates).

@nclarkjudd nclarkjudd mentioned this issue Mar 2, 2021
Merged
@robertpyke robertpyke mentioned this issue Jun 11, 2021
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@tkf @opus111 @nclarkjudd @vogt31337 @danvip10