build error on debian

[YangChuanMakeSense]

Describe what happened:
I got the following error when I compile the code

I've installed python3.7.6 and pkg-config --cflags -- python3 works fine

Describe what you expected:
No error

Steps to reproduce the issue:

[christian-korneck]

hi @YangChuanMakeSense,

go-python3 works fine for me on Debian 10. In my setup the env var PKG_CONFIG_PATH points to a dir with a file exactly named python3.pc (I needed to create a symlink with that name).

Here's a Debian 10 Dockerfile example that works for me (even if you don't use Docker you should find the relevant parts in there):

FROM docker.io/library/debian:10
RUN apt-get -y update
RUN apt-get -y install golang python3.7 python3.7-dev pkg-config libpython3.7 libpython3.7-dev git
RUN ln -s /usr/lib/x86_64-linux-gnu/pkgconfig/python-3.7.pc /usr/lib/x86_64-linux-gnu/pkgconfig/python3.pc
ENV PKG_CONFIG_PATH=/usr/lib/x86_64-linux-gnu/pkgconfig
COPY hello /hello
RUN cd /hello && go get . && go build -o /usr/local/bin/hello .
CMD hello

(hello is a go program that prints a python string, all files are here).

quickstart:

$ git clone https://github.com/christian-korneck/Dockerfiles
$ cd Dockerfiles/debian-go-python3-demo/
$ docker build -t hello-demo .
$ docker run --rm hello-demo
hello from python in go

Hope this helps.

[YangChuanMakeSense]

Hi, @christian-korneck , thank you very much for your patient.
I use debian 9 dokcer base image, python3.4 already installed on it. I can not changed to debian 10 for some reasons. And I noticed that apt-get install python3.7 and python3.7-dev not works（ Couldn't find any package by regex 'python3.7-dev'）, I've tried some ways to make it works such as add ppa, all failed.
finally, I installed python3.7 by wget python3.7 source code and compile it. but go-python3 compile still failed(have set the PKG_CONFIG_PATH env). I think it is because python3.7-dev is not installed in my docker image, but I can not figure out how to install it even after google it for several days.

I am very appreciate for your help

[christian-korneck]

hi @YangChuanMakeSense ,

for building python did you use configure --enable-shared? (this gives a working <prefix>/lib/pkgconfig/python3.pc) I just tried on Debian 9 and this Dockerfile works for me:

I've split this into three build steps. The first one builds python from src, the second one builds the go program against our custom python and the third one is for running the compiled Go program (a more minimal container with no build tools installed and both our go program and our custom python in /opt). You could of course also all do it in one container or machine.

# temp container to build python from source
FROM docker.io/library/debian:9 as builder-python
RUN apt-get -y update && apt-get -y install curl build-essential zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libreadline-dev libffi-dev
RUN curl https://www.python.org/ftp/python/3.7.10/Python-3.7.10.tgz -o python.tgz && tar xf python.tgz
RUN cd Python-3.7.10 && ./configure --enable-shared --prefix=/opt/py
RUN cd Python-3.7.10 && make -j `nproc`
RUN cd Python-3.7.10 && make install

# temp container to build the go program
FROM docker.io/library/debian:9 as builder-golang
COPY --from=builder-python /opt/py /opt/py
RUN echo /opt/py/lib > /etc/ld.so.conf.d/py.conf && ldconfig
RUN apt-get -y update && apt-get -y install git curl pkg-config gcc
RUN curl -L https://golang.org/dl/go1.16.2.linux-amd64.tar.gz -o go.tar.gz && tar -C /usr/local -xzf go.tar.gz
ENV PKG_CONFIG_PATH=/opt/py/lib/pkgconfig
COPY hello /hello
RUN cd /hello && /usr/local/go/bin/go get . && /usr/local/go/bin/go build -o /opt/hello/bin/hello .

# container to distribute the go program
FROM docker.io/library/debian:9-slim
COPY --from=builder-golang /opt/py /opt/py
COPY --from=builder-golang /opt/hello /opt/hello
RUN echo /opt/py/lib > /etc/ld.so.conf.d/py.conf && ldconfig
CMD /opt/hello/bin/hello

To try:

$ git clone https://github.com/christian-korneck/Dockerfiles
$ cd Dockerfiles/debian-go-python3-demo/
$ docker build -t hello-debian9 -f ./Dockerfile.debian9 .
$ docker run --rm hello-debian9
hello from python in go

[YangChuanMakeSense]

@christian-korneck thank you very much, --enable-shared helps a lot.

[floki2020]

hi @christian-korneck ，i had the same problem ,and i used pyenv to install the python3.7 with options configure --enable-shared.but it still doesnt work. i can't use docker. thanks for your help~
os：ubuntu 20.04

[christian-korneck]

@floki2020 this works for me on Ubuntu 20.04:

apt-get -y update && apt-get -y install curl build-essential zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libreadline-dev libffi-dev git curl pkg-config gcc

# build/install Python 3.7 to /opt/py (note: this needs to be exactly Py 3.7)
curl https://www.python.org/ftp/python/3.7.10/Python-3.7.10.tgz -o python.tgz && tar xf python.tgz
cd Python-3.7.10
./configure --enable-shared --prefix=/opt/py
make -j `nproc`
make install
echo /opt/py/lib > /etc/ld.so.conf.d/py.conf && ldconfig

# build/install latest stable Go
curl -L https://golang.org/dl/go1.16.6.linux-amd64.tar.gz -o go.tar.gz && tar -C /usr/local -xzf go.tar.gz
export PKG_CONFIG_PATH=/opt/py/lib/pkgconfig

# fetch demo project
cd $HOME
git clone https://github.com/christian-korneck/Dockerfiles.git
cd Dockerfiles/debian-go-python3-demo/hello

# build demo project
/usr/local/go/bin/go get . && /usr/local/go/bin/go build -o /opt/hello/bin/hello .

# run demo project
/opt/hello/bin/hello

and you should see output

hello from python in go

(sorry if it's a bit clunky, copied from an ssh client on my phone. But hopefully it should get you an idea how to get it to work?)