Skip to content

Commit dbfcfab

Browse files
zultronzultron
authored
Merge pull request #320 from cerna/emcapplication-introduction
Add documentation for Machinekit EMCApplication project
2 parents 1356a82 + 3b1c2ae commit dbfcfab

File tree

4 files changed

+231
-58
lines changed

4 files changed

+231
-58
lines changed

‎docs/developing/common-developing.asciidoc

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
---
2+
---
3+
4+
:skip-front-matter:
5+
6+
These instructions assume that you have `sudo` rights and that you can execute `bash` scripts on your machine. You will need to install these programs to follow the instructions to build (and develop) Machinekit organization's project. These programs were chosen to make the workflow easier and cleaner for the everybody, however if you were to choose to, you don't neccessarily need them. But support given through Machinekit channels expect you to have these installed.
7+
8+
As a first step make sure you can execute *Docker CLI* commands by installing https://docker.com[Docker]. The current steps needed for your platform are published in the official https://docs.docker.com/get-docker/[Docker documentation]. For the vast majority of cases following the Debian Buster (current Debian stable version) https://docs.docker.com/get-docker/[here] will be the right one. Make sure you can run Docker commands without the need for `sudo` command!
9+
10+
Test the installation by running few basic commands:
11+
12+
[source,bash]
13+
----
14+
machinekit@machinekit:~$ docker --version
15+
Docker version 0.0.0-20190727010531-15bdbd76a5, build 15bdbd76a5
16+
machinekit@machinekit:~$ docker run hello-world
17+
Unable to find image 'hello-world:latest' locally
18+
latest: Pulling from library/hello-world
19+
1b930d010525: Pull complete
20+
Digest: sha256:fc6a51919cfeb2e6763f62b6d9e8815acbf7cd2e476ea353743570610737b752
21+
Status: Downloaded newer image for hello-world:latest
22+
23+
Hello from Docker!
24+
This message shows that your installation appears to be working correctly.
25+
26+
To generate this message, Docker took the following steps:
27+
1. The Docker client contacted the Docker daemon.
28+
2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
29+
(amd64)
30+
3. The Docker daemon created a new container from that image which runs the
31+
executable that produces the output you are currently reading.
32+
4. The Docker daemon streamed that output to the Docker client, which sent it
33+
to your terminal.
34+
35+
To try something more ambitious, you can run an Ubuntu container with:
36+
$ docker run -it ubuntu bash
37+
38+
Share images, automate workflows, and more with a free Docker ID:
39+
https://hub.docker.com/
40+
41+
For more examples and ideas, visit:
42+
https://docs.docker.com/get-started/
43+
44+
machinekit@machinekit:~$
45+
46+
----
47+
48+
Then you will need to have installed (and from terminal accessible) **git CLI**. The current steps needed for your platform are published in the official https://git-scm.com/book/en/v2/Getting-Started-Installing-Git[Git documentation]. For Debian system and standard official repository this means running:
49+
50+
[source,bash]
51+
----
52+
sudo apt install git
53+
----
54+
55+
After test that you can successfully use git by running:
56+
57+
[source,bash]
58+
----
59+
machinekit@machinekit:~$ git --version
60+
git version 2.27.0
61+
machinekit@machinekit:~$
62+
63+
----

‎docs/developing/developing.asciidoc

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,9 @@ starting at point 1 should lead you thru the next steps and return you to this p
1212

1313
. link:/docs/getting-started/installing-packages#configure-apt[Configure Apt]
1414
. link:/docs/getting-started/installing-packages#install-RT-kernel[Install the kernel of your choice]
15+
. link:/docs/developing/common-developing[Set up common development environment for all Machinekit projects]
1516
. link:/docs/developing/machinekit-developing[Install developing packages for Machinekit]
17+
. link:/docs/developing/emcapplication-developing[Set up building environment for EMCApplication]
1618

1719
== Other instructions for specific development
1820
. link:/docs/developing/CAN-developing[CAN developing for Machinekit]

‎docs/developing/emcapplication-developing.asciidoc

Lines changed: 165 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,165 @@
1+
---
2+
---
3+
4+
:skip-front-matter:
5+
6+
== [[packages-developing-rip]]Setting up for development
7+
=== [[packages-developing-docker]]Docker .deb build
8+
9+
. <<install-development-packages-docker,Install packages required for building Debian packages with Docker>>
10+
. <<get-source-and-build-docker,Get and build the Debian packages>>
11+
12+
=== [[packages-developing-rip]]Run-In-Place build
13+
14+
. <<install-development-packages,Install packages required for building from source>>
15+
. <<get-source-and-build,Get and build the source>>
16+
17+
=== [[aditional-notes]]Additional notes
18+
19+
. <<machinekit-hal-pinning,Why do I have to pass Machinekit-version when building debian packages>>
20+
21+
== [[install-development-packages-docker]]Install packages required for building Debian packages with Docker
22+
23+
These instructions assume that you have `sudo` rights and that you can execute `bash` scripts on your machine. You can build the *EMCApplication* debian package this way which can be then installed on target machine by the `apt` utility. You will also need to have installed **Git** executable and for container specific instructions also **Docker CLI** and **Docker daemon**. Directions for both can be found in link:/docs/developing/common-developing[Set up common development environment for all Machinekit projects].
24+
25+
EMCApplication is tightly tracking https:github.com/linuxCNC/linuxcnc[LinuxCNC@master] branch with minimal set of additional patches to use the Machinekit-HAL project. You can try to follow https://linuxcnc.org/docs/devel/html/[LinuxCNC'd documentation], but be aware that any part pertaining RuntimeAPI (RTAPI), Hardware Abstraction Layer (HAL) and most HAL modules and Python code will not be accurate as these components come from Machinekit-HAL package.
26+
27+
The supported distributions for which you can build _apt_ packages is the intersection of supported distribution of Machinekit-HAL and supported distributions for LinuxCNC.
28+
29+
Currently, Machinekit-HAL code stack is fully Python 3 ported, LinuxCNC code stack is made so both Python 2 and Python 3 are supported (with some parts [mainly GUI] requiring Python 2), however the Debian packaging recipes still support only Python 2 (this problem is being intensively worked on in https://github.com/LinuxCNC/linuxcnc/pull/943[PR#943]). This means that for building of successful packages the input version of _machinekit-hal_ and _machinekit-hal-dev_ packages have to be https://cloudsmith.io/~machinekit/repos/machinekit-hal/packages/?q=version%3A0.4.20868*[**0.4.20868**] maximum.
30+
31+
Following steps presume that you have cloned the EMCApplication repository to local folder and that you know the _UID_ and _GID_ of your user.
32+
33+
[source,bash]
34+
----
35+
git clone https://github.com/machinekit/emcapplication.git
36+
id
37+
----
38+
39+
Starting from current version of _Debian Buster_ image downloaded from DockerHUB (replace with version for which you want to build the packages), the following commands would be used for building:
40+
41+
[source,bash]
42+
----
43+
docker run -it --rm -v "$(pwd):/machinekit/emca" debian:buster bash -i
44+
apt update
45+
apt upgrade
46+
apt install build-essential fakeroot devscripts equivs sudo curl python lsb-release apt-cudf
47+
addgroup machinekit --gid 1000
48+
adduser machinekit --uid 1000 --gid 1000
49+
echo "machinekit ALL = (ALL) NOPASSWD: ALL" >> /etc/sudoers
50+
chown -R machinekit:machinekit /machinekit
51+
su machinekit
52+
curl -1sLf \
53+
'https://dl.cloudsmith.io/public/machinekit/machinekit/cfg/setup/bash.deb.sh' \
54+
| sudo -E bash
55+
curl -1sLf \
56+
'https://dl.cloudsmith.io/public/machinekit/machinekit-hal/cfg/setup/bash.deb.sh' \
57+
| sudo -E bash
58+
cd /machinekit/emca
59+
./debian/configure machinekit-hal=0.4.20894-1.gitebe1344a0~$(lsb_release -cs) no-docs
60+
mk-build-deps -irs sudo -t 'apt-cudf-get --solver aspcud -o Debug::pkgProblemResolver=0 -o APT::Install-Recommends=0'
61+
dpkg-buildpackage -us -uc
62+
cp ../*.deb .
63+
exit
64+
exit
65+
----
66+
67+
At the end you will have several new **.deb** files in your folder.
68+
69+
You should replace `--gid 1000`s and `--uid 1000` with numbers which you got from `id` command in previous step.
70+
71+
As EMCApplication packages are pinning the version of Machinekit-HAL packages dependency to one specific version, we need to pass this version string to _debian/configure_ script as `debian/configure machinekit-hal=${VERSION}`. One can use either version available from one of accessible Debian repositories, or already installed version.
72+
73+
To determine which Machinekit-HAL packages are currently installed on the system, run:
74+
75+
[source,bash]
76+
----
77+
apt list --installed | grep machinekit
78+
----
79+
80+
As standard Debian _mk-build-deps_ script with _apt_ manager cannot resolve build dependencies fixed to one particular version, we need to install _apt-cudf_ package to be able to choose different dependency solver.
81+
82+
LinuxCNC does not support **cross-building** of Debian packages, which is the reason why you will need to build the _EMCApplication_ natively. Because of time needed to build the whole application on _arm_ devices, the best course of action is to use _QEMU-user-static_ binaries for running the foreign architecture code on _amd64_ machine. There are multiple ways how to reach functional system: First, use https://github.com/multiarch/qemu-user-static[Multiarch/QEMU-user-static] Docker image (please heed advice in https://github.com/multiarch/qemu-user-static/issues/17#issuecomment-518660227[this] issue if you get `sudo` related problem), or Second, install the https://packages.debian.org/sid/qemu-user-static[qemu-user-static] Debian package (preferably the latest, _Sid_ version).
83+
84+
Then run the Docker container with specific architecture (here _arm32_):
85+
86+
[source,bash]
87+
----
88+
docker run -it --rm -v "$(pwd):/machinekit/emca" arm32v7/debian:buster bash -i
89+
dpkg --print-architecture && uname -m
90+
apt update
91+
apt upgrade
92+
apt install build-essential fakeroot devscripts equivs sudo curl python lsb-release apt-cudf
93+
addgroup machinekit --gid 1000
94+
adduser machinekit --uid 1000 --gid 1000
95+
echo "machinekit ALL = (ALL) NOPASSWD: ALL" >> /etc/sudoers
96+
chown -R machinekit:machinekit /machinekit
97+
su machinekit
98+
sudo apt install debian-keyring debian-archive-keyring apt-transport-https
99+
curl -k 'https://dl.cloudsmith.io/public/machinekit/machinekit/cfg/gpg/gpg.A9B6D8B4BD8321F3.key' | sudo apt-key add -
100+
curl -k https://dl.cloudsmith.io/public/machinekit/machinekit/cfg/setup/config.deb.txt?distro=$(lsb_release -is | tr "[:upper:]" "[:lower:]")\&codename=$(lsb_release -cs) | sudo tee -a /etc/apt/sources.list.d/machinekit-machinekit.list
101+
curl -k 'https://dl.cloudsmith.io/public/machinekit/machinekit-hal/cfg/gpg/gpg.D35981AB4276AC36.key' | sudo apt-key add -
102+
curl -k https://dl.cloudsmith.io/public/machinekit/machinekit-hal/cfg/setup/config.deb.txt?distro=$(lsb_release -is | tr "[:upper:]" "[:lower:]")\&codename=$(lsb_release -cs) | sudo tee -a /etc/apt/sources.list.d/machinekit-machinekit-hal.list
103+
sudo apt update
104+
cd /machinekit/emca
105+
./debian/configure machinekit-hal=0.4.20894-1.gitebe1344a0~$(lsb_release -cs) no-docs
106+
mk-build-deps -irs sudo -t 'apt-cudf-get --solver aspcud -o Debug::pkgProblemResolver=0 -o APT::Install-Recommends=0'
107+
dpkg-buildpackage -us -uc
108+
cp ../*.deb .
109+
exit
110+
exit
111+
----
112+
113+
Running Docker containers has a problem with _SLL/certificates/PGP_ (generally speaking there is some problem with _encryption_). This is why the previous example is using the manual installation of Debian registry instead of the automatic script like native build. (https://cloudsmith.io/~machinekit/repos/machinekit/setup/#formats-deb[Here] is the original procedure from Cloudsmith.) _Information why is this happening and how to solve this issue are greatly appreciated._
114+
115+
All other notes mentioned with native build previously are also valid here.
116+
117+
== [[install-development-packages]]Install packages required for building Run-In-Place
118+
119+
Building EMCApplication to run as separate and isolated software bundle (known as a RIP build from LinuxCNC world) is not possible. In all cases the Machinekit-HAL will need to be installed onto the system in the form of Debian packages.
120+
timeAPI to find them. (This condition could be circumvented with changing _loader_ rules.)
121+
122+
All that would be compiled and ruI to find them. (This condition could be circumvented with changing _loader_ rules.)
123+
124+
All that would be compiled and run from the given directory, will be the Enhanced Machine Controller and Graphical User Interfaces.
125+
126+
From root of the EMCApplication source directory run:
127+
128+
[source,bash]
129+
----
130+
apt install build-essential fakeroot devscripts equivs python lsb-release apt-cudf
131+
curl -1sLf \
132+
'https://dl.cloudsmith.io/public/machinekit/machinekit/cfg/setup/bash.deb.sh' \
133+
| sudo -E bash
134+
curl -1sLf \
135+
'https://dl.cloudsmith.io/public/machinekit/machinekit-hal/cfg/setup/bash.deb.sh' \
136+
| sudo -E bash
137+
debian/configure machinekit-hal=0.4.20894-1.gitebe1344a0~$(lsb_release -cs) no-docs
138+
mk-build-deps -irs sudo -t 'apt-cudf-get --solver aspcud -o Debug::pkgProblemResolver=0 -o APT::Install-Recommends=0'
139+
cd src
140+
./autogen.sh
141+
./configure --with-hal=machinekit-hal
142+
make -j$(nproc)
143+
sudo make install
144+
source ../scripts/rip-environment
145+
linuxcnc
146+
----
147+
148+
Now you should have a functioning Machinekit-HAL + EMCApplication installation. You will need to rerun `source ../scripts/rip-environment` code in each new terminal to populate the environment with correct values.
149+
150+
It is also viable to disallow _apt_ to upgrade the _machinekit-hal_ package to higher version:
151+
152+
[source,bash]
153+
----
154+
sudo apt-mark hold machinekit-hal
155+
----
156+
157+
== [[machinekit-hal-pinning]]Why do I have to pass Machinekit-version when building Debian packages?
158+
159+
It is important to understand what Machinekit EMCApplication really is. From programming point of view it is just set of patches applied over _LinuxCNC@master_ branch which it periodically tracks. (Meaning changes happening in the _upstream_ LinuxCNC@master are from time to time _merged into_ the _EMCApplication@machinekit/master_ branch.) And all what these additional patches are doing is to allow parts of LinuxCNC CNC stack (historically called **E**nhanced **M**otion **C**ontroller) to use external HAL dependency.
160+
161+
Machinekit-HAL project and LinuxCNC project both have certain development vector independent of each other, i.e. Machinekit-HAL does not care about LinuxCNC and LinuxCNC does not care about Machinekit-HAL. This represents quite interesting problem when creating **dpkg** packages, because **EMCApplication** package has to depend on **Machinekit-HAL** package, but also specific version of _EMCApplication_ has to depend on specific version of _Machinekit-HAL_ (or specific range) as later or previous versions of Machinekit-HAL can include incompatible code. And the easiest way how to guarantee the compatibility is to require the same version of Machinekit-HAL package as install dependency of EMCApplication as was used in build stage.
162+
163+
This way the _apt_ will make sure that the correct version of Machinekit-HAL is installed for the EMCApplication issue-less operation.
164+
165+
Using the _**mk-build-deps**_ script from the _devscripts_ package for creating environment capable of problem-less building of packages by invocating `dpkg-buildpackage` or compiling and linking the software during **R**un-**I**n-**P**lace build needs the `debian/control` file with correctly stated `Build-Depends: machinekit-ha (=` dependency. This is why we need to pass the _machinekit-hal_ package version early in stage when calling the `debian/configure machinekit-hal=${VERSION}` script.

‎docs/developing/machinekit-developing.asciidoc

Lines changed: 1 addition & 58 deletions
Original file line numberDiff line numberDiff line change
@@ -16,64 +16,7 @@
1616

1717
== [[install-development-packages-docker]]Install packages required for building Debian packages with Docker
1818

19-
These instructions assume that you have `sudo` rights and that you can execute `bash` scripts on your machine. You can build the *Machinekit-HAL* debian package this way which can be then installed on target machine by the `apt` utility.
20-
21-
As a first step make sure you can execute *Docker CLI* commands by installing https://docker.com[Docker]. The current steps needed for your platform are published in the official https://docs.docker.com/get-docker/[Docker documentation]. For the vast majority of cases following the Debian Buster (current Debian stable version) https://docs.docker.com/get-docker/[here] will be the right one. Make sure you can run Docker commands without the need for `sudo` command!
22-
23-
Test the installation by running few basic commands:
24-
25-
[source,bash]
26-
----
27-
machinekit@machinekit:~$ docker --version
28-
Docker version 0.0.0-20190727010531-15bdbd76a5, build 15bdbd76a5
29-
machinekit@machinekit:~$ docker run hello-world
30-
Unable to find image 'hello-world:latest' locally
31-
latest: Pulling from library/hello-world
32-
1b930d010525: Pull complete
33-
Digest: sha256:fc6a51919cfeb2e6763f62b6d9e8815acbf7cd2e476ea353743570610737b752
34-
Status: Downloaded newer image for hello-world:latest
35-
36-
Hello from Docker!
37-
This message shows that your installation appears to be working correctly.
38-
39-
To generate this message, Docker took the following steps:
40-
1. The Docker client contacted the Docker daemon.
41-
2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
42-
(amd64)
43-
3. The Docker daemon created a new container from that image which runs the
44-
executable that produces the output you are currently reading.
45-
4. The Docker daemon streamed that output to the Docker client, which sent it
46-
to your terminal.
47-
48-
To try something more ambitious, you can run an Ubuntu container with:
49-
$ docker run -it ubuntu bash
50-
51-
Share images, automate workflows, and more with a free Docker ID:
52-
https://hub.docker.com/
53-
54-
For more examples and ideas, visit:
55-
https://docs.docker.com/get-started/
56-
57-
machinekit@machinekit:~$
58-
59-
----
60-
61-
Then you will need to have installed (and from terminal accessible) `git` CLI. The current steps needed for your platform are published in the official https://git-scm.com/book/en/v2/Getting-Started-Installing-Git[Git documentation]. For Debian system and standard official repository this means running:
62-
63-
[source,bash]
64-
----
65-
sudo apt install git
66-
----
67-
68-
After test that you can successfully use git by running:
69-
70-
[source,bash]
71-
----
72-
machinekit@machinekit:~$ git --version
73-
git version 2.27.0
74-
machinekit@machinekit:~$
75-
76-
----
19+
These instructions assume that you have `sudo` rights and that you can execute `bash` scripts on your machine. You can build the *Machinekit-HAL* debian package this way which can be then installed on target machine by the `apt` utility. You will also need to have installed **Git** executable and for container specific instructions also **Docker CLI** and **Docker daemon**. Directions for both can be found in link:/docs/developing/common-developing[Set up common development environment for all Machinekit projects].
7720

7821
Another dependency is the `Python SH` module which is extensively used in the Python 3 scripts functioning as wrappers around the calls to programs actually doing the build. The official https://amoffat.github.io/sh/[documentation] has a tutorial how to install it by invoking Pip, but for standard Debian system you can just get it by running standard APT installation:
7922

0 commit comments

Comments
 (0)