Skip to content

Commit a27e663

Browse files
jgray-19Copilot
andauthored
Add transfer of dicts. (#21)
* Add dicts to pymadng. * Adjust pymadng to require explicit bool for copy of tables * Fix examples and documentation Co-authored-by: Copilot <[email protected]> * Update version and changelog * Update MAD-NG to version 1.1.3 * Fix typos * Actually test send dict * Fix table indexing. If a table is a list it will index from 0, if it has any keys, it indexes from 1 --------- Co-authored-by: Copilot <[email protected]>
1 parent 22cc5e7 commit a27e663

29 files changed

+415
-271
lines changed

.github/workflows/python-publish.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -37,8 +37,8 @@ jobs:
3737
- name: Get MAD Binaries
3838
run: |
3939
mkdir src/pymadng/bin
40-
curl https://madx.web.cern.ch/releases/madng/1.1/mad-linux-1.1.2 -o src/pymadng/bin/mad_Linux
41-
curl https://madx.web.cern.ch/releases/madng/1.1/mad-macos-1.1.2 -o src/pymadng/bin/mad_Darwin
40+
curl https://madx.web.cern.ch/releases/madng/1.1/mad-linux-1.1.3 -o src/pymadng/bin/mad_Linux
41+
curl https://madx.web.cern.ch/releases/madng/1.1/mad-macos-1.1.3 -o src/pymadng/bin/mad_Darwin
4242
chmod +x src/pymadng/bin/mad_Linux src/pymadng/bin/mad_Darwin
4343
- name: Build package
4444
run: python -m build

.github/workflows/test-pymadng.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -33,8 +33,8 @@ jobs:
3333
- name: Get MAD Binaries
3434
run: |
3535
mkdir ./src/pymadng/bin
36-
curl https://madx.web.cern.ch/releases/madng/1.1/mad-linux-1.1.2 -o ./src/pymadng/bin/mad_Linux
37-
curl https://madx.web.cern.ch/releases/madng/1.1/mad-macos-1.1.2 -o ./src/pymadng/bin/mad_Darwin
36+
curl https://madx.web.cern.ch/releases/madng/1.1/mad-linux-1.1.3 -o ./src/pymadng/bin/mad_Linux
37+
curl https://madx.web.cern.ch/releases/madng/1.1/mad-macos-1.1.3 -o ./src/pymadng/bin/mad_Darwin
3838
chmod +x ./src/pymadng/bin/mad_Linux ./src/pymadng/bin/mad_Darwin
3939
- name: Install dependencies
4040
run: |

CHANGELOG.md

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,14 @@
1-
0.6.3 (2024/30/05)
1+
0.7.0 (2025/06/05)
2+
Update to MAD-NG 1.1.3 \
3+
Breaking change: tables in lua are always returned as references, so you must use `eval` to get the value of the table. Or use the optional second argument in `py:send` such as `py:send(data, true)` to return the value of the table. \
4+
Dictionaries in python can now be sent to MAD-NG, and will be converted to a lua table. \
5+
Add an optional parameter to to_df and convert_to_dataframe methods to allow the user to specify to always return a pandas dataframe, instead of a tfs dataframe, when tfs is installed. \
6+
Update the documentation and examples to work again. \
7+
Remove iter restriction on MAD-NG objects that are not sequences, now all objects can be iterated over. \
8+
Renamed redirect_sterr to redirect_stderr in the MAD object, fixing a typo. \
9+
10+
11+
0.6.3 (2025/04/30)
212

313
Update to MAD-NG 1.1.2
414

docs/source/advanced_features.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -67,7 +67,7 @@ MAD-NG supports callbacks and iterative evaluations, which can be tied into Pyth
6767
In MAD:
6868
```lua
6969
function twiss_and_send()
70-
local tbl, flow = twiss {sequence=seq, method=4}
70+
local tbl, flow = twiss {sequence=seq}
7171
py:send({tbl.s, tbl.beta11})
7272
return tbl, flow
7373
end

docs/source/communication.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -89,8 +89,8 @@ If the object is not an `mtable`, a `TypeError` will be raised.
8989
```
9090

9191
See:
92-
```{literalinclude} ../../examples/ex-ps-twiss/ps-twiss.py
93-
:lines: 18, 24, 41-49
92+
```{literalinclude} ../../examples/ex-ps-twiss/ex-ps-twiss.py
93+
:lines: 19, 25, 42-53
9494
:linenos:
9595
```
9696

docs/source/contributing.md

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -64,8 +64,7 @@ python -m unittest tests/*.py
6464
## Best Practices
6565

6666
### Code Style
67-
- Follow PEP8 (enforced via linters)
68-
- Use descriptive names for MAD objects (e.g. `tw`, `flow`, `seq`)
67+
- Use descriptive names for everything
6968
- Keep high-level user APIs separate from internal helpers
7069
- Use Ruff for code and import formatting.
7170

docs/source/ex-lhc-couplingLocal.rst

Lines changed: 46 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ LHC Example
55
:local:
66
:depth: 2
77

8-
The file :ref:`ex-lhc-couplingLocal/lhc-couplingLocal.py <ex-lhc>` contains an example of loading the required files to use and run the LHC, while including a method to receive and plot intermediate results of a match.
8+
The file :ref:`ex-lhc-couplingLocal/ex-lhc-couplingLocal.py <ex-lhc>` contains an example of loading the required files to use and run the LHC, while including a method to receive and plot intermediate results of a match.
99

1010
Loading the LHC
1111
---------------
@@ -14,8 +14,8 @@ The following lines loads the required variables and files for the example. ``as
1414

1515
To grab variables from the MAD-X environment, we use ``mad.load("MADX", ...)``.
1616

17-
.. literalinclude:: ../../examples/ex-lhc-couplingLocal/lhc-couplingLocal.py
18-
:lines: 12-22
17+
.. literalinclude:: ../../examples/ex-lhc-couplingLocal/ex-lhc-couplingLocal.py
18+
:lines: 20-28
1919
:linenos:
2020

2121
Receving intermediate results
@@ -25,80 +25,80 @@ The most complicated part of the example includes the following set of lines.
2525

2626
From lines 4 - 8 below, we define a function that will be invoked during the optimization process at each iteration. Within this function, we perform a twiss for the match function to use, while also sending some information on the twiss to python, on line 6.
2727

28-
From lines 10 - 21, we run a match, with a **reference** to the match result returned to the variable ``match_rtrn``. Line 22 is a very important line, as this is something you place in the pipe to MAD-NG for MAD-NG to execute once the match is done. Lines 23-25 receive the first result returned during the match, so that we can start plotting the results.
28+
From lines 10 - 23, we run a match, with a **reference** to the match result returned to the variable ``match_rtrn``. Line 24 is a very important line, as this is something you place in the pipe to MAD-NG for MAD-NG to execute once the match is done. Lines 23-25 receive the first result returned during the match, so that we can start plotting the results.
2929

30-
The plotting occurs between lines 27 - 36, wtih the while loop continuing until twiss result is ``None``, which occurs when the match is done, as requested on line 22.
30+
The plotting occurs between lines 29 - 38, wtih the while loop continuing until twiss result is ``None``, which occurs when the match is done, as requested on line 24.
3131

32-
Finally, on lines 38 and 39, we retrieve the results of the match from the variable ``match_rtrn``. Since ``match_rtrn`` is a *temporary variable*, there is a limit to how many of these that can be stored (see :doc:`/advanced_features` for more information on these), we delete the reference in python to clear the temporary variable so that is is available for future use.
32+
Finally, on lines 40 and 41, we retrieve the results of the match from the variable ``match_rtrn``. Since ``match_rtrn`` is a *temporary variable*, there is a limit to how many of these that can be stored (see :doc:`/advanced_features` for more information on these), we delete the reference in python to clear the temporary variable so that is is available for future use.
3333

3434
.. important::
3535
As MAD-NG is running in the background, the variable ``match_rtrn`` contains *no* information and instead must be queried for the results. During the query, python will then have to wait for MAD-NG to finish the match, and then return the results. On the other hand, if we do not query for the results, the match will continue to run in the background, we can do other things in python, and then query for the results later.
3636

37-
.. literalinclude:: ../../examples/ex-lhc-couplingLocal/lhc-couplingLocal.py
38-
:lines: 39-77
37+
.. literalinclude:: ../../examples/ex-lhc-couplingLocal/ex-lhc-couplingLocal.py
38+
:lines: 50-90
3939
:linenos:
4040

41-
LHC Speed Tests
42-
---------------
41+
.. LHC Speed Tests
42+
.. ---------------
4343
44-
This file creates functions within MAD-NG, ``LHC_load``, which loads the LHC and ``reg_expr``, which looks through the MADX environment and places anything that is a deferred expression into the table ``expr``.
44+
.. This file creates functions within MAD-NG, ``LHC_load``, which loads the LHC and ``reg_expr``, which looks through the MADX environment and places anything that is a deferred expression into the table ``expr``.
4545
46-
The ``LHC_load`` function sends a string ``"done"`` afterwards, so that Python can stay in sync and time the LHC correctly.
46+
.. The ``LHC_load`` function sends a string ``"done"`` afterwards, so that Python can stay in sync and time the LHC correctly.
4747
48-
The code below just runs the function and times it.
48+
.. The code below just runs the function and times it.
4949
50-
.. literalinclude:: ../../examples/ex-recv-lhc/ex-defexpr.py
51-
:lines: 44-47
50+
.. .. literalinclude:: ../../examples/ex-recv-lhc/ex-defexpr.py
51+
.. :lines: 44-47
5252
5353
54-
The ``reg_expr`` fucntion is recursive so, after calling the function, to ensure Python stays in sync, Python is required to ask MAD-NG for the string ``"done"``.
54+
.. The ``reg_expr`` fucntion is recursive so, after calling the function, to ensure Python stays in sync, Python is required to ask MAD-NG for the string ``"done"``.
5555
56-
.. literalinclude:: ../../examples/ex-recv-lhc/ex-defexpr.py
57-
:lines: 50-53
56+
.. .. literalinclude:: ../../examples/ex-recv-lhc/ex-defexpr.py
57+
.. :lines: 50-53
5858
59-
Next, we have the first of two methods to evaluate every deferred expression in the LHC and receive them, where Python performs a loop through the number of deferred expressions and has to ask MAD-NG everytime to receive the result.
59+
.. Next, we have the first of two methods to evaluate every deferred expression in the LHC and receive them, where Python performs a loop through the number of deferred expressions and has to ask MAD-NG everytime to receive the result.
6060
61-
.. literalinclude:: ../../examples/ex-recv-lhc/ex-defexpr.py
62-
:lines: 60-64
61+
.. .. literalinclude:: ../../examples/ex-recv-lhc/ex-defexpr.py
62+
.. :lines: 60-64
6363
64-
The second method involves making MAD-NG do a loop at the same time as python and therefore does not require back on forth communication, which speeds up the transfer of data
64+
.. The second method involves making MAD-NG do a loop at the same time as python and therefore does not require back on forth communication, which speeds up the transfer of data
6565
66-
.. literalinclude:: ../../examples/ex-recv-lhc/ex-defexpr.py
67-
:lines: 67-71
66+
.. .. literalinclude:: ../../examples/ex-recv-lhc/ex-defexpr.py
67+
.. :lines: 67-71
6868
69-
The two methods above do not store the data, so the next bit of code is identical to above, but uses list comprehension to store the data into a list automatically, storing the lists into variables ``exprList1`` and ``exprList2``. The main point of seperating the methods above and below was to identify if storing the variables into a list was a bottleneck.
69+
.. The two methods above do not store the data, so the next bit of code is identical to above, but uses list comprehension to store the data into a list automatically, storing the lists into variables ``exprList1`` and ``exprList2``. The main point of seperating the methods above and below was to identify if storing the variables into a list was a bottleneck.
7070
71-
.. literalinclude:: ../../examples/ex-recv-lhc/ex-defexpr.py
72-
:lines: 74-77, 79-83
71+
.. .. literalinclude:: ../../examples/ex-recv-lhc/ex-defexpr.py
72+
.. :lines: 74-77, 79-83
7373
7474
75-
While we have the LHC loaded, the next example grabs the name of every element in the sequence ``lhcb1``, demonstrating the ability and speed of pymadng and MAD-NG, other attributes could also be grabbed, but for simplicity this code just gets the names. This bit of code also uses list comprehension while making MAD-NG loop at the same time as Python.
75+
.. While we have the LHC loaded, the next example grabs the name of every element in the sequence ``lhcb1``, demonstrating the ability and speed of pymadng and MAD-NG, other attributes could also be grabbed, but for simplicity this code just gets the names. This bit of code also uses list comprehension while making MAD-NG loop at the same time as Python.
7676
77-
.. literalinclude:: ../../examples/ex-recv-lhc/ex-defexpr.py
78-
:lines: 89-99
77+
.. .. literalinclude:: ../../examples/ex-recv-lhc/ex-defexpr.py
78+
.. :lines: 89-99
7979
80-
Another method, not shown above could be to create an entire list on the side of MAD-NG and then send the entire list to python. Which is done below, where instead all the names from the sequence ``lhcb2`` are taken from MAD-NG.
80+
.. Another method, not shown above could be to create an entire list on the side of MAD-NG and then send the entire list to python. Which is done below, where instead all the names from the sequence ``lhcb2`` are taken from MAD-NG.
8181
82-
.. literalinclude:: ../../examples/ex-recv-lhc/ex-defexpr.py
83-
:lines: 104-114
82+
.. .. literalinclude:: ../../examples/ex-recv-lhc/ex-defexpr.py
83+
.. :lines: 104-114
8484
85-
You can run the file yourself to retrieve the timings, but below is one run on an Intel® Core™ i7-8550U CPU @ 1.80GHz × 8 in Ubuntu 22.04.1 LTS
85+
.. You can run the file yourself to retrieve the timings, but below is one run on an Intel® Core™ i7-8550U CPU @ 1.80GHz × 8 in Ubuntu 22.04.1 LTS
8686
87-
.. code-block:: console
87+
.. .. code-block:: console
8888
89-
Load time: 0.3955872058868408 sec
90-
reg_expr time: 0.034337759017944336 sec
89+
.. Load time: 0.3955872058868408 sec
90+
.. reg_expr time: 0.034337759017944336 sec
9191
92-
For evaluating the deferred expressions
92+
.. For evaluating the deferred expressions
9393
94-
.. code-block:: console
94+
.. .. code-block:: console
9595
96-
eval time method 1: 0.5888900756835938 sec
97-
eval time method 2: 0.2224569320678711 sec
98-
eval time method 3: 0.6652431488037109 sec
99-
eval time method 4: 0.21885156631469727 sec
96+
.. eval time method 1: 0.5888900756835938 sec
97+
.. eval time method 2: 0.2224569320678711 sec
98+
.. eval time method 3: 0.6652431488037109 sec
99+
.. eval time method 4: 0.21885156631469727 sec
100100
101-
.. code-block:: console
101+
.. .. code-block:: console
102102
103-
time to retrieve every element name in lhcb1 sequence 0.024236202239990234 sec
104-
time to retrieve every element name in lhcb2 sequence 0.0245511531829834 sec
103+
.. time to retrieve every element name in lhcb1 sequence 0.024236202239990234 sec
104+
.. time to retrieve every element name in lhcb2 sequence 0.0245511531829834 sec

docs/source/examples.rst

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ PS Twiss Example
1919

2020
.. _ex-ps-twiss:
2121

22-
.. literalinclude:: ../../examples/ex-ps-twiss/ps-twiss.py
22+
.. literalinclude:: ../../examples/ex-ps-twiss/ex-ps-twiss.py
2323
:linenos:
2424

2525

@@ -28,7 +28,7 @@ LHC Example
2828

2929
.. _ex-lhc:
3030

31-
.. literalinclude:: ../../examples/ex-lhc-couplingLocal/lhc-couplingLocal.py
31+
.. literalinclude:: ../../examples/ex-lhc-couplingLocal/ex-lhc-couplingLocal.py
3232
:linenos:
3333

3434
Managing References Example

docs/source/quickstartguide.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,7 @@ mad.seq.beam = mad.beam()
5353
### Low-Level:
5454

5555
```python
56-
mad.send("seq.beam = beam")
56+
mad.send("seq.beam = beam {}")
5757
```
5858

5959
---
@@ -63,13 +63,13 @@ mad.send("seq.beam = beam")
6363
### High-Level:
6464

6565
```python
66-
mad["tbl", "flow"] = mad.twiss(sequence=mad.seq, method=4)
66+
mad["tbl", "flw"] = mad.twiss(sequence=mad.seq)
6767
```
6868

6969
### Low-Level:
7070

7171
```python
72-
mad.send("tbl, flow = twiss {sequence=seq, method=4}")
72+
mad.send("tbl, flw = twiss {sequence=seq}")
7373
mad.send("py:send(tbl)")
7474
tbl = mad.recv()
7575
```

examples/ex-LowLevel/ex-send-multypes.py

Lines changed: 36 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,9 @@
1-
from pymadng import MAD
2-
import numpy as np
31
import time
42

3+
import numpy as np
4+
5+
from pymadng import MAD
6+
57
arr0 = np.zeros((10000, 1000)) + 1j # 2*10000*1000*8 -> 160 MB
68

79
mad = MAD()
@@ -20,46 +22,59 @@
2022
{0} = cm1 {1} {2}
2123
py:send({0})"""
2224

23-
mad.send(cmatrixString.format("cm4", "*", 1)) ## Set cm4 to cm1 * 1 and send it to Python
24-
mad.send(cmatrixString.format("cm1", "*", 2)) ## Set cm1 to cm1 * 2 and send it to Python
25-
mad.send(cmatrixString.format("cm2", "*", 2)) ## Set cm2 to cm1 * 2 and send it to Python
26-
mad.send(cmatrixString.format("cm3", "/", 3)) ## Set cm3 to cm1 / 3 and send it to Python
25+
mad.send(
26+
cmatrixString.format("cm4", "*", 1)
27+
) ## Set cm4 to cm1 * 1 and send it to Python
28+
mad.send(
29+
cmatrixString.format("cm1", "*", 2)
30+
) ## Set cm1 to cm1 * 2 and send it to Python
31+
mad.send(
32+
cmatrixString.format("cm2", "*", 2)
33+
) ## Set cm2 to cm1 * 2 and send it to Python
34+
mad.send(
35+
cmatrixString.format("cm3", "/", 3)
36+
) ## Set cm3 to cm1 / 3 and send it to Python
2737

2838
## Create a vector in MAD and send it to Python
2939
mad.send("""
3040
local v1 = (MAD.vector(45):seq()*2 + 1)/3
3141
py:send(v1)
3242
""")
33-
start_time = time.time() # Start timer
43+
start_time = time.time() # Start timer
3444

3545
# Receive the matrices and vectors
36-
m1 = mad.recv()
46+
m1 = mad.recv()
3747
cm4 = mad.recv()
3848
cm1 = mad.recv()
3949
cm2 = mad.recv()
4050
cm3 = mad.recv()
4151
v1 = mad.recv()
4252

43-
print(time.time() - start_time) # Print time
53+
print(time.time() - start_time) # Print time
4454

4555
# Check if the matrices have been correctly sent
46-
print(np.all(cm1 == arr0*2))
47-
print(np.all(cm2 == arr0*2*2))
48-
print(np.all(cm3 == arr0*2/3))
56+
print(np.all(cm1 == arr0 * 2))
57+
print(np.all(cm2 == arr0 * 2 * 2))
58+
print(np.all(cm3 == arr0 * 2 / 3))
4959
print(np.all(cm4 == arr0))
5060

5161
# Send a list to MAD and receive a changed version back
52-
myList = [[1, 2, 3, 4, 5, 6, 7, 8, 9]] * 2
62+
my_list = [[1, 2, 3, 4, 5, 6, 7, 8, 9]] * 2
5363
mad.send("""
54-
local list = py:recv()
64+
list = py:recv() -- Note: no local, so it can be accessed outside this block
5565
list[1][1] = 10
5666
list[2][1] = 10
5767
py:send(list)
5868
""")
59-
mad.send(myList)
60-
myList[0][0] = 10
61-
myList[1][0] = 10
62-
print("receiving lists", mad.recv() == myList)
69+
mad.send(my_list)
70+
my_list[0][0] = 10
71+
my_list[1][0] = 10
72+
mad_list = mad.recv("list")
73+
for i, inner_list in enumerate(mad_list):
74+
for j, val in enumerate(inner_list):
75+
print(
76+
f"List value at [{i}][{j}]: {val} == {my_list[i][j]}", val == my_list[i][j]
77+
)
6378

6479
# Send an integer to MAD and receive a changed version back
6580
myInt = 4
@@ -94,14 +109,14 @@
94109
py:send(myNil)
95110
""")
96111
mad.send(None)
97-
print("Nil/None", mad.recv() == None)
112+
print("Nil/None", mad.recv() is None)
98113

99114
# Receive ranges from MAD
100115
mad.send("""
101116
py:send(3..11..2)
102117
py:send(MAD.nrange(3.5, 21.4, 12))
103118
py:send(MAD.nlogrange(1, 20, 20))
104119
""")
105-
print("irng", mad.recv() == range(3 , 12 , 2)) #Py not inclusive, mad is
120+
print("irng", mad.recv() == range(3, 12, 2)) # Py not inclusive, mad is
106121
print("rng", mad.recv() == np.linspace(3.5, 21.4, 12))
107-
print("lrng", np.allclose(mad.recv(), np.geomspace(1, 20, 20)))
122+
print("lrng", np.allclose(mad.recv(), np.geomspace(1, 20, 20)))

0 commit comments

Comments
 (0)