handle permutations in S_n with n > 2^16

[edgarcosta]

This resolves #39998, as the issue is gap having small and large permutations:
https://github.com/gap-system/gap/blob/f0b438db8fa5786d19975e6040076fd46b08bd85/src/permutat.h#L18-L26

📝 Checklist

• The title is concise and informative.
• The description explains in detail what this PR is about.
• I have linked a relevant issue or discussion.
• I have created tests covering the changes.
• I have updated the documentation and checked the documentation preview.

[github-actions]

Documentation preview for this PR (built with commit 4306a45; changes) is ready! 🎉
This preview will update shortly after each push to this PR.

[saraedum]

I guess the failing tests have nothing to do with these changes. If so, feel free to set this to positive review.

[tscrim]

LGTM except I have one question: did you look at the memory usage of your doctest? Does it require a lot of memory?

[edgarcosta]

In theory it just needs to allocate an array 2**17 of int32_t, but something else is going on behind the scenes.
Nonetheless, in practice the memory usage jumps by 112 MB

>>> (310099968 - 192577536)/1024.**2
112.078125

via:

/usr/bin/time -lh ./sage -c ""                                                                                                                                                          
           192577536  maximum resident set size
                   0  average shared memory size
                   0  average unshared data size
                   0  average unshared stack size
               36503  page reclaims
                 110  page faults
                   0  swaps
                   0  block input operations
                   0  block output operations
                   0  messages sent
                   0  messages received
                   0  signals received
                  10  voluntary context switches
                 158  involuntary context switches
           116133618  instructions retired
            47630290  cycles elapsed
             1458488  peak memory footprint

vs

/usr/bin/time -lh ./sage -c "SymmetricGroup(2**17)((2**16,2**17-1))._libgap_()" 
	1.00s real		0.52s user		0.25s sys
           310099968  maximum resident set size
                   0  average shared memory size
                   0  average unshared data size
                   0  average unshared stack size
               48026  page reclaims
                 114  page faults
                   0  swaps
                   0  block input operations
                   0  block output operations
                   0  messages sent
                   0  messages received
                   0  signals received
                  13  voluntary context switches
                7762  involuntary context switches
           116675395  instructions retired
            46895391  cycles elapsed
             1491280  peak memory footprint

[dimpase]

should this extreme high RAM test be marked as # long time ?

[edgarcosta]

I added # long time (100 mb)

and tried to make the example less memory intensive, but didn't succeed on that front:

sage: from resource import *
sage: print(getrusage(RUSAGE_SELF)[2]/1024.**2)
220.046875000000
sage: SymmetricGroup(2**16+1)((2**16,2**16+1))._libgap_()
(65536,65537)
sage: print(getrusage(RUSAGE_SELF)[2]/1024.**2)
319.703125000000

[dimpase]

can't you make sparse permutations in GAP? (probably not, as I vaguely recall). If not, then any test is bound to be RAM-expensive.

[edgarcosta]

There are small and large permutations, both stored in the one line representation, see
https://github.com/gap-system/gap/blob/master/src/permutat.h

[dimpase]

OK

[edgarcosta]

Thank you

[tscrim]

Yea, it doesn't seem like there is a better solution. I slightly lean towards making it a # not tested, but I guess it is not that big given even fairly low-memory modern computers... Perhaps we should make a brief mention on sage-devel about this?

[edgarcosta]

I don't think is such a big issue, I am sure there other tests that take similar amount of memory.
Note that sage at startup already uses 220 MB

[dimpase]

by the way, was it tested on a 32-bit system?
It might be necessary to tag this test appropriately

[edgarcosta]

I do not know, as I do not have access to such machines.

I don't think the behavior will change in 32 bits, as the short permutations use 16 bit arrays, and large ones use 32 bits arrays.

Also, gap doesn't support permutation groups larger than 2^32, but one will also encounter other problems before getting there

sage/src/sage/groups/perm_gps/permgroup_named.py

Line 194 in e0cf1e4

domain = list(range(1, domain + 1))

[dimpase]

if this test crashes on 32-bit CPU, it should be tagged appropriately.
so you get one result on 64-bit, and another on 32-bit (also Sage should not crash in the latter case).

(unless I missed an announcement that we dropped 32-bit)

[edgarcosta]

I don't see why it would crash a 32bit, we are switching between arrays of

typedef uint16_t UInt2;

to arrays of

typedef uint32_t UInt4;

https://github.com/gap-system/gap/blob/master/src/common.h#L94-L97

both types are supported in 32 bits.

I am just saying that we never need to go to larger than that

[dimpase]

above you point to a line of code and say that on 32-bit one won't get there

[edgarcosta]

I was just answering the hypothetical question. Can we handle S_n for n >= 2^32? I don't think so

[tscrim]

I don't think is such a big issue, I am sure there other tests that take similar amount of memory. Note that sage at startup already uses 220 MB

I would be curious if there are actually other such tests. This isn’t something we considered so explicitly before, but a policy about this might be good to have. This ended up being such a clear case.

[tscrim]

With regards to 32 bit, I think we will just have to wait to see what Volker says from his testing… Sorry in advance Volker if this breaks on 32 bit.