Packaging Pyarmor obfuscated script

[mightbecharles]

Hi-

I am working on a project where I am hoping to obfuscate the project with PyArmor, then package it with PyInstaller.

I am able to obfuscate the project with PyArmor. I am also able to package the unobfuscated project with PyInstaller. But, I am not able to package the obfuscated code.

I can see from the terminal output that PyInstaller misses Processing standard module hook for my pyarmor runtime file. This file looks to pass in a lot of imports via bytecode:

from pyarmor_runtime_000000 import __pyarmor__
__pyarmor__(__name__, __file__, b'............

My suspicion is that the bytecode is too long for PyInstaller, as when I package a smaller test project with less dependencies, PyInstaller does not miss importing the hook.

Is there a limit to the length of bytecode that can be imported in a hook?
Is there something else I should be looking at?

Here is a link to the discussion on PyArmor that brought me here.

Thank you

[mightbecharles]

I believe I provided the incorrect file reference before.

The file that is not getting processed is hook-pyarmor_runtime_000000.py. The contents of which is all of the project dependencies in this format: hiddenimports=['foo', 'bar']

The test project with fewer dependencies (106) works fine.
The project with more dependencies (2959) does not have this hook- processed, even though the dir it is in gets scanned.

[rokm]

Is there a limit to the length of bytecode that can be imported in a hook?

I don't think so. The following example generates a package with 10k sub-modules and a hook that adds them to hiddenimports, and it seems to work for me:

import sys
import os
import pprint

NUM_MODULES=10_000

hook_dir = 'extra-hooks'

module_names = [
    f"mypackage.module_{i}" for i in range(NUM_MODULES)
]

# Create package and its modules
for module_name in module_names:
    module_file = module_name.replace('.', os.sep) + '.py'
    module_dir = os.path.dirname(module_file)
    os.makedirs(module_dir, exist_ok=True)

    with open(module_file, 'w') as fp:
        print("# Test module", file=fp)
        print(f"""print("Importing module {module_name!r}!")""", file=fp)


with open('mypackage/__init__.py', 'w') as fp:
    print("import importlib", file=fp)
    for module_name in module_names:
        print(f"importlib.import_module({module_name!r})", file=fp)

with open('program.py', 'w') as fp:
    print("import mypackage", file=fp)

# Create hook
os.makedirs(hook_dir, exist_ok=True)
hook_file = os.path.join(hook_dir, 'hook-mypackage.py')
with open(hook_file, 'w') as fp:
    print(f"hiddenimports={pprint.pformat(module_names)}", file=fp)

# Now run: pyinstaller --clean --noconfirm --additional-hooks-dir=extra-hooks program.py

So if the hook is not ran, there are couple of possibilities I can think of:

• the additional hook location is not passed (or passed correctly) to pyinstaller for some reason (via --additional-hooks-dir command-line option, or hookspath argument to Analysis constructor in the .spec file)
• the hook name (hook-something.py) does not match the module name (something), where something seems to be pyarmor_runtime_000000 in your case
• the location of pyarmor_runtime_000000 module is not in python's / pyinstaller's search path during analysis, and so the module is not found at all - hence the hook cannot be processed either. You can check this by opening build/{name}/xref-{name}.html (or warn-{name}.txt), and see if pyarmor_runtime_000000 is marked as missing.

[mightbecharles]

Thank you for this detailed information. I believe it is the third issue that you mentioned. I will give it a shot and follow up.