Pyinstaller with crawl4ai module not working? #9036

Rapid1898-code · 2025-02-23T10:17:48Z

Rapid1898-code
Feb 23, 2025

I have created the following program using the crawl4ai module and now try to compile this to an executable file for windows -
Program code which is running fine generally:

import asyncio
from crawl4ai import AsyncWebCrawler, CrawlerRunConfig, CacheMode, MemoryAdaptiveDispatcher, CrawlerMonitor, DisplayMode
from crawl4ai.async_configs import BrowserConfig, CrawlerRunConfig
import json
import pprint as pp

urls = ["https://www.1aes.de","https://www.1afruechte.de","https://www.1luvfoods.com","https://www.1o8.co","https://www.1qbit.com"]

async def crawl_batch():
    browser_config = BrowserConfig(headless=True, verbose=False)
    run_config = CrawlerRunConfig(
        cache_mode=CacheMode.BYPASS,
        stream=False  # Default: get all results at once
    )

    dispatcher = MemoryAdaptiveDispatcher(
        memory_threshold_percent=70.0,
        check_interval=1.0,
        max_session_permit=10,
        monitor=CrawlerMonitor(
            display_mode=DisplayMode.DETAILED
        )
    )

    async with AsyncWebCrawler(config=browser_config) as crawler:
        # Get all results at once
        results = await crawler.arun_many(
            urls=urls,
            config=run_config,
            dispatcher=dispatcher
        )

        resultList = []
        # Process all results after completion
        for result in results:
            if result.success:
                resultList.append(result.markdown)
                # print(result.markdown)
                # await process_result(result)
            else:  
                print(f"Failed to crawl {result.url}: {result.error_message}")

        for e in resultList:
          print(e)
          input("Press!")
        print(len(resultList))
        
if __name__ == "__main__":
    asyncio.run(crawl_batch())

I used the following command for the executable creation:

pyinstaller --onefile --add-data="C:/DEVNEU/.venv/crawl4ai/Lib/site-packages/playwright_stealth/js;./playwright_stealth/js" --collect-data fake_http_header.data --collect-data fake_useragent.data crawl_multi_url.py

When i run the above code as python-file its working fine - but when i run the created executable program i get this error:

C:\DEVNEU\Python-Diverses\crawl4ai>crawl_multi_url.exe
                                                         Crawler Performance Monitor
╭──────────┬─────────────────────────────────────┬───────────┬─────────────┬───────────┬──────────┬──────────────────────────────────────────╮
│ Task ID  │ URL                                 │ Status    │ Memory (MB) │ Peak (MB) │ Duration │ Info                                     │
├──────────┼─────────────────────────────────────┼───────────┼─────────────┼───────────┼──────────┼──────────────────────────────────────────┤
│ SUMMARY  │ Total: 14                           │ Active: 0 │         2.8 │     105.6 │  0:00:12 │ ✓0 ✗14                                   │
├──────────┼─────────────────────────────────────┼───────────┼─────────────┼───────────┼──────────┼──────────────────────────────────────────┤
│ a6920beb │ https://www.1aes.de                 │ FAILED    │         0.3 │       0.3 │  0:00:00 │ Unexpected error in _crawl_web at line 1 │
│ 6d36514c │ https://www.1st-service.org         │ FAILED    │         0.3 │       0.3 │  0:00:00 │ Unexpected error in _crawl_web at line 1 │
│ d089d7f2 │ https://www.1afruechte.de           │ FAILED    │         0.3 │       0.3 │  0:00:00 │ Unexpected error in _crawl_web at line 1 │
│ aa3cd0ef │ https://www.1luvfoods.com           │ FAILED    │         0.3 │       0.3 │  0:00:00 │ Unexpected error in _crawl_web at line 1 │
│ 96750f07 │ https://www.1stforfoliage.com       │ FAILED    │         0.0 │       0.0 │  0:00:00 │ Unexpected error in _crawl_web at line 1 │
│ 1ac49dc3 │ https://www.1stcarpetcleaning.co.uk │ FAILED    │         0.3 │       0.3 │  0:00:00 │ Unexpected error in _crawl_web at line 1 │
│ bdad503e │ https://www.1o8.co                  │ FAILED    │         0.3 │       0.3 │  0:00:02 │ Unexpected error in _crawl_web at line 1 │
│ 39cf4c3f │ https://www.1qbit.com               │ FAILED    │         0.3 │       0.3 │  0:00:03 │ Unexpected error in _crawl_web at line 1 │
│ 9d5c2624 │ https://www.1stforsupplements.com   │ FAILED    │         0.0 │       0.0 │  0:00:03 │ Unexpected error in _crawl_web at line 1 │
│ 571ace60 │ https://www.1zf.com                 │ FAILED    │         0.0 │       0.0 │  0:00:03 │ Unexpected error in _crawl_web at line 1 │
│ 335ca9ce │ https://www.1stcalltrees.com        │ FAILED    │         0.3 │       0.3 │  0:00:04 │ Unexpected error in _crawl_web at line 1 │
│ 639a25fe │ https://www.1ste-edeldestillerie.de │ FAILED    │         0.3 │       0.3 │  0:00:05 │ Unexpected error in _crawl_web at line 1 │
│ 22ddf78f │ https://www.1toone.co.uk            │ FAILED    │         0.0 │       0.0 │  0:00:08 │ Unexpected error in _crawl_web at line 1 │
│ dc48d360 │ https://www.1stcut.net              │ FAILED    │         0.3 │       0.3 │  0:00:10 │ Unexpected error in _crawl_web at line 1 │
╰──────────┴─────────────────────────────────────┴───────────┴─────────────┴───────────┴──────────┴──────────────────────────────────────────╯
Failed to crawl https://www.1aes.de: Unexpected error in _crawl_web at line 1354 in _crawl_web (crawl4ai\async_crawler_strategy.py):
Error: Failed on navigating ACS-GOTO:
Page.goto: net::ERR_CONNECTION_CLOSED at https://www.1aes.de/
Call log:
  - navigating to "https://www.1aes.de/", waiting until "domcontentloaded"


Code context:

Failed to crawl https://www.1st-service.org: Unexpected error in _crawl_web at line 1354 in _crawl_web (crawl4ai\async_crawler_strategy.py):
Error: Failed on navigating ACS-GOTO:
Page.goto: net::ERR_CONNECTION_CLOSED at https://www.1st-service.org/
Call log:
  - navigating to "https://www.1st-service.org/", waiting until "domcontentloaded"


Code context:

Failed to crawl https://www.1afruechte.de: Unexpected error in _crawl_web at line 12 in load_js_script (crawl4ai\js_snippet\__init__.py):
Error: Script update_image_dimensions not found in the folder C:\Users\RapidTech1898\AppData\Local\Temp\_MEI258482\crawl4ai\js_snippet

Code context:

Failed to crawl https://www.1luvfoods.com: Unexpected error in _crawl_web at line 12 in load_js_script (crawl4ai\js_snippet\__init__.py):
Error: Script update_image_dimensions not found in the folder C:\Users\RapidTech1898\AppData\Local\Temp\_MEI258482\crawl4ai\js_snippet

Code context:

Failed to crawl https://www.1stforfoliage.com: Unexpected error in _crawl_web at line 12 in load_js_script (crawl4ai\js_snippet\__init__.py):
Error: Script update_image_dimensions not found in the folder C:\Users\RapidTech1898\AppData\Local\Temp\_MEI258482\crawl4ai\js_snippet

Code context:

Failed to crawl https://www.1stcarpetcleaning.co.uk: Unexpected error in _crawl_web at line 12 in load_js_script (crawl4ai\js_snippet\__init__.py):
Error: Script update_image_dimensions not found in the folder C:\Users\RapidTech1898\AppData\Local\Temp\_MEI258482\crawl4ai\js_snippet

Code context:

Failed to crawl https://www.1o8.co: Unexpected error in _crawl_web at line 12 in load_js_script (crawl4ai\js_snippet\__init__.py):
Error: Script update_image_dimensions not found in the folder C:\Users\RapidTech1898\AppData\Local\Temp\_MEI258482\crawl4ai\js_snippet

Code context:

Failed to crawl https://www.1qbit.com: Unexpected error in _crawl_web at line 12 in load_js_script (crawl4ai\js_snippet\__init__.py):
Error: Script update_image_dimensions not found in the folder C:\Users\RapidTech1898\AppData\Local\Temp\_MEI258482\crawl4ai\js_snippet

Code context:

Failed to crawl https://www.1stforsupplements.com: Unexpected error in _crawl_web at line 12 in load_js_script (crawl4ai\js_snippet\__init__.py):
Error: Script update_image_dimensions not found in the folder C:\Users\RapidTech1898\AppData\Local\Temp\_MEI258482\crawl4ai\js_snippet

Code context:

Failed to crawl https://www.1zf.com: Unexpected error in _crawl_web at line 12 in load_js_script (crawl4ai\js_snippet\__init__.py):
Error: Script update_image_dimensions not found in the folder C:\Users\RapidTech1898\AppData\Local\Temp\_MEI258482\crawl4ai\js_snippet

Code context:

Failed to crawl https://www.1stcalltrees.com: Unexpected error in _crawl_web at line 12 in load_js_script (crawl4ai\js_snippet\__init__.py):
Error: Script update_image_dimensions not found in the folder C:\Users\RapidTech1898\AppData\Local\Temp\_MEI258482\crawl4ai\js_snippet

Code context:

Failed to crawl https://www.1ste-edeldestillerie.de: Unexpected error in _crawl_web at line 12 in load_js_script (crawl4ai\js_snippet\__init__.py):
Error: Script update_image_dimensions not found in the folder C:\Users\RapidTech1898\AppData\Local\Temp\_MEI258482\crawl4ai\js_snippet

Code context:

Failed to crawl https://www.1toone.co.uk: Unexpected error in _crawl_web at line 12 in load_js_script (crawl4ai\js_snippet\__init__.py):
Error: Script update_image_dimensions not found in the folder C:\Users\RapidTech1898\AppData\Local\Temp\_MEI258482\crawl4ai\js_snippet

Code context:

Failed to crawl https://www.1stcut.net: Unexpected error in _crawl_web at line 1354 in _crawl_web (crawl4ai\async_crawler_strategy.py):
Error: Failed on navigating ACS-GOTO:
Page.goto: net::ERR_CONNECTION_CLOSED at https://www.1stcut.net/
Call log:
  - navigating to "https://www.1stcut.net/", waiting until "domcontentloaded"


Code context:

0

How can i create a working exe-file using pyinstaller for this?

bwoodsend · 2025-02-23T12:06:43Z

bwoodsend
Feb 23, 2025
Maintainer

Error: Script update_image_dimensions not found in the folder C:\Users\RapidTech1898\AppData\Local\Temp_MEI258482\crawl4ai\js_snippet

Does collecting data files for crawl4ai fix it?

0 replies

Rapid1898-code · 2025-02-23T12:36:39Z

Rapid1898-code
Feb 23, 2025
Author

I think that helped - using this command it seems to work now:

pyinstaller --onefile --add-data="C:/DEVNEU/.venv/crawl4ai/Lib/site-packages/playwright_stealth/js;./playwright_stealth/js" --collect-data fake_http_header.data --collect-data fake_useragent.data --collect-data crawl4ai crawl_multi_url.py

Thanks for your help!

0 replies

Rapid1898-code · 2025-02-26T12:25:52Z

Rapid1898-code
Feb 26, 2025
Author

Hello - i have a follow up question - i now try to bundle this also on a Mac computer using the following command:

pyinstaller --onefile --add-data="/Users/markus/Documents/DEV/venv/crawl4ai/lib/python3.13/site-packages/playwright_stealth/js;./playwright_stealth/js" --collect-data fake_http_header.data --collect-data fake_useragent.data --collect-data crawl4ai test.py

But i get this error:

(crawl4ai) markus@Macmini crawl4ai % pyinstaller --onefile  --add-data="/Users/markus/Documents/DEV/venv/crawl4ai/lib/python3.13/site-packages/playwright_stealth/js;./playwright_stealth/js" --collect-data fake_http_header.data --collect-data fake_useragent.data --collect-data crawl4ai test.py
usage: pyinstaller [-h] [-v] [-D] [-F] [--specpath DIR] [-n NAME]
                   [--contents-directory CONTENTS_DIRECTORY]
                   [--add-data SOURCE:DEST] [--add-binary SOURCE:DEST]
                   [-p DIR] [--hidden-import MODULENAME]
                   [--collect-submodules MODULENAME]
                   [--collect-data MODULENAME] [--collect-binaries MODULENAME]
                   [--collect-all MODULENAME] [--copy-metadata PACKAGENAME]
                   [--recursive-copy-metadata PACKAGENAME]
                   [--additional-hooks-dir HOOKSPATH]
                   [--runtime-hook RUNTIME_HOOKS] [--exclude-module EXCLUDES]
                   [--splash IMAGE_FILE]
                   [-d {all,imports,bootloader,noarchive}] [--optimize LEVEL]
                   [--python-option PYTHON_OPTION] [-s] [--noupx]
                   [--upx-exclude FILE] [-c] [-w]
                   [--hide-console {minimize-early,hide-early,hide-late,minimize-late}]
                   [-i <FILE.ico or FILE.exe,ID or FILE.icns or Image or "NONE">]
                   [--disable-windowed-traceback] [--version-file FILE]
                   [--manifest <FILE or XML>] [-m <FILE or XML>] [-r RESOURCE]
                   [--uac-admin] [--uac-uiaccess] [--argv-emulation]
                   [--osx-bundle-identifier BUNDLE_IDENTIFIER]
                   [--target-architecture ARCH] [--codesign-identity IDENTITY]
                   [--osx-entitlements-file FILENAME] [--runtime-tmpdir PATH]
                   [--bootloader-ignore-signals] [--distpath DIR]
                   [--workpath WORKPATH] [-y] [--upx-dir UPX_DIR] [--clean]
                   [--log-level LEVEL]
                   scriptname [scriptname ...]
pyinstaller: error: argument --add-data: Wrong syntax, should be --add-data=SOURCE:DEST

How can i bundle this also on Mac?

1 reply

rokm Feb 26, 2025
Maintainer

How can i bundle this also on Mac?

Use correct separator - as written in the error message, it should be colon (:), not semi-colon (;).

With older PyInstaller versions (< 6.x), this used to be OS-dependent (; on Windows, : elsewhere), but now : works everywhere (and ; still works only on Windows).

Rapid1898-code · 2025-02-26T13:23:25Z

Rapid1898-code
Feb 26, 2025
Author

ok i see - thanks a lot again - i tried to bundle this now using the following command:

pyinstaller --onefile --add-data="/Users/markus/Documents/DEV/venv/crawl4ai/lib/python3.13/site-packages/playwright_stealth/js:./playwright_stealth/js" --collect-data fake_http_header.data --collect-data fake_useragent.data --collect-data crawl4ai test.py

but on Mac i get this error:

Last login: Wed Feb 26 14:10:56 on ttys002
/Users/markus/Documents/DEV/Python-Diverses/crawl4ai/test ; exit;
markus@Macmini ~ % /Users/markus/Documents/DEV/Python-Diverses/crawl4ai/test ; exit;
Traceback (most recent call last):
  File "test.py", line 22, in <module>
  File "asyncio/runners.py", line 195, in run
  File "asyncio/runners.py", line 118, in run
  File "asyncio/base_events.py", line 725, in run_until_complete
  File "test.py", line 13, in main
  File "crawl4ai/async_webcrawler.py", line 241, in __aenter__
  File "crawl4ai/async_webcrawler.py", line 225, in start
  File "crawl4ai/async_crawler_strategy.py", line 887, in __aenter__
  File "crawl4ai/async_crawler_strategy.py", line 897, in start
  File "crawl4ai/async_crawler_strategy.py", line 402, in start
  File "playwright/async_api/_generated.py", line 14418, in launch
  File "playwright/_impl/_browser_type.py", line 95, in launch
  File "playwright/_impl/_connection.py", line 61, in send
  File "playwright/_impl/_connection.py", line 528, in wrap_api_call
playwright._impl._errors.Error: BrowserType.launch: Executable doesn't exist at /private/var/folders/h5/3736gxwx56ndr2pplhr5nvk00000gn/T/_MEIYvcmyI/playwright/driver/package/.local-browsers/chromium-1155/chrome-mac/Chromium.app/Contents/MacOS/Chromium
╔════════════════════════════════════════════════════════════╗
║ Looks like Playwright was just installed or updated.       ║
║ Please run the following command to download new browsers: ║
║                                                            ║
║     playwright install                                     ║
║                                                            ║
║ <3 Playwright Team                                         ║
╚════════════════════════════════════════════════════════════╝
[PYI-19190:ERROR] Failed to execute script 'test' due to unhandled exception!

Saving session...
...copying shared history...
...saving history...truncating history files...
...completed.

[Prozess beendet]

Is it probably necessary to use here something else after the colon:

:./playwright_stealth/js

1 reply

rokm Feb 26, 2025
Maintainer

playwright will not work with PyInstaller on macOS, because we do not properly preserve the browser's .app bundle that playwright tries to pass as data files.

If you were building in onedir mode, you could manually copy browser files post-hoc to remedy the situation, but with onefile, you have no such option.

Rapid1898-code · 2025-02-26T13:31:20Z

Rapid1898-code
Feb 26, 2025
Author

ok i think i understand -
What do in need to copy post-hoc manually so playwright is working using onedir mode?

1 reply

rokm Feb 26, 2025
Maintainer

What do in need to copy post-hoc manually so playwright is working using onedir mode?

The browser files that should be in playwright/driver/package/.local-browsers/chromium-1155/chrome-mac (you should remove the files that PyInstaller copied there, if any, and then copy the files from your python environment, while making sure that symlinks (if any) are preserved).

Rapid1898-code · 2025-02-26T14:10:06Z

Rapid1898-code
Feb 26, 2025
Author

I checked this path

/Users/markus/Documents/DEV/venv/crawl4ai/lib/python3.13/site-packages/playwright/driver/package

but was not able to find a ".local-browsers" in there

1 reply

rokm Feb 26, 2025
Maintainer

Then you did not install browser(s) into the package as per playwright's instructions.

Rapid1898-code · 2025-02-27T13:03:26Z

Rapid1898-code
Feb 27, 2025
Author

Hello it seems that i now get a different error when doing:

pyinstaller --onedir --collect-data fake_http_header.data --collect-data fake_useragent.data --collect-data crawl4ai test.py

Any idea what i should do now?

28332 INFO: Removing dir /Users/markus/Documents/DEV/Python-Diverses/crawl4ai/dist/test
28350 INFO: Building COLLECT COLLECT-00.toc
Traceback (most recent call last):
  File "/Users/markus/Documents/DEV/venv/crawl4ai/lib/python3.13/site-packages/PyInstaller/building/utils.py", line 315, in process_collected_binary
    osxutils.sign_binary(cached_name, codesign_identity, entitlements_file)
    ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/markus/Documents/DEV/venv/crawl4ai/lib/python3.13/site-packages/PyInstaller/utils/osx.py", line 391, in sign_binary
    raise SystemError(f"codesign command ({cmd_args}) failed with error code {p.returncode}!\noutput: {p.stdout}")
SystemError: codesign command (['/usr/bin/codesign', '-s', '-', '--force', '--all-architectures', '--timestamp', '/Users/markus/Library/Application Support/pyinstaller/bincache00py31364bit/arm64/adhoc/no-entitlements/playwright/driver/package/.local-browsers/chromium-1155/chrome-mac/Chromium.app/Contents/MacOS/Chromium']) failed with error code 1!
output: /Users/markus/Library/Application Support/pyinstaller/bincache00py31364bit/arm64/adhoc/no-entitlements/playwright/driver/package/.local-browsers/chromium-1155/chrome-mac/Chromium.app/Contents/MacOS/Chromium: replacing existing signature
/Users/markus/Library/Application Support/pyinstaller/bincache00py31364bit/arm64/adhoc/no-entitlements/playwright/driver/package/.local-browsers/chromium-1155/chrome-mac/Chromium.app/Contents/MacOS/Chromium: bundle format unrecognized, invalid, or unsuitable
In subcomponent: /Users/markus/Library/Application Support/pyinstaller/bincache00py31364bit/arm64/adhoc/no-entitlements/playwright/driver/package/.local-browsers/chromium-1155/chrome-mac/Chromium.app/Contents/Frameworks/Chromium Framework.framework


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/markus/Documents/DEV/venv/crawl4ai/bin/pyinstaller", line 8, in <module>
    sys.exit(_console_script_run())
             ~~~~~~~~~~~~~~~~~~~^^
  File "/Users/markus/Documents/DEV/venv/crawl4ai/lib/python3.13/site-packages/PyInstaller/__main__.py", line 231, in _console_script_run
    run()
    ~~~^^
  File "/Users/markus/Documents/DEV/venv/crawl4ai/lib/python3.13/site-packages/PyInstaller/__main__.py", line 215, in run
    run_build(pyi_config, spec_file, **vars(args))
    ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/markus/Documents/DEV/venv/crawl4ai/lib/python3.13/site-packages/PyInstaller/__main__.py", line 70, in run_build
    PyInstaller.building.build_main.main(pyi_config, spec_file, **kwargs)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/markus/Documents/DEV/venv/crawl4ai/lib/python3.13/site-packages/PyInstaller/building/build_main.py", line 1270, in main
    build(specfile, distpath, workpath, clean_build)
    ~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/markus/Documents/DEV/venv/crawl4ai/lib/python3.13/site-packages/PyInstaller/building/build_main.py", line 1208, in build
    exec(code, spec_namespace)
    ~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/markus/Documents/DEV/Python-Diverses/crawl4ai/test.spec", line 36, in <module>
    coll = COLLECT(
        exe,
    ...<5 lines>...
        name='test',
    )
  File "/Users/markus/Documents/DEV/venv/crawl4ai/lib/python3.13/site-packages/PyInstaller/building/api.py", line 1122, in __init__
    self.__postinit__()
    ~~~~~~~~~~~~~~~~~^^
  File "/Users/markus/Documents/DEV/venv/crawl4ai/lib/python3.13/site-packages/PyInstaller/building/datastruct.py", line 184, in __postinit__
    self.assemble()
    ~~~~~~~~~~~~~^^
  File "/Users/markus/Documents/DEV/venv/crawl4ai/lib/python3.13/site-packages/PyInstaller/building/api.py", line 1168, in assemble
    src_name = process_collected_binary(
        src_name,
    ...<7 lines>...
        strict_arch_validation=(typecode == 'EXTENSION'),
    )
  File "/Users/markus/Documents/DEV/venv/crawl4ai/lib/python3.13/site-packages/PyInstaller/building/utils.py", line 334, in process_collected_binary
    raise SystemError(f"Failed to process binary {cached_name!r}!") from e
SystemError: Failed to process binary '/Users/markus/Library/Application Support/pyinstaller/bincache00py31364bit/arm64/adhoc/no-entitlements/playwright/driver/package/.local-browsers/chromium-1155/chrome-mac/Chromium.app/Contents/MacOS/Chromium'!

1 reply

rokm Feb 27, 2025
Maintainer

Yeah, we don't properly handle browser .app bundle that playwright is trying to pass as data. That's the part of the issues I was referring to earlier.

If you really must use playwright + macOS + PyInstaller, you will first need to disable collection of playwright/driver/package/.local-browsers; there are several ways:

move playwright/driver/package/.local-browsers out of the playwright package (same as if you did not install the browser into package) into some external directory (from which you can later copy the files into frozen application); OR
modify playwright's PyInstaller hooks in playwright/_impl/__pyinstaller/ and comment out collect_data() calls; OR
create your own hooks with same names (hook-playwright.sync_api.py, hook-playwright.async_api.py) that do nothing, and pass their directory path to PyInstaller via --additional-hooks-dir command-line option

Then, once the build finishes, copy the browser files into dist/<name>/_internal/playwright/driver/package/.local-browsers.

That's about the best you can try at the moment; if it doesn't work, then consider playwright + macOS + PyInstaller to be unsupported combination until we get around to solving the underlying issues. Or find a way that allows you to use playwright without having to bundle browser with your application (e.g., download and install to persistent location on first use, or use system-installed browser, if possible).

Rapid1898-code · 2025-02-27T13:55:48Z

Rapid1898-code
Feb 27, 2025
Author

Hello due your great help and patience i think i found a solution:

due the reasson that --add-data is not possible on MAC we need to create the folder with --onedir and then copy the .local browsers to the created dir

pyinstaller --onedir --collect-data fake_http_header.data --collect-data fake_useragent.data --collect-data crawl4ai crawl_data.py
(this is NOT working when .local-browsers are allready installed in the venv-folders!!!)

copy playwright_stealth folder from venv (crawl4ai/lib/python3.13/site-packages) to program folder (prgname/_internal/)
eg. fullpathFROM: '/Users/markus/Documents/DEV/Python-Diverses/crawl4ai/dist/crawl_data/_internal/playwright_stealth/js/chrome.csi.js'
eg. fullpathTO: /Users/markus/Documents/DEV/venv/crawl4ai/lib/python3.13/site-packages/playwright_stealth

install PLAYWRIGHT_BROWSERS_PATH=0 playwright install chromium
(or copy from a previous created working solution before)

copy playwright folder for .local-browsers from venv (or previous created working solution) to program folder (prgname/dist/crawl_data/_internal/playwright/driver/package/.local-browsers)
eg. fullpathFROM1 (from VENV): /Users/markus/Documents/DEV/venv/crawl4ai/lib/python3.13/site-packages/playwright/driver/package/.local-browsers
eg. fullpathFRO2 (from previous working program version): /Users/markus/Documents/DEV/Python-Diverses/crawl4ai/dist/test/_internal/playwright/driver/package/.local-browsers
eg. fullpathTO: /Users/markus/Documents/DEV/Python-Diverses/crawl4ai/dist/crawl_data/_internal/playwright/driver/package/.local-browsers

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PyInstaller

Pyinstaller with crawl4ai module not working? #9036

{{title}}

Replies: 8 comments 5 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

PyInstaller

Pyinstaller with crawl4ai module not working? #9036

Rapid1898-code Feb 23, 2025

Replies: 8 comments · 5 replies

bwoodsend Feb 23, 2025 Maintainer

Rapid1898-code Feb 23, 2025 Author

Rapid1898-code Feb 26, 2025 Author

rokm Feb 26, 2025 Maintainer

Rapid1898-code Feb 26, 2025 Author

rokm Feb 26, 2025 Maintainer

Rapid1898-code Feb 26, 2025 Author

rokm Feb 26, 2025 Maintainer

Rapid1898-code Feb 26, 2025 Author

rokm Feb 26, 2025 Maintainer

Rapid1898-code Feb 27, 2025 Author

rokm Feb 27, 2025 Maintainer

Rapid1898-code Feb 27, 2025 Author

Rapid1898-code
Feb 23, 2025

Replies: 8 comments 5 replies

bwoodsend
Feb 23, 2025
Maintainer

Rapid1898-code
Feb 23, 2025
Author

Rapid1898-code
Feb 26, 2025
Author

rokm Feb 26, 2025
Maintainer

Rapid1898-code
Feb 26, 2025
Author

rokm Feb 26, 2025
Maintainer

Rapid1898-code
Feb 26, 2025
Author

rokm Feb 26, 2025
Maintainer

Rapid1898-code
Feb 26, 2025
Author

rokm Feb 26, 2025
Maintainer

Rapid1898-code
Feb 27, 2025
Author

rokm Feb 27, 2025
Maintainer

Rapid1898-code
Feb 27, 2025
Author