Logs from browserforge

[vdusek]

Before

• Before browserforge (Crawlee < 0.6)

2025-03-04T09:35:42.108Z ACTOR: Pulling Docker image of build vbJ3dgYm28POtiN1j from repository.
2025-03-04T09:35:42.311Z ACTOR: Creating Docker container.
2025-03-04T09:35:42.358Z ACTOR: Starting Docker container.
2025-03-04T09:35:44.092Z [apify] INFO  Initializing Actor...
2025-03-04T09:35:44.094Z [apify] INFO  System info ({"apify_sdk_version": "2.3.1", "apify_client_version": "1.9.2", "crawlee_version": "0.6.1", "python_version": "3.13.2", "os": "linux"})
2025-03-04T09:35:44.119Z [apify] DEBUG Debug message
2025-03-04T09:35:44.123Z [apify] INFO  Info message
2025-03-04T09:35:44.124Z [apify] WARN  Warning message
2025-03-04T09:35:44.126Z [apify] ERROR Error message
2025-03-04T09:35:44.128Z [apify] ERROR Exception message
2025-03-04T09:35:44.131Z       Traceback (most recent call last):
2025-03-04T09:35:44.133Z         File "/usr/src/app/src/main.py", line 25, in main
2025-03-04T09:35:44.134Z           raise ValueError('Dummy ValueError')
2025-03-04T09:35:44.136Z       ValueError: Dummy ValueError
2025-03-04T09:35:44.139Z [apify] INFO  Multi
2025-03-04T09:35:44.141Z line
2025-03-04T09:35:44.142Z log
2025-03-04T09:35:44.144Z message
2025-03-04T09:35:44.146Z [apify] ERROR Actor failed with an exception
2025-03-04T09:35:44.147Z       Traceback (most recent call last):
2025-03-04T09:35:44.149Z         File "/usr/src/app/src/main.py", line 33, in main
2025-03-04T09:35:44.150Z           raise RuntimeError('Dummy RuntimeError')
2025-03-04T09:35:44.152Z       RuntimeError: Dummy RuntimeError
2025-03-04T09:35:44.154Z [apify] INFO  Exiting Actor ({"exit_code": 91})

After

• With browserforge (Crawlee ~= 0.6)

2025-03-04T09:35:42.108Z ACTOR: Pulling Docker image of build vbJ3dgYm28POtiN1j from repository.
2025-03-04T09:35:42.311Z ACTOR: Creating Docker container.
2025-03-04T09:35:42.358Z ACTOR: Starting Docker container.
2025-03-04T09:35:43.463Z Downloading model definition files...
2025-03-04T09:35:43.725Z input-network.zip             OK!
2025-03-04T09:35:43.727Z browser-helper-file.json      OK!
2025-03-04T09:35:43.729Z header-network.zip            OK!
2025-03-04T09:35:43.731Z headers-order.json            OK!
2025-03-04T09:35:43.732Z fingerprint-network.zip       OK!
2025-03-04T09:35:44.092Z [apify] INFO  Initializing Actor...
2025-03-04T09:35:44.094Z [apify] INFO  System info ({"apify_sdk_version": "2.3.1", "apify_client_version": "1.9.2", "crawlee_version": "0.6.1", "python_version": "3.13.2", "os": "linux"})
2025-03-04T09:35:44.119Z [apify] DEBUG Debug message
2025-03-04T09:35:44.123Z [apify] INFO  Info message
2025-03-04T09:35:44.124Z [apify] WARN  Warning message
2025-03-04T09:35:44.126Z [apify] ERROR Error message
2025-03-04T09:35:44.128Z [apify] ERROR Exception message
2025-03-04T09:35:44.131Z       Traceback (most recent call last):
2025-03-04T09:35:44.133Z         File "/usr/src/app/src/main.py", line 25, in main
2025-03-04T09:35:44.134Z           raise ValueError('Dummy ValueError')
2025-03-04T09:35:44.136Z       ValueError: Dummy ValueError
2025-03-04T09:35:44.139Z [apify] INFO  Multi
2025-03-04T09:35:44.141Z line
2025-03-04T09:35:44.142Z log
2025-03-04T09:35:44.144Z message
2025-03-04T09:35:44.146Z [apify] ERROR Actor failed with an exception
2025-03-04T09:35:44.147Z       Traceback (most recent call last):
2025-03-04T09:35:44.149Z         File "/usr/src/app/src/main.py", line 33, in main
2025-03-04T09:35:44.150Z           raise RuntimeError('Dummy RuntimeError')
2025-03-04T09:35:44.152Z       RuntimeError: Dummy RuntimeError
2025-03-04T09:35:44.154Z [apify] INFO  Exiting Actor ({"exit_code": 91})

Result

• As a result test test_actor_logging is skipped.

Possible solutions

• Remove the browserforge output and rm skip flag from test_actor_logging.
• Leave it as it is and rm (or update) the test_actor_logging test.
  • in case of updating - there is no line order guaranteed.

[vdusek]

It is important to note that during every Actor run, zip files are now downloaded from the apify/fingerprint-suite repository.

[Erol444]

Not sure if related, but I get this error every time I run an actor:

ACTOR: Pulling Docker image of build {MY_ID} from repository.
2025-03-08T12:46:21.303Z ACTOR: Creating Docker container.
2025-03-08T12:46:21.387Z ACTOR: Starting Docker container.
2025-03-08T12:46:23.361Z Downloading model definition files...
2025-03-08T12:46:23.615Z Error downloading fingerprint-network.zip: [Errno 13] Permission denied: '/usr/local/lib/python3.13/site-packages/browserforge/fingerprints/data/fingerprint-network.zip'
2025-03-08T12:46:23.618Z Error downloading fingerprint-network.zip: [Errno 13] Permission denied: '/usr/local/lib/python3.13/site-packages/browserforge/fingerprints/data/fingerprint-network.zip'
2025-03-08T12:46:23.620Z Error downloading fingerprint-network.zip: [Errno 13] Permission denied: '/usr/local/lib/python3.13/site-packages/browserforge/fingerprints/data/fingerprint-network.zip'
2025-03-08T12:46:23.622Z Error downloading fingerprint-network.zip: [Errno 13] Permission denied: '/usr/local/lib/python3.13/site-packages/browserforge/fingerprints/data/fingerprint-network.zip'
2025-03-08T12:46:23.625Z Error downloading fingerprint-network.zip: [Errno 13] Permission denied: '/usr/local/lib/python3.13/site-packages/browserforge/fingerprints/data/fingerprint-network.zip'
2025-03-08T12:46:23.627Z Downloading model definition files...
2025-03-08T12:46:23.645Z Error downloading input-network.zip: [Errno 13] Permission denied: '/usr/local/lib/python3.13/site-packages/browserforge/headers/data/input-network.zip'
2025-03-08T12:46:23.648Z Error downloading input-network.zip: [Errno 13] Permission denied: '/usr/local/lib/python3.13/site-packages/browserforge/headers/data/input-network.zip'
2025-03-08T12:46:23.650Z Error downloading input-network.zip: [Errno 13] Permission denied: '/usr/local/lib/python3.13/site-packages/browserforge/headers/data/input-network.zip'
2025-03-08T12:46:23.652Z Error downloading input-network.zip: [Errno 13] Permission denied: '/usr/local/lib/python3.13/site-packages/browserforge/headers/data/input-network.zip'
2025-03-08T12:46:23.658Z Traceback (most recent call last):
2025-03-08T12:46:23.667Z   File "<frozen runpy>", line 198, in _run_module_as_main
2025-03-08T12:46:23.669Z   File "<frozen runpy>", line 88, in _run_code
2025-03-08T12:46:23.671Z   File "/usr/src/app/src/__main__.py", line 3, in <module>
2025-03-08T12:46:23.674Z     from .main import main
2025-03-08T12:46:23.676Z   File "/usr/src/app/src/main.py", line 10, in <module>
2025-03-08T12:46:23.678Z     from apify import Actor, Request
2025-03-08T12:46:23.680Z   File "/usr/local/lib/python3.13/site-packages/apify/__init__.py", line 15, in <module>
2025-03-08T12:46:23.682Z     from apify._actor import Actor
2025-03-08T12:46:23.684Z   File "/usr/local/lib/python3.13/site-packages/apify/_actor.py", line 28, in <module>
2025-03-08T12:46:23.686Z     from apify._charging import ChargeResult, ChargingManager, ChargingManagerImplementation
2025-03-08T12:46:23.689Z   File "/usr/local/lib/python3.13/site-packages/apify/_charging.py", line 17, in <module>
2025-03-08T12:46:23.692Z     from apify.storages import Dataset
2025-03-08T12:46:23.694Z   File "/usr/local/lib/python3.13/site-packages/apify/storages/__init__.py", line 3, in <module>
2025-03-08T12:46:23.698Z     from ._request_list import RequestList
2025-03-08T12:46:23.701Z   File "/usr/local/lib/python3.13/site-packages/apify/storages/_request_list.py", line 13, in <module>
2025-03-08T12:46:23.703Z     from crawlee.http_clients import HttpClient, HttpxHttpClient
2025-03-08T12:46:23.705Z   File "/usr/local/lib/python3.13/site-packages/crawlee/http_clients/__init__.py", line 6, in <module>
2025-03-08T12:46:23.707Z     from ._httpx import HttpxHttpClient
2025-03-08T12:46:23.710Z   File "/usr/local/lib/python3.13/site-packages/crawlee/http_clients/_httpx.py", line 13, in <module>
2025-03-08T12:46:23.712Z     from crawlee.fingerprint_suite import HeaderGenerator
2025-03-08T12:46:23.714Z   File "/usr/local/lib/python3.13/site-packages/crawlee/fingerprint_suite/__init__.py", line 1, in <module>
2025-03-08T12:46:23.717Z     from ._browserforge_adapter import BrowserforgeFingerprintGenerator as DefaultFingerprintGenerator
2025-03-08T12:46:23.732Z   File "/usr/local/lib/python3.13/site-packages/crawlee/fingerprint_suite/_browserforge_adapter.py", line 10, in <module>
2025-03-08T12:46:23.734Z     from browserforge.fingerprints import Fingerprint as bf_Fingerprint
2025-03-08T12:46:23.736Z   File "/usr/local/lib/python3.13/site-packages/browserforge/fingerprints/__init__.py", line 5, in <module>
2025-03-08T12:46:23.738Z     from browserforge.headers import Browser
2025-03-08T12:46:23.741Z   File "/usr/local/lib/python3.13/site-packages/browserforge/headers/__init__.py", line 5, in <module>
2025-03-08T12:46:23.743Z     from .generator import Browser, HeaderGenerator
2025-03-08T12:46:23.745Z   File "/usr/local/lib/python3.13/site-packages/browserforge/headers/generator.py", line 80, in <module>
2025-03-08T12:46:23.747Z     class HeaderGenerator:
2025-03-08T12:46:23.750Z     ...<470 lines>...
2025-03-08T12:46:23.752Z             )
2025-03-08T12:46:23.756Z   File "/usr/local/lib/python3.13/site-packages/browserforge/headers/generator.py", line 86, in HeaderGenerator
2025-03-08T12:46:23.759Z     input_generator_network = BayesianNetwork(DATA_DIR / "input-network.zip")
2025-03-08T12:46:23.761Z   File "/usr/local/lib/python3.13/site-packages/browserforge/bayesian_network.py", line 103, in __init__
2025-03-08T12:46:23.763Z     network_definition = extract_json(path)
2025-03-08T12:46:23.766Z   File "/usr/local/lib/python3.13/site-packages/browserforge/bayesian_network.py", line 288, in extract_json
2025-03-08T12:46:23.768Z     with zipfile.ZipFile(path, 'r') as zf:
2025-03-08T12:46:23.770Z          ~~~~~~~~~~~~~~~^^^^^^^^^^^
2025-03-08T12:46:23.772Z   File "/usr/local/lib/python3.13/zipfile/__init__.py", line 1367, in __init__
2025-03-08T12:46:23.774Z     self.fp = io.open(file, filemode)
2025-03-08T12:46:23.776Z               ~~~~~~~^^^^^^^^^^^^^^^^
2025-03-08T12:46:23.779Z FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/lib/python3.13/site-packages/browserforge/headers/data/input-network.zip'

[Erol444]

@vdusek you should look into this asap - new py actors will by default use 2.4.0 apify, which throws the error above when starting the actor on apify platform. Thanks to this issue in actor's requirements.txt I pinned apify==2.3.0 which fixes the error above (I was messing with Dockerfile at first).

[Pijukatel]

Proposed solution:

• Create new PyPi package apify_fingerprint_datapoints: feat: Fingerprint datapoints python package fingerprint-suite#353
• Change browserforge to not download any files, but explicitly import those files from newly created package apify_fingerprint_datapoints. PR to be created from this branch
  after package is published to Pypi
• Update all Apify requirements constraints to depend on the latest browserforge version

[Pijukatel]

It should be fixed now for new templates that will use new base docker image. It is quite an uggly import-time-patch workaround because the PR in the upstream third party repo did not get merged yet.