Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Browserpool slices of last char of proxy url #2878

Open
1 task
harm-matthias-harms opened this issue Mar 7, 2025 · 1 comment
Open
1 task

Browserpool slices of last char of proxy url #2878

harm-matthias-harms opened this issue Mar 7, 2025 · 1 comment
Assignees
Labels
bug Something isn't working. t-tooling Issues with this label are in the ownership of the tooling team.

Comments

@harm-matthias-harms
Copy link

Which package is this bug report for? If unsure which one to select, leave blank

@crawlee/browser-pool

Issue description

We encounter a problem trying to use socks proxies with browser-pool. In particular, we lose the last digit of the port number when slicing the URL.

https://github.com/apify/crawlee/blob/master/packages/browser-pool/src/anonymize-proxy.ts#L19

Here, the last character of the URL is sliced. This is fine for HTTP proxies because they end with /, but socks proxies don't have a trailing slash. From our first test, HTTP proxies with trailing slashes work as expected, and we don't need to slice them.

For http

proxy: http://username:password@proxy:1000
url.href: http://username:password@proxy:1000/
url.href.slice(0, -1): http://username:password@proxy:1000

For socks5

proxy: socks5://username:password@proxy:1000
url.href: socks5://username:password@proxy:1000
url.href.slice(0, -1):  socks5://username:password@proxy:100

Code sample

Package version

latest

Node.js version

lts

Operating system

linux

Apify platform

  • Tick me if you encountered this issue on the Apify platform

I have tested this on the next release

No response

Other context

No response

@harm-matthias-harms harm-matthias-harms added the bug Something isn't working. label Mar 7, 2025
@github-actions github-actions bot added the t-tooling Issues with this label are in the ownership of the tooling team. label Mar 7, 2025
@janbuchar
Copy link
Contributor

Hello, and thanks for your interest in Crawlee. It seems that the issue is missing a code sample that reproduces the issue, could you please add that? Without that, it is difficult for us to troubleshoot this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working. t-tooling Issues with this label are in the ownership of the tooling team.
Projects
None yet
Development

No branches or pull requests

2 participants