Unknown subtask ArkoseLogin

[karashiiro]

Not sure if this is the issue or if there are multiple separate issues right now, but I ran the scraper tests just now and couldn't log in:

    Unknown subtask ArkoseLogin

      109 |         next = await this.handleSuccessSubtask(next);
      110 |       } else {
    > 111 |         throw new Error(`Unknown subtask ${next.subtask.subtask_id}`);
          |               ^
      112 |       }
      113 |     }
      114 |     if ('err' in next) {

      at TwitterUserAuth.login (src/auth-user.ts:111:15)
      at Scraper.login (src/scraper.ts:410:5)
      at getScraper (src/test-utils.ts:55:5)
      at Object.<anonymous> (src/tweets.test.ts:331:19)

The login flow involves a bunch of task messaging back and forth, and I haven't seen this task name before, so it's possible that the login flow changed, in which case a bunch of APIs will stop working. Still need to figure out how exactly this changed, though.

Originally posted by @karashiiro in #113 (comment)

[karashiiro]

Actually, when I tried debugging this just now, I manually logged into the account I was using for unit tests to compare against the real login flow for that particular account in the browser console. It was the same as what it had been before it broke, and then when I ran the unit tests again they all worked as expected.

I wonder if this means this issue was some kind of suspicious account check, and logging in through the website cleared it? At any rate, there doesn't seem to be any new fundamental issue with the scraper. I do still want to understand why this alternate auth path happened before I logged in on the website, though.

Originally posted by @karashiiro in #113 (comment)

[karashiiro]

@danbednarski I won't be able to look into this for a couple of weeks (until 1/2) because I'll be traveling with very little downtime, but the comments I copied to this issue are my best understanding of what's going on so far.

[catdevnull]

i'm sure you know this already but this seems like a semi-randomized arkose labs challenge (something like a recaptcha.) maybe twitter-scraper could provide the facilities for a user to hook into 2captcha.com or something similar?

[nvsd]

To further confirm what everyone is thinking, I cycle between 3 accounts with the scraper. All three had this issue and had a captcha challenge when signing out and back in.