ENH(string dtype): Make str.decode return str dtype

[rhshadrach]

• closes #xxxx (Replace xxxx with the GitHub issue number)
• Tests added and passed if fixing a bug or adding a new feature
• All code checks passed.
• Added type annotations to new arguments/methods/functions.
• Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file if fixing a bug or adding a new feature.

[rhshadrach]

The change from ser.map to using Series is just to make this test a bit more explicit. Using ser.map(...).astype("str") also passes.

[jorisvandenbossche]

Looks good!

[jorisvandenbossche]

You can probably simplify this and always to .to_numpy()? (or np.asarray(..))
In the case of object dtype in the else branch, that will return the same (and be as cheap) as _values I think

[rhshadrach]

Confirmed - thanks.

[jorisvandenbossche]

@rhshadrach can you update this?

[rhshadrach]

@jorisvandenbossche - the issue with .to_numpy on NumPy-backed Series is that we set the underlying data to read-only. In pytables, we switch out NA values in libwriters.string_array_replace_from_nan_rep, which is causing the tests to fail.

Perhaps there could be a way (e.g. Series._to_numpy) to always get a corresponding NumPy array that isn't read-only? Barring this, it seems to me we could either always (and unnecessarily) make a copy, or use my original branching logic. Open to other ideas too.

[jorisvandenbossche]

Hmm, good point. Ideally we would be able to solve this without using private APIs, I think, because it is a good case study for what also other people (external code) could run into.

So I think what we have said before is that downstream users could do data.flags.writeable = True on the result of to_numpy() if they know what they are doing (and in this case we know that we indeed own the memory, because we are reading a file and created that data and not yet returned it to the user).

But this also makes me wonder if we should re-discuss if we have to add some keyword to to_numpy() to get this (e.g. something like writeable=True)

[rhshadrach]

@jorisvandenbossche - ran into a couple more test changes.

[mroeschke]

Thanks @rhshadrach

[lumberbot-app]

Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

git checkout 2.3.x
git pull

2. Cherry pick the first parent branch of the this PR on top of the older branch:

git cherry-pick -x -m1 c36da3f6ded4141add4b3b16c252cedf4641e5ea

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

git commit -am 'Backport PR #60709: ENH(string dtype): Make str.decode return str dtype'

4. Push to a named branch:

git push YOURFORK 2.3.x:auto-backport-of-pr-60709-on-2.3.x

5. Create a PR against branch 2.3.x, I would have named this PR:

"Backport PR #60709 on branch 2.3.x (ENH(string dtype): Make str.decode return str dtype)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the Still Needs Manual Backport label once the PR gets merged.

If these instructions are inaccurate, feel free to suggest an improvement.

[rhshadrach]

Backport PR: #60821