Clarify value of "authorized" in relation to "indexed"

[rwstauner]

I fixed the display issue in metacpan/metacpan-web#1251
but I still wonder if the api should be changed at all (or docs be clarified or something):

However, if we were going to change the indexer/api data, i'm not sure what would be appropriate...
if indexed is false, authorized could be true (b/c it's not _un_authorized) or maybe authorized should technically be null (undefined), though I'm not sure how ES feels about that.

If indexed is false, should consumers simply ignore authorized (we can leave it as is)?
Or to demonstrate that it should be ignored should we change the value?

[rwstauner]

I had trouble finding this other comment (#373 (comment)) so I'm quoting it here to collect the comments in one place:

The documentation for "indexed"
says "include in the search index", however that sounds different to me than the more verbose documentation for "set_indexed".
It was once described to me that "indexed" essentially meant "not no_indexed by the META file"
and that seems like roughly what "set_indexed" is doing.

The "authorized" value defaults to true and gets set to false if the module is in 06perms but the releaser is not associated with it.

So by those definitions both of these "cpanm" files from each dist would get marked true/true.
That's obviously not very helpful, though, because in this case,
while neither is explicitly not indexed (by META), nor not authorized (by 06perms),
they are also not actually indexed (in 02packages) nor actually authorized (in 06perms)...
So I'm still not sure what the right answer is.
If the pod files were indexed:false and authorized:false they wouldn't be searchable on the web site.

I'd love to come up with a clearer definition for these values.
If we can come up with better attributes we could put it in the next version of the API.

[oalders]

After discussing this today, I really do think we need to come up with something better. It was confusing for me to explain and after re-reading this I realize that it's actually just confusing.

[ranguard]

We want to work out WHAT we should do, even if we don't actually do it at MetaHack

[tsibley]

I've pushed some docs on the state of these very confusing fields to the branch tsibley/document-indexer-flags.

I haven't carefully audited queries for the ways these fields are used the search, download_url endpoint, etc, but the fields are used in many queries.