select with refs no longer infers fields after typescript 3.8

[Naddiseo]

Environment

Knex version: 0.19.5 / 0.21.2
Database + version: postgresql 11
OS: Ubuntu 20.04

Select applicable template from below.
If issue is about oracledb support, tag @ atiertant. For MSSql tag @ smorey2.
Rest of dialects don't need tags.
If issue is about TypeScript definitions, tag @lorefnon.

Bug

1. Explain what kind of behaviour you are getting and how you think it should do
   When using select([knex.ref('id'), knex.ref('field')]) the return type no longer contains all fields starting with typescript 3.8, and will only contain a single field

2. Error message

• None, typescript propblem

3. Reduced test code, for example in https://npm.runkit.com/knex or if it needs real
   database connection to MySQL or PostgreSQL, then single file example which initializes
   needed data and demonstrates the problem.

import Knex from 'knex';
interface Table {
    id: string
   field_a: string
}
const knex = Knex({ client: 'pg', connection: 'postgresql://user:pass@db:5432/db' });

const KTable = knex<Table>('table');
// expected type = `{ "id": string, "field_a": string };
/*
starting with typescript 3.8, the actual return type can be `{"id": string}` or `{"field_a": string}` depending on how it gets passed to `select()`
*/
const result = await KTable.select([knef.ref('id'), knef.ref('field_a')]);

Feature discussion / request

1. Explain what is your use case

• Using the same way as the docs suggest, but breaks as of TS3.8

2. Explain what kind of feature would support this

3. Give some API proposal, how the feature should work

I'm not sure if this is a TS bug, or a knex bug, but since it works in multiple versions of knex, but stopped working in TS3.8, I'm thinking it could be that TS changed something. I had a look at the typings, and it could be that the UnionToIntersection utility is no longer working in TS3.8 and would need updating?

[Naddiseo]

This can sort of be reduced to:

import Knex from knex;
const knex = Knex();
// should be (Knex.Ref<'id', {id:'id'}> | Knex.Ref<'field_a', {field_a:'field_a'}>)[]
// but is Knex.Ref<'id', {id:'id'}>[]
const fields = [knef.ref('id'), knef.ref('field_a')]; 

This leads me to think it's a change in TS that needs to be accounted for? Or is it a bug in TS? The strange thing is, if I try to reproduce in the TS playground with minimal representations of knex types, I can get it to work:

https://www.typescriptlang.org/play/index.html#code/MYGwhgzhAEBKCmAzAPAFQMoCdjXgDwBd4A7AExnQMwEtiBzAGmlQFkwAHd2u3Qk86AG8AvgD4hAKGjRMSAFwZsU6MAD2xCFQCuwAqswAKWYjnMswAJSTp0ggAtqEAHTHoAXhlIA3MuES-ErREmIhgwPDQ6PAg8LpoCGqYpLxEZDAi7tBgxACeTKgIEFogBG5axADWxKoA7sQA2gC64iISgcTBoeHQAIpa8Jg5AJIdA13w8bH6yfipAhke2XnMhcUEmUstbX6gkDB9A8OjIWETBVNJKfzpwhu5+asldzlb-m0Su1DQANLE+JOJGZ8NJCW6Le4reBFJ4ecpVWoNZpXEEHQYjTqnAHTB5QtbiayeFCKHCza7QTQ0eiiAwQbCmYkWUwIInmJiCerfaC0MzYRr08xiISeAhaTDEaB-GpwJA07AWLzQPzCHztDHdVBgABGMQJ1FIpgp3GUxDAAFt4HJDfQ3h91JpoABHTWZSU-P54NBamLU+VtNQadaIajRAQeepOlwygDkeqjFiYEeMBijJvNccaXiAA

[lorefnon]

@Naddiseo Thanks for bringing this to our attention, and for creating the reproducer.

Unfortunately it is quite easy to come across such edge cases in TS, and a few of my issues in TS repo have been open for months.

In this particular case, it seems like differently parameterised Ref<*> types are getting treated as same because the type parameters of the interface are not actually being used in any of the members of the interface. I have introduced a phantom property to mark them as used, and that appears to resolve the problem here.

I am not entirely sure why this doesn't happen in the playground, but that too is not the first time this would have happened.

[Naddiseo]

@lorefnon, I've done a bisect on typescript to see where/why this started breaking, and it leads to this PR: microsoft/TypeScript#36261 ( commit microsoft/TypeScript@1a10e71 ) which is supposed to fix microsoft/TypeScript#35805

I'm not sure if that helps. The type variance stuff is slightly beyond me, but it seems to be changing how it measures if two types are the same (like you're saying) and not looking at the nested parameters the same way. So, I'm not sure if this needs to be reported in TS as a regression or not? (I'd be happy to report it if you want)

This is how I've been testing it:

import { expectType } from 'tsd';
import Knex from 'knex';

const knex = Knex({});
interface Table {
    id: number
    name: string
}

(async () => {
    const T = knex<Table>();
    type expectedType = { id: number, name: string };

    const res = await T.select([knex.ref('id'), knex.ref('name')]);
    expectType<expectedType[]>(res);
})();

[lorefnon]

Oh this is interesting. I had come across the invariance of Fn<*, ...> before, and its good that they fixed it. But this seems like an unintentional side effect of that.

Let me try to come up with a minimal reproducer for that and file a bug with them. If you can come up with a good isolated example, please feel free to report.

[Naddiseo]

Another interesting datapoint:

const res = await T.select(   knex.ref('id'), knex.ref('name')   ); // WORKS
const res = await T.select( [ knex.ref('id'), knex.ref('name') ] ); // DOES NOT WORK

This seems to point to being a bug in the way TS infers array types?

[Naddiseo]

A slightly different test case, using chained selects works:

const res1 = await T.select(   knex.ref('id')   ).select(   knex.ref('name')    ); // works
const res2 = await T.select( [ knex.ref('id') ] ).select( [ knex.ref('name') ]  ); // works

This is again seeming to indicate there's something wrong with array handling. I'm still working on getting a better minimal test case, but knowing to look at the array side is helping narrow it down

[kibertoad]

Released in 0.21.3