Skip to content

Use unique declaration references #192

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 4 commits into from
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .eslintrc.js
Original file line number Diff line number Diff line change
Expand Up @@ -10,5 +10,6 @@ module.exports = {
ignorePatterns: ['temp', 'scip.ts', 'snapshots'],
rules: {
'no-sync': 'off',
'jsdoc/check-indentation': 'off',
},
}
4 changes: 2 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@
"tslint": "tslint -p tsconfig.json --format stylish",
"eslint": "eslint --cache '**/*.ts?(x)'",
"build": "node ./node_modules/typescript/bin/tsc -b .",
"test": "uvu -r ts-node/register --ignore dist",
"update-snapshots": "uvu -r ts-node/register --ignore dist --update-snapshots",
"test": "TS_NODE_TRANSPILE_ONLY=true uvu -r ts-node/register --ignore dist",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added new test-fast and update-snapshots-fast scripts. I prefer to get the type errors in the terminal because typechecking is reasonably fast in this codebase and debugging runtime exceptions is slower than debugging type errors.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good! To clarify the reasoning behind this: while iterating on changes locally, I often have type errors because I want to test some logic but don't want to invest time into making Typescript happy about it because there's no intent to commit it. Type-checking slows down the process of trying different ideas on existing tests.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm +1 to adding these scripts to package.json as long as we keep the typechecking variants as well. For sg/sg, I appreciate that sg start doesn't typecheck because it's a big codebase slowing down the edit/reload/debug feedback loop. In scip-typescript, typechecking has been fast enough for my needs.

"update-snapshots": "TS_NODE_TRANSPILE_ONLY=true uvu -r ts-node/register --ignore dist --update-snapshots",
"prepare": "cd snapshots && yarn && cd input/multi-project && yarn"
},
"repository": {
Expand Down
4 changes: 3 additions & 1 deletion snapshots/input/syntax/src/namespace.d.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
import * as ts from 'typescript'

declare namespace a {
function hello(): string
function hello(): ts.StringLiteral
interface Interface {
hello: string
}
Expand Down
Binary file added snapshots/output/multi-project/index.scip
Binary file not shown.
Binary file added snapshots/output/pure-js/index.scip
Binary file not shown.
1 change: 0 additions & 1 deletion snapshots/output/pure-js/src/main.js
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,6 @@
// documentation ```ts\nvar a: number\n```
print_fib(a)
//^^^^^^^^^ reference pure-js 1.0.0 src/`main.js`/print_fib().
// ^ reference pure-js 1.0.0 src/`main.js`/a.
// ^ reference pure-js 1.0.0 src/`main.js`/a.

function forever() {
Expand Down
Binary file added snapshots/output/react/index.scip
Binary file not shown.
Binary file added snapshots/output/syntax/index.scip
Binary file not shown.
2 changes: 0 additions & 2 deletions snapshots/output/syntax/src/local.ts
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,6 @@
// ^ reference local 8
// ^^ reference local 9
// ^^^^^^ reference typescript 4.8.4 lib/`lib.es5.d.ts`/Array#reduce().
// ^^^^^^ reference typescript 4.8.4 lib/`lib.es5.d.ts`/Array#reduce().
// ^^^^^^ reference typescript 4.8.4 lib/`lib.es5.d.ts`/Array#reduce().
// ^^^^^^^^^^^^^ definition local 16
// documentation ```ts\n(parameter) previousValue: string\n```
// ^^^^^^^^^^^^ definition local 17
Expand Down
12 changes: 9 additions & 3 deletions snapshots/output/syntax/src/namespace.d.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,17 @@
declare namespace a {
import * as ts from 'typescript'
// definition syntax 1.0.0 src/`namespace.d.ts`/
//documentation ```ts\nmodule "namespace.d.ts"\n```
// ^^ reference typescript 4.8.4 lib/`typescript.d.ts`/ts/
// ^^^^^^^^^^^^ reference typescript 4.8.4 lib/`typescript.d.ts`/

declare namespace a {
// ^ definition syntax 1.0.0 src/`namespace.d.ts`/a/
// documentation ```ts\na: typeof a\n```
function hello(): string
function hello(): ts.StringLiteral
// ^^^^^ definition syntax 1.0.0 src/`namespace.d.ts`/a/hello().
// documentation ```ts\nfunction hello(): string\n```
// documentation ```ts\nfunction hello(): StringLiteral\n```
// ^^ reference typescript 4.8.4 lib/`typescript.d.ts`/ts/
// ^^^^^^^^^^^^^ reference typescript 4.8.4 lib/`typescript.d.ts`/ts/StringLiteral#
interface Interface {
// ^^^^^^^^^ definition syntax 1.0.0 src/`namespace.d.ts`/a/Interface#
// documentation ```ts\ninterface Interface\n```
Expand Down
1 change: 0 additions & 1 deletion snapshots/output/syntax/src/structural-type.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@
// ^^^^^^^ reference typescript 4.8.4 lib/`lib.es2015.symbol.wellknown.d.ts`/Promise#
// ^^^^^^^ reference typescript 4.8.4 lib/`lib.es2018.promise.d.ts`/Promise#
// ^^^^^^^ reference typescript 4.8.4 lib/`lib.es2015.promise.d.ts`/PromiseConstructor#resolve().
// ^^^^^^^ reference typescript 4.8.4 lib/`lib.es2015.promise.d.ts`/PromiseConstructor#resolve().
// ^^^^^^ definition syntax 1.0.0 src/`structural-type.ts`/member0:
// documentation ```ts\n(property) member: number\n```
}
Expand Down
104 changes: 72 additions & 32 deletions src/FileIndexer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,21 @@ export class FileIndexer {
if (ts.isIdentifier(node) || ts.isStringLiteralLike(node)) {
const sym = this.getTSSymbolAtLocation(node)
if (sym) {
this.visitSymbolOccurrence(node, sym)
/**
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree it's not super clear why this.declarationName(node.parent) === node identifies a definition. To improve readability, I'd prefer to add a function private isDefinition(...): boolean with a docstring than adding an inline comment like this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

* For example:
*
* const a = 1
* ^ node
* ^ node.parent.name
* ^^^^^ node.parent
*/
const isDefinition = this.declarationName(node.parent) === node

if (isDefinition) {
return this.indexDefinition(node, sym)
}

this.indexReference(node, sym)
}
}
ts.forEachChild(node, node => this.visit(node))
Expand Down Expand Up @@ -97,42 +111,48 @@ export class FileIndexer {
return symbol
}

private visitSymbolOccurrence(node: ts.Node, sym: ts.Symbol): void {
const range = Range.fromNode(node).toLsif()
let role = 0
const isDefinition = this.declarationName(node.parent) === node
if (isDefinition) {
role |= scip.scip.SymbolRole.Definition
private indexDefinition(node: ts.Node, sym: ts.Symbol): void {
const [declaration] = sym?.declarations || []
const scipSymbol = this.scipSymbol(declaration)

if (scipSymbol.isEmpty()) {
return
}
for (const declaration of sym?.declarations || []) {
const scipSymbol = this.scipSymbol(declaration)

if (scipSymbol.isEmpty()) {
// Skip empty symbols
continue
}
const range = Range.fromNode(node).toLsif()

this.document.occurrences.push(
new scip.scip.Occurrence({
range,
symbol: scipSymbol.value,
symbol_roles: scip.scip.SymbolRole.Definition,
})
)

this.addSymbolInformation(node, sym, declaration, scipSymbol)
this.handleShorthandPropertyDefinition(declaration, range)
this.handleObjectBindingPattern(node, range)
}

private indexReference(node: ts.Node, sym: ts.Symbol): void {
const range = Range.fromNode(node).toLsif()

for (const symbol of this.uniqueDeclarationSymbolValues(sym)) {
this.document.occurrences.push(
new scip.scip.Occurrence({
range,
symbol: scipSymbol.value,
symbol_roles: role,
symbol,
symbol_roles: scip.scip.SymbolRole.UnspecifiedSymbolRole,
})
)
if (isDefinition) {
this.addSymbolInformation(node, sym, declaration, scipSymbol)
this.handleShorthandPropertyDefinition(declaration, range)
this.handleObjectBindingPattern(node, range)
// Only emit one symbol for definitions sites, see https://github.com/sourcegraph/lsif-typescript/issues/45
break
}
}
}

/**
* Emits an additional definition occurrence when destructuring an object
* pattern. For example:
* ```
* interface Props { property: number}
* interface Props { property: number }
* const props: Props[] = [{ property: 42 }]
* props.map(({property}) => property) = {a}
* // ^^^^^^^^ references `Props.property` and defines a local parameter `property`
Expand All @@ -148,15 +168,11 @@ export class FileIndexer {
}
const tpe = this.checker.getTypeAtLocation(node.parent.parent)
const property = tpe.getProperty(node.getText())
for (const declaration of property?.declarations || []) {
const scipSymbol = this.scipSymbol(declaration)
if (scipSymbol.isEmpty()) {
continue
}
for (const symbol of this.uniqueDeclarationSymbolValues(property)) {
this.document.occurrences.push(
new scip.scip.Occurrence({
range,
symbol: scipSymbol.value,
symbol,
})
)
}
Expand Down Expand Up @@ -233,6 +249,7 @@ export class FileIndexer {
): scip.scip.Relationship[] {
const relationships: scip.scip.Relationship[] = []
const isAddedSymbol = new Set<string>()

const pushImplementation = (
node: ts.NamedDeclaration,
isReferences: boolean
Expand Down Expand Up @@ -305,7 +322,7 @@ export class FileIndexer {
return undefined
}

private scipSymbol(node: ts.Node): ScipSymbol {
private scipSymbol = (node: ts.Node): ScipSymbol => {
const fromCache: ScipSymbol | undefined =
this.globalSymbolTable.get(node) || this.localSymbolTable.get(node)
if (fromCache) {
Expand Down Expand Up @@ -361,7 +378,7 @@ export class FileIndexer {
return this.scipSymbol(decl)
}
} catch {
// TODO: https://github.com/sourcegraph/lsif-typescript/issues/34
// TODO: https://github.com/sourcegraph/scip-typescript/issues/34
// continue regardless of error, the TypeScript compiler tends to
// trigger stack overflows in getTypeOfSymbolAtLocation and we
// don't know why yet.
Expand All @@ -378,7 +395,11 @@ export class FileIndexer {
return this.cached(node, this.scipSymbol(node.parent))
}

if (ts.isImportSpecifier(node) || ts.isImportClause(node)) {
if (
ts.isImportSpecifier(node) ||
ts.isImportClause(node) ||
ts.isNamespaceImport(node)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this change related to de-duplication? It would be nice to isolate this change and have a diff in the snapshot tests to better understand the exact motivation for this change.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

) {
const tpe = this.checker.getTypeAtLocation(node)
for (const declaration of tpe.symbol?.declarations || []) {
return this.scipSymbol(declaration)
Expand Down Expand Up @@ -608,6 +629,25 @@ export class FileIndexer {

return this.checker.getTypeAtLocation(node)
}

/**
* @param sym ts.Symbol with declarations to process.
* @returns an array of unique non-empty SCIP symbol values extracted
* from ts.Symbol declarations.
*/
private uniqueDeclarationSymbolValues(sym?: ts.Symbol): string[] {
if (!sym?.declarations) {
return []
}

return Array.from(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This expression is fairly expensive considering it's in a very hot loop and that we can achieve a similar results with a single if condition, as is done in
#194

It's very important that we try to keep performance parity with tsc -b and the fewer redundant collections we allocate the better.

new Set(
sym.declarations
.map(declaration => this.scipSymbol(declaration).value)
.filter(Boolean)
)
)
}
}

function isAnonymousContainerOfSymbols(node: ts.Node): boolean {
Expand Down