Skip to content

Use unique declaration references #192

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 4 commits into from
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .eslintrc.js
Original file line number Diff line number Diff line change
@@ -10,5 +10,6 @@ module.exports = {
ignorePatterns: ['temp', 'scip.ts', 'snapshots'],
rules: {
'no-sync': 'off',
'jsdoc/check-indentation': 'off',
},
}
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -2,4 +2,4 @@ dist
node_modules
.eslintcache
yarn-error.log
snapshots/output/**/*.lsif-typed
snapshots/output/**/*.scip
4 changes: 2 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
@@ -11,8 +11,8 @@
"tslint": "tslint -p tsconfig.json --format stylish",
"eslint": "eslint --cache '**/*.ts?(x)'",
"build": "node ./node_modules/typescript/bin/tsc -b .",
"test": "uvu -r ts-node/register --ignore dist",
"update-snapshots": "uvu -r ts-node/register --ignore dist --update-snapshots",
"test": "TS_NODE_TRANSPILE_ONLY=true uvu -r ts-node/register --ignore dist",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added new test-fast and update-snapshots-fast scripts. I prefer to get the type errors in the terminal because typechecking is reasonably fast in this codebase and debugging runtime exceptions is slower than debugging type errors.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good! To clarify the reasoning behind this: while iterating on changes locally, I often have type errors because I want to test some logic but don't want to invest time into making Typescript happy about it because there's no intent to commit it. Type-checking slows down the process of trying different ideas on existing tests.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm +1 to adding these scripts to package.json as long as we keep the typechecking variants as well. For sg/sg, I appreciate that sg start doesn't typecheck because it's a big codebase slowing down the edit/reload/debug feedback loop. In scip-typescript, typechecking has been fast enough for my needs.

"update-snapshots": "TS_NODE_TRANSPILE_ONLY=true uvu -r ts-node/register --ignore dist --update-snapshots",
"prepare": "cd snapshots && yarn && cd input/multi-project && yarn"
},
"repository": {
4 changes: 3 additions & 1 deletion snapshots/input/syntax/src/namespace.d.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
import * as ts from 'typescript'

declare namespace a {
function hello(): string
function hello(): ts.StringLiteral
interface Interface {
hello: string
}
1 change: 0 additions & 1 deletion snapshots/output/pure-js/src/main.js
Original file line number Diff line number Diff line change
@@ -67,7 +67,6 @@
// documentation ```ts\nvar a: number\n```
print_fib(a)
//^^^^^^^^^ reference pure-js 1.0.0 src/`main.js`/print_fib().
// ^ reference pure-js 1.0.0 src/`main.js`/a.
// ^ reference pure-js 1.0.0 src/`main.js`/a.

function forever() {
2 changes: 0 additions & 2 deletions snapshots/output/syntax/src/local.ts
Original file line number Diff line number Diff line change
@@ -37,8 +37,6 @@
// ^ reference local 8
// ^^ reference local 9
// ^^^^^^ reference typescript 4.8.4 lib/`lib.es5.d.ts`/Array#reduce().
// ^^^^^^ reference typescript 4.8.4 lib/`lib.es5.d.ts`/Array#reduce().
// ^^^^^^ reference typescript 4.8.4 lib/`lib.es5.d.ts`/Array#reduce().
// ^^^^^^^^^^^^^ definition local 16
// documentation ```ts\n(parameter) previousValue: string\n```
// ^^^^^^^^^^^^ definition local 17
12 changes: 9 additions & 3 deletions snapshots/output/syntax/src/namespace.d.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,17 @@
declare namespace a {
import * as ts from 'typescript'
// definition syntax 1.0.0 src/`namespace.d.ts`/
//documentation ```ts\nmodule "namespace.d.ts"\n```
// ^^ reference typescript 4.8.4 lib/`typescript.d.ts`/ts/
// ^^^^^^^^^^^^ reference typescript 4.8.4 lib/`typescript.d.ts`/

declare namespace a {
// ^ definition syntax 1.0.0 src/`namespace.d.ts`/a/
// documentation ```ts\na: typeof a\n```
function hello(): string
function hello(): ts.StringLiteral
// ^^^^^ definition syntax 1.0.0 src/`namespace.d.ts`/a/hello().
// documentation ```ts\nfunction hello(): string\n```
// documentation ```ts\nfunction hello(): StringLiteral\n```
// ^^ reference typescript 4.8.4 lib/`typescript.d.ts`/ts/
// ^^^^^^^^^^^^^ reference typescript 4.8.4 lib/`typescript.d.ts`/ts/StringLiteral#
interface Interface {
// ^^^^^^^^^ definition syntax 1.0.0 src/`namespace.d.ts`/a/Interface#
// documentation ```ts\ninterface Interface\n```
1 change: 0 additions & 1 deletion snapshots/output/syntax/src/structural-type.ts
Original file line number Diff line number Diff line change
@@ -17,7 +17,6 @@
// ^^^^^^^ reference typescript 4.8.4 lib/`lib.es2015.symbol.wellknown.d.ts`/Promise#
// ^^^^^^^ reference typescript 4.8.4 lib/`lib.es2018.promise.d.ts`/Promise#
// ^^^^^^^ reference typescript 4.8.4 lib/`lib.es2015.promise.d.ts`/PromiseConstructor#resolve().
// ^^^^^^^ reference typescript 4.8.4 lib/`lib.es2015.promise.d.ts`/PromiseConstructor#resolve().
// ^^^^^^ definition syntax 1.0.0 src/`structural-type.ts`/member0:
// documentation ```ts\n(property) member: number\n```
}
104 changes: 72 additions & 32 deletions src/FileIndexer.ts
Original file line number Diff line number Diff line change
@@ -65,7 +65,21 @@ export class FileIndexer {
if (ts.isIdentifier(node) || ts.isStringLiteralLike(node)) {
const sym = this.getTSSymbolAtLocation(node)
if (sym) {
this.visitSymbolOccurrence(node, sym)
/**
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree it's not super clear why this.declarationName(node.parent) === node identifies a definition. To improve readability, I'd prefer to add a function private isDefinition(...): boolean with a docstring than adding an inline comment like this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

* For example:
*
* const a = 1
* ^ node
* ^ node.parent.name
* ^^^^^ node.parent
*/
const isDefinition = this.declarationName(node.parent) === node

if (isDefinition) {
return this.visitDefinition(node, sym)
}

this.visitReference(node, sym)
}
}
ts.forEachChild(node, node => this.visit(node))
@@ -97,42 +111,48 @@ export class FileIndexer {
return symbol
}

private visitSymbolOccurrence(node: ts.Node, sym: ts.Symbol): void {
const range = Range.fromNode(node).toLsif()
let role = 0
const isDefinition = this.declarationName(node.parent) === node
if (isDefinition) {
role |= scip.scip.SymbolRole.Definition
private visitDefinition(node: ts.Node, sym: ts.Symbol): void {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not convinced this refactoring is an improvement in maintainability. In the future, I want to add more symbol roles such as import and write and then the symbol_roles bitmask will need to be updated in two different places.

const [declaration] = sym?.declarations || []
const scipSymbol = this.scipSymbol(declaration)

if (scipSymbol.isEmpty()) {
return
}
for (const declaration of sym?.declarations || []) {
const scipSymbol = this.scipSymbol(declaration)

if (scipSymbol.isEmpty()) {
// Skip empty symbols
continue
}
const range = Range.fromNode(node).toLsif()

this.document.occurrences.push(
new scip.scip.Occurrence({
range,
symbol: scipSymbol.value,
symbol_roles: scip.scip.SymbolRole.Definition,
})
)

this.addSymbolInformation(node, sym, declaration, scipSymbol)
this.handleShorthandPropertyDefinition(declaration, range)
this.handleObjectBindingPattern(node, range)
}

private visitReference(node: ts.Node, sym: ts.Symbol): void {
const range = Range.fromNode(node).toLsif()

for (const symbol of this.uniqueDeclarationSymbolValues(sym)) {
this.document.occurrences.push(
new scip.scip.Occurrence({
range,
symbol: scipSymbol.value,
symbol_roles: role,
symbol,
symbol_roles: scip.scip.SymbolRole.UnspecifiedSymbolRole,
})
)
if (isDefinition) {
this.addSymbolInformation(node, sym, declaration, scipSymbol)
this.handleShorthandPropertyDefinition(declaration, range)
this.handleObjectBindingPattern(node, range)
// Only emit one symbol for definitions sites, see https://github.com/sourcegraph/lsif-typescript/issues/45
break
}
}
}

/**
* Emits an additional definition occurrence when destructuring an object
* pattern. For example:
* ```
* interface Props { property: number}
* interface Props { property: number }
* const props: Props[] = [{ property: 42 }]
* props.map(({property}) => property) = {a}
* // ^^^^^^^^ references `Props.property` and defines a local parameter `property`
@@ -148,15 +168,11 @@ export class FileIndexer {
}
const tpe = this.checker.getTypeAtLocation(node.parent.parent)
const property = tpe.getProperty(node.getText())
for (const declaration of property?.declarations || []) {
const scipSymbol = this.scipSymbol(declaration)
if (scipSymbol.isEmpty()) {
continue
}
for (const symbol of this.uniqueDeclarationSymbolValues(property)) {
this.document.occurrences.push(
new scip.scip.Occurrence({
range,
symbol: scipSymbol.value,
symbol,
})
)
}
@@ -233,6 +249,7 @@ export class FileIndexer {
): scip.scip.Relationship[] {
const relationships: scip.scip.Relationship[] = []
const isAddedSymbol = new Set<string>()

const pushImplementation = (
node: ts.NamedDeclaration,
isReferences: boolean
@@ -305,7 +322,7 @@ export class FileIndexer {
return undefined
}

private scipSymbol(node: ts.Node): ScipSymbol {
private scipSymbol = (node: ts.Node): ScipSymbol => {
const fromCache: ScipSymbol | undefined =
this.globalSymbolTable.get(node) || this.localSymbolTable.get(node)
if (fromCache) {
@@ -361,7 +378,7 @@ export class FileIndexer {
return this.scipSymbol(decl)
}
} catch {
// TODO: https://github.com/sourcegraph/lsif-typescript/issues/34
// TODO: https://github.com/sourcegraph/scip-typescript/issues/34
// continue regardless of error, the TypeScript compiler tends to
// trigger stack overflows in getTypeOfSymbolAtLocation and we
// don't know why yet.
@@ -378,7 +395,11 @@ export class FileIndexer {
return this.cached(node, this.scipSymbol(node.parent))
}

if (ts.isImportSpecifier(node) || ts.isImportClause(node)) {
if (
ts.isImportSpecifier(node) ||
ts.isImportClause(node) ||
ts.isNamespaceImport(node)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this change related to de-duplication? It would be nice to isolate this change and have a diff in the snapshot tests to better understand the exact motivation for this change.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

) {
const tpe = this.checker.getTypeAtLocation(node)
for (const declaration of tpe.symbol?.declarations || []) {
return this.scipSymbol(declaration)
@@ -608,6 +629,25 @@ export class FileIndexer {

return this.checker.getTypeAtLocation(node)
}

/**
* @param sym ts.Symbol with declarations to process.
* @returns an array of unique non-empty SCIP symbol values extracted
* from ts.Symbol declarations.
*/
private uniqueDeclarationSymbolValues(sym?: ts.Symbol): string[] {
if (!sym?.declarations) {
return []
}

return Array.from(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This expression is fairly expensive considering it's in a very hot loop and that we can achieve a similar results with a single if condition, as is done in
#194

It's very important that we try to keep performance parity with tsc -b and the fewer redundant collections we allocate the better.

new Set(
sym.declarations
.map(declaration => this.scipSymbol(declaration).value)
.filter(Boolean)
)
)
}
}

function isAnonymousContainerOfSymbols(node: ts.Node): boolean {