Skip to content

API Extractor and TypeScript guide for ts-node users #222

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
116 changes: 116 additions & 0 deletions websites/api-extractor.com/docs/pages/guides/invoke-tsc.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
---
title: Invoking TypeScript Before API Extractor
---

Suppose you prefer to work in TypeScript files, and you use ts-node to run your TypeScript files. Great! Now you want to use API Extractor and API Documenter to generate documentation.

There's just one catch. API Extractor inputs are _type declaration_ files. (`*.d.ts`.) You may write some type declaration files, but the whole point of ts-node is to _not emit_ compiled files when you don't need to. Now you need to.

The overall approach is:

1. Copy a configuration file into your source directory.
2. Use TypeScript to emit declaration files into an ignored snapshot directory.
3. Copy type declaration files from your source directory into the snapshot directory.
4. Run API Extractor against the snapshot.

## Configuring TypeScript

You will need a TypeScript configuration file specifically for emitting type declaration files. Here's a quick sample:

```json
// typings-tsconfig.json
{
"compilerOptions": {
"lib": ["ESNext"],
"module": "es2022",
"target": "esnext",
"moduleResolution": "node",

"declaration": true,
"emitDeclarationOnly": true,
"outDir": "../typings-snapshot/source",
"newLine": "lf"
},

"extends": "@tsconfig/node18/tsconfig.json"
}
```

Since you're using ts-node already, you certainly have one of these files already (`tsconfig.json`), but it probably isn't configured the same. The key properties for this purpose above are `declaration`, `emitDeclarationOnly`, and `outDir`. These guarantee a specific output location, for only type declaration files.

Speaking of the output directory (which is `snapshotDir` below), yes, you need one. It should be a directory your version control system ignores (using `.gitignore` or `.hgignore` files, most likely). It should also _not_ be a directory which already exists. This directory is a transient one to feed to API Extractor.

Save this configuration file in a directory you don't ignore. We're going to copy it into the source directory.

## Emitting type declaration files

As a prerequisite, you have to know where the TypeScript compiler resides on your file system. If it's a developer dependency, you'll probably find it at `node_modules/typescript/bin/tsc` from your project's root directory.

Invoking it takes a few steps:

```typescript
import fs from 'fs/promises';
import { fork } from 'child_process';
import path from 'path';
import url from 'url';

declare const projectDir: string; // your actual project root, with the node_modules directory as a child
declare const sourceDir: string; // your TypeScript sources file
declare const snapshotDir: string; // this is your temporary output directory

const tsconfigFile = path.join(url.fileURLToPath(import.meta.url), '../typings-tsconfig.json');
const tsconfigSourceFile = path.join(sourceDir, 'typings-tsconfig.json');

await fs.rm(snapshotDir, { force: true, recursive: true });
await fs.mkdir(snapshotDir);

await fs.copyFile(tsconfigFile, tsconfigSourceFile);

const pathToTSC = path.join(projectDir, `node_modules/typescript/bin/tsc`);
const parameters = ['--project', tsconfigSourceFile];

try {
// set up a promise to resolve or reject when tsc exits
type PromiseResolver<T> = (value: T | PromiseLike<T>) => unknown;
type PromiseRejecter = (reason?: unknown) => unknown;

let resolve: PromiseResolver<void>, reject: PromiseRejecter;
const tscPromise = new Promise((res, rej) => {
resolve = res;
reject = rej;
});

// run the TypeScript compiler!
const tsc = fork(pathToTSC, parameters, {
silent: false,
// this ensures you can see TypeScript error messages
stdio: ['ignore', 'inherit', 'inherit', 'ipc']
});
tsc.on('exit', (code) => (code ? reject(code) : resolve()));

await tscPromise;
} finally {
// clean up
await fs.rm(tsconfigSourceFile);
}
```

## Copying existing type declaration files

You're not done yet. TypeScript converted your `**.ts` files to `**.d.ts`, but it didn't migrate your _existing_ type declaration files. You need to do that yourself.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds like the observation here is that tsc normally does not copy .d.ts files from src to lib, when it is transpiling src/*.ts to produce lib/*.d.ts. Normally such .d.ts files should either be stored in a relative folder (e.g. ../typings/*.d.ts) or else the toolchain can copy them. By "toolchain", I mean that mature projects generally do not build using a shell command like tsc && eslint && jest && api-extractor -- instead they will use a standard CLI that can intelligently chain these tasks, cache the results, and support interactive watch mode. (The Rush Stack toolchain is Heft.)

By contrast, the ts-node is aimed at lightweight scripting scenarios, where the user wants to execute .ts files directly without any build process or configuration. Instead of running node script.js, you can run ts-node script.ts that merely adds basic type checking and nothing else.

But as you observed, this simplistic approach causes trouble when trying to integrate with other stages like API Extractor. When such requirements arise, the typical solution would be to switch to using a proper toolchain. The code snippets here are basically starting down that path -- this script invokes tsc, then copies over the .d.ts files, then invokes API Extractor. (Oh and it probably should copy .json files as well. And oh it probably needs a way to clean them. Which means the CLI needs a --clean parameter, so really we should have --help, etc. Wait a second, why not just use a real toolchain again? 😄 )

It feels like it's drifting away from the premise of ts-node, which to avoid a build process. Thus I wonder if it might be more fruitful to implement a simple NPM package like ts-node-api-extractor that combines these ideas into a simple command line. It would:

  • Read the real *tsconfig.json
  • In memory, magically add the declaration, emitDeclarationOnly, and outDir settings
  • Then invoke tsc to produce *.d.ts files in a temporary directory
  • Copy over the *.d.ts inputs
  • Then invoke API Extractor on the output

Basically just package up all this code so that a person who uses ts-node can generate their .d.ts rollup or .api.md report without having to configure anything. What do you think?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait a second, why not just use a real toolchain again? 😄

Because I built this incrementally and slapped API Extractor in later? 😁 But your point is extremely well-taken. It could be a form of technical debt I've built up in my project. At some point, maybe I should do that conversion and this PR should be turned away - or into a "here's why you should convert to a toolchain" article.

I have no problem throwing away code or ideas that are less efficient.

Basically just package up all this code so that a person who uses ts-node can generate their .d.ts rollup or .api.md report without having to configure anything. What do you think?

Two catches. One, I had to do some massaging of the generated type definition files before I could feed it to API Extractor. So such a package would need to include a hook script option for that purpose.

Two, and I really believed this would've been fixed by now, ts-node has stopped working out-of-the-box on NodeJS 18.19+. Various people have suggested five to seven solutions, again and again. Most of those solutions are "move off of ts-node, here's a similar project that works". It's really quite disheartening. This is another reason I'm rethinking the home-grown build system I've already got in place in my project. I had looked at both Grunt and Gulp, and turned them down - one for vulnerabilities, one for ESM incompatibility.

I've been on a bit of a break from my project since I released version 1.0 a couple weeks ago.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well stepping back and thinking about this, I suppose my hesitation with this doc is that it includes enough interesting code that people will want to complete the code sample, and code review it, and improve it over time, which is difficult to do in an .md page. If you don't have time to maintain a new GitHub project yourself, another idea would be to contribute it to the Rush Stack repo as a rushstack-samples or build test project. (We'd also be onboard with building something like this directly into API Extractor, a mode where it invokes tsc automatically into a temp folder to produce the .d.ts inputs -- but that is probably an even bigger time commitment. 😄)

Anyway you can think about the best way to go. There's no hurry.


```typescript
let files = await fs.readdir(sourceDir, { encoding: 'utf-8', recursive: true });
files = files.filter((f) => f.endsWith('.d.ts'));

await Promise.all(
files.map(async (file) => {
await fs.mkdir(path.dirname(path.join(snapshotDir, file)), { recursive: true });
await fs.copyFile(path.join(sourceDir, file), path.join(snapshotDir, file));
})
);
```

## Now you can use API Extractor

Make sure your `projectFolder` in `api-extractor.jsonc` points to the snapshot directory you've just created. Then you're good to go, configuring API Extractor as you need.
6 changes: 6 additions & 0 deletions websites/api-extractor.com/sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,12 @@ const sidebars = {
'pages/setup/help'
]
},
{
type: 'category',
label: 'Guides',
collapsible: false,
items: ['pages/guides/invoke-tsc']
},
{
type: 'category',
label: 'Doc comment syntax',
Expand Down