On this page

The engine of doc-kit. It loads your configuration, runs generators in dependency order across worker threads, and provides the JSON generators and the parsing stages that every output builds on.

You usually reach it through the doc-kit CLI, which depends on it. Import it yourself to run doc-kit from a script or to write a generator.

npm install --save-dev @doc-kit/cli

Installing the CLI installs this package, and with it the JSON generators. This command writes one JSON document per Markdown file to out/:

npx doc-kit generate -t json -i "docs/**/*.md" -o out

To import @doc-kit/core from your own code, add it to your own dependencies too. Package managers that don't hoist dependencies, such as pnpm, won't let you import it otherwise:

npm install --save-dev @doc-kit/core

Every output comes from a generator, and each generator names the one it reads from with dependsOn. On every run, doc-kit:

  1. Merges the configuration: command-line options override the configuration file, which overrides the defaults.
  2. Resolves each target: a built-in name such as json to the package that provides it, a path to that file, and anything else to a package import.
  3. Loads the targets and every generator they depend on, and runs each one once, as soon as its input is ready. Your sources are parsed once, however many outputs read them, and independent generators run at the same time.
  4. Splits per-file work into chunks of chunkSize items across up to threads worker threads, and frees each result once the last generator reading it has it.

This package provides the JSON targets:

TargetWritesReference
jsonOne JSON document per source file, described by a JSON schemaJSON
json-allEvery json document, bundled in one all.jsonJSON bundle
json-simpleThe API entries in one api-docs.json, for debugging doc-kitDebug JSON

It also provides the pipeline stages. They write nothing, and run whenever a generator you ask for depends on them:

StageWhat it doesReference
astParses the Markdown and MDX files matched by input into syntax treesParsing Markdown
metadataTurns those trees into typed API entries, which most generators readAPI entries
ast-jsParses the .js files matched by input, for api-linksParsing JavaScript

The other generators, and the rest of doc-kit, live in companion packages:

PackageProvides
@doc-kit/cliThe doc-kit command, which runs this package
@doc-kit/generator-reacthtml (the website), section-pages, orama-db (the search index), sitemap, llms-txt, llms-txt-full, and the jsx-ast stage
@node-core/doc-kitman-page, api-links and addon-verify, for building Node.js itself
@doc-kit/typedocA TypeDoc plugin that writes the API of a TypeScript or JSDoc project as doc-kit Markdown

The generators overview describes them all.

This package finds the configuration file (where it looks), merges it with the command-line options and the defaults, and hands each generator its options. The file's top level takes:

export default {
  target: ['json', 'json-all'],
  global: {
    input: ['docs/**/*.md'],
    output: 'out',
  },
  // A generator's own options override the global ones for it
  'json-all': { minify: true },
};
JavaScript
Attributes
target:string | string[]
The generators to run: built-in names, paths, or import specifiers.
global:Object
The options every generator reads, such as input, output, project and repository. See Configuration for all of them.
threads?:number
The most worker threads to use. Default: the number of CPUs, at most 4 (1 on riscv64).
chunkSize?:number
How many items each worker task processes. Default: 10.
extends:string | string[]
Presets to build on: modules whose default export is a configuration. Paths resolve from the configuration file. The file's own options override its presets', and a later preset overrides an earlier one.

Each generator's own options go in a section named after it, such as json or html. A generator takes whatever its section leaves out from global.

The CLI's generate command comes down to two calls, which you can make yourself:

import createGenerator from '@doc-kit/core';
import { setConfig } from '@doc-kit/core/utils/configuration/index.mjs';

// The options of `doc-kit generate`, in camelCase. The configuration file is
// found and merged as on the command line.
const config = await setConfig({
  input: ['docs/**/*.md'],
  target: ['json', 'json-all'],
  output: 'out',
});

const { runGenerators } = createGenerator();

// One result per target, in the order given
const [documents, bundle] = await runGenerators(config);
JavaScript

setConfig() takes the options of doc-kit generate: input, target, output, ignore, threads, chunkSize, version, changelog, gitRef, index, minify, typeMap and configFile. It ignores anything else, so options such as project, repository or a generator's own section go in the configuration file. It also stores the configuration generators read with getConfig(), so a process runs one configuration at a time.

Without an output, the JSON generators write nothing and only return their results.

A generator is a module whose default export describes it, passed to the CLI by its path (-t ./my-generator.mjs). Creating generators walks through writing one, with worker threads, streaming and dependents.

The default export needs:

Attributes
name:string
The generator's name, which is also the name of its configuration section and what getConfig() takes.
generate:Function
generate(input, worker) builds the output from input, the result of its dependsOn (undefined without one). It returns the result, or an async generator yielding arrays of results, which generators reading it receive joined into one array.

It can also have:

Attributes
description:string
What it makes, in one line.
dependsOn:string
The generator it reads from: an import specifier such as @doc-kit/core/metadata, or a built-in name.
dependent:string | string[]
Generators to deliver its output through. It is spliced into each one's pipeline, in front of the first generator there that reads the same dependsOn, and asking for it runs them.
defaultConfiguration:Object | Function
The defaults of its configuration section, or a function taking the user's configuration and returning them.
hasParallelProcessor?:boolean
Whether generate gets a worker, whose worker.stream(items, extra) runs processChunk on worker threads and yields each chunk's results as it completes. Default: false.
processChunk:Function
processChunk(items, indices, extra) processes the items at indices on a worker thread, and returns an array of results. extra is copied to the thread, so it must be cloneable.
markdown:Object
The unified plugins it processes Markdown with, as remarkPlugins, rehypePlugins and recmaPlugins lists, with '...' where the configured plugins go.
ImportProvides
@doc-kit/corecreateGenerator(), the default export, which returns { runGenerators }
@doc-kit/core/<name>Each generator and stage above (json, metadata, …) as the default export, for a generator's dependsOn
@doc-kit/core/utils/configuration/index.mjssetConfig(options), and getConfig(name), the default export: a generator's options, merged with global
@doc-kit/core/utils/file.mjswriteFile() and writeJSON(), which create missing directories
@doc-kit/core/utils/markdown/processor.mjsgetProcessor(name): the unified processor of a generator's markdown
@doc-kit/core/plugins/shiki/highlighter.mjsgetHighlighter(name): the code highlighter of a generator's markdown
@doc-kit/core/logger/index.mjsThe logger, with logger.child(name) for one of your own
@doc-kit/core/generators/typesThe generator types, such as GeneratorMetadata and ParallelWorker, declared as globals

This package lives in the nodejs/doc-kit monorepo. Set it up as the contributing guide describes, then, from this directory:

# Run this package's tests, once or on every change
node --run test
node --run test:watch

# Measure their coverage
node --run test:coverage

# Update the snapshots after changing an output on purpose
node --run test:update-snapshots
Shell

From the repository root, node --run test runs the tests of every package. A change to published behavior needs a changeset: run node --run changeset at the root.

Previous@doc-kit/cliNext@doc-kit/generator-react