Skip to content

Instantly share code, notes, and snippets.

@laurentsenta

laurentsenta/notes.md

Created March 10, 2022 13:20
Show Gist options
  • Save laurentsenta/aa7c70a0b83880abe9b6ad0ffd34d488 to your computer and use it in GitHub Desktop.
Save laurentsenta/aa7c70a0b83880abe9b6ad0ffd34d488 to your computer and use it in GitHub Desktop.
Download ZIP
js doc research notes
Raw
notes.md

JS Doc

Meta

Issue:

ipfs/js-ipfs#4021

Ideally we:

Example

https://github.com/nftstorage/nft.storage/blob/main/packages/client/src/lib.js

Note from Hugo Mr Dias:

The end goal was always to use https://typedoc.org/ to gen the docs, we didnt do it because the output wasn't always the best because of legacy cjs code. You can check nft.storage client workflow https://github.com/nftstorage/nft.storage/blob/main/.github/workflows/client.yml#L79 and output https://nftstorage.github.io/nft.storage/client

Structure:

First attempt from the package

building the project then running typedoc fails

in ~/dev/plabs/ipfs/js-ipfs/packages/ipfs-core on test-doc
› npx typedoc src/index.js                                                        (node v16.13) ☺
TypeDoc exiting with unexpected error:
Error: Expected a symbol for node with kind FirstNode at /Users/laurent/dev/plabs/ipfs/js-ipfs/node_modules/ipfs-utils/dist/src/files/glob-source.d.ts:10
    at Context.expectSymbolAtLocation (/Users/laurent/dev/plabs/ipfs/js-ipfs/node_modules/typedoc/dist/lib/converter/context.js:101:19)
    at Object.convert (/Users/laurent/dev/plabs/ipfs/js-ipfs/node_modules/typedoc/dist/lib/converter/types.js:390:32)
    at convertType (/Users/laurent/dev/plabs/ipfs/js-ipfs/node_modules/typedoc/dist/lib/converter/types.js:70:30)
    at /Users/laurent/dev/plabs/ipfs/js-ipfs/node_modules/typedoc/dist/lib/converter/types.js:600:64
    at Array.map (<anonymous>)
    at Object.convert (/Users/laurent/dev/plabs/ipfs/js-ipfs/node_modules/typedoc/dist/lib/converter/types.js:600:50)
    at convertType (/Users/laurent/dev/plabs/ipfs/js-ipfs/node_modules/typedoc/dist/lib/converter/types.js:70:30)
    at Converter.convertType (/Users/laurent/dev/plabs/ipfs/js-ipfs/node_modules/typedoc/dist/lib/converter/converter.js:56:40)
    at Object.convertProperty (/Users/laurent/dev/plabs/ipfs/js-ipfs/node_modules/typedoc/dist/lib/converter/symbols.js:369:41)
    at convertSymbol (/Users/laurent/dev/plabs/ipfs/js-ipfs/node_modules/typedoc/dist/lib/converter/symbols.js:123:88)

Looks like there’s a bug in typedoc we need to fix. It might also be a typescript version issue. (similar issue ‣). Same issue with the command aegir docs

If I remove the problematic line in the node modules, typedoc completes:

typedoc generates a doc with content that is not very useful: IPFS types are defined in another project, ipfs-core-utils and it doesn’t resolve the references apparently (more tweaks might be found here)

https://github.com/ipfs/js-ipfs/blob/1082fce9/packages/ipfs-core/src/index.js#L9

* @typedef {import('ipfs-core-types').IPFS} IPFS

Trying to use the monorepo features

https://typedoc.org/guides/options/#entrypointstrategy

https://github.com/TypeStrong/typedoc/blob/abc6263238fda4619ce7e4bf8662ff1abf559461/README.md#monorepo-with-manually-specified-sub-packages-to-document

With just a dot:

in ~/dev/plabs/ipfs/js-ipfs on test-doc
› ./node_modules/.bin/typedoc --entryPointStrategy packages .                    (node v16.13) ☹
Error: The file /Users/laurent/dev/plabs/ipfs/js-ipfs/packages/interface-ipfs-core/src/index.js does not contain a sourceMappingURL
Error: Could not determine TS entry point for package /Users/laurent/dev/plabs/ipfs/js-ipfs/packages/interface-ipfs-core/package.json

Looking at the code, it tries to rollback from a js build file to a typescript file, which doesn’t works with our stack.

I tried to set typedocMain = ./types/src/index.d.ts

› ./node_modules/.bin/typedoc --entryPointStrategy packages .                    (node v16.13) ☹
a: /Users/laurent/dev/plabs/ipfs/js-ipfs/packages/interface-ipfs-core/package.json
Error: Entry point "/Users/laurent/dev/plabs/ipfs/js-ipfs/packages/interface-ipfs-core/types/src/index.d.ts" does not appear to be built by the tsconfig found at "/Users/laurent/dev/plabs/ipfs/js-ipfs/packages/interface-ipfs-core/tsconfig.json"

with typedocMain = ./src/index.js

(same as first attempt)
Error: Could not determine TS entry point for package /Users/laurent/dev/plabs/ipfs/js-ipfs/packages/interface-ipfs-core/package.json
in ~/dev/plabs/ipfs/js-ipfs on test-doc
› npx typedoc --entryPointStrategy packages ./packages/ipfs-core                 (node v16.13) ☹
Error: The file /Users/laurent/dev/plabs/ipfs/js-ipfs/packages/ipfs-core/src/index.js does not contain a sourceMappingURL
Error: Could not determine TS entry point for package /Users/laurent/dev/plabs/ipfs/js-ipfs/packages/ipfs-core/package.json

So that’s confusing, you can pass typedoc a js file: npx typedoc ./src/index.js and it works,

but in packages mode, it won’t work because it tries to find a .ts source file. And if you pass it a ts file it will complain that the package was not built by the tsconfig.

typedoc on ipfs-core-types

Default typedoc shows a lot of missing values

in ~/dev/plabs/ipfs/js-ipfs/packages/ipfs-core-types on test-doc
› npm run doc                                                                    (node v16.13) ☺

> [email protected] doc
> npx typedoc

Warning: Bases, defined at src/index.ts:53, is referenced by IPFS.bases but not included in the documentation.
Warning: API, defined at src/bitswap/index.ts:4, is referenced by IPFS.bitswap but not included in the documentation.
Warning: API, defined at src/block/index.ts:4, is referenced by IPFS.block but not included in the documentation.
Warning: API, defined at src/bootstrap/index.ts:4, is referenced by IPFS.bootstrap but not included in the documentation.
Warning: Codecs, defined at src/index.ts:58, is referenced by IPFS.codecs but not included in the documentation.
Warning: API, defined at src/config/index.ts:4, is referenced by IPFS.config but not included in the documentation.
Warning: API, defined at src/dag/index.ts:4, is referenced by IPFS.dag but not included in the documentation.
Warning: API, defined at src/dht/index.ts:5, is referenced by IPFS.dht but not included in the documentation.
Warning: API, defined at src/diag/index.ts:3, is referenced by IPFS.diag but not included in the documentation.
Warning: API, defined at src/files/index.ts:6, is referenced by IPFS.files but not included in the documentation.
Warning: Hashers, defined at src/index.ts:63, is referenced by IPFS.hashers but not included in the documentation.
Warning: API, defined at src/key/index.ts:3, is referenced by IPFS.key but not included in the documentation.
Warning: API, defined at src/log/index.ts:3, is referenced by IPFS.log but not included in the documentation.
Warning: API, defined at src/name/index.ts:5, is referenced by IPFS.name but not included in the documentation.
Warning: API, defined at src/object/index.ts:6, is referenced by IPFS.object but not included in the documentation.
Warning: API, defined at src/pin/index.ts:5, is referenced by IPFS.pin but not included in the documentation.
Warning: API, defined at src/pubsub/index.ts:3, is referenced by IPFS.pubsub but not included in the documentation.
Warning: RefsAPI, defined at src/index.ts:25, is referenced by IPFS.refs but not included in the documentation.
Warning: API, defined at src/repo/index.ts:4, is referenced by IPFS.repo but not included in the documentation.
Warning: API, defined at src/stats/index.ts:5, is referenced by IPFS.stats but not included in the documentation.
Warning: API, defined at src/swarm/index.ts:4, is referenced by IPFS.swarm but not included in the documentation.
Warning: AddResult, defined at src/root.ts:236, is referenced by IPFS.add.add but not included in the documentation.
Warning: IDResult, defined at src/root.ts:273, is referenced by IPFS.id.id but not included in the documentation.
Warning: IPFSEntry, defined at src/root.ts:141, is referenced by IPFS.ls.ls but not included in the documentation.
Warning: MountResult, defined at src/root.ts:323, is referenced by IPFS.mount.mount but not included in the documentation.
Warning: PingResult, defined at src/root.ts:307, is referenced by IPFS.ping.ping but not included in the documentation.
Warning: VersionResult, defined at src/root.ts:288, is referenced by IPFS.version.version but not included in the documentation.
Warning: ImportCandidate, defined at src/utils.ts:25, is referenced by IPFS.add.add.entry but not included in the documentation.
Warning: AddOptions, defined at src/root.ts:153, is referenced by IPFS.add.add.options but not included in the documentation.
Warning: ImportCandidateStream, defined at src/utils.ts:21, is referenced by IPFS.addAll.addAll.source but not included in the documentation.
Warning: AddAllOptions, defined at src/root.ts:215, is referenced by IPFS.addAll.addAll.options but not included in the documentation.
Warning: IPFSPath, defined at src/utils.ts:119, is referenced by IPFS.cat.cat.ipfsPath but not included in the documentation.
Warning: CatOptions, defined at src/root.ts:248, is referenced by IPFS.cat.cat.options but not included in the documentation.
Warning: DNSOptions, defined at src/root.ts:299, is referenced by IPFS.dns.dns.options but not included in the documentation.
Warning: GetOptions, defined at src/root.ts:259, is referenced by IPFS.get.get.options but not included in the documentation.
Warning: IDOptions, defined at src/root.ts:269, is referenced by IPFS.id.id.options but not included in the documentation.
Warning: ListOptions, defined at src/root.ts:265, is referenced by IPFS.ls.ls.options but not included in the documentation.
Warning: MountOptions, defined at src/root.ts:318, is referenced by IPFS.mount.mount.options but not included in the documentation.
Warning: PingOptions, defined at src/root.ts:303, is referenced by IPFS.ping.ping.options but not included in the documentation.
Warning: ResolveOptions, defined at src/root.ts:313, is referenced by IPFS.resolve.resolve.options but not included in the documentation.
Warning:
Unsupported highlight language "console" will not be highlighted. Run typedoc --help for a list of supported languages.
target code block :
        $ npm install ipfs-core-types
source files :undefined
output file :
        undefined
Info: Documentation generated at ./docs

As if typedoc doesn’t know how to walk the tree.

maybe related: TypeStrong/typedoc#1771

Trying with file as an entrypoint

trying npx typedoc src/index.ts src/**/*.ts gives something interesting,

I still get these weirds errors of packages not referenced by the documentation:

in ~/dev/plabs/ipfs/js-ipfs/packages/ipfs-core-types on test-doc
› npx typedoc src/index.ts src/root.ts  src/**/*.ts                              (node v16.13) ☺
Warning: ValidStat, defined at src/pin/remote/service/index.ts:55, is referenced by pin/remote/service.Stat but not included in the documentation.
Warning: InvalidStat, defined at src/pin/remote/service/index.ts:60, is referenced by pin/remote/service.Stat but not included in the documentation.
Warning: Bases, defined at src/index.ts:53, is referenced by index.IPFS.bases but not included in the documentation.
Warning: Codecs, defined at src/index.ts:58, is referenced by index.IPFS.codecs but not included in the documentation.
Warning: Hashers, defined at src/index.ts:63, is referenced by index.IPFS.hashers but not included in the documentation.
Warning: RefsAPI, defined at src/index.ts:25, is referenced by index.IPFS.refs but not included in the documentation.
Warning:
Unsupported highlight language "console" will not be highlighted. Run typedoc --help for a list of supported languages.
target code block :
        $ npm install ipfs-core-types
source files :undefined
output file :
        undefined
Info: Documentation generated at ./docs

Despite running typedoc with EVERYTHING as an entrypoint. I tried to use the typedoc.json config, it ignores my entrypoints parameters.

Best I got:

cd ipfs-core-types;
npx typedoc --plugin typedoc-plugin-markdown ./src/index.ts ./src/**/*.ts

(using markdown for generation)

See my comment here: ipfs/js-ipfs#4021 (comment)

JSDoc

I gave jsdoc a shot too, just in case we could get a quick win there,

No success.

The configuration file:

// jsdoc.json
{
  "opts": {
    "template": "./node_modules/better-docs"
  },
  "tags": {
    "allowUnknownTags": ["optional"]
  },
  "plugins": [
    "./node_modules/better-docs/typescript",
    "./node_modules/better-docs/typedef-import"
  ],
  "source": {
    "includePattern": "\\.(jsx|js|ts|tsx)$"
  }
}

The result

in ~/dev/plabs/ipfs/js-ipfs on test-doc
› npx jsdoc ./packages/ipfs-core/src/index.js -c ./jsdoc.json                                                                                                                        (node v16.13) ☹

ERROR: Unable to parse a tag's type expression for source file /Users/laurent/dev/plabs/ipfs/js-ipfs/packages/ipfs-core/src/index.js in line 8 with tag title "typedef" and text "{import('ipfs-core-types').IPFS} IPFS": Invalid type expression "import('ipfs-core-types').IPFS": Expected "!", "=", "?", "[]", "|", or end of input but "(" found.
ERROR: Unable to parse a tag's type expression for source file /Users/laurent/dev/plabs/ipfs/js-ipfs/packages/ipfs-core/src/index.js in line 8 with tag title "typedef" and text "{import('./types').Options} Options": Invalid type expression "import('./types').Options": Expected "!", "=", "?", "[]", "|", or end of input but "(" found.
ERROR: Unable to parse a tag's type expression for source file /Users/laurent/dev/plabs/ipfs/js-ipfs/packages/ipfs-core/src/index.js in line 8 with tag title "typedef" and text "{import('./types').Libp2pFactoryFn} Libp2pFactoryFn": Invalid type expression "import('./types').Libp2pFactoryFn": Expected "!", "=", "?", "[]", "|", or end of input but "(" found.
ERROR: Unable to parse a tag's type expression for source file /Users/laurent/dev/plabs/ipfs/js-ipfs/packages/ipfs-core/src/index.js in line 8 with tag title "typedef" and text "{import('./types').Libp2pFactoryFnArgs} Libp2pFactoryFnArgs": Invalid type expression "import('./types').Libp2pFactoryFnArgs": Expected "!", "=", "?", "[]", "|", or end of input but "(" found.
ERROR: Unable to parse a tag's type expression for source file /Users/laurent/dev/plabs/ipfs/js-ipfs/packages/ipfs-core/src/index.js in line 8 with tag title "typedef" and text "{import('./types').InitOptions} InitOptions": Invalid type expression "import('./types').InitOptions": Expected "!", "=", "?", "[]", "|", or end of input but "(" found.
ERROR: Unable to parse a tag's type expression for source file /Users/laurent/dev/plabs/ipfs/js-ipfs/packages/ipfs-core/src/index.js in line 8 with tag title "typedef" and text "{import('./types').Re
...
ERROR: Unable to parse a tag's type expression for source file /Users/laurent/dev/plabs/ipfs/js-ipfs/packages/ipfs-core/src/index.js in line 25 with tag title "typedef" and text "{import('./types').MfsPreload} MfsPreload": Invalid type expression "import('./types').MfsPreload": Expected "!", "=", "?", "[]", "|", or end of input but "(" found.
ERROR: Unable to parse a tag's type expression for source file /Users/laurent/dev/plabs/ipfs/js-ipfs/packages/ipfs-core/src/index.js in line 25 with tag title "typedef" and text "{import('./types').LoadBaseFn} LoadBaseFn": Invalid type expression "import('./types').LoadBaseFn": Expected "!", "=", "?", "[]", "|", or end of input but "(" found.
ERROR: Unable to parse a tag's type expression for source file /Users/laurent/dev/plabs/ipfs/js-ipfs/packages/ipfs-core/src/index.js in line 25 with tag title "typedef" and text "{import('./types').LoadCodecFn} LoadCodecFn": Invalid type expression "import('./types').LoadCodecFn": Expected "!", "=", "?", "[]", "|", or end of input but "(" found.
ERROR: Unable to parse a tag's type expression for source file /Users/laurent/dev/plabs/ipfs/js-ipfs/packages/ipfs-core/src/index.js in line 25 with tag title "typedef" and text "{import('./types').LoadHasherFn} LoadHasherFn": Invalid type expression "import('./types').LoadHasherFn": Expected "!", "=", "?", "[]", "|", or end of input but "(" found.
ERROR: Unable to parse a tag's type expression for source file /Users/laurent/dev/plabs/ipfs/js-ipfs/packages/ipfs-core/src/index.js in line 25 with tag title "typedef" and text "{import('./types').IPLDOptions} IPLDOptions": Invalid type expression "import('./types').IPLDOptions": Expected "!", "=", "?", "[]", "|", or end of input but "(" found.

Moving forward

https://github.com/ipfs/aegir/blob/master/package.json#L133

FWIW aegir uses typescript 4.4, but I found different versions of typescript in our monorepo

Ideally we could use https://www.npmjs.com/package/typedoc-plugin-monorepo

But we need the entrypointStrategy = packages to pass first.

There are also documentation discussion in multiformats:

multiformats/js-multiformats#47

Note on 2022-2-24:

👋  I took a quick shot at this issue to try to move it forward, without much success yet,

It's not straightforward to point typedoc to `ipfs-core/index.ts` and get a "good looking" documentation,
it looks like typedoc doesn't resolve [references import](https://github.com/ipfs/js-ipfs/blob/6a5ea700/packages/ipfs-core/src/index.js#L9), so all we get are empty API types. (also some of our interfaces seemed to break typedoc's parser)

There is a monorepo configuration with typedoc, but I couldn't get it to work. It looks like our setup confuses typedoc: it tries to load our `index.js` and find its source typescript. Which doesn't exist.

I did manage to generate an equivalent of the current doc by pointing typedoc at the `ipfs-core-types` folder and marking every ts file as an entry point.

- [In Markdown](https://github.com/laurentsenta/js-ipfs/blob/test-doc/packages/ipfs-core-types/docs/md/modules.md)
- [In HTML form](https://test-docs-js-ipfs.netlify.app/interfaces/index.ipfs)
- For comparison:
  - [generated bitswap](https://github.com/laurentsenta/js-ipfs/blob/test-doc/packages/ipfs-core-types/docs/md/interfaces/bitswap.API.md)
  - [original](https://github.com/ipfs/js-ipfs/blob/master/docs/core-api/BITSWAP.md)

I see a few options:

- Invest into typedoc to make it work with our codebase and use it /everywhere/,
- Build our own solution, just the minimum that works with this codebase,
- Look into other tools such as jsdoc with better-docs plugin.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment