👋 I’ve been trying to use this project on vanilla JavaScript code and can’t get the example from the README to work as expected due to shell expansion issues (I think). Here is my project’s structure, which is just the output of npx create-react-app my-app with the src folder copied at different levels of nesting:

$ tree .
.
├── README.md
├── dump.lsif
├── package-lock.json
├── package.json
├── src
│   ├── App.css
│   ├── App.js
│   ├── App.test.js
│   ├── copy
│   │   ├── App.css
│   │   ├── App.js
│   │   ├── App.test.js
│   │   ├── copy
│   │   │   ├── App.css
│   │   │   ├── App.js
│   │   │   ├── App.test.js
│   │   │   ├── index.css
│   │   │   ├── index.js
│   │   │   ├── logo.svg
│   │   │   ├── serviceWorker.js
│   │   │   └── setupTests.js
│   │   ├── index.css
│   │   ├── index.js
│   │   ├── logo.svg
│   │   ├── serviceWorker.js
│   │   └── setupTests.js
│   ├── index.css
│   ├── index.js
│   ├── logo.svg
│   ├── serviceWorker.js
│   └── setupTests.js
└── yarn.lock

This has 15 .js files. The first issue I have is that when I get the indexer to work, there are 17 files indexed. It’s unclear to me whether this is because the .json files are also processed, or something else.

This also fails in zsh when my project contains a node_modules folder:

lsif-tsc **/*.js --AllowJs --checkJs
zsh: argument list too long: lsif-tsc

I assume this is because this relies on shell expansion of the glob pattern. Not sure whether this is a feature or a bug.

Trying to bypass shell expansion doesn’t work either:

lsif-tsc '**/*.js' --AllowJs --checkJs

0 file(s), 0 symbol(s)
Processed in 0.003s

Switching to bash instead of zsh, the shell expansion doesn’t support arbitrarily-deep folder structures:

lsif-tsc **/*.js --AllowJs --checkJs
.....
7 file(s), 113 symbol(s)
Processed in 0.326s

Practically I would want to run this in CI, so can probably figure out which shell this should run inside of with which pattern, but it would be nicer for this to be more portable. And the pattern in the README should "just work" IMHO.

Packages like fast-glob / globby should solve this by making the glob patterns portable across different shells. Then there’s the problem of whether the indexer is meant to go through node_modules, or auto-magically ignore it, or whether there should be a way to supply ignore patterns. Or whether it’s a bad idea to use **/*.js as documented, and users should be told to use more specific patterns.

Running yarn install gives:

warning " > @sourcegraph/[email protected]" has incorrect peer dependency "tslint@^5.15.0".
warning "@sourcegraph/tslint-config > [email protected]" has incorrect peer dependency "tslint@^5.1.0".

I'm not entirely sure if this is a problem (and maybe it needs to be fixed in tslint-config?) but I figured I'd at least mention it.

Sample code: const foo = Symbol("foo")

When I hover on foo in a js file in vscode I get const foo: typeof foo. When I put the same code in a TS file I get const foo: unique symbol. On Sourcegraph.com, I always get the JS type, even if I'm browsing a TS file.

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Error type: undefined. Note: this is a nested preset so please contact the preset author if you are unable to fix it yourself.

When generating an LSIF index for lodash we get the following illegal URIs:

{"id":115885,"type":"vertex","label":"document","uri":"file:///test/findLastIndex-and-lastIndexOf.test.js","languageId":"typescript"}

Index was generated with lsif-tsc *.js test/*.js --AllowJs --checkJs.

Since version 4.1 of TypeScript React 17 JSX factories are supported. To do this, the key "jsx" must be set to "react-jsx" in "tsconfig.json".

https://devblogs.microsoft.com/typescript/announcing-typescript-4-1/#jsx-factories
https://www.typescriptlang.org/tsconfig#jsx

Unfortunately, lsif-tsc throws the following error

error TS6046: Argument for '--jsx' option must be: 'preserve', 'react-native', 'react'.
    at loadConfigFile (/usr/local/lib/node_modules/@sourcegraph/lsif-tsc/dist/src/main.js:85:15)
    at processProject (/usr/local/lib/node_modules/@sourcegraph/lsif-tsc/dist/src/main.js:110:18)
    at run (/usr/local/lib/node_modules/@sourcegraph/lsif-tsc/dist/src/main.js:259:11)
    at Object.<anonymous> (/usr/local/lib/node_modules/@sourcegraph/lsif-tsc/dist/src/main.js:265:1)
    at Module._compile (internal/modules/cjs/loader.js:1201:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1221:10)
    at Module.load (internal/modules/cjs/loader.js:1050:32)
    at Function.Module._load (internal/modules/cjs/loader.js:938:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
    at internal/main/run_main_module.js:17:47
```

For golang there is an example: https://docs.gitlab.com/ee/user/project/code_intelligence.html#configuration
But none for node, I supposed that it will follow the same syntax but looks like it's not.

Here is what I did:

code_navigation:
 image: sourcegraph/lsif-node
 allow_failure: true # recommended
 script:
   - lsif-node
 artifacts:
   reports:
     lsif: dump.lsif

But it looks like the script is taking arguments in that case: search, api, repos...

What should I do to reproduce what golang is doing?

The latest available @sourcegraph/lsif-tsc on NPM is v0.6.6 which depends on https://github.com/dbaeumer/TypeScript/releases/download/3.8.3-lsif/typescript-3.8.3-lsif.tgz

The latter seems to have been removed from github for some reason and so we can no longer npm install @sourcegraph/lsif-tsc.
Can you publish a newer version of @sourcegraph/lsif-tsc from the master branch since I see that the dbaeumer/TypeScript has been updated to one that's available?

You might also want to decouple from dbaeumer/TypeScript if they keep on deleting old releases.

A customer reported performance issues with lsif-node.

Key points from the discussion:

• The project involved 4 tsconfig files, each of which had a ~700M LSIF dump, and indexing took in total about 400 seconds on a laptop.
• It wasn't clear to the customer how they could/should profile the performance.
• It wasn't clear to the customer how to parallelize builds (and whether it was safe to do so).
• The customer noted that some colleagues pointed out that the compiler was relatively slow, so the time taken by indexing was "not really unexpected" but she said that it would be nice to speed it up.
• The customer mentioned that if indexing took over 20 minutes, the index may be invalid by the time it is uploaded.

Happens to some of my lsif-tsc generations. Either due to an infinite loop or it goes way too deep.

....RangeError: Maximum call stack size exceeded
at checkPropertyAccessExpressionOrQualifiedName (/Users/william.chau/.config/yarn/global/node_modules/typescript-lsif/lib/typescript.js:71051:132)
at checkPropertyAccessExpression (/Users/william.chau/.config/yarn/global/node_modules/typescript-lsif/lib/typescript.js:70862:17)
at checkExpressionWorker (/Users/william.chau/.config/yarn/global/node_modules/typescript-lsif/lib/typescript.js:76001:28)
at checkExpression (/Users/william.chau/.config/yarn/global/node_modules/typescript-lsif/lib/typescript.js:75918:38)
at getTypeOfExpression (/Users/william.chau/.config/yarn/global/node_modules/typescript-lsif/lib/typescript.js:75859:24)
at getMatchingUnionConstituentForObjectLiteral (/Users/william.chau/.config/yarn/global/node_modules/typescript-lsif/lib/typescript.js:66116:40)
at discriminateContextualTypeByObjectMembers (/Users/william.chau/.config/yarn/global/node_modules/typescript-lsif/lib/typescript.js:69270:20)
at getApparentTypeOfContextualType (/Users/william.chau/.config/yarn/global/node_modules/typescript-lsif/lib/typescript.js:69284:105)
at getContextualTypeForObjectLiteralElement (/Users/william.chau/.config/yarn/global/node_modules/typescript-lsif/lib/typescript.js:69174:24)
at getContextualType (/Users/william.chau/.config/yarn/global/node_modules/typescript-lsif/lib/typescript.js:69380:28)

Another example:

RangeError: Maximum call stack size exceeded
at Object.createSymbolKey (/Users/william.chau/.config/yarn/global/node_modules/@sourcegraph/lsif-tsc/dist/src/typescripts.js:156:25)
at Symbols.findBaseMembers (/Users/william.chau/.config/yarn/global/node_modules/@sourcegraph/lsif-tsc/dist/src/lsif.js:787:23)
at Symbols.findBaseMembers (/Users/william.chau/.config/yarn/global/node_modules/@sourcegraph/lsif-tsc/dist/src/lsif.js:811:47)
at Symbols.findBaseMembers (/Users/william.chau/.config/yarn/global/node_modules/@sourcegraph/lsif-tsc/dist/src/lsif.js:811:47)
at Symbols.findBaseMembers (/Users/william.chau/.config/yarn/global/node_modules/@sourcegraph/lsif-tsc/dist/src/lsif.js:811:47)
at Symbols.findBaseMembers (/Users/william.chau/.config/yarn/global/node_modules/@sourcegraph/lsif-tsc/dist/src/lsif.js:811:47)
at Symbols.findBaseMembers (/Users/william.chau/.config/yarn/global/node_modules/@sourcegraph/lsif-tsc/dist/src/lsif.js:811:47)
at Symbols.findBaseMembers (/Users/william.chau/.config/yarn/global/node_modules/@sourcegraph/lsif-tsc/dist/src/lsif.js:811:47)
at Symbols.findBaseMembers (/Users/william.chau/.config/yarn/global/node_modules/@sourcegraph/lsif-tsc/dist/src/lsif.js:811:47)
at Symbols.findBaseMembers (/Users/william.chau/.config/yarn/global/node_modules/@sourcegraph/lsif-tsc/dist/src/lsif.js:811:47)

In the following code, references to the query variable aren't tracked correctly. Using "Find references" on the query variable doesn't return the object in the console.log statement.

export function foo(test: string) {
  const query = 'test'
  console.log({ query })
}

Reproduction example

View the code on Sourcegraph and see that query within the console.log statement isn't shown in references.

hey, I realized that v0.7.0 was recently released: https://github.com/sourcegraph/lsif-node/releases/tag/v0.7.0

but I can't see it on Docker Hub: https://hub.docker.com/r/sourcegraph/lsif-node/tags?page=1&ordering=last_updated

we're using the docker image to run this in our CI, so it would be great to have access to the latest version!

Currently, we don't automatically publish new releases to Docker Hub on git tag push, causing issues like #163. We should add GitHub Actions workflows to automate this step so that Docker Hub is always in sync with what is published to npm.

Copy of sourcegraph/sourcegraph#14272

https://github.com/sourcegraph/sourcegraph/pull/14256/files#diff-389012daf6ffbfb95a148bde9e6800bbL145

Go to definition on LinkOrButton, it just jumps to the import in the same file, instead of to https://sourcegraph.com/github.com/sourcegraph/sourcegraph@b2cd5ddf5a013b12df9691c670d8644dcf26a936/-/blob/shared/src/components/LinkOrButton.tsx#L46

User from https://app.hubspot.com/contacts/2762526/company/436902646

There is a mix of Typescript and Javascript in this repo.
I’m using the lsif-tsc tool to generate the LSIF file for the repository. I followed this docbut I’ve encountered several problems:
• When indexing JS files over the whole repo, the given command (lsif-tsc **/*.js --allowJs --checkJs) fails because the number of JS files is greater than the bash limit for arguments
• Cross references between TS and JS does not work
• JSX file seems not to be indexed

I'm trying to use lsif-node to generate index files for my Js project.
The command I executed was lsif-tsc **/*.js --AllowJs --checkJs. The *.js files were successfully indexed but *.jsx files were not indexed and some problems could be found in the dump.lsif.

{
    "id":281904,
    "type":"vertex",
    "label":"diagnosticResult",
    "result":[
        {
            "severity":1,
            "code":6142,
            "message":"Module './Header.jsx' was resolved to '/xxx/Header.jsx', but '--jsx' is not set.",
            "range":{
                "start":{
                    "line":4,
                    "character":19
                },
                "end":{
                    "line":4,
                    "character":33
                }
            }
        },
        
        {
            "severity":1,
            "code":17004,
            "message":"Cannot use JSX unless the '--jsx' flag is provided.",
            "range":{
                "start":{
                    "line":31,
                    "character":6
                },
                "end":{
                    "line":31,
                    "character":35
                }
            }
        }
       
    ]
}

I tried to use lsif-tsc **/*.js* --AllowJs --jsx --checkJs but it didn't work. I was wondering how to index a Js repository with both js and jsx files analysed.

This package (which is a fork of microsoft/lsif-node) depends on a fork of the typescript compiler maintained by the microsoft/lsif-node author.

We should eventually either:

• maintain our own fork of that fork, since right now we're only half decoupled from their implementation, OR
• merge our changes to lsif-node back into the original so we don't have to maintain it any more

We use composite projects in our main repo (references in tsconfig.json).
The LSIF exporter fails on these with

No input files specified

Repro: sourcegraph/sourcegraph#11467

I can't get any codeintel in https://github.com/sourcegraph/sourcegraph/pull/15099/files#diff-d0f2ec764d1c50118c2c5465271cb246c8d06830456e42eab5db0936476cf089, even on local variables. I see LSIF requests being made by the browser extension, but they come back with an empty definition (but with hover content). All LSIF checks passed for the PR.

Copied from: sourcegraph/sourcegraph#15185

Jump to definition doesn't work in many cases, e.g.

https://sourcegraph.com/github.com/sourcegraph/sourcegraph@2f5873684d268d60d2d4c46e0961918a526262a2/-/blob/client/browser/src/shared/platform/settings.ts#L16:10

This could be related to the destination being under a different TypeScript project (tsconfig.json), we use project references.

Dependent projects appear to be indexed but references to files outside of the current project's root are not accessible within the dump (nor are monikers generated as its not a reference to an external library).

Copy of #15150

VS Code:

https://github.com/sourcegraph/sourcegraph/pull/15131/files#diff-d0f2ec764d1c50118c2c5465271cb246c8d06830456e42eab5db0936476cf089L293

When generating an index for moment we run into the following exception:

Error: Shouldn't be called
    at AliasedSymbolData.getOrCreateReferenceResult (/usr/local/lib/node_modules/@sourcegraph/lsif-tsc/dist/src/lsif.js:462:15)
    at SymbolDataPartition.end (/usr/local/lib/node_modules/@sourcegraph/lsif-tsc/dist/src/lsif.js:623:51)
    at AliasedSymbolData.nodeProcessed (/usr/local/lib/node_modules/@sourcegraph/lsif-tsc/dist/src/lsif.js:405:23)
    at DataManager.nodeProcessed (/usr/local/lib/node_modules/@sourcegraph/lsif-tsc/dist/src/lsif.js:1224:32)
    at Visitor.doVisit (/usr/local/lib/node_modules/@sourcegraph/lsif-tsc/dist/src/lsif.js:1397:26)
    at Visitor.visit (/usr/local/lib/node_modules/@sourcegraph/lsif-tsc/dist/src/lsif.js:1348:22)
    at Visitor.visitProgram (/usr/local/lib/node_modules/@sourcegraph/lsif-tsc/dist/src/lsif.js:1335:18)
    at Object.lsif (/usr/local/lib/node_modules/@sourcegraph/lsif-tsc/dist/src/lsif.js:1970:26)
    at processProject (/usr/local/lib/node_modules/@sourcegraph/lsif-tsc/dist/src/main.js:161:19)
    at run (/usr/local/lib/node_modules/@sourcegraph/lsif-tsc/dist/src/main.js:222:11)

Index was generated with lsif-tsc **/*.js --AllowJs --checkJs.

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• chore(deps): update dependency ts-node to ^8.10.2
• chore(deps): update dependency typescript to ^3.9.10
• fix(deps): update dependency minimist to v1.2.6 [security]
• fix(deps): update dependency npm to ^6.14.17
• fix(deps): update dependency uuid to ^7.0.3 (uuid, @types/uuid)
• fix(deps): update dependency vscode-uri to ^2.1.2
• chore(deps): update dependency @sourcegraph/eslint-config to v0.31.0
• chore(deps): update dependency sourcegraph/src-cli to v3.42.0
• chore(deps): update node.js
• chore(deps): update node.js to >=v10.24.1
• fix(deps): update dependency lsif-protocol to v0.5.3
• chore(deps): update actions/checkout action to v3
• chore(deps): update actions/setup-go action to v3
• chore(deps): update actions/setup-node action to v3
• chore(deps): update dependency @types/npm to v7
• chore(deps): update dependency eslint to v8
• chore(deps): update dependency ts-node to v10
• chore(deps): update dependency typescript to v4
• chore(deps): update docker/build-push-action action to v3
• chore(deps): update docker/login-action action to v2
• chore(deps): update docker/setup-buildx-action action to v2
• chore(deps): update node.js to v15
• chore(deps): update node.js to v16 (node, @types/node)
• chore(deps): update node.js to v18
• fix(deps): update dependency lru-cache to v7 (lru-cache, @types/lru-cache)
• fix(deps): update dependency npm to v8
• fix(deps): update dependency uuid to v8 (uuid, @types/uuid)
• fix(deps): update dependency vscode-uri to v3
• Click on this checkbox to rebase all open PRs at once

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

I get a TS5023: Unknown compiler option 'noUncheckedIndexedAccess' error message when I use the current Docker image.

The code can be found here: https://gitlab.com/anti-tracking/ClearURLs/core/-/tree/dev
The CI output here: https://gitlab.com/anti-tracking/ClearURLs/core/-/jobs/1313500016

$ lsif-tsc -p .
Error: error TS5023: Unknown compiler option 'noUncheckedIndexedAccess'.
    at loadConfigFile (/usr/local/lib/node_modules/@sourcegraph/lsif-tsc/dist/src/main.js:85:15)
    at processProject (/usr/local/lib/node_modules/@sourcegraph/lsif-tsc/dist/src/main.js:110:18)
    at run (/usr/local/lib/node_modules/@sourcegraph/lsif-tsc/dist/src/main.js:259:11)
    at Object.<anonymous> (/usr/local/lib/node_modules/@sourcegraph/lsif-tsc/dist/src/main.js:265:1)
    at Module._compile (internal/modules/cjs/loader.js:1201:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1221:10)
    at Module.load (internal/modules/cjs/loader.js:1050:32)
    at Function.Module._load (internal/modules/cjs/loader.js:938:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
    at internal/main/run_main_module.js:17:47

In the past, we've run into OOMs with 2GB of RAM in CI. For example: https://github.com/sourcegraph/sourcegraph/runs/4479534778?check_suite_focus=true

(copied relevant part of the log below in case the log gets deleted)

Run yarn eslint -f lsif
yarn run v1.22.4
$ eslint --cache '**/*.[tj]s?(x)' -f lsif

<--- Last few GCs --->

[328:0x55eb601eb760]   480668 ms: Scavenge (reduce) 2031.1 (2051.5) -> 2030.1 (2051.5) MB, 1.9 / 0.0 ms  (average mu = 0.389, current mu = 0.503) allocation failure 
[328:0x55eb601eb760]   480677 ms: Scavenge (reduce) 2031.2 (2051.5) -> 2030.2 (2051.5) MB, 2.0 / 0.0 ms  (average mu = 0.389, current mu = 0.503) allocation failure 
[328:0x55eb601eb760]   480682 ms: Scavenge (reduce) 2031.2 (2051.5) -> 2030.3 (2051.8) MB, 2.1 / 0.0 ms  (average mu = 0.389, current mu = 0.503) allocation failure 


<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
error Command failed with signal "SIGABRT".
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
Error: Process completed with exit code 1.

Explore moniker generation and determine why import monikers are not specified where expected.