Tested on:
WebdriverIO service for testing VSCode extensions.
This WebdriverIO service allows you to seamlessly test your VSCode extensions from end to end in the VSCode Desktop IDE or as web extension. You only need to provide a path to your extension and the service does the rest by:
- ๐๏ธ Installing VSCode (either
stable
,insiders
or a specified version) - โฌ๏ธ Downloading Chromedriver specific to given VSCode version
- ๐ Enables you to access the VSCode API from your tests
- ๐ฅ๏ธ Starting VSCode with custom user settings (including support for VSCode on Ubuntu, MacOS and Windows)
- ๐ Or serves VSCode from a server to be accessed by any browser for testing web extensions
- ๐ Bootstrapping page objects with locators matching your VSCode version
This project was highly inspired by the vscode-extension-tester project which is based on Selenium. This package takes the idea and adapts it for WebdriverIO.
To initiate a new WebdriverIO project, run:
npm create wdio ./
An installation wizard will guide you through the process. Ensure you select TypeScript as compiler and don't have it generate page objects for you given this project comes with all page objects needed. Then make sure to select chromedriver
and vscode
within the list of services:
Note: remove chromedriver
from the list of services in the generated wdio.conf.ts
afterwards. See also configuration example below.
For more information on how to install WebdriverIO
, please check the project docs.
To use the service you need to add vscode
to your list of services, optionally followed by a configuration object. This will make WebdriverIO download given VSCode binaries and appropiate Chromedriver version:
// wdio.conf.ts
export const config = {
outputDir: 'trace',
// ...
capabilities: [{
browserName: 'vscode',
browserVersion: '1.71.0', // "insiders" or "stable" for latest VSCode version
'wdio:vscodeOptions': {
extensionPath: __dirname,
userSettings: {
"editor.fontSize": 14
}
}
}],
services: ['vscode'],
/**
* optionally you can define the path WebdriverIO stores all
* VSCode and Chromedriver binaries, e.g.:
* services: [['vscode', { cachePath: __dirname }]]
*/
// ...
};
If you define wdio:vscodeOptions
with any other browserName
but vscode
, e.g. chrome
, the service will serve the extension as web extension. If you test on Chrome no additional driver service is required, e.g.:
// wdio.conf.ts
export const config = {
outputDir: 'trace',
// ...
capabilities: [{
browserName: 'chrome',
'wdio:vscodeOptions': {
extensionPath: __dirname
}
}],
services: ['vscode'],
// ...
};
Note: when testing web extensions you can only choose between stable
or insiders
as browserVersion
.
In your tsconfig.json
make sure to add wdio-vscode-service
to your list of types:
{
"compilerOptions": {
"types": [
"node",
"webdriverio/async",
"@wdio/mocha-framework",
"expect-webdriverio",
"wdio-vscode-service"
],
"target": "es2019",
"moduleResolution": "node",
"esModuleInterop": true
}
}
You can then use the getWorkbench
method to access the page objects for the locators matching your desired VSCode version:
describe('WDIO VSCode Service', () => {
it('should be able to load VSCode', async () => {
const workbench = await browser.getWorkbench()
expect(await workbench.getTitleBar().getTitle())
.toBe('[Extension Development Host] - README.md - wdio-vscode-service - Visual Studio Code')
})
})
If you like to execute certain automation through the VSCode API you can do that by running remote commands via the custom executeWorkbench
command. This command allows to remote execute code from your test inside the VSCode environment and enables to access the VSCode API. You can pass arbitrary paramaters into the function which will then be propagated into the function. The vscode
object will be always passed in as first argument following the outer function parameters. Note that you can not access variables outside of the function scoped as the callback is executed remotely. Here is an example:
const workbench = await browser.getWorkbench()
await browser.executeWorkbench((vscode, param1, param2) => {
vscode.window.showInformationMessage(`I am an ${param1} ${param2}!`)
}, 'API', 'call')
const notifs = await workbench.getNotifications()
console.log(await notifs[0].getMessage()) // outputs: "I am an API call!"
For the full page object documentation, check out the docs. You can find various usage examples in this projects test suite.
Through service configuration you can manage the VSCode version as well as user settings for VSCode:
Service options are options needed for the service to setup the test environment. They are a superset of the Chromedriver options which can be applied for this service as well.
Define a cache path to avoid re-downloading all bundles. This is useful for CI/CD to avoid re-downloading VSCode and Chromedriver for every testrun.
Type: string
Default: process.cwd()
In order to run tests through VSCode you have to define vscode
as browserName
. You can specify the VSCode version by providing a browserVersion
capability. Custom VSCode options are then defined within the custom wdio:vscodeOptions
capability. The options are the following:
Path to a local installed VSCode installation. If option is not provided the service will download VSCode based on given browserVersion
(or stable
if not given).
Type: string
Define the directory to the extension you want to test.
Type: string
Define custom user settings to be applied to VSCode.
Type: Record<string, number | string | object | boolean>
Default: {}
Opens VSCode for a specific workspace. If not provided VSCode starts without a workspace opened.
Type: string
Opens VSCode with a specific file opened.
Type: string
Additional start-up arguments as object, e.g.
vscodeArgs: { fooBar: true, 'bar-foo': '/foobar' }
will be passed in as:
--foo-bar --fooBar --bar-foo=/foobar
Type: Record<string, string | boolean>
Default: see constants.ts#L5-L14
If set to true, service logs VSCode output from the extension host and console API.
Type: boolean
Default: false
VSCode API proxy configurations define how WebdriverIO connects to the VSCode workbench to give you access to the VSCode API.
Type: VSCodeProxyOptions
Default:
{
/**
* If set to true, the service tries to establish a connection with the
* VSCode workbench to enable access to the VSCode API
*/
enable: true,
/**
* Port of the WebSocket connection used to connect to the workbench.
* By default set to an available port in your operating system.
*/
// port?: number
/**
* Timeout for connecting to WebSocket inside of VSCode
*/
connectionTimeout: 5000,
/**
* Timeout for command to be executed within VSCode
*/
commandTimeout: 5000
}
You can re-use the components used in this service for your own webview page objects. For that first create a file that defines all your selectors, e.g.:
// e.g. in /test/pageobjects/locators.ts
export const componentA = {
elem: 'form', // component container element
submit: 'button[type="submit"]', // submit button
username: 'input.username', // username input
password: 'input.password' // password input
}
Now you can create a page object as following:
// e.g. in /test/pageobjects/loginForm.ts
import { PageDecorator, IPageDecorator, BasePage } from 'wdio-vscode-service'
import * as locatorMap, { componentA as componentALocators } from './locators'
export interface LoginForm extends IPageDecorator<typeof componentALocators> {}
@PageDecorator(componentALocators)
export class LoginForm extends BasePage<typeof componentALocators, typeof locatorMap> {
/**
* @private locator key to identify locator map (see locators.ts)
*/
public locatorKey = 'componentA' as const
public login (username: string, password: string) {
await this.username$.setValue(username)
await this.password$.setValue(password)
await this.submit$.click()
}
}
Now in your test you can use your page object as follows:
import { LoginForm } from '../pageobjects/loginForm'
import * as locatorMap from '../locators'
// e.g. in /test/specs/example.e2e.ts
describe('my extension', () => {
it('should login', async () => {
const loginForm = new LoginForm(locatorMap)
await loginForm.login('admin', 'test123')
// you can also use page object elements directly via `[selector]$`
// or `[selector]$$`, e.g.:
await loginForm.submit$.click()
// or access locators directly
console.log(loginForm.locators.username)
// outputs: "input.username"
})
})
If you use WebdriverIO with TypeScript make sure to add wdio-vscode-service
to your types
in your tsconfig.json
, e.g.:
{
"compilerOptions": {
"moduleResolution": "node",
"types": [
"webdriverio/async",
"@wdio/mocha-framework",
"expect-webdriverio",
// add this service to your types
"wdio-devtools-service"
],
"target": "es2019"
}
}
The following VS Code extensions use wdio-vscode-service
:
- Marquee (27k downloads)
- Live Server (27.8m downloads)
- DVC Extension for Visual Studio Code (11.2k downloads)
- Nx Console (1.2m downloads)
Before posting a pull request, please run the following:
git clone [email protected]:webdriverio-community/wdio-vscode-service.git
cd wdio-vscode-service
npm install
npm run build
npm run test
(ornpm run ci
)
If you want to learn more about testing VSCode Extensions, check out Christian Bromann's talk at OpenJS World 2022:
For more information on WebdriverIO check out the project homepage.