ImageJ is an open source image processing program designed for scientific multidimensional images. It was originally written in Java and available mainly on desktop computers. We compiled it into WebAssembly and Javascript with a Java compiler called CheerpJ with additional features to connect it with the ImJoy plugin ecosystem.
You can run ImageJ with all mainstream browsers with one click, no installation and no Java runtime environment needed. It also works on mobile devices.
Try ImageJ.JS here: https://ij.imjoy.io
This project is currently under development, please expect frequent changes.
To facilitate the sharing of images, macro, and plugins, ImageJ.JS web app supports loading predefined images, macro or plugin by constructing a URL. The following options are supported:
open
: used to open an image or macro script automatically when the user click the link. For example:https://ij.imjoy.io/?open=https://github.com/imjoy-team/imagej.js/blob/master/src/assets/img/screenshot-1.png
. Experimental feature: it can also be a NGFF zarr image URL, for example: https://ij.imjoy.io/?open=https://uk1s3.embassy.ebi.ac.uk/idr/zarr/v0.1/6001237.zarrrun
: used to directly run a macro script stored under a url, e.g. on Github or Gist. For example:https://ij.imjoy.io/?run=https://gist.github.com/oeway/ab45cc8295efbb0fb5ae1c6f9babd4ac
.plugin
: used to load an ImJoy plugin when the user click the link. For example:https://ij.imjoy.io/?plugin=https://github.com/imjoy-team/imjoy-plugins/blob/master/repository/ImageAnnotator.imjoy.html
spec
: For ImJoy plugins in Python, by default a free compute service MyBinder will be used to run Python plugins, thisspec
is used to specify the specification of MyBinder server (see details in the ImJoy Plugin Development)engine
: For ImJoy plugins in Python, to specify a local Jupyter notebook server as the ImJoy plugin engine (see details ImJoy Plugin Development) If you want to use two optionsopen
andrun
, then use&
to connect them:https://ij.imjoy.io/?open=https://github.com/imjoy-team/imagej.js/blob/master/src/assets/img/screenshot-1.png&run=https://gist.github.com/oeway/ab45cc8295efbb0fb5ae1c6f9babd4ac
You can also use multple times the same option, for example open multiple open
, you just need to connect them with &
as the above example.
Note for the URL: not every url can be loaded into ImageJ.JS, it must be starts with https
and the site should have CORS enabled. In general you can load files from Github repository, Gist and Zenodo. Typically, Github or Gist is suitable for storing macro files or small test images, if you have large images Zenodo is recommended.
For small script file, one can directly encode the content into the URL (so you don't need to upload it to a server). The easiest way to use this feature is to open the script/macro editor, then click "Share -> Share via URL" in the menu. You will get a long encoded string followed after https://ij.imjoy/?open=
.
As a tip, you can use URL shorten services such as tiny.cc to make a very short URL. This will make the url more readable and also fit in a tweet message for example.
Another tip is, if you want to run the script directly, simply change open
to run
in the url.
If you are a developer, this is implemented by decompress a long string (i.e. file name + content) encoded with a javascript library named lz-string. If you want to generate a valid string, you need to first make a json object with the file name and content: {"name": <FILE_NAME>, "content": <FILE_CONTENT>}
then convert it a string and compress it via LZString.compressToEncodedURIComponent
. Finally, add the compressed string after https://ij.imjoy/?open=
or https://ij.imjoy/?run=
.
Besides Javascript, you can find lz-string
implementation in other programming languages here.
If you use ImageJ.JS in your project, it is recommended to add one of our ImageJ.JS badge to your project repository (e.g. on Github) or website. We have two official badges: and
. A typical use case is to use the badges with the custom ImageJ.JS URL in a Github repo.
For starting ImageJ.JS (e.g. with preloaded macro or image), you can use the badge.
Markdown:
[![launch ImageJ.JS](https://ij.imjoy.io/assets/badge/launch-imagej-js-badge.svg)](https://ij.imjoy.io)
reStructuredText:
.. image:: https://ij.imjoy.io/assets/badge/launch-imagej-js-badge.svg
:target: https://ij.imjoy.io
For opening images or macro, you can use the badge.
Markdown:
[![open in ImageJ.JS](https://ij.imjoy.io/assets/badge/open-in-imagej-js-badge.svg)](https://ij.imjoy.io/?open=YOUR_IMAGE_URL)
reStructuredText:
.. image:: https://ij.imjoy.io/assets/badge/open-in-imagej-js-badge.svg
:target: https://ij.imjoy.io/?open=YOUR_IMAGE_URL
If you made a ImageJ macro that you want to share, you can store them in your project repo on Github, or simply go to Gist(https://gist.github.com), paste it and get the URL. For example, we stored a demo macro here: https://gist.github.com/oeway/ab45cc8295efbb0fb5ae1c6f9babd4ac. Note: set the file extension as .ijm
.
Then wen can construct an URL for sharing where user can directly click and run: https://ij.imjoy.io/?run=https://gist.github.com/oeway/ab45cc8295efbb0fb5ae1c6f9babd4ac . This URL can be shared as is, or further shorten by one of the short URL service (e.g. tiny.cc).
Currently, we don't support install plugin downloaded locally, all the plugines need to be installed from the ImageJ.JS site.
To install a new ImageJ plugin, please make an issue with the following information:
- name of the plugin
- what does the plugin do and how to use it
- a publicly accessible url to the jar file, preferably a link to github release
NOTE: Only ImageJ-1 plugins are supported for now.
ImageJ.JS supports two-way integration with ImJoy, meaning you can either use it as an ImJoy plugin or load other ImJoy plugins into ImageJ.JS. This a a powerful combination, since it brings useful features from ImageJ including file IO, image processing plugins, macro scripting together with web plugins in ImJoy for building easy to use modern UI and powerful deep learning libraries.
In the standalone mode (simply go to https://ij.imjoy.io), you will have acess to ImJoy features via the ImJoy icon located in the top-left corner of ImageJ. You can load ImJoy plugins into the workspace via a plugin URL, especially for those plugin that calls ImageJ.JS.
For example, the following code shows how to call ImageJ.JS api from an ImJoy plugin:
const ij = await api.getWindow("ImageJ.JS")
...
// assuming we have an input image to be processed
await ij.viewImage(inputImage)
await ij.runMacro("...my ijm macro for image processing...")
const result = await ij.getImage()
You can use also ImageJ.JS as an ImJoy plugin by using its url:
const ij = await api.createWindow({src: "https://ij.imjoy.io"})
// assuming we have an input image to be processed
await ij.viewImage(inputImage)
await ij.runMacro("...my ijm macro for image processing...")
const result = await ij.getImage()
The most common use case is to use it with Python, e.g. in a Jupyter notebook or in ImJoy.
Load an image into ImageJ.JS. For example, you can pass images from Python with numpy arrays or raw bytes in other image file formats (e.g PNG). The shape can have the following format:
- [height, width]
- [height, width, 1]
- [height, width, 3] (will show as RGB image)
- [height, width, z-stack]
- [z-stack, height, width, 1]
- [z-stack, height, width, 3] (will show as a stack of RGB image)
config
is optional for numpy array, you can use it to specify file name. For example: {"name": "my image"}
import imageio
image = imageio.imread("https://images.proteinatlas.org/115/672_E2_1_blue_red_green.jpg")
ij = await api.createWindow(src="https://ij.imjoy.io")
await ij.viewImage(image)
If you pass raw bytes of an image in other formats, you need to specify the file name with the corresponding file extension. For example: ij.viewImage(image_bytes, {"name": "my_image.png"})
.
This function is in experimental state
Load a virtual stack that provide data in a lazy fashion. The input argument img
should contain the following properties:
- name: String, the name of the virtual stack image
- dtype: String, the data type of the virtual stack image, should be one of the following:
"uint8"
,"uint16"
or"float32"
. - width: Integer, width of the virtual stack image
- height: Integer, height of the virtual stack image
- nSlices: Integer, the total number of slides in the virtual stack image
- getSlice: Function, a function that takes an index (
Integer
) as input and return the anArrayBuffer
(for Javascript) orbytes
(for Python) with the specified image plane.
If successful, it will return a virtual stack ID, with which you can close the virutal stack via closeVirtualStack(ID)
.
An example in Javascript can be found here, this plugin uses constructs a virtual stack which can be used to dynamically pull data from the HPA cell atlas, you can also see it in action here (After loading click "HPACellAtlasImages" menu under the ImJoy icon).
Note: this function can only support 3D image stack, if you want to load 4D or 5D images, you can run the Stack to Hyperstack
macro to convert it into a Hyperstack, for example:
await ij.runMacro(`run("Stack to Hyperstack...", "order=xyzct channels=4 slices=30 frames=10 display=Grayscale");`)
This function is in experimental state
Close the virtual stack by its ID.
This function is in experimental state
Show a zarr image stored with NGFF format. The input argument config
is an object contains the following field:
- source: an URL or a valid Zarr Group object
- name: name of the image
- offsetX: starting position for x axis (used to display a small portion of the image plane)
- sizeX: the size for x axis (used to display a small portion of the image plane)
- offsetY: starting position for y axis (used to display a small portion of the image plane)
- sizeY: the size for y axis (used to display a small portion of the image plane)
For example:
const ij = await api.getWindow("ImageJ.JS")
await ij.viewZarr({source: "https://uk1s3.embassy.ebi.ac.uk/idr/zarr/v0.1/6001241.zarr", name: '6001241'})
A Zarr Group object can be constructed in either Javascript or Python, for example:
from imjoy import api
import zarr
from fsspec.implementations.http import HTTPFileSystem
from imjoy_rpc import register_default_codecs
register_default_codecs()
fs = HTTPFileSystem()
class Plugin:
async def setup(self):
pass
async def run(self, ctx):
ij = await api.createWindow(
type="ImageJ.JS", src="https://ij.imjoy.io"
)
http_map = fs.get_mapper('https://uk1s3.embassy.ebi.ac.uk/idr/zarr/v0.1/6001241.zarr')
z_group = zarr.open(http_map, mode='r')
await ij.viewZarr({"source": z_group})
api.export(Plugin())
For more Python examples, you can also take a look at the vizarr repo. For more example NGFF images, you can get them from here: https://www.openmicroscopy.org/2020/11/04/zarr-data.html.
Note: This function uses Virtual Stack in ImageJ to display, this means it will load the image plane as a whole -- as a result, it doesn't support loading image with large width and height (e.g. much less than 50000 pixels on each dimension ). You will need to use offsetX
, offsetY
, sizeX
, sizeY
to crop the image plane.
Get the current image (current slice for a stack), for example, in Python you can get it as numpy array by setting the format to "ndarray".
ij = await api.createWindow(src="https://ij.imjoy.io")
img = await ij.getImage("ndarray")
Optionally, you can specify it as "tiff", "png", "jpeg", "gif", "zip", "raw", "avi", "bmp", "fits", "pgm", "text image", "lut", "selection", "measurements", "xy Coordinates" or "text".
For a stack, format
can be set as an object with keys: channel
, slice
, frame
(the values are one-based indexes), otherwise it will return the data of the current slice. If you want to get whole image with all the dimensions, pass {"all": true}
as format
.
The following example plugin shows how to get a stack of image and visualize it in the itk-vtk-viewer:
class ImJoyPlugin {
async setup() {
const ij = await api.getWindow('ImageJ.JS')
await ij.runMacro('run("Fly Brain");')
}
async run(ctx) {
const ij = await api.getWindow('ImageJ.JS')
const img = await ij.getImage({format: 'ndarray', all: true})
const viewer = await api.createWindow({src: "https://kitware.github.io/itk-vtk-viewer/app/"})
await viewer.setImage(img)
}
}
api.export(new ImJoyPlugin())
Return the dimensions of the image as an array of [width
, height
, nChannels
, nSlices
, nFrames
].
Select the current window by its title.
Get the ROI selection in the current image.
The returned roi is a encoded byte string, to parse it, you can use https://github.com/hadim/read-roi/.
Add a menu item to the Plugins
menu. You can pass the following arguments:
label
: The label of the menu itemcallback
: The callback function for the menu itemgroup
: Name of the menu group, the default group isPlugins
position
: A index number for the insert position within the menu group, for submenu items, you can pass a string with numbers separated by.
. For example:position="3.2"
means the second sub-menu of the third menu item.
ij = await api.createWindow(src="https://ij.imjoy.io")
def sayHello():
api.alert('hello')
img = await ij.addMenuItem({"label": "Say Hello", "callback": sayHello})
Run macro string with an optional arguments.
ij = await api.createWindow(src="https://ij.imjoy.io")
await ij.runMacro("close();")
Install a macro (string).
ij = await api.createWindow(src="https://ij.imjoy.io")
myMacro = open("my_macro_to_install.ijm", "r").read()
await ij.installMacro(myMacro)
Install a macro tool (string).
ij = await api.createWindow(src="https://ij.imjoy.io")
myMacroTool = open("my_macro_tool_to_install.ijm", "r").read()
await ij.installTool(myMacroTool)
Run plugin by its class name with an optional arguments.
Open an image.
Note, you can also pass an optional path, but since ImageJ.JS won't be able to access your file system, so most cases, you can just skip the path
and it will popup a dialog to ask the user to select.
Save the current image.
Tutorial: https://imjoy.io/docs/#/i2k_tutorial
ImageJ.JS supports all the ImJoy plugin types, including those written in Python.
The Python plugin type can be either web-python
or native-python
. If you choose web-python
, then Pyodide will be used to run Python directly in the browser (no server needed).
?> You can specify python packages via requirements
(e.g. {"type": "web-python", "requirements": ["numpy", "pandas"]}
). However, Pyodide only support limited number of libraries, including numpy
, scipy
, pandas
, scikit-learn
, pillow
etc.
For accessing more Python libraries, you need to use another type of python which is native-python
. In order to run it, ImJoy will need to connect to a Jupyter notebook server (JupyterHub or BinderHub). By default, it will automatically connect to MyBinder.org which is a free online Jupyter service.
?> MyBinder works by building a Docker image based on a specified Github Repo with specification files (e.g. requirements.txt
for pip, or environment.yml
for conda). The default spec is oeway/imjoy-binder-image/master
, and you can change by pass a URL parameter named spec
to the ImJoy docs (e.g. https://ij.imjoy.io/?spec=oeway/pyimagej-binder-image/master
)
?> If you want to use your own Jupyter notebook server, you can 1) start your notebook server and obtain the URL with token 2) Pass another URL parameters with your notebook URL as engine
, for example: https://ij.imjoy.io/?engine=http://127.0.0.1:8080/?token=fad30034546630efk3923l304s3134o20d2592a3f060f3795
.
We did a preliminary benchmark to check the performance, and it seems ImageJ.JS is ~3x slower than the native.
We ran the following macro on both ImageJ.JS and the native Java version on MacOS.
run("AuPbSn 40");
for(i=0;i<5000;i++){
run("Smooth");
}
print("done");
ImageJ.JS took 15s, ImageJ took 5s.
However, running plugins can be much slower (e.g. 6x or more) since it uses on-the-fly with a simpler compiler.
run("AuPbSn 40");
for(i=0; i<60; i++){
run("Fast Filters", "link filter=mean x=5 y=5 preprocessing=none offset=128");
}
print("done")
ImageJ.JS takes 14.20s, ImageJ takes 2.40s.
We are currently investigating ways to compile plugins in advance to boost the performance.
Before start, please make sure you have nodejs installed.
git clone https://github.com/imjoy-team/imagej.js
cd imagej.js
npm install
npm run build # files will be saved into ./dist folder
sh get-imagej.sh # fetch precompiled imagej
npm run serve
Now you can go to http://127.0.0.1:9090/ to visit your local site.
If you make any changes to the code, you might need to reload the page to see them.
If you have an ImageJ-1 plugin and you want to use it with ImageJ.JS, you can compile it yourself. Here are the steps:
-
get ImageJ.JS running locally by following the steps in Run the ImageJ.JS site locally. Make sure you have the
ij-*.jar
file andplugins
folder under the folder nameddist/ij153
. -
download and install cheerpj compiler
Change the following path to your actual path of cheerpj:
# set CHEERPJ_DIR export CHEERPJ_DIR=/path/to/your/cheerpj/installation
-
we can place the plugin jar file under
dist/ij153/plugins
and compile it with cheerpj. Here we can take MorphoLibJ plugin as an example:cd dist/ij153 # download MorphoLibJ curl https://github.com/ijpb/MorphoLibJ/releases/download/v1.4.2.1/MorphoLibJ_-1.4.2.1.jar -LO mv MorphoLibJ_-1.4.2.1.jar plugins/MorphoLibJ_-1.4.2.1.jar # compile MorphoLibJ ${CHEERPJ_DIR}/cheerpjfy.py --deps=ij.jar plugins/MorphoLibJ_-1.4.2.1.jar # extract plugins.config jar xf plugins/MorphoLibJ_-1.4.2.1.jar plugins.config mv plugins.config MorphoLibJ_-1.4.2.1.jar.config
Now you should get
MorphoLibJ_-1.4.2.1.jar
,MorphoLibJ_-1.4.2.1.jar.js
andMorphoLibJ_-1.4.2.1.jar.config
in sidedist/ij153/plugins
-
Now. we need to update the text file
dist/ij153/plugins/index.list
, and add a new line with the jar file name (i.e.:MorphoLibJ_-1.4.2.1.jar
). With this file, ImageJ.JS will know there is a new plugin.
Finally, you can go to http://127.0.0.1:9090/ and see if the plugin works (make sure you started the server via npm run serve
). In case it doesn't work, you can use the Chrome dev tool to check errors in the console.
The above step runs get-imagej.sh
to obtain the precompiled imagej with plugins. You can compile them locally by following these:
Before start, you need to:
-
download and install maven
-
download and install cheerpj compiler
Change the following path to your actual path of cheerpj:
# set CHEERPJ_DIR export CHEERPJ_DIR=/path/to/your/cheerpj/installation
-
compile imagej with maven
git clone https://github.com/imjoy-team/ImageJA.JS cd ImageJA.JS # compile imagej mvn install:install-file -Dfile=${CHEERPJ_DIR}/cheerpj-dom.jar -DgroupId=com.learningtech -DartifactId=cheerpj-dom -Dversion=1.0 -Dpackaging=jar mvn -Pdeps package
-
compile imagej jar into js
cd target mv ij-1.53m.jar ij.jar # compile ij.jar and we should get ${CHEERPJ_DIR}/cheerpjfy.py ij.jar
The compiled files can be placed under imagej.js/dist folder, and used by your local imagej.js site.
Note: you may need to adjust the imagej jar file name to keep consistent with the one defined in index.js
.
ImageJ.JS is a part of the ImJoy project, please consider citing the ImJoy paper on Nature Methods (https://www.nature.com/articles/s41592-019-0627-0, free access: https://rdcu.be/bYbGO ):
Ouyang, W., Mueller, F., Hjelmare, M. et al. ImJoy: an open-source computational platform for the deep learning era. Nat Methods (2019) doi:10.1038/s41592-019-0627-0