This is an open source mirror of Redwood Research's transformer interpretability tools. This is all the code behind our UI at our demo website, as well as our interpretability-specific transformer implementation and gradient query framework.
- You need access to pretrained models (or train them yourself), and OpenWebText data. You can download our pretrained models (and OpenAI models ported into our codebase) fro s3. To download these from S3 you need the S3 cli, but not an AWS account:
Our code looks for models in the path specified by the environment variable
# Download all available models locally > mkdir ~/rr-models/ > aws s3 cp s3://rrserve/interp-assets/models/ --no-sign-request ~/rr-models/ --recursive # List available models > aws s3 ls s3://rrserve/interp-assets/ --no-sign-request --recursive
INTERPRETABILITY_MODELS_DIR
for models andINTERP_DATA_DIR
for data, so setINTERPRETABILITY_MODELS_DIR
to~/rr-models
or something. pip install -r requirements.txt
- If you see an error about attrs, do
pip uninstall attrs; pip install attrs
- Go to /interp
python local_dev.py
- In a seperate terminal, go to interp/app
npm run start
- Your browser should open to localhost:3000, and the GUI should start working after a few seconds.
- You can write Python modules in the tensor_makers folder and they will immediately become available in the website.
- Start our JS server, as described above
- In a Jupyter notebook, write code like the following:
import interp.cui as cui
import numpy as np
from interp.ui.very_named_tensor import VeryNamedTensor
await cui.init(port=6789)
hidden_size = 20
token_strings = ["Hello","There"]
# using fake data in the shape of a transformer hidden state
tensor = np.random.normal(0,1,(len(token_strings),hidden_size))
my_vnt = VeryNamedTensor(
tensor,
dim_names=["Token","Embedding"],
dim_types=["seq","hidden"],
dim_idx_names=[
token_strings,
[str(i) for i in hidden_size]
],
units="activation",
title="Random Data",
)
await cui.show_tensors(my_vnt)
- See
cui_demo.ipynb
for another example using the CUI.