看了了一下TEXTOIR-DEMO的代码方法以及整个的架构图，

我的理解是TEXTOIR用的intents detect（新意图检测）和intents discover（新意图发现）

如样本【A1,A2,B1,B2,B3,C1, UNK1,UNK2】
前者新意图检测（如ADB）主要是发现可能的新类别，即“找出”UNK1,UNK2

后者新意图发现（如DeepAlignCluster）主要是用有监督指导无监督聚类，我的问题在于，
如果我把上述的数据输入到DAC，最后会得到（A1,A2,UNK1）（B1,B2,B3）（C1）（UNK2）三个聚类结果

所以我对新意图发现的理解是，
Q1：该方法论可以实现“相同意图下的样本扩充”（如->A1,A2,UNK1）
Q2: 也可以实现可能的新类的发现与聚合（如UNK2），是不是我直接DAC就能解决新意图检测的目的了，只是效果差别而已

不知道这两个Q我理解的对不，

I was following the instructions given in Quick Start guide. I am using macOS Monterey 12.4 on my 13 inch Macbook Pro, M1, 2020.
I have not install cudatoolkit as it is not supported for my machine.

On doing pip install -r requirements.txt it is showing following error:

ERROR: Could not find a version that satisfies the requirement mkl-fft==1.2.0 (from versions: 1.0.0.17, 1.0.2, 1.0.6)
ERROR: No matching distribution found for mkl-fft==1.2.0

I modified the requirements.txt file to install mkl-fft>=1.2.0 but it is showing same error.

Is this solvable or shall I try installing it on CUDA supported machine?

Hello,

First of thank you so much for developing and open-sourcing this.
I have used DeepAligned on the Banking dataset to train and save a model. Now I would like to use this model to run inference on another set of data. How can I do this? Thanks!

Is the example found in open_intent_detection/examples/run_DA-ADB.sh meant to reproduce the published results? Running this and comparing to ADB gives significantly worse results. I took a look to see why this is briefly but didn't find an obvious answer.

The average of ADB across the experiments gave 84% and DA-ADB gave 47%

Example of ADB results from which averages of Acc where calculated

The only change was the location of the bert model in hyper-parameters which was the same for both methods:
'bert_model': "bert-base-uncased",
Instead of: 'bert_model': "/home/sharing/disk1/pretrained_embedding/bert/uncased_L-12_H-768_A-12/",

Thanks for your help!

如何手动添加模型呢？

只有您进行比对的十种方法

I have a dataset with about 10% labeled data and I want to use this labeled data to do clustering on the rest. But the dataset in 'data' folder are all provided with ground truth label. How can I prepare data to use this work?

Hello,
First thank you for releasing such useful and well-written source!
I have used ADB for detection task. However, I find that in file "ADB/manager.py" line #68, "self.model.train()" is used. I am confused about using train() mode instead of eval() mode, since there are no parameters in "self.model" need to be trained. I try to turn it off, and the performance drops drastically. It seems that this setting is on purpose. How should I understand this? Thanks!

Hello,
is it possible to include also this work - https://github.com/tgargiani/Adaptive-Boundary. I am one of the co-author and it would be interesting to compare with your methods (which were used as root idea for our algorithm)
Best regards Petr

你好，抱歉打扰，我想询问一下open_intent_discovery部分是否能运行，我在运行的时候出现AttributeError: 'NoneType' object has no attribute 'tokenize'的问题，
发现在读取bert时有使用一个参数args.bert_model，但arg中并没有，在我添加了此参数后依旧无法正常运行，想请教一下~

谢谢~

Hi!

Thanks for providing this excellent demo. However, when I tried to run some baseline models for unsupervised open intent discovery like "SAE-KM", "DCN" and "DEC", I found that the performance is different from what is reported in paper "Discovering New Intents with Deep Aligned Clustering".

For instance, with "DCN" model in this demo, after averaging 3 runs on "banking", I get NMI 54.87, ARI 18.19, ACC 33.03. As opposed NMI 67.54, ARI 26.81, ACC 41.99 reported in the paper.

Notice that these models are all related with "SAE" embedding. I first wondered if it is because of different hyper-parameter choice. But the parameters I used strictly follows your implementation except the paths.

Also notice that I can not install all the dependencies by running pip install -r requirements.txt. Thus I have to change the versions for several packages by myself. The differences are:

1. keras-nightly==2.5.0.dev2021032900
2. tensorflow-gpu==2.5.1
3. Pillow==8.4.0
4. mkl-random==1.0.1.1
5. mkl-fft==1.0.6
6. numpy==1.19.5
7. remove torchaudio

To make it clearer in a future version, could you please:

1. add the performance list using your environment into README as a comparison for reproduction.
2. clean and remake the requirements.txt.

Thanks a lot! And appreciate your great work!

Excuse me, I can't find the DA-ADB model file under the Methods folder. Thank you very much for your help！

@inproceedings{Zhou2022KNNContrastiveLF,
  title={KNN-Contrastive Learning for Out-of-Domain Intent Classification},
  author={Yunhua Zhou and Peiju Liu and Xipeng Qiu},
  booktitle={Annual Meeting of the Association for Computational Linguistics},
  year={2022}
}

对应代码：https://github.com/zyh190507/KnnContrastiveForOOD

这里的超链接应该对应到 ./open_intent_detection/dataloaders/init.py，但似乎对应到了./open_intent_detection/configs/init.py，需修改

from utils import get_prebuilt_trulens_recorder

tru_recorder = get_prebuilt_trulens_recorder(query_engine,
app_id="App_1")
validation error for TruLlama
records_with_pending_feedback_results
Input should be an instance of Queue [type=is_instance_of, input_value=<queue.Queue object at 0x0000023A244C3190>, input_type=Queue]

Hi,

Many thanks for this repo. I ran intent detection and it runs smoothly. However, I am facing issues with intent discovery. The requirements were the first issue I encountered as some of the libraries were not getting installed. I then used python 3.7 and all installations were completed except deepspeed and triton. I manually installed deepspeed==0.3.16 but could'nt get triton installed. Regardless, I continued along the tutorial and ran run.py file but got below error. Please can you offer any advice? Thanks

Below is the syntax I used:
python run.py --setting semi_supervised --dataset banking --known_cls_ratio 0.75 --train --save_results --results_file_name results_DeepAligned.csv --save_model --labeled_ratio 0.2

Just as an example:

conda install pytorch torchvision torchaudio cudatoolkit=11.0 -c pytorch -c conda-forge

Results in:

Collecting package metadata (current_repodata.json): done
Solving environment: unsuccessful initial attempt using frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: unsuccessful initial attempt using frozen solve. Retrying with flexible solve.

PackagesNotFoundError: The following packages are not available from current channels:

  - cudatoolkit=11.0

Current channels:

  - https://conda.anaconda.org/pytorch/osx-64
  - https://conda.anaconda.org/pytorch/noarch
  - https://conda.anaconda.org/conda-forge/osx-64
  - https://conda.anaconda.org/conda-forge/noarch
  - https://repo.anaconda.com/pkgs/main/osx-64
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/r/osx-64
  - https://repo.anaconda.com/pkgs/r/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

Any easy workaround that could be stated within the readme?

would this be open-source in the future? I have read the paper but still wonder how it differs with ADB in details

Learn a lot from the work, by the way. LOL

Hi,
in your paper you wrote: "for example, the primary backbone is the pre-trained BERT (Devlin et al., 2019) model, and it supports adding new bert-based models with different downstream tasks."
But I would like to add other models for the Open Intent Detection part aswell, which are not bert-based, like Prototypical Networks, in order to compare more Open Intent Detection methods within this framework.
So is this possible?

Hi,

I'm trying to use your code for some time but I keep running into issues. I was wondering whether is was doing something wrong and if you could help me.

The main error I run into now is that when I try to run
sh examples/run_unsup_usnid.sh

it give me the error:
File "/TEXTOIR/open_intent_discovery/backbones/bert.py", line 341, in forward if not self.args.wo_ce: AttributeError: 'EasyDict' object has no attribute 'wo_ce'

But since I keep running into problems, I was wondering whether I was doing something wrong. Here is a overview of the steps I took:
Based on the requirements.txt I changed conda create --name textoir python=3.6 to conda create --name textoir python=3.8.13

I ran the rest of the code:

conda` activate textoir
conda install pytorch torchvision torchaudio cudatoolkit=11.0 -c pytorch -c conda-forge  
git clone [email protected]:thuiar/TEXTOIR.git
cd TEXTOIR
cd open_intent_discovery
pip install -r requirements.txt

However based on the following series of errors:

ERROR: Could not find a version that satisfies the requirement python==3.8.13 (from versions: none)
ERROR: No matching distribution found for python==3.8.13

ERROR: Could not find a version that satisfies the requirement faiss==1.7.2 (from versions: none)
ERROR: No matching distribution found for faiss==1.7.2

ERROR: Cannot install scikit-learn==0.24.1 and scikit_learn==1.2.0 because these package versions have conflicting dependencies.
ERROR: Cannot install -r requirements.txt (line 13) and Keras==2.4.3 because these package versions have conflicting dependencies.
ERROR: Cannot install -r requirements.txt (line 10), -r requirements.txt (line 11), -r requirements.txt (line 12), -r requirements.txt (line 14), -r requirements.txt (line 5), -r requirements.txt (line 6), -r requirements.txt (line 9) and numpy==1.23.1 because these package versions have conflicting dependencies.

I changed the requirements.txt to:

# python>=3.8.13
easydict>=1
# faiss
# faiss_gpu>=1
Keras>=2
matplotlib>=3
nltk>=3
numpy>=1
pandas>=2
scikit_learn>=1
scipy>=1
sentence_transformers>=2
tensorflow>=2
# tensorflow_gpu>=2
torch>=1
tqdm>=4
transformers>=4
scikit-learn>=1

Next i ran sh examples/run_unsup_usnid.sh which gave me the following error:

Traceback (most recent call last):
  File "run.py", line 3, in <module>
    from backbones.base import ModelManager
  File "/TEXTOIR2/open_intent_discovery/backbones/__init__.py", line 5, in <module>
    from .sae import get_stacked_autoencoder
  File "/TEXTOIR/open_intent_discovery/backbones/sae.py", line 5, in <module>
    from keras.engine.topology import Layer, InputSpec

So in /TEXTOIR/open_intent_discovery/backbones/sae.py i changed from keras.engine.topology import Layer, InputSpec to from tensorflow.keras.layers import Layer, InputSpec

I got this error:

Traceback (most recent call last):
  File "/home/iskander/miniforge3/envs/textoir2/lib/python3.8/site-packages/transformers/utils/hub.py", line 385, in cached_file
    resolved_file = hf_hub_download(
  File "/home/iskander/miniforge3/envs/textoir2/lib/python3.8/site-packages/huggingface_hub/utils/_validators.py", line 110, in _inner_fn
    validate_repo_id(arg_value)
  File "/home/iskander/miniforge3/envs/textoir2/lib/python3.8/site-packages/huggingface_hub/utils/_validators.py", line 158, in validate_repo_id
    raise HFValidationError(
huggingface_hub.utils._validators.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/home/sharing/disk1/pretrained_embedding/bert/uncased_L-12_H-768_A-12/'. Use `repo_type` argument if needed.

/home/sharing/disk1/pretrained_embedding/bert/uncased_L-12_H-768_A-12/ seems to be hard coded somewhere. I don't remember nor can I find how i fixed it.

Anyway after fixing that I got the main error I am asking you about:
File "/TEXTOIR/open_intent_discovery/backbones/bert.py", line 341, in forward if not self.args.wo_ce: AttributeError: 'EasyDict' object has no attribute 'wo_ce'

Again I cannot find any mentions of this argument anywhere.

Anyway thanks for composing TEXTOIR. I would greatly appreciate it if you could help me.

Hi~
Thank you so much for your open-source platform.
I want to run MSP method in intent detection.
I run this commonds sh examples/run_ADB.sh and sh examples/run_MAP.sh.
The platform would first run methods/__init__.py, but in methods/ADB/pretrain.py that cannot import name 'sigmoid_rampup' and 'linear_rampup'.
I found that 'sigmoid_rampup' and 'linear_rampup' not in utils/functions.py.
What should I do to run the method (MSP, ADB, etc.) in the examples ?

Hello,

I'm reaching out with a question while using TEXTOIR.

Hugging Face offers the latest library versions, but I'm curious as to why older versions of BERT, RoBERTa, etc., are being used. Could you explain the reason for this?