Refactor Directory Structure

Title says it all.

Task List

• move all operators and pipelines from /towhee/tests to /operators and /pipelines
• remove emulated_pipeline from tests directory.
• move /towhee/tests to /tests/unittests and update relevant paths in test files.
• combine /towhee/cnn_trainer and /towhee/trainer, data.
• combine array, and dataframe.
• remove data_source.
• remove utils folder (so it does not become an outpost for random functions) and move existing files within, if possible move hub_tools.py to separate hub folder and handler_mixin.py to engine

towhee [project]
  |- towhee [codebase]
  |- tests  # shares same directory tree as towhee [codebase]
  |- operators
  |- pipelines
  |- docs
  |- bootcamp
  |- setup.py
  |- README.md
  |- pylint.conf

towhee [codebase]
  |- engine
  |- models
    |- layers
    |- individual models
    |- utils
  |- trainer
    |- trainer.py  # abstract Trainer class
    |- train_args.py
    |- cnn_trainer.py
    |- transformer_trainer.py
    |- gan_trainer.py
  |- data
    |- dataset
      |- PytorchImageDataset.py
      |- PytorchVideoDataset.py
      |- TFImageDataset.py
    |- transforms
  |- cache
    |- hub
  |- common
    |- decorators
    |- mixins
  |- logger.py
  |- error.py
  |- type.py

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

Use flatmap_pipeline: https://github.com/towhee-io/towhee/blob/main/towhee/tests/test_util/test_flatmap/flatmap_pipeline.yaml
& operator: https://github.com/towhee-io/towhee/blob/main/towhee/tests/mock_operators/repeat_operator/repeat_operator.py

It only returns 1st result. However, if print [Outputs(num) for i in range(self._repeat)] in operator before return, it prints all results.
Sample result:

# result printed in operator before return
[Outputs(num=3), Outputs(num=3), Outputs(num=3), Outputs(num=3), Outputs(num=3)]

# actual result returned by pipeline
(3,)

Expected Behavior

Expect to return all results, not only the first one.

example:
test_pipeline(3) returns (3, 3, 3, 3, 3) instead of (3,) when init_args repeat is 5.

Steps To Reproduce

1. Use setup.py to install towhee
2. Download pipeline: https://github.com/towhee-io/towhee/blob/main/towhee/tests/test_util/test_flatmap/flatmap_pipeline.yaml
& operator: https://github.com/towhee-io/towhee/blob/main/towhee/tests/mock_operators/repeat_operator/repeat_operator.py
3. Try this pipeline

Environment

- Towhee version: towhee==0.1.3
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Anything else?

No response

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

>>> from towhee import pipeline
>>> from PIL import Image
>>> img_path = '/Users/binbin/Towhee_projects/2021825-19039.jpg'
>>> img_path = '/Users/binbin/Towhee_projects/2021825-19039.jpg'
>>> embedding_pipeline = pipeline('shiyu/img_object_embedding_pytorch_yolov5_resnet50')
>>> embedding = embedding_pipeline(img_path)
Downloading: "https://github.com/ultralytics/yolov5/archive/master.zip" to /Users/binbin/.cache/torch/hub/master.zip
No module named 'seaborn'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/binbin/Towhee_projects/towhee-test-env/lib/python3.8/site-packages/towhee/__init__.py", line 78, in __call__
    out_df = self._pipeline(in_df)
  File "/Users/binbin/Towhee_projects/towhee-test-env/lib/python3.8/site-packages/towhee/engine/pipeline.py", line 111, in __call__
    graph_ctx.outputs.wait_sealed()
  File "/Users/binbin/Towhee_projects/towhee-test-env/lib/python3.8/site-packages/towhee/dataframe/dataframe.py", line 156, in wait_sealed
    self._seal_cv.wait()
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/threading.py", line 302, in wait
    waiter.acquire()

Expected Behavior

Embedding successfully

Steps To Reproduce

from towhee import pipeline
from PIL import Image
img_path = '/Users/binbin/Towhee_projects/2021825-19039.jpg'
embedding_pipeline = pipeline('shiyu/img_object_embedding_pytorch_yolov5_resnet50')
embedding = embedding_pipeline(img_path)

Environment

- Towhee version(e.g. v0.1.3 or 8b23a93): 0.2.2
- OS(Ubuntu or CentOS): mac
- CPU/Memory: 
- GPU: 
- Others:

Anything else?

No response

Is there an existing issue for this?

• I have searched the existing issues

What would you like to be added?

1. When using the python hub_tools.py create command to create a hub, need to enter a password, but the entered password will not be hidden. I suggest that you can hide the entered password in the Python code.

2. It is recommended to change Please enter the repo type to Please enter the repo type, choose one from "model | operator | pipeline | dataset"

Why is this needed?

1. A hidden password can protect privacy.
2. It is necessary to show the optional repo type.

Anything else?

No response

The actual meaning of OperatorContext.is_finished is 'all tasks are ready', which is quite misleading.

Is there an existing issue for this?

• I have searched the existing issues

What would you like to be added?

If a repository (either pipeline or operator) on the Towhee hub is updated, the changes are not automatically reflected during local runs if a prior version of said repository exists.

Why is this needed?

This seems to be unintended behavior.

Anything else?

No response

DAG Implementation Checklist

Based on the agreed upon architecture, the DAG will serve as a template for the way the engine moves data around in the pipeline. In essence, each engine component has a corresponding representation in the DAG:

• GraphRepr: A simple container for operators and dataframes. It does not explicitly hold any data regarding the dataflow between operators and dataframes.
• OperatorRepr: An operator abstraction. This will hold the operator function along with input and output dataframes and the iterators used for the respective dataframes.
• DataframeRepr: A dataframe representation. Essentially a collection of variable representations with no information about the operators which use or supply said dataframes.
• VariableRepr: A variable representation. Stores information about the individual variable, i.e. variable type (ndarray, Image, etc) without knowing which Dataframe it is a part of.

Task List

Graph

• base implementation (attributes & helper functions)
• unit tests
• from_yaml
• to_yaml
• error checking: no cyclical loops

Operator

• base implementation (attributes & helper functions)
• unit tests
• error checking: no orphan operators
• error checking: all operators have both inputs and outputs

Dataframe

• base implementation (attributes & helper functions)
• unit tests
• from_input_annotations
• from_output_annotations
• error checking: no orphan dataframes
• error checking: all dataframes have input operators (except data sources)

Variable

• base implementation (attributes & helper functions)
• unit tests

Other Tests

• load yaml -> dump yaml identity checking tests
• engine can create pipeline using graph

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

Pipeline "towhee/video_embedding_resnet50" hangs when embedding avi type video

Expected Behavior

Embedding successfully

Steps To Reproduce

from towhee import pipeline
p = pipeline('towhee/video_embedding_resnet50')
video_path_1 = "/Users/binbin/Towhee_projects/test.avi"
video_path_1_vec = p(video_path_1)

Environment

- Towhee version(e.g. v0.1.3 or 8b23a93): 0.2.3.dev3
- OS(Ubuntu or CentOS): mac
- CPU/Memory: 
- GPU: 
- Others:

Anything else?

No response

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

The response time of Image embedding is about double that of video embedding

Expected Behavior

The response time of Image embedding should be smaller or equal with that of video embedding

Steps To Reproduce

No response

Environment

- Towhee version(e.g. v0.1.3 or 8b23a93):
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Anything else?

No response

Needs an issue template giving users more guides on what needs to be added for opening an issue.

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

did not return

>>> video_vec = p("/Users/binbin/Towhee_projects/binbin1.mp4")
OpenCV: Couldn't read video stream from file "/Users/binbin/Towhee_projects/binbin1.mp4"

>>> video_vec = p("/Users/binbin/Towhee_projects/")
hangs here

Expected Behavior

Report error just like image embedding

Steps To Reproduce

1 setup env and install towhee:

 python
python3 -m venv towhee-test-env
pip install -U pip
pip install towhee

2 into python env: python3:
video_path with no existed video

from towhee import pipeline
p = pipeline('towhee/video_embedding_resnet50')
video_vec = p(video_path)

### Environment

```markdown
- Towhee version(e.g. v0.1.3 or 8b23a93): 0.2
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Anything else?

No response

Say we have an operator which takes the path of a model weight file as the init arg like following:

If the user provides the weight file in the operator repo, which in this case is clmr_magnatagatune&towhee$main/checkpoints/clmr_checkpoint_10000.pt
How do we deal with the init arg in pipeline yaml file?
We cannot use abs path since the different users have different user directory, we cannot user relative path since we don't know the under which directory will the users run the pipeline.

Due to the seal mechanism of DataFrame, the GraphContext can not be resued.
I suggest to add clear() method to DataFrame.
Each time a GraphContext process a row, it will firstly reset its self, mainly by clear its DataFrames

In the latest version of GraphRepr, the _operators and _dataframes are lists. Alsoo, we only have the name of input/output dataframes inside the OperatorRepr.
In this situation, for each input/output, we have to traverse the _dataframes list to find the corresponding dataframe.

Steps/Code to reproduce:
1 setup env and install towhee:

python3 -m venv towhee-test-env
pip install -U pip
pip install towhee

2 into python env: phthon3:

from towhee import pipeline
p = pipeline('image-embedding')
embedding = p()

Expected result:
Return error

Actual results:
pipeline crashed with eexception:

>>> embedding = p()
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/Users/binbin/Towee/towhee-test-env/lib/python3.8/site-packages/towhee/engine/engine.py", line 85, in run
    self._task_sched.schedule_forever(self._config.sched_interval_ms)
  File "/Users/binbin/Towee/towhee-test-env/lib/python3.8/site-packages/towhee/engine/task_scheduler.py", line 78, in schedule_forever
    self.schedule_step()
  File "/Users/binbin/Towee/towhee-test-env/lib/python3.8/site-packages/towhee/engine/task_scheduler.py", line 139, in schedule_step
    tasks = op_ctx.pop_ready_tasks(n_tasks=1)
  File "/Users/binbin/Towee/towhee-test-env/lib/python3.8/site-packages/towhee/engine/operator_context.py", line 99, in pop_ready_tasks
    op_input_params = self._reader.read()
  File "/Users/binbin/Towee/towhee-test-env/lib/python3.8/site-packages/towhee/engine/_operator_io.py", line 87, in read
    return self._to_op_inputs(data[0])
  File "/Users/binbin/Towee/towhee-test-env/lib/python3.8/site-packages/towhee/engine/_operator_io.py", line 63, in _to_op_inputs
    ret[key] = cols[index].value
IndexError: tuple index out of range

Environment:
Mac OS

Currently, the Pipeline can not gurantee the consistency of its inputs and outputs order.
We need to add index to each call of GraphContext, and add reorder mechanism triggered by`GraphContext.on_finish_handlers.

https://chris.beams.io/posts/git-commit/

A good guide to follow.

Add a README to improve community awareness.

Attach branch in order to not confuse between different versions of an op in one pipeline.

A Hello World Example

class Inc(Operator):
    @create_op_in_pipeline
    def __call__(self, x:int) -> int:
        return x+1

class Add(Operator):
    @create_op_in_pipeline
    def __call__(self, x1:int, x2:int) -> int:
        return x1+x2

@create_pipeline
def my_pipeline(x1:int, x2:int):
    inc1 = Inc()
    inc2 = Inc()
    add = Add()
    return add(inc1(x1), inc2(x2))

In this example, we have three operators in a pipeline, where add depends on the results of inc1 and inc2. Behind the scene, Towhee's compiler will create a DAG during the execution of my_pipeline. The DAG construction is driven by the operator decorator and pipeline decorator. When the program executes the line add(inc1(x1), inc2(x2)), the decorator @create_op_in_pipeline will be called before the Operators' call method. It will put inc1, inc2, add into the pipeline's context, link the pipeline's input x1, x2 to inc1, inc2 respectively, and set inc1, inc2's output as add's input. The operators' dependencies are also settled during the DAG construction.

Main Components

To support the example mentioned above, Towhee needs five major components.

• Operator An operator is a set of code that performs one step in the pipeline, such as preprocessing, model inference, postprocessing, etc. An operator is analogous to a function, in that it has a name, parameters, return values, and a body. Once involved in a pipeline, an operator will be construct as a node in the DAG.

• Compiler There are two phase compilation. The first step is to convert a pipeline description (eg. my_pipeline) and operators (eg. Inc, Add) into a intermediate graph, where the variables, operators, dependencies are precisely described. The second step is to convert the intermediate graph to a backend executable, such as a local python-driven DAG, or a kubeflow pipeline.

• Variable The data abstraction of pipeline, operator's inputs and outputs. It is partitionable and iterable. The difference of device memory, the cross-device memory copy, data transfer is also handled by Variable.

• Pipeline The runtime scheduling context of a pipeline, including variables, tasks, and execution flow. All the statefull part during taks execution is maintained in Pipeline.

• Engine The engine driving the task execution on a pipeline. It takes a Pipeline as its scheduling context, and performs all the necessary data paritition, operator parallelization, resource management, etc.

Works Remaining

Operator

• interface design
• internal key classes design
• docstrings
• design proposal: operator abstraction

Pipeline

• interface design
• internal key classes design
• docstrings
• design proposals

Engine

• interface design
• internal key classes design
• docstrings
• design proposals

Variable

• interface design
• internal key classes design
• docstrings
• design proposal: variable abstraction

Compiler

• interface design
• internal key classes design
• docstrings
• design proposal: intermediate graph representation
• design proposal: graph construction driven mechanism
• design proposal: convert intermediate graph to initial pipeline context

Enforcing a linear commit history prevents collaborators from pushing merge commits to the branch. This means that any pull requests merged into the protected branch must use a squash merge or a rebase merge. A strictly linear commit history can help teams reverse changes more easily. it also makes it easier to review the Git timeline, as it won't look like a giant ball of spaghetti.

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

>>> video_vec = p("")
Downloading Operator: video_representation_frame
Downloading __init__.py: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 592/592 [00:00<00:00, 153kiB/s]
Downloading README.md: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 55.0/55.0 [00:00<00:00, 13.2kiB/s]
Downloading video_representation_frame.py: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.85k/1.85k [00:00<00:00, 241kiB/s]
OpenCV: Couldn't read movie file ""
[ERROR:0] global /Users/runner/work/opencv-python/opencv-python/opencv/modules/videoio/src/cap.cpp (166) open VIDEOIO(CV_IMAGES): raised OpenCV exception:

OpenCV(4.5.4-dev) /Users/runner/work/opencv-python/opencv-python/opencv/modules/videoio/src/cap_images.cpp:293: error: (-215:Assertion failed) !_filename.empty() in function 'open'

Expected Behavior

Report error

Steps To Reproduce

1 setup env and install towhee:


python3 -m venv towhee-test-env
pip install -U pip
pip install towhee


2  into python env: python3:

``` python
from towhee import pipeline
p = pipeline('towhee/video_embedding_resnet50')
embedding = p("")

### Environment

```markdown
- Towhee version(e.g. v0.1.3 or 8b23a93): 0.2
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Anything else?

No response

Steps/Code to reproduce:
1 setup env

python3 -m  venv towhee-test-env

2 update pip and install towhee

pip install -U pip
pip install towhee

Expected result:
towhee install successfully

Actual results:

pip install towhee
Collecting towhee
  Downloading towhee-0.1-py3-none-any.whl (134 kB)
     |████████████████████████████████| 134 kB 511 kB/s
Collecting pillow>=8.3.1
  Downloading Pillow-8.3.2-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
     |████████████████████████████████| 3.0 MB 2.0 MB/s
Collecting pyyaml>=5.3
  Using cached PyYAML-5.4.1-cp36-cp36m-manylinux1_x86_64.whl (640 kB)
ERROR: Could not find a version that satisfies the requirement numpy>=1.20.3 (from towhee) (from versions: 1.3.0, 1.4.1, 1.5.0, 1.5.1, 1.6.0, 1.6.1, 1.6.2, 1.7.0, 1.7.1, 1.7.2, 1.8.0, 1.8.1, 1.8.2, 1.9.0, 1.9.1, 1.9.2, 1.9.3, 1.10.0.post2, 1.10.1, 1.10.2, 1.10.4, 1.11.0, 1.11.1, 1.11.2, 1.11.3, 1.12.0, 1.12.1, 1.13.0rc1, 1.13.0rc2, 1.13.0, 1.13.1, 1.13.3, 1.14.0rc1, 1.14.0, 1.14.1, 1.14.2, 1.14.3, 1.14.4, 1.14.5, 1.14.6, 1.15.0rc1, 1.15.0rc2, 1.15.0, 1.15.1, 1.15.2, 1.15.3, 1.15.4, 1.16.0rc1, 1.16.0rc2, 1.16.0, 1.16.1, 1.16.2, 1.16.3, 1.16.4, 1.16.5, 1.16.6, 1.17.0rc1, 1.17.0rc2, 1.17.0, 1.17.1, 1.17.2, 1.17.3, 1.17.4, 1.17.5, 1.18.0rc1, 1.18.0, 1.18.1, 1.18.2, 1.18.3, 1.18.4, 1.18.5, 1.19.0rc1, 1.19.0rc2, 1.19.0, 1.19.1, 1.19.2, 1.19.3, 1.19.4, 1.19.5)
ERROR: No matching distribution found for numpy>=1.20.3

Environment:

Linux version 5.4.0-84-generic (buildd@lcy01-amd64-007) (gcc version 7.5.0 (Ubuntu 7.5.0-3ubuntu118.04)) #9418.04.1-Ubuntu SMP Thu Aug 26 23:17:46 UTC 202

use DataFrameRepr instead

1. setup.py
2. coverage config
3. init.py file
4. test demo
5. pylint.conf

make a file 'docs' to create documents for Towhee

Refactor DataFrame as column based, so that multiple DataFrames can share Arrays in a non-copy way.
related classes:

• towhee.Array
• towhee.DataFrame
• OperatorContext (construction of OperatorContext's inputs and outputs)

currently, OperatorContext.init only works in the case that every OperatorContext takes one DataFrame as its input and one DataFrame as its output.

Current hub operator download logic might result in race conditions due to multiple separate processes checking and downloading the same files. This issue is worsened by the fact we need to support a force_download tag on the operator which might cause a deadlock in deleting and redownloading.

The main file these issues will be found in is here .

• reset of GraphContext
• assign index number to each GraphContext call
• reorder of GraphContext's concurrent results

Steps/Code to reproduce:
1 setup env and install towhee:

python3 -m venv towhee-test-env
pip install -U pip
pip install towhee

2 into python env: phthon3:

from towhee import pipeline
pipeline = pipeline('image-embedding')

Expected result:
pipeline return successfully

Actual results:
error:
Can not find pipeline by name resnet50_embedding

Environment:
Mac OS

Towhee is going to mirror open sourced models to towhee hub for reliability when the hub is ready.

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

>>> p = pipeline('towhee/video_embedding_resnet50')
>>> video_list = {"video1": "/home/zong/binbin/Towhee/binbin.mp4"}
>>>
>>> vec = p(video_list)
ERROR:root:OpenCV(4.5.4-dev) :-1: error: (-5:Bad argument) in function 'VideoCapture'
> Overload resolution failed:
>  - Can't convert object of type 'dict' to 'str' for 'filename'
>  - Required argument 'apiPreference' (pos 2) not found
>  - Argument 'index' is required to be an integer
>  - Required argument 'apiPreference' (pos 2) not found

>

Expected Behavior

Report specific error

Steps To Reproduce

1 install towhee

python3 -m venv towhee-test-env
pip install -U pip
pip install towhee
 


2  into python env: python3:

``` python
from towhee import pipeline
p = pipeline('towhee/video_embedding_resnet50')
video_list = {"video1": "/home/zong/binbin/Towhee/binbin.mp4"}
vec = p(video_list)

Environment

- Towhee version(e.g. v0.1.3 or 8b23a93):0.2
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Anything else?

No response

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

Embedding hangs:

>>> video_vec = p("/Users/binbin/Towhee_projects/2021825-19039.jpg")

Expected Behavior

Report error

Steps To Reproduce

python
python3 -m venv towhee-test-env
pip install -U pip
pip install towhee

2 into python env: python3:

from towhee import pipeline
p = pipeline('towhee/video_embedding_resnet50')
embedding = p("/Users/binbin/Towhee_projects/2021825-19039.jpg") Here embedding image

### Environment

```markdown
- Towhee version(e.g. v0.1.3 or 8b23a93): 0.2
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Anything else?

No response

Hub Integration:

@filip-halt @fzliu @Chiiizzzy

To Do's For Hub Integration

• Create tools for hub.towhee.io interaction
• Incorporate rudimentary pipeline cache-check and download
• Incorporate rudimentary operator cache-check and download
• Figure out how to deal with multi executor file downloads and race conditions: #182
• Figure out how to deal with force_redownload logic with operators and interaction when operator called multiple times by multiple executors
• #229
• Figure out where to download requirements.txt (python virual env/conda/etc) and how to launch operation there
• Decide on where local files will be worked on in order to avoid checking cache for them (local/ maybe)
• Have users manually use git for uploading pipelines or build into towhee

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

Pipeline imported failed from towhee on OS ubuntu

>>> from towhee import pipeline
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/zong/binbin/towhee/towhee-test-env/lib/python3.6/site-packages/towhee/__init__.py", line 22, in <module>
    from towhee.engine.engine import Engine, start_engine
  File "/home/zong/binbin/towhee/towhee-test-env/lib/python3.6/site-packages/towhee/engine/engine.py", line 18, in <module>
    from towhee.engine.pipeline import Pipeline
  File "/home/zong/binbin/towhee/towhee-test-env/lib/python3.6/site-packages/towhee/engine/pipeline.py", line 17, in <module>
    from towhee.engine.graph_context import GraphContext
  File "/home/zong/binbin/towhee/towhee-test-env/lib/python3.6/site-packages/towhee/engine/graph_context.py", line 21, in <module>
    from towhee.engine.operator_context import OperatorContext, OpStatus
  File "/home/zong/binbin/towhee/towhee-test-env/lib/python3.6/site-packages/towhee/engine/operator_context.py", line 19, in <module>
    from towhee.engine.operator_runner.runner_base import RunnerStatus
  File "/home/zong/binbin/towhee/towhee-test-env/lib/python3.6/site-packages/towhee/engine/operator_runner/__init__.py", line 22, in <module>
    from towhee.engine.operator_io.reader import DataFrameReader
  File "/home/zong/binbin/towhee/towhee-test-env/lib/python3.6/site-packages/towhee/engine/operator_io/__init__.py", line 16, in <module>
    import towhee.engine.operator_io.reader as io_reader
AttributeError: module 'towhee.engine' has no attribute 'operator_io'

Expected Behavior

pipeline imported from towhee successfully

Steps To Reproduce

from towhee import pipeline

Environment

- Towhee version(e.g. v0.1.3 or 8b23a93): 0.2.3.dev3
- OS(Ubuntu or CentOS): Ubuntu 18.04
- CPU/Memory: 
- GPU: 
- Others:

Anything else?

No response

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

Two different videos are embedded into equal vectors

Expected Behavior

Two different videos are embedded into not equal vectors

Steps To Reproduce

1 install towhee

python3 -m venv towhee-test-env
pip install -U pip
pip install towhee

2 into python env: python3:

from towhee import pipeline
p = pipeline('towhee/video_embedding_resnet50')
video_path_1 = "/Users/binbin/Towhee_projects/1.mp4"
video_vec_1 = p(video_path_1)
video_path_2 = "/Users/binbin/Towhee_projects/binbin.mp4"
video_vec_2 = p(video_path_2)
video_vec_1[0][0][0] == video_vec_2[0][0][0]

tensor([True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True])

Environment

- Towhee version(e.g. v0.1.3 or 8b23a93): 0.2.3.dev3
- OS(Ubuntu or CentOS): mac
- CPU/Memory: 
- GPU: 
- Others:

Anything else?

No response

complete a call stack class, which includes the following functions:

• collect function, collect all the frames of the current call stack
• num_frmes, return the number of frames in the current stack
• find_func, given a function name, find the first match from the outermost frame in the stack, return its index
• hash, given the start and end indexes and an items list, return the hash value of the attributes in the items list between start and end.

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

Embedding is failed for "Image" type object with pipeline "shiyu/img_object_embedding_pytorch_yolov5_resnet50" reporting:

Can't convert object of type 'Image' to 'str' for 'filename'

>>> embedding = embedding_pipeline(img)
Using cache found in /Users/binbin/.cache/torch/hub/ultralytics_yolov5_master
YOLOv5 🚀 2021-12-1 torch 1.10.0 CPU

Fusing layers...
Model Summary: 213 layers, 7225885 parameters, 0 gradients
Adding AutoShape...
/Users/binbin/Towhee_projects/towhee-test-env/lib/python3.8/site-packages/torch/autocast_mode.py:141: UserWarning: User provided device_type of 'cuda', but CUDA is not available. Disabling
  warnings.warn('User provided device_type of \'cuda\', but CUDA is not available. Disabling')
Can't convert object of type 'Image' to 'str' for 'filename'

Expected Behavior

Embedding is successful

Steps To Reproduce

from PIL import Image
from towhee import pipeline

img = Image.new(mode="RGB", size=(200, 200), color=(155, 155, 155))
pipeline_name = "shiyu/img_object_embedding_pytorch_yolov5_resnet50"
embedding_pipeline = pipeline(pipeline_name)
embedding = embedding_pipeline(img)

Environment

- Towhee version(e.g. v0.1.3 or 8b23a93): 0.2.2
- OS(Ubuntu or CentOS): mac
- CPU/Memory: 
- GPU: 
- Others:

Anything else?

No response

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

>>> embedding = p("")
ERROR:root:'str' object has no attribute 'read'

Expected Behavior

when input not valid, return useful error message

Steps To Reproduce

1 setup env and install towhee:

python3 -m venv towhee-test-env
pip install -U pip
pip install towhee

2 into python env: python3:

from towhee import pipeline
p = pipeline('image-embedding')
embedding = p("")

Environment

- Towhee version(e.g. v0.1.3 or 8b23a93): 0.2
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Anything else?

No response

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

Embedding is empty for pipeline "'shiyu/img_object_embedding_pytorch_yolov5_resnet50'"

>>> from towhee import pipeline
>>> from PIL import Image
>>> img_path = '/Users/binbin/towhee/docs/towhee_logo.png'
>>> embedding_pipeline = pipeline('shiyu/img_object_embedding_pytorch_yolov5_resnet50')
>>> embedding = embedding_pipeline(img_path)
Using cache found in /Users/binbin/.cache/torch/hub/ultralytics_yolov5_master
YOLOv5 🚀 2021-12-1 torch 1.10.0 CPU

Fusing layers...
Model Summary: 213 layers, 7225885 parameters, 0 gradients
Adding AutoShape...
/Users/binbin/Towhee_projects/towhee-test-env/lib/python3.8/site-packages/torch/autocast_mode.py:141: UserWarning: User provided device_type of 'cuda', but CUDA is not available. Disabling
  warnings.warn('User provided device_type of \'cuda\', but CUDA is not available. Disabling')
>>>
>>>
>>> embedding
[]

Expected Behavior

Embedding vector is created

Steps To Reproduce

from towhee import pipeline
from PIL import Image
img_path = '/Users/binbin/towhee/docs/towhee_logo.png'
embedding_pipeline = pipeline('shiyu/img_object_embedding_pytorch_yolov5_resnet50')
embedding = embedding_pipeline(img_path)

Environment

- Towhee version(e.g. v0.1.3 or 8b23a93):0.2.2
- OS(Ubuntu or CentOS): mac
- CPU/Memory: 
- GPU: 
- Others:

Anything else?

No response

Describe the bug:

When I add an operator, it will show the error TypeError: 'NoneType' object is not iterable if __int__ without the argument. It can refer to #166.

Environment:

• Towhee version: v0.1.3
• OS: Mac OS

Engine Implementation Checklist

The engine lies at Towhee's core, driving the dataflow, partitioning tasks to devices, and managing the overall state of the pipeline. When a user calls a pipeline() function, an Pipeline object and all associated contexts and device managers are created. Execution threads for each device is created through a TaskExecutor, and management of graph state is done through GraphContext and OperatorContext classes. Each OperatorContext is responsible for passively managing its own input iterators, while the Scheduler actively (periodically) asks each operator whether or not it has any tasks ready for processing. Each task is then assigned by the Scheduler to one of many TaskExecutor objects, resulting in parallel processing of simultaneous tasks.

This architecture allows a single pipeline to be run across one or many interconnected machines, each of which may have multiple devices.

Task List

GraphContext and OperatorContext (~19 days)

• graph state management
• operator state management
• data management and dataflow between operators
• task acquisition, if input data is available
• unit tests

Dataframe and Array (~3 days)

• integration with data source
• map iterator (v0.1)
• unit tests

Task, TaskExecutor, and OperatorPool (~9 days)

• operator acquisition & release
• task management
• task execution
• unit tests

Scheduler (~5 days)

• looping FIFO scheduler implementation
• basic integration tests with simple Operators
• unit tests

Pipeline (~2 days)

• GraphContext wrapper implementation
• unit tests

my-first-pipeline (~3 days)

• successful creation of pipeline using DAG
• successful processing with PyTorch models

we need to find a proper place for this file. Maybe somewhere in dag

OperatorContext should have only one DataFrame as its output.
The current implementation is a list of DataFrames

Is your feature request related to a problem? Please describe.

The resnet50_embedding pipeline only input a PIL.image, but the Yolo object detection operator will return a list of PIL.image, so I cannot directly embed the detected objects with the YAML pipeline.

Describe the solution you'd like.

I would like a pipeline to get the embedding of the object detected image through the pytorch_yolo_detector_operator.

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

Install towhee by setup.py:
Run python setup.py install does NOT install any packages listed in install_requires.

• New clean environment
• Python version: 3.8.8

Expected Behavior

Run python setup.py install should install packages listed in install_requires.

Steps To Reproduce

No response

Environment

- Towhee version: v0.1.3
- OS(Ubuntu or CentOS): CentOS
- CPU/Memory: 
- GPU: 
- Others:

Anything else?

No response

hello world.

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

Run coverage on ubuntu system,
it failed reporting:

test_flatmap_writer (towhee.tests.engine.test_engine.TestEngine) ... Downloading Operator: repeat_operator
ERROR:root:mock_operators/repeat_operator repo doesnt exist. 

Expected Behavior

No failure

Steps To Reproduce

coverage run setup.py test -s towhee.tests

Environment

- Towhee version(e.g. v0.1.3 or 8b23a93): 0.2.0
- OS(Ubuntu or CentOS): Ubuntu 
- CPU/Memory: 
- GPU: 
- Others:

Anything else?

No response

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

1. With VPN disabled & the pipeline repo at local, run in python: p = pipeline('author/repo', force_download = True)
   --> It returns connection error: "ConnectionRefusedError: [Errno 61] Connection refused"

2. Then enable VPN & re-run in python: p = pipeline('author/repo', force_download = True)
   --> It returns error: failed to find local path of repo

Expected Behavior

• Option 1: print out message that the local repo has been deleted after step 1, before error out.
• Option 2: pipeline force_download can cover the case of no local repo.

Steps To Reproduce

No response

Environment

- Towhee version: 0.3.1.dev2
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Anything else?

No response