tencentyouturesearch / crowdcounting-p2pnet Goto Github PK
View Code? Open in Web Editor NEWThe official codes for the ICCV2021 Oral presentation "Rethinking Counting and Localization in Crowds: A Purely Point-Based Framework"
License: Other
The official codes for the ICCV2021 Oral presentation "Rethinking Counting and Localization in Crowds: A Purely Point-Based Framework"
License: Other
Hello,
thanks for uploading the code!
When trying to run a test using supplied pretrained Shanghai Dataset Part A, an error is raised about VGG_BN pretrained weights:
No such file or directory: '/apdcephfs/private_changanwang/checkpoints/vgg16_bn-6c64b313.pth
Could you share it?
Thanks
Very nice work.
I want to know wehre can obtain the evaluation code of nAP.
Hi everyone, 你們好,
When I run the train.py, it occurs the following problems:
當我運行train.py時,出現了以下問題:
The annotation points in the Txt file follow as instructions.
每一張圖片的annotation points 都是根據要求寫的
x1 y1
x2 y2
Each image will contain at least 4 points
每張圖片至少有4個點
Some of the annotation points and images have been done.
已經有一部份圖片處理了.
Just wondering how to solve this problem.
請問一下我要如何解決這個問題?
感謝感謝
请问将训练结果如何转换onnx格式
我在试着复现程式
请问regression head的部分是如何建构的?
以及regression loss在没有权重调整前数值大概是多少?
论文的结果非常impressive,不知道作者什么时候可以release代码让大家共同学习一下呢?
hello ~thanks for this great job.
I get some trouble when resizing images in run_tset.py.
I modify new_width and new_height like this:
# load the images
img_raw = Image.open(data_path+img_path).convert('RGB')
# round the size
width, height = img_raw.size
img_raw = img_raw.resize((int(width*0.5), int(height*0.5)), Image.ANTIALIAS)
there is the error message:
Namespace(backbone='vgg16_bn', gpu_id=0, line=2, output_dir='./logs/', row=2, weight_path='./weights/SHTechA.pth')
File "run_test.py", line 140, in
main(args)
File "run_test.py", line 107, in main
result = self.forward(*input, **kwargs)
File "D:\working_space_HJ\CrowdCounting-P2PNet-main\models\p2pnet.py", line 215, in forward
features_fpn = self.fpn([features[1], features[2], features[3]])
File "C:\ProgramData\Anaconda3\envs\crowd\lib\site-packages\torch\nn\modules\module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "D:\working_space_HJ\CrowdCounting-P2PNet-main\models\p2pnet.py", line 183, in forward
P4_x = P5_upsampled_x + P4_x
RuntimeError: The size of tensor a (134) must match the size of tensor b (135) at non-singleton dimension 2
please help me and I'm really appreciate it if anyone could answer these questions. Thanks very much. ^ ^
训练过程中,显存会随着epoch忽高忽低。有时会导致显卡爆炸。
有人遇到这种问题吗?
Can you provide processed shha datasets or preprocessing code? thanks!
如题
Hi!
How do one add support training on images without heads? Is it possible?
In models/p2pnet.py
:
Line 278: losses['loss_point'] = loss_bbox.sum() / num_points
Line 335: weight_dict = {'loss_ce': 1, 'loss_points': args.point_loss_coef}
one is "loss_points" and another is "loss_point" (an "s" is different) ! This would cause that the point loss is actually not used, which is fully wrong!
Hi all!
I'd love to use this model in our ONNX flows but wasn't able to convert it to ONNX. Is there any known way of converting this model to ONNX?
Code I am using:
import os
import sys
import torch
sys.path.append(os.path.abspath(f"{os.getcwd()}/model"))
# Available after the above append
# it's in the model folder
from model.models.p2pnet import P2PNet
from model.models.backbone import Backbone_VGG
def main():
onnx_model_name = sys.argv[1] or "model"
onnx_model_name = f"{onnx_model_name}.onnx"
print("Loading Model")
# Create the model
model_backbone = Backbone_VGG("vgg16_bn", True)
model = P2PNet(model_backbone, 2, 2)
# Load Weights
checkpoint = torch.load("./model/weights/SHTechA.pth", map_location=torch.device('cpu'))
model.load_state_dict(checkpoint["model"])
model.eval() # Put in inference mode
# Create dummy input
dummy_input = torch.randn(1, 3, 640, 640)
# dummy_input1 = torch.randn(1, 3, 1024, 1024)
# dummy_input = (dummy_input0, dummy_input1)
# Export as ONNX
print(f"Exporting as ONNX: {onnx_model_name}")
torch.onnx._export(
model,
dummy_input,
onnx_model_name, # Output name
opset_version=13, # ONNX Opset Version
export_params=True, # Store the trained parameters in the model file
do_constant_folding=True, # Execute constant folding for optimization
input_names = ['input'], # the model's input names
# output_names = ['pred_logits', 'pred_points'], # the model's output names (see forward in the architecture)
output_names = ['pred_logits', 'pred_points'], # the model's output names (see forward in the architecture)
dynamic_axes={
# Input is an image [batch_size, channels, width, height]
# all of it can be variable so we need to add it in dynamic_axes
'input': {
0: 'batch_size',
1: 'channels',
2: 'width',
3: 'height'
},
'pred_logits': [0, 1, 2],
'pred_points': [0, 1, 2],
}
)
if __name__ == "__main__":
main()
Error I receive:
[ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running Add node. Name:'Add_88' Status Message: Add_88: right operand cannot broadcast on dim 1 LeftShape: {1,40960,2}, RightShape: {1,25600,2}
我直接训练最好得到54,但是我用给出的权重(51.9,阈值保持默认为0.5)跑出来是53.13,但是阈值改成0.45是可以到52的,请问51.9是在当前代码默认setting下训练出来的还是有所调整呢?
如果是在当前代码下跑出来的,请问能给一下你们的SHTechA的数据集和组织格式吗?因为我只改了loader。
I found
loss_dict_reduced_scaled = {k: v * weight_dict[k] for k, v in loss_dict_reduced.items() if k in weight_dict}
the key in weight_dict and loss_dict_reduced is not consistent with each other.
one is loss_point, the other is ;oss_points.
使用NWPU数据集训练时出现如下错误,使用ShanghaiTech PartA ,PartB训练OK
Start training
Traceback (most recent call last):
File "train.py", line 222, in
main(args)
File "train.py", line 159, in main
stat = train_one_epoch(
File "/home/model/CrowdCounting-P2PNet/engine.py", line 85, in train_one_epoch
for samples, targets in data_loader:
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/dataloader.py", line 521, in next
data = self._next_data()
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/dataloader.py", line 1183, in _next_data
return self._process_data(data)
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/dataloader.py", line 1229, in _process_data
data.reraise()
File "/usr/local/lib/python3.8/dist-packages/torch/_utils.py", line 425, in reraise
raise self.exc_type(msg)
IndexError: Caught IndexError in DataLoader worker process 6.
Original Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop
data = fetcher.fetch(index)
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/model/CrowdCounting-P2PNet/crowd_datasets/SHHA/SHHA.py", line 73, in getitem
img, point = random_crop(img, point)
File "/home/model/CrowdCounting-P2PNet/crowd_datasets/SHHA/SHHA.py", line 133, in random_crop
idx = (den[:, 0] >= start_w) & (den[:, 0] <= end_w) & (den[:, 1] >= start_h) & (den[:, 1] <= end_h)
IndexError: too many indices for array: array is 1-dimensional, but 2 were indexed
您好,非常感谢您的代码,我在使用的时候有个问题,使用如下代码:
“CUDA_VISIBLE_DEVICES=0 python run_test.py --weight_path ./weights/SHTechA.pth --output_dir ./logs/”执行后得到:“Namespace(backbone='vgg16_bn', gpu_id=0, line=2, output_dir='./logs/', row=2, weight_path='./weights/SHTechA.pth')”
后面就没有反应了,代码没有任何更改。
有没有朋友在这几个数据集上实验过,什么样的参数设置可以取得比较好的结果?感谢分享
CrowdCounting-P2PNet/models/p2pnet.py
Line 278 in 5c91a81
这里是typo?应该是loss_points吧
RuntimeError: CUDA out of memory. Tried to allocate 88.00 MiB (GPU 0; 8.00 GiB total capacity; 6.17 GiB already allocated; 63.69 MiB free; 6.23 GiB reserved in total by PyTorch)
Traceback (most recent call last):
File "/home/mfo/Projects/crowd-counting-gui/demo_gui.py", line 16, in <module>
from models import build_model
File "/home/mfo/Projects/crowd-counting-gui/CrowdCounting-P2PNet/models/__init__.py", line 1, in <module>
from .p2pnet import build
File "/home/mfo/Projects/crowd-counting-gui/CrowdCounting-P2PNet/models/p2pnet.py", line 5, in <module>
from util.misc import (NestedTensor, nested_tensor_from_tensor_list,
File "/home/mfo/Projects/crowd-counting-gui/CrowdCounting-P2PNet/util/misc.py", line 26, in <module>
from torchvision.ops import _new_empty_tensor
ImportError: cannot import name '_new_empty_tensor' from 'torchvision.ops' (/home/mfo/miniconda3/envs/p2pnet/lib/python3.9/site-packages/torchvision/ops/__init__.py)
While reading the paper, I came to know that the authors used 13 convs from VGG16 network to extract deep features. Since VGG nets have been around for a quite a long time, why didn't you choose some more efficient and accurate networks? Or is there something I don't know of? Thank you.
CrowdCounting-P2PNet/engine.py
Line 146 in 5c91a81
Originally posted by @Tracyummy in #7 (comment)
Can vgg11_bn be used instead of vgg16_bn in the code?
I attempted to change the model to the pre-trained vgg11_bn model provided by PyTorch, but I encountered an error. The error message states that some keys are missing in the state_dict for the P2PNet model, such as backbone.body1.5.weight, backbone.body1.5.bias, backbone.body1.5.running_mean, backbone.body1.5.running_var, and many others. In addition, there are unexpected keys in the state_dict, such as backbone.body1.10.weight, backbone.body1.10.bias, backbone.body1.11.weight, backbone.body1.11.bias, and so on. There is also a size mismatch for some of the parameters, such as backbone.body1.4.weight and backbone.body1.4.bias.
I found this error:
RuntimeError: Error(s) in loading state_dict for P2PNet:
Missing key(s) in state_dict: "backbone.body1.5.weight", "backbone.body1.5.bias", "backbone.body1.5.running_mean", "backbone.body1.5.running_var", "backbone.body2.0.weight", "backbone.body2.0.bias", "backbone.body2.0.running_mean", "backbone.body2.0.running_var", "backbone.body2.3.weight", "backbone.body2.3.bias", "backbone.body2.3.running_mean", "backbone.body2.3.running_var", "backbone.body2.6.weight", "backbone.body2.6.bias", "backbone.body3.0.weight", "backbone.body3.0.bias", "backbone.body3.0.running_mean", "backbone.body3.0.running_var", "backbone.body3.3.weight", "backbone.body3.3.bias", "backbone.body3.3.running_mean", "backbone.body3.3.running_var", "backbone.body3.6.weight", "backbone.body3.6.bias", "backbone.body4.0.weight", "backbone.body4.0.bias", "backbone.body4.0.running_mean", "backbone.body4.0.running_var", "backbone.body4.3.weight", "backbone.body4.3.bias", "backbone.body4.3.running_mean", "backbone.body4.3.running_var".
Unexpected key(s) in state_dict: "backbone.body1.10.weight", "backbone.body1.10.bias", "backbone.body1.11.weight", "backbone.body1.11.bias", "backbone.body1.11.running_mean", "backbone.body1.11.running_var", "backbone.body1.11.num_batches_tracked", "backbone.body1.3.weight", "backbone.body1.3.bias", "backbone.body1.4.running_mean", "backbone.body1.4.running_var", "backbone.body1.4.num_batches_tracked", "backbone.body1.7.weight", "backbone.body1.7.bias", "backbone.body1.8.running_mean", "backbone.body1.8.running_var", "backbone.body1.8.num_batches_tracked", "backbone.body2.7.weight", "backbone.body2.7.bias", "backbone.body2.8.weight", "backbone.body2.8.bias", "backbone.body2.8.running_mean", "backbone.body2.8.running_var", "backbone.body2.8.num_batches_tracked", "backbone.body2.1.weight", "backbone.body2.1.bias", "backbone.body2.2.running_mean", "backbone.body2.2.running_var", "backbone.body2.2.num_batches_tracked", "backbone.body2.4.weight", "backbone.body2.4.bias", "backbone.body2.5.weight", "backbone.body2.5.bias", "backbone.body2.5.running_mean", "backbone.body2.5.running_var", "backbone.body2.5.num_batches_tracked", "backbone.body3.7.weight", "backbone.body3.7.bias", "backbone.body3.8.weight", "backbone.body3.8.bias", "backbone.body3.8.running_mean", "backbone.body3.8.running_var", "backbone.body3.8.num_batches_tracked", "backbone.body3.1.weight", "backbone.body3.1.bias", "backbone.body3.2.running_mean", "backbone.body3.2.running_var", "backbone.body3.2.num_batches_tracked", "backbone.body3.4.weight", "backbone.body3.4.bias", "backbone.body3.5.weight", "backbone.body3.5.bias", "backbone.body3.5.running_mean", "backbone.body3.5.running_var", "backbone.body3.5.num_batches_tracked", "backbone.body4.7.weight", "backbone.body4.7.bias", "backbone.body4.8.weight", "backbone.body4.8.bias", "backbone.body4.8.running_mean", "backbone.body4.8.running_var", "backbone.body4.8.num_batches_tracked", "backbone.body4.1.weight", "backbone.body4.1.bias", "backbone.body4.2.running_mean", "backbone.body4.2.running_var", "backbone.body4.2.num_batches_tracked", "backbone.body4.4.weight", "backbone.body4.4.bias", "backbone.body4.5.weight", "backbone.body4.5.bias", "backbone.body4.5.running_mean", "backbone.body4.5.running_var", "backbone.body4.5.num_batches_tracked".
size mismatch for backbone.body1.4.weight: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128, 64, 3, 3]).
size mismatch for backbone.body1.4.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.body1.8.weight: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256, 128, 3, 3]).
size mismatch for backbone.body1.8.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone.body2.2.weight: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([256, 256, 3, 3]).
size mismatch for backbone.body3.2.weight: copying a param with shape torch.Size([512]) from checkpoint, the shape in current model is torch.Size([512, 512, 3, 3]).
size mismatch for backbone.body4.2.weight: copying a param with shape torch.Size([512]) from checkpoint, the shape in current model is torch.Size([512, 512, 3, 3]).
I would greatly appreciate any response. Thank you!
Can anyone reproduce the accuracy of UCF-QNRF? I hava conducted acout ten experiments on UCF-QNRF, however, the MAE and RMSE is 100 and 180. I can not achieve the reported MAE:85 and RMSE:154.5, even I use the settings in this paper.
scene01,scene02这些怎么处理。比如shanghaiA
train.txt test.txt
Hello, thank you for this project. My question is, how can I have a video input instead of a single image? I would like to test it with a video or a webcam in real time.
Why I tested the best SHTechA.pth and got a result of 53 MAE?
Line 187 in 5c91a81
您好,我也是一样的情况,在出现了Namespace(backbone='vgg16_bn', gpu_id=0, line=2, output_dir='./logs/', row=2, weight_path='./weights/SHTechA.pth')之后,后面就没有反应了。不知道有没有顺利解决这个问题的
Originally posted by @fatakWang in #16 (comment)
你好,请问我使用ShanghaiTech_Crowd_Counting_Dataset/part_A_final数据进行训练,参数使用的是readme里面的参数,但是精度很差,是什么原因?
2779][lr 0.0001000][11.72s]mae:329.14285714285717, mse:466.4266075383996, time:10.338678121566772, best mae:295.75274725274727metric/mae@2778: 329.14285714285717metric/mse@2778: 466.4266075383996loss/loss@2780: 0.04982551398712236loss/loss_ce@2780: 0.04982551398712236[ep 2780][lr 0.0001000][12.00s]
Thank you for your great code and algorisms.
I'm trying to use this code, but still not work.
When I run train.py, I got this massage as blow.
Namespace(backbone='vgg16_bn', batch_size=8, checkpoints_dir='./ckpt', clip_max_norm=0.1, data_root='/content/drive/MyDrive/DATA_ROOT', dataset_file='SHHA', eos_coef=0.5, epochs=3500, eval=False, eval_freq=5, frozen_weights=None, gpu_id=0, line=2, lr=0.0001, lr_backbone=1e-05, lr_drop=3500, num_workers=8, output_dir='./logs', point_loss_coef=0.0002, resume='', row=2, seed=42, set_cost_class=1, set_cost_point=0.05, start_epoch=0, tensorboard_dir='./runs', weight_decay=0.0001)
number of params: 21579344
Start training
Averaged stats: Traceback (most recent call last):
File "train.py", line 222, in
main(args)
File "train.py", line 161, in main
args.clip_max_norm)
File "/content/CrowdCounting-P2PNet/engine.py", line 120, in train_one_epoch
print("Averaged stats:", metric_logger)
File "/content/CrowdCounting-P2PNet/util/misc.py", line 186, in str
"{}: {}".format(name, str(meter))
File "/content/CrowdCounting-P2PNet/util/misc.py", line 85, in str
median=self.median,
File "/content/CrowdCounting-P2PNet/util/misc.py", line 64, in median
return d.median().item()
RuntimeError: median cannot be called with empty tensor
I seem it cannot read dataset from list file.
my train.list is
train/RIMG6463.JPG train/RIMG6463.txt
train/RIMG6464.JPG train/RIMG6464.txt
train/RIMG6465.JPG train/RIMG6465.txt
train/RIMG6466.JPG train/RIMG6466.txt
train/RIMG6467.JPG train/RIMG6467.txt
that under DATA_ROOT.
What is wrong about this?
Thank you for your help.
Is it possible to use it for other tasks such as animal detection or objects other than human?
Hi,
I could reproduce MAE results on ShanghaiTech part A using supplied arguments for train script. However, with same parameters, for ShanghaiTech part B I can not reproduce results, getting a MAE of 11.18 instead of article results which get a 6.25 .
I have created an script to build train files out of ShanghaiTech datasets and I can confirm that this part is ok because I can reproduce MAE with generated files (for part A).
For partB, I have tried with same supplied parameters and also changing learning rate but MAE does not go below 11.
Any more tweaks to be done in dataset loading? Any different parameter values?
Many thanks for your great article and code,
Ricard
你好,何时开放training代码,每天都在盼星星盼月亮地期待着。
Please let us know your parameter settings of QNRF to achieve the results reported in the paper.
Thank you in advance.
I am getting below error while running this on CPU machine(i haven't installed Cuda enabled Pytorch as my computer don't have GPU)
AssertionError: Torch not compiled with CUDA enabled
can anyone please guide me where to get ShanghaiTechA dataset in a format given in repository, as I find the dataset containing .mat file.
Awesome work!
But it seems that the distributed training doesn't work, i.e., the argument --gpu_id
is unused.
This is very meaningful work.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.