Giter VIP home page Giter VIP logo

retinaface-cpp's Introduction

RetinaFace-Cpp

RetinaFace detector with C++

official RetinaFace

I convert mobilenet-0.25 mxnet model (trained by yangfly) to caffe model

I convert R50 mxnet model to caffe model BaiDuYun密码:6evh | Google Drive

  • I have checked the output of the two models be the same.

  • For same input images, the output of the two detector (python version and cpp version) is same.

(the code is too simple, only for reference 23333)

This is anchor parsing demo, NO INFERENCE, you should replace your own inference header and code (caffe/ncnn/feather .etc) in the source code

I create QQ group: 760688309 for communicating


Update 2019.6.4

I update time test tools for ncnn and rokidnn, only arm64-v8a version (I'm too lazy to compile armeabi-v7a version, if you need, I can update later)

how to run

use adb push to device and run ./retina_det imagepath modelpath thread_num loop_num

It will calculate average time (including input time, forward time and anchor parsing time, nms time) the output image is saved as "res.jpg"

Time Benchmark

VGA (640x480)

RokidNN vs NCNN rokidnn_vs_ncnn

Next week I will provide apk built with NCNN and RokidNN for you to compare.

platform net threads 1/2/4(ms) inference
qcom625 mnet 418/262/202 NCNN
qcom625 mnet 379/244/180 NCNN(optmize)
qcom835 mnet 137.37/82.97/62.79 by hanson-young NCNN
qcom835 mnet 125.10/74.52/75.09 by hanson-young NCNN(optmize)

Update 2019.5.28

R50: Because of the GPU memory limited, I set max(width, height) to 1000 and test it on WiderFace_val set for SINGLE SCALE, NO MULTI-SCALE, NO FLIP, results as follows:

wider val easy medium hard
python version 92.25 88.86 64.02
cpp version 91.36 87.15 62.22

Update 2019.5.27

mnet: I test on WiderFace_val set for SINGLE SCALE, NO MULTI-SCALE, NO FLIP, results as follows:

wider val easy medium hard
python version 83.28 77.02 39.52
cpp version 83.04 76.84 39.43

You can see the two detect results as results/cpp_detect and results/python_detect. The main cause of the difference in test results is the numerical precision difference between C++ and python.

TODO

  • implement bbox_vote
  • multi-scale test
  • speed benchmark (CPU/ARM)
  • new models

retinaface-cpp's People

Contributors

charrin avatar hanson-young avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

retinaface-cpp's Issues

MXNet2caffe

你好,请问可以开源你MXNet2caffe的代码吗,我模型转换碰到问题了

Different size for input image

when i modify the size of input image to 640 * 640, got no result for stride8, but got the result for stride16 and stride32. but I don't know why.

Cube not defined

int FilterAnchor(const Cube* cls, const Cube* reg, const Cube* pts, std::vector& result);

@Charrin you must forget somthing...

the inference time not stable

inference time 11.890137
inference time 77.138916
inference time 14.939941
inference time 153.685059
inference time 11.651123
inference time 11.635986
inference time 18.880127
inference time 21.743896
inference time 18.770996
inference time 210.691895
inference time 15.924072
inference time 13.067871
inference time 11.803955
inference time 11.813965
inference time 12.151123
face detection time cost: min = 8.04ms max = 733.61ms avg = 64.47ms
why the inference time are not stable ?

批量裁剪图片时,预测出错

您好,我更改了您的代码用于批量裁剪图片。结果在裁剪了数张图之后,预测出现问题,返回错误坐标。
YNEX3H2%Z)ZY FE9}_%5$47
6ZUM`L4QD(Z@8@71_XZ2CWV

这是我更改后的Main.cpp

#include <stdio.h>
#include <vector>
#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc/imgproc.hpp>

#include <iostream>
#include <string>
#include <fstream>
#include <sstream>
#include <stdlib.h>
#include <cstdio>

#include "platform.h"
#include "net.h"
#include<windows.h>
#include "anchor_generator.h"
#include "config.h"
#include "tools.h"

using namespace std;

const int target_size = 300;
//------------------------------------------------------------------------------------------
struct Object
{
	cv::Rect_<float> rect;
	int label;
	float prob;
};

static int detect_mobilenet(const cv::Mat& bgr, std::vector<Anchor>& result, const char* paramPath, const char* modelPath)
{
	ncnn::Net retina;

	retina.load_param(paramPath);
	retina.load_model(modelPath);

	int img_w = bgr.cols;
	int img_h = bgr.rows;

	ncnn::Mat in = ncnn::Mat::from_pixels_resize(bgr.data, ncnn::Mat::PIXEL_BGR2RGB, bgr.cols, bgr.rows, img_w, img_h);
	//ncnn::Mat in = ncnn::Mat::from_pixels_resize(bgr.data, ncnn::Mat::PIXEL_BGR, bgr.cols, bgr.rows, target_size, target_size);
	
	//const float mean_vals[3] = { 103.939f, 116.779f, 123.68f }; //retina
	//const float norm_vals[3] = { 1.0 / mean_vals[0],1.0 / mean_vals[1],1.0 / mean_vals[2] };
	const float mean_vals[3] = { 0,0,0 }; //retina
	const float norm_vals[3] = { 1,1,1 };
	in.substract_mean_normalize(mean_vals, norm_vals);

	ncnn::Extractor ex = retina.create_extractor();

	ex.input("data", in);

	//--------------------------------------------------------
	std::vector<AnchorGenerator> ac(_feat_stride_fpn.size());
	for (int i = 0; i < _feat_stride_fpn.size(); ++i) {
		int stride = _feat_stride_fpn[i];
		ac[i].Init(stride, anchor_cfg[stride], false);
	}
	std::vector<Anchor> proposals;
	proposals.clear();

	for (int i = 0; i < _feat_stride_fpn.size(); ++i) {
		ncnn::Mat cls;
		ncnn::Mat reg;
		ncnn::Mat pts;

		char clsname[100]; sprintf(clsname, "face_rpn_cls_prob_reshape_stride%d", _feat_stride_fpn[i]);
		char regname[100]; sprintf(regname, "face_rpn_bbox_pred_stride%d", _feat_stride_fpn[i]);
		char ptsname[100]; sprintf(ptsname, "face_rpn_landmark_pred_stride%d", _feat_stride_fpn[i]);

		ex.extract(clsname, cls);
		ex.extract(regname, reg);
		ex.extract(ptsname, pts);

		ac[i].FilterAnchor(cls, reg, pts, proposals);

	}
	
	nms_cpu(proposals, nms_threshold, result);

	retina.clear();
	ac.clear();
	proposals.clear();

	return 0;
}

void crop_objects(cv::Mat& bgr, std::vector<Anchor> result, const char* imagepath) {
	//resize(bgr, bgr, cv::Size(target_size, target_size));
	cv::Mat image = bgr.clone();
	
	if (result.size() == 0) {
		cout << "[ " << imagepath << " ] crop fail!\n" << endl;
		system("pause");
		exit(0);
	}
	Anchor res = result[0];
	for (int i = 1; i < result.size(); i++)
	{	
		cout << res.score << endl;
		cout << result[i].score << endl;

		if (res.score < result[i].score) {
			res = result[i];
			cout << "!" << endl;
		}

	}
    
	cout << res.score << endl << endl;
	cout << res.finalbox.x << " " << res.finalbox.y << " " << res.finalbox.width << " " << res.finalbox.height << endl;

	float w,h;
	w = h = 0;
	w = res.finalbox.width - res.finalbox.x + 1;
	h = res.finalbox.height - res.finalbox.y + 1;

	if (res.finalbox.x < 0) {
		res.finalbox.x = 0;
	}
	if ((res.finalbox.x + w) > image.cols) {
		w = image.cols - res.finalbox.x;
	}

	if (res.finalbox.y < 0) {
		res.finalbox.y = 0;
	}
	if ((res.finalbox.y + h) > image.rows) {
		h = image.rows - res.finalbox.y;
	}

	cout << res.finalbox.x << " " << res.finalbox.y << " " << w << " " << h << endl;
	image = bgr(cvRect(res.finalbox.x, res.finalbox.y, w, h));

	cv::imwrite(imagepath, image);
	//cv::imshow("t", image);
	//cv::waitKey(true);
	//Sleep(200);
	cout << "[ " << imagepath << " ] crop complete!\n" << endl;
}

int main(int argc, char** argv)
{
	string cropListPathStr = "C:/Users/SEARECLUSE/Desktop/RetinaFaceDemo/x64/Release/dataList.txt";
	string paramPathStr = "C:/Users/SEARECLUSE/Desktop/RetinaFaceDemo/x64/Release/model/mnet.ncnn.param";
	string modelPathStr = "C:/Users/SEARECLUSE/Desktop/RetinaFaceDemo/x64/Release/model/mnet.ncnn.bin";

	if (argc > 1) {
		cropListPathStr = argv[1];
		paramPathStr = argv[2];
		modelPathStr = argv[3];
	}

	cout << "确认裁剪列表: " << cropListPathStr << endl;
	cout << "确认模型结构: " << paramPathStr << endl;
	cout << "确认模型文件:" << modelPathStr << endl;
	cout << "准备就绪。" << endl;
	system("pause");

	const char* cropListPath = cropListPathStr.data();
	const char* paramPath = paramPathStr.data();
	const char* modelPath = modelPathStr.data();

	ifstream infile;
	infile.open(cropListPath, ios::in);
	while (!infile.eof()) {
		string str;
		getline(infile, str);

		const char* imagepath = str.data();
		cv::Mat m = cv::imread(imagepath, 1);

		cout << imagepath << endl;

		if (m.empty())
		{
			fprintf(stderr, "cv::imread %s failed\n", imagepath);
			return -1;
		}

		std::vector<Anchor> result;
		detect_mobilenet(m, result, paramPath, modelPath);
		crop_objects(m, result, imagepath);
		result.clear();
	}

	system("pause");
	return 0;
}

crop

作者你好,crop时特征图有时大小刚好相等,也有特征图不等的情况,那么offset是怎么设置的呢,prototxt中没有offset的值,caffe这个层不太懂,没有设置offset是默认居中裁剪吗?

使用vulkan 进行GPU加速

您好, 已经跑通您的这个程序, 但我使用Vulkan的时候, 发现不工作 (ncnn 提供的例子可以使用gpu加速),
ncnn::Net _net;

ncnn::create_gpu_instance();
_net.opt.use_vulkan_compute = 1;
int gpu_count = ncnn::get_gpu_count();
ncnn::VulkanDevice vkdev; 
_net.set_vulkan_device(&vkdev);

_net.load_param(param_path.data());
_net.load_model(bin_path.data());

mxnet model convert to ncnn

您好,
请问你的ncnn模型是用的 mnet.25-symbol.json mnet.25-0000.params 这两个文件转的吗,
我用ncnn最新代码转,运行会报错.查看了一下转换后的ncnn模型和你工程中的差别蛮大的, 你转换的时候用了什么技巧吗,你ncnn的tag 是多少

谢谢

模型问题

请问
RetinaFace-Cpp/Demo/ncnn/models/
这个目录下的
retina.bin 和 retina.param
是对应于您该链接中:
I convert R50 mxnet model to caffe model BaiDuYun密码:6evh | Google Drive
中提供的模型吗

Run on CPU only

Hello,
will this run on CPU?
I can't spot where you are loading the model.

关于用MNN推理框架在树莓派4B推理速度

测试了一下caffe的mnet模型在树莓派4B上的速度,推理框架用的阿里的MNN,树莓派4B的cpu型号是BCM2711(四核Cortex A72,主频1.5GHz),测试分辨率为VGA (640*480),loop10次取平均:

核心数 fp32计算耗时(ms) 量化后int8计算耗时(ms)
1 167 183
2 116 102
3 105 76
4 96 61

MNN框架的加速优化还是做得挺不错的。

about Cube struct

Cube cls;
        Cube reg;
        Cube pts;

        // get blob output
        char clsname[100]; sprintf(clsname, "face_rpn_cls_prob_reshape_stride%d", _feat_stride_fpn[i]);
        char regname[100]; sprintf(regname, "face_rpn_bbox_pred_stride%d", _feat_stride_fpn[i]);
        char ptsname[100]; sprintf(ptsname, "face_rpn_landmark_pred_stride%d", _feat_stride_fpn[i]);

        net.Extract(nc, clsname, cls);
        net.Extract(nc, regname, reg);
        net.Extract(nc, ptsname, pts);
```when i use caffe C++, i find Cube not defined. Can you show me its structure?

Any Speedups Compared to Python?

Thanks for the conversion.

Is there any speedups compared to the Python version? Can you provide some simple benchmarks?

Thanks

Onnx model

Hi! Could you provide onnx model for R50?

is crop necessary?

both input of the crop layer have identical shape. Do we really need crop layer?

About arm platform (ncnn model)

Do you have any plans to transfer to ncnn on arm platform? I failed to convert ncnn with caffe model which you provide.

:~/Documents/3rdpart/ncnn/build/tools/caffe$ ./caffe2ncnn ./mnet.prototxt ./mnet.prototxt.caffemodel ./retina.param ./retina.bin
Segmentation fault (core dumped)

运行环境

您好, 这个代码 运行的环境,您能详细说下吗,非常感谢!

R50 mxnet convert to ncnn parameter file has inconsistent

1、convert R50 use mxnet2ncnn then load model
std::string param_path = "./models/retina-R50.param";
std::string bin_path = "./models/retina-R50.bin";
ncnn::Net _net;
_net.load_param(param_path.data());
_net.load_model(bin_path.data());

2、when run it error in _extractor.extract(clsname, cls) error:load_model error at layer 282, parameter file has inconsistent
clsname char [100] "face_rpn_cls_prob_reshape_stride16"

Is the output same?

Hi,
Thanks for your good job.
I tried your retinaface-R50 caffe model, the output size is not same as the MXNET output.
For example, the stride 8 output size, if the input size is 640x640, the caffe model output size is 81x81, but MXNET output size is 80x80

Is it same as your caffe model output?

why resize to 300x300 before extraction

hi @Charrin ,

ncnn::Mat input = ncnn::Mat::from_pixels_resize(img.data, ncnn::Mat::PIXEL_BGR2RGB, img.cols, img.rows, 300, 300);
cv::resize(img, img, cv::Size(300, 300));

300x300 is really small for bigger images, is it the network size? however, I found the param is 640:
Input data 0 1 data 0=640 1=640 2=3

How can I elevate the model performance?

The inference cost time is more than 200ms in my device... while I use 4 threads to match the core num... (network input size is 640*480)
However , when I use the inference in "SpeedTest", The forward time is less than 100ms, remains 4 thread...
Could U share the SpeedTest's source code? Thank you

error C2065: “anchor”: 未声明的标识符

void AnchorGenerator::landmark_pred(const CRect2f anchor, const std::vectorcv::Point2f& delta, std::vectorcv::Point2f& pts) {
float w = anchor[2] - anchor[0] + 1;
float h = anchor[3] - anchor[1] + 1;
float x_ctr = anchor[0] + 0.5 * (w - 1);
float y_ctr = anchor[1] + 0.5 * (h - 1);
pts.resize(delta.size());
for (int i = 0; i < delta.size(); ++i) {
pts[i].x = (delta[i].x*w + x_ctr)ratiow;
pts[i].y = (delta[i].y
h + y_ctr)*ratioh;
}
}中出现该问题, @Charrin

How to run inference using provided models

How to run inference using the model provided in convert_models folder ?
What piece of code should I place in the space provided in "detect.cpp" ?
What are the preferred libraries to be used ?
The detect.cpp file mentions "Inference ,NetContext and Cube objects" . Where can I find these classes ? (Line number 16 17 and 37 respectively

Got different output between Mxnet and caffe anything wrong?

I'm using mobilenet-0.25 Retinaface on TensorRT & caffe. They getting the same output value but no bbox output. Then I compare the output of caffe to Mxnet, it is different(Mxnet can get the bbox).
The preprocess is like below
`cv::Mat img = cv::imread("./test.jpg");

    cv::cvtColor(img, img, CV_BGR2RGB);

cv::resize(img, img, cv::Size(INPUT_H, INPUT_W));

cv::Mat frame_copy = img.clone();

cv::resize(frame_copy, frame_copy, cv::Size(INPUT_W, INPUT_H));

float* data = new float[INPUT_C*INPUT_H*INPUT_W];

std::vector<cv::Mat> fd_input_channels;

for (int i = 0; i < 3; ++i) {
	cv::Mat channel(INPUT_H, INPUT_W, CV_32FC1, data);
	fd_input_channels.push_back(channel);
	data += INPUT_H * INPUT_W;
}

cv::Mat sample_float;

frame_copy.convertTo(sample_float, CV_32FC3, 1.0 / 128, -127.5 / 128);

cv::split(sample_float, fd_input_channels);`

The input is normalize to -1~1 planar,right?

Widerface valuation

Hi,Charrin:
what format should I save the widerface test result in order to test the widerface val dataset,I could not find a doc that explain it.Thanks.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.