The pillcounter from jeunetoujour

image size and scale, plus other thoughts...

I found major undercounts (eg 14 instead of 500) for some images, which I was able to fix by pre-enlarging the image, up to the largest that I could use before the python code ran out of memory.

Not knowing enough python to fix it in the script, I did it from the command-line instead, with this 'count' script which takes the image file name as its parameter:

#!/bin/bash
convert $1 -resize 8244x6183 +repage pills.jpg
RESPONSE=`curl --silent --request POST -F "[email protected]" http://127.0.0.1:5000/upload`
COUNT=`echo $RESPONSE | awk -F":" '{print $2}'`
echo `basename $1 .jpg`,$COUNT,`date +%Y%m%d`

I'm using cell-phone photos at 2576x1932 resolution.

I couldn't get cv2 to install under python3 so everything had to be python2. There are tweaks in the python script below to make it work with cv2. (There's also some commented-out debugging)

11c11
< UPLOAD_FOLDER = '/home/slewis0/uploads'
---
> UPLOAD_FOLDER = '/home/gtoal/src/pillcounter/uploads'
25c25
<     print("Inside")
---
>     ##print("Inside")
83c83,84
<             _, contours, hierarchy = cv2.findContours(
---
>             ##_, contours, hierarchy = cv2.findContours(
>             contours, hierarchy = cv2.findContours(
113c114,115
<             _, contours, hierarchy = cv2.findContours(
---
>             ##_, contours, hierarchy = cv2.findContours(
>             contours, hierarchy = cv2.findContours(
147c149,150
<             _, contours, hierarchy = cv2.findContours(
---
>             ##_, contours, hierarchy = cv2.findContours(
>             contours, hierarchy = cv2.findContours(
185c188,189
<               _, contours, hierarchy = cv2.findContours(
---
>               ##_, contours, hierarchy = cv2.findContours(
>               contours, hierarchy = cv2.findContours(
232c236,237
<                     _, new_contours, new_hierarchy = cv2.findContours(
---
>                     ##_, new_contours, new_hierarchy = cv2.findContours(
>                     new_contours, new_hierarchy = cv2.findContours(
243c248,249
<                     _, new_contours, new_hierarchy = cv2.findContours(
---
>                     ##_, new_contours, new_hierarchy = cv2.findContours(
>                     new_contours, new_hierarchy = cv2.findContours(
327c333,334
<             _, contours, hierarchy = cv2.findContours(
---
>             ##_, contours, hierarchy = cv2.findContours(
>             contours, hierarchy = cv2.findContours(
414a422,423
>     ##print "contours:"
>     ##print contours
418c427,430
<         bisect.insort(areaarray, cv2.contourArea(cnt))
---
>         temp = cv2.contourArea(cnt)
>         ##print "Temp:"
>         ##print temp
>         bisect.insort(areaarray, temp)
422a435,438
>     ##
>     ##print "Index = " + str(index)
>     ##print areaarray
>     ##
458c474
<     print ("New trueavg = " + str(trueavg))
---
>     ##print ("New trueavg = " + str(trueavg))
463c479
<     app.run(host='localhost', port=5000)
---
>     app.run(host='0.0.0.0', port=5000)

The upload directory doesn't have to be hard-coded. It could use an 'upload' folder in the same directory as the python program, with the python program name and path being determined at runtime. (At least on unix/linux. I know how to do this in bash but can't help with a python implementation.)

The times when 100's of pills were reported as a handful were easy to spot, but a more insidious error was where say 500 were reported as 300 - easy to miss. I have a suggestion for adding an internal consistency check: create a histogram of blob sizes, and determine the pixel area of a single pill from the most frequent blob size. Then for blobs greater than one pill, divide the pixel area of the blob by the pixel area of one pill to estimate the number of pills in the blob and present it as extra information, eg "214 pills plus possibly 120 more that I am not sure of"...

Another issue: sometimes you can have two different versions of the same medicament in the one pill jar. (ie a refill came from a different vendor and you keep them all in the one container for simplicity). Different versions can be a different shape, size, or colour!
Possibly report this as "You have 66 pills of one type and 14 of another".

I have some pills that have been cut in half for when only a half dose is wanted. I picked those out before photographing and noted them separately in my inventory spreadsheet. If half pills could be recognised that would be great, but it's not critical. Half pills could possibly be handled by either the area counting or the alternative shape suggestions.

I'll attach my pill images so you can have more real-life test data.