Comments (16)
V33m
commented on September 26, 2024
1
Just a heads-up regarding using hdparm to query hard drives as hdparm is known to wake-up drives when doing so. Consider using smartctl by smartmontools instead to query a device to see the status by using smartctl -i -n standby /dev/sdX.
from hd-idle.
adelolmo
commented on September 26, 2024
1
In /etc/default/hd-idle. Just append it to the variable HD_IDLE_OPTS.
e.g.
HD_IDLE_OPTS="-i 600 -c ata -a /dev/disk/by-partlabel/disk2 -c scsi -a /dev/disk/by-label/parity1 -c scsi -l /var/log/hd-idle.log -d"
from hd-idle.
adelolmo
commented on September 26, 2024
1
Thank you for the logs.
After all it seems that you found a bug. I was reading (what it seems) the wrong fields on /proc/diskstats for checking reads and writes.
So far you have the only system where this bug can be tested. I'd like to ask you to test the fixed version before I release it.
Could you build the package and test it? or should I provide you with the binary?
This is the branch with the fix: https://github.com/adelolmo/hd-idle/tree/fix/read-writes
from hd-idle.
V33m
commented on September 26, 2024
1
It's not up to me, but I guess it's probably faster (and easier) if you build the amd64 package, adelolmo π
from hd-idle.
adelolmo
commented on September 26, 2024
1
The fix is included in the latest release: https://github.com/adelolmo/hd-idle/releases/tag/v1.10
@auanasgheps Let me know if it works any better
from hd-idle.
auanasgheps
commented on September 26, 2024
1
@adelolmo
The update works great, take a look at these wonderful spindown times!
Thank you :)
Aug 9 20:41:51 nas hd-idle[21056]: disk=sdc command=scsi spunDown=true reads=216823770 writes=8556274 idleTime=600 idleDuration=27934 spindown=2020-08-09T13:06:25 spinup=2020-08-09T12:56:16 lastIO=2020-08-09T12:56:16
Aug 9 20:41:51 nas hd-idle[21056]: disk=sdb command=scsi spunDown=true reads=218896299 writes=2040018 idleTime=1200 idleDuration=9637 spindown=2020-08-09T18:21:34 spinup=2020-08-09T18:01:14 lastIO=2020-08-09T18:01:14
Aug 9 20:41:51 nas hd-idle[21056]: disk=sda command=scsi spunDown=true reads=203668732 writes=2119898 idleTime=1200 idleDuration=9637 spindown=2020-08-09T18:21:34 spinup=2020-08-09T18:01:14 lastIO=2020-08-09T18:01:14
from hd-idle.
auanasgheps
commented on September 26, 2024
Just a heads-up regarding using hdparm to query hard drives as hdparm is known to wake-up drives when doing so. Consider using smartctl by smartmontools instead to query a device to see the status by using
smartctl -i -n standby /dev/sdX.
Thanks, sometimes I woke up my drives using hdparm but happened rarely. I will try smartctl.
from hd-idle.
adelolmo
commented on September 26, 2024
hd-idle relies on /proc/diskstats to determine disk activity. It implements a simple state machine with only two stages (disk spunup or disk spundown). It will transition from down to up only if it detects reads or writes on the disk. This info is retrieved from the 4th and 8th columns in /proc/diskstats (reads and writes respectively). e.g. here
My guess is that /proc/diskstats changed and that's why the disk state changed from spundown to spunup.
You can enable debug mode by adding the flag -d.
It is extremely verbose, but it might bring some light to the issue.
hd-idle[24300]: disk=sda command=ata spunDown=true reads=7940696 writes=1177729 idleTime=600 idleDuration=42840 spindown=2020-08-06T12:17:32 spinup=2020-08-06T12:01:32 lastIO=2020-08-06T12:06:32
hd-idle[24300]: disk=sdb command=ata spunDown=true reads=12651447 writes=30483 idleTime=600 idleDuration=42180 spindown=2020-08-06T12:28:32 spinup=2020-08-06T12:17:32 lastIO=2020-08-06T12:17:32
hd-idle[24300]: sdc spinup
hd-idle[24300]: disk=sdc command=scsi spunDown=false reads=11690123 writes=90095 idleTime=600 idleDuration=0 spindown=2020-08-06T17:08:32 spinup=2020-08-07T00:00:32 lastIO=2020-08-07T00:00:32
hd-idle[24300]: disk=sdd command=scsi spunDown=true reads=36306043 writes=1147380 idleTime=600 idleDuration=95579 spindown=2020-08-05T21:38:32 spinup=2020-08-05T21:27:32 lastIO=2020-08-05T21:27:32
lastIO indicates last read/write on the disk.
from hd-idle.
auanasgheps
commented on September 26, 2024
Here's another example from today.
Currently drives are in standby. They have been used about until 12:15. Hd-idle is set to spindown drives after 20 minutes so I believe at 12:36-37 they actually went to standby.
Aug 7 12:36:05 nas hd-idle[916]: sda spindown
Aug 7 12:36:05 nas hd-idle[916]: sdb spindown
Aug 7 12:37:06 nas hd-idle[916]: sdc spindown
Aug 7 12:37:06 nas hd-idle[916]: sda spinup
Aug 7 12:37:06 nas hd-idle[916]: sdb spinup
Aug 7 12:38:07 nas hd-idle[916]: sdc spinup
Aug 7 12:51:20 nas hd-idle[916]: sdc spindown
Aug 7 12:52:21 nas hd-idle[916]: sdc spinup
But logs let me believe they have immediately spun-up.
The command below as suggested by @V33m confirms that the standby is actually working
I don't know if this helps, but I have seen a similar behavior on another system I just installed a couple of days ago, in a similar software combination: hd-idle 1.9, Debian 10 (openmediavault 5). I haven't done this kind of troubleshooting on that system, but I went to look on mine and discovered the same log behavior, hence reporting it here.
from hd-idle.
adelolmo
commented on September 26, 2024
Here you can find an explanation on the hd-idle log question.
I understand that there's room for improvement in the README, so i'm planning on adding the explanation to it.
from hd-idle.
auanasgheps
commented on September 26, 2024
hey @adelolmo, I sent the last message and then I saw your updates.
Thanks for the log explanation, will look into it. If I get it right I might create a PR.
My guess is that
/proc/diskstatschanged and that's why the disk state changed from spundown to spunup.
You can enable debug mode by adding the flag-d.
Can you please detail where I should add the -d flag?
Also worth noting that when I implemented hd-ide 1.9 in March, this log behavior was not happening. I checked logs for a week because I was obsessed by the old hd-idle not working properly and everything looked fine. Obviously since then my system has been running system updates so maybe a 3rd party has broken logs?
from hd-idle.
auanasgheps
commented on September 26, 2024
Thanks, will do it now.
In the meantime, I was looking at hd-idle and found out when the behaviour changed, and I see a new pattern since the 11th May in the morning.
10th May was Sunday and some updates were applied. Here's what changed that day.
The following packages will be REMOVED:
python3-tornado4
The following NEW packages will be installed:
linux-image-5.5.0-0.bpo.2-amd64
The following packages will be upgraded:
base-files borgbackup distro-info-data fuse iputils-ping libcups2 libfuse2
libnss-myhostname libpango-1.0-0 libpangocairo-1.0-0 libpangoft2-1.0-0
libsystemd0 libudev1 linux-image-amd64 openmediavault postfix proftpd-basic
salt-common salt-minion systemd systemd-sysv tzdata udev wpasupplicant
Then when the server rebooted on 11th May morning, the behaviour changed.
This is my current config, which is dated to 25th March (file edit timestamp).
START_HD_IDLE=true
###Spindown parity disk after 10 mins, others after 20
HD_IDLE_OPTS="-i 0 -a /dev/disk/by-label/HDD1 -i 600 -a /dev/disk/by-label/HDD2 -i 1200 -a /dev/disk/by-label/HDD3 -i 1200 -l /var/log/hd-idle.log"
And here's the hd-idle extract from that time. Ignore the time gap, my server is not always on.
date: 2020-05-10, time: 18:47:18, disk: sda, running: 1281, stopped: 1524
date: 2020-05-10, time: 18:48:19, disk: sdb, running: 1220, stopped: 1646
date: 2020-05-10, time: 21:29:56, disk: sda, running: 1281, stopped: 8476
date: 2020-05-10, time: 21:30:57, disk: sdb, running: 2257, stopped: 7500
date: 2020-05-10, time: 22:30:55, disk: sdb, running: 1219, stopped: 2379
date: 2020-05-10, time: 22:30:55, disk: sda, running: 1280, stopped: 2379
date: 2020-05-10, time: 23:05:28, disk: sdb, running: 1219, stopped: 853
date: 2020-05-10, time: 23:05:28, disk: sda, running: 1219, stopped: 853
date: 2020-05-11, time: 00:00:21, disk: sdb, running: 1402, stopped: 1890
date: 2020-05-11, time: 00:00:21, disk: sda, running: 1523, stopped: 1769
date: 2020-05-11, time: 00:01:22, disk: sdc, running: 671, stopped: 35186
date: 2020-05-11, time: 11:12:46, disk: sdc, running: 610, stopped: 60
date: 2020-05-11, time: 11:23:57, disk: sda, running: 1281, stopped: 60
date: 2020-05-11, time: 11:23:57, disk: sdb, running: 1281, stopped: 60
date: 2020-05-11, time: 11:23:57, disk: sdc, running: 610, stopped: 61
date: 2020-05-11, time: 11:35:07, disk: sdc, running: 608, stopped: 61
date: 2020-05-11, time: 11:46:17, disk: sdc, running: 609, stopped: 61
date: 2020-05-11, time: 11:57:28, disk: sdc, running: 609, stopped: 61
from hd-idle.
adelolmo
commented on September 26, 2024
Let me explain what this log is telling:
date: 2020-05-11, time: 11:57:28, disk: sdc, running: 609, stopped: 61
The disk sdc spunup at 11:57:28 after 61 seconds spundown. The previous running period lasted 609 seconds.
This is certainly not good, because the disk should be for way longer periods on standby.
This one looks way better:
date: 2020-05-11, time: 00:01:22, disk: sdc, running: 671, stopped: 35186
35186 seconds of standby.
from hd-idle.
auanasgheps
commented on September 26, 2024
Yeah, now I've understood what these logs tell
All logs now seem to tell this, but it does not match what I can measure: spindown just working as expected.
I will soon post results with the-d command.
from hd-idle.
auanasgheps
commented on September 26, 2024
Here are the verbose logs.
Ignore sdd, it's not in hd-idle config and is not an hard drive.
According to the smart plug I use to monitor the server, during this period drives were down all the time. If drives were spinning up, I would have seen higher power consumption.
Aug 8 16:35:10 nas hd-idle[923]: sda spindown
Aug 8 16:35:10 nas hd-idle[923]: disk=sda command=scsi spunDown=true reads=443749 writes=415 idleTime=1200 idleDuration=1219 spindown=2020-08-08T16:35:10 spinup=2020-08-08T16:14:50 lastIO=2020-08-08T16:14:50
Aug 8 16:35:10 nas hd-idle[923]: disk=sdb command=scsi spunDown=false reads=424454 writes=341 idleTime=600 idleDuration=365 spindown=2020-08-08T16:28:03 spinup=2020-08-08T16:29:04 lastIO=2020-08-08T16:29:04
Aug 8 16:35:10 nas hd-idle[923]: disk=sdd command=scsi spunDown=false reads=11100 writes=3973 idleTime=0 idleDuration=0 spindown=0001-01-01T00:00:00 spinup=2020-08-08T11:51:36 lastIO=2020-08-08T16:35:10
Aug 8 16:36:11 nas hd-idle[923]: sdc spindown
Aug 8 16:36:11 nas hd-idle[923]: disk=sdc command=scsi spunDown=true reads=366447 writes=403 idleTime=1200 idleDuration=1219 spindown=2020-08-08T16:36:11 spinup=2020-08-08T16:15:51 lastIO=2020-08-08T16:15:51
Aug 8 16:36:11 nas hd-idle[923]: sda spinup
Aug 8 16:36:11 nas hd-idle[923]: disk=sda command=scsi spunDown=false reads=443750 writes=415 idleTime=1200 idleDuration=0 spindown=2020-08-08T16:35:10 spinup=2020-08-08T16:36:11 lastIO=2020-08-08T16:36:11
Aug 8 16:36:11 nas hd-idle[923]: disk=sdb command=scsi spunDown=false reads=424454 writes=341 idleTime=600 idleDuration=426 spindown=2020-08-08T16:28:03 spinup=2020-08-08T16:29:04 lastIO=2020-08-08T16:29:04
Aug 8 16:36:11 nas hd-idle[923]: disk=sdd command=scsi spunDown=false reads=11100 writes=3992 idleTime=0 idleDuration=0 spindown=0001-01-01T00:00:00 spinup=2020-08-08T11:51:36 lastIO=2020-08-08T16:36:11
Aug 8 16:37:12 nas hd-idle[923]: sdc spinup
Aug 8 16:37:12 nas hd-idle[923]: disk=sdc command=scsi spunDown=false reads=366448 writes=403 idleTime=1200 idleDuration=0 spindown=2020-08-08T16:36:11 spinup=2020-08-08T16:37:12 lastIO=2020-08-08T16:37:12
Aug 8 16:37:12 nas hd-idle[923]: disk=sda command=scsi spunDown=false reads=443750 writes=415 idleTime=1200 idleDuration=61 spindown=2020-08-08T16:35:10 spinup=2020-08-08T16:36:11 lastIO=2020-08-08T16:36:11
Aug 8 16:37:12 nas hd-idle[923]: disk=sdb command=scsi spunDown=false reads=424454 writes=341 idleTime=600 idleDuration=487 spindown=2020-08-08T16:28:03 spinup=2020-08-08T16:29:04 lastIO=2020-08-08T16:29:04
Aug 8 16:37:12 nas hd-idle[923]: disk=sdd command=scsi spunDown=false reads=11100 writes=3996 idleTime=0 idleDuration=0 spindown=0001-01-01T00:00:00 spinup=2020-08-08T11:51:36 lastIO=2020-08-08T16:37:12
Aug 8 16:38:13 nas hd-idle[923]: disk=sdc command=scsi spunDown=false reads=366448 writes=403 idleTime=1200 idleDuration=61 spindown=2020-08-08T16:36:11 spinup=2020-08-08T16:37:12 lastIO=2020-08-08T16:37:12
Aug 8 16:38:13 nas hd-idle[923]: disk=sda command=scsi spunDown=false reads=443750 writes=415 idleTime=1200 idleDuration=122 spindown=2020-08-08T16:35:10 spinup=2020-08-08T16:36:11 lastIO=2020-08-08T16:36:11
Aug 8 16:38:13 nas hd-idle[923]: disk=sdb command=scsi spunDown=false reads=424454 writes=341 idleTime=600 idleDuration=548 spindown=2020-08-08T16:28:03 spinup=2020-08-08T16:29:04 lastIO=2020-08-08T16:29:04
Aug 8 16:38:13 nas hd-idle[923]: disk=sdd command=scsi spunDown=false reads=11100 writes=4006 idleTime=0 idleDuration=0 spindown=0001-01-01T00:00:00 spinup=2020-08-08T11:51:36 lastIO=2020-08-08T16:38:13
from hd-idle.
auanasgheps
commented on September 26, 2024
wow, Thanks for the fast update! I'm installing the update now and will keep you posted.
from hd-idle.
Related Issues (20)
- Question. Don't start HDD after suspend and sleep HDD on standby HOT 1
- HDD in external bay not detected anymore HOT 3
- Question / Clarification: How does hd-idle parse options (disks vs. partitions, LUKS, precedence in case of contradictions) HOT 1
- Help understanding how to use the -p power_condition flag for SAS drives HOT 15
- WD Re HDD over USB does not spin down. HOT 23
- Can't spin down my internal WD Blue HDD HOT 5
- HD-Idle command does not seem to get set? HOT 4
- Drive standby instead of sleep. HOT 2
- Logfile won't be generate dataγ HOT 5
- Hook-function to be called after spinning down HOT 2
- Spin up not recognized HOT 2
- Disk spin down not working after scheduled S.M.A.R.T test
- hd-idle task gets out of sync with real disk spindown HOT 11
- logfile hard to read - how can we have spindown/spinup log entries? HOT 1
- Deprecated Key? HOT 2
- RFE/RFC: hd-idle extensibility: add option for a state file HOT 6
- help identifying actions in a log HOT 1
- Possible to spin-down based on Schedule? HOT 3
- Logfile won't be created HOT 4
- Nothing after hd-idle command HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from hd-idle.