Script skips everything (too old) about discord_feedbot HOT 13 CLOSED

The "too old; skipping" means that the item (post) in the feed is older than the "max_age" for that feed. Doesn't look like any kind of permissions problem for the channel (or if there is, that's a separate issue)

When I examine that RSS feed by hand, I see that the first item in the feed has a date of March 25... Try this feed_url instead: https://www.reddit.com/r/kpop/new/.rss

from discord_feedbot.

With the new feed URL, I'm still getting the same "too old; skipping" error. Really have no idea why...

With debug set to 3, these are the last lines. All previous ones say "too old" as well, even though I'm using the URL for the new feed rather than default.

DEBUG:__main__:kpopreddit:item:checking database history for this item
INFO:__main__:kpopreddit:item t3_4cqjh6 unseen, processing:
INFO:__main__:kpopreddit:too old; skipping
DEBUG:__main__:kpopreddit:sleeping for 500 seconds
WARNING:asyncio:Executing <Task pending coro=<background_check_feed() running at feed2discord.py:320> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/tasks.py:503> created at feed2discord.py:333> took 0.247 seconds

from discord_feedbot.

What date/time does your system think it is? (run date -u to see)

from discord_feedbot.

Server's time/date is Fri Apr 1 01:52:57 UTC 2016 should be fine, right?

Also on a potentially unrelated note, I tried this same thing replaced with an Instagram tag feed url (http://widget.websta.me/rss/tag/taghere) and it worked provided I set the max_age to a very high number, and started sending them into my Discord channel, but then when I tried a feed url for just one user (http://widget.websta.me/rss/n/usernamehere), it gave back this:

DEBUG:__main__:instagramtest:processing entries
DEBUG:__main__:instagramtest:item:processing this entry
DEBUG:__main__:instagramtest:item:checking database history for this item
INFO:__main__:instagramtest:item http://websta.me/p/1214601160999041871_329452045 $ unseen, processing:
ERROR:__main__:instagramtest:Unexpected error:
ERROR:__main__:(<class 'TypeError'>, TypeError('Tuple or struct_time argument r$ quired',), <traceback object at 0x7f3c364d6ec8>)
ERROR:__main__:instagramtest:giving up

So now I've got this and the Reddit error too... I've installed NTP and synced servers properly, but still the same errors for "too old" as well.

from discord_feedbot.

Yes, that date/time looks about right...

Please do a "git pull", set debug to 4 and try again with the reddit feed. I've added some additional debugging that will output all the relevant dates...

That TypeError on the instagram feed is definitely a bug, but a different bug. Can you put that into a new ticket?

from discord_feedbot.

I think I have an error with accidentally treating an item's date as if it's in the local timezone (but without converting it)... I'll have to look at that more carefully.

from discord_feedbot.

I can't figure out how to post the log because it's so long, so I've uploaded the text file with log here in it's entirety. Debug was set to 4 for this. Will make a ticket for the TypeError too. Also, my server's time is currently set to KST, if that makes a difference. Can be changed if needed.

from discord_feedbot.

This is the bit I needed:

DEBUG:__main__:kpopreddit:item:processing this entry
DEBUG:__main__:kpopreddit:item:id:t3_4czg2d
DEBUG:__main__:kpopreddit:item:checking database history for this item
INFO:__main__:kpopreddit:item t3_4czg2d unseen, processing:
INFO:__main__:kpopreddit:too old; skipping
DEBUG:__main__:kpopreddit:now:1459561548.9034908
DEBUG:__main__:kpopreddit:now:gmtime:time.struct_time(tm_year=2016, tm_mon=4, tm_mday=2, tm_hour=1, tm_min=45, tm_sec=48, tm_wday=5, tm_yday=93, tm_isdst=0)
DEBUG:__main__:kpopreddit:now:localtime:time.struct_time(tm_year=2016, tm_mon=4, tm_mday=2, tm_hour=10, tm_min=45, tm_sec=48, tm_wday=5, tm_yday=93, tm_isdst=0)
DEBUG:__main__:kpopreddit:pubDate:2016-04-02T01:45:32+00:00
DEBUG:__main__:kpopreddit:pubDate_parsed:time.struct_time(tm_year=2016, tm_mon=4, tm_mday=2, tm_hour=1, tm_min=45, tm_sec=32, tm_wday=5, tm_yday=93, tm_isdst=0)
DEBUG:__main__:{'id': 't3_4czg2d', 'link': 'https://www.reddit.com/r/kpop/comments/4czg2d/what_song_do_you_have_on_repeat_right_now/', 'guidislink': True, 'author': '/u/YeBeAWitch', 'updated': '2016-04-02T01:45:32+00:00', 'content': [{'type': 'text/html', 'base': '', 'value': '<!-- SC_OFF --><div class="md"><p>I&#39;m normally into more upbeat stuff but for me it&#39;s <a href="https://www.youtube.com/watch?v=2fBODWRrnWM">Sunggyu&#39;s kontrol</a>. For the last week I&#39;ve been playing it first no matter what I&#39;m listening to.</p> </div><!-- SC_ON --> &#32; submitted by &#32; <a href="https://www.reddit.com/user/YeBeAWitch"> /u/YeBeAWitch </a> <br/> <span><a href="https://www.reddit.com/r/kpop/comments/4czg2d/what_song_do_you_have_on_repeat_right_now/">[link]</a></span> &#32; <span><a href="https://www.reddit.com/r/kpop/comments/4czg2d/what_song_do_you_have_on_repeat_right_now/">[comments]</a></span>', 'language': None}], 'href': 'https://www.reddit.com/user/YeBeAWitch', 'authors': [{'name': '/u/YeBeAWitch', 'href': 'https://www.reddit.com/user/YeBeAWitch'}], 'updated_parsed': time.struct_time(tm_year=2016, tm_mon=4, tm_mday=2, tm_hour=1, tm_min=45, tm_sec=32, tm_wday=5, tm_yday=93, tm_isdst=0), 'tags': [{'scheme': None, 'term': 'kpop', 'label': '/r/kpop'}], 'links': [{'rel': 'alternate', 'type': 'text/html', 'href': 'https://www.reddit.com/r/kpop/comments/4czg2d/what_song_do_you_have_on_repeat_right_now/'}], 'title': 'What song do you have on repeat right now?', 'summary': '<!-- SC_OFF --><div class="md"><p>I&#39;m normally into more upbeat stuff but for me it&#39;s <a href="https://www.youtube.com/watch?v=2fBODWRrnWM">Sunggyu&#39;s kontrol</a>. For the last week I&#39;ve been playing it first no matter what I&#39;m listening to.</p> </div><!-- SC_ON --> &#32; submitted by &#32; <a href="https://www.reddit.com/user/YeBeAWitch"> /u/YeBeAWitch </a> <br/> <span><a href="https://www.reddit.com/r/kpop/comments/4czg2d/what_song_do_you_have_on_repeat_right_now/">[link]</a></span> &#32; <span><a href="https://www.reddit.com/r/kpop/comments/4czg2d/what_song_do_you_have_on_repeat_right_now/">[comments]</a></span>', 'title_detail': {'type': 'text/plain', 'base': '', 'value': 'What song do you have on repeat right now?', 'language': None}, 'author_detail': {'name': '/u/YeBeAWitch', 'href': 'https://www.reddit.com/user/YeBeAWitch'}}

from discord_feedbot.

Yeah, I think this is likely a timezone bug... Probably have to change how I do those date comparisons.

In the meantime, I think you can work around this by setting the "TZ" environment variable to "UTC" when you run the script. Like: TZ=UTC python feed2discord.py

from discord_feedbot.

TZ=UTC python feed2discord.py did the trick, posts fine now and I've set it up to posts 3 subreddits so far, works beautifully. Thanks!

from discord_feedbot.

I just pushed a change that should force it to do the equivalent on its own. If you git pull you should be able to run without manually doing the "TZ=UTC" thing anymore.

from discord_feedbot.

Runs fine with just python feed2discord.py now, much better.

from discord_feedbot.

FYI, there's some major improvements to the timezone handling. Should work on Windows or Linux, and does things even smarter than that workaround. A "git pull" to get the latest should work, or you can look at #24 to see the details.

from discord_feedbot.