Comments (15)
Or Instagram, but especially Mastodon
from weeknote-generator.
There was Instagram support, until they started locking down their API. That stalled at #19 but given that in 2019 they further limited their API I'm going to mark #19 as "won't fix" because Instagram are web-hostile.
I've changed the title of this one so we'll know when it's fixed (if there are other sources people would like, we should open separate issues for them).
I think we should make a copy of the images rather than hot-linking to them, so we don't hit the same "images don't show up" bug as we have with Twitter hot-linked images #30. As I said in a comment on that issue, it'd be nice to also get permission to use the images the first time someone includes one in a weeknote. (Initially it can just store the consent locally, and we can add it to Optimism
as a similar-system-to-noting-inductions)
The main thing I'm not sure about is where we'd store a copy of the images. Should it be on a subdomain of ours - like assets.doesliverpool.com
or similar? Then we can have images/weeknotes/USER/file.jpg
for ones with just-weeknotes
consent, and images/full/USER/file.jpg
for those with full consent.
That could live on the same VPS as organisers.doesliverpool.com
does, I guess. Any reasons I shouldn't do that, or better suggestions?
from weeknote-generator.
Your image caching suggestions sound reasonable. How would we get the consent though, automatically or.. otherwise?
from weeknote-generator.
Automatically I think. Have the weeknote generator code keep a list of people it's got consent from (until there's an API for that in the members section of optimism ;-) and if it encounters an image from someone it doesn't know, it sends them a DM to ask for consent, with a "reply with yes / no / just weeknotes".
Might want to run that bit more frequently than once-a-week, so the consent has come through before we need to publish things.
from weeknote-generator.
Made some progress on this last night.
There doesn't seem to be an API to Mastodon to get posts for a given hashtag, but you can get the hashtags as an RSS feed, so that works. I've got some code that fetches that and parses it before trying to fetch the details of each post.
Fetching the details of the post works for Mastodon, but fails on Pleroma (at least), so it'll need to be a bit more intelligent in how t gets details in order to message the user to ask for consent.
I'm thinking that we'd run a script to find new posts on Mastodon frequently (one-a-day or something), which then saves them into a full-consent-awaiting-processing
or just-weeknotes-awaiting-processing
folder if we've already got consent from the user (and discards them if they refused consent). And messages any new users to ask for consent and saves those posts into a pending-consent
folder. I think it'd also check for any consent replies, and then move any posts accordingly.
Then the generate_weeknotes.rb
script will process anything from the ...-awaiting-processing
folders.
I'm tracking the work on this on a new branch: feature/31-add-mastodon-support
from weeknote-generator.
Found the timelines
section of the API, which lets us search better through tags and returns all the relevant details in JSON so it no longer crashes on Pleroma posts. I updated the find_toots.rb
script on the branch to use that.
I also added code to send a DM with a poll in it to ask for consent:
It needs to only send one request per person, and needs to do something with the replies, so that'll be next.
from weeknote-generator.
Nice. Small comments, the screenshot shows it asking for a reply when the user needs to fill in the poll, and perhaps it would be good to link to the week notes to explain what that bit means (then again maybe we can assume they know given they used the hashtag).
from weeknote-generator.
Yeah, partly that's because the wording was written when I was last working on it, and the poll was an addition last night 😁 I'm not 100% sure that all bits of the Fediverse support polls though, so maybe a "Vote or reply [all / weeknotes / no]" would work? Including a link to our intro weeknotes post isn't a bad idea either.
I did wonder whether or not it should come from @[email protected] or some new @[email protected] or similar - and also whether it should be a DM (which needs them to follow us) or a "private" message (not quite sure what that is, but it's one of the options for visibility alongside "direct")
from weeknote-generator.
Good question about the account, I guess technically if we're posting from a bot we should mark the account as a bot, but then that might confuse people about who they should follow if this bot was never actually posting anything else. I forget if people had to be following DoES, or vice versa, for the toots to show up?
from weeknote-generator.
I've tweaked the wording.
@johnmckerrell, I think at the moment we'll see any #weeknotes
toots that the does.social
server sees, but we can check for them following us (I'm sure). I think sending them from @DoESLiverpool makes more sense - maybe we could "sign" them with "^ Weeknotes-Bot" at the end of the message text if we want to make it clearer it's a semi-bot?
This evening I got it to only ask each user once, rather than every time it encountered a toot from them; and it now stores the consent in a YAML file (we can make it a more complex database in the future if need be). I've also worked out the API endpoints we'll need to hit to check for responses to the consent request, and started on a script (check_consent_requests.rb
) which will hit those endpoints.
I did discover that there isn't any paper trail for who voted (at least I've not seen it yet), just the number of votes. As you can't vote on your own polls, and it's a DM to a single user, I think that'll be okay, but I'd prefer to have a URL to store in the consent database which points to something on the user's instance. We'll have that for replies. That said, the user could delete a reply in future and the URL we've stored will 404, so maybe a poll is better in that case 🤷
from weeknote-generator.
Got the checking for replies to the consent request done today. Need to save the found toots accordingly now (based on both the consent, and also a pending area for ones where we ask for consent), and then process them to include in the weeknotes.
from weeknote-generator.
The found toots get saved: ones where we've asked for consent are put into one folder (from the settings), inside a sub-folder for the user; ones where we've got consent are stored in a folder ready for publication, based on the consent. When consent is given, any pending toots are copied over ready for publication accordingly (or deleted if consent is refused)
We now ignore toots from accounts that we don't follow too.
Next we can actually process them for inclusion in the weeknotes! Will need to download any media and store it somewhere on a server we can serve it from, and generate the relevant HTML for inclusion in the weeknotes blogpost.
from weeknote-generator.
Started on the processing them for inclusion in the weeknotes bit last night. Need to handle downloading the images, so we can serve them from media.doesliverpool.com
and then update the message content to point to the new locations.
And then get it deployed for use...
from weeknote-generator.
Added downloading the images and generating the output for Mastodon posts. Needs to only find toots in the past week, and have the consent handling actually ask everyone it finds, rather than printing out what it would've sent and only really ask me (that my debugging has done).
from weeknote-generator.
This has been deployed (see #34 for details) and so far seems to be working as expected. Actually, the poll when it asks for consent only lasts for an hour, which was good for testing but less useful in production. I've upped it to 14 days.
from weeknote-generator.
Related Issues (20)
- The generated page should include how to create posts for the website HOT 1
- Pull in images for tweets containing Flickr links
- Strip out any RTs
- Multiple image support has stopped working HOT 1
- Pull #weeknotes from the DoES Slack channel
- Submit app to Instagram for review of the API HOT 2
- Multiple images aren't included inline HOT 1
- Summarise updated issues?
- Images using http:// URLs HOT 2
- Update Twitter integration to cope with longer tweets
- Highlight open issues on GitHub HOT 1
- Tag each section to allow deep-linking
- Event times are sometimes incorrect
- Not handling some repeating calendar entries
- Switch to authenticated github API access HOT 2
- Firefox sometimes blocks hot-linked Twitter images HOT 2
- Calendar entries set to "private" crash the weeknotes generator
- Dockerize the setup for easier deployment HOT 1
- Deploy to the organisers.doesliverpool.com VPS HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from weeknote-generator.