jamesvandyne / tanzawa Goto Github PK

Add a view for logging in / logging out.

According to the spec scope and me are optional, but the IndieAuthAuthorizationSerializer is requiring them.

Currently there is only a single page that is publicly visible. Add a status list and detail page so we can provide permalinks and share individual statuses.

Micropub only for starters

Current user model is the default user model. Add support for storing more information about posters so we can use it for micro formats about the author.

• Twitter
• Instagram
• Email
• LinkedIn?
• City
• State
• Country

Add support for plugins. Plugins should be able to eventually allow you to modify the public and admin site (of their own pages) and the pubic facing pages. They should be able to store data (run migrations) from settings pages. They should be able to enabled / disabled via the admin pages.

In order to achieve this, Tanzawa should autodiscover plugins.py modules using from django.utils.module_loading import autodiscover_modules

The autodisocvered modules can then be listed in a page in the admin with a button to activate / deactivate.

Activation should add the application to installed apps and run migrations like below.

from collections import OrderedDict
from django.apps import apps
from django.conf import settings
from django.core import management

new_app_name = "my_new_app"

settings.INSTALLED_APPS += (new_app_name, )
apps.app_configs = OrderedDict()
apps.apps_ready = apps.models_ready = apps.loading = apps.ready = False
apps.clear_cache()
apps.populate(settings.INSTALLED_APPS)

management.call_command('makemigrations', new_app_name, interactive=False)

management.call_command('migrate', new_app_name, interactive=False)

source

The general flow should be:

• django starts up
• plugins app autodiscovers plugins and compares them against activated plugins
• activated plugins will be added to installed apps / urls.py
• migrations should only be run on activation or when activated and the system as a whole is running migrations

Tanzawa needs support for uploading images. Image support initially will mean adapting the Trix attachment.js example to Tanzawa, but it would also be nice to be able to insert existing images on the system.

Since I'm planning on adding micropub support eventually - making trix use the micropub media endpoint seems logical. Selecting images (a gallery view) is lower priority.

File Storage

Tanzawa is designed to run with minimal requirements. Rather than storing images on S3 or other cloud provider's systems, we will store images locally on the server. File path should be as follows /YYYY/MM/DD/<uuid>/<md5>

Once the file is stored we want to process:

• store original as-is
• strip geo coordinates
• compress with pngcrush etc...
• create thumbnails / resized versions
  • resize for theme (currently 655px wide on my 15" Macbook Pro) - serve this by default as it should be much smaller

File Retrieval

Rather than serve all photos directly from Apache/nginx directly, I'll implement a simple retrieval api. This will allow me to provide a single stable url to users / the web at large and cover for any changes in file storage / permission changes going forward.

GET /media/<uuid> -> if data is found, return an x-accel-redirect to the actual file path.

This api could also take arguments to allow tanzawa to return the most appropriate (smallest) version possible.

File Models

I will need a way to manage and track the files in the database. As a single file could be used in multiple posts in the future, we do not want a single ForeignKey, but rather an M2M relationship.

Follow these instructions for installing spatialite to make SQLite geoaware.

• TFile
  • db_table: t_file
  • id: auto
  • created_at: datetime
  • updated_at: datetime
  • path: string
  • filename: string
  • point: Point (if possible with sqlite, otherwise lat/long fields)
  • private: bool - default: false
• TPostFile
  • db_table: t_post_file
  • id: auto
  • t_file -> TFile FoeignKey
  • t_post -> TPost ForeignKey

Features that need to be built

• model implementation
• Micropub media endpoint ( cookie-auth for now, Bearer auth can be added with indieauth support)
• media retrieval endpoint
• trix JS upload handler
• image processing

As the title says, I'd like people to be able spin up a Tanzawa blog on Digital Ocean with a single click. This issues is more of a milestone ticket than a specific task.

Beyond the work integrating any image building into the DO ecosystem, there are a number of tasks that must be completed first.

1. Update the Docker image to run a uWSGI server.
2. Create a packer script that will setup a VM with:
   1. docker
   2. nginx
   3. lets encrypt
   4. tanzawa running in docker
   5. a cron job to automatically update the system ( daily )
   6. a cron job to automatically pull the latest version of Tanzawa. (hourly? )
   7. Proper host/container filesystem mappings for staticfiles, media (file uploads), and the database
3. Instructions for how I can make new releases of the docker image either automatically, or a github action to build a new image
4. The packer VM must accept variables somehow for setting up the domain name to configure the .env files and nginx's lets encrypt.

Once we have a VM that can build a production ready Tanzawa instance, then the work can start to massage that script for DO integration.

Image processing for stage one will consist of the two biggest wins: strip exif data and automatically using optimized file formats.

Rather than introduce celery and some other background tasks to convert formats, we'll perform the optimization on the first request. This way we don't fill up the disk with files that aren't requested and only the first request is a bit slow.

Just as the title says. They're aligned properly on desktop, but funky on mobile.

• save resized photos
• add file filters?
• allow inserting / selecting images from Trix
  • select image toolbar, click image -> insert

One reason I disliked being on IndieWeb was the number of moving parts. Which is also why I'm liking this project - it's all a one big implementation that relies on just one server.

It would be cool to support native syndication rather than using brid.gy for Twitter and Mastodon, for starters.

Sorry if this is already done - I just couldn't find anything about this anywhere!

The current interface when posting / updating interfaces is too basic for planned features.

This may could be improved by making it a dropdown (similar with Wordpress), especially when we take into account #35 ( Post Visibility)

Add support for sending webmentions. It seems like we can use webmention-tools to help implementation.

Hi James,

Is there currently — or going to be — a mobile posting workflow? Perhaps related to that, does Tanzawa have the ability to sync with GitHub and build posts from Markdown?

Cheers,
-Pete

Streams are the name for the multiblog. They'll appear down the left. Each stream can consist of multiple post kinds.

There must be a minimum of 1 stream. The default stream will be called "Status". Each stream can have a custom name and emoji.

When exposed via micropub, a stream is a category. A single post can be in multiple streams. Each stream will have its own feed for just those items at /feed/stream_slug.

All streams will feed into the main feed. Eventually I'd like to have unlisted streams (along with unlisted posts) that don't appear anywhere unless you know the url / are an admin.

Building the frontend JS/CSS currently requires two commands:

1. npm run build (tailwind via postcss)
2. npm run wp (JS via webpack)

This should all be a single command via webpack.

Moved the django-webpack-loader / hashing requirements to #71 .

Carryover from #66 . We've now got webpack building 3 different files: app.js (JS for the admin), public.js ( JS for the public site), and style.css (overall styles). These assets sometimes get cached by browsers, which can cause new functionality to not work until there's a hard reload.

Output files should include their hash so we can avoid caching issues. Once using hashes, integrate django-webpack-loader so django knows where to find the built files when running collectstatic.

Records that are store when Tanzawa sends a Webmention don't appear to be being created properly any longer. Investigate the issue and add some tests to ensure it doesn't break again.

Currently there is only a single page that is publicly visible. Add a status list and detail page so we can provide permalinks and share individual statuses.

Tracking issue for:

• https://github.com/jamesvandyne/tanzawa/security/code-scanning/5

I want to start properly documenting Tanzawa outside of the README. mkdocs seems to be a good solution to help keep documentation and code in the same repository. This will also make it easier to make sure features and documentation are updated together.

Preparing to import my Instagram posts and having a photo post type would be nice.

Currently I'm just embedding photos into an 'e-content' and calling it a day, however 'u-photo' should exist outside of 'e-content' in much the same way a checkin does.

As the title says document the commands required to get Tanzawa running locally for development / trying it out. Add the commands and an explanation to README.md

• build Docker image
• migrate / createsuperuser
• runserver (develop) gunicorn (prod)

Design the author profile page.

Currently just shows a list of posts.

It should:

• Show profile photo
• Show a blurb about the author (if exists)
• Links to author on other platforms ( if exists in profile)
• posts

interpret_comment's gets the first object from the h-entry and uses its content, rather than respecting the source url.

example: The content from colin's comment is for the first post ( p#1 but his tanzawa webmention is p#2 ). The link is correct but the content is off.

https://tanzawa.jamesvandyne.com/220b68bb-460b-4636-b206-e78e0030ef0e

Work around by making our own interpret_comment function. Once confirmed working send a pull request to mf2py repo.

Integrate django-environ to Tanzawa.

Filepaths, secret keys, debugging settings, allowed_hosts etc.. are all different depending on the environment and not something that should be managed in the git repository (public or private).

Hi James,

I'm still configuring it but as a new user I want to understand what I need to do to fix all these problems and document it, so that next users don't have to try it blindly to make it work.

I'm testing indieAuth, Indiewebify.me and login to the indieweb.org wiki with my tanzawa.

for IndieAuth I'm getting this :

{ "\n<!DOCTYPE_html>\n<html_lang": "\"en\">\n<head>\n <meta http-equiv=\"content-type\" content=\"text/html; charset=utf-8\">\n <meta name=\"robots\" content=\"NONE,NOARCHIVE\">\n <title>403 Forbidden</title>\n <style type=\"text/css\">\n html * { padding:0; margin:0; }\n body * { padding:10px 20px; }\n body * * { padding:0; }\n body { font:small sans-serif; background:#eee; color:#000; }\n body>div { border-bottom:1px solid #ddd; }\n h1 { font-weight:normal; margin-bottom:.4em; }\n h1 span { font-size:60%; color:#666; font-weight:normal; }\n #info { background:#f6f6f6; }\n #info ul { margin: 0.5em 4em; }\n #info p, #summary p { padding-top:10px; }\n #summary { background: #ffc; }\n #explanation { background:#eee; border-bottom: 0px none; }\n </style>\n</head>\n<body>\n<div id=\"summary\">\n <h1>Forbidden <span>(403)</span></h1>\n <p>CSRF verification failed. Request aborted.</p>\n\n\n <p>You are seeing this message because this site requires a CSRF cookie when submitting forms. This cookie is required for security reasons, to ensure that your browser is not being hijacked by third parties.</p>\n <p>If you have configured your browser to disable cookies, please re-enable them, at least for this site, or for \u201csame-origin\u201d requests.</p>\n\n</div>\n\n<div id=\"info\">\n <h2>Help</h2>\n \n <p>Reason given for failure:</p>\n <pre>\n CSRF cookie not set.\n </pre>\n \n\n <p>In general, this can occur when there is a genuine Cross Site Request Forgery, or when\n <a\n href=\"https://docs.djangoproject.com/en/3.2/ref/csrf/\">Django\u2019s\n CSRF mechanism</a> has not been used correctly. For POST forms, you need to\n ensure:</p>\n\n <ul>\n <li>Your browser is accepting cookies.</li>\n\n <li>The view function passes a <code>request</code> to the template\u2019s <a\n href=\"https://docs.djangoproject.com/en/dev/topics/templates/#django.template.backends.base.Template.render\"><code>render</code></a>\n method.</li>\n\n <li>In the template, there is a <code>{% csrf_token\n %}</code> template tag inside each POST form that\n targets an internal URL.</li>\n\n <li>If you are not using <code>CsrfViewMiddleware</code>, then you must use\n <code>csrf_protect</code> on any views that use the <code>csrf_token</code>\n template tag, as well as those that accept the POST data.</li>\n\n <li>The form has a valid CSRF token. After logging in in another browser\n tab or hitting the back button after a login, you may need to reload the\n page with the form, because the token is rotated after a login.</li>\n </ul>\n\n <p>You\u2019re seeing the help section of this page because you have <code>DEBUG =\n True</code> in your Django settings file. Change that to <code>False</code>,\n and only the initial error message will be displayed. </p>\n\n <p>You can customize this page using the CSRF_FAILURE_VIEW setting.</p>\n</div>\n\n</body>\n</html>\n" }

add support for bookmarks and replies.

Add site settings:

• Title
• RSS feed style (full content vs links?)
• Menu items on the left / blog title.

Need to confirm that points are being stored properly lat (y) lon (x) in the database for TLocation and TFile records. If not, after fixing the issue, add a data-migration to fix existing records.

Add support for webmentions

This is what I got today. Had happened a few days ago too. The body text in front of Bookmark: appears to be broken.

There should be a way to set publish dates for posts. This will allow you to schedule posts to appear in the future, as well as make it easier to insert posts when backfilling statuses or adding photos to a trip.

• Use BeautifulSoup to parse the wordpress export file and create entries.
• Download images
• How to handle video attachments?

Use phpserialize to handle post meta-data deserialization

Snippets:

# Post Type
>>> [x.text for x in set(soup.find_all('wp:post_type'))]
['post', 'attachment', 'custom_css', 'page', 'nav_menu_item']

# Created At
>>> [x.text for x in set(soup.find_all('wp:post_type'))]


# Post Format
>>> [x.text for x in set(soup.find_all('category', attrs={"domain": "post_format"}))]
['Status', 'Quote', 'Link', 'Video', 'Image', 'Aside']

# Category
>>> [x.text for x in set(soup.find_all('category', attrs={"domain": "category"}))]
['Uncategorized', 'Japan', 'Replies', 'Status', 'Sustainability', 'The Week', 'Thoughts', 'EV', 'Running', 'Checkin', 'Tech']

# Post Kind
>>> [x.text for x in set(soup.find_all('category', attrs={"domain": "kind"}))]
['Article', 'Bookmark', 'Note', 'Reply', 'Checkin', 'Photo']

# Extracting Meta Data From A Post

>>> p.find('wp:meta_key')
<wp:meta_key>_wpcom_is_markdown</wp:meta_key>
>>> p.find('wp:meta_key').find_next('wp:meta_value')
<wp:meta_value>1</wp:meta_value>

Add support for RSS. Should have an "all feed" as well as a feed for each different kind (note, blog, photo, workout etc...)

Add support for IndieAuth so services like ownyourswarm and bridgy can integrate.

Authorization Endpoint / Flow GET

• Add <link rel="authorization_endpoint" href="https://indieauth.example.org/"> link tag to my head
• Add a Authorization Endpoint that accepts the following GET input
  • me
    Full URI of the user's homepage
  • client_id
    Full URI of the application's/website's home page. Used to identify the application. An authorization endpoint may show the application's icon and title to the user during the auth process.
  • redirect_uri
    Full URI to redirect back to when the login process is finished
  • state
    A random value the app makes up, unique per request. The authorization server just passes it back to the app.
    Optional. Auth endpoints MUST support them, though.
  • response_type
    id (identification only) or code (identification + authorization)
    Optional. Defaults to id.
  • scope
    Not used and omitted in identification mode (response_type=id)
    For authorization, the scope contains a space-separated list of scopes that the web application requests permission for, e.g. "create". Multiple values are supported, e.g. create delete
• Verify the user
  • Display a UI to approve the request and select their scope
• Redirect to the callback up with the code and state GET parameters 302 FOUND

Verification Endpoint POST

• Input
  • code
  • redirect_url
  • client_id
• Output
• me (a url indicating who is signed in)

HTTP/1.1 200 OK
Content-Type: application/json
{
  "me": "https://tanzawa.jamesvandyne.com/"
}

Navigation for the posting interface leaves is difficult to use.

• fix layout - set a max width for the main content. Fixed width for the left nav.
• fix draft interface - if published change button text to "Unpublish"
• pagination
• Add a "dashboard".
  • list reviewed webmentions

More of a question/remark than anything else:

When I load https://jamesvandyne.com/feed/ and subsequently read, e.g., https://jamesvandyne.com/ed9f2982-9adf-47a7-b1e8-5624c498728e in my feed reader, it (obviously) displays the blockquote on top (because it's in the feed item).

But when I parse https://jamesvandyne.com/ as a microformats feed (which I prefer, because then I get all of the additional metadata, like reply-to URLs, etc.), the blockquote goes missing: my reader's mf2 parser (mf2-php), I believe, just drops it (it is not in e-content, after all).

In a way, that's fine (I have the link to the original article, after all), but if you were replying to a specific paragraph, for instance, that context would be lost (for mf2 followers).

When I try to add a new Bookmark in my localhost installation (still testing tanzawa)
I get this error :

[07/Nov/2021 10:25:52] "GET /a/bookmark/create/ HTTP/1.1" 200 1880 Internal Server Error: /a/bookmark/meta_info/ Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/django/core/handlers/exception.py", line 47, in inner response = get_response(request) File "/usr/local/lib/python3.8/dist-packages/django/core/handlers/base.py", line 181, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/local/lib/python3.8/dist-packages/django/views/generic/base.py", line 70, in view return self.dispatch(request, *args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/django/utils/decorators.py", line 43, in _wrapper return bound_method(*args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/django/contrib/auth/decorators.py", line 21, in _wrapped_view return view_func(request, *args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/django/utils/decorators.py", line 43, in _wrapper return bound_method(*args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/django/contrib/auth/decorators.py", line 21, in _wrapped_view return view_func(request, *args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/django/views/generic/base.py", line 98, in dispatch return handler(request, *args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/django/views/generic/edit.py", line 142, in post return self.form_valid(form) File "/app/apps/entry/views.py", line 185, in form_valid linked_page = extract_reply_details_from_url(form.cleaned_data["url"]) File "/app/apps/indieweb/extract.py", line 44, in extract_reply_details_from_url data = extruct.extract(response.text) File "/usr/local/lib/python3.8/dist-packages/extruct/_extruct.py", line 108, in extract output[syntax] = list(extract(document, base_url=base_url)) File "/usr/local/lib/python3.8/dist-packages/extruct/rdfa.py", line 154, in extract_items jsonld_string = g.serialize(format='json-ld', auto_compact=not expanded).decode('utf-8') AttributeError: 'str' object has no attribute 'decode'

Receiving webmentions was handled in #18 . This issue is to track actual displaying them publicly on the web. Webmentions can add a bunch of visual noise, so I think they should be hidden by default.

Detail page should use an html5 details/summary tag.

<details>
        <summary>Interactions</summary>
        <Webmentions displayed / sorted here>
</details>

List pages should not display anything about webmentions - or if they do, it should be in aggregate.

Disclaimer: not too big a fan of checkins. That said, I noticed, e.g., X-Ray (which I use for my home-grown feed reader, too, and relies on mf2-php) doesn't seem to pick up https://xray.p3k.app/parse?url=https%3A%2F%2Fjamesvandyne.com%2F0bb1651e-c6fc-4e41-8c6b-081b7fb79626&pretty=true as a checkin, while, e.g., https://xray.p3k.app/parse?url=https%3A%2F%2Fdavid.shanske.com%2F2021%2F08%2F22%2F5128%2F&pretty=true works fine.

Thinking it's a missing u-checkin on (e.g.) the Swarm/Foursquare link. (Guessing you'd also want to add this to checkins on the homepage.)

Wondering if with it, it'll also pick up the coordinates.

I would like to customise the user interface to change colours and fonts and so on. I also want to put each entry / posting into a sort-of "card". I'm not able to figure out where the frontend code might be. It looks like the UI is completely implemented in J Javascript. I also tried searching for the CSS styles that I might be able to adapt to my liking.

Any guidelines on how to customise the UI would be very helpful. Thank you for this wonderful project.

Right now the only way to delete a post is via the Django admin. It would be nice to have a button in the posting interface to delete posts.

Add support for visibility controls to posts.

• Public
• Unlisted
• Only me (logged in users)

Unlisted and private posts should not appear in feeds / streams / sitemaps unless the user is logged in. Should indicate that it's not visible somehow on the page.

I didn't think I'd want "like" posts, but I'm finding I want to interact with sites without bookmarking or replying. Not sure if these should appear in the main feed?

Search e_content and title.

Geo search (hidden field?) when clicking on the name of a location map.

Add support for entries / blog posts.
Main difference between statuses is a title and how it syndicates to Twitter.

Statuses will syndicate as a note with the entire e_entry text.

Blogs will syndicate as title plus link.

This is a milestone issue to keep track of the issues that I should need to be fixed before releasing:

• Pick a license
• Set verbose names for each model so they appear nicely in the Django admin
• Change t_post / t_entry to a OneToOne ForeignKey
• Mark public

Hi James,

Encountering a couple of errors from the docker prep steps:

1. When trying to clone from the git instructions, I receive the following message:
   [email protected]: Permission denied (publickey).
   fatal: Could not read from remote repository.
   Please make sure you have the correct access rights
   and the repository exists.
   I got around this by using the https link instead

2. I keep getting the following error at the docker build stage:
   "docker image build" requires exactly 1 argument.
   See 'docker image build --help'.
   Usage: docker image build [OPTIONS] PATH | URL | -
   Build an image from a Dockerfile

Any idea why it’s not working?

Cheers,
-Pete