warriordog / activitypubsharp Goto Github PK

SimpleClient is in a really bad state after rushing to get it workable. It needs (at least) the following work:

• Extract the state (_focus, mostly) into a stateful singleton service
• Extract each command into a separate handler class. Closely related commands can be grouped if convenient / appropriate.
• Cache reflection info at initialization wherever possible
• Normalize error output / exception messages. Maybe even use a Result type for non-exception errors?
• Implement the help command

AS types should be compared with exact case. An older version of this code was incorrectly case-ignorant, and some vestiges of that still remain. Everything should be fixed to be case-sensitive, with tests to verify this behavior.

Implement a type to model a BCP47 Language-Tag. These are used in NaturalLanguageString, ASLink.HRefLang, and possibly other places.

Proposed implementation:

• Create LanguageTag utility type
  • Implicit conversion to/from string
• Create LanguageTagConverter to convert to/from JSON
• Change type of ASLink.HRefLang to LanguageTag

Tests will be needed for type, converter, and the modified uses.

Related to #12.

The JSON-LD model, as currently designed and implemented, will not and can not work due to this open issue in System.Text.Json: dotnet/runtime#63791. Until that feature gap is resolved, we will need to fall back to a lower-level implementation. This is the proposed fallback:

• Simple types (not polymorphic, no special AS logic or encoding) will be handled by System.Text.Json as usual.
• Complex types will implement an interface exposing static virtual Serialize and Deserialize methods, which will work with JsonNode/JsonElement respectively.
• Complex types will also implement protected static virtual methods Read and Write. Read will populate an existing object from a reader, and write will write properties into a pre-initialized JsonObject. These will allow serialization logic to be "inherited" by calling into the base class members.
• Top-level code will create readers and writers manually. We should wrap this up in some kind of helper class, ideally with DI so that we can easily remove it later.

This rewrite will be completed in the sample-app branch. Work will be considered done when the sample app is able to query a simple object from a remote instance.

@context only exists in the document root. We need to strip it from nested objects. Although technically, we should probably compare nested contexts against the root context and generate a scoped context for the specific property. But that's out of scope for now. TODO: create a separate item for that

After scanning, assemblies should be tracked in some kind of list or hashset. If already in the list, then scanning should be skipped. This removes a negative performance hit if something mistakenly calls RegisterAssembly or RegisterAllAssemblies multiple times.

During the push to implement functional JSON, many internal types were switched to public as a quick-fix. Additionally, private / internal types were dropped just anywhere instead of in the correct location. This needs to be fixed.

Full JSON-LD is probably not practical to support, but these features are critical for ActivityPub support:

• Set context appropriately (serialization)
• Read & verify context (de-serialization)
• Allow multiple contexts - this is necessary to communicate with most real-world ActivityPub software (both)

Additionally, these would be nice to have:

• Map fully-qualified property names (de-serialization)
• Remap properties based on context (de-serialization)
• Not limited to fixed list of known contexts (both)
• Use System.Text.JSON instead of a third-party library (both)
• Support polymorphic properties (both)

There exists a package to support Dependency Injection in XUnit tests. Integrate it so that tests don't have to manually create entire dependency graphs just to construct an object. Ensure that the implementation allows complete isolation (by replacing implementations with fakes) if desired.

The spec-compliant API is garbage - either OneOf or AnyOf can contain the list of options, and code is supposed to check which is populated in order to determine the question type. This is clunky and uncomfortable, so we will do this instead:

• Merge OneOf and AnyOf into Options (or another name)
• Add a new boolean property AllowMultiple (add [JsonIgnore])
• Implement a [CustomJsonDeserializer]
  • Map oneOf to Options and set AllowMultiple to false
  • Map anyOf to Options and set AllowMultiple to true
• Implement a CustomJsonSerializer]
  • If AllowMultiple is false, then write Options to oneOf
  • If AllowMultiple is true, then write Options to anyOf

Note: the custom serializer API doesn't handle this case well - it may need an upgrade. We don't want to manually convert every property.

The following behavior is incorrect, but currently displayed by the JSON implementation:

• Properties with a null value are written to the output
• Empty collections are written as an empty array ([])

I think both can be fixed with some changes in ASTypeConverter.

There's an example of this in the documentation here: https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json/converters-how-to?pivots=dotnet-7-0#sample-factory-pattern-converter

IActor / ASActor do not exist as types in the ActivityPub spec. Instead, they are shapes that any object can implement. The description of these is extremely lacking and types have been largely inferred. Hopefully - there is a technical spec somewhere that be referenced.

This issue is to find such a spec, or if no such spec exists, then to study what existing implementations do. Use that knowledge to verify our model.

Use this as a starting point: https://www.w3.org/TR/activitypub/#actor-objects

This should allow us to remove all the Equals() stuff. Need unit tests to ensure everything behaves the same. Also check if records support inheritance - that's important for JsonLDTerm and JsonLDExpandedTerm.

We don't want to implement many at this stage, but it would be good to have the test project(s) created in the solution.

We currently have JsonLDContext to model the object-form, but the property itself is modeled as HashSet<JsonLDContext>. This works, but makes prevents us from adding special methods and makes it clunky to pass the context around.

We should create a dedicated type to contain the HashSet. I recommend JsonLDContext, and we rename the existing JsonLDContext to JsonLDContextObject. Thus, we will end up with this heirarchy:

JsonLDContext 1->x JsonLDContextObject
                     (IsExternal) 1->1 IRI (actually just a string)
                     (IsInternal) 1->x JsonLDContextTerm
                                       JsonLDExpandedTerm

Not exactly... nice, but "nice" and "JSON-LD" don't exactly go together 🤷‍♀️

We currently support JSON-LD's @context field in either string or string[] forms, but we don't support the object form. This is needed, at the very least, for #15.

Various tweaks and small refactors to improve compatibility between AS and .NET. Depends on changes in #8.

Create new types:

• ASUri (use ASUriConverter)
• LinkRel (use LinkRelConverter)

Implement these implicit casts between AS / CLR types:

• ASUri <-> string, Uri
• ASLink <-> ASUri, Uri
• MentionLink <-> ASUri, Uri, string
• Linkable<T> <-- ASLink, ASUri, Uri, string, T
• Linkable<T> --> ASLink?, T?
• LinkRel <-> string
• ASCollection <-- List<T>, T
• ASOrderedCollection <-- List<T>, T
• ASCollectionPage <-- List<T>, T
• ASOrderedCollectionPage <-- List<T>, T

Implement these custom JsonConverters:

• ASUriConverter <-> JsonTokenType.String
• LinkRelConverter <-> string

Change the type of these properties:

• ASLink.HRef to type ASUri
• ASLink.Rel to type LinkRel

REMOVE these implicit casts:

• NaturalLanguageString <-> string (this will be redesigned in #12)

Additionally, add unit tests for all new types and behavior.

Currently, you have to type the case-sensitive .NET property name. We should ignore case and also allow the JSON property name.

Implement IEnumerable<T> to iterate through Items. This will allow ForEach over a collection.

Output is meant to be pretty-printed. This was working, but broke during refactors. Depends on #25.

In the meantime, a simple workaround is to put this somewhere in Program.cs. It must precede the first JSON call, or an exception will be thrown!

var serializer = host.Services.GetRequiredService<IJsonLdSerializer>();
serializer.SerializerOptions.WriteIndented = true;

For some APIs, we have to derive an instance of JsonNodeOptions from the provided JsonSerializerOptions. This is currently done once for every call to Write, but that may be avoidable. Factory-style JSON converters (including ASTypeConverter) should be bound to a particular options instance that can be passed from the non-generic factory. If that options never changes, then we can also avoid changing the node options. Both can be cached as readonly fields in the class.

This is to implement a naive MVP implementation of JSON-LD, just to get the project unblocked. Easy version is this:

-[ ] Start with the System.Text.Json fork
-[ ] Map all fields to their basic names. hope for no conflicts lol

This would allow for better DX in some edge cases. There are several important requirements:

• Must be abstract
• Must be read-only
• Must contain ASType elements
• Should implement IEnumerable (delegate to Items)
• Should have an implicit conversion to List

See README.md for a list of package names

The Range type is clunky and results in bad UX. Its barely used anywhere so replace it with overloaded properties for each type, with backing fields.

We need a polymorphic JSON converter for ASType. It should read the "Type" field and select the appropriate subclass to construct. Missed part of #6.

Implement a property in ASType to capture overflow JSON. This will capture and contain any properties that exist within an incoming object but don't map to any known properties. Overflow data should persist all the way to re-serialization.

This should be a good starting point: How to handle overflow JSON or use JsonElement or JsonNode in System.Text.Json

This is to explore a possible implementation of JSON-LD based on interfaces and Roslyn Source Generators. If successful, this will resolve #2.

The first step is to model all AP / AS types as interfaces rather than classes. This accomplishes two key things:

• Resolves multiple inheritance (as with OrderedCollectionPage)
• Allows an application to include multiple extensions of the native types

And has two key flaws:

• You can't create an instance of an interface
• Library users would have to manually define an implementation for every type they use

Both flaws are dealbreakers, which leads to the second step - implementing a source generator. The source generator will automatically produce all necessary implementation classes at compile time. Since this runs at application compile time, rather than library compile time, the implementation class is guaranteed to have all fields supported by the application. Application users would just need to create an empty partial class for each type needed.

Instead of SingeString / LanguageMap mode, base it on Lang->Variant->Value map with a default fallback. Use BCP47 Language-Tags and possibly an existing library since this is a lot to deal with.

API should look something like this:

public class NaturalLanguageString
{
    /// <summary>
    ///     The language-agnostic value of this string.
    ///     Will be used as a fallback if the requested language is not available, or if no user preference is available.
    /// </summary>
    public string? DefaultValue { get; set; }
    
    /// <summary>
    ///     Gets or sets the value of a string for a given language.
    ///     If no language tags are specified, then this will access <see cref="DefaultValue" /> instead.
    /// </summary>
    /// <seealso cref="GetFor" />
    /// <seealso cref="SetFor" />
    public string? this[params string[] language]
    {
         get => GetFor(language);
         set => SetFor(value, language);
    }

    /// <summary>
    ///     Sets the value of the string for a given language.
    ///     If no language tags are specified, then sets the value of <see cref="DefaultValue" /> instead.
    /// </summary>
    /// <remarks>
    ///     The value must be provided <em>first</em> due to the use of <see langword="params" />.
    /// </remarks>
    public void SetFor(string value, params string[] language);


    /// <summary>
    ///     Gets the value of the string for a given language.
    ///     If no language tags are specified, then returns the value of <see cref="DefaultValue" /> instead.
    /// </summary>
    /// <seealso cref="TryGetFor" />
    /// <seealso cref="SetFor" />
    public string? GetFor(params string[] language);

    /// <summary>
    ///     Attempts to get the value of the string for a given language.
    ///     Returns true on success, or false if no matching string was found.
    ///     This method does <em>not</em> consider <see cref="DefaultValue" />.
    /// </summary>
    /// <seealso cref="GetFor" />
    public bool TryGetFor(params string[] language, [NotNullWhen(true)] out string? value);

    /// <summary>
    ///     Returns this string as a map of language tags to values.
    ///     The default value will be mapped to the <see langword="null" /> key.
    /// </summary>
    public IDictionary<string?, string> ToDictionary();
}

This will involve three steps:

1. Change type of ASType.JsonLdContexts to support the object form
2. Extend ASUri to resolve shortened forms given a specific context
3. Find some way to provide context during deserialization

Depends on #14

We could substitute them like this:

• Linkable<T> -> ASType
• LinkableList<T> -> List<ASType>

Pros:

• Reduced code complexity
• Eliminate several JSON converters
• Can model something like MentionLink | ASObject

Cons:

• Can no longer model something like ASLink | PersonActor - user would need to do something like if (asType is ASLink link) {} or if (asType is PersonActor person) {}
• We will break ActorEndpoints - it does not derive from ASObject so ASType can't be used as a base

1. ActorEndpoints should not be an ASType
2. It's linkable (type is Object | Link) but the type is not an AS type

Currently, IgnoreEmptyCollectionsModifier is automatically attached to the JsonSerializerOptions to strip out empty collections. This should be configurable through some kind of toggle.

Just a convenience. We can still add all our custom logic, but collection initializers and other goodies will be available.

All the DevOps-y parts of OSS. Configure the github repo, create a build pipeline, establish contributor guidelines and technical standards, etc.

Phase 1 - Groundwork:

• Add license to project
• Create standard issue labels
  • Create basic issue labels
  • Add "blocked" label
• Convert PROGRESS.md into GitHub issues
• Establish documentation standards
• Define testing standards
• Create milestones for MVP
• Document design decisions (monorepo, DI focus, etc)
• Create issues for all current TODO items

Phase 2 - MVP:

• Enable security checks
• Enable Dependabot updates
• Create build pipeline
• Usable documentation
  • Scaffold documentation files (#20)
  • XMLDoc for all API
  • Document standard use cases

Phase 3 - Best Practices:

• Create issue template
• Create PR template
• Publish releases from pipeline
• Code checks on PR (github actions?)
• Complete documentation
• Move project to GitHub organization (and use PRs for my own changes)

Place a readme.md file at the root of each distinct package / C# project. Describe the goals, function, and status of each project. Link each package from the table in the root readme.

Finalize the library architecture for the upper packages. This needs to be worked out while its still early and things are flexible. The API shape of Client and Server packages will likely shape the entire rest of the library.

We have these goals to balance:

• Out-of-box experience - we want the default configuration to be as close to production ready as is reasonably possible. This applies at all levels of abstraction - not just the upper packages.
• Customization - user code must be able to customize and/or replace parts of the library logic. This should also be doable without too much friction - there shouldn't be a need to re-implement giant complex interfaces just to tweak a condition.
• Flexibility - ActivityPubSharp should be usable for multiple types for fedi software, so we can't just pick one "target use case" and go with that. We need to ensure that the high-level API is flexible enough to be used in different workloads.
• Extension compatibility - built-in code must support extension use cases that can't be predicted in advance.
• Dual uses - library could be used in user-facing or automated environments. In the former, we want very strict access controls. In the latter, we want the opposite. Doing this easily will probably require some tiered structure, with most logic implemented from a batch perspective and user access logic implemented in an additional layer that wraps the previous one.

For now, lets go with this:

• ActivityPub.Types - type model and JSON conversion
• ActivityPub.Common - core utilities, services, and config
• ActivityPub.Client - client services and use cases
• ActivityPub.Server - server services and use cases (including federation)
• ActivityPub.Server.AspNetCore - integration with ASP.NET Core
• ActivityPub.Extension.Mastodon - implementation of Mastodon extensions

Related: #117

Needs to split into one of these based on properties:

• ASTargetedActivity - has "target"
• ASTransitiveActivity - has "object" but not "target"
• ASActivity - has neither

Ignore ASIntransitiveActivity as it has an entirely different AS type

The HasOnlyHRef property does not account for ASType.UnknownJsonProperties. If that dictionary contains any elements, then HasOnlyHRef should return false to force object-mode serialization.

This doesn't happen with base ActivityStreams, but the spec allows for it. The resulting type is essentially A & B. C# can't model that so we'll offload most of that to extensions (#36), but we still need to support the types list. Its possible for something like this to occur:

{
  "@context" {
    "https://www.w3.org/ns/activitystreams",
    "https://example.com/some-funky-extension"
  },
  "types": [
    "CustomType",
    "AnotherCustomType",
    "TypeWeDon'tEvenCareAbout",
    "PersonActor"
  ]
}

In the above scenario, we need to skip over the first three types and convert to PersonActor. The other types should be preserved in ASType.Types.

This computed property is used to determine whether an ASLink is serialized as a string or object. The current implementation is a hack and won't work with extensions or subtypes. It should be replaced with something more durable and extendable by subtypes.

Currently, ASObject.Url is typed as ASLink | null, but it should be List<ASLink> | ASLink | null. List -> T conversion is already handled by the globally-registered LinkableConverter, so this change will likely be as simple as changing the type to List<ASLink>?. Tests are needed to ensure that everything works as expected.

Extensions are a critical and actively used part of the ActivityPub spec, but they're also remarkably difficult to implement in C#. This poses a research challenge - we need to find an implementation that meets all these requirements simultaneously:

Deserialization:

• Can extract a subset of unmapped properties from incoming JSON
• Shape is defined by code that won't exist at library build time (but will exist when linked with user code)
• Conversion ONLY takes place if the appropriate context is included
• User code should not need to pass around multiple objects. That is, extension data must be directly linked to the base object type.
• Properties may be remapped at runtime by an alias in an inline or external context. currently out of scope - will be handled in a separate issue
• Property values can be remapped by an extension. currently out of scope - will be handled in a separate issue

Serialization:

• Extensions must be linked to and serialized along with the base object - users should not need to make multiple calls
• Extension context must be promoted to @context property
• Multiple extensions may define overlapping properties, and they must be aliased or expanded to resolve conflicts.

For more information, see previous discussions in #2.

Hello!

I recently became interested in the fediverse concept, and learned about the underlying protocols (ActivityPub/WebFinger) under which Mastodon relies. I was curious if the .net ecosystem had any implementation of ActivityPub. This is one of two repositories I've found, both without a license. This repository is more recent, and is more in line with an implementation that I was considering myself, so I was considering contributing instead of starting from scratch.

But without a license, I cannot safely do so. I see that adding an open source license is on your To-Do list already, but I figured adding an issue might encourage you to do it sooner rather than later. 🙂

If you're your having difficulty deciding which license to use, I'm more than happy to discuss it with you.

this is important for things like pretty printing.

This is really just a DI upgrade. IJsonLdSerializer already exposes a SerializerOptions property with a configurable instance. We need to run a callback or something after the service collection is built, then we can adjust it.

Currently, you have to type "push Value" or similar which is both annoying and extremely unintuitive. We should unwrap this automatically.

This is used in every object (ASType.MediaType) and could benefit from a more specific type (at least for some use cases). If one exists in the framework, then we should use that. Otherwise we should create our own. In either case, we will need a custom converter. Make sure to update the property to use the new type.

Do to a bad merge, the main branch does not currently build or compile. It needs to be restored to a working state, even if that means removing code that seems to support JSON-LD. Whatever is there doesn't work and will be replaced anyway with #2.