Giter VIP home page Giter VIP logo

federation-rs's Introduction

federation-rs

This repository is responsible for all of the deno-powered TypeScript <--> Rust interop. Currently this includes composition and query planning.

Branch Strategy

This repository has one long-running branch, main. The federation repository itself maintains two separate branches, version-0.x for Federation 1 and main for Federation 2.

Workspaces

You'll notice that there are three cargo workspaces in this repository. The first is the root which contains xtask and apollo-federation-types. The others are federation-1 and federation-2. Each of these workspace directories contains some binaries and some libraries. Building specific packages is orchestrated by xtask in CI.

IMPORTANT: If you are working locally, changes made to federation-1 and federation-2 WILL NOT be picked up by rust-analyzer or your cargo build/cargo test commands. There are a few tools to help you out here.

  1. You can run code federation-rs.code-workspace to open a VS Code Workspace that will run Rust-Analyzer properly.
  2. You can run cargo xtask test from the root workspace directory to run tests across all workspaces
  3. You can open a new VS Code window for federation-1 and/or federation-2 and rust-analyzer will work as expected. There might be some way to get something to work with rust-analyzer.linkedProjects but it wouldn't be straightforward and opening a new VS Code window is easy enough.

federation-rs Crates

Each crate listed here has their own README with much more information than what's here.

harmonizer

The harmonizer crate is a library that provides the federation composition algorithm to the rest of Apollo's Rust ecosystem.

supergraph

The supergraph crate is a binary that provides the federation composition algorithm as a CLI, primarily for integration with rover.

apollo-federation-types

The apollo-federation-types crate provides types for all versions of harmonizer and supergraph, and is used by Rover to read the output from the supergraph binary.

router-bridge

The router-bridge crate is a library that provides the federation query-planning algorithm, primarily for integration with the router

federation-rs's People

Contributors

abernix avatar apollo-bot2 avatar benjamn avatar benweatherman avatar bnjjj avatar bryncooke avatar clenfest avatar dariuszkuc avatar dbanty avatar dependabot[bot] avatar doumanash avatar everlastingbugstopper avatar garypen avatar geal avatar github-actions[bot] avatar glasser avatar goto-bus-stop avatar jeffjakub avatar jsegaran avatar lennyburdette avatar michael-watson avatar o0ignition0o avatar peakematt avatar renovate[bot] avatar sachindshinde avatar simonsapin avatar svc-secops avatar tinnou avatar trevor-scheer avatar xuorig avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

federation-rs's Issues

get tests to pass for harmonizer

in the current state of the repo, harmonizer fed 2 tests do not pass due to an issue with rollup - it's likely because I'm not exactly sure what the effects of the first compile stage are in the federation repo and we need to copy over artifacts or something.

router-bridge: Re-introduce `validate` on the `main` (v2) branch

When I initially switched to using the new query-planner, via apollographql/federation@9cb2201, I removed our invocation of validate on the premise that we did all the validations inside of the Federation's buildQueryPlan and operationFromDocument. While we do a lot in those, we do not do the complete suite of validations and we should add that back.

This can be seen rather clearly in the diff on the above commit, but roughly this consists of:

  1. Adding back
      const validationErrors = validate(composedSchema, query);
      if (validationErrors.length > 0) {
        return { errors: validationErrors };
      }
  2. Making sure the Router itself can recognize that as a validation error and thread that through to this code (see apollographql/router#309)
  3. Adding a test to the Router to make sure that we test for validation to have happened.

router-bridge: Instantiate a v8 runtime once, and reuse it for query planning

We're currently instantiating a new v8 runtime through deno_core everytime we want to get a query plan for a query.

Moreover, we also have to reinstantiate a QueryPlanner, validate the schema etc.

We could avoid all of that by Instantiating a v8 runtime once, and reinstantiating a QueryPlanner just when the schema changes.

add `composition-latest-0` and `composition-latest-2` tags when running `cargo xtask tag`

in orbiter, we need a way to get the latest version of the supergraph CLIs for either v0 or v2. easiest way to do this is with a tag that we update for each. i've manually created these tags based on what's here but we need to automate updating it. i've already created the first versions of these tags locally. until this is closed we should manually update them to point to the correct place.

router-bridge: consider alternative packaging approaches

The rust router-bridge is currently relying on build.rs to invoke npm run compile:for-router-bridge-build-rs and generate the required .js files it will invoke at runtime.

Relying on a build.rs step has an impact on downstream build times.
Moreover the build.rs step invokes an npm command, which requires downstream (rust) users to have a nodejs environment available.

Goals:

  • We would like to get rid of the build.rs file
  • It would be great if we didn't need a nodejs / npm environment to build downstream libraries (the router for example).
  • We want to make sure the bridge remains in sync with the underlying js / ts code.

A couple of options:

1 - Check in the js output:

  • generate the js files by running the npm run compile:for-router-bridge-build-rs while developing.
  • check in the output directory to the repository

pros:

  • the build step is gone
  • downstream users won't need to have an npm / node environment at hand (rusty_v8 gets pulled in when running cargo build)
  • the bridge remains in sync (and tests make sure the bridge keeps working)

cons:

  • these are a lot of files
  • this will introduce noise to any pull request that changes query planning / query introspection

2 - move router-bridge to a separate repository

  • move router-bridge to a separate repository
  • have router-bridge depend to a specific version of query planner
  • regularly update the dependency

pros:

  • the federation team doesn't have to maintain router-bridge anymore
  • one less build step

cons:

  • hard to see when the js code is out of sync
  • hard to track breaking changes in the query-planner

there are probably other ways to deal with it, I'm not really sure which solution would be best, what does the federation team think?

router-bridge: Packaging and distribution approaches

This conversation is revisiting the #15 decisions now that the product has evolved.

Context

The rust router-bridge is currently relying on build.rs to invoke npm and generate the required .js files it will invoke at runtime.

This leads to:

  • Router developers need to have Nodejs and NPM installed and in their path
  • Increased compile time when building the Router

Previously identified options (have a look at the linked issue for more info):

1 - Check in the js output:

  • generate the js files by running the npm run compile:for-router-bridge-build-rs while developing.
  • check in the output directory to the repository

2 - move router-bridge to a separate repository

  • move router-bridge to a separate repository
  • have router-bridge depend to a specific version of query planner
  • regularly update the dependency

3 - publish as apollo-router-bridge

Essentially, this will work the same way as harmonizer does today, where the actual .js is in the published crate. The router would then merely depend on that.

New option:

4 - pre generate snapshots for each of our target architectures.

Pros:

  • we wouldn't need to check in the js-dist
  • we could get rid of the whole build-rs (much faster build times)

Cons:

  • we need a publish command that generates these
  • we need to make sure we can generate the snapshots for each of our supported platforms

Previous decisions

We did act on option number 2; but we are still suffering the need for our downstream users to have NPM and Nodejs installed.

This becomes more of a pain point now that our users are trying to build their own routers; they hit this more.

Publishing apollo-router-bridge looks like the most sensible approach to me; since it would allow our downstream users to not have to deal with the npm dance; and we would still have a good local development experience.

Query planner: No operation found in provided document.

TLDR: let's add a test with no operation in the query and no provided operation name.

(Another case to seriously consider is the No operation found in provided document. case, which happens if you don't provide an operation name and there's no operation at all in the document. Skimming the graphql-js validation rules I don't think there's an explicit rule for "there must be at least one operation". However, the graphql-js parser requires there to be at least one top-level definition (see parseDocument and note that this.many matches a non-empty list of things). And the validation rules do include a rule that requires all definitions to be executable (not type-system) which is to say operations and fragments. So the only way to have no operations is to have a document consisting only of fragments. But there's another rules that says all fragments must be used (perhaps transitively) in an operation. (Interestingly the spec merely says that each fragment must be spread somewhere, not that it must be reachable from an operation, but since fragment spreads cannot form cycles you eventually have to find a spread from an operation.) Thus, a document without any operations is either a parse failure or a validation failure and so the No operation found in provided document. case should not be possible. I encourage you to double check my logic here!)

Originally posted by @glasser in #106 (comment)

harmonizer::harmonize should return a `Result<String, impl std::error::Error>`

Pretty much what the title says.

Library consumers of harmonizer will want an easy integration with libraries such as anyhow that take any error type implementing std::error::Error. I'll be able to get away with a map_err for the time being, but the API would be nicer to use and integrate with other tools more easily if it implemented std::error::Error

automate keeping the submodules up to date

every time a commit is pushed to main or version-0.x in the federation repo, this repo's submodule should be updated on the according branch to point to the latest commit

better separation between config errors and composition errors

builds are a multi-stage process. first the config file is resolved (get all the SDL from the various subgraphs) and THEN run composition afterwards. this means that BuildErrors will either be all config errors or all composition errors. currently we have that designation one layer under where I think we should have them. It'd be a better representation to have the error type on the BuildErrors type rather than the BuildError type.

chore: provide prebuilt binaries for `aarch64-apple-darwin`

Situation

Developers want to run rover compiled for the ARM architecture which runs on their new Mac computers with Apple processors. Rover downloads pre-built supergraph binaries that are built from source code in this repository. We currently provide pre-built binaries for architectures x86_64-apple-darwin, x86_64-pc-windows-msvc, and x86_64-unknown-linux-gnu. Our next releases (2.0.6 and 0.36.2) will also have support for the aarch64-unknown-linux-gnu architecture.

Problem

We're blocked on support for aarch64-apple-darwin for a few reasons:

  1. CircleCI can only run workflows on MacOS with an x86_64 processor
  2. You can cross-compile for aarch64-apple-darwin on an x86_64 machine, but v8 will still generate snapshots (bytecode) in x86_64. This means that running supergraph compose on an M1 when it was compiled on Intel will fail miserably.

Resolution for now

Only deliver prebuilt aarch64-unknown-linux-gnu binaries, and forgo native ARM support for Macs processors. Many other tools at the moment do not provide binaries for this architecture because Apple makes it incredibly hard to support. This should be fine! M1 users can run x86_64-apple-darwin binaries because of Apple's emulation software Rosetta. This should work by default with no configuration because we sign our binaries. Running in Docker will also work because we provide pre-built binaries for aarch64-unknown-linux-gnu.

Long term resolutions

There are a few paths forward here:

  1. If Circle (or Apple) start renting out CI servers that run on Apple's processors, we'll use that to provide these binaries.
  2. If Federation gets oxidized (see: rewritten in ๐Ÿฆ€ Rust), this shouldn't be a problem anymore since we won't need to build and embed v8 snapshots into these plugin binaries - we'll just cross compile from x86_64 macs and it will Just Work
  3. For unsupported architectures, we could enable Rover's installer script to clone Rover and run cargo build (this would be a slow installation but maybe better than nothing)
  4. We decide that x86_64 emulation is good enough and never build for aarch64-apple-darwin

Changes to support API schema

The following changes need to be made to support API schemas:

  • Given a schema generate the API schema.
  • Validation for before query planning should use API schema.

Returning of API schema needs to be a separate call to query planning as we will need to return the API schema for introspection.

Introspection fails if a variable is non-null

sf.

const { data, errors } = graphqlSync({ schema, source: query });

Given this pair of schema and query, introspection will always fail because the variable $num has a non-null type while the input variable set is empty.

Schema

schema {
    query: Query
    mutation: Mutation
}

type Bar {
    foobar: String!
}

type Foo {
    subquery(num: Int!): Bar!
}

type Query {
    foo: Foo!
}

type Mutation {
    update(num: Int!): Int!
}

Query

query GetFoo($num: Int!) {
  foo {
    subquery(num: $num) {
      __typename
    }
  }
}

This is also related to the apollo-router, since it always performs introspection on this query because of the request for a __typename field.

harmonizer: strongly typed hints

originally brought to our attention by @EverlastingBugstopper:

right now we're just calling .toString() on the hints from the new composition-js module, but there's actually more info there. we should strongly type the hints on harmonizer's side as well, and make sure we implement things like displaying AST node locations in Rover.

No actionable error message on locally missing subgraph type (`supergraph compose`)

supergraph compose fails with no actionable error messages when a subgraph depends on a type that it does not define, but which some other subgraph does.

For example, given a.graphql:

extend type Query {
  fieldA: Uri
}

and b.graphql:

extend type Query {
  fieldB: Uri
}

scalar Uri

and a supergraph.yml config file pointing at these files:

subgraphs:
  films:
    routing_url: https://a.example.com
    schema:
      file: ./a.graphql
  people:
    routing_url: https://b.example.com
    schema:
      file: ./b.graphql

then we get:

rover supergraph compose --config supergraph.yml
error[E029]: Encountered 1 build error while trying to build a supergraph.

Caused by:
    Something went wrong! No build errors were recorded, but we also build a valid supergraph SDL.
    
        The subgraph schemas you provided are incompatible with each other. See https://www.apollographql.com/docs/federation/errors/ for more information on resolving build errors.

This leads to the end-developer needing to perform a binary search, which is hard with an SDL because you have to be careful not to remove types that are depended on by other types, otherwise you get completely different errors.

cannot build rust harmonizer 0.1.2

harmonizer 0.1.2 seems to include an outdated rusty_v8 dependency that is brought in by an outdated deno_core, meaning we cannot currently build a release binary for rover. I believe dependabot has already made a patch to harmonizer but we need that patch to land so we can build for Rover's release.

In addition to this patch release, I think it would be good to remove the Cargo.lock line from harmonizer's .gitignore, and to also run tests in CI with the --locked flag. I'm not sure if that would have prevented this since I think it's denos fault for releasing something broken, but it wouldn't hurt!

xtask: create artefacts but don't publish them

Debugging build issues locally is a bit painful at the moment, because there is no simple way for us to generate a harmonizer package that looks the same as the one we would publish.

This means it was a bit hard for me to find out what was exactly happening in apollographql/rover#1030

a cargo xtask prepublish command would come in handy, it would:

  • Create a new folder, say prepublish
  • Copy the sources one would publish over there
  • Copy the Cargo.publish.toml and rename it to Cargo.toml

This means local rover developers would for example be able to target this prepublished directory and make sure everything is working as expected.

Moreover this would allow us to be more confident that what we publish is exactly what we're seeing.

create `RELEASE_CHECKLIST.md`

this should be similar to rover's release checklist, with a few key differences.

we will be cutting releases for each of the packages in this repository, described below:

source directory package delivery
./apollo-federation-types apollo-federation-types crates.io library
./harmonizer-0 harmonizer crates.io library
./harmonizer-2 harmonizer crates.io library
./router-bridge router-bridge crates.io library
./supergraph-0 supergraph GitHub release binary
./supergraph-2 supergraph GitHub release binary

the harmonizer and supergraph packages will be tested and published at the same time, therefore a release for those packages is kicked off when a composition/vx.x.x tag is pushed. our circle config will correctly create the proper stage from the version number (composition/v0* vs. composition/v2* will run cargo xtask prep -z 0 or -z 2). The other packages are easier and can just be published from source.

Below is a table describing what git tags will kick off a release for what packages and what directory the package will be published from.

initial tag source dir(s) xtask command publish dir(s)
[email protected] ./apollo-federation-types ./apollo-federation-types
[email protected] ./harmonizer-0, ./supergraph-0 cargo xtask prep -z 0 ./stage/harmonizer, ./stage/supergraph
[email protected] ./harmonizer-2, ./supergraph-2 cargo xtask prep -z 2 ./stage/harmonizer, ./stage/supergraph
[email protected] ./router-bridge ./router-bridge

it should also be noted in the checklist that composition jobs will fail to publish if apollo-federation-types has bumped its version and has not been published.

a release checklist should likely include this table or a link to these tables, and the instructions should be clear for each package.

investigate issue with creating GitHub release

i was able to work around this issue by using circleci's rerun with ssh, but i got a super cryptic error after the crate was published:

#!/bin/bash -eo pipefail
gh release create $RELEASE_NAME \
--title $RELEASE_NAME \
--notes 'This release was automatically created by [CircleCI](./.circleci/config.yml).

If you would like to verify that the binary you have downloaded was built from the source code in this repository, you can compute a checksum of the zipped tarball and compare it to the checksums that are included as release artifacts.

Binaries built for MacOS are signed, notarized, and automatically verified with [Gatekeeper](https://support.apple.com/guide/deployment-reference-macos/using-gatekeeper-apd02b925e38/web).' \
artifacts/*
stat [email protected]: no such file or directory

Exited with code exit status 1
CircleCI received exit code 1

GraphQL parsing errors could have a location

Description

When running a composition and one of the schemas does not parse successfully, a location (including ideally a reference to the schema file) could be displayed.

Repro where this is missing:

pcarrier@cat ~/repro> cupa *
--- 8< --- 8< --- demo.graphql --- 8< --- 8< ---
"Demo"
extend type Demo { id: ID! }
--- 8< --- 8< --- supergraph.yaml --- 8< --- 8< ---
subgraphs:
  demo:
    routing_url: undefined
    schema:
      file: ./demo.graphql
--- >8 --- >8 --- >8 --- >8 --- >8 ---
pcarrier@cat ~/repro> ~/.rover/bin/rover supergraph compose --config supergraph.yaml
error[E029]: Encountered 1 build error while trying to build a supergraph.

Caused by:
    Encountered 1 build error while trying to build the supergraph.
    UNKNOWN: Syntax Error: Unexpected Name "extend".

        The subgraph schemas you provided are incompatible with each other. See https://www.apollographql.com/docs/federation/errors/ for more information on resolving build errors.

Unable to use locally built supergraph binary on M1 macbook

When I run scripts/install.sh and then try to compose a supergraph, I get an error

โžœ rover supergraph compose --elv2-license accept --skip-update --config examples/basic/supergraph.yaml
error: Output from `/Users/itme/.rover/bin/supergraph-v2.0.0-preview.10 compose` was malformed.

Caused by:
    0: /Users/itme/.rover/bin/supergraph-v2.0.0-preview.10 compose output:
    1: EOF while parsing a value at line 1 column 0
        This error was unexpected! Please submit an issue with any relevant details about what you were trying to do: https://github.com/apollographql/rover/issues/new/choose

I can run the binary to get the version and it will tell me if I feed it an invalid YAML, but if once it gets to "real composition" I get an illegal hardware instruction

โžœ ~/.rover/bin/supergraph-v2.0.0-preview.10 compose --version
supergraph-v2.0.0-preview.10-compose 2.0.0-preview.10

โžœ ~/.rover/bin/supergraph-v2.0.0-preview.10 compose composition-tests/bonk.yaml 
{"Err":{"build_errors":[{"message":"Config for subgraph(s) [\"other\"] are not fully resolved. name, routing_url, and sdl must be present.","code":null,"type":"config"}]}}

โžœ ~/.rover/bin/supergraph-v2.0.0-preview.10 compose composition-tests/bare.yaml
[1]    36549 illegal hardware instruction  ~/.rover/bin/supergraph-v2.0.0-preview.10 compose composition-tests/bare.yaml

NOTE: I've permanently disabled Gatekeeper by running sudo spctl --master-disable since we were investigating if it was related to apollographql/rover#398

`cargo xtask tag` should sync local `composition-latest-X` tags

when running cargo xtask tag locally after someone other than you has updated the composition-latest-0/composition-latest-2 magic tags, your local tag will not match the remote, thus causing a massive git explosion as seen here:

$ cargo xtask tag --package [email protected] --real-publish
   Compiling xtask v0.1.0 (/home/a/work/federation-rs/xtask)
    Finished dev [unoptimized + debuginfo] target(s) in 1.69s
     Running `target/debug/xtask tag --package '[email protected]' --real-publish`
info: running `git fetch` in `/home/a/work/federation-rs`
info: running `git branch --show-current` in `/home/a/work/federation-rs`
info: running `git status -uno` in `/home/a/work/federation-rs`
info: running `cargo test --locked -- --nocapture` in `/home/a/work/federation-rs/federation-1`
info: running `cargo test --locked -- --nocapture` in `/home/a/work/federation-rs/federation-2`
info: running `cargo test --locked -- --nocapture` in `/home/a/work/federation-rs/federation-2/router-bridge`
info: running `cargo test --locked -- --nocapture` in `/home/a/work/federation-rs`
info: running `cargo fmt --all -- --check` in `/home/a/work/federation-rs/federation-1`
info: running `cargo clippy -- -D warnings` in `/home/a/work/federation-rs/federation-1`
info: running `cargo fmt --all -- --check` in `/home/a/work/federation-rs/federation-2`
info: running `cargo clippy -- -D warnings` in `/home/a/work/federation-rs/federation-2`
info: running `cargo fmt --all -- --check` in `/home/a/work/federation-rs/federation-2/router-bridge`
info: running `cargo clippy -- -D warnings` in `/home/a/work/federation-rs/federation-2/router-bridge`
info: running `cargo fmt --all -- --check` in `/home/a/work/federation-rs`
info: running `cargo clippy -- -D warnings` in `/home/a/work/federation-rs`
info: running `cargo build` in `/home/a/work/federation-rs/federation-1`
info: successfully compiled all packages in workspace root `/home/a/work/federation-rs/federation-1`
info: running `cargo build` in `/home/a/work/federation-rs/federation-2`
info: successfully compiled all packages in workspace root `/home/a/work/federation-rs/federation-2`
info: running `cargo build` in `/home/a/work/federation-rs/federation-2/router-bridge`
info: successfully compiled all packages in workspace root `/home/a/work/federation-rs/federation-2/router-bridge`
info: running `cargo build` in `/home/a/work/federation-rs`
info: successfully compiled all packages in workspace root `/home/a/work/federation-rs`
info: running `git fetch` in `/home/a/work/federation-rs`
info: running `git branch --show-current` in `/home/a/work/federation-rs`
info: running `git status -uno` in `/home/a/work/federation-rs`
info: running `git pull` in `/home/a/work/federation-rs`
info: running `git fetch --tags` in `/home/a/work/federation-rs`
From https://github.com/apollographql/federation-rs
 ! [rejected]        composition-latest-2 -> composition-latest-2  (would clobber existing tag)

Error: Encountered an issue while executing `git fetch --tags`

Caused by:
    Exited with status code 1

BOO!! Now our release isn't happening ๐Ÿ‘Ž๐Ÿผ ๐Ÿ˜ข ๐Ÿ˜ฟ

@benweatherman pointed this out in #126 and attempted to document it, but I shot him down thinking that cargo xtask tag should just take care of this. "For sure! It will loop over all the local tags and delete them one by one, and then fetch the tags from the remote." No. Not at all. What a fool I was! It turns out that in this case (src), we never reach the point where we would delete all the local tags, because we run git fetch --tags before we try to iterate over the tags to delete. Since composition-latest-2 exists on the remote and locally, the fetch fails. Since this step fails, xtask tag does not continue and we get no release party.

I think we should be able to avoid all of this nonsense and remove some steps in xtask tag to make releasing a bit smoother - let's just update git fetch --tags to git fetch --tags --force and call it a day.

A path to upgrade deno on harmonizer (so it can be upgraded workspace wide)

the router-bridge is currently stuck relying on deno-core version 0.124.0 because there s a dependency conflict with once_cell

error: failed to select a version for `once_cell`.
    ... required by package `deno_core v0.135.0`
    ... which satisfies dependency `deno_core = "^0.135.0"` of package `router-bridge v0.1.0 (/Users/ignition/projects/apollo/federation-rs/federation-2/router-bridge)`
versions that meet the requirements `^1.10.0` are: 1.12.0, 1.11.0, 1.10.0

all possible versions conflict with previously selected packages.

  previously selected package `once_cell v1.9.0`
    ... which satisfies dependency `once_cell = "=1.9.0"` of package `deno_core v0.118.0`
    ... which satisfies dependency `deno_core = "^0.118.0"` of package `harmonizer v2.0.2 (/Users/ignition/projects/apollo/federation-rs/federation-2/harmonizer)`
    ... which satisfies path dependency `harmonizer` (locked to 2.0.2) of package `supergraph v2.0.2 (/Users/ignition/projects/apollo/federation-rs/federation-2/supergraph)`

failed to select a version for `once_cell` which could resolve this conflict

The main issue preventing us from bumping harmonizer is that a lot of things changed between deno-core 0.118.0 and 0.124.0.

Here's a rough list of things that changed, the actual upgrade that happened in the bridge can be found in this comment

callback declaration is done through the #[op] macro:

A lot of the complexity/machinery in worker.rs is due to our planner being long lived, and passing messages back and forth from / to Rust and the Deno runtime. This probably doesn't need to be ported since the cli is probably short lived overall.

I would highly recommend us to first try to bump Harmonizer to 0.124.0, before we try to bump both to 0.135.0.

@EverlastingBugstopper are there some caveats in harmonizer land that we should keep in mind if we wanna tackle this? wdyt? ๐Ÿค—

feat: add context to composition errors

While using rover supergraph compose with multiples sdl schema, it would save time to add the context of the error: the file that will generate the error and the line.

To add these errors context, it would need to update the harmonize javascript library before adding it to rover, so I opened an issue here.

feat(composition): add hint levels to output types

hints don't need to be addressed! they're helpful to have, but having them as warnings may push folks to want to fix/remove them. we should just call them hints instead and let people keep them around.
these warning prefixes can be found here

automate the `harmonizer` and `router-bridge` releases to crates.io

We should have an automatic pipeline to release the harmonizer crate to Crates.io, much like the rest of the npm packages in this repository can be published from CI.

Today, the harmonizer package is manually published to Crates.io, rather than using any sort of automated process that we have setup for the rest of this repository using lerna.

As a preliminary consideration though, we might want to consider whether this entire mechanism might best live outside of this repository. E.g.,

  1. Move harmonizer to another small repository that has:

    1. An npm package that depends on the npm package @apollo/federation.
    2. The harmonizer crate source
  2. Set it up so that Renovate automatically bumps the npm dependency @apollo/federation when it is published from this federation repository.

  3. During a an automatic CI workflow:

    1. Set the harmonizer's Cargo.toml version to be === to the @apollo/federation package.json's version property.
    2. Wholesale publish the harmonizer package.

A possible downside to this is that it would be more difficult to build rover (which depends on harmonizer) while iterating on a version of @apollo/federation. We should determine if that's a requirement.

There is only one published Rust crate in this repository, so I don't believe anything like lerna is necessary right now, but it's worth noting that https://github.com/pksunkara/cargo-workspaces exists as a solution that claims to do the DAG ordering/resolution of multi-package Cargo workspaces and that there is no such thing as cargo publish --all for workspaces right now.

Related TODOs

Running `cargo xtask dist` on OSX result in an error

โžœ cargo xtask dist --debug                              
    Finished dev [unoptimized + debuginfo] target(s) in 0.07s
     Running `target/debug/xtask dist --debug`
error: 'unknown-target' isn't a valid value for '--target <target>'
        [possible values: x86_64-apple-darwin, x86_64-pc-windows-msvc, x86_64-unknown-linux-gnu]

Passing a --target explicitly fixes the error:

โžœ cargo xtask dist --target x86_64-apple-darwin  --debug
    Finished dev [unoptimized + debuginfo] target(s) in 0.07s
     Running `target/debug/xtask dist --target x86_64-apple-darwin --debug`
info: running `cargo build --workspace --target x86_64-apple-darwin` in `/Users/weatherman/code/apollographql/federation-rs/federation-1`
info: running `cargo build --workspace --target x86_64-apple-darwin` in `/Users/weatherman/code/apollographql/federation-rs/federation-2`
info: running `cargo build --workspace --target x86_64-apple-darwin` in `/Users/weatherman/code/apollographql/federation-rs`
Success!

I would expect running cargo xtask dist would run successfully on OSX without needing to specify a --target.

remove `js_types` and only use `apollo_federation_types`

this should happen in both harmonizer-0 and harmonizer-2. additionally we should make sure that the errors sent from harmonizer-0 are structured properly with a [{message: ${message}}] instead of just [err] so that we don't get so many "unknown errors" in federation 1 output

When `CannotFindBinaryPath`, display name of binary that cannot be found

Currently, if you try building this without npm installed, you'll receive a message like this:

`Result::unwrap()` on an `Err` value: CannotFindBinaryPath', /usr/local/cargo/git/checkouts/federation-rs-edb4f8c0f5624a5c/33659ef/federation-2/router-bridge/build.rs:23:35

Nowhere in the message does it say what binary cannot be found. Can we improve that message to help self-diagnosis and resolution?

Reproduction steps

You can probably just get npm out of your path and reproduce this โ€”ย idk, mv npm npm.bu โ€” but Docker is also a reasonable place to try this!

create function in all composition packages for determining if fed1 or fed2

in harmonizer-0 and harmonizer-2:

pub fn get_compatible_versions(subgraph_definitions: &[SubgraphDefinition]) -> semver::VersionReq

and in supergraph-0 and supergraph-2 there should be a supergraph version detect --config resolved_supergraph.yml command that spits out json including information about the current binary and information about the version of federation required by the resolved subgraphs. something like this: { "current_version: 0.33.8", "required_version": "^2.0.0" }

then in rover we'll do the work of making sure that we download an appropriate version. the installer will come pre-loaded with the latest versions of the supergraph-0 and supergraph-2 binaries, and will download newer ones as appropriate. This makes it so we can keep all the deno bits in this repo and not have to leak it into rover or create some separate crate.

CI: publish dry runs ?

Once I'm done publishing the bridge, I would like to see if we can think of a way to run publish dry runs in CI.

This would allow us to catch and prevent our next publish from being painful.

It may or may not be an issue once everything is wired up.

rust harmonizer missing some implementations

The Rust API Guidelines list a bunch of common traits that are eagerly implemented for most library types. I think we should implement most of these for ServiceDefinition (and maybe rename it SubgraphDefinition) so it's easier to work with in tests.

I'd love to do this:

        let actual_film_subgraph = subgraph_definitions.get(0).unwrap();
        let actual_people_subgraph = subgraph_definitions.get(1).unwrap();

        let expected_film_subgraph = SubgraphDefinition::new(
            "films",
            "https://films.example.com",
            "there is something here",
        );
        let expected_people_subgraph = SubgraphDefinition::new(
            "people",
            "https://people.example.com",
            "there is also something here",
        );

        assert_eq!(actual_film_subgraph.clone(), expected_film_subgraph);
        assert_eq!(actual_people_subgraph.clone(), expected_people_subgraph);

Unfortunately I can't seem to compare these two quite like this because ServiceDefinition does not implement Clone and either Eq or PartialEq... or at least I think that's why I can't do the above. I've got something working but would be nice to add these implementations for sure!

This is what I'm doing for now:

        assert_eq!(actual_film_subgraph.name, "films");
        assert_eq!(actual_film_subgraph.url, "https://films.example.com");
        assert_eq!(actual_film_subgraph.type_defs, "there is something here");
        assert_eq!(actual_people_subgraph.name, "people");
        assert_eq!(actual_people_subgraph.url, "https://people.example.com");
        assert_eq!(actual_people_subgraph.type_defs, "there is also something here");
        ```

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.