Is your feature request related to a problem? Please describe.
The current test suite is quite limited to simple schemas that I was able to come up with. It will be nice to have tests against real-life GraphQL schemas, similar to what hypothesis-jsonschema does.

Describe the solution you'd like
Build a test corpus from something like https://github.com/APIs-guru/graphql-apis and run query generation tests against them.

Describe the bug
When a GraphQL query type has multiple possible types on the same level (e.g via union) and those types have fields with the same name but different types, then hypothesis-graphql generates invalid queries

To Reproduce
Schema:

interface Conflict {
  id: ID
}

type FloatModel implements Conflict {
  id: ID,
  query: Float!
}

type StringModel implements Conflict {
  id: ID,
  query: String!
}

type Query {
  getData: Conflict
}

Generated query:

{
  getData {
    ... on FloatModel {
      query
    }
    ... on StringModel {
      query
    }
  }
}

Error: Fields 'query' conflict because they return conflicting types 'Float!' and 'String!'. Use different aliases on the fields to fetch both if this was intentional.

Additional context
Comes from this job

If there are no strategies provided (as #22 is implemented), skip fields with user-defined types rather than raise errors.

It could be a config option passed to the query function

Now it is not bounded, which may lead to very long builds and general performance issues. I assume we can utilize recursive or deferred here

At the moment, data generation is relatively slow. On my machine (i7-8700K), generation of 100 queries for this schema takes on average 0.8 seconds:

type Book {
  title: String
  author: Author
}

type Author {
  name: String
  books: [Book]
}

enum Color {
  RED
  GREEN
  BLUE
}

input QueryInput {
  eq: String
  ne: String
}

input NestedQueryInput {
  code: QueryInput
}

type Model {
  int: Int,
  float: Float,
  string: String
  id: ID,
  boolean: Boolean
  color: Color
}

type Query {
      getModel(int: Int): Model
}

The version without AST nodes was ~3 times faster. Not a priority right now definitely, but we can investigate the cause in the future and improve performance

Describe the bug
If arguments in the input query type contain required fields, they should be present in generated queries. Otherwise, such a query is invalid.

Example:

input RequiredInput {
  eq: Float!,
}

type Query { 
  getData(query: RequiredInput): Float! 
}

Generated query:

{
  getData(query: {})
}

Error: Field 'RequiredInput.eq' of required type 'Float!' was not provided.

Similar to #49 but for input arguments

Currently query doesn't generate arguments. E.g.

type Query {
    getBooksByAuthor(name: String) [Book]
}

name will not be generated.

Can be done by adding arguments to a field node

It will be really helpful to verify generated queries against a real GraphQL server

Otherwise it is an error like this:

Field \"continents\" of type \"[Continent!]!\" must have a selection of subfields. Did you mean \"continents { ... }\"?

At the moment, when a query is generated here a dictionary is created on the first pass, then it is traversed again and converted into an AST tree. It could be done in one pass, which will be more efficient

Queries are supported, mutations aren't much different. The most of the required machinery is already there

At the moment only GraphQLScalarType, GraphQLList and GraphQLObjectType are handled. But there could be more different types which are listed in a union type GraphQLOutputType:

• - GraphQLScalarType
• - GraphQLObjectType
• - GraphQLInterfaceType
• - GraphQLUnionType
• - GraphQLEnumType
• - GraphQLWrappingType

Is your feature request related to a problem? Please describe.
At the moment, if a custom scalar is nullable, it is skipped from arguments. But null is a valid value, which is not generated. Example query:

{
  countries(filter: {code: {eq: null}}) {
    languages {
      name
      native
    }
  }
}

Describe the solution you'd like
Generate NullValueNode for such cases

Even though that on the network layer non-printable chars might be compressed, etc it may decrease debuggability. Consider:

Falsifying example: test(
    case=GraphQLCase(body='{\n  search {\n    operationLocations {\n      UIC\n    }\n  }\n}\n'),
)

vs. (some minimal whitespaces):

Falsifying example: test(
    case=GraphQLCase(body='{ search { operationLocations { UIC } } }'),
)

Of course, it might be pretty-printed, but Hypothesis output is the primary source of description, and it displays a raw string. Therefore it is crucial to have it as readable as possible.

Implementation notes:

• Create a subclass of graphql.language.printer.PrintAstVisitor;
• Redefine methods that do pretty-printing and make them return a more compact definition

Mostly it should be enough to replace some "\n" with " "

It is helpful for packagers if the source tarball includes tests.

Describe the bug
The GraphQL spec defines Int! as an integer value in [-2 ** 31:2 ** 31 - 1] range. The current implementation may generate queries that will be rejected by graphql.validate with Int cannot represent non 32-bit signed integer value: -2147483649.

Sometimes types contain default parameters - use them in data generation

Is your feature request related to a problem? Please describe.
Interfaces are ignored now which leads to invalid queries and cases when fields from implementers are not used at all

tox, tests, etc.

Is your feature request related to a problem? Please describe.
Unit types are completely ignored at the moment, which leads to invalid queries.

There is a bug that occurs when a GraphQL schema has an optional enum argument on an input object. One of the values that hypothesis-graphql generates is: EnumValueNode. I'm guessing that this is happening during the case when the enum value is supposed to be null.

To demonstrate, run the following with pytest -s:

import schemathesis

schema = schemathesis.graphql.from_file(
    """
    enum MyEnum {
        FOO
        BAR
        BAZ
    }
    
    input MyInput {
        value: MyEnum
    }
    
    type Query {
        hello(i: MyInput!): String
    }
    """
)

@schema.parametrize()
def test_schema(case):
    print(case.body)

The printed output is the following:

{
  hello(i: {value: FOO})
}

{
  hello(i: {value: EnumValueNode})
}

{
  hello(i: {value: BAZ})
}

{
  hello(i: {value: FOO})
}

{
  hello(i: {value: BAR})
}

I am using:
schemathesis 3.6.3
hypothesis-graphql 0.4.1
graphql-core 3.1.4

To avoid using username / password we could use something like this job

If a field is chosen for generation and contains a not-nullable scalar type, it will lead to TypeError. It will be better to skip such fields from the generation in the first place - then it will be possible to generate queries.

I'm trying to figure out why I keep getting the following error: UnicodeEncodeError: 'utf-8' codec can't encode characters in position 0-1: surrogates not allowed

For some reason, the text strategy for arguments of type String is generating unicode data that cannot be encoded as UTF-8, because it includes surrogate characters.

From what I can tell, hypothesis-graphql is using hypothesis.strategies.text() for String-based arguments: https://github.com/Stranger6667/hypothesis-graphql/blob/master/src/hypothesis_graphql/_strategies/primitives.py#L46

The documentation for hypothesis.strategies.text() explicitly states:

The default alphabet strategy can generate the full unicode range but excludes surrogate characters because they are invalid in the UTF-8 encoding.

https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.text

So I'm very confused why surrogate characters are being produced.

To Reproduce
Run this test using pytest. You should get a UnicodeEncodeError. You can include --log-cli-level=INFO to see all the values being generated for the user argument.

import logging
import graphql
import schemathesis

schema = schemathesis.graphql.from_file(
    """
    type Query {
        hello(user: String!): String
    }
    """
)

logger = logging.getLogger()

@schema.parametrize()
def test_schema(case):
    document = graphql.parse(case.body)
    argument_node = document.definitions[0].selection_set.selections[0].arguments[0]
    if argument_node.name.value == "user":
        value = argument_node.value.value
        logger.info(f"User argument = {value!r}")
        encoded = value.encode("utf8")

Expected behavior
The text strategy used for String-based arguments includes only characters that can be UTF-8 encoded.

Environment (please complete the following information):

• OS: Linux
• Python version: 3.8.5
• schemathesis version: 3.6.1
• hypothesis-graphql version: 0.4.0
• hypothesis version: 6.8.9
• graphql-core version: 3.1.4
• pytest version: 6.2.3

This is a great task for new developers.

This library has type annotations, but is missing py.typed file.
But, it is required to make types distributable.
See https://www.python.org/dev/peps/pep-0561/

For example, if there is a custom scalar type:

scalar Date

Then hypothesis-graphql doesn't know how to generate data for it, but it would be nice to have a way to generate it via custom strategies provided by the user. Something like this:

from hypothesis_graphql import strategies as gql_st
from hypothesis import strategies as st

@given(case=gql_st.query(schema, custom_types={"Date": st.dates().map(str)}
def test_something(case):
    ...

Implementation notes:

• Add custom_types argument to the hypothesis_graphql._strategies.queries.query function. Its type is Dict[str, SearchStrategy];
• Pass it down to the hypothesis_graphql._strategies.primitives.scalar function;
• At the end of the function, add a check if the scalar name is in the custom_types dictionary. Then use a strategy from this there;
• If type is nullable, then combine that strategy with st.none() and return it to the caller

The current Graphene version does not support graphql-core>=3 and it will take some more time ... It seems like the AST part we're using is not much difference between these versions and it should be easy to support both versions of graphql-core

Still there. The TMDB schema from the corpus and the following query is generated:

{
  people {
    trending(
      last: 370363096
      after: ""
      timeWindow: Day
      first: null
      before: "\u001bh\u0015"
    ) {
      totalCount
    }
    person(id: 1427696479) {
      profilePicture(size: W45)
      knownForDepartment
      isAdult
      alsoKnownAs
      name
      imdbID
    }
    popular(
      last: 10124
      after: "\u0087\u00d4\uc03d\ub8da\u0011\u00ae;\u22fd\u0084\u008f)\uf674\u1fd2\u00899"
      first: null
      before: null
    ) {
      pageInfo {
        startCursor
      }
      totalCount
      edges {
        cursor
      }
    }
    search(last: -249, after: "\u00ceh\u8341", term: "", first: null, before: null) {
      totalCount
    }
  }
  movies {
    topRated(
      last: -126
      after: "\u00a9\u00c5\u0092j\u00e3(*PLE"
      first: 149
      before: "\u0084\u00c4i\u00b8\u6364\u754dw"
    ) {
      edges {
        cursor
      }
      totalCount
    }
    trending(
      last: -132
      after: "\u0099"
      timeWindow: Day
      first: -61425
      before: null
    ) {
      totalCount
      pageInfo {
        endCursor
        hasNextPage
        hasPreviousPage
        startCursor
      }
    }
    popular(last: null, after: null, first: -102257250, before: null) {
      pageInfo {
        hasNextPage
        hasPreviousPage
        endCursor
      }
    }
    upcoming(last: -77, after: null, first: null, before: null) {
      edges {
        cursor
        node {
          title
          releaseDate
          id
          videos {
            type
          }
          translations {
            iso639_1
          }
          ... on DetailedMovie {
            runtime
            images {
              posters {
                width
                voteCount
                image(size: W92)
              }
            }
            poster(size: W92)
            translations {
              iso3166_1
              info {
                overview
                title
              }
              iso639_1
            }
            externalIds {
              instagram
              facebook
              twitter
              imdb
            }
            backdrop(size: W1280)
            productionCompanies {
              id
              name
              logo(size: W154)
              originCountry
            }
            releaseDate
            homepage
            imdbID
          }
          ... on MovieResult {
            reviews(first: null, before: null, last: 197, after: null) {
              pageInfo {
                startCursor
              }
            }
            originalTitle
            alternativeTitles {
              type
              title
            }
          }
        }
      }
      pageInfo {
        hasNextPage
      }
      totalCount
    }
    nowPlaying(last: null, after: null, first: null, before: null) {
      pageInfo {
        endCursor
        startCursor
        hasNextPage
        hasPreviousPage
      }
    }
    search(
      last: null
      after: "\u001c\u0013"
      term: "\u009b\u0089\t\u3068"
      first: null
      before: "\u249b\u0094\ue856\u00fa`\u9847\u00d8Ql\u0080\u009fU"
    ) {
      totalCount
      pageInfo {
        endCursor
      }
      edges {
        cursor
      }
    }
  }
  trending(
    last: null
    after: "\u00adh\uebd3"
    timeWindow: Week
    first: null
    before: "\u001c"
  ) {
    pageInfo {
      startCursor
      endCursor
      hasNextPage
      hasPreviousPage
    }
    edges {
      node {
        ... on PersonListResult {
          isAdult
        }
        ... on MovieResult {
          originalTitle
        }
        ... on PersonListResult {
          details {
            alsoKnownAs
            birthday
            isAdult
            externalIds {
              imdb
            }
            details {
              details {
                placeOfBirth
              }
            }
            images {
              voteCount
              height
              aspectRatio
              width
              iso639_1
              image(size: Original)
              voteAverage
            }
          }
          popularityIndex
        }
        ... on PersonListResult {
          externalIds {
            twitter
            imdb
          }
          images {
            width
            height
          }
        }
        ... on PersonListResult {
          profilePicture(size: Original)
          images {
            width
          }
        }
        ... on MovieResult {
          title
        }
        ... on TVShowResult {
          firstAirDate
          translations {
            iso639_1
            localizedLanguage
            iso3166_1
          }
          poster(size: W185)
          videos {
            type
            thumbnail
            links {
              fireTV
              androidTV
              tvOS
            }
            iso3166_1
            site
          }
          numberOfRatings
          details {
            id
            productionCompanies {
              id
            }
            rating
            firstAirDate
            backdrop(size: W300)
            homepage
          }
          externalIds {
            twitter
          }
          overview
          originalName
          rating
          recommendations(first: null, before: null, last: 44267, after: null) {
            totalCount
            pageInfo {
              endCursor
            }
          }
          popularityIndex
          originalLanguage
          streamingOptions {
            bestOffering {
              links {
                web
                androidTV
                tvOS
                fireTV
              }
              resolution
              type
            }
            offerings {
              links {
                fireTV
                androidTV
              }
            }
          }
        }
        ... on PersonListResult {
          profilePicture_1: profilePicture(size: Original)
          images {
            width
          }
        }
        ... on MovieResult {
          title
        }
        ... on TVShowResult {
          firstAirDate
          translations {
            iso639_1
            localizedLanguage
            iso3166_1
          }
          poster_1: poster(size: W185)
          videos {
            type
            thumbnail
            links {
              fireTV
              androidTV
              tvOS
            }
            iso3166_1
            site
          }
          numberOfRatings
          details {
            id
            productionCompanies {
              id
            }
            rating
            firstAirDate
            backdrop_1: backdrop(size: W300)
            homepage
          }
          externalIds {
            twitter
          }
          overview
          originalName
          rating
          recommendations_1: recommendations(
            first: null
            before: null
            last: 44267
            after: null
          ) {
            totalCount
            pageInfo {
              endCursor
            }
          }
          popularityIndex
          originalLanguage
          streamingOptions {
            bestOffering {
              links {
                web
                androidTV
                tvOS
                fireTV
              }
              resolution
              type
            }
            offerings {
              links {
                fireTV
                androidTV
              }
            }
          }
        }
      }
    }
  }

Error:

Fields 'popularityIndex' conflict because they return conflicting types 'Float!' and 'Float'. Use different aliases on the fields to fetch both if this was intentional.

GraphQL request:207:11
206 |           }
207 |           popularityIndex
    |           ^
208 |         }

GraphQL request:347:11
346 |           }
347 |           popularityIndex
    |           ^
348 |           originalLanguage

I am not sure how good shrinking works with the current implementation and if it could be improved. We can try to build some app first, add a target error and check how good it will be in finding a minimal example.

There is no concrete plan on my mind now since I don't have much experience with Hypothesis internals, maybe we can get some advice from Hypothesis devs

Describe the bug
Unable to import from_schema from hypothesis_graphql

To Reproduce
Steps to reproduce the behaviour:

1. My GraphQL schema is - the sample schema provided in the README
2. Run this command - python3 hypothesis_graphql.py
3. See error -
   ImportError: cannot import name 'from_schema' from partially initialized module 'hypothesis_graphql' (most likely due to a circular import) (file-path/hypothesis_graphql.py)

Expected behavior
Imports to go through correctly

Environment (please complete the following information):

• OS: Mac
• Python version: 3.9.6 and also noticed with 3.11.1
• hypothesis-graphql version: 0.11.0
• hypothesis version: 6.100.0