Automated tests about pony-stable HOT 8 OPEN

@rawkode assume. i just sent you an invite to be part of our "contributor" team. it doesn't give you much in the way of privileges but does allow for you to appear in the "Assignee" list and be assigned to issues you are interested in working on. Baby steps towards committer!

from pony-stable.

@jemc @Theodus any thoughts on what automated tests for stable would look like?

from pony-stable.

Maybe it could be an integration test that sets up an example bundle.json that clones its own repo (pony-stable), and imports some types to prove that it worked?

from pony-stable.

@jemc i was just thinking the same thing. and then compile the program with stable env ponyc.

we can verify

• bundle file is correct after the add (this could be expanded for each type eventually)
• stable env ponyc compiles program

eventually i think we'd want to test other features like

• tag
• local-git
• github
• stable add

from pony-stable.

@rawkode any interest in this one? it's a little more advanced but probably manageable as a start. there's a bit of "how much can we use ponytest for" (a lot see https://github.com/ponylang/changelog-tool/blob/master/tests/main.pony tests)? and some would have to be some kind of automated integration test(s), so there's a bit of complication but its pretty straightforward. if you are interested, i'd say drop idea here for tests and what the runner would be and @jemc and I could give feedback.

from pony-stable.

@SeanTAllen Yeah, I'll look at what I can do and come back with a plan 👍

from pony-stable.

So this is pretty tough.

My initial thought is that we need a JSON Schema that will evolve with the tool and provide a means of documenting the format for other integrations; as well a confidence that stable conforms with said schema.

From what I can tell, we could probably use ponytest for arbitary comparison tests, but it wouldn't be pretty and we'd need to provide the json for comparison. Which means, in order for the test not to be flaky, we need to add branching/tag/commit sha support to stable add

assert_eq("json", loaded_bundle.json)

Hopefully any assumptions I've made are correct 😄

from pony-stable.

I'll try pushing this forward. #70 gets some scaffolding in place, leaving open the following test TODOs:

• stable fetch
• stable add
• stable env: #73

from pony-stable.