Comments (5)
Hey - I've done exactly what you're looking for with this code:
This assumes rspec running with --json
output (rather than junit, although you could do something similar with junit), written to the spec/reports/rspec*.json
directory. It works well for us in conjunction with the github artifacts uploader/downloader collecting the json formatted spec result from each node:
- name: Upload json rspec reports
uses: actions/upload-artifact@v3
if: always()
continue-on-error: true
with:
name: rspec-json-reports
retention-days: 10
path: |
/your/app/spec/reports/*.json
when rspec is run like:
bundle exec rspec --format json --out spec/reports/rspec_<RSPEC_NODE_NUMBER>.json
# frozen_string_literal: true
require_relative "rspec_report_merger"
require "terminal-table"
require "json"
class GithubActionsStepSummaryCreator
def initialize(overall_summary = RSpecReportMerger.new.overall_summary,
per_dir_summary = RSpecReportMerger.new.per_dir_summary)
@overall_summary = overall_summary
@per_dir_summary = per_dir_summary
@spec_reports = Dir["spec/reports/rspec*.json"]
raise "Error: No spec reports discovered!" if @spec_reports.empty?
end
def headings
["directory"] + @overall_summary.keys
end
def main_app_suite_data
[["**All tests**"] + @overall_summary.values.map{ |v| v.round(2) }]
end
def per_dir_rows
@per_dir_summary.map do |dir_spec|
[
dir_spec.first,
dir_spec.last["duration"],
dir_spec.last["example_count"],
dir_spec.last["failure_count"],
dir_spec.last["pending_count"],
]
end
end
def add_step_summary
rows = main_app_suite_data + per_dir_rows
rows.sort_by!{ |data| -data[1] } # sort by duration
table = Terminal::Table.new(
headings: headings,
rows: rows,
style: { border: :markdown }
)
puts table
message = "### Spec report: :rocket: \n \n #{table}"
File.write(ENV["GITHUB_STEP_SUMMARY"] || "/tmp/summary", message, mode: "a+")
end
def run
add_step_summary
end
end
GithubActionsStepSummaryCreator.new.run
# frozen_string_literal: true
class RSpecReportMerger
def initialize(options = {})
@spec_reports = Dir["spec/reports/rspec*.json"]
@options = options
raise "Error: No spec reports discovered!" if @spec_reports.empty?
end
def read_reports(key)
@spec_reports.flat_map do |file|
JSON.parse(File.read(file)).fetch(key)
end
end
def per_dir_summary
# examples grouped by path
grouped = read_reports("examples").group_by do |example|
example["file_path"].split("/").first(3).last(2)
end
result = {}
grouped.each_key do |key|
duration = grouped[key].map{ |ex| ex["run_time"] }.sum
status_counts = grouped[key].map{ |ex| ex["status"] }.tally
result[key.join("/")] = {
"example_count" => grouped[key].count,
"failure_count" => status_counts["failed"] || 0,
"pending_count" => status_counts["pending"] || 0,
"duration" => (duration/60).round(2)
}
end
result
end
def overall_summary
result = {
"duration" => 0,
"example_count" => 0,
"failure_count" => 0,
"pending_count" => 0,
}
summaries = read_reports("summary")
summaries.each do |summary|
result["duration"] += summary["duration"]
result["example_count"] += summary["example_count"]
result["failure_count"] += summary["failure_count"]
result["pending_count"] += summary["pending_count"]
end
result
end
end
The code is quick and dirty and could definitely be improved, but it works. Here's what it looks like:
Note if you're missing rspec data and are running in queue mode, you might need to follow these instructions:
https://docs.knapsackpro.com/ruby/rspec/#queue-mode
from knapsack_pro-ruby.
@singhprd Thank you for sharing this example!
from knapsack_pro-ruby.
We can use rspec_junit_formatter
gem to generate an XML report with tests results. We would need something to convert the XML report to a nice markdown summary.
from knapsack_pro-ruby.
Ideas
Here are ideas what to do before we close this issue. Info for Knapsack team:
-
Maybe we could add an example of using the above script and GitHub Actions to our Rails example project. Or we could create an article on our blog.
-
In the case of Queue Mode, we could generate JSON reports this way.
-
We could then update cookbook docs and add the example config that uses GitHub Actions job summaries.
-
In the installation guide, we could let users know that they can check the cookbook if they want.
story
from knapsack_pro-ruby.
@singhprd If you would like to publish on our blog we are open for Pull Request. Some of our users already published their solutions.
from knapsack_pro-ruby.
Related Issues (20)
- Queue mode is running much slower than regular mode with Knapsack Pro HOT 23
- Ability to ignore some CI nodes HOT 11
- NameError: uninitialized constant RSpec::Core::Version HOT 10
- Feature requests(adding --seed & --bisect to rspec debug command) HOT 4
- Is it possible to retry failing specs on another worker? HOT 6
- Add configuration to write to /tmp directory instead of .knapsack_pro HOT 9
- Log output understanding question HOT 4
- Ci failing with missing file error HOT 4
- Ability to set an exit code for RSpec failure HOT 4
- Include error messages from dry run json report in stdout when knapsack_pro:rspec_test_example_detector fails HOT 8
- rspec_junit_formatter still producing invalid XML after applying suggestion from #40 HOT 3
- Queue mode causes rspec_junit_formatter to produce invalid XML when a node runs a spec with all examples commented out. HOT 5
- Thread.join causes knapsack to hang on circleCI HOT 13
- Enabled KNAPSACK_PRO_RSPEC_SPLIT_BY_TEST_EXAMPLES and now get uninitialized constant at end of run HOT 10
- Regular mode (non-queue) rspec outputs all spec file names on startup HOT 4
- All examples skipped after exception in before: suite hook in queue mode HOT 2
- Memory leak HOT 2
- Lots of failures with rolify gem after 4.1 -> 5.0 bump HOT 12
- Docs need to specify ENV var to detect CI provider HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from knapsack_pro-ruby.