Giter VIP home page Giter VIP logo

Comments (6)

kadrach avatar kadrach commented on July 27, 2024

The patch below enables scraping of subgroups, as well as scraping "all" projects by default. Note that with a large number of projects visible, this directly leads to "too many open files" failures. Some sort of queueing and backoff is probably required here, have not looked at this in detail.

diff --git a/cmd/config.go b/cmd/config.go
index 8340e10..460ced5 100644
--- a/cmd/config.go
+++ b/cmd/config.go
@@ -35,8 +35,9 @@ type Project struct {
 type Wildcard struct {
 	Search string
 	Owner  struct {
-		Name string
-		Kind string
+		Name             string
+		Kind             string
+		IncludeSubgroups bool `yaml:"include_subgroups"`
 	}
 	Refs string
 }
diff --git a/cmd/gitlab.go b/cmd/gitlab.go
index f3b9b3e..e720f85 100644
--- a/cmd/gitlab.go
+++ b/cmd/gitlab.go
@@ -66,6 +66,7 @@ func (c *Client) listProjects(w *Wildcard) ([]Project, error) {
 			gps, resp, err = c.Groups.ListGroupProjects(
 				w.Owner.Name,
 				&gitlab.ListGroupProjectsOptions{
+					IncludeSubgroups: &w.Owner.IncludeSubgroups,
 					ListOptions: listOptions,
 					Archived:    &falseVal,
 					Simple:      &trueVal,
@@ -73,7 +74,14 @@ func (c *Client) listProjects(w *Wildcard) ([]Project, error) {
 				},
 			)
 		default:
-			return projects, fmt.Errorf("Invalid owner kind '%s' must be either 'user' or 'group'", w.Owner.Kind)
+			gps, resp, err = c.Projects.ListProjects(
+				&gitlab.ListProjectsOptions{
+					ListOptions: listOptions,
+					Archived:    &falseVal,
+					Simple:      &trueVal,
+					Search:      &w.Search,
+				},
+			)
 		}
 
 		if err != nil {

from gitlab-ci-pipelines-exporter.

mvisonneau avatar mvisonneau commented on July 27, 2024

From what I understood of @vakaobr request, he wants the ability to automatically discover submodules within projects. This is going to be quite challenging to implement as this would imply fetching the necessary git content of the repository and then parse it. I do not plan to work on this myself but PRs are welcome.

On the other end, thanks for the patch @kadrach, I'll try to implement it!

from gitlab-ci-pipelines-exporter.

mvisonneau avatar mvisonneau commented on July 27, 2024

done in bace584

thanks again @kadrach 🙏

from gitlab-ci-pipelines-exporter.

vakaobr avatar vakaobr commented on July 27, 2024

hooray!

from gitlab-ci-pipelines-exporter.

toniiiik avatar toniiiik commented on July 27, 2024

good job but...

There is wrong property name in README.md config.yml
includeSubgroups: true
must be
include_subgroups: true

from gitlab-ci-pipelines-exporter.

mvisonneau avatar mvisonneau commented on July 27, 2024

README.md has been fixed since and we do not have it as part of the example config.yml anymore, thanks for the info @toniiiik.

I will close this issue as I reckon that you can probably workaround it by playing with wildcards.

from gitlab-ci-pipelines-exporter.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.