voxpupuli / puppet-ghostbuster Goto Github PK
View Code? Open in Web Editor NEW๐ป Dead code detector for Puppet
License: Apache License 2.0
๐ป Dead code detector for Puppet
License: Apache License 2.0
When I tried to run it with puppet-lint 2.0.2 installed on my system I got:
$ find -type f -exec puppet-lint --only-checks ghostbuster_classes,ghostbuster_defines,ghostbuster_facts,ghostbuster_files,ghostbuster_functions,ghostbuster_hiera_files,ghostbuster_templates,ghostbuster_types {} \+
/Users/amos.shapira/.rvm/rubies/ruby-2.1.3/lib/ruby/site_ruby/2.1.0/rubygems/specification.rb:2112:in `raise_if_conflicts': Unable to activate puppet-ghostbuster-0.7.3, because puppet-lint-2.0.2 conflicts with puppet-lint (~> 1.0) (Gem::ConflictError)
from /Users/amos.shapira/.rvm/rubies/ruby-2.1.3/lib/ruby/site_ruby/2.1.0/rubygems/specification.rb:1280:in `activate'
from /Users/amos.shapira/.rvm/rubies/ruby-2.1.3/lib/ruby/site_ruby/2.1.0/rubygems.rb:198:in `rescue in try_activate'
from /Users/amos.shapira/.rvm/rubies/ruby-2.1.3/lib/ruby/site_ruby/2.1.0/rubygems.rb:195:in `try_activate'
from /Users/amos.shapira/.rvm/rubies/ruby-2.1.3/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:126:in `rescue in require'
from /Users/amos.shapira/.rvm/rubies/ruby-2.1.3/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:39:in `require'
from /Users/amos.shapira/.rvm/gems/ruby-2.1.3/gems/puppet-ghostbuster-0.7.3/lib/puppet-lint/plugins/check_ghostbuster_classes.rb:1:in `<top (required)>'
from /Users/amos.shapira/.rvm/gems/ruby-2.1.3/gems/puppet-lint-2.0.2/lib/puppet-lint/plugins.rb:18:in `load'
from /Users/amos.shapira/.rvm/gems/ruby-2.1.3/gems/puppet-lint-2.0.2/lib/puppet-lint/plugins.rb:18:in `block (2 levels) in load_from_gems'
from /Users/amos.shapira/.rvm/gems/ruby-2.1.3/gems/puppet-lint-2.0.2/lib/puppet-lint/plugins.rb:17:in `each'
from /Users/amos.shapira/.rvm/gems/ruby-2.1.3/gems/puppet-lint-2.0.2/lib/puppet-lint/plugins.rb:17:in `block in load_from_gems'
from /Users/amos.shapira/.rvm/gems/ruby-2.1.3/gems/puppet-lint-2.0.2/lib/puppet-lint/plugins.rb:16:in `each'
from /Users/amos.shapira/.rvm/gems/ruby-2.1.3/gems/puppet-lint-2.0.2/lib/puppet-lint/plugins.rb:16:in `load_from_gems'
from /Users/amos.shapira/.rvm/gems/ruby-2.1.3/gems/puppet-lint-2.0.2/lib/puppet-lint/plugins.rb:74:in `<top (required)>'
from /Users/amos.shapira/.rvm/rubies/ruby-2.1.3/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:54:in `require'
from /Users/amos.shapira/.rvm/rubies/ruby-2.1.3/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:54:in `require'
from /Users/amos.shapira/.rvm/gems/ruby-2.1.3/gems/puppet-lint-2.0.2/lib/puppet-lint.rb:214:in `<top (required)>'
from /Users/amos.shapira/.rvm/rubies/ruby-2.1.3/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:54:in `require'
from /Users/amos.shapira/.rvm/rubies/ruby-2.1.3/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:54:in `require'
from /Users/amos.shapira/.rvm/gems/ruby-2.1.3/gems/puppet-lint-2.0.2/bin/puppet-lint:5:in `<top (required)>'
from /Users/amos.shapira/.rvm/gems/ruby-2.1.3/bin/puppet-lint:23:in `load'
from /Users/amos.shapira/.rvm/gems/ruby-2.1.3/bin/puppet-lint:23:in `<main>'
from /Users/amos.shapira/.rvm/gems/ruby-2.1.3/bin/ruby_executable_hooks:15:in `eval'
from /Users/amos.shapira/.rvm/gems/ruby-2.1.3/bin/ruby_executable_hooks:15:in `<main>'
any chance of making it work with puppet-lint 2?
We currently make a new connection for each db query ๐
I'm quite sure we could speed up things by reusing existent connection.
We should see how to do this with the puppetdb class.
Hi Camp2Camp,
Would there be any interest in transferring this repository to Vox Pupuli?
With @raphink moving on, the community might worry that the project will descend further into obsoleteness. Vox Pupuli has expressed interest, and requested that I lodge this issue to determine if you would be willing to migrate?
Bundler could not find compatible versions for gem "puppet-lint":
In Gemfile:
puppet-lint (= 3.0.1)
puppet-ghostbuster (~> 0.9) was resolved to 0.9.0, which depends on
puppet-lint (< 3.0, >= 1.0)
With PuppetDB 3 and later the v2/v3 API endpoints have been retired. The puppetdb-ruby client this depends on does not support the v4 API without some minor patching but the project seems dead.
Essentially the patch I did locally was:
PuppetDB::Client.new({
:server => configuration.puppetdbserverurl,
:pem => {
'key' => configuration.hostprivkey,
'cert' => configuration.hostcert,
'ca_file' => configuration.localcacert,
}
}, 4)
(notice the , 4
thing before the closing )
)
And then calling it with -s https://my-puppetdb-host:8081/pdb/query
appears to do the trick.
Great to see this module :)
I'd love a flag to use a non-ssl integration with PuppetDB. I'm not sure I'll have any time soon to send a PR for this, but would love to see it!
Seems we have accidentally broken something by trying to support hiera datadir. See #86
@bastelfreak @alexjfisher @h0tw1r3
Tagging you here to keep this all in one place and keep track.
It would be cool to have a single resources.json
and have a lib that mocks the calls to the PuppetDB and returns only the subset of resources matching the query.
So that we can open an ssh tunnel to the host that hosts the puppetdb
Hello
I came across the following error when using ghostbuster_facts:
PUPPETDB_URL='http://localhost:8080/' HIERA_YAML_PATH='/tmp/hiera.yaml' find . -type f -
exec puppet-lint --only-checks ghostbuster_facts {} \+
Whoops! It looks like puppet-lint has encountered an error that it doesn't
know how to handle. Please open an issue at https://github.com/rodjek/puppet-lint
and paste the following output into the issue description.
---
puppet-lint version: 2.3.3
ruby version: 2.1.9-p490
platform: x86_64-darwin16.0
file path: ./modules/apt/lib/facter/apt_updates.rb
file contents:
apt_package_updates = nil
Facter.add("apt_has_updates") do
confine :osfamily => 'Debian'
if File.executable?("/usr/bin/apt-get")
apt_get_result = Facter::Util::Resolution.exec('/usr/bin/apt-get -s upgrade 2>&1')
if not apt_get_result.nil?
apt_package_updates = [[], []]
apt_get_result.each_line do |line|
if line =~ /^Inst\s/
package = line.gsub(/^Inst\s([^\s]+)\s.*/, '\1').strip
apt_package_updates[0].push(package)
security_matches = [
/ Debian[^\s]+-updates /,
/ Debian-Security:/,
/ Ubuntu[^\s]+-security /,
/ gNewSense[^\s]+-security /
]
re = Regexp.union(security_matches)
if line.match(re)
apt_package_updates[1].push(package)
end
end
end
end
end
setcode do
if not apt_package_updates.nil? and apt_package_updates.length == 2
apt_package_updates != [[], []]
end
end
end
Facter.add("apt_package_updates") do
confine :apt_has_updates => true
setcode do
if Facter.version < '2.0.0'
apt_package_updates[0].join(',')
else
apt_package_updates[0]
end
end
end
Facter.add("apt_updates") do
confine :apt_has_updates => true
setcode do
Integer(apt_package_updates[0].length)
end
end
Facter.add("apt_security_updates") do
confine :apt_has_updates => true
setcode do
Integer(apt_package_updates[1].length)
end
end
error:
ArgumentError: invalid byte sequence in UTF-8
/Users/schowdhury/.rvm/gems/ruby-2.1.9/gems/puppet-ghostbuster-0.9.0/lib/puppet-lint/plugins/check_ghostbuster_facts.rb:40:in `==='
/Users/schowdhury/.rvm/gems/ruby-2.1.9/gems/puppet-ghostbuster-0.9.0/lib/puppet-lint/plugins/check_ghostbuster_facts.rb:40:in `each'
/Users/schowdhury/.rvm/gems/ruby-2.1.9/gems/puppet-ghostbuster-0.9.0/lib/puppet-lint/plugins/check_ghostbuster_facts.rb:40:in `grep'
/Users/schowdhury/.rvm/gems/ruby-2.1.9/gems/puppet-ghostbuster-0.9.0/lib/puppet-lint/plugins/check_ghostbuster_facts.rb:40:in `block (2 levels) in check'
/Users/schowdhury/.rvm/gems/ruby-2.1.9/gems/puppet-ghostbuster-0.9.0/lib/puppet-lint/plugins/check_ghostbuster_facts.rb:39:in `each'
/Users/schowdhury/.rvm/gems/ruby-2.1.9/gems/puppet-ghostbuster-0.9.0/lib/puppet-lint/plugins/check_ghostbuster_facts.rb:39:in `block in check'
/Users/schowdhury/.rvm/gems/ruby-2.1.9/gems/puppet-ghostbuster-0.9.0/lib/puppet-lint/plugins/check_ghostbuster_facts.rb:28:in `each'
/Users/schowdhury/.rvm/gems/ruby-2.1.9/gems/puppet-ghostbuster-0.9.0/lib/puppet-lint/plugins/check_ghostbuster_facts.rb:28:in `check'
/Users/schowdhury/.rvm/gems/ruby-2.1.9/gems/puppet-lint-2.3.3/lib/puppet-lint/checkplugin.rb:21:in `run'
/Users/schowdhury/.rvm/gems/ruby-2.1.9/gems/puppet-lint-2.3.3/lib/puppet-lint/checks.rb:60:in `block in run'
/Users/schowdhury/.rvm/gems/ruby-2.1.9/gems/puppet-lint-2.3.3/lib/puppet-lint/checks.rb:57:in `each'
/Users/schowdhury/.rvm/gems/ruby-2.1.9/gems/puppet-lint-2.3.3/lib/puppet-lint/checks.rb:57:in `run'
/Users/schowdhury/.rvm/gems/ruby-2.1.9/gems/puppet-lint-2.3.3/lib/puppet-lint.rb:196:in `run'
/Users/schowdhury/.rvm/gems/ruby-2.1.9/gems/puppet-lint-2.3.3/lib/puppet-lint/bin.rb:59:in `block in run'
/Users/schowdhury/.rvm/gems/ruby-2.1.9/gems/puppet-lint-2.3.3/lib/puppet-lint/bin.rb:56:in `each'
/Users/schowdhury/.rvm/gems/ruby-2.1.9/gems/puppet-lint-2.3.3/lib/puppet-lint/bin.rb:56:in `run'
/Users/schowdhury/.rvm/gems/ruby-2.1.9/gems/puppet-lint-2.3.3/bin/puppet-lint:7:in `<top (required)>'
/Users/schowdhury/.rvm/gems/ruby-2.1.9/bin/puppet-lint:23:in `load'
/Users/schowdhury/.rvm/gems/ruby-2.1.9/bin/puppet-lint:23:in `<main>'
/Users/schowdhury/.rvm/gems/ruby-2.1.9/bin/ruby_executable_hooks:15:in `eval'
/Users/schowdhury/.rvm/gems/ruby-2.1.9/bin/ruby_executable_hooks:15:in `<main>'
Here are the gem's I currently have loaded:
gem list
*** LOCAL GEMS ***
bigdecimal (default: 1.2.4)
bundler-unload (1.0.2)
CFPropertyList (2.3.5)
executable-hooks (1.3.2)
facter (2.5.1 universal-darwin)
fast_gettext (1.1.0)
gem-wrappers (1.3.2)
gettext (3.2.4)
gettext-setup (0.28)
hiera (3.4.2)
httparty (0.15.6)
io-console (default: 0.4.3)
json (default: 1.8.1)
locale (2.1.2)
minitest (default: 4.7.5)
multi_xml (0.6.0)
psych (default: 2.0.5)
puppet (5.3.2 universal-darwin)
puppet-ghostbuster (0.9.0)
puppet-lint (2.3.3)
puppetdb-ruby (1.1.1)
rake (default: 10.1.0)
rdoc (default: 4.1.0)
rubygems-bundler (1.4.4)
rvm (1.11.3.9)
test-unit (default: 2.1.9.0)
text (1.3.1)
Has anyone else came across this issue as well?
We're still on PuppetDB 2.x so it would be useful to add an option to switch between the versions of PuppetDB.
Due to problems during the migration from the camptocamp GitHub org to Vox Pupuli we had some issues with the v1.0.0 Release. The tag exists but the release isn't available on rubygems.org.
<%= scope.function_template(['foo/bar/baz.erb']) -%>
If the _module path ends_ with a slash you get an error.
โ 12:51 ~/workspace/puppet-ghostbuster [ master | โ 1 โฆ3 ] $ puppet-ghostbuster --key ssl/private.pem --cert ssl/public.pem --ca ssl/ca.pem -s http://localhost:8080 ~/workspace/puppet/modules/module_name_with_ending_slash/
I, [2016-01-04T12:51:29.207418 #17394] INFO -- PuppetGhostbuster: Now trying to find unused classes
I, [2016-01-04T12:51:29.224780 #17394] INFO -- PuppetGhostbuster: Now trying to find unused defines
I, [2016-01-04T12:51:29.225225 #17394] INFO -- PuppetGhostbuster: Now trying to find unused templates
/home/tuxinaut/workspace/puppet/modules/module_name_with_ending_slash//templates/env.erb
/home/tuxinaut/workspace/puppet-ghostbuster/lib/puppet-ghostbuster.rb:127:in `block in find_unused_templates': undefined method `captures' for nil:NilClass (NoMethodError)
from /home/tuxinaut/workspace/puppet-ghostbuster/lib/puppet-ghostbuster.rb:124:in `each'
from /home/tuxinaut/workspace/puppet-ghostbuster/lib/puppet-ghostbuster.rb:124:in `find_unused_templates'
from /home/tuxinaut/workspace/puppet-ghostbuster/lib/puppet-ghostbuster.rb:180:in `run'
from /home/tuxinaut/workspace/puppet-ghostbuster/lib/puppet-ghostbuster/bin.rb:26:in `run'
from /home/tuxinaut/workspace/puppet-ghostbuster/bin/puppet-ghostbuster:7:in `<top (required)>'
from /home/tuxinaut/.rvm/gems/ruby-2.2-head@puppet_ghostbuster/bin/puppet-ghostbuster:23:in `load'
from /home/tuxinaut/.rvm/gems/ruby-2.2-head@puppet_ghostbuster/bin/puppet-ghostbuster:23:in `<main>'
from /home/tuxinaut/.rvm/gems/ruby-2.2-head@puppet_ghostbuster/bin/ruby_executable_hooks:15:in `eval'
from /home/tuxinaut/.rvm/gems/ruby-2.2-head@puppet_ghostbuster/bin/ruby_executable_hooks:15:in `<main>'
โ 12:51 ~/workspace/puppet-ghostbuster [ master | โ 1 โฆ3 ] $
Would be great to find the ensure => absent resources in order to be able to clean a bit the catalogs.
The puppetdb-ruby gem provides a useful client but seems unable to encode more complex queries correctly causing APIError's to be thrown. Especially the classes query could benefit from a more efficient query on larger installations.
I'm having some trouble with Ruby right now but this should more or less be it:
require 'json'
require 'net/http'
require 'ghostbusters/version'
class PuppetGhostbuster
class PuppetDB
def self.client
uri = URI.parse("#{ENV['PUPPETDB_URL'] || 'http://puppetdb:8080'}")
@@client ||= Net::HTTP.new(uri.host, uri.port)
@@client.key = ENV['PUPPETDB_KEY_FILE']
@@client.cert = ENV['PUPPETDB_CERT_FILE']
@@client.ca_file = ENV['PUPPETDB_CACERT_FILE']
end
[..]
def self.request(endpoint, query = nil)
uri_prefix = '/pdb/query/v4'
headers = {'user-agent' => "ghostbusters/#{PuppetGhostbuster::VERSION}", 'accept' => 'application/json' }
if query.nil?
request = Net::HTTP::Get.new("#{uri_prefix}/#{endpoint}", headers)
else
request = Net::HTTP::Get.new("#{uri_prefix}/#{endpoint}?query=#{URI.encode(query)}", headers)
end
resp = client.request(request)
case resp
when Net::HTTPSuccess
JSON.parse resp.body
else
resp.value # calling value will raise an exception if we didn't get a 200
end
end
[..]
def self.classes
query = '["extract", [["function", "count"], "title"], ["=", "type", "Class"], ["group_by", "title"]]'
@@classes ||= client.request('resources', query).select { |r| r.title }
end
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.