Three Questions about salt-states HOT 12 CLOSED

Devs / @digitalsleuth (sorry bud to always tag you)
Also just caught that docker-compose is at 1.29.2 from PyPi from where it is sourced for this project. But I guess Docker has been bad and has not been updating their PyPi since 1.29.2 was their update from May 2021. But if you go to their git https://github.com/docker/compose it was last updated a few days ago (Nov 23rd). You might need to consider switching sourcing on dependency from the PyPi to their Git in the future to ensure you are getting up to date versions of it.
Cheers!

from salt-states.

Hey, sorry I'm just now seeing this, not sure how I missed it :) I'll do my best to answer your questions in order:

1. Regarding the installation as root, the primary motive around this was to make sure the applications were available to all users on the system, and not just the current one. They don't all need to be run as root, but some of them may if they have to use system devices. Installing in the users local means they won't necessarily be available for any other users created on the system.

2. Some of the packages that are installed via git instead of Pypi (pip) are that way either because the Pypi version at the time wasn't up-to-date, or there is/was no Pypi package. I'll take a review of those we have installed by git and see if they can be updated to be pip installed.

3. I have been working on moving some of these tools to virtualenv's, but am also trying to make sure they'll work on Jammy for support. Some of them don't play well with virtualenv's, but they're coming along.

4. (Docker Compose) This is definitely something that can be looked at, but there are some breaking changes with the move to the newer version which I'll have to test. Ideally, it should be pretty easy.

Since there are a few things on the docket here, it might take a week or so to get to them, but I'll be looking at them this weekend (as well as peepdf).

Cheers!

from salt-states.

@digitalsleuth
All good. I wasn't worried so much about getting answers for the three questions as much making sure that the Docker Compose thing was noticed. But thank you for answering them.
Hopefully you can grab a newer version (whatever is most stable) for the project to use and not something 2 years out of date (thanks for that Docker!).
I may have spoken too soon on the Docker thing. https://github.com/docker/compose/issues/9571. Python was a part of their legacy v1 of their project which is no longer supported. V2 is what is on their Git which is now written in Go. Not sure if this changes things.

As for the other things:
Running as root for all to be able to access the tools makes sense. I may play around with that on my own just to mess with local installing to see what I can find out.
The Pypi vs git was just some things I noticed. I wasn't sure if installing from Git or Pypi was easier so figure I would give some updates for some of the gits just in case PyPi was easier.
I have been trying to play around with virtualenvs too today but with little success. Still trying to wrap my head around Salt and all. And I keep finding more things to throw the devs way lol. I would love to see Salt support Pipx one day so that they could all get their own virtualenv for each tool and their dependencies. Hopefully one day soon that will come.

Cheers!

from salt-states.

Not sure if the dev's have someone familiar with both Salt and Go but the integration does not seem easy from what I'm finding so far. I think I found a package that can tie them together to be able to run Go commands in Salt to build the package. It requires Ruby and Docker to run it but it looks like those are already install requirements for the current Remnux package so nothing super extra there. https://github.com/saltstack-formulas/golang-formula/.
I am trying to install and mess with it now to see if I can make it work. If I figure it out I'll upload my salt.
The old Docker-Compose Python route may be more feasible for now till a full solution can be found to make it work or decision on whether or not to just stick with the v1 of the program.

from salt-states.

Ok so it has been a pain so far but here is what I got for this golang package I linked above:

• Git cloned the package onto my system and ran the commands as it says in the README to install it, however ran into a error on install that required troubleshooting to see if I could fix it.
  -- Troubleshooting the issue I ran into trying to do this install process, it turns out that this salt-go package or Kitchen or something uses Pigz which uses Zlib to uncompress some required files. There was a big bug in like v2.6 of Pigz that causes Pigz not to recognize the Zlib version, even if you have the most current version of Zlib.
  -- Another issue is that Pigz does not update its repo on Ubuntu for older OS versions to get its more current versions. So 20.04 can only pull 2.4, 22.04 can only get 2.6, and the most current version being 2.8.
  -- The Zlib version issue in Pigz has been patched as of August, but the newer version with this fix can't be pulled via repo.
  -- The fix I found is to either pull Pigz source from their git https://github.com/madler/pigz or their site http://zlib.net/pigz/ and compile it (a simple make command) to make the most current version. Then move the newly compiled programs (pigz & unpigz) into the /usr/bin to override the current version installed (if it is already installed). This will get the system to recognize that 2.8 is installed.
• Once the newly compiled files are dropped in, it should work no problem to run the kitchen test fully
• After this I am stuck since I have no experience with Salt and building pillars so trying to figure out how to integrate the commands from this new integration to allow for go states is above my head.

It seems like it may be a complex path to try to build a Salt to make this to work with the Remnux package just to get a newer version of Docker Compose to work with it. Again not sure if just sticking with the old v1 in Python may be the best bet just to lessen complications. But I'm stuck at this point and will leave it to someone else to see if they can find a simpler way or if they continue forward, a way to Salt all of this.

from salt-states.

The golang build shouldn't be an issue at all, it's more along the lines of how compose is used and the support for various versions of the compose files. There is a compose binary which can simply be installed and bypass these issues.

from salt-states.

Ah, glad you have a simpler solution than what I tried to figure out. I'm sure you got this.
Cheers!

from salt-states.

Just an update (haven't forgotten about this):

• I've created virtualenvs for qiling, speakeasy, and thug and have finished most of the testing. The last bit of testing I'll need to do is ensure the old versions of these tools are completely removed before installing the new virtualenv versions. This will likely be done tomorrow sometime.
• I've adapted docker-compose to the latest version, but it doesn't use the new V2 plugin, it still uses the old command. However, it does still function as the V2 plugin should.
• I haven't finished reviewing all of the git install methods via pip, but I've flagged a few which will need to remain. More will follow in the next day or two.

from salt-states.

@digitalsleuth Hey buddy, It's all good. I know we all got life and other things going on and you would get to these issues when you could. But I am glad to see you managed to find some fixes through everything else you may currently have going on. I'm on vacation for a while longer but I'll do a pull for updates to Remnux once I get back to see if there is any issues that I can find. Thank you for the update and Happy New Years!
Cheers!

from salt-states.

@geeksailor Enjoy the rest of your vacation, and thanks and Happy New Years to you too!

from salt-states.

@geeksailor With recent releases, you should find that the questions you asked above should now all be answered. Please let us know if you have any other concerns, and if this issue can be closed.

Cheers!

from salt-states.

@digitalsleuth Sounds good! I'll close this thread out. I am back at work this week so I'm pushing updates to my system today and then I'll see if find any new issues after that (which I hopefully I shouldn't). If I do find anything though I'll start up a new thread.
Thank you again for all your work to fix these issues.
Cheers!

from salt-states.