Giter VIP home page Giter VIP logo

cracklord's Introduction

Cracklord

GoDoc Build Status Rawsec's CyberSecurity Inventory

What Is It?

CrackLord is a system designed to provide a scalable, pluggable, and distributed system for both password cracking as well as any other jobs needing lots of computing resources. Better said, CrackLord is a way to load balance the resources, such as CPU, GPU, Network, etc. from multiple hardware systems into a single queueing service across two primary services: the Resource and Queue. It won't make these tasks faster, but it will make it easier to manage them.

System Components

There are three primary components to CrackLord as shown in the above image:

  • Queue - The Queue is a service that runs on a single system, providing an interface for users to submit, pause, resume, and delete jobs. These jobs are then processed and sent to available Resources to perform the actual work and handle the results.
  • Resource / Resource Managers - Resources are the individual servers that are connected into the queue. They are managed by a resource manager plugins. These are code that allow various types of resources to be connected. Managers can directly connect to physical resources you own, or use cloud services to spawn resources as necessary.
  • Tools - Tools are a set of plugins, configured on resources, that perform the underlying tasks such as running oclHashcat to crack passwords. Tools are written in the Go programming language and have a standard interface to make them easy to write or enhance. They are wrappers of the various tools used that require great deals of resources, such as John, HashCat, etc.

Server Installation

We have a set of packages built for every release we make, if you'd like to just use that you can do it by simply following the instructions here.

If you'd like to get things build from source, it will first require you to have a working Go build environment with the GOPATH setup. Additionally, you'll probably want Git and Mercurial setup to gather the various libraries and plugins that we've used in the code.

  1. First, you'll need to get cracklord itself.
    go get github.com/jmmcatee/cracklord

  2. Next we need to get all of the dependencies downloaded for both the resource daemon and queue daemon.
    go get github.com/jmmcatee/cracklord/cmd/queued
    go get github.com/jmmcatee/cracklord/cmd/resourced

  3. Now we can actually build the queue daemon and resource daemon
    go build github.com/jmmcatee/cracklord/cmd/queued
    go build github.com/jmmcatee/cracklord/cmd/resourced

  4. Finally, we can run both the resource and queue daemons, which will both be in the cmd/queued and cmd/resourced directories. You will also need to setup the various configuration files, information for those can be found in our wiki.

Contributing

Addons

Probably the easiest way to get involved is to write a new tool plugin. If you have tools that you use as part of testing, research, or work and would like to get them integrated, you can very easily write a new tool and send us a pull request. We'll make sure to get it integrated in as soon as possible. In the plugins directory we have created an empty tool to provide some guidance and help. If you also have a neat way to interact with resources, you would also write a resource manager plugin, maybe for a cloud service that we don't support yet or some new way to do the work.

Because of the way the Go language works, we have to compile all of the tools in, so if you do something you'd like to share please send us a pull request and we'll test it and get it out for everyone to use.

Scripts / GUI

We have a standard API that the queue daemon publishes out for access. We went ahead and wrote a standard web GUI which also uses the same API. That doesn't mean you couldn't make a better one! We're also looking at writing a few scripts to automate common jobs in our workflow, if you end up making them send us links or a pull request and we'll make sure to find a home / give you a shout out!

Documentation

We're working hard to try and keep the documentation up to date with everything we're doing, but there's always room for a how-to, tutorial, or example and we'd love any help you can provide on those. Head on over to our wiki and see what needs fixing or adding!

Bugs / Issues

Of course, there's nothing saying you can't work on the CrackLord queue and resource daemons themselves. We have our issues list and any help getting those fixed would be greatly appreciated.

cracklord's People

Contributors

emperorcow avatar gifts avatar jmmcatee avatar mychickenninja avatar noraj avatar ramzeth avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cracklord's Issues

Code queue reorder confirmation

Right now we need to send the entire queue in order. You should add some logic to move all done/failed/quit jobs to the bottom.

Tests are broken and not used

Right now the tests are broken and we aren't really using them, we should revisit these to get them back in and be good developers...

Use TLS for RPC resource calls

The Queue currently calls resource over a standard TCP connection. This needs to be over TLS and ignore certificate issues for the time being.

Need to streamline install process

Need to change main page readme to document

  • Differences between server and resourceserver
  • go get should be for the individual item, otherwise it doesn't grab deps
  • Need to copy public and the INI files over is there a way to do this during go install?
  • Need to add instructions on gopath and goroot

Generated form select sorting

It is likely in Go that most of the select fields used for generated forms will be stored internally as Go maps. Maps in Go return their internals randomly, so could we have all selects just sort by name so they are presented to the user in a constant order?

Queue attempts to send jobs to disconnected resources

When a resource is disconnected, the queue attempts to send a job to it and then doesn't try the next successful resource. Here is a log from the keeper:

2015/03/30 10:29:23 ----Queue Keeper is running....
2015/03/30 10:29:23 ----Getting Queue Lock
2015/03/30 10:29:23 ----Updating the the Queue
2015/03/30 10:29:23 ==Res(5b1baa61-9f1c-4f96-8657-0add7b19c7dc): gpu:%!b(bool=true)
2015/03/30 10:29:23 Starting a new job on:5b1baa61-9f1c-4f96-8657-0add7b19c7dc
2015/03/30 10:29:23 connection is shut down
2015/03/30 10:29:23 ==Res(8b732cd3-76ef-4130-abca-e59d66b45fc7): gpu:%!b(bool=true)
2015/03/30 10:29:23 ----Releasing the lock

Add nonce to all API requests

We may or may not want to do this, but I think the API structs should be updated to utilize one for all requests. How hard would this be on the backend?

Add more details to Job JSON return

Right now we don't have a lot of details, I'm thinking it could be nice to return some things like total guesses, track a history of guesses (yay graphs!) every job check, etc.

Returned parameter div improvements

The div that displays returned parameters should list the first few (4 or 5) then have a more information link. This should pop-up a menu that shows all parameters returned by the job (hopefully without sensitive data) and also provide all the OutputTitle and Data fields as well.

Null pointer exception

panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x639eaf]

goroutine 25 [running]:
net/rpc.(_Client).send(0x0, 0xc2081915e0)
/usr/local/go/src/net/rpc/client.go:72 +0x3af
net/rpc.(_Client).Go(0x0, 0x8c7e30, 0x10, 0x809260, 0xc208196200, 0x72f8a0, 0xc2081ca870, 0xc208084d20, 0xc208196200)
/usr/local/go/src/net/rpc/client.go:309 +0x13e
net/rpc.(_Client).Call(0x0, 0x8c7e30, 0x10, 0x809260, 0xc208196200, 0x72f8a0, 0xc2081ca870, 0x0, 0x0)
/usr/local/go/src/net/rpc/client.go:315 +0xa5
github.com/jmmcatee/cracklord/queue.(_Queue).updateQueue(0xc2080801a0)
/usr/local/cracklord/src/github.com/jmmcatee/cracklord/queue/queue.go:684 +0x385
github.com/jmmcatee/cracklord/queue.func·001()
/usr/local/cracklord/src/github.com/jmmcatee/cracklord/queue/queue.go:573 +0x1e7
created by github.com/jmmcatee/cracklord/queue.(*Queue).keeper
/usr/local/cracklord/src/github.com/jmmcatee/cracklord/queue/queue.go:669 +0x8c

goroutine 1 [IO wait]:
net.(_pollDesc).Wait(0xc2080d7f70, 0x72, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(_pollDesc).WaitRead(0xc2080d7f70, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(_netFD).accept(0xc2080d7f10, 0x0, 0x7f18c0697d48, 0xc2082089e0)
/usr/local/go/src/net/fd_unix.go:419 +0x40b
net.(_TCPListener).AcceptTCP(0xc20802c050, 0xc2082089c0, 0x0, 0x0)
/usr/local/go/src/net/tcpsock_posix.go:234 +0x4e
net/http.tcpKeepAliveListener.Accept(0xc20802c050, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/http/server.go:1976 +0x4c
crypto/tls.(_listener).Accept(0xc208117ee0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/crypto/tls/tls.go:46 +0x6b
net/http.(_Server).Serve(0xc2080849c0, 0x7f18c069b918, 0xc208117ee0, 0x0, 0x0)
/usr/local/go/src/net/http/server.go:1728 +0x92
net/http.(*Server).ListenAndServeTLS(0xc2080849c0, 0x8aad10, 0x8, 0x8aacf0, 0x8, 0x0, 0x0)
/usr/local/go/src/net/http/server.go:1879 +0x55b
net/http.ListenAndServeTLS(0xc20800b980, 0xb, 0x8aad10, 0x8, 0x8aacf0, 0x8, 0x7f18c0699a78, 0xc2080564e0, 0x0, 0x0)
/usr/local/go/src/net/http/server.go:1841 +0xef
main.main()
/usr/local/cracklord/src/github.com/jmmcatee/cracklord/queueserver/queueserver.go:215 +0x21e4

goroutine 33 [syscall, 887 minutes, locked to thread]:
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:2232 +0x1

goroutine 107 [IO wait]:
net.(_pollDesc).Wait(0xc2081ec300, 0x72, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(_pollDesc).WaitRead(0xc2081ec300, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(_netFD).Read(0xc2081ec2a0, 0xc2080de800, 0x800, 0x800, 0x0, 0x7f18c0697d48, 0xc2082095a0)
/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(_conn).Read(0xc20802c068, 0xc2080de800, 0x800, 0x800, 0x0, 0x0, 0x0)
/usr/local/go/src/net/net.go:121 +0xdc
crypto/tls.(_block).readFromUntil(0xc2080e90b0, 0x7f18c069bad8, 0xc20802c068, 0x5, 0x0, 0x0)
/usr/local/go/src/crypto/tls/conn.go:454 +0xe6
crypto/tls.(_Conn).readRecord(0xc20807c580, 0x17, 0x0, 0x0)
/usr/local/go/src/crypto/tls/conn.go:539 +0x2da
crypto/tls.(_Conn).Read(0xc20807c580, 0xc20816d000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/local/go/src/crypto/tls/conn.go:904 +0x166
net/http.(_liveSwitchReader).Read(0xc2080dc0e8, 0xc20816d000, 0x1000, 0x1000, 0x8, 0x0, 0x0)
/usr/local/go/src/net/http/server.go:214 +0xab
io.(_LimitedReader).Read(0xc2080fa6c0, 0xc20816d000, 0x1000, 0x1000, 0x800, 0x0, 0x0)
/usr/local/go/src/io/io.go:408 +0xce
bufio.(_Reader).fill(0xc2081ba180)
/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(_Reader).ReadSlice(0xc2081ba180, 0xc2080e910a, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/bufio/bufio.go:295 +0x257
bufio.(_Reader).ReadLine(0xc2081ba180, 0x0, 0x0, 0x0, 0xc2080e9100, 0x0, 0x0)
/usr/local/go/src/bufio/bufio.go:324 +0x62
net/textproto.(_Reader).readLineSlice(0xc208201b60, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/textproto/reader.go:55 +0x9e
net/textproto.(_Reader).ReadLine(0xc208201b60, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/textproto/reader.go:36 +0x4f
net/http.ReadRequest(0xc2081ba180, 0xc2081c0410, 0x0, 0x0)
/usr/local/go/src/net/http/request.go:598 +0xcb
net/http.(_conn).readRequest(0xc2080dc0a0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/http/server.go:586 +0x26f
net/http.(_conn).serve(0xc2080dc0a0)
/usr/local/go/src/net/http/server.go:1162 +0x69e
created by net/http.(*Server).Serve
/usr/local/go/src/net/http/server.go:1751 +0x35e

Wrap panics in Resource calls

We should wrap all calls to the Tasker interface to avoid panics that bring down the Resource because of a bug in the plugin.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.