openwall / johnny Goto Github PK

Today, I worked on preparing myself to make a Windows release with the installer that we discussed on the list. However, I found out a problem with current jumbo related to pausing an attack(was Windows event loop thread on the list).
Shinnok's fix works on core john and jumbo(for making it exit). However, we recently started to verify the exit status of the Qt process in Johnny to show error messages to users (in the TODO task) .

Core john returns QProcess::NormalExit and jumbo returns QProcess::CrashExit. This has the effect to show pop-up John crash, verify console log for details everytime we pause an attack. Also, the buttons are blocked(because of the return; -- which I fixed in one of my PR). Aleksey told me that if we don't see the "session ended" message in the console log, it's probably a real crash exit and not a false positive.

I sent a detailed message to john-dev to ask question to jumbo developers http://www.openwall.com/lists/john-dev/2015/06/09/12 .

This is around 30% of a cpu core constantly. It looks like a bug with Qt:
https://bugreports.qt.io/browse/QTBUG-35312
https://bugreports.qt.io/browse/QTBUG-20880

It might have been fixed in Qt 5.5 https://codereview.qt-project.org/#/c/112379/ .

Thus don't use the define anymore. In JohnProcess the Q_OS_UNIX check should be moved to the cpp in place.

I can do the initial work on Yosemite and then Mathieu can test the results on the older OS X versions.

In all settings, we use combo box in place where there should be line edit. In fact, there is the drop down icon which tell the user that there might be predefined values(like the format comboBox) but in fact there aren't for almost all our options.

Document the Windows installer procedure. Start with the Qt Installer Framework, what did you find out, what does it produce, is it easy to rebuild?

Here is a problem that we have at the moment. Imagine the release folder is as follow.
./johnny.exe (or any other executable format)
./XXX.dll
./yy.dll
./JohnTheRipper/run/john.exe

We already search for john.exe in the same directory because when you have an installer, it should be easy to make johnny works without configuration. However, since the m_pathToJohn setting is system-wide, if another john older version was specified in the past by the user, or for example comes from another release or folder of johnny, we'll remember an existing old path and we won't do our auto-detect. So we bundle a new john with johnny, but it's not gonna get used.

I'm not sure if I was clear enough to make you understand what I mean.

We need to start disseminating the upcoming Release. The first step towards coming up with a plan is to ask for feedback from john-dev and john-users lists regarding which platforms should we support for:

1. Source build instructions and testing;
2. Subsequent binary packaging;

For no.2, the installers, let's focus on platforms people are using Johnny on already or express a written desire or need to do so but can't for lack of an installer.

Something happened in johnny, I don't know under which circumstances, I was doing normal things that I do most of the time. It didn't seem to affect johnny as it seemed to work correctly but this print in the debugging terminal.
EDIT : The message is really long and the formatting was bad when I put it in github, go see http://pastebin.com/w65uFFJ1

Tasks to do in regards to this release:

1. Resolve #37 #35 #34 #22 #17 #16
2. Update INSTALL distfile with source building instructions for all supported platforms. See #17
3. Prepare OS X and Windows redistributables and test them
4. Freeze changes and tag version 2.0 in git
5. Prepare final redistributables
6. Add CHANGELOG distfile
7. Revise Johnny's main Wiki page on openwall.info
8. Write announcement e-mail (should be sent to both john-users and john-dev); include details on what's new, what's changed, what's ahead and where to get packages, source and more info
9. After review of the announcement e-mail, proceed to public unvelining

NB: Some tasks are to be handled by me, pay attention to the Assignee tag for issues or ask me directly

We need a validator that checks if the path that is being set is a valid existing file, readable and executable. Further more we could check if it is john indeed by greping the output of ./john with no args and also checking to see if it's jumbo or core right there and then reconsider the UI.

Let's use github issues to keep track of things that are easy to loose on the list.
This TODO might need more than a few lines so it might need a separate task if it is judged important. @AlekseyCherepanov Did you qualify that as something important ?
From MainWindow::on_actionStart_Attack_triggered()
// TODO: Saving so two instances of johnny overwrite description
// but not .rec so they become not synchronized.
QFile description(m_session + ".johnny");
if (!description.open(QIODevice::WriteOnly | QIODevice::Text)) {
QMessageBox::critical(
this,
tr("Johnny"),
tr("Johnny could not open file to save session description!"));
return;
}

callJohnShow() often lead to weird message popping out in the terminal "QProcess::start: Process is already running". As the doc says, If the "QProcess object is already running a process, a warning may be printed at the console, and the existing process will continue running unaffected. ".

It's because we call john show process another time beforelast call is finished. This is harmless(and doesn't really affect feature, since we'll still get one john show results). However, it doesn't look really profesionnal to have a lot of those messages in the terminal. Also, now that we call john show every 15 seconds, it's probably gonna happens even more often.

Use case :

1. Start a new attack which will end really fast, for example, choose external mode with no name. It'll end really fast because john says: Option requires a parameter: "--external="
2. Start a new attack again, you'll get the QProcess warning in the terminal since the previous johnShow process hasn't finished.

For now we could make the User column field for the cracked passwords bold.

Currently, when we open a last session, we use --restore and the .rec which doesn't seem to save the things related to environment variables such as OpenMP and other environment variables that we can specify in the field I added to Johnny a few weeks ago requested by Solar. @AlekseyCherepanov Is that correct ?

As part of the multiple session management, I'll save those environment variables to QSettings to restore the UI correctly. When restoring a session, my idea is to put them filled by default(ex: var1=XXX) so the user get the comportment he had last time with this session(ex: the same nb of openMP threads). But the user can still edit with var2=YYY or remove some variables and resume the attack with the changes working(unless other settings like --fork witch are directly written in the .rec and will take effects only if you start a new attack)

Let's check off this list the platforms that have been tested and are confirmed(via either source or redistributable installer):

1. Windows 7 and 8; .msi installer; Based on latest Qt5
2. OS X Mavericks and Yosemite; .dmg package; Based on latest Qt5
3. Linux(Ubuntu, Debian, Mint, Fedora, various distros); source build
4. Compatibility with Qt as low as version 4.6.2 (for old Linuxes and OS X Leopard);

Filling the Password field with NULL or EMPTY PASSWORD and with a gray color to indicate that this field is special should take care of this.

In the attack option tab, I think it'd be a great idea to add the found hashes formats of the files in the formatList(auto-detect, des etc..). What do you think @shinnok, is it a great idea ?

Investigate using the macdeployqt tool.
http://doc.qt.io/qt-5/osx-deployment.html

On OS X these include:

• on quit via CMD+q (possibly alt+F4 on Windows too)
• on quit via the OS X dock

This does not happen when closing via the X button on the MainWindow.

Should include Copy text action for now.

Tasks to do in regards to this release:

• Freeze changes and tag version 2.1 release
• Test source build on all supported platforms
• Prepare updated OS X and Windows redistributables and test them
• Add 2.1 entry to CHANGELOG and update INSTALL
• Update Johnny's main Wiki page on openwall.info
• Write announcement e-mails. One for john-users done by @Math07. I'll forward that to john-dev as I did previously.

current/master with crappy old qt (centos 6 - qt 4.6.2)
[root@localhost johnny]# qmake
uic: Error in line 17, column 38 : Unexpected attribute alignment

Since qmake fails, building fails ...

We need to indicate non-printable or control chars too in the Password field. Maybe encoding to something like UTF would work for passwords containing such chars.

Via --show=types option. Can use the existing HashTypeChecker handler for this.

We don't do that apparently. One has to hit resume attack for that to happen.

Apple doesn't allow text inside progress bars according to the latest HI guide:
https://developer.apple.com/library/mac/documentation/UserExperience/Conceptual/OSXHIGuidelines/ControlsIndicators.html#//apple_ref/doc/uid/20000957-CH50-SW1

In our release thread, some users requested a lineEdit to put rules into wordlist mode. I added this but I saw in the thread of Aleksey and tested by myself that core john don't support --rules="OPTION". I'll make the same check as --show hash type

Take a look at commits between 86ecbf5 and 8304a34. These implement the JohnHandler class interface that is going to be the base for all future handler wrappers around JtR. A sample implementation has been provided in JohnAttack handler, the main cracking process.

Things to keep in mind regarding the implementation:

• All handler specific functionality that is compute intensive, like post or pre processing of JtR output, should be handled in methods pertaining to that handler class.
• Each JohnHandler has thread affinity to a separate own thread(the moveToThread in the constructor), thus all signal slots connections to and from a JohnHandler should use Qt::QueuedConnection so that we are guaranteed to be honoring the thread affinity.
• All data that is signaled to the UI from a handler should be presented in a fashion readily consumable by the UI code (no compute intensive stuff so that we don't lag in the UI)
• Don't invoke handler methods that are compute intensive or busy wait directly from the UI(i.e. synchronously: handler.exampleMethod()). If you find your self in need of calling slot directly without a meaningful connect() opportunity, please use QMetaObject::invokeMethod() so that we ensure the method is called on the handler's thread.
• I use JohnHandler directly for the simple show and version handlers for now, until we need to expand on their functionality.

What you need to do next is to base HashTypeChecker and the password guessing handler on JohnHandler.

Since we now have "not only one last session", now we have a lot of saved sessions with the session history, maybe it'd be more appropriate to store the options to repopulate the UI in the .johnny file rather than with QSettings ?
As I see it, the first step to get to a multi-session environment would be to start from the session history PR and add a way to save each options and repopulate the UI when we load a new session. I'd like to start by that monday.

Reminder : I should update the french translation files before release since we did a lot of changes in johnny since the beggining of GSOC

When right clicking on the main tool bar(the one with open pwd file, open last session, start attack etc.. actions), a context menu appears and clicking on it make this mandatory toolbar disappears from the screen. There is no way to put it back. So the user can no longer do anything, he has to restart Johnny. We should not permit doing this.

1. Johnny seems to update only console log when attack ends. For example, sometimes I get the 4 hashes loaded message only when I pause attack.
2. Maybe status button could be interesting, or automatic status(but we still have the problem we can't use stdin to send an input) every X minute.

On the same note, I've seen a TODO
void MainWindow::readJohnShow()
{
// TODO: Read John's output while it runs. Do not wait before it
// finishes.

In current master, the options related to the tabs(which depends on the modes), are lost when you start an attack/finish(or pause)/go look at the console log(or other views)/start the same attack again. I suspect it is related to c332cc1 .

The new conditions in getAttackParameters such as "else if (m_ui->externalModeTab->isVisible())" all returns visible=false in the use-case described, we should not use the isVisible() method to determine the selected mode.

More detailed OUTPUT:
I start attack with external tab selected(visible)
13:25:33 : /home/math/jumbo/run/john --format=Raw-MD5 --external=Strip --users=bob --session=/home/math/_john/johnny/default /home/math/3.raw-md5.pw
Warning: external mode 'Strip' can't be resumed if aborted
No password hashes loaded (see FAQ)

I didn't change anything, except changing view, and started a new attack. I lost the --external=Strip
13:25:46 : /home/math/jumbo/run/john --format=Raw-MD5 --users=bob --session=/home/math/_john/johnny/default /home/math/3.raw-md5.pw
No password hashes loaded (see FAQ)

I'm using latest jumbo official release 1.8.0.6-jumbo-1 (the only official release which contains the CTRL_C stuff) on Windows. I don't know what's going on with encoding. The character ":" that we use in our session name is written as a weird character as you can see in the screenshot. This prevents our session history feature to work as the .rec is never found by johnny since it doesn't have the same name ..

I updated openwall wiki, let's use this issue to give me your feedback.
http://openwall.info/wiki/john/johnny

We need to display the default hash format type in the progressbar (and elsewhere), in the case where the user didn't specify one.

As we started using QTabWidget for selecting the attack mode, I discovered that mouse over a tab + mouse wheel(which should be used to scroll down the options) will change the selected index of the tab widget(which changes the attack mode), It's pretty annoying and annoying for the user, I need to fix that !

This bug is discussed in the qt website : https://bugreports.qt.io/browse/QTBUG-8428
This summarizes the bug(I copied paster this from the qt bug report as it explains my problem really great) :
Start a Qt app that has a QTabBar
Mouse over the QTabBar
using your mouse wheel (or ball) will now cause the QTabBar to change the active tab.

This is very disconcerting and non-standard behavior. For example, I'm develping a multi-tabbed app. If the user is trying to scroll the content in a particular view, and accidentially places the mouse over the tab bar, suddenly they zip away from the intended view. More likely, they click on the tab bar handle to switch views, then immediately try to scroll the content with the mouse wheel, only to have it disappear. At the very least this is not the native behavior on Mac, but I suggest this weird behavior be turned off for all Qt platforms.

Similar to what has been done here Tarsnap/tarsnap-gui#15.

As proposed by Shinnok, get rid of apply, don't apply, save, don't save, etc.." and automatically save all changes.

Move the INSTALL instructions to INSTALL distfile.

Not just the attack one.

When opened in Finder, the title on the volume is /Users/admin/etc..

It should be just Johnny, can you look into it? See what you did wrong with the macdeployqt command.

PS: If the commands used are more than one line, it might be useful to create a small shell script and add it to the repo.

When we code-freeze a branch for release v2.0, we should not forget to add to johnny.pro
CONFIG += RELEASE in that branch since release builds are faster than debug ones.

Otherwise, git status will show these as Untracked files.
Also, johnny should be removed on make clean.

I didn't implement the feature but I make a draft of the .ui based on what you said on the list in the thread Mathieu's weekly report 8. What do you think of that @shinnok ? Imagine that there is one checkbox from PR #54 in each row of the table view.

Here is the use case.
Imagine the .pot file has some already cracked password in it. If we open a new file, start a new attack or resume another one with a different format, callJohnShow() method call ./john --formatXXX --show pw1.txt. In the tableview, the already cracked password from the .pot file from format YYY won't appears.

1. I understand this was probably done to speed up john --show by Aleksey. Maybe the first call of john show should be done without the --format thing to get a more coherent tableview and further calls could be made with the --format to keep the speed ? Or maybe, was it the behaviour that you guys wanted ?