[john-users] Most efficient way to brute force

[Powen Cheng]

Hello,

This is the test setup that I am stuck with so I want to make sure that
these two commands are the most efficient way to brute force with 8 threads
per video card.

As per magnumripper, using two separate terminals.

OMP_NUM_THREADS=8 ./john -dev=0 -node=1/2 -form=tezos-opencl
-ses=tezos1 tezos -inc

OMP_NUM_THREADS=8 ./john -dev=1 -node=2/2 -form=tezos-opencl
-ses=tezos2 tezos -inc


I was told to use --incremental and I read that I could also create and use
my own custom Incremental.

[Incremental:Custom]
File = custom.chr
CharCount = 95
MinLen = 6
MaxLen = 8

So to use my own custom incremental. I would simply add -inc:custom -
is this correct?

OMP_NUM_THREADS=8 ./john -dev=0 -node=1/2 -form=tezos-opencl
-ses=tezos1 tezos -inc:custom

OMP_NUM_THREADS=8 ./john -dev=1 -node=2/2 -form=tezos-opencl
-ses=tezos2 tezos -inc:custom


Other related questions.

Since the MinLen starts at 6. I am guessing that it would start with
000000 up to charset?
Then when Length of 6 is done, it would move to 7 or 0000000, etc.

Please help me understand how incremental work with John.

I want to make sure that I using this brute force as efficient as possible.

Thanks,
Po

[Solar Designer]

Post by Powen Cheng
This is the test setup that I am stuck with so I want to make sure that
these two commands are the most efficient way to brute force with 8 threads
per video card.
As per magnumripper, using two separate terminals.
OMP_NUM_THREADS=8 ./john -dev=0 -node=1/2 -form=tezos-opencl
-ses=tezos1 tezos -inc
OMP_NUM_THREADS=8 ./john -dev=1 -node=2/2 -form=tezos-opencl
-ses=tezos2 tezos -inc

These may be fine (assuming you have at least 16 logical CPUs), but most
importantly you need to focus the attack based on what you know/recall
about the password. You previously tried asking about that, and I
recommended that you use mask mode, possibly along with other modes:

https://www.openwall.com/lists/john-users/2018/10/28/3

This remains my current recommendation. Have you tried it? How?
What were the results?

Post by Powen Cheng
I was told to use --incremental and I read that I could also create and use
my own custom Incremental.

You could, but why would you? Chances are that whatever you know/recall
about the password is best expressed as a mask.

Post by Powen Cheng
[Incremental:Custom]
File = custom.chr
CharCount = 95
MinLen = 6
MaxLen = 8
So to use my own custom incremental. I would simply add -inc:custom -
is this correct?
OMP_NUM_THREADS=8 ./john -dev=0 -node=1/2 -form=tezos-opencl
-ses=tezos1 tezos -inc:custom
OMP_NUM_THREADS=8 ./john -dev=1 -node=2/2 -form=tezos-opencl
-ses=tezos2 tezos -inc:custom

Yes, but you probably don't need to do that.

Post by Powen Cheng
Since the MinLen starts at 6. I am guessing that it would start with
000000 up to charset?
Then when Length of 6 is done, it would move to 7 or 0000000, etc.
Please help me understand how incremental work with John.

Under the hood, and in terms of ordering of candidate passwords tried,
it's far more complex than that. It will be switching lengths back and
forth, and will be testing weird-looking sequences of characters, trying
to optimize for non-increasing estimated probability of each being the
password. It estimates those probabilities based on previously known
passwords - the training set used when the .chr file was generated. For
the .chr files bundled with JtR, the training set is the RockYou leak.

If you generate your own .chr file, you re-train based on whatever is in
your john.pot at that time.

Post by Powen Cheng
I want to make sure that I using this brute force as efficient as possible.

What approach is most efficient depends on what you know/recall about
the password.

Alexander

[Powen Cheng]

Hi Alexander,

I have tested mask mode and thank you for this suggestion.

As for now, I still need to figure how to create a GPU system properly
before I let this system run for the real task.
I am very limited with what driver and hardware I could use with Ubuntu
14.04.1 as in 16.04 the opencl was removed unless I am using Nvidia cards
and I still need to get pass this error.
OpenCL CL_INVALID_DEVICE (-33) error in opencl_common.c:452 - Error
querying PLATFORM_NAME

Thanks again for your help,
Po

Post by Powen Cheng

Post by Powen Cheng
This is the test setup that I am stuck with so I want to make sure that
these two commands are the most efficient way to brute force with 8

threads

Post by Powen Cheng
per video card.
As per magnumripper, using two separate terminals.
OMP_NUM_THREADS=8 ./john -dev=0 -node=1/2 -form=tezos-opencl
-ses=tezos1 tezos -inc
OMP_NUM_THREADS=8 ./john -dev=1 -node=2/2 -form=tezos-opencl
-ses=tezos2 tezos -inc

These may be fine (assuming you have at least 16 logical CPUs), but most
importantly you need to focus the attack based on what you know/recall
about the password. You previously tried asking about that, and I
https://www.openwall.com/lists/john-users/2018/10/28/3
This remains my current recommendation. Have you tried it? How?
What were the results?

Post by Powen Cheng
I was told to use --incremental and I read that I could also create and

use

Post by Powen Cheng
my own custom Incremental.

You could, but why would you? Chances are that whatever you know/recall
about the password is best expressed as a mask.

Post by Powen Cheng
[Incremental:Custom]
File = custom.chr
CharCount = 95
MinLen = 6
MaxLen = 8
So to use my own custom incremental. I would simply add -inc:custom -
is this correct?
OMP_NUM_THREADS=8 ./john -dev=0 -node=1/2 -form=tezos-opencl
-ses=tezos1 tezos -inc:custom
OMP_NUM_THREADS=8 ./john -dev=1 -node=2/2 -form=tezos-opencl
-ses=tezos2 tezos -inc:custom

Yes, but you probably don't need to do that.

Post by Powen Cheng
Since the MinLen starts at 6. I am guessing that it would start with
000000 up to charset?
Then when Length of 6 is done, it would move to 7 or 0000000, etc.
Please help me understand how incremental work with John.

Under the hood, and in terms of ordering of candidate passwords tried,
it's far more complex than that. It will be switching lengths back and
forth, and will be testing weird-looking sequences of characters, trying
to optimize for non-increasing estimated probability of each being the
password. It estimates those probabilities based on previously known
passwords - the training set used when the .chr file was generated. For
the .chr files bundled with JtR, the training set is the RockYou leak.
If you generate your own .chr file, you re-train based on whatever is in
your john.pot at that time.

Post by Powen Cheng
I want to make sure that I using this brute force as efficient as

possible.
What approach is most efficient depends on what you know/recall about
the password.
Alexander

[Solar Designer]

Post by Powen Cheng
I have tested mask mode and thank you for this suggestion.

I recommend that you share more detail on what you tried and how it's
working for you or not. That way, the community should be able to help
you focus the attack for your specific case, given what you recall about
the password. This is far more important than using your GPUs better or
at all. Even a 100x difference in performance isn't as important as
properly focusing the attack.

Post by Powen Cheng
As for now, I still need to figure how to create a GPU system properly
before I let this system run for the real task.
I am very limited with what driver and hardware I could use with Ubuntu
14.04.1 as in 16.04 the opencl was removed unless I am using Nvidia cards
and I still need to get pass this error.
OpenCL CL_INVALID_DEVICE (-33) error in opencl_common.c:452 - Error
querying PLATFORM_NAME

I'm out of context on this, but "opencl was removed unless I am using
Nvidia cards" makes no sense to me. I'm not setting up new GPU boxes
often, but FWIW here are (edited) commands I saved from my shell history
on a client's Ubuntu 18.x setup I helped make a few months ago:

sudo bash
apt-get install libssl-dev opencl-headers ocl-icd-opencl-dev clinfo
cd /path/to/amdgpu-pro-18.30-641594
./amdgpu-pro-install --opencl=pal,legacy --headless

JtR worked there with the AMD GPUs fine.

Alexander

[oayz]

I need a rule with quotation mark. This rule works fine:
: $[."]
This obviously doesn't (quote inside quotes):
: Az"[."]"
So I tried usual "escape" character, still getting "invalid rule":
: Az"[.\"]"
I think it should work - a bug?
Thanks,Oscar

[Solar Designer]

Post by oayz
: $[."]
: Az"[."]"

You can change this line to e.g.:

Az,[."],

which the preprocessor expands under-the-hood into two rules:

Az,.,
Az,",

Of course, there's no reason to use the "A" command for just one
character, but I assume you gave this as an example, and your actual use
is with longer strings.

Post by oayz
: Az"[.\"]"
I think it should work - a bug?

No, and no. The quote character is just a character you provide to the
"A" command telling it that you'll terminate the string with that
character. It is not special to the preprocessor, so no reason to
escape it for the preprocessor. In your case, the line expanded to:

Az"."
Az"""

where the second expanded rule is invalid: it appends an empty string
and is followed by an extra quote character in place where the parser
expects a command character. Luckily, there's no command of that name,
so you get an error reported.

Alexander

[oayz]

Thanks for response. I should apply RTFM to myself:
"Although the use of the double-quote character is good for readability,you may use any other character not found in STR instead.  This isparticularly useful when STR contains the double-quote character." 
Thanks again - for great tool and superb support
Oscar

Post by oayz
: $[."]
: Az"[."]"

You can change this line to e.g.:

Az,[."],

which the preprocessor expands under-the-hood into two rules:

Az,.,
Az,",

Of course, there's no reason to use the "A" command for just one
character, but I assume you gave this as an example, and your actual use
is with longer strings.

Post by oayz
: Az"[.\"]"
I think it should work - a bug?

No, and no.  The quote character is just a character you provide to the
"A" command telling it that you'll terminate the string with that
character.  It is not special to the preprocessor, so no reason to
escape it for the preprocessor.  In your case, the line expanded to:

Az"."
Az"""

where the second expanded rule is invalid: it appends an empty string
and is followed by an extra quote character in place where the parser
expects a command character.  Luckily, there's no command of that name,
so you get an error reported.

Alexander