JTR with bigcrypt/crypt16

[Stephen Cartwright]

I tried to use JTR on a password file that uses bigcrypt to hash the
passwords. However the program does not seem to work with this type of
hash. Is there anyway to get JTR to work with bigcrypt?

Many Thanks,

Stephen

[Solar Designer]

Stephen,

Post by Stephen Cartwright
I tried to use JTR on a password file that uses bigcrypt to hash the
passwords. However the program does not seem to work with this type of
hash. Is there anyway to get JTR to work with bigcrypt?

For the JtR-supported flavor of bigcrypt/crypt16, it should just work,
splitting the hashes into their halves on load. There's another (even
weaker) flavor that JtR does not support (no demand).

What OS did the password file come from? To my knowledge, bigcrypt-alikes
are used on at least some versions of HP-UX, SCO, OSF/1 (DU, Tru64),
Ultrix.

What version of JtR?

Can you please post one line from your password file that JtR doesn't
load? (Obviously, change that password immediately.)

Thanks,

[Stephen Cartwright]

The OS is: Tru64 51b

John version is 1.6

Here is a sample (password has been changed):

sgcartwr:QpwuMaH29uXKkXXs6n5D01vg:500169:500169:Stephen
Cartwright:/home/users/sgcartwr:/usr/local/bin/bash

Thanks for your time!

Stephen

Post by Solar Designer
Stephen,

Post by Stephen Cartwright
I tried to use JTR on a password file that uses bigcrypt to hash the
passwords. However the program does not seem to work with this type of
hash. Is there anyway to get JTR to work with bigcrypt?

For the JtR-supported flavor of bigcrypt/crypt16, it should just work,
splitting the hashes into their halves on load. There's another (even
weaker) flavor that JtR does not support (no demand).
What OS did the password file come from? To my knowledge, bigcrypt-alikes
are used on at least some versions of HP-UX, SCO, OSF/1 (DU, Tru64),
Ultrix.
What version of JtR?
Can you please post one line from your password file that JtR doesn't
load? (Obviously, change that password immediately.)
Thanks,
--
Alexander Peslyak <solar at openwall.com>
GPG key ID: B35D3598 fp: 6429 0D7E F130 C13E C929 6447 73C3 A290 B35D 3598
http://www.openwall.com - bringing security into open computing environments
Was I helpful? Please give your feedback here: http://rate.affero.net/solar

[Solar Designer]

Stephen,

This is the JtR-supported flavor. Both John 1.6 and 1.6.38 loaded this
line just fine and cracked the second half within 10 seconds (of
course, 1.6.38 cracked it quicker than 1.6 did). Your password ends in
"ow"; the first 8 characters will take longer to crack.

Are you experiencing any problems with using John on this password file?
--
Alexander Peslyak <solar at openwall.com>
GPG key ID: B35D3598 fp: 6429 0D7E F130 C13E C929 6447 73C3 A290 B35D 3598
http://www.openwall.com - bringing security into open computing environments

Was I helpful? Please give your feedback here: http://rate.affero.net/solar

Post by Stephen Cartwright
The OS is: Tru64 51b
John version is 1.6
sgcartwr:QpwuMaH29uXKkXXs6n5D01vg:500169:500169:Stephen
Cartwright:/home/users/sgcartwr:/usr/local/bin/bash
Thanks for your time!
Stephen

[Stephen Cartwright]

I am sorry, I was confused, I looked at the pot file and saw that they
were broken up and since before I was looking at random passwords to
test I did not see that it was breaking them properly, I just jumped
to the conclusion that it was mis-interpreting the type of hash. Also
I did not use the -show option.

It is working fine! My apologies and thanks for your time.

One more thing the computer I am running Jack on has 4 CPUs. To take
advantage of this my understanding is that I should use the
-session:FILE option and start four different processes in the
background with different session files and -wordfile:FILE word
lists?

Do you reccomend I try DJohn instead?

Stephen

Post by Solar Designer
Stephen,
This is the JtR-supported flavor. Both John 1.6 and 1.6.38 loaded this
line just fine and cracked the second half within 10 seconds (of
course, 1.6.38 cracked it quicker than 1.6 did). Your password ends in
"ow"; the first 8 characters will take longer to crack.
Are you experiencing any problems with using John on this password file?
--
Alexander Peslyak <solar at openwall.com>
GPG key ID: B35D3598 fp: 6429 0D7E F130 C13E C929 6447 73C3 A290 B35D 3598
http://www.openwall.com - bringing security into open computing environments
Was I helpful? Please give your feedback here: http://rate.affero.net/solar

Post by Stephen Cartwright
The OS is: Tru64 51b
John version is 1.6
sgcartwr:QpwuMaH29uXKkXXs6n5D01vg:500169:500169:Stephen
Cartwright:/home/users/sgcartwr:/usr/local/bin/bash
Thanks for your time!
Stephen

[Solar Designer]

Stephen,

Post by Stephen Cartwright
I did not use the -show option.

Yes, you should be using -show rather than looking at john.pot file
contents.

Post by Stephen Cartwright
One more thing the computer I am running Jack on has 4 CPUs. To take
advantage of this my understanding is that I should use the
-session:FILE option and start four different processes in the
background with different session files and -wordfile:FILE word
lists?

Yes. However, wordlist-based cracking is generally quite fast, so you'd
probably want to play with "incremental" mode settings instead. Once
you're done with "single crack" mode and the wordlists (don't forget to
enable -rules), you could set your four instances to try lengths 0-5, 6,
7, and 8, respectively. You would define john.conf sections like this:

[Incremental:All05]
File = $JOHN/all.chr
MinLen = 0
MaxLen = 5
CharCount = 95

...

[Incremental:All8]
File = $JOHN/all.chr
MinLen = 8
MaxLen = 8
CharCount = 95

This example is for the 1.6.x development versions - which you should be
using if you care about performance.

Since the multiple instances of John would not exchange information
about already cracked passwords on the fly, you may want to interrupt
and continue the sessions after they've cracked most weak passwords
(e.g., after a few hours). Then the recovered sessions won't load
password hashes already cracked by other instances. In fact, you may do
this more than once for even better efficiency.

You would also want to set this:

[Options]
# Use idle cycles only
Idle = Y

Post by Stephen Cartwright
Do you reccomend I try DJohn instead?

No. Unfortunately, all John the Ripper hacks for parallel processing
that I am aware of result in a far less optimal order in which candidate
passwords are tried. They also tend to be unreliable.

[Alen Williams]

Post by Solar Designer
No. Unfortunately, all John the Ripper hacks for parallel processing
that I am aware of result in a far less optimal order in which
candidate
passwords are tried. They also tend to be unreliable.

Hmmm, as I'm in the middle of writing one of those hacks, how should
that be tackled?

Right now the candidates in my hack are presented in the same order as
dJohn. How should I improve that?

Alen.

[Solar Designer]

Post by Alen Williams

Post by Solar Designer
Unfortunately, all John the Ripper hacks for parallel processing
that I am aware of result in a far less optimal order in which
candidate passwords are tried. They also tend to be unreliable.

Hmmm, as I'm in the middle of writing one of those hacks, how should
that be tackled?

I am sorry, but I've given up wasting time on responding to this kind of
questions via private e-mail. Now that we're on an archived mailing
list, things are a bit different. I might find the time (several hours)
to describe my thoughts on this and to answer any follow-up questions in
here eventually... although I do not expect anything to come out of
this. I feel that it would be more productive for me to implement the
thing myself (I did have a prototype working back in 1997).

Post by Alen Williams
Right now the candidates in my hack are presented in the same order as
dJohn.

Well, this only works fine in two cases:

1. When you're programming this for fun rather than for actual use, or

2. When you intend to search a pre-defined portion of the keyspace
_exhaustively_ and somehow don't care to get most of your passwords
cracked early on during the search.

In most practical cases, I would expect John running on one CPU to be
far more effective at getting weak passwords cracked within a certain
(limited) amount of time than dJohn running on 10 CPUs.

Post by Alen Williams
How should I improve that?

Well, here's a really short answer: there exist fairly straightforward
enhancements of John the Ripper's "incremental" mode algorithm that
allow for parallelization, with no or little effect on the efficiency of
the order of candidates.

It is also entirely possible to support nodes of differing and changing
performance, add/remove nodes on the fly, and handle node and network
failures gracefully.

[Stephen Cartwright]

What I ended up doing is I got a huge dictionary (found every wordlist
I could and then stripped out all non-printible characters, converted
all uppercase to lowercase, sorted and uniqed to give about a 50 mb
wordlist) then I broke up the passwd file into four sections and just
ran 4 seperate instances of John (in four different directories) --
one on each section. I am using the default settings except for the
new wordlist.

Does this sound like optimal usage for my 4-cpu machine? Any
reccomendations? I can let this run for a long time and I have no need
to be stealthly, so my goal is only to break as many hashes as
possible.

One more thing... I have been getting some false posititves. Some
passwords have worked fine, but others that JTR found did not work.
Any idea why?

Thanks in advance!

Stephen

[Solar Designer]

Post by Stephen Cartwright
What I ended up doing is I got a huge dictionary (found every wordlist
I could and then stripped out all non-printible characters, converted
all uppercase to lowercase, sorted and uniqed to give about a 50 mb
wordlist)

It is not clear to me why you had to do that on your own. You could
have simply downloaded this:

ftp://ftp.openwall.com/pub/wordlists/all.gz

or you could have purchased it and more on CD:

http://www.openwall.com/wordlists/

Post by Stephen Cartwright
then I broke up the passwd file into four sections

That way you've likely made the hashes per salt ratio worse for all 4
instances, thereby decreasing the effective c/s rate.

How many password hashes did you have in your full password file (the
halves, the way John reports them on load)? If it's under 1000, then
the hit of using this approach is not that bad. However, if you had,
say, 10,000+ of them, then there may be no gain from running this on 4
CPUs like that (as opposed to using just one CPU) - and you need to use
a better approach (see below).

Basically, you need to add up the numbers of different salts as reported
by your four instances of John and compare that against the number of
different salts that one instance of John would report when run on the
original password file. If these numbers are close, you're fine. If
there's a substantial difference, that is how much processing power
you're wasting by using this approach.

Post by Stephen Cartwright
and just
ran 4 seperate instances of John (in four different directories) --

You did not need to use different directories. They can share the same
john.pot just fine - this is a feature. You merely needed to supply
different session names.

Post by Stephen Cartwright
one on each section. I am using the default settings except for the
new wordlist.

Do you mean, you've changed the "Wordlist = ..." in john.conf, then ran
the 4 instances with no options (for batch mode)? OK, that would
explain why you needed different directories.

Post by Stephen Cartwright
Does this sound like optimal usage for my 4-cpu machine?

No - see above about the hashes per salt ratio.

Post by Stephen Cartwright
Any reccomendations?

1. You need to run "single crack" on all of your accounts at once. John
might crack more passwords this way.

./john -single passwd

2. It is unimportant whether or not you distribute wordlist-based
cracking over all of your CPUs. However, if you do, you may want to
split the wordlist into 4 parts and not split the password file. Rather
than use 4 separate directories, you'd run:

./john -se=w1 -w=words1.lst -rules passwd
./john -se=w2 -w=words2.lst -rules passwd
./john -se=w3 -w=words3.lst -rules passwd
./john -se=w4 -w=words4.lst -rules passwd

(on four different terminals - or simply background the commands and
logout).

3. Once you're done with the wordlists, proceed with "incremental" mode
as I had suggested in my previous response (have different instances try
different password lengths).

Of course, you can script these three steps.

4. Set "Idle = Y" in john.conf.

5. Oh, and you should be using John 1.6.38 (development) - or whatever
version is current - for best performance.

Post by Stephen Cartwright
One more thing... I have been getting some false posititves. Some
passwords have worked fine, but others that JTR found did not work.
Any idea why?

It is unrealistic that John would crack a password "wrongly". More
likely, the system has other measures preventing access to those
accounts, like account or password expiration. It is also possible that
another password database is being consulted at login time, not the
password file which you've grabbed.

[Cheyenne England]

Believe it or not, I am not a hacker.... I have an account on yahoo that is mine, but i forgot the password to. It have very valuable pictures on it, and I really wont them back, can you please help me? I have not clue what u r talkin about in those emails, so could you please explain in easier terms please? I really wont those pictires back. PLEASE?


P.S. I AM NOT A HACKER AND NEVER WILL BE, I HATE HACKER THEY CAUSE TOO MANY PROBLEMS SO WILL YOU PLEASE HELP ME?????
THANK YOU

Post by Stephen Cartwright
What I ended up doing is I got a huge dictionary (found every wordlist
I could and then stripped out all non-printible characters, converted
all uppercase to lowercase, sorted and uniqed to give about a 50 mb
wordlist)

It is not clear to me why you had to do that on your own. You could
have simply downloaded this:

ftp://ftp.openwall.com/pub/wordlists/all.gz

or you could have purchased it and more on CD:

http://www.openwall.com/wordlists/

Post by Stephen Cartwright
then I broke up the passwd file into four sections

That way you've likely made the hashes per salt ratio worse for all 4
instances, thereby decreasing the effective c/s rate.

How many password hashes did you have in your full password file (the
halves, the way John reports them on load)? If it's under 1000, then
the hit of using this approach is not that bad. However, if you had,
say, 10,000+ of them, then there may be no gain from running this on 4
CPUs like that (as opposed to using just one CPU) - and you need to use
a better approach (see below).

Basically, you need to add up the numbers of different salts as reported
by your four instances of John and compare that against the number of
different salts that one instance of John would report when run on the
original password file. If these numbers are close, you're fine. If
there's a substantial difference, that is how much processing power
you're wasting by using this approach.

Post by Stephen Cartwright
and just
ran 4 seperate instances of John (in four different directories) --

You did not need to use different directories. They can share the same
john.pot just fine - this is a feature. You merely needed to supply
different session names.

Post by Stephen Cartwright
one on each section. I am using the default settings except for the
new wordlist.

Do you mean, you've changed the "Wordlist = ..." in john.conf, then ran
the 4 instances with no options (for batch mode)? OK, that would
explain why you needed different directories.

Post by Stephen Cartwright
Does this sound like optimal usage for my 4-cpu machine?

No - see above about the hashes per salt ratio.

Post by Stephen Cartwright
Any reccomendations?

1. You need to run "single crack" on all of your accounts at once. John
might crack more passwords this way.

./john -single passwd

2. It is unimportant whether or not you distribute wordlist-based
cracking over all of your CPUs. However, if you do, you may want to
split the wordlist into 4 parts and not split the password file. Rather
than use 4 separate directories, you'd run:

./john -se=w1 -w=words1.lst -rules passwd
./john -se=w2 -w=words2.lst -rules passwd
./john -se=w3 -w=words3.lst -rules passwd
./john -se=w4 -w=words4.lst -rules passwd

(on four different terminals - or simply background the commands and
logout).

3. Once you're done with the wordlists, proceed with "incremental" mode
as I had suggested in my previous response (have different instances try
different password lengths).

Of course, you can script these three steps.

4. Set "Idle = Y" in john.conf.

5. Oh, and you should be using John 1.6.38 (development) - or whatever
version is current - for best performance.

Post by Stephen Cartwright
One more thing... I have been getting some false posititves. Some
passwords have worked fine, but others that JTR found did not work.
Any idea why?

It is unrealistic that John would crack a password "wrongly". More
likely, the system has other measures preventing access to those
accounts, like account or password expiration. It is also possible that
another password database is being consulted at login time, not the
password file which you've grabbed.
--
Alexander Peslyak
GPG key ID: B35D3598 fp: 6429 0D7E F130 C13E C929 6447 73C3 A290 B35D 3598
http://www.openwall.com - bringing security into open computing environments

Was I helpful? Please give your feedback here: http://rate.affero.net/solar

__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com

[Michael Behrisch]

Post by Solar Designer

Post by Stephen Cartwright
Any reccomendations?

1. You need to run "single crack" on all of your accounts at once. John
might crack more passwords this way.
./john -single passwd
2. It is unimportant whether or not you distribute wordlist-based
cracking over all of your CPUs. However, if you do, you may want to
split the wordlist into 4 parts and not split the password file. Rather
./john -se=w1 -w=words1.lst -rules passwd
./john -se=w2 -w=words2.lst -rules passwd
./john -se=w3 -w=words3.lst -rules passwd
./john -se=w4 -w=words4.lst -rules passwd
(on four different terminals - or simply background the commands and
logout).
3. Once you're done with the wordlists, proceed with "incremental" mode
as I had suggested in my previous response (have different instances try
different password lengths).
Of course, you can script these three steps.

There is an easier and more flexible way to parallelize step 2 and 3
at least if your OS (and your filesystem) supports named pipes (probably
any flavor of Un*x).

You can simply create a named pipe (Linux: "mkfifo johnpipe")
and let one instance of john write the password candidates into:
Step 2:
john --stdout -w=words.lst -rules > johnpipe
Step 3:
john --stdout -inc > johnpipe

Then you can create an arbitrary number of real cracking sessions which
read the candidates from the pipe:
./john -se=w1 --stdin passwd < johnpipe
./john -se=w2 --stdin passwd < johnpipe
...

That's it. No hacks, no patches not even a change to the configuration files.

A further advantage of this approach is that you can dynamically add
and remove sessions just by aborting / starting them and you don't
have to care how to split the password range. Additionally if you
are using nfs the single john sessions donot even need to run
on the same computer. They simply need the same filesystem.

One remark:
The candidate creation instance of john will use almost no computing time
so you can savely start it together with four real crackers on a 4CPU system
(I started it once with 12 "clients" and it had about 5% cpu load on a 1GHz
machine)

Yours,
Michael

P.S: I did not invent this method, but I forgot where I read this first.

[Stephen Cartwright]

That is great info! Thank you very much Solar and Michael for those posts!

If you think of anything else don't hesitate to post it... this is very helpful.

Stephen

Post by Michael Behrisch

Post by Solar Designer

Post by Stephen Cartwright
Any reccomendations?

1. You need to run "single crack" on all of your accounts at once. John
might crack more passwords this way.
./john -single passwd
2. It is unimportant whether or not you distribute wordlist-based
cracking over all of your CPUs. However, if you do, you may want to
split the wordlist into 4 parts and not split the password file. Rather
./john -se=w1 -w=words1.lst -rules passwd
./john -se=w2 -w=words2.lst -rules passwd
./john -se=w3 -w=words3.lst -rules passwd
./john -se=w4 -w=words4.lst -rules passwd
(on four different terminals - or simply background the commands and
logout).
3. Once you're done with the wordlists, proceed with "incremental" mode
as I had suggested in my previous response (have different instances try
different password lengths).
Of course, you can script these three steps.

There is an easier and more flexible way to parallelize step 2 and 3
at least if your OS (and your filesystem) supports named pipes (probably
any flavor of Un*x).
You can simply create a named pipe (Linux: "mkfifo johnpipe")
john --stdout -w=words.lst -rules > johnpipe
john --stdout -inc > johnpipe
Then you can create an arbitrary number of real cracking sessions which
./john -se=w1 --stdin passwd < johnpipe
./john -se=w2 --stdin passwd < johnpipe
...
That's it. No hacks, no patches not even a change to the configuration files.
A further advantage of this approach is that you can dynamically add
and remove sessions just by aborting / starting them and you don't
have to care how to split the password range. Additionally if you
are using nfs the single john sessions donot even need to run
on the same computer. They simply need the same filesystem.
The candidate creation instance of john will use almost no computing time
so you can savely start it together with four real crackers on a 4CPU system
(I started it once with 12 "clients" and it had about 5% cpu load on a 1GHz
machine)
Yours,
Michael
P.S: I did not invent this method, but I forgot where I read this first.
--
Michael Behrisch (Tel. [phone])
HU Berlin, Institut fuer Informatik, Arbeitsgruppe Algorithmen
http://www.informatik.hu-berlin.de/~behrisch/

[Solar Designer]

Post by Michael Behrisch
You can simply create a named pipe (Linux: "mkfifo johnpipe")
john --stdout -w=words.lst -rules > johnpipe
john --stdout -inc > johnpipe
Then you can create an arbitrary number of real cracking sessions which
./john -se=w1 --stdin passwd < johnpipe
./john -se=w2 --stdin passwd < johnpipe
...
That's it. No hacks, no patches not even a change to the configuration files.

Unfortunately, while this would be an elegant solution, it does not work
reliably. John uses stdio to read wordlists (including from stdin), and
of course stdio makes no attempt to ensure it reads whole lines out of
the file descriptor. Whenever one of the real cracking instances of
John would happen to read a block of data ending in the middle of a
line (and this would happen most of the time!), the next instance to
read data out of the pipe would get a partial line instead of a full one -
and would check a wrong candidate password, missing the one that it was
meant to check. With 4 KB stdio buffers and lines consisting of 10
characters on average, about 0.5% of candidate passwords would not be
checked because of this problem alone (and it's not the only problem!),
resulting in John possibly missing even the weakest passwords.

Another problem is that standards such as the Single Unix Specification
do not require serialization of reads/writes from/to file descriptors.
Whenever two or more threads read off the same fd simultaneously, the
behavior is undefined. Some Unix-like kernels actually happen to
serialize reads/writes (for some or all file types), some don't. Linux
doesn't, except with Openwall patches for 2.4.x kernels where I've
enforced serialization to fight a class of potential security-relevant
in-kernel race conditions (but this hardening measure has no chance to
be accepted upstream). With small reads off pipes things happen to work
"as desired" in practice most of the time, though.

Finally, with the proposed approach, it is impossible to reliably
recover interrupted sessions - even if the problems I've explained above
wouldn't exist.

Post by Michael Behrisch
A further advantage of this approach is that you can dynamically add
and remove sessions just by aborting / starting them

Each time you abort a real cracking session, you miss up to a few
hundred of candidate passwords (typical) that this session had buffered
(in the stdio I/O buffer and in John's own data structures). No other
instance of John would pick those since they would have already been
read off the pipe.

Post by Michael Behrisch
P.S: I did not invent this method, but I forgot where I read this first.

I had seen it before, too. In fact, I've been toying with the idea to
enhance John to actually make a similar method work reliably, -- but I
never actually did that (it's far from trivial).

[Solar Designer]

Post by Solar Designer
Another problem is that standards such as the Single Unix Specification
do not require serialization of reads/writes from/to file descriptors.
Whenever two or more threads read off the same fd simultaneously, the
behavior is undefined. [...]

A correction: this particular problem does not apply to the approach
described by Michael because the concurrent reads actually happen via
different fd's, not via duplicates of the same fd. Similarly, the
hardening measure that I've put into Linux 2.4.x-ow kernels which would
serialize concurrent reads off the same fd does not apply here.

However, SUSv3 also says this:

"The behavior of multiple concurrent reads on the same pipe, FIFO, or
terminal device is unspecified."

So we're back to the same conclusion.

[Stephen Cartwright]

Breaking the password file apart randomly causes duplicate salts to be used
across the password files, but what if you broke the password file apart
based upon salt?

Then for every salt, there would only be 1 set of passwords still. right?

Another thing I am wondering about is how does John crack partial passwords?
Many passwords have just two or three characters cracked. However I thought
that with a hash, the output varies widely depending on the input. This
should means that you can not just try words and see if some of the
characters match up. Since this is obviously not the case (unless I am
missing something), how is it possible to find these partial passwords?


Thanks!

Stephen

Post by Cheyenne England

Post by Stephen Cartwright
What I ended up doing is I got a huge dictionary (found every wordlist
I could and then stripped out all non-printible characters, converted
all uppercase to lowercase, sorted and uniqed to give about a 50 mb
wordlist)

It is not clear to me why you had to do that on your own. You could
ftp://ftp.openwall.com/pub/wordlists/all.gz
http://www.openwall.com/wordlists/

Post by Stephen Cartwright
then I broke up the passwd file into four sections

That way you've likely made the hashes per salt ratio worse for all 4
instances, thereby decreasing the effective c/s rate.
How many password hashes did you have in your full password file (the
halves, the way John reports them on load)? If it's under 1000, then
the hit of using this approach is not that bad. However, if you had,
say, 10,000+ of them, then there may be no gain from running this on 4
CPUs like that (as opposed to using just one CPU) - and you need to use
a better approach (see below).
Basically, you need to add up the numbers of different salts as reported
by your four instances of John and compare that against the number of
different salts that one instance of John would report when run on the
original password file. If these numbers are close, you're fine. If
there's a substantial difference, that is how much processing power
you're wasting by using this approach.

Post by Stephen Cartwright
and just
ran 4 seperate instances of John (in four different directories) --

You did not need to use different directories. They can share the same
john.pot just fine - this is a feature. You merely needed to supply
different session names.

Post by Stephen Cartwright
one on each section. I am using the default settings except for the
new wordlist.

Do you mean, you've changed the "Wordlist = ..." in john.conf, then ran
the 4 instances with no options (for batch mode)? OK, that would
explain why you needed different directories.

Post by Stephen Cartwright
Does this sound like optimal usage for my 4-cpu machine?

No - see above about the hashes per salt ratio.

Post by Stephen Cartwright
Any reccomendations?

1. You need to run "single crack" on all of your accounts at once. John
might crack more passwords this way.
./john -single passwd
2. It is unimportant whether or not you distribute wordlist-based
cracking over all of your CPUs. However, if you do, you may want to
split the wordlist into 4 parts and not split the password file. Rather
./john -se=w1 -w=words1.lst -rules passwd
./john -se=w2 -w=words2.lst -rules passwd
./john -se=w3 -w=words3.lst -rules passwd
./john -se=w4 -w=words4.lst -rules passwd
(on four different terminals - or simply background the commands and
logout).
3. Once you're done with the wordlists, proceed with "incremental" mode
as I had suggested in my previous response (have different instances try
different password lengths).
Of course, you can script these three steps.
4. Set "Idle = Y" in john.conf.
5. Oh, and you should be using John 1.6.38 (development) - or whatever
version is current - for best performance.

Post by Stephen Cartwright
One more thing... I have been getting some false posititves. Some
passwords have worked fine, but others that JTR found did not work.
Any idea why?

It is unrealistic that John would crack a password "wrongly". More
likely, the system has other measures preventing access to those
accounts, like account or password expiration. It is also possible that
another password database is being consulted at login time, not the
password file which you've grabbed.
--
Alexander Peslyak <solar at openwall.com <http://openwall.com>>
GPG key ID: B35D3598 fp: 6429 0D7E F130 C13E C929 6447 73C3 A290 B35D 3598
http://www.openwall.com - bringing security into open computing environments
http://rate.affero.net/solar

[Frank Dittrich]

Post by Stephen Cartwright
Breaking the password file apart randomly causes duplicate salts
to be used across the password files, but what if you broke the
password file apart based upon salt?

That's certainly a better alternative.
You could even collect the most frequently used salts of uncracked
passwords (after running single crack mode and simple password rules)
into a separate password file.
Since the number of different salts is smaller, you'll be able to
test a larger part of the search space on these hashes.

Post by Stephen Cartwright
Then for every salt, there would only be 1 set of passwords still.

Right, except for a few somewhat broken password algorithms,
see comments regarding bigcrypt.

Post by Stephen Cartwright
Another thing I am wondering about is how does John crack partial passwords?
Many passwords have just two or three characters cracked.

That's caused by the way bigcrypt works.
Instead of restrictinc the significant password length to 8
characters, bigcrypt calculates a hash using "Traditional DES" and a
2 character hash. If the password is longer than 8 characters, the
remaining characters of the password (starting from offset 8) are
encrypted separately (again using "Traditional DES", the salt,
however, is composed of the first 2 characters of the hash computed
from the first part of the password).

That's why, the bigcrypt hash QpwuMaH29uXKkXXs6n5D01vg
(from your previous mail) can be split into 2 DES hashes
QpwuMaH29uXKk
is the hash for the first part of your password (8 characters)
wuXXs6n5D01vg
is the hash of the second part of your password (2 characters, which
we know because the "password" of length 2 has been easily cracked.

Post by Stephen Cartwright
However I thought that with a hash, the output varies widely
depending on the input.

For bigcrypt, you have 2 separate hashes which can be cracked
individually.


That also means, you shouldn't just split all your bigcrypt hashes
into individual password files.
Instead, you could split each bigcrypt hash into 2 DES hashes
after trying single crack mode on the bigcrypt password file.


$ cut --fields=2 --delimiter=":" pw_bigcrypt | cut --bytes=1-13 \

Post by Stephen Cartwright
| grep -n "." >> bigcrypt.part1

$ cut --fields=2 --delimiter=":" pw_bigcrypt | cut --bytes=3-4,14- \

Post by Stephen Cartwright
| grep -n "." >> bigcrypt.part2

Splitting the part 1 and 2 does only make sense because it's much
more likely that shorter "passwords" will have been used here and
that part 1 of the passwords could more likely be cracked using
wordlists and --rules.

For incremental mode, you should combine these and then split
based on the DES hash.

For john --show, it doesn't matter whether you use pw_bigcrypt
(john will try to compose the complete passwords from the parts
which have beeen cracked) or bigcrypt.part1/2 (john will report
dummy user names (the line numbers from "grep -n") - the same
applies for cracked passwords (command line and log file)).


Regards,
Frank

[Frank Dittrich]

Post by Frank Dittrich
$ cut --fields=2 --delimiter=":" pw_bigcrypt | cut --bytes=1-13 \

Post by Stephen Cartwright
| grep -n "." >> bigcrypt.part1

^ just to avoid confusion: this is the continuation of the
command line, and it starts with the "pipe" character, |

Post by Frank Dittrich
Splitting the part 1 and 2 does only make sense because it's much
more likely that shorter "passwords" will have been used here

"here" means: in part 2 of the split passwords

Post by Frank Dittrich
For incremental mode, you should combine these and then split
based on the DES hash.

I wanted to say: "based on the DES salt".

Frank

[Solar Designer]

Post by Stephen Cartwright
Breaking the password file apart randomly causes duplicate salts to be used
across the password files, but what if you broke the password file apart
based upon salt?
Then for every salt, there would only be 1 set of passwords still. right?

Yes, and this works fine for most salted hash types, but with bigcrypt
there is the difficulty so nicely explained by Frank.

Also, whenever you split your password files rather than your keyspace,
you increase the key generation and setup "overhead". I'll try to
explain this.

John obtains (or generates) its candidate passwords to try using certain
algorithms (cracking modes). While those algorithms are pretty
efficient, they nevertheless do consume quite a few CPU cycles to
generate each candidate password. With the more advanced (slower)
password hash types, this "overhead" is negligible, while with the
weaker/quicker ones (e.g., LM hashes) it has to be taken into account.
The traditional DES-based crypt(3) hashes are a marginal case, with the
password generation "overhead" being very low but still measurable.

More importantly, John is smart enough to separate the "key setup"
from the actual hashing (the details of how this may or may not be done
are specific to each hash type). The key setup only needs to be done
once per candidate password, whereas the hashing has to be done for each
{candidate password, salt} combination. For traditional DES-based
crypt(3) hashes, the cost of key setup is roughly 10% of the cost of
actual hashing.

So we've got two tasks which are done per-candidate rather than
per-salt. When running multiple instances of John with each trying the
same candidate passwords (even against different salts), you have each
instance do some duplicate work (the same that other instances do).

If the number of salts per John instance is not very low, this overhead
will be acceptably low (e.g., around 1% for traditional crypt(3) with 10
salts per John instance). However, if you would be splitting just 4
different salts across 4 CPUs in this way, the overhead could be 10%.

Of course, if splitting password files rather than the keyspace happens
to help achieve a more optimal order of candidate passwords, that may
make up for the overhead. So it is not obvious which approach is better
after all.

[Frank Dittrich]

Post by Solar Designer
Yes, and this works fine for most salted hash types, but with bigcrypt
there is the difficulty so nicely explained by Frank.

I just remenbered what Google told me after I read about
bigcrypt in this mailing list.
(And the typos are due to me being tired
at the end of a hard day.)

Frank

[Solar Designer]

Stephen,

Here's one thing you could do to help John improve:

On your Tru64 system, set a password that is longer than 16 characters
(that is, 17 or more). Then post in here both the password you've set
and the resulting hash.

Anyone else who has a system with bigcrypt/crypt16-alike hashes is
welcome to do the same.

Thanks,

[sky lab din]

 
Dear friend
can you please explain step by step method how to crack the hashes we catch by using the ettercap tool in the network(encrypted).How to add dictionary to that.

Post by Solar Designer
Stephen,
This is the JtR-supported flavor. Both John 1.6 and 1.6.38 loaded this
line just fine and cracked the second half within 10 seconds (of
course, 1.6.38 cracked it quicker than 1.6 did). Your password ends in
"ow"; the first 8 characters will take longer to crack.
Are you experiencing any problems with using John on this password file?
--
Alexander Peslyak <solar at openwall.com>
GPG key ID: B35D3598 fp: 6429 0D7E F130 C13E C929 6447 73C3 A290 B35D 3598
http://www.openwall.com - bringing security into open computing environments
Was I helpful? Please give your feedback here: http://rate.affero.net/solar

Post by Stephen Cartwright
The OS is: Tru64 51b
John version is 1.6
sgcartwr:QpwuMaH29uXKkXXs6n5D01vg:500169:500169:Stephen
Cartwright:/home/users/sgcartwr:/usr/local/bin/bash
Thanks for your time!
Stephen

Have a nice day
YOURS
skylabudeen

[Solar Designer]

Post by sky lab din
Dear friend
can you please explain step by step method how to crack the hashes we catch by using the ettercap tool in the network(encrypted).How to add dictionary to that.

[ I've changed the message subject and omitted the irrelevant quote from
a previously discussed topic. Please learn some netiquette. ]

This mailing list exists primarily for sharing experience with using
John the Ripper, and for asking and answering reasonable (preferably
advanced) questions. Unfortunately, your question, as formulated above,
cannot be answered without further clarifications (what hash type?)

The generic answer, though, is that John the Ripper with no unofficial
patches does not support any on-the-wire-only hash types (e.g., those
used in various challenge-response protocols). If you happen to capture
"regular" password hashes, such as those normally stored on a server,
those might be supported. Otherwise, you need patches, which might not
be readily available for a particular hash type.