Discussion:
[john-users] john --make-charset=custom.chr: Can't get the hang of using it. :-(
Eric Watson
2018-05-22 20:14:57 UTC
Permalink
Hi,

I have a .txt file containing a few characters from which a password was
made. I have the hash of the password.

I use the command:

./john --make-charset=custom.chr mypasswd.txt

where "mypasswd.txt" contains characters (AbCdEf)

I get the error:

Loaded 0 plaintexts, exiting...

Looks like I could use a manual! However,I am told that one does not
exist. I will create my own, step by step :-)

Please assist in using that john command. What I read seems to relate to
password lists:

From john examples:

john --make-charset=custom.chr passwd1 passwd2
[ Configure your custom "incremental" mode now. See below. ]
john -i=custom passwd3


Where does passwd3 appear from?

Eric
Solar Designer
2018-05-23 09:02:43 UTC
Permalink
This post might be inappropriate. Click to display it.
Rich Rumble
2018-05-23 14:02:57 UTC
Permalink
Post by Solar Designer
Hi Eric,
Post by Eric Watson
I have a .txt file containing a few characters from which a password was
made. I have the hash of the password.
./john --make-charset=custom.chr mypasswd.txt
where "mypasswd.txt" contains characters (AbCdEf)
Loaded 0 plaintexts, exiting...
The intended use for the "--make-charset" option is to process whatever
passwords you have already cracked in order to optimize further attacks.
The cracked passwords are read from john.pot. When you also list any
"password files" on the command line, (1) those must be of one of the
usual formats that JtR normally reads for cracking (that is, they should
contain password hashes, as well as possibly other fields) and (2) they
are only used to filter john.pot contents. In other words, you specify
them along with "--make-charset=custom.chr" only in order to limit the
resulting contents of custom.chr to overlap of what's in john.pot
(hashes and plaintexts) and what's in the specified files (hashes only).
Your use is unintended. You may, however, achieve what you want by
echo :AbCdEf > john.pot
./john --make-charset=custom.chr
Please note that incremental mode cares not only about the character
set, but also about password lengths, character positions, character
frequencies given specific up to two preceding characters. So in the
above example, it will generate the specific string AbCdEf first (if
you allow it to generate candidate passwords of length 6 at all, and
don't apply any other restrictions).
You might want to use mask mode instead, which is intended use and is
./john -2='AbCdEf' -mask='?2?2?2?2' mypasswd.hash
This attacks your password hash directly, without generating any
intermediate charset file.
Post by Eric Watson
Looks like I could use a manual! However,I am told that one does not
exist. I will create my own, step by step :-)
Where are you told that a manual does not exist?
Post by Eric Watson
Please assist in using that john command. What I read seems to relate to
john --make-charset=custom.chr passwd1 passwd2
[ Configure your custom "incremental" mode now. See below. ]
john -i=custom passwd3
Where does passwd3 appear from?
All of the passwd* files in this example are expected to contain
password hashes. passwd1 and passwd2 contain hashes that you already
have some passwords cracked for (they're in john.pot), and you use these
files for filtering your john.pot contents (in case it also contains
cracked passwords for unrelated hashes). passwd3 is the password hash
file that you intend to crack.
"If you've got a password file for which you already have a lot of
passwords cracked or obtained by other means, and the passwords are
unusual, then you may want to generate a new charset file, based on
character frequencies from that password file only"
Then it proceeds to give examples for one such file and eventually for
multiple related files (the example you quoted here). Perhaps we need
to clarify these examples with a mention that cracked passwords are read
from john.pot.
Alexander
Other helpful resources in addition to the DOC
<https://github.com/magnumripper/JohnTheRipper/tree/bleeding-jumbo/doc>
folder are: http://openwall.info/wiki/john
http://openwall.info/wiki/john/tutorials
One I'm fond of: https://xinn.org/blog/JtR-AD-Password-Auditing.html (Skip
down to "Using Jtr")
-rich
Eric Watson
2018-05-23 14:48:19 UTC
Permalink
Thank you Alexander for that detailed advice.

I read somewhere in the john documents that there was a manual in
another group such as raspberry Pi but the advice was not to read it but
to use the documents instead. If you wish I will post the location
if/when I come across it again.

May I continue with this query?

The password in question contains one or two numerical strings of known
length and known numerals. It also contains words of known length and
characters. It also contains two *, ! characters.

The words may be capitalised and in any position as the numerical
strings could also be.

I tried the method, (actually just before receiving you reply):

echo :AbCdEf > john.pot
./john --make-charset=custom.chr


It resulted in the numbers being treated individually which made me ask
about a 'group' set.

What method would you suggest in this case?

Eric
Post by Solar Designer
Hi Eric,
Post by Eric Watson
I have a .txt file containing a few characters from which a password was
made. I have the hash of the password.
./john --make-charset=custom.chr mypasswd.txt
where "mypasswd.txt" contains characters (AbCdEf)
Loaded 0 plaintexts, exiting...
The intended use for the "--make-charset" option is to process whatever
passwords you have already cracked in order to optimize further attacks.
The cracked passwords are read from john.pot. When you also list any
"password files" on the command line, (1) those must be of one of the
usual formats that JtR normally reads for cracking (that is, they should
contain password hashes, as well as possibly other fields) and (2) they
are only used to filter john.pot contents. In other words, you specify
them along with "--make-charset=custom.chr" only in order to limit the
resulting contents of custom.chr to overlap of what's in john.pot
(hashes and plaintexts) and what's in the specified files (hashes only).
Your use is unintended. You may, however, achieve what you want by
echo :AbCdEf > john.pot
./john --make-charset=custom.chr
Please note that incremental mode cares not only about the character
set, but also about password lengths, character positions, character
frequencies given specific up to two preceding characters. So in the
above example, it will generate the specific string AbCdEf first (if
you allow it to generate candidate passwords of length 6 at all, and
don't apply any other restrictions).
You might want to use mask mode instead, which is intended use and is
./john -2='AbCdEf' -mask='?2?2?2?2' mypasswd.hash
This attacks your password hash directly, without generating any
intermediate charset file.
Post by Eric Watson
Looks like I could use a manual! However,I am told that one does not
exist. I will create my own, step by step :-)
Where are you told that a manual does not exist?
Post by Eric Watson
Please assist in using that john command. What I read seems to relate to
john --make-charset=custom.chr passwd1 passwd2
[ Configure your custom "incremental" mode now. See below. ]
john -i=custom passwd3
Where does passwd3 appear from?
All of the passwd* files in this example are expected to contain
password hashes. passwd1 and passwd2 contain hashes that you already
have some passwords cracked for (they're in john.pot), and you use these
files for filtering your john.pot contents (in case it also contains
cracked passwords for unrelated hashes). passwd3 is the password hash
file that you intend to crack.
"If you've got a password file for which you already have a lot of
passwords cracked or obtained by other means, and the passwords are
unusual, then you may want to generate a new charset file, based on
character frequencies from that password file only"
Then it proceeds to give examples for one such file and eventually for
multiple related files (the example you quoted here). Perhaps we need
to clarify these examples with a mention that cracked passwords are read
from john.pot.
Alexander
Solar Designer
2018-05-23 15:03:19 UTC
Permalink
Post by Eric Watson
I read somewhere in the john documents that there was a manual in
another group such as raspberry Pi but the advice was not to read it but
to use the documents instead. If you wish I will post the location
if/when I come across it again.
Oh, you probably mean my own recent comments about the Debian man page.
Yes, I recommend reading our documentation under doc/ instead of that
unofficial man page.
Post by Eric Watson
May I continue with this query?
Sure.
Post by Eric Watson
The password in question contains one or two numerical strings of known
length and known numerals. It also contains words of known length and
characters. It also contains two *, ! characters.
The words may be capitalised and in any position as the numerical
strings could also be.
If you can arrive at a reasonably small number of different masks that
represent your possible passwords, then I recommend that you use mask
mode (run it multiple times with the different masks). For example:

./john --mask='[Ff]irstword197[0-4][Ss]econdword123[*!][*!]' passwd

and so on for other word orderings, etc.
Post by Eric Watson
echo :AbCdEf > john.pot
./john --make-charset=custom.chr
It resulted in the numbers being treated individually which made me ask
about a 'group' set.
You may also try training incremental mode on multiple samples similar
to your password - not on mere lists of characters.

Alexander
Eric Watson
2018-05-24 10:46:02 UTC
Permalink
Hi,

Can I comment on your reply?

But first to mention that one of the links supplied by "rich" does not work.

https://xinn.org/blog/JtR-AD-Password-Auditing.html

Alexander, you mention, "training" incremental method. Can you explain
"training" please?

I find it confusing until it is pointed out that "wordlist" and "passwd"
are not in fact words but instead are hashes. I must remember that
everything revolves around hashes.

I appreciate that users such as I are "jumping in" with only basic
knowledge whereas most users appear to be very familiar with terms and
probably attended schooling.

Not a criticism just a comment to explain my apparent ignorance.

Eric
Post by Solar Designer
Post by Eric Watson
I read somewhere in the john documents that there was a manual in
another group such as raspberry Pi but the advice was not to read it but
to use the documents instead. If you wish I will post the location
if/when I come across it again.
Oh, you probably mean my own recent comments about the Debian man page.
Yes, I recommend reading our documentation under doc/ instead of that
unofficial man page.
Post by Eric Watson
May I continue with this query?
Sure.
Post by Eric Watson
The password in question contains one or two numerical strings of known
length and known numerals. It also contains words of known length and
characters. It also contains two *, ! characters.
The words may be capitalised and in any position as the numerical
strings could also be.
If you can arrive at a reasonably small number of different masks that
represent your possible passwords, then I recommend that you use mask
./john --mask='[Ff]irstword197[0-4][Ss]econdword123[*!][*!]' passwd
and so on for other word orderings, etc.
Post by Eric Watson
echo :AbCdEf > john.pot
./john --make-charset=custom.chr
It resulted in the numbers being treated individually which made me ask
about a 'group' set.
You may also try training incremental mode on multiple samples similar
to your password - not on mere lists of characters.
Alexander
Solar Designer
2018-05-25 17:39:51 UTC
Permalink
Post by Eric Watson
But first to mention that one of the links supplied by "rich" does not work.
https://xinn.org/blog/JtR-AD-Password-Auditing.html
Works for me. I think this is Rich's own website. For him to be able
to correct whatever problem there might be, you'd need to explain how
exactly this link "does not work" for you and maybe in what web browser.
Post by Eric Watson
Alexander, you mention, "training" incremental method. Can you explain
"training" please?
As doc/MODES says, incremental "mode deals with trigraph frequencies,
separately for each character position and for each password length,
to crack as many passwords as possible within a limited time." It
obtains those frequencies when you "train" it on previously-cracked
passwords, using the "--make-charset" option.
Post by Eric Watson
I find it confusing until it is pointed out that "wordlist" and "passwd"
are not in fact words but instead are hashes. I must remember that
everything revolves around hashes.
I don't know what mention of "wordlist" you refer to. I think there's
no such confusion about "wordlist". When we say "wordlist", we mean a
list of words or other strings to base candidate passwords on. We do
not mean hashes.

Yes, "passwd" traditionally refers to a file with hashes - that's due to
Unix having password hashes in a file called /etc/passwd.

Alexander
Eric Watson
2018-05-25 18:23:35 UTC
Permalink
I am using Xubuntu, Opera browser via a VPN.

'Does not work' =


This site can’t be reached

*xinn.org*took too long to respond.


Thank you for your succinct reply my questions.

Eric
Post by Solar Designer
Post by Eric Watson
But first to mention that one of the links supplied by "rich" does not work.
https://xinn.org/blog/JtR-AD-Password-Auditing.html
Works for me. I think this is Rich's own website. For him to be able
to correct whatever problem there might be, you'd need to explain how
exactly this link "does not work" for you and maybe in what web browser.
Post by Eric Watson
Alexander, you mention, "training" incremental method. Can you explain
"training" please?
As doc/MODES says, incremental "mode deals with trigraph frequencies,
separately for each character position and for each password length,
to crack as many passwords as possible within a limited time." It
obtains those frequencies when you "train" it on previously-cracked
passwords, using the "--make-charset" option.
Post by Eric Watson
I find it confusing until it is pointed out that "wordlist" and "passwd"
are not in fact words but instead are hashes. I must remember that
everything revolves around hashes.
I don't know what mention of "wordlist" you refer to. I think there's
no such confusion about "wordlist". When we say "wordlist", we mean a
list of words or other strings to base candidate passwords on. We do
not mean hashes.
Yes, "passwd" traditionally refers to a file with hashes - that's due to
Unix having password hashes in a file called /etc/passwd.
Alexander
Eric Watson
2018-05-27 00:28:13 UTC
Permalink
I am struggling with making a suitable mask having tried numerous attempts.

I am trying to achieve:

./john --mask'[*!][Ff]irrstword followed by a known number such as 11334
but could also be 43311 followed by [*!][Ss]econdword followed by 11334
or 43311' passwd

The number digits would not change and would always be in the same order
following the first and second word so could be treated as a "word".

The firstwordnumber and secondwordnumber could also be reversed ie.

./john --mask'[*!][Ss]econdword followed by 11334 or 43311
[*!][Ff]irrstword followed by 11334 or 43311' passwd

I assume this would need two passes, one for each combination?

Can you assist please?

Eric
Post by Solar Designer
Post by Eric Watson
I read somewhere in the john documents that there was a manual in
another group such as raspberry Pi but the advice was not to read it but
to use the documents instead. If you wish I will post the location
if/when I come across it again.
Oh, you probably mean my own recent comments about the Debian man page.
Yes, I recommend reading our documentation under doc/ instead of that
unofficial man page.
Post by Eric Watson
May I continue with this query?
Sure.
Post by Eric Watson
The password in question contains one or two numerical strings of known
length and known numerals. It also contains words of known length and
characters. It also contains two *, ! characters.
The words may be capitalised and in any position as the numerical
strings could also be.
If you can arrive at a reasonably small number of different masks that
represent your possible passwords, then I recommend that you use mask
./john --mask='[Ff]irstword197[0-4][Ss]econdword123[*!][*!]' passwd
and so on for other word orderings, etc.
Post by Eric Watson
echo :AbCdEf > john.pot
./john --make-charset=custom.chr
It resulted in the numbers being treated individually which made me ask
about a 'group' set.
You may also try training incremental mode on multiple samples similar
to your password - not on mere lists of characters.
Alexander
Solar Designer
2018-05-27 10:12:34 UTC
Permalink
Post by Eric Watson
I am struggling with making a suitable mask having tried numerous attempts.
./john --mask'[*!][Ff]irrstword followed by a known number such as 11334
but could also be 43311 followed by [*!][Ss]econdword followed by 11334
or 43311' passwd
The number digits would not change and would always be in the same order
following the first and second word so could be treated as a "word".
The firstwordnumber and secondwordnumber could also be reversed ie.
./john --mask'[*!][Ss]econdword followed by 11334 or 43311
[*!][Ff]irrstword followed by 11334 or 43311' passwd
I assume this would need two passes, one for each combination?
If I understood you correctly, it's these 8 masks:

[*!][Ff]irstword11334[*!][Ss]econdword11334
[*!][Ff]irstword11334[*!][Ss]econdword43311
[*!][Ff]irstword43311[*!][Ss]econdword11334
[*!][Ff]irstword43311[*!][Ss]econdword43311
[*!][Ss]econdword11334[*!][Ff]irstword11334
[*!][Ss]econdword11334[*!][Ff]irstword43311
[*!][Ss]econdword43311[*!][Ff]irstword11334
[*!][Ss]econdword43311[*!][Ff]irstword43311

To use them with JtR, you can use this lengthy command-line in bash:

while read mask; do ./john --mask="$mask" passwd; done < masks.txt

or you can do the 8 invocations of JtR manually, substituting the
different masks in there (if so, use single quotes around the masks to
prevent the shell's processing of special characters in there).

You can also choose to optimize for fewer masks, given that the number
of different candidate passwords to test is low anyway (let the computer
waste some time, instead of wasting your own time):

./john --mask='[*!][Ff]irstword[14][13]3[31][41][*!][Ss]econdword[14][13]3[31][41]' passwd
./john --mask='[*!][Ss]econdword[14][13]3[31][41][*!][Ff]irstword[14][13]3[31][41]' passwd

This will test a total of 8192 candidate passwords instead of only 128,
but either is a low number for a computer.

Alexander
Eric Watson
2018-05-27 13:08:58 UTC
Permalink
Thank you Alexander.

Can I ask for guidance on another method?

Incremental mode excluding a list of characters which would not be
included in the password?

Eric
Post by Solar Designer
Post by Eric Watson
I am struggling with making a suitable mask having tried numerous attempts.
./john --mask'[*!][Ff]irrstword followed by a known number such as 11334
but could also be 43311 followed by [*!][Ss]econdword followed by 11334
or 43311' passwd
The number digits would not change and would always be in the same order
following the first and second word so could be treated as a "word".
The firstwordnumber and secondwordnumber could also be reversed ie.
./john --mask'[*!][Ss]econdword followed by 11334 or 43311
[*!][Ff]irrstword followed by 11334 or 43311' passwd
I assume this would need two passes, one for each combination?
[*!][Ff]irstword11334[*!][Ss]econdword11334
[*!][Ff]irstword11334[*!][Ss]econdword43311
[*!][Ff]irstword43311[*!][Ss]econdword11334
[*!][Ff]irstword43311[*!][Ss]econdword43311
[*!][Ss]econdword11334[*!][Ff]irstword11334
[*!][Ss]econdword11334[*!][Ff]irstword43311
[*!][Ss]econdword43311[*!][Ff]irstword11334
[*!][Ss]econdword43311[*!][Ff]irstword43311
while read mask; do ./john --mask="$mask" passwd; done < masks.txt
or you can do the 8 invocations of JtR manually, substituting the
different masks in there (if so, use single quotes around the masks to
prevent the shell's processing of special characters in there).
You can also choose to optimize for fewer masks, given that the number
of different candidate passwords to test is low anyway (let the computer
./john --mask='[*!][Ff]irstword[14][13]3[31][41][*!][Ss]econdword[14][13]3[31][41]' passwd
./john --mask='[*!][Ss]econdword[14][13]3[31][41][*!][Ff]irstword[14][13]3[31][41]' passwd
This will test a total of 8192 candidate passwords instead of only 128,
but either is a low number for a computer.
Alexander
Solar Designer
2018-05-27 13:25:31 UTC
Permalink
Post by Eric Watson
Incremental mode excluding a list of characters which would not be
included in the password?
It's probably a poor fit for what you described so far, but in general
yes you can generate a custom incremental mode .chr file using "cracked
passwords" that never contain certain characters (the "excluded" ones),
and the passwords that incremental mode will generate using that file
will also never include those characters (except if you use "Extra =" in
the corresponding john.conf section, which in this case you won't).

You can do something like this:

$ cat > masks.txt << EOF
[*!][Ff]irstword11334[*!][Ss]econdword11334
[*!][Ff]irstword11334[*!][Ss]econdword43311
[*!][Ff]irstword43311[*!][Ss]econdword11334
[*!][Ff]irstword43311[*!][Ss]econdword43311
[*!][Ss]econdword11334[*!][Ff]irstword11334
[*!][Ss]econdword11334[*!][Ff]irstword43311
[*!][Ss]econdword43311[*!][Ff]irstword11334
[*!][Ss]econdword43311[*!][Ff]irstword43311
EOF
$ while read mask; do ./john --mask="$mask" --stdout; done < masks.txt | sed 's/^/:/' > john.pot
$ ./john --make-charset=custom.chr
$ ./john --incremental=custom passwd

where "$" denotes your shell prompt (so that you see that the masks and
the EOF line are input to "cat" and not to the shell).

Alexander

Eric Oyen
2018-05-23 14:47:07 UTC
Permalink
mine is a slightly different issue. I run the command as stated in the example (./john --make-charset custom.chr custom.txt) where custom.txt happens to be a crunch created wordlist for letters, numbers and only 2 special chars (exclamation point and period).

the command runs and nothing else happens (it just sits there on a blank line).

so, what gives?

-eric


PGP fingerprint: 6DFB D6B0 3771 90F1 373E 570C 7EA2 1FF3 6B68 0386
Hi,
I have a .txt file containing a few characters from which a password was made. I have the hash of the password.
./john --make-charset=custom.chr mypasswd.txt
where "mypasswd.txt" contains characters (AbCdEf)
Loaded 0 plaintexts, exiting...
Looks like I could use a manual! However,I am told that one does not exist. I will create my own, step by step :-)
john --make-charset=custom.chr passwd1 passwd2
[ Configure your custom "incremental" mode now. See below. ]
john -i=custom passwd3
Where does passwd3 appear from?
Eric
Solar Designer
2018-05-23 15:10:48 UTC
Permalink
This post might be inappropriate. Click to display it.
Eric Oyen
2018-05-23 18:04:53 UTC
Permalink
ah, ok.
well, I did some further reading. I am simply going to do a wordlist based session with rules mangling in play. It's a bit slower, but I am reasonably sure that my password wil show up eventually (and not in 13.75 billion years either!). :)

-eric

PGP fingerprint: 6DFB D6B0 3771 90F1 373E 570C 7EA2 1FF3 6B68 0386
Post by Solar Designer
Post by Eric Oyen
I run the command as stated in the example (./john --make-charset custom.chr custom.txt) where custom.txt happens to be a crunch created wordlist
As I explained in my reply to the other Eric, this is not what the
example is about and is not what it says. "--make-charset" was
originally intended for processing your previously-cracked passwords to
train incremental mode on. It will not read a wordlist. You can,
sed 's/^/:/' < custom.txt > john.pot
./john --make-charset=custom.chr
But you should probably simply use mask mode instead of this hack around
crunch and incremental mode. Mask mode is intended to do just what you
seem to be trying to do.
I am unaware of any use case for crunch along with JtR given what JtR's
built-in mask mode is capable of.
Alexander
Loading...