The following changes were made between 1.8.0-jumbo-1 and 1.9.

0-jumbo-1:

magnum (2623):
inc.c: Instead of sending a "" to crk_process_key(), use a safe static
buffer of PLAINTEXT_BUFFER_SIZE. No problem seen here but in other
places with similar code.
Move lastpass format back in tree. See #890
OpenCL: Support reporting OpenCL 2.0 versions.
OpenCL: Pass DEV_VER_MAJOR and DEV_VER_MINOR to OpenCL compiler. Eg.
for Catalyst 1234.5 they will be 1234 and 5 respectively. For
strings like "text 123 text 56" they will be 123 and 56. Closes
#909.
DEScrypt OpenCL: Tweak the selection of "fast" versus "safe" version
of the kernel. The selection logic will very likely grow over time.
OpenCL: If kernel build fails, force outputting build options.
OpenCL: AMD devices have some weird bug when using pinned buffers.
They seem to allocate twice the memory. We need to account for this
when finding out limits.
Speed up some --list stuff with OpenCL formats. See #919.
Even more speedup for --list stuff with OpenCL formats. See #919.
Tweak for --list=format-[all-]details and OMP formats. See #919.
OMP tuning and some other performance fixes. See #877.
AC: Fix ASAN configuration. Closes #921.
AC: More fixes to ASAN configuration. #921.
Fix a problem with --list=format-tests and Unicode formats. Closes
#919
DEScrypt OpenCL: Tweak the selection of "fast" versus "safe" version
of the kernel. The __Tahiti__ macro is not all upper case, as I had
it. And only nvidia GM204 seem to be faster using the "safe" kernel
- Kepler and Fermi wants the other.
Vectorized OpenCL formats: Correction for max memory use.
Vectorized OpenCL formats: Correction for max memory use.
raw-sha256 bug fix. It didn't unify case for all hashes. Closes #925,
#926, #927, #928, #931.
OpenCL: Fix a bad bug with multi-field test vectors. Found by ASAN.
OpenCL: Auto-tune max. size calc. fix for vectorized formats (RAKP and
NTLMv2). See #922
Office2013 OpenCL: Disable vectorizing on VLIW5 because the register
pressure is too high. Discussion in #908.
AC: Add -D__USE_MINGW_ANSI_STDIO for MinGW.
Office2013 OpenCL: Allow forcing vectorizing.
Fix a minor problem in AESNI Makefile.
RAKP OpenCL: On VLIW5, limit vectorizing to 2x due to GPR pressure.
See #908.
Various OpenCL formats: On VLIW5, limit vectorizing to 2x due to GPR
pressure. See #908.
Oops I committed the define of JTR_RELASE_BUILD into bleeding.
Makefile: Fix a typo in luks2john's LDFLAGS. See #933
jumbo.c: Pure whitespace fixes.
OpenCL: Drop some obsolete code. Fix a lot of formats for using LWS=0
(auto tune LWS) or LWS=9999 (set to ceiling instead of forcing auto
tune). Also plenty other minor tweaks and fixes.
Change the $HEX$ loader bodge to use strncmp() instead of strstr(),
for performance.
Drop OCL_CONFIG in favor of FORMAT_LABEL. It will be stripped to it's
base and lower-cased, eg. rar-opencl -> rar, so the end result is
supposedly the same (or bugs got fixed automagically, which is the
very intention with this). Closes #940.
Drop a couple of old formats from the unused/ directory. That
 directory is not a permanent graveyard - Git is.
Drop a couple of old Unicode "original" files from unused. We do not
need them - the licensing information is stated in the files we
actually use.
Fix a problem with opencl_cryptsha512_fmt after my recent changes.
Drop no-op code from Truecrypt. See #941.
Fix indentation and widescreen comments in memory.c
raw-sha1-linkedin bodge for loader.c. Closes #924.
RAKP-opencl, fix a problem with uninitialized index buffer when used
with -skip. Closes #947.
mscash-cuda. Update valid() and prepare() from CPU format. Closes #942
sha256crypt-cuda. Use shared valid() and binary() functions. Closes
#943.
sha512crypt-cuda. Use shared valid() and binary() functions. Closes
#944.
Revert an incorrect fix in formats.c from faf28f43. You must not call
clear_keys() in the middle of a batch of keys!
Revert an incorrect fix in mssql from faf28f43.
Put workaround back in RSVP since we reverted the fix in formats.c
Bugfix for xsha512-cuda found by ASAN.
Add more test vectors to CUDA formats that only had one.
Encfs-opencl and sxc-opencl: Add test vectors from CPU format.
OpenCL keyring format: Add second test vector from CPU format. This
immediately made a bad bug show up, which was fixed too.
Revert "loader: incresed number of parms being loaded from file, for
scrypt"
pwsafe-opencl: Reject uppercase in valid().
Drop 'test' but keep .gitmodules. Experimental.
Drop some unused code from Keccak and Blake.
Fix some header inclusions that should be <brackets> but was "quotes"
Add an option to configure (--enable-plugin-dependencies) that creates
plugin dependencies in Makefile. Highly experimental and
best-effort. See #955.
Add a self-test for salt() ensuring it returns same data for same
salt. See #953.
Base64_convert code: Some off-by-one and buffer cleaning problems
fixed. See #953
Fix some more header inclusions that should be <brackets> but were
"quotes".
Add Makefile dependencies for non-plugins, "best effort".
Default to enable plugin dependencies generation.
Add a debug print to formats.c salt dupe check.
PFX bugfix. We were clearing salt after setting things in it. This
does not fix #953 - it appears we really have to use a dynamic salt.
Change wording of self-test warning for dupe salt detection. There's
not really any doubt it's broken.
OpenCL DES: Use an LWS of 8 for Intel, for good measure (they
recommend it in their white papers). Closes #958.
Revert 53442ed and 123a654. They were never needed. See #958
Allow environment variable OMP_SCALE to override john.conf setting
FormatBlockScaleTuneMultiplier. And clean up code.
luks2john: Add missing LF in error output.
Fix bugs in LUKS format. Some optimist had changed things without
checking results of it. We were freeing memory that was not
alloc'ed, and non-inlined hash handling was terrificly b0rken in
valid().
Add autoconfig-stamp-h.in to .gitignore
DEScrypt-opencl: Add a kludge for older buggy Intel CPU driver.
Double entry in .gitignore after mid-air collision. Removed.
Make base64_convert less Windows-ish by lifting the limit of 50K max.
number of lines read from stdin. Fix some bugs while at it. The max
line length is now 256K as stated (it was coded as 256M). FWIW our
current LINE_BUFFER_SIZE is less at 0x30000.
base64_convert: Put a line length limit notice back in usage blob.
Fix edge-case issues with enc_lc() and enc_uc(). The destination size
is now buffer size, not string length.
Fix a good bunch of formats that did not work with the new OMP_SCALE
test/devel stuff. See #877.
Fix some more formats that did not work with the new OMP_SCALE
test/devel stuff. Plus some no-ops for unified code. See #877.
Revert CP737 to what the Unicode database says.
Manual sync of libclamav unrar code from upstream. This did not fix
#1000.
Fix an ASAN problem in unrar.c where a read ahead buffer could read
past end of buffer. Closes #1000.
Fix a bug in -list=format-details where multi-field ciphertexts were
not printed. Also drop some for loops in -list=format-tests in favor
of strchr().
unrar.c, bump over-alloc more for protection against bad data. See
#1000 and closes #1001.
Document how to crack CRAM-MD5 using the HMAC-MD5 format. Closes #999.
Add salt_compare method and plaintext_min_length param to format
struct. Closes 996, closes #997.
Add --list info for new format struct members. Closes #1004.
unrar.c: Implement a similar protection for rar_getbits() that Jim did
in rarvm_getbits(). Unfortunately this did not help #1008.
Wordlist mode had no consecutive dupe suppression when running
unbuffered and without rules. This reverts 15949eb and fixes it in a
better way. Closes #1012.
Reduce some chatter when -verbosity is decreased.
If verbosity is increased, report tunable cost even when all hashes
have same cost. This was suggested in #816.
Allow -cost with -test. See #816.
Clean up cost-with-test code, see #816.
Cost-with-test: Added printing of costs used in test. Closes #816.
Cost-with-test: Changed message for costs used in test. See #816.
Benchmark w/ -cost issues. Closes #816. - Never output cost
information w/ --test=0. - Also mute it if verbosity was decreased
from default. - Give a better error message when -cost pruned all
test vectors. - Fail with a proper notice if a required cost is not
defined.
Import pp.c (as prince.c) from Atom's Princeprocessor and implement a
prince mode with it. See #1011.
Add means for setting PRINCE parameters. Also add logging of them. See
#1011.
PRINCE: Add ETA/progress. See #1011.
PRINCE: Minor tweaks. See #1011.
PRINCE: Implement session save/restore. See #1011.
PRINCE: Minor tweaks. See #1011.
PRINCE: Fix a problem where we broke out of inner loop but not the
outer. See #1014.
PRINCE: Fix another problem where we broke out of inner loop but not
the outer. Closes #1014.
Fix a potential "ASan problem" in mask mode.
PRINCE: Always report DONE when everything is cracked. This is not
necessarily a good idea (will open another issue) but all other
modes do so.
PRINCE: Support node/fork. See #1011.
PRINCE: Better node/fork distribution. See #1011.
PRINCE: Fix resume problem. Closes #1016.
PRINCE: Cosmetic & performance resume tweaks. See #1016.
Drop PRINCE from usage blob and other stuff, unless GMP is available.
PRINCE: Sync pp.c from Atom's repo.
PRINCE: Add proper GMP cleanup for JtR-specific variables.
PRINCE: Optionally define PRINCE-specific wordlist in john.conf.
Autoconf: Add a PRINCE notice to summary (for GMP library).
PRINCE: Mute a compiler warning.
Add a header in pkzip_plug.c muting a compiler warning.
Sync upstream PRINCE changes. Major improvement for resuming.
PRINCE: Much faster node distribution - but distributes poorly for
very small keyspaces.
PRINCE: Sync prince.c from upstream (d9bbf908) pp.c.
PRINCE: Sync prince.c from upstream (be309320) pp.c (major bugfix).
Autoconf: Check whether compiler supports some int128 type(s).
PRINCE: Prefer uint128_t over GMP if available, for a 44% boost. This
only works with 64-bit builds and not with all compilers (it should
work with gcc 3.1 and above). Otherwise fall back to GMP.
PRINCE: Support 128-bit integer for legacy Makefile builds.
PRINCE: Fix a bug with 128-bit restore.
Show default parameters for PRINCE mode in usage blob.
PRINCE: Bugfix for legacy builds.
Add instructions for working around bugs in the system headers of OS X
Yosemite. See #1025.
PRINCE: Merge upstream.
OSX: Backup originals if patching system headers o.O
Mask mode (pure): Fix progress/ETA for node/fork runs. Closes #1024.
PRINCE: Drop the inner-loop node distribution alternative, plus some
error output tweaks.
PRINCE: Update from upstream.
PRINCE: Ensure session store/resume compatibility between GMP and
non-GMP versions. And fix input/output of large numbers in the
int128 version. Also fix what I hope is a bug upstream (else we got
a problem).
PRINCE: Update from upstream. 6.4x boost. Drop the int128 version for
now (WIP).
PRINCE: Add options --prince-skip=N and --prince-limit=N.
PRINCE: Add int128 alternative to GMP again, this time using a header
with nefarious macros.
Re-add AC status output for int128 PRINCE.
PRINCE: Add --prince-keyspace option. Also add --disable-int128 to
autoconf.
mpz_int128.h: Support base 10 and 16 for mpz_set_str()
PRINCE: Show "bits used" when printing keyspace. Also revert a bug
workaround that should no longer be needed (div by zero while
skipping).
Add unrule.pl embryo. This does wot epixoip said, plus removes dupes
and sorts by count. Reduces rockyou.txt by 60% in 30 seconds. See
#998.
mpz_int128.h: Add overflow checks, saturating to INT128_MAX (and to
zero for subtraction)
mpz_int128.h: Minor tweaks.
blf.h: Use "stdint.h" and not <stdint.h>
Sync pp.c from upstream.
Add a warning about saturation in mpz_int128.h in case someone tries
to use it for something other than PRINCE.
PRINCE: Fix a bug in JtR-specific code introduced when merging
upstream.
Drop OS X 'as' wrapper script in favor of -q option to native 'as'.
mpz_int128.h: Fix saturation checks for case where rop and op1 is the
same variable.
PRINCE: Update from upstream v 0.20.
PRINCE: Tweak parameter checking.
PRINCE: Fix (JtR specific) resume bug.
PRINCE: Update pp.c from upstream and add new -prince-case-permute
amplifier option to JtR.
PRINCE: Allow output lengths up to 125. Note that this mode currently
limits output length to 16 unless you use -max-len to bump it. For
most other use cases, -max-len is a limiter.
Print timestamp with --log-stderr.
PRINCE: Bug fixes for case when output length > max input length.
Jim's portability hack of pp.c was broken: * If -DJTR_MODE we don't
need getopt.h at all * if that macro is not defined we must not
depend on other object files.
PRINCE: Reduce memory use.
PRINCE: Fix a bug when output length > max. input length.
PRINCE: Fix output length bugs. We can't support longer output than 32
unless we switch to u64, but it will too slow anyway.
PRINCE: Fix a session bugs with node/fork. This is incompatible with
saved sessions from previous versions - there was no way around it.
MIPS tweaks for autoconf script. Suggested by Solar after tests.
PRINCE: Update from upstream.
AC: Add mips* back (and a new mips64*) for catching obscure target
triples.
WPAPSK: Better version of previous patch. See #1031
PRINCE: Add support for encodings. This does not affect performance
much, and only during initial wordlist load.
PRINCE: Add loopback mode. Note that we do not yet have dupe
suppression which is almost a must for loopback.
PRINCE: Add dupe-suppression.
PRINCE: Update from (sort of) upstream.
Some tweaks for --max-length option. Do not allow it to over-ride a
limited --stdout=N length, and bail with error if trying to use
larger than allowed for PRINCE mode (currently 32).
Change the inline error output from keepass format to reference the
issue for fixing it, and use error() instead of exit(). See #1023
and #1026.
Do not test for int128 unless we are really 64-bit or better.
PRINCE: Fix resume bug.
PRINCE: Update from upstream. Dupe-suppression is now default.
PRINCE: Yet again tweak session save/resume. It's hard to get
node/fork right.
Replace exit() calls with error() so we get a proper bailout with
logging.
PRINCE: Optionally memory-map the input file (not for loopback mode
nor when using case permutations). This is JtR-specific, the
original prince-processor still only supports loading from stdin.
Dupe suppression is no longer default except in loopback mode, good
old --dupe-suppression option does it otherwise.
Tweaks to the Yosemite patch, for not breaking clang builds.
Pacify a clang compiler warning in pp.c.
PRINCE: Add logging of memory use (JtR only).
PRINCE: Upstream bug fix, and some more fixes for dupe-suppression.
Comment out an unused variable in MD4 OpenCL kernel.
PRINCE: Revert a bogus change in last commit.
PRINCE: JtR-specific bugfix similar to the recently fixed one
upstream.
PRINCE: Automatically decrease elem-cnt-max to pw_max if applicable.
Drop an odd thing Frank accidentally(?) added to bash completion.
Fix bash completion for --prince=FILE. Closes #1020.
bash completion: Allow ':' instead of '=' for --prince=FILE. See
#1020.
MPI/fork: Mute dupe "slow loading" warning messages for NTLMv1.
PRINCE: Ensure session-save does not store a position we never
reached. Not sure there was a problem but better than sorry.
PRINCE: Add rules support. This is very slow right now because we run
each word through all rules instead of the other way round. Closes
#1019.
Add a format that is based on dummy but use no encoding at all.
ciphertext == plaintext. This is easier to use for some tests.
Some --list OpenCL tweaks for Apple.
Cosmetic --list CUDA tweak for cc 2.1.
Add a comment in plaintext format. This format is faster than dummy if
using same max lengths.
Drop some weird garbage from Makefile.in
Support -DSYSTEMWIDE using Autoconf --with-systemwide. Shared binary
directory defaults to /usr/libexec/john unless you use the --prefix
option to configure [which also implies --with-systemwide].
PRINCE: Updates from upstream.
logger.c: Only flush stdout if we're indeed running fork. It should
not be needed for MPI.
Minor tweak for SecureMode = Y.
Fix p/s counter for GPU generation.
PRINCE: Enable hybrid mask mode.
Various fixes for PRINCE hybrid mask mode.
Add a newly needed GMP function for emulation.
Some file locking tweaks: Ensure a LOCK_SH before pot sync was not
interrupted by a signal (timer). This should not matter much, if at
all. Also show whether locking is supported or not in
--list=build-info.
Add some file locking debug output for -DLOCK_DEBUG and also have it
imply -DPOTSYNC_DEBUG.
Add a flock-test script to unused.
Change a couple #ifdef POTSYNC_DEBUG to LOCK_DEBUG for consistency.
Simplify pot-reload file open. This should really not make any
difference in the end (we're doing the same things) but it looks
nicer and it no longer feels like we're missing a call to close()
(as opposed to fclose() which we do call).
Make it possible to *completely* disable pot syncing just by editing
john.conf. Also clarify some descriptions.
Plaintext format: Add a test vector including spaces, and some
comments.
PRINCE: Enable use of mmap with loopback, plus various bugfixes.
Fix some logging that happened before log_init() so never made it into
log.
Finish the alternative fcntl() file locking. After having consulted
Solar, we also make it the default.
Improved -DLOCK_DEBUG a little.
BFEgg is FMT_NOT_EXACT. See #1045.
BFEgg should have a max. password length of 72.
Drop FMT_NOT_EXACT from BFEgg again, and document (in source) the
optional per-run alternative --keep-guessing. See #1045.
Tweaks and a bug fix for fcntl locking.
Update some obsolete information in README.mpi.
Clarify description of FMT_NOT_EXACT in formats.h, and drop that flag
from CRC32 format. That format should be run with --keep-guessing
 option instead (if/when actually needed). Also drop some comments
from formats so they don't hit a "git grep FMT_NOT_EXACT" while not
actually having that flag.
Mute clang compiler warnings about unused variables (that we can't
drop cause we need the sizeof() them) in gpg2john.
Fix usage blob for --prince-mmap, a restriction was dropped.
The --stress-test option did not work very well with OpenMP formats
that have things like "self->params.max_keys_per_crypt *= omp_t"
since this would be evaluated before each test and figures would
soon get silly. So we save them and restore them. Also applies to
plaintext_length as that might be eg. "*= 3" for UTF-8.
Update doc/OPTIONS with some missing bits and pieces.
Work around a very strange MPI segfault when trying to resume a
session that does not exist.
Fix a bug that was introduced with the ShowUIDinCracks option, even
when not enabled. pwdump-format input files ended up showing random
garbage.
doc/OPTIONS, tweak description of --keep-guessing.
ShowUIDinCracks: Honor --field-sep-char.
CUDA md5crypt, clean salt.
Add salt_hash() to a couple of formats. See #809.
AC: Rename --disable-native-macro to --disable-native-march which is a
whole lot less confusing. Also ensure that --disable-native-tests
always implies --disable-native-march.
AC: All x86 cross-compile tests were busted and returned true for
anything.
Document a problem when using Ubuntu 12.04 with modern (AVX2)
hardware. Closes #1051.
PRINCE: Updates from upstream.
Always use mem_alloc(), or check result of malloc(), for good measure.
Our "mute ASAN" fix in bench.c affected benchmark speeds a whole lot
for fast formats. This mitigates it at least enough not to show
slower than real-life speed (compared to mask mode).
DES-opencl: Use mem_alloc wrapper.
Update Unicode external dumb/repeats modes for Unicode 7.0, and add
some codepages for possibly dealing with "Equation Group" hashes as
published by Kaspersky. All generated from the Unicode Database.
Update doc/ENCODINGS.
Fix bogus include paths (using quotes where there should be brackets).
Ensure Makefile dependencies for opencl_DES_hst_dev_shared.h
DEScrypt-opencl: Bugfix for dupe salts. The list of binaries for a
given salt may have NULL entries (already cracked).
OpenCL: Ensure that we get AMD-specific functionality even if JtR was
built with nvidia headers, and vice versa.
Add a few more vendor-specific OpenCL macros in case they are not
present in headers.
DEScrypt OpenCL: get rid of a warning about comparison of integers of
different signs.
Cosmetic tweak (on error) for fcntl locks.
OpenCL: Use locks when reading/writing cache files.
memdbg: Fix calloc bugs. Closes #1062.
DES-opencl bugfix. I got "Get key error! 50 1575" until I did this,
and with this fix it cracked a hash at that point.
Copy test vector plains to full-size buffers outside of hot loops so
we get a better raw speed benchmark figure. Closes #1054.
Add CRAM-MD5 support to HMAC-MD5 format, using the raw Base64 strings
prepended with a Hashcat-compatible tag. Closes #1018.
We've had an odd prototype for mem_calloc() ever since implemented in
36416c5. As much as I hate this no-op commit it makes it easier to
 import code in the future.
Fix regex.h for new path of rexgen header.
hmac-SHA384 bug fix. Closes #1063.
Fix a bug in hmac-MD5 and hmac-SHA1, and unify HMAC code.
Revert recent changes to HMAC formats and make the correct fixes.
Better fix for #1063 (the previous fixes were OK but used larger
buffer sizes than necessary).
Disable raw-sha1-linkedin if SHA1_SSE_PARA > 3. Closes #454.
Bug fix for edge cases in wordlist mode using mmap(). Hopefully
addresses #1025.
Bug fix (ASAN) in HMAC-MD5 for CRAM-MD5 conversion. See #1067
Add our own strnlen() unless system has one.
Add a self-test for catching buggy SIMD implementations using 0x80 as
end of data. 0x80 can be anything, for example a euro-sign in
CP1252. You can't ever use it for detecting end of data!
XSHA512: Fix a bug that would truncate any plaintext containing 0x80.
MSSQL12 SIMD support. There's some overhead in set_key() that's hard
to work around without writing totally unreadable code.
MSSQL12: KISS fixes for SIMD: Simpler buffer cleaning and use flat
buffers. Speed is about the same (or better) and code is much
simpler and potential bugs are squashed before they showed up.
Replace more deprecated des_ methods and calls with the DES_
counterparts.
Fix bogus line endings in cryptsha256_common.h (it had a disgusting
MIX of LF and CRLF).
sha256crypt: Allow - but truncate - salts longer than 16.
sha256crypt-cuda: Bump max length to 23. See #1077. This format
probably should be able to do 24, but then it loses some in the Test
Suite due to a bug somewhere.
Pending core change in cracker.c. Ensure get_key() is not called (via
status.c) after clear_keys() was called. Closes #1076.
Pending core change in cracker.c. Ensure get_key() is not called (via
status.c) after clear_keys() was called. Take 2. Closes #1076.
Drop a few empty clear_keys() functions and use fmt_default*
Revert "bcrypt / opencl: created a BF_common set of functions / data"
Enforce C89 for gcc-5.x (it defaults to C11). See #1093.
Notify user of ASan affecting speed if she tries to benchmark (we
already did for -DDEBUG). I have ended up chasing non- problems a
couple of times because I forgot I was running an ASAN build o.O
Move our strnlen() function to jumbo.c, where we have other such
functions. Basically, misc.c is for our "own" functions while
jumbo.c is for functions that many systems already have, but some
systems do not. Like in this case: We only implement the strnlen()
function unless it is available in libc.
Revert "siemens: only validate a 1. Since we now have split, a 0
should not be valid any more"
MSSQL12 bugfix for non-MMX/SSE2
Add a bunch of known mappings between OpenCL driver version and
Catalyst driver version (and name).
OpenCL, --list=opencl-devices for Catalyst (cosmetic)
Drop the "[CPU]" after OpenCL version number, it's redundant.
Revert a test vector in siemens S7 format to it's original format.
Cosmetical changes to OpenCL SHA512 formats: PLAINTEXT_LENGTH was used
in a confusing way. Change to match our conventions. These changes
are verified with TS for good measure.
Merge an optimization for HH() in md5.c originally commented as "Help
the compiler detect a common subexpression between steps in round
3."
MD5_std.c, md5_go.c, md4.c, NT and some GPU MD4/MD5: "common
 subexpressions in round 3" optimizations.
sse-intrinsics.c: Use _mm_set1_epi32(n) instead of
_mm_set_epi32(n,n,n,n) when applicable. This is 100% cosmetical.
Refactor MMX_COEF -> SIMD_COEF_32
MSSQL12: Avoid build warnings on non-SIMD
Add README-jumbo on top of README.md
OpenCL: More use of shared code.
Change mem_alloc_tiny() to malloc/free for large buffers in a bunch of
formats. This mitigates memory leaks when running --test with a huge
number of threads (eg. MIC/Xeon Phi) or with --stress-test. It
matters less for a real crack run. I picked some of most common
formats here plus the ones with largest OMP_SCALE figures for a
starter.
Dynamic: Use malloc/free for buffers. This fixes the real show
stoppers, but there's still plenty of memory leaking when running
eg. "../run/john -stress-test=0 -form:dynamic".
Use malloc/free for buffers; Another bunch of formats.
Mass refactoring of saved_key_length -> saved_len to ease scripted
changes.
Use malloc/free for buffers; Yet another bunch of formats.
bench.c now resets the Unicode system, formats need not care.
Revert a torrent of trigger-happy-commits and start using
posix_memalign() or aligned_alloc() for mem_alloc_align - drop the
extra pointers.
Waste a couple of cycles for each alloc just to mute the ridiculous
compiler warning "possibly uninitialized variable" - which is isn't.
Fix a couple of ASan crashes caused by erroneous $tag$*.
Avoid re-declaring _XOPEN_SOURCE (core code clash with autoconf)
If mem_alloc_align() fails, exit with message telling what size was
requested.
OpenCL fixes. More unifying of SHA1 ahead.
RAR5 OpenCL bugfix: Wrong "cost" (iteration count) reported due to
casting to incorrect (GPU side, should be core side) salt type.
OpenCL shared SHA1 and MD4/MD5 cleanup and bugfixes.
AMD bug workaround re-implemented in new shared SHA1 for OpenCL.
Fix OpenCL formats for strtokm(). See #1108.
Fix for that stupid AMD driver bug workaround for SHA1, for mixed
vector/scalar formats.
Use -Og for debug targets if compiler supports it. Closes #1113.
GPG-opencl failed due to changes half-assed merged from CPU format.
Closes #1119.
Change strok() to strtokm() in all formats but SKEY. See #1108.
It turns out a bug workaround for Catalyst 14.9 *triggers* an other
bug in older drivers... Doh. Addresses #908.
Change strok() to strtokm() in shared files that were missed. See
#1108.
PDF had a `strtok (` that didn't get replaced. Closes #1108 again.
OpenCL: Implement an optimized SHA512 for the case where half of the
block is zero (as in PBKDF2). Just a slight boost, I guess the
compiler did much of it already.
GPG: Needed to treat $gpg$* as the effective tag. And bump plaintext
size.
GPG: Bump max length to 125, and fix some code to adopt to
PLAINTEXT_LENGTH. The OpenCL format already behaved well in that
aspect but had some other non critical bugs leading to suboptimal
code. I also unified some code to ease comparing side by side.
GPG OpenCL: Alternate leaner code, trying to make it work on more
nvidias.
GPG OpenCL: Much leaner code. Still does not work on all nvidias.
GPG OpenCL: Disable leaner code for nvidias.
Disable "cost" support in CUDA sha2crypt formats. They were buggy and
not trivial to fix. Closes #1118.
Strip format, fix an ASan regression from 70f3b95a (I guess)
Cosmetic: Address clang warnings. Closes #1126.
Fix a "binary() returned misaligned pointer" in phpass/dynamic. See
#1127.
AC: protect a variable with quotes in some tests.
OpenCL raw-md4: Correct mask mode behavior when GWS is supplied. We'll
use the figure as-is but we still print a message about internal
generation.
Add some debug/benchmark stuff to raw-md4-opencl. For GCN and Maxwell,
I see no difference at all between different versions of the HH()
optimization - or not using at all.
OpenCL raw-MD5 GPU-side mask: Change GWS divisor to a more suiting
figure. This is significantly faster (on both GCN and Maxwell) but
still a compromise: Both auto-tunes to 4M, the HD7970 works best at
8K and Maxwell at 16K. We need to auto-tune *with* candidate mask!
SHA256 OpenCL: Mute complaint from picky compilers
pass_gen.pl: Do not list formats as supported when they are not.
Closes #1132.
mem_calloc_align had its parameters reversed. Not that it matters.
Change calloc_tiny to calloc/free in many formats. See #1110.
Fix a bug introduced in b9e13c38, closing #1139.
Change calloc_tiny to calloc/free in 14 formats. See #1110.
AC: Include CPPFLAGS when creating fmt_externs.h and fmt_registers.h
just in case it matters in some situation. Closes #1137.
Fixes for building "icc .S files". See #1065.
Update the icc pre-built see-intrinsics.S files, using the icc on
super: "icc version 14.0.0 (gcc version 4.4.7 compatibility)" Closes
#1065.
Fix a bug in MD5 introduced in 2e33cfd. Closes #1142.
Fix a copy/paste bug in DMD5. Closes #1143.
Most OpenCL formats: Auto-tune in reset().
Bugfix in o5logon-opencl.
Fix GPG-opencl for nvidias.
Optimizations for oldoffice-opencl that are also workarounds for some
AMD driver bugs.
OpenCL: The recent AMD change for PUTCHAR re-triggered the damn AMD
bug in oldoffice. Introduce a new macro for preferring the non-cast
alternative but default to the cast one.
Stop Lotus5-opencl auto-tune from overflowing on fast devices.
Lotus5-opencl, auto-tune limit.
README-OPENCL update.
Common OpenCL auto-tune: Temporary changes to handle auto-tune with
GPU-side mask. See #1136.
Common OpenCL code messed around with bench_running flag, this lead to
problems and should no longer be needed anyway.
mask.c: Allow actually using some other mask than options.mask
(possibly for auto-tune or benchmark).
Fix search/replace mistakes in comments in OpenCL MD4.
encoding_data.h: CP720 had (lack of) case conversion incorrectly
specified. Closes #1158.
Support fallback to older clang syntax for Address Sanitizer.
Fix a segfault when --users option resulted in no hashes loaded, and
an encoding was used. Closes #1164.
Prince format lacked some length checks. Closes #1163.
Fix a "theoretical" bug (ASan) in Prince. Closes #1162.
Fix alignment issues. Closes #1133.
Fix a bad bug introduced by f6af7d7: --show=left was broken for all
formats but dynamic. Closes #1130.
Refactor salt() -> get_salt() and binary -> get_binary() in most
(Jumbo) formats that did not already use those names.
Do not allow negative numbers for --mkpc. Closes #1166.
Prefer posix_memalign() over aligned_alloc() as a work-around for an
ASan shortcoming. See #1159.
Complete the previous patch: Prefer posix_memalign() over
aligned_alloc() as a work-around for an ASan shortcoming. See #1159.
Comply with the strange restriction of aligned_alloc() that "size
should be a multiple of alignment".
Less intrusive implementation of --mkpc. We do not alter format's view
of max_keys_per_crypt, just cracker.c's. Closes #1167.
--mkpc is now a no-op for --test so do not allow it.
Disallow some options unless we are running a cracking mode.
Handle --mkpc in an even safer way. This handles the situation where
max_keys_per_crypt changes during a run.
Raw-sha1-ng and shared intrinsics code incorrectly reported using AVX,
while they actually use at most SSE4.1 if no XOP.
Change the dump_stuff() debug functions so they don't spew out
warnings.
Add a pseudo_intrinsics.h function, for abstracting intrinsics from
the actual vector width.
raw-sha512-ng: Use pseudo-intrinsics.
raw-sha256-ng: Use pseudo-intrinsics.
raw-sha1-ng: Use pseudo intrinsics.
Q'n'D fix for a problem with cprepair.
vtestz was not available on Travis' clang.
Fix regression problems with clang from recent patches.
Drop an extraneous paren from pseudo-intrinsics header.
Fix a benign bug in raw-sha1-ng
Drop an unused variable in raw-sha256-ng.
Fix an issue with non-AVX SSE4.1 or SSSE3 systems Closes #1180.
Fix incorrect function declarations for cmp_all() and cmp_one()
Drop experimental code for OMP_SCALE, closing #1181.
When we do not request membdg, really do not let it interfere with
anything at all. See #1125.
Fix for cross-compiles on OS X, using SSE4.1 but not AVX.
Add a parameter to configure, --enable-experimental-code, that sets
USE_EXPERIMENTAL macro. This way we can merge incomplete features
but have them disabled unless user asks for them.
Make it possible to leave Dynamic format out entirely from a build.
And do so by default for AVX2, unless the option
--enable-experimental-code was given to configure.
Fix for the AC handling of AVX2 vs Dynamic.
Use pseudo-intrinsics for sse-intrinsics.c, making nearly all
remaining SIMD formats AVX2/AVX512 capable.
Various fixes for AVX2.
Fix NT2 format for AVX2.
Drop SIMD_COEF_BITS and just divide by SIMD_COEF instead. The
optimizer will change it to a bit shift anyway and code gets easier
to understand.
Fix more hard-coded vector sizes. And set all the SSE_PARA's to 2 for
__AVX2__, for ironing out problems.
Wpapsk bugfixes for AVX2. Practically 2x boost over AVX.
Fix hardcoded SIMD width in Office format.
Hardcoded vector widths in Dynamic. Just the top of the iceberg. Now,
dynamic_30 works. Not many more of them...
Hardcoded vector width in alignment.
Finally nailed the MD5 problem for AVX2. Now all raw formats work.
Fix a load of hardcoded vector widths.
Fix algorithm names for AVX2/AVX512.
CPU_NAME -> SIMD_TYPE
More fixes for hardcoded vector width.
Fixed SAP/H width issues.
Fixed IPB and mscash2 width issues.
Fixed mysql-sha1 and nsldap width issues.
Slow workaround for Office format. I have yet to get the original
implementation to work with AVX2 - something is not right with the
width macros.
Vector width fix for md5crypt. But it still doesn't work.
Fix hardcoded vector widths in RAKP.
Fix hardcoded vector widths in salted-sha1.
Fix hardcoded vector widths in sunmd5. Still does not work with AVX2.
Fix hardcoded vector widths in SAP/B.
Fix regression problems from Lei's recent fix.
Finally fix the inherent problem with raw-sha1-linkedin's source(),
that would not cope with certain combinations of vector width and
interleaving, and also did not allow crippled test vectors.
Fix 64-bit index calculation for widths > 4.
Fix hmac-sha256 vector width.
Fix hard-coded widths SAP F/G format.
SunMD5, fix hard-coded vector widths. Closes #1191.
Oracle11, fix hard-coded vector widths. Closes #1190.
Finally nailed md5crypt vector width. Closes #1192.
It appears we can use same PARA values for AVX2 as for AVX.
Add a few more easter egg format classes: -form:avx, -form:avx2 and
-form:avx512.
Some cleanup in pseudo_intrinsics.h, and an optimization for MIC
emulation of _mm512_loadu_si512() that should do lots of good in
most cases.
pseudo-intrinsics.h cosmetical/whitespace/comments.
sse-intrinsics.c whitespace/comments cleanup. And finally drop the old
80x4 SHA1 that we stopped using long ago.
Dynamic: Change a few hardcoded width figures to correct formulas.
These changes make a few more formats pass self-test. See #1189.
Width fixes in Dynamic. See #1189.
Algorithm name tweaks.
Minor algorithm name fixes.
More algorithm name fixes. We are not displaying "block loops", it's
just confusing.
More algorithm name fixes. Some were completely backwards.
It turns out I'm the one who was backwards w/ algorithm names.
Last bunch of fixes to algorithm names.
Yet some algorithm name fixes.
Drop confusing mentions of "sse2" in config. Closes #1193.
Add an ifdef'ed out scalar version of vtranspose_epi32() (in the AVX2
section) for reference and benchmarking. See #1201.
The _mm*_i32gather_epi32() intrinsic has different parameters for
different architectures:
Cleanups for raw-sha1-ng. Drop vtranspose_epi32 from
pseudo-intrinsics.h and revert the 4x code to a format-local macro
with its original name. Also re-write some ugly width-dependent code
to be more canonical and fix some white space problems.
Separate int128.h from mpz_int128.h, and fix raw-sha1-ng in that
regard.
Add some documentation about AVX2 vs. Dynamic format.
Auto-disable dynamic format for k1om (MIC) also.
AVX2 fixes for 32-bit builds. See #1205.
Fix for legacy generic build after AVX2 support was added.
Fixes for generic builds.
Fixes for generic. See #1205.
Some plugins could not just be deleted, so make them non-plugs. Closes
#1170.
Drop the binary plugin interface that no-one uses. Closes #1124.
Fix a typo in configure.ac
Allow building with no plugins present. Closes #1207.
Add some documentation about how to disable AVX2.
Autoconf: Add support for detecting AVX512. Also fixes a bug with XOP
erroneously regarded as better as AVX2.
Autoconf: Move the OS X as(1) quirk prior to all feature tests.
Autoconf: Do not test for AVX512 unless we have AVX2.
Drop dupe macros from pseudo_intrinsics.h and tweak non-XOP rotates.
Cosmetic change (unify code) to pseudo_intrinsics.h.
wordlist.c: Use pseudo-intrinsics for SIMD mgetl().
Fix a bug in vroti_epi16() (which is not used yet anyway).
sse-intrinsics.c: Implement the "MD4/5 common XOR" but leave it
defined out since it seems to be slower no matter how I do it
(reason yet to be examined).
MD5: 9% boost just by moving code around (this was with gcc 4.8.2 and
AVX2)
Pseudo-intrinsics tweaks for AVX2.
Change a few int's to unsigned in sse-intrinsics.c. 1% boost in eg.
wpapsk and md5crypt.
Revert some intrinsics tweaks that had dubious benefit.
OpenCL RC4: Use restrict pointers.
Autoconf: Detect AVX512BW as well.
AC: Assume that there's no point in testing for AVX512BW unless we
found AVX512F.
AC: Same as previous commit but for cross compiles.
Drop a vim swap file accidentally added in 2f8bc4871
Correct some algorithm names.
Fix the length stuffing in dynamic, taking us a good bit closer to
AVX2+ support. See #1189.
Fix more hardcoded width crap in Dynamic. See #1189.
Cruical AVX2+ fix for dynamic. Fixes most (all?) segfaults but some
formats still fail self-test. See #1189.
Yet another bunch of width fixes in dynamic. See #1189.
Possibly the last bunch of width fixes in dynamic. Closes #1189.
Do not disable dynamic format for AVX2 anymore. Closes #1189.
Drop obsolete dynamic vs. AVX2 notices.
Dynamic, change lots of variables to unsigned, enabling optimizations.
Ensure unsigned divisions with SIMD_COEF_32/64 for faster code.
Major white-space clean-up to common-opencl.[ch] First: indent -kr
-i4 -ts4 -nlp -nbbo -ncs -l79 -lc79 -bad -il0 common-opencl.[ch] and
then: astyle --style=kr -t4 -U -H -p -xC79 -c -k3 -z2
common-opencl.[ch]
Fix a bug in --list=opencl-devices that caused "Max clock (MHz)" to
show a totally bogus number on some (rare) devices.
Drop obsolete header from wordlist.c (we now use pseudo- intrinsics.h)
Fix a logic error in #ifdefs for including mman.h. This should close
#1214. It's strange we did not get complaints on many systems.
Add an experimental clone of raw-md5 that uses flat buffers for SIMD.
For now it uses Tavis' SSE set_key() from his raw-sha1-ng format,
but this will be dropped in favor of some upcoming code in
sse-intrinsics.c. See #1220.
Reformat Jim's unreadable 650-column wide comments in
 see-intrinsics-load-flags.h, and add a few more flags (yet to be
actually implemented) to the enum. See #1220.
Bug fixes for raw-md5-flat. And drop the SIMD set_key().
Fix mixed declarations and code in pseudo-intrinsic.h
Fix memdbg for alignments larger than 16. Also fixes some other minor
problems (warnings). Closes #1200.
Always define _DEFAULT_SOURCE when defining _BSD_SOURCE. Closes #1227.
pkzip: Fix a cast that could lead to undefined behavior from
overflowing signed integer. Closes #1232.
OMP_SCALE tweak for raw-md5.
Promote "nt2" to be main "NT" format. Rename old "NT" to "NT-old".
Closes #1226.
NT OpenMP overhaul. This format should not run in HT threads.
listconf.c: Always call fmt_init() now that OpenCL formats are fixed.
03404dfd broke for any system using gcc but not glibc. Just comment it
out for now.
Fix a bug in pp.c where munmap() was called even when we do not
HAVE_MMAP. And add a warning notice when building without mmap.
AC: Change the way we look for mmap(). Closes #1214.
Add a format_all_done() for silencing memory leak checkers. Closes
#1244.
Fix a gigantic memory leak in dynamic when using --show and an input
file containing many different formats. See #1244.
Fix minor memory leaks in dual-use formats. See #1244.
Define _POSIX_SOURCE for glibc builds, for exposing fileno(3).
Let AC handle definition of _POSIX_SOURCE. Closes #1263.
Add a self-test fail for get_key() returning NULL.
Improve StatusShowCandidates so it logs the exact candidate number for
a crack. Screen output will still be rounded up to batch size.
Closes #1269.
memdbg: Fix alignment up to sizeof(vtype) for 32-bit builds. Closes
#1270.
memdbg: We were adding too much padding for 32-bit. It worked but was
wasteful. See #1270.
Revert "crypt-descrypt: for length-13, we never disable, since they
are in ST code"
Fix the slow AVX2 bug workaround in Office 2013 and add other
optimizations to boot. 65% speedup for -cost=2013:2013 (and now an
actual 2x speedup over AVX). Also hopefully fixes latent bugs with
interleaving.
Stop requiring libz. It's only needed for pkzip format and gpg2john.
Add some missing codepages to opencl_unicode.h
DMG (debug): Use fcntl locks if we have it.
Update .travis.yml
Update README.mozilla for mozilla-ng format
Update INSTALL-UBUNTU for latest rexgen (and drop obsolete libs).
Drop some artifacts from old MMX support.
Drop @Z_LIBS@ from Makefile.in - or it will be added either twice or
when not appropriate.
Revert "Drop @Z_LIBS@ from Makefile.in - or it will be added either"
Fix OS X as(1) workaround in cases where CC is not "gcc". Closes
#1293.
Minor and cosmetic tweaks for raw-sha1-ng.
krb5pa-sha1 formats: Salt cleansing, for pot sync to work properly.
Fix OpenSSL/CommonCrypto output. Closes #1299.
options.c: Bail with error if an invalid field separator char (eg.
multi-byte character) is used.
Reduce scope of a pot-sync variable.
sunmd5: Fix #1301 and also implement various changes for OpenMP that
 was discussed on john-dev.
Fix a repeated bug in many SIMD formats that had min_keys_per_crypt
set to 1 or to SIMD_COEF without para multiplier. It's supposed to
be set to whatever number is calculated at minimum, so for SIMD
formats it should be (SIMD_COEF * SIMD_PARA).
OMP_SCALE tune for sha1crypt. See #1303. Higher numbers may be
slightly better but impacts self-test duration too much.
Work around AES-NI problems with 32-bit builds.
Fix misaligned return from binary() in IPB2 and Lotus5. Closes #1291,
closes #1304.
Add struct alignment in sunmd5 that was missed in 7910681c
gpg2john: implicit int declaration.
Add another solution to #1093. This make it possible to build using
gcc 5 without --std=gnu89 (although I kept the latter for now). See
also #1250.
ALWAYS_INLINE: Revert last commit and implement a different solution.
See #1093.
keyring: Alloc a huge buffer instead of trying to create it on stack.
Works around a problem on some 32-bit builds and does not affect
speed anyway. Closes #1292.
pufferfish: Fix incorrect source file permissions.
Add missing FORMAT_NAME for streebog formats.
Stribog: Properly emulate epi64 intrinsics for 32-bit builds. And
disable format completely unless SSE4.1 is available. Tested on
32-bit OSX. Closes #1331.
Sledge-hammer bodge for pp.c fileno() warning in Cygwin.
Default to not enable OpenMP for fast formats. Closes #1325.
Fix Pomelo algorithm_name for SSE2 case. I believe the format uses
SSE2 in single-hash calculation, so it says 1x.
Stribog formats use SSE4.1, but not for doing several hashes in
parallel. So change algo to 1x.
Do not treat SHA256/512-ng formats as "fast" ones when it comes to
OpenMP.
truecrypt_volume2john: Add a missing LF
Revert "Sledge-hammer bodge for pp.c fileno() warning in Cygwin."
Repeats16/32: Increment char before length (b will now come before
aa). This greatly benefits formats where length variations cause
thread (or branch) divergence.
Fix a bug in rar2john for decoding non-ASCII filenames when stored in
compressed form. Did not affect crack, just meta data (eg. Single
mode).
Tweak error output in wpapcap2john.
wpapcap2john: Add some rigidity against bad input. Closes #1340.
Finally start defaulting to UTF-8. Closes #1215.
Pomelo: Drop some checks that are always false (unsigned < 0). Closes
#1343.
keystore2john: Drop some checks that are always false. Closes #1342.
Fix a problem with debug builds and gcc 5.x. Closes #1286.
Bump max length in GOST R 34.11-94 format. There's no reason for us to
artificially limit it. Also fix HMAC-GOST code (not yet used).
Stribog: Don't compare too much in cmp_all() - but on the other hand
*do* compare all of the hash in cmp_one().
Many formats: Don't compare too much in cmp_all(). There is a reason
we separate cmp_all() and cmp_one() for crying out loud. Hopefully
this Q'n'D fix compiles to a simple binary[0] == crypt_key[0] in
many cases.
BFEgg has a binary size less than 8, do not memcmp ARCH_SIZE.
mem_alloc_tiny(): If -DDEBUG or -DMEMDBG, never return better
alignment than requested (for triggering bugs). See #1195.
Truecrypt: Fix a debug code buffer that was undersized.
Fix non-AVX2 builds after the previous patch.
Add SIMD_PARA_SHA256/512 macros to mic.h and x86-sse.h.
Dynamic: SHA2 interleaving fixes.
Show SHA2 interleaving factor in algorithm names if applicable.
Do not use "pragma GCC optimize" for ASan builds.
SHA512 SIMD interleaving bug fix.
sha512crypt: Interleaving fixes. Tested with para 3 so far.
sha256crypt: Interleaving fixes and some unification with sha512crypt.
Tested with para 2 & 3 so far, and AVX & AVX2.
SHA2 interleaving fixes. More to come.
sse-intrinsics.c: Asan builds was unbearably slow without using
#pragma GCC optimize... To get that, use Asan and build with "make
debug".
Fix non-OpenMP build problems introduced with b96ed88f.
Add all aliases to --list=encodings, and fix bash completion for
multiple "or". Closes #1352.
HMAC-SHA2 formats, interleaving fixes (not complete).
XSHA512 interleaving fixes (not complete).
SHA2 interleaving fixes.
Drop redundant SHA256/512_BUF_SIZ macros. They are all 16 so we don't
need different ones.
SHA2 interleaving fixes.
AIX-SSHA512 bugfix.
SHA2 interleaving fixes.
Blackberry: Interleaving bugfix, and correct algorithm name.
rawSHA224/256 formats: Fix interleaving (now passes Test Suite).
rawSHA384/512 formats: Fix interleaving. Still fails but I'm pretty
sure the problem is in sse-intrinsics.c. Except I can't find any
problem there...
Fix a SHA512 interleaving bug in sse-intrinsics.c. Lei did the right
thing, then I blew it.
Interleaving bugfix in Drupal. Still doesn't work.
Final interleaving bugfixes in Drupal.
Interleaving bugfixes in MSSQL12
Interleaving bugfixes in XSHA512.
Minor regression fixes after SHA2 interleaving battle.
Add a testparas.sh script. See #1297.
testparas.sh LF and OMP fixes.
Refactor MD5_SSE_PARA -> SIMD_PARA_MD5 and so on. See #1217.
Pomelo and SunMD5: Fixes for non-SIMD builds.
SybaseASE: Handle truncated key length. See #1367.
Implement SSEi_FLAT_OUT for simpler format code. SHA512 and MIC may
hopefully use the new scatter instructions here.
Use new SSEi_FLAT_OUT in formats that did marshalling in caller.
Bump Keepass max. length to 125.
Fix a minor OMP_SCALE regression from 87fab5b.
sse-intrinsics.c missed LF on last line from 84bd73e0c. I've seen
compilers choke on that.
Use vscatter for SSEi_FLAT_OUT in MD4 and MD5 too.
Add codepage.pl as seen on john-users mailing list.
testparas.sh: Ensure using OMP_SCALE=1 while testing. Also change all
formats to use any existing OMP_SCALE if defined. This is for
automated testing (obviously not limited to testparas.sh).
Fix SSE2 vswap64() macro. We change in-place *and* return the result.
See #1373.
More #ifndef OMP_SCALE wrappers. Last one-liner wasn't good enough.
Add SIMD support to new Oracle 12c format.
sse-intrinsics.c: Wider interleaving loops, and as a result use
 non-array temp variables. See #1217.
SHA1 intrinsics: Drop a whole array of redundant temp. We can use the
existing pad array! And decrease SHA1_PARA to 1 because that's
faster now.
Fix bug for SSEi_FLAT_IN introduced in b3fb17c7b, and drop yet another
temp variable. Closes #1375.
sse-intrinsics: Revert to using tmp[SIMD_PARA].
SHA512 intrinsics: Decrease the w[80] pad to w[16] using the same
sliding-window technique used in SHA1 and SHA256. Just a 3-4% boost
here but may gain more on some systems. See #1217.
Unify SHA256 and SHA512 intrinsics. Effectively a no-op, but simpler
code. See #1217.
Add an EMMS at HDAA MMX function exit (icc warned about it).
Mute a warning from some compilers about see-intrinsics cast.
Intrinsics: Safer casting (union) between vtype and integer.
Replace testparas.sh with testparas.pl, with Markdown table output,
ready for pasting in GitHub.
Add non-OMP figures in testparas.pl
Improve testparas.pl for quicker run and real non-OMP figures. This is
very hacky but does the job.
krb5pa-md5: Improve comments on input format.
Keepass: Avoid a compiler bug related to SALT_ALIGN. Closes #1284.
Add CRC-32C format with SSE4.2 support, just for trying it out. See
#1379.
Use a unique tag for CRC-32C.
Re-enable SIMD for Truecrypt/SHA512. Closes #1421.
Fix a bug in last commit spotted just by review. We need test vectors
for this! See #1421.
Add error handling for keyfile operations in truecrypt.
doc/MASK: Fix a typo and do not use tabs.
Drop the separate CRC-32C format (CRC-32 format now hosts both).
Drop the recently introduced Lineage format. It wasn't ready.
Add "cost" support to CRC-32 format, mainly to facilitate benchmarking
CRC-32C separately (using --cost=1).
Whitespace/cosmetic in raw-md4-opencl
Add GPU-side mask mode to raw-md5.
Drop some obsolete/redundant stuff from CRC32 format.
Tweaks to README.md: Slightly smaller heading, put license last.
Fix a segfault in some OpenCL formats when ran after another.
raw-sha512-opencl: Fix re-init bugs.
Fix for plugins that are #ifdef _OPENMP (they were never included).
OpenCL formats: Fix a problem with --list where fmt_done() is called
without create_clobj() being called first. Closes #1237.
Drop an old kludge for nvidia rotates and fix weirdness in pwsafe code
(cosmetical).
Core merge for Makefile.legacy. Git lost track of the connection with
master:src/Makefile after some screw-ups we did in the past.
Update rexgen.h for latest librexgen.
Support Tera prefix for C/s numbers.
Revert bogus change of raw-md5-opencl format tag, plus some minor
(cosmetic) fixes.
Add generic pbkdf2-hmac-md5 OpenCL format.
Add SIMD support to pbkdf2-hmac-md5. Closes #1446.
Add 1000x test vectors to pbkdf2-hmac-md5.
OpenCL raw-md4/5 formats: Correction for assert.h, it was given in
quotes instead of brackets.
Another try handling the CircleCI problem with MD5 internal CTX
struct.
Add 'const' to our own MD4 and MD5 update functions.
Mute some compiler warnings (gcc 4.4.7, possibly false positives).
Add a pbkdf2-hmac-md4-opencl format, for testing purposes.
Correct some confusing format struct names.
Add a pbkdf2-hmac-md4 format, for testing/benchmarking. Also change
testparas.pl to use the PBKDF2 formats for MD4 and MD5 and add some
test vectors so we always do 1000 iterations.
Selectively mute some output from testparas.pl.
Fix for clang not compatible with gcc despite claiming to *be* gcc
(which proves that's a dumb ass claim).
Add const to MD4/MD5 Update functions. I already did this but jfoug
decided to drop them without discussion. Do not do that, I didn't
add them by accident.
MD4/5 const stuff.
#undef MIN/MAX macros before defining.
Add CPPFLAGS to cprepair target.
Drop a dupe macro definition from raw-sha1-opencl
bcrypt-opencl: Only use privat mem (as opposed to local) for Maxwell
and above.
Ignore "Idle=Y" when running an OpenCL or CUDA format. See #1460.
Work around an OSX OpenCL driver bug for Intel HD4000 where profiling
info sometimes end up with end-time == 0. In the cases seen, we can
just ignore it (set end-time to start-time). Let's hope we don't see
more of this bug.
Improve OpenCL version detection.
OpenCL 1.2 formats: Move slow function call out of inner loop.
Add a workaround for what I believe is a driver bug in OS X for nvidia
(seen on GT650M driver version 10.2.7 310.41.25f01). See #1459.
BF-opencl: Use private memory for nvidia sm_30 & sm_35 too.
Turns out only Kepler benefits from using private memory in
bcrypt-opencl. The Maxwell is 6.5x faster using local memory.
OpenCL fixes after #1456 and #1457.
SHA512 OpenCL: Workarounds for OSX' nvidia & HD4000 driver bugs.
Closes #1464 (even though HD4000 now fails instead of crashes).
Unify OpenCL formats / use shared code.
pass_gen.pl: Drop misplaced 'require' lines for Haval. The point of
using 'require' instead of 'use' is we only place them where
actually needed. I can't currently build the Haval stuff from CPAN,
for some reason (fails self test).
Revert "Add build timestamp to --list=build-info"
Add -Wdate-time if compiler supports it. JTR_FLAG_CHECK is a gem!
Add git commit hash to --list=build-info. Also move non-version
strings like "debug" and "OMP" to the *build* info.
I was hoping this would fix a dependency problem with version.h, but
it doesn't. It does update version.h if needed, but listconf.c may
already have decided dependencies being OK.
Fixed dependency problem (I think) with version.h
Ensure "usage" version is same as "--list=build-info".
Drop a level macro that broke all real builds.
We can't use OpenCL 1.2 commands without wrapping them in #ifdef.
Addresses #1470.
If we are a "release build", never show Git hash in version string.
Kludge for ADL device id matching with Catalyst > 14.9 and dual-chip
devices. I'm out of (printable) words for AMD.
Change our OpenCL run-time support macros: APPLE -> __APPLE__
DEVICE_IS_CPU -> __CPU__ And add __GPU__ too. The __CPU__ and
__GPU__ macros are defined by AMD but we make sure they get defined
when not using AMD too.
pbkdf2-hmac-sha512-opencl: Apply some bugfixes from CPU version.
Closes #1476.
Allow an option argument string to contain an equal sign when the
delimiter is a colon. Closes #1481.
Revert 734dc42 and re-implement parts of it in a way that doesn't
force you to use a fkn pager to read the output.
Drop an unjustified chain of ifdefs for debug build warnings in
bench.c
Fix a bug in configure from d9293dc.
Some dynamic formats had the ambiguous text "first 32 bytes". Change
it to "hash truncated to length 32". Closes #1482.
Bugfix for POSIX va_args usage in dynamic_compiler.c. You can't re-use
the va_list, you need to copy it.
See if we can have Travis test OpenCL (CPU) builds.
Fix a perl warning and a missing linefeed in testparas.pl.
testparas.pl: show OMP/GOMP params
2nd try getting OpenCL into Travis
OSX OpenCL: We can't -D__APPLE__ because it clashes with system stuff
when using the CPU device. Refactor to __OS_X__.
RAR3 formats: share about 700 lines of code. See #978.
RAR5 formats: Reject bogus iteration count. Closes #1493.
OpenCL verbosity tweaks.
DEScrypt-opencl: Use "safe goto" for newer AMD drivers. Closes #1496.
nt-opencl: Silence a compiler warning.
OpenCL: Verbosity tweaks.
Core: Always pass database to reset(), if we have one.
OpenCL formats: Adopt to core change for reset().
OpenCL mscash: Adopt to core change for reset().
OpenCL SHA2 formats: Bug fixes after core change for reset().
Mute compiler warnings when building without OpenMP.
Fix -DDYNAMIC_DISABLED.
Fix name case for pbkdf2-hmac-sha512-opencl
Fix typo in Makefiles ($shell) -> $(shell)
Silence warnings about redefining bswap32. See #1511
Fix OpenCL version mismatch in GPU mask formats. See #1508 for
discussion.
One OpenMP pragma missed in 5bd720090
Add --list=build-info to 'make check'. See #1516.
Silence warnings about redefining bswap64. See #1511
Port SXC fixes to OpenCL version.
Port Strip fixes to OpenCL version. And fix a fundamental align
problem with the "unsplit" PBKDF2 salt struct.
Add -list=build-info to CircleCI. Closes #1516.
Strip-opencl: Fix alignment warning. Closes #1519.
Do not trust byteswap.h for things like swap(*x++). See #1520.
magnum decided to return the address holding a pointer instead of the
address it contained. Dubious idea, and did not quite work out well.
Comment out defining DEBUG in build_table.c. Closes #1521.
Boost NT format by a percent or two skipping final add, which is a
very modest first attempt to reversing steps...
More reversing steps for NT SIMD format (and shared MD4 SIMD). Total
boost now 10% from original speed.
Add whirlpool_plug.o to dynacomptest rule.
Reverse steps in raw-md4 format. 12% boost. See #1523.
Ensure out max-length self-tests doesn't make case-insensitive formats
fail.
Fix cmp_all() in reversing MD4 SIMD formats. It would only check 32
bits when reversing steps. We don't have the half baked buffer so we
need to do a full MD4 in cmp_exact() but that wont affect
performance anyway. See #1523.
Reverse steps (more of an early exit) in raw-md5 (SIMD). See #1523.
Reverse last add of constants in SIMD raw SHA1. See #1523.
Reverse final add in all (SIMD) MSSQL formats. See #1523.
OpenCL formats: Do not compile kernel in init(). It slows things down.
Bugfix for reversing final add in SHA512.
Partially revert "Generate more test output for `make check`, for
Travis build clues"
Pacify warning about unused variables in MD5 kernel.
Pacify warnings about unused variables in MD4 and SHA1 kernels.
Pacify warnings about unused variable in NT kernel.
Fix non-SIMD builds of MSSQL formats.
Fix #1533 (and a memory leak while at it). Sorry about whitespace but
it was a horrible mess of tabs and spaces.
Add a heading for --list=format-details. Defined out for now.
mscash-opencl claimed to fully support UTF-8 and codepages, while it
doesn't. This is a bad bug: False negatives!
Fix version.h rule: -N is not a universal option to diff and we don't
need it anyway. Closes #1555.
Fix for when using attribute format gnu_printf instead of printf. See
#1555 and 6406e51.
It turns out even -q is not in POSIX diff. Closes #1555 again.
Fix a bug that prevented NETLM and NETHALFLM to be removed even though
they are plugins. Also prepare for upcoming OpenCL version of LM
format.
Add tag to bare hashes $u -> $$U preparation (if needed). Also fixes
some CRLF issues. See #1541.
Fix for dynacomptest workaround. See #1541
LM-opencl: Make device query error for warp size non-fatal. Also fixes
minor things like format name and benchmark len. Closes #1561.
Print CUDA version numbers in --list=cuda-devices. Closes #1534.
Minor fixes to sha256crypt while trying to nail #1532.
Reverse final add in raw SHA2 formats and LinkedIn format. Closes
#1523.
Linkedin format was accidentally disabled for all non-SIMD builds.
Dynamic compiler: Normalize $password -> $p. See #1543.
Add cmpt_cp.pl, closes #1568.
Some more white-space fixes. See #1567.
cmpt_cp.pl was accidentally put in base directory instead of src.
zip2john: Change default to never use file magic. Closes #1569.
zip2john: Fix usage blob after last commit. See #1569.
editorconfig: Let user decide his tab width. That's the very beauty of
using tabs!
Fix a bug in pot sync found by Christien Rioux, where split is used
twice. Since split returns a static buffer, a strcmp of the two
results always returned 0. Closes #1573.
Improve SSEi_REVERSE_STEPS for SHA1 for another 5% boost. The Linkedin
format can't reverse steps because it needs a source() with larger
part of binary as input to be safe.
Update icc-precompiled intrinsics (compiled on Super).
Fix memdbg bug in mscash2-opencl helper. Closes #1574.
Fix "reverse sum" bug in raw SHA-224/384 and also decrease binary size
in a bunch of raw SHA-2 formats for smaller memory footprint when
loading many hashes. Closes #1577.
MSSQL off by one bug introduced with recent reverse-steps code.
Fix a couple of bugs in MSSQL formats before jfoug finds them ;-)
Work around an OpenCL driver bug in OSX. See #1583.
Rewrite some hairy code in oldoffice kernel. I'm not sure it was legal
(and guaranteed left-to-right) at all but even if it were, some
drivers had problems with it. See #1571.
OpenCL: Add a -D__MESA__ for MESA platforms. And use it to enforce
 "safe goto" DES kernel version for decrypt. See #1470.
Since Mesa incorrectly claims supporting OpenCL 1.2, add an autoconf
function test checking for actual presence in the libs found. See
#1470.
Minor fix for the OpenCL 1.2 AC detection stuff.
Fix for non-opencl builds.
Fix a bug in LM OpenCL for use of clGetKernelArgInfo. See #1587.
LM OpenCL: Fix for OSX nvidia. Closes #1587.
Bugfix for selection of LM Incremental mode.
Automagic selection of LM-opencl codepage.
sha256crypt-opencl: Protect some macro arguments with parens. Closes
#1601.
Correct algorithm_name for non-SIMD sha512crypt.
Bump nt-opencl to max. length 27. This adds six additions to each
calculation and increases a buffer size with 8 bytes.
Autoconf: Don't enable CUDA unless user asked for it.
Autoconf: Change --enable-rexgen -> --disable-rexgen (default is we
use it if found; user may want to disable it).
Oracle12c: The SIMD implementation never worked except for index 0 - a
whole loop was missing. See #1607.
CRC32 bugfix: We used a function pointer to crypt_all that was set in
set_salt, but cracker.c and others may have already "cached" its
dereference. See #1607.
Fix for non-SIMD OMP Oracle12c.
Drop old "testpara" make targets (they are obsoleted by testparas.pl).
Closes #1297.
CircleCI forced me into actually testing a non-SIMD build myself and
fix the Oracle12c problems.
Tune lm-opencl for OSX nvidia.
Emit a warning if CUDA was found but not OpenCL.
Update README-CUDA.
Warn if using an 8-bit incremental mode with a 7-bit format. Closes
#1619.
-enc=raw (or -enc=ascii) disables CPstoreUTF8 option. Closes #1633.
Document the fact --loopback now defaults to using rules. See #1655.
Change %zd to %zu in a number of files. Silly.
Regard GPU temp readings of < 10°C or > 125°C as invalid.
Change all code to use our AES-NI (run-time detection) implementation
(with fall-back to OpenSSL). Heaven knows why we didn't do this ages
ago.
Fix aes/ legacy Makefile for recent AES changes.
Revert Office, LastPass and GPG formats to use OpenSSL AES since our
run-time-detecting AES-NI code only handles CBC mode. Closes #1683.
7z-opencl: New kernel, optimized similar to RAR3. About 65% boost and
now faster than oclHashcat.
Boost 7z formats' many-salts speed a *lot*, it's KDF is effectively
unsalted as long as the iteration count doesn't change. Closes
#1679.
Emit a warning if "many salts" benchmark did not test the full
BENCHMARK_MANY number of salts.
Bitcoin: Drop OpenSSL EVP high-level crap in favor of our own AES.
Hopefully fixes #1667.
RAR3 formats: Stop using OpenSSL EVP, see #1686. This ought to give a
noticeable boost, especially for OpenCL.
Add missing header protection wrapper (wtf is that called?) for aes.h.
7z: Add a missing OpenMP pragma for non-SIMD and fix algorithm names.
Add a bunch of test vectors to 7z formats.
Fix bug in 7z for SIMD: Max supported length seem to be 27. Also fix
my own bug with OpenMP allocation. There are still some SIMD
 problems so the '#undef SIMD_COEF_32' stays.
Re-write the "extra pot files" code to use a list section. See #1623.
doc/CONFIG: Replace a line that got lost from its section, and add a
bit more details/examples.
Drop an unused variable from encfs_common_plug.c.
OpenCL PBKDF2-HMAC-MD4/5 contrived formats: Use 2x interleaving on
nvidia Kepler.
pass_gen.pl: Add a -vectors option for test vector "source code"
output.
Drop silly warnings from pass_gen.pl
md5crypt-opencl: Prepare for supporting length > 15.
formats.c: Use strcasecmp for checking (most) format names. Also
trim/fix some error output.
pass_gen.pl: Refactor a sub for checking (surrogate) UTF-16 length.
7z: Bugfixes for SIMD. Buffers were not sufficiently allocated for
worst-case scenario. Also move more code into "if (new_keys)".
Closes #1692.
7z: Fix for non-SIMD builds.
Add a macro for detecting "nvidia sm_32 or better" and use it for
enabling Solar's inline PTX for md5crypt format.
OpenCL NTLMv2 & SHA1crypt: Run 2x interleaved on Kepler.
OpenCL shared pbkdf2-hmac-sha1: Tweaks for reducing GPR pressure.
Update Catalyst major/minor -> "version" list.
Drop pufferfish format. Closes #1702.
Don't use __attribute__((max_constant_size())) on OS X, it's not
supported (emits warnings). Closes #1657.
Loopback mode: Never assemble LM halves when target format is
descrypt. Closes #1654 (or so I hope).
Tweak "recommended"/"not recommended" output for AMD versions.
7z: Canonical fix for the confusing alloc formulas. See #1692.
Single mode: Re-write Jim's patch for retesting guessed plains against
all other salts. See #1706.
Use fewer test-vectors in 7z-opencl unless -DDEBUG - it takes too long
to self-test.
Tweak for new auto-tune, fixing problem with some formats.
Shared OpenCL code: Drop obsoleted functions.
New OpenCL auto-tune: Print nanoseconds in human format.
OpenCL autotune: Drop obsolete functions get_task_max_size() and
get_default_workgroup().
Add ability to read all pot files in a directory. See #1623.
EditorConfig: Use a tab width of 8 for core files.
Break out the Jumbo additions to single mode rules to a separate
section (but do include that section where it was located, so this
is a no-op).
SessionFileProtect: Adopt to coding style, simplify code and fix error
messages. Closes #1710.
Try to address #1711. I have no idea why this doesn't work.
Logic bug with SessionFileProtect.
OpenCL: Change BENCH_CLERROR so it can always be used in crypt_all().
This reduces the risk of auto-tune aborting the process if hitting a
limit.
More output from auto-tune: Show LWS when tuning GWS and vice versa.
OpenCL: Change some %zu to portable macro.
Add a global variable telling if we're currently auto-tuning.
Change default (LWS == NULL) to a device query for warp size. But also
include other alternatives in a ladder of #if's.
RAR-opencl: Use the new ocl_autotune_running variable to get rid of
the crypt_all_benchmark() function.
WPAPSK-opencl: Use the new ocl_autotune_running variable to get rid of
 the crypt_all_benchmark() function.
7z-opencl: Use the new ocl_autotune_running variable to get rid of the
crypt_all_benchmark() function.
Office OpenCL formats: Drop the separate crypt_all_benchmark().
encfs-opencl: Drop the separate crypt_all_benchmark().
krb5pa-sha1-opencl: Drop the separate crypt_all_benchmark().
sha1crypt-opencl: Drop the separate crypt_all_benchmark().
rar5-opencl: Drop the separate crypt_all_benchmark().
pwsafe-opencl: Drop the separate crypt_all_benchmark().
OpenCL PBKDF2 formats: Drop the separate crypt_all_benchmark().
Bunch of OpenCL formats: Replace unreadable code with an existing
macro.
Many OpenCL formats: Speed up autotune and make it more reliable.
OpenCL: Make a macro vector-aware.
RAR-opencl: Autotune fix just for good measure. I saw no problems.
OpenCL: Drop local v_width, use the global one. Refactor
opencl_v_width to ocl_v_width.
Tweak BENCH_CLERROR macro.
Workaround for OS X weirdness with CPU OpenCL device.
OpenCL bugfix: BENCH_CLERROR got the logic backwards after change.
Tweak error output during autotune.
OpenCL: Add a global variable holding the max LWS of *all* kernels.
For this to work, format's get_task_max_work_group_size() must be
correct.
OpenCL: Don't show a build log only containing warnings/notices,
unless verbosity is bumped. This was the case for release builds
already.
OpenCL kernel functions: Never use static, always use inline.
GET_MULTIPLE_OR_BIGGER macro must not be vector aware. Add another one
instead.
Add john.conf option for auto-tune initial LWS enumeration.
OpenCL RAR formats: Bugfixes for LWS=0.
md5crypt-opencl: Add a pragma needed for amd_bitalign with older
drivers.
Change some exit() to error() for proper logging and easier debugging.
See #1714.
Add a header to win32_memmap.c needed for error().
Update dependencies in Makefile.in
7z and RAR: Trim the waste-space allocation for length sorting. Closes
#1692.
Correct dependency for error() is misc.h.
7z: Missing paren. I did say it was tricky.
Undocumented hack: If you give the benchmark time as a negative (eg.
-test=-1) it will be minimum that time, but will also wait for
completion of "many salts" test.
Keccak formats: Stop lying about SIMD support. There is none. Closes
#1712.
SHA-3 format: Stop lying about SIMD support. There is none. See #1712.
Fix 1711: Correct detection of whether we're restoring or not.
Fix a logic bug that made (currently) impossible things like "-markov
-rules" not bail out with error. Closes #1697.
Revert exit() -> error() for a CUDA file.
OpenCL: Move Solar's bitalign macros to opencl_misc.h.
OpenCL: Drop the fragile nvidia_sm32plus macro in favor of separate
SM_MAJOR and SM_MINOR definitions supplied to the kernel.
Handle partial NVML (missing nvmlDeviceGetIndex() function). Closes
#1630.
Minor tweaks to --list:cuda/opencl-devices output (stop using tabs,
for a starter. Looks bad when pasted)
Bugfix for the many-salts extra test.
OpenCL: Ensure self-test and autotune does not involve changing length
of test vectors' plaintext. Some formats have too big a penalty for
that so we need to let the format decide (by means of what test
vectors it declares).
OpenCL autotune: Rework keysetting a bit more (and fix a bug that lead
to irregularities).
formats.c: The fmt_set_key() wrapper now has its buffer SIMD- aligned.
Also fixes a few callers that did not use the wrapper.
Fix --format option when -DDYNAMIC_DISABLED. Closes #1722.
Change a format expression in dynamic.con so we don't end up seeing
"peoplesoft" when gripping out primitives. See #1716.
Tweak format descriptions for dynamic Cisco formats so we don't end up
seeing "cisco" when grep'ing out primitives. See #1716.
RAR-opencl: Minor optimization, 5-10% boost.
RAR OpenCL: allow less work per loop-kernel call.
OpenCL auto-tune: log resulting work sizes.
Minor 7z-opencl optimization.
Tweaks for .editorconfig
Suppress an erroneus comment about UTF-8 with -enc=raw.
Add FMT_TRUNC format flag. This is set for formats that originally
truncate at our max. length (eg. LM, DEScrypt) but not for formats
where our max. length is limited (usually due to optimizations), eg.
NT and MD5crypt. See #1724.
Lotus5-opencl bugfix (segfault in some situations). Closes #1726.
If BF_X2 is already defined (eg. CFLAGS), don't change it. See #1199.
Retire the "md4-gen" format, just like "sha1-gen" (they both live in
unused/ for a while). It was obsoleted long ago.
If BF_ASM is 1, don't allow redefining BF_X2. See #1199.
--list=build-info: Show salt hash size.
Separate --log-stderr from --nolog. Now the former does not imply the
latter, but both can be used.
Salt sort: Allow user to disable the default fallback to "most used
first" (it was hard-coded until now).
Lower default iteration count for pbkdf2-hmac-sha2 OpenCL formats, for
easier testing. Use -cost:10000 to get the old figures.
Unify SHA-1's H() a.k.a F3() a.k.a SHA-2's Maj() as well as SHA-2's
Ch() implementations, using better optimized ones. OpenCL and CUDA
formats. See #1727.
Unify SHA-1's H() a.k.a F3(), a.k.a SHA-2's Maj() as well as SHA-2's
Ch() implementations, using better optimized ones. Intrinsics
formats. See #1727.
7z-opencl: Bugfix for keeping within workgroup limits.
DMG-opencl segfaulted on super because of OpenSSL thread safety issues
with EVP. Disable OpenMP for now. See #1686.
raw MD5 flat format (experimental): Drop bogus header dependency
Intrinsics: Use vandnot version for vcmov, if available (currently
always). Both versions are 3 ops but the andnot one has parallelism.
Pseudo-intrinsics: Comments (no-op).
OpenCL: Ensure auto-tune plaintext length is within limits.
OpenCL: Add an alternative SHA-2 Ch() a.k.a MD4/5 F() for archs with
and-not instruction. And stop defining USE_BITSELECT for Maxwell.
OpenCL: Fix definition of OLD_NVIDIA. It was always intended for
sm_1x.
DMG formats: Drop use of EVP in favor of low-level API and re-enable
OMP support for the OpenCL format. See #1686.
Several formats: Use shared PKCS padding-check function. See #1686.
OpenCL shared MD5: Use Solar's newly found or-not bitselect.
Dynamic: Drop a couple of spurious space characters.
opencl_misc.h: Do not define HAVE_ANDNOT for GCN. This is a no-op
though since it'll use bitselect instead anyway.
Re-enable use of bitselect instructions for Maxwell. The previous
tests seem to have been invalid due to kernel binary caching.
Add a "kernel-clean" make target, dropping nvidia's cached binaries in
~/.nv/ComputeCache as well as our own cache for other vendors.
Neither of them is safe from a change in just a header file included
by a kernel.
7z formats: Add a salt_compare function so we are sure to exploit any
same-iteration-count weakness. Closes #1728.
Update --list=format-methods with missing methods. Change ordering to
same as in the struct. Do not output get/ binary_hash methods that
are NULL.
OpenCL: Work around an Apple driver bug for Intel HD Grahpics. See
#1723.
Recent change to keychain formats increased number of FP.
Intrinsics: Sourcing of misc.h was trying to abandon the pack.
Intrinsics: Change Solar's "#if 0" around MD4/5's SSEi_FLAT_OUT to
"#if USE_EXPERIMENTAL" and add comments on why it's left out.
mem_align function: Require power of 2 alignment, and use faster code.
See #1731.
Change stack allocations of MEM_ALIGN_SIMD to be safe. Closes #1731.
Intrinsics: Pacify a compiler warning on XOP where PARA >= 3.
Drop support for "icc precompiled intrinsics". We keep breaking it and
apparently no-one is using it nowadays anyway.
XOP builds: Use 2x interleaving for SHA-256.
NTLMV1/MSCHAP2 bugfix (for currently unused code path).
testparas.pl: Finish with printing a complete suggested command line
for ./configure. Also add a second optional parameter for number of
cores (for make -j option).
Document testparas.pl (very briefly).
Ensure all PBKDF2 formats have 10K test vectors. To bench them, use
"-test -cost:10000"
Optimization for emulated vloadu/vstoreu for the case address is
already aligned. See #1731.
testparas.pl: Bug when run with no args.
The "-test=-1" hack for testing all 256 "many salts" did not work for
2nd or later formats in a test run.
The "-test=-1" hack for testing all 256 "many salts" still wasn't
quite OK. Took too long for some raw formats. Closes #1738.
New canonical code for "format class" selection. If you say something
like -format=@rc4 you will only get formats that has "RC4" within
its algorithm name. This can also be combined with rejecting
dynamics. Examples:
Trying to beat Jim to fixing a free() <-> MEM_FREE() problem in john.c
that I very recently introduced.
Format-class selection: Allow rejecting OpenCL formats.
Ensure unsigned divisions (optimized to shifts) in various index
calculations.
Dynamic compiler: Add max-length and zero-length test vectors.
Dynamic compiler: Bugfix for mixed-case plaintext test vector.
Add missing parens in dynamic_compiler_lib.c. Add missing dependencies
for dynamic_compiler.c and dynamic_compiler_lib.c.
Fix all dependencies in Makefile.in. Closes #1745.
Do not consider lc() and uc() for SALT_AS_HEX. See #1748.
Pull back Myrice's OpenCL SHA-512 formats (to unused, they need some
updates before you can build them).
Face-lift for Myrice's formats, still in unused. They need the shared
auto-tune, and things moved from init() to reset().
Drop interim formats, so next commit get the "moves" right.
Put Myrice's formats back in business.
Add a config option WordlistMemoryMap, defaulting to Y but letting
user disable use of mmap in wordlist mode.
Auto-tune had a really silly bug for picking plaintext length,
resulting in always zero.
Found Myrice's XSHA512 kernel. Its removal was hidden in an unrelated
commit (b0f1fad4).
Add autotune to Myrice's SHA-2 formats.
Address some test vector length issues in dynamic compiler format. See
#1751.
Add some debug output for CircleCI. See #1751.
Fix for non-SIMD dynamic='md5($p)' max-length test vector. See #1751.
Drop the "#pragma GCC optimize 3" from simd-intrinsics.c, and the XOP
workaround that was needed when we had it.
Revert some mem_align() uses. Closes #1735.
AVX-512: undef VCMOV_EMULATED when using ternary intrinsics to emulate
it (it's just as good as the vcmov instruction).
Re-introduce the mmxput3() switch alternative (but ifdef'ed out).
Use some shared OpenCL code for Myrice' SHA-512, and optimized SHA-2
subfunctions, but keep the actual SHA functions for now.
Dynamic: Drop the phrase "(MD5_body)" from non-SIMD MD5 algorithm
name. It made no sense anyway (it's just a function name) and it
screwed up dynamic compiler format's parsing of algorithm_name vs.
label. Closes #1763.
Intrinsics: Add missing curly braces around initializer.
Add "Best64" rules from Hashcat. Some rules are not working in JtR so
was commented out, and one 'f' rule was added to end up with exactly
64 rules.
Stop defaulting to "ReloadAtCrack = Y". It hurts performance when a
lot of hashes are cracked very fast, and the remaining pot sync
alternatives will take care of the important business anyway.
Centralize reversing/unreversing of (scalar) binaries, for easier
maintenance of code. And re-implement source() for raw formats using
reversed steps (SHA-1 still to-do).
Drop the john.conf default for DefaultInternalEncoding.
Do NOT lock potfile during pot sync.
Implement source() for raw-SHA1 format.
Fix SessionProtect for forked (or MPI) sessions, as well as for
JOHN_SYSTEMWIDE builds (Jim had hard-coded the session name). Closes
#1764.
Move the logic for "Disabled Formats" out of loader.
Do not define CRK_PREFETCH in cracker.c (it's defined in params.h).
Reverse three steps for SHA-2. 4-5% boost. Closes #1758.
Replace 0x7ffffff in all binary/hash_6 functions with PH_MASK_6 which
is defined as (PASSWORD_HASH_SIZE_6 - 1) in params.h.
mssql12 non-SIMD bugfix.
Change all binary/hash function masks to params.h macros.
Dependency tweak for plugins. See #1767.
Prince mode: Pure formatting.
Prince mode: 8x speedup for rules.
Prince mode: Optimization for when rules are used but no rules were
accepted (or eg. --rules=none).
Introduce macros for telling that vloadu or vstoreu are emulated.
Bugfix for -DDEBUG.
Change wordlist memory buffer default threshold to 1 GB, and use that
default unless -save-memory >= 2. Also tweaks some other related
things.
Fixes for generator/amplifier logic for mask and regex. Closes #1756,
 see also #1757 (more work needed there).
Mask mode: More rigid parsing of ranges/placeholders. Closes #1765.
Mask mode: Allow '??' as literal '?'.
Make -stdout "FMT_TRUNC" so we can chose at run-time whether to
truncate (-stdout=N) or reject (-max-len=N).
Loader: Fix a bug where a loop depending on signed behavior was using
size_t.
Revert "john-huge-loader-mt.diff, splitting initialization of bitmap"
mem_calloc: Actually use calloc().
Fix a few bugs not caused by, but triggered by, the recent loader
allocation tweaks. See #1769.
Ensure ShowUidOnCrack is turned off when --save-memory is active.
More tweaks for --save-memory.
Raw MD5 non-SIMD bugfixes.
MSSQL: Fix for reverse steps: We have to uppercase.
Add a config note about --save-memory vs. ShowUIDinCracks.
Refactor a variable to match its john.conf name.
Minor optimization for loader.
http://www.openwall.com/lists/john-dev/2015/09/16/32
Re-write code that used comma separator so it's easier to read.
Drop code that was obsoleted by fd3c440.
Fix a bug with hybrid mask used with wordlist + rules when an internal
encoding is active.
Mask mode: Use fmt_null_key instead of creating our own.
Minor overhaul of regex hybrid mode. See #1678.
Fix bugs introduced in previous commit and close #1678.
Add the cool rexgen example from #1678 to our arsenal of presets.
Allow external filters even with Hybrid Mask. The logic was there
already.
Disable CRK_PREFETCH for now, segfaults seen.
http://www.openwall.com/lists/john-dev/2015/09/20/6
Linkedin format: Bugfixes for reading {SHA}-style pot entries.
Disable UTF-8 checks when -enc:raw.
Optimization for registering dynamics. Shaves the number of calls to
dynamic_IS_VALID() by half (from 10,000).
Print hash counts to stderr for --show=invalid. See #1773.
Update INSTALL-UBUNTU (after I did just that).
Drop summary line from -show=invalid - it's not populated at all.
Add parsing of 1800.11 -> Catalyst 15.9.
Fix and improve OpenCL logging.
OpenCL: Force saving of kernel "binary" with env DUMP_BINARY. This
will have kernel's name with ".bin" extension added, stripped from
any path (so will end up in current directory). It will also
over-ride the caching we sometimes do for non-nvidia devices (the
nvidia driver caches for us).
OpenCL bug fix (or just rigidity fix) for auto-tune.
Add full Unicode/codepage support for NT-opencl, including for mask
mode with --internal-encoding. See #1037.
Mask mode: If mask contains UTF-8 characters, require
--internal-encoding so we don't get false negatives from improper
use. See #1037.
--show=invalid counted incorrectly. See #1773.
Add full Unicode/codepage support for mscash-opencl, including for
mask mode with --internal-encoding. See #1037.
New DEScrypt OpenCL code: Portability macros for %zu.
Comment out the silly "Get key error!". I have only ever seen false
alarms and Sayantan wont even comment it. See #1037.
Fix a bug in all UTF-8 conversions on GPU. We are processing strings
that are not NULL-terminated, yet checked for NULL in some places
 instead of checking length. The result was a performance drop.
OpenCL: Add one more instance of "spinning wheel" cursor for slow
benchmarks with --test=-1 (force all "many-salts" test).
Mask mode debug stuff.
Code style: Don't write pointers as multiplications.
OpenCL autotune: Fix bugs when LWS is required to be a power of 2.
NT-OpenCL: Use shared auto-tune.
OpenCL auto-tune fixes. And a hack: If mask mode is active, we
decrease max duration to a tenth.
AC: Change AC_CHECK_LIB tests from "!= no" to "= yes".
Put OpenCL UCS-2/UTF-16 support selection under the wings of
./configure. Closes #1779.
DES-opencl cleanup, and a tweak in shared code. Closes #1778.
MSCash-OpenCL: Use shared auto-tune. See #1037.
Revert "MSCash-OpenCL: Use shared auto-tune. See #1037."
MSCash-OpenCL: Use shared auto-tune. 2nd try, see #1037.
mscash-opencl: Unicode salt support and test vectors. Closes #1781,
see also #1780.
Bump SALT_HASH_LOG from core's 12 to 20.
Makefile.in: Add recent loader.o dependency for base64_convert.h.
OpenCL: Unicode/codepage fixes (edge cases, even literally) for mscash
& NT formats. Both formats now pass systematic testing (TS).
Workaround for AMD problem with mscash-opencl. Closes #1780.
gcc-4.4.7 complained about me breaking strict aliasing rules in
mscash-opencl. I agreed to disagree but had to fix it anyway.
Fix some mis-match between mask mode and its documentation. Also
deprecate ?h and ?H in favor of ?B and ?b since ?b is used by
Hashcat.
Mask mode fix: If a custom mask placeholder was defined containing
UTF-8 characters, we bailed out for "internal encoding required"
even when that placeholder was not actually used.
OpenCL common code: Ensure use of portable basename().
Drop spurious file add by Jim
Fix bug when using default (from john.conf) mask, and some other cases
of hybrid/pure mask & regex modes.
Mask: When iterating over length, optionally emit a status line after
each length is done.
SHA-256 intrinsics: Reverse 4 more steps for another 3% boost. This
can't be done for SHA-224.
Mask mode: Tweak an error message for easier understanding.
SHA-512 intrinsics: Reverse 4 more steps. This can't be done for
SHA-384.
MSSQL12: Reverse seven steps.
MSSQL12: Non-SIMD fixes after last commit.
Mask mode: Change "internal encoding" error output to be more
specific.
Deprecate -internal-encoding in favor of -internal-codepage, for
clarity. Change corresponding john.conf option
DefaultInternalEncoding to DefaultInternalCodepage. The old option
names are still supported (and will be for a long time). Also
refactors the related variables.
OpenCL auto-tune: Tune with GPU-side mask, when applicable.
Autotune: Print Gp/s instead of a wall of digits.
Pacify an idiotic warning.
Revert most of the recent auto-tune-with-mask patch. Regression-tests
revealed it already happened, LOL.
Cross-compile fixes for 'ar' and 'strip', and for using DESTDIR for
bash (and zsh) completion. See #1783.
Change most opencl_build() calls from Sayantan's formats to
 opencl_build_kernel() so they cache like the other formats.
OpenCL: Use a hash of parameters when deciding a cached kernel file
name, instead of creating filenames that exceed all sane limits. And
since the hash also includes the actual full source, we get extra
"security".
Bugfix in our implementation of RC4.
NT-opencl: Use our optimized MD4 macros. See #1729.
mscash-opencl: Use our optimized MD4 macros and the rotate function.
Closes #1729.
hmacmd5.h "bugfix": do not depend on caller including md5.h.
krb5-tgs: Major cleanup, code-style, stop re-inventing the wheel, fix
major and minor bugs, add proper Unicode conversion, add OpenMP
support, much less promiscuous valid().
OpenCL NT & mscash: Copy-paste bug of sorts.
Further boost for krb5-tgs format.
Cosmetic fixes in kirbi2john.py.
Complete the implementation of FMT_REMOVE. Closes #1793.
Minor enhancement of FMT_REMOVE code. See #1793.
Update docs for OS X 10.11 "El Capitan".
OpenCL bug fix. 57a6cce change a buffer to alloced memory but we were
still using sizeof() on it. Closes #1797.
Fix a bug in --reject-printable for split formats.
Revert "mask: ETA was 1/x the correct time in -fork=x mode"
Try to get rid of the Travis CI failures occurring ever since 09f7f91.
Bugfix for --test-full
Revert "Try to get rid of the Travis CI failures occurring ever since"
Myrice's OpenCL formats: Hopefully fix intermittent failures on CPU
devices.
Implement LOP3.LUT as inline PTX, but do not enable it (not yet seen
doing any good - only tested for md5crypt).
DUMP_BINARY environment variable over-rides any normal binary caching.
CUDA WPAPSK: Bugfix after get_hash macros was introduced.
Common OpenCL: Fix a potential memory leak.
lotus5-opencl: Drop that weird "PADDING" thing. This alone seemed to
fix all problems - closes #1726.
Revert "configure: changed != xno to = xyes for some display results
(crypt, mmap)"
Bump dynamic's DYNA_SALT_HASH_BITS to same as SALT_HASH_LOG.
Add proper logic for enabling LOP3.LUT (although we still don't
actually enable it because only regression was seen).
Minor CUDA fixes.
Tweaks for LOP3.LUT. Refactor the function to lut3() like what Roman
used in published DES BS s4. Always expose that function (if
available).
Add kernel filename to "Options used" when building OpenCL kernels.
DES & LM bit-slice OpenCL: Implement DeepLearningJohnDoe's LUT3
s-boxes, but with s4 replaced with Roman's published one (one less
gate).
Shared OpenCL MD4, MD5, SHA-1 and SHA-2 code: Optional explicit
LOP3.LUT.
Stop mis-using the "warn" parameter of opencl_build_kernel. It's for
kernels that take like over 10 seconds to build.
Stop mis-using the "warn" parameter of opencl_build_kernel (one last
format that was missed).
Use Makefile logic to copy/update OpenCL stuff to run/kernels.
Tweak Makefile logic for OpenCL stuff in run/kernels.
Tweak Makefile logic for OpenCL stuff in run/kernels.
Revert recent Makefile changes. They are problematic.
I seem to have a bad hair day.
Fixes for generation of Makefile OpenCL dependencies. Also updates
most other dependencies (auto-generated with plugin_deps.pl)
Fix a mis-leading notion in list=build-info.
Mask mode: Default to emitting status when switching length. And fix
some progress reporting issues.
Use __FUNCTION__, not __func__.
Some no-ops while trying to work out what's wrong with lotus5. Looks
like an nvidia driver bug. Intermittent failure when not
auto-tuning. See #1726.
Yet some tweaks required for OpenCL dependency generation.
Extra pot-file feature: When traversing directory, only consider files
with extension of ".pot". This is how the Windows code path already
handled it. Also, accept symlinks. And clean up code, moving this
to its own function to avoid indenting off screen... Closes #1623.
Windows fix for extra pot files. Windows does not grok symlinks.
Using S_IFLNK applies to lstat(2), not stat(2).
john.conf comment grammar fix.
Unicode core files overhaul: Code-style fixes and tweaks.
--stdout: Allow using a codepage output encoding.
External mode: Optionally support handling UTF-32 characters,
converted to target encoding just before crk_process_key(). This
speeds up things like -external:dumb32 with 4x and opens up for
plenty of other tricks.
Dumb16/32 and Repeats16/32 external modes: Use new UTF-32 external
mode feature. 4x boost (or even more if target encoding is a
codepage).
Implement a proper lut3_64() function, using two 32-bit ops.
OpenCL: Use bitalign for ror64() in shared code and in Myrice's
formats. See #1819.
The bitalign-rotate can't be used with vectors.
opencl_mask_extras.h: Pure whitespace clean-up.
Missed tab in whitespace clean-up.
Pot sync tweak for split OpenCL kernels.
Add a trivial script for calculating truth table for
_mm512_ternarylogic_epi32 or LOP3.LUT. Example:
Some mostly no-ops and whitespace noise while trying to nail the
nvidia LOP3.LUT bug.
LOP3.LUT bug fixes. Wasn't a driver bug... was PEBCAK!
Bugfix in the example in comments of truth.pl...
Enable LOP3.LUT for SHA-1 formats.
Stop using cl_nv_pragma_unroll for detecting nvidia; it does not work
on OS X. Also update mscash2-opencl with optimized functions.
truth.pl: Optionally show all alternatives for LOP.LUT3 (or AVX-512
ternary) truth tables.
Regression fixes for LOP3.LUT. We can't use that inline asm function
with vector variables.
Add SaltLen to new dynamic_1034. See #1825.
Poke a little at Myrice's formats to get rid of intermittent failures.
Usage blurb tweaks. Thou shall not hit 80th column (not even counting
from 1, due to Windows' silly terminal).
Allow "-test -mask" for benchmarking GPU-side mask mode performance.
Currently, self-tests are not performed when this is used. Mask can
be given or the default one from john.conf will be used. See #1822.
Enhance the "-test -mask" feature. See #1822.
Tweaks for --test --mask. See #1827.
Auto-tune tweak for mask mode. See #1827
Mask mode ETA bug fix: mask_tot_cand is supposed to hold total
multiplier *including* GPU-side. Parent modes' progress reporting
depend on it.
Some refactoring of mask mode global names, per code style.
Mask mode: Decrease max. number of placeholders from 255 to 127. This
still supports the global core max length of a plaintext and should
help footprint and locality.
Implement a shared OpenCL function to get a very rough "speed index"
for an OpenCL device, and use it to tune mask mode. Closes #1831.
Global tweaks to recent mask mode tuning logic. See #1831.
Print our "speed index" estimation in --list=opencl-devices. See
#1831.
More tweaks for mask tuning. See #1831.
Mask benchmark tweak. We won't generate candidates longer than
format's first test vector.
Hack fix for #1828.
Put a max-length test vector first in LM-opencl for better auto-tune.
mscash-opencl: First test vector should be length 8 or so and
definitely not a null-string. This is for shared auto-tune reasons.
Bugfix for building non-OpenCL.
Reduce default logging for rules and incremental mode. Now -verb=4 or
higher is required to get gigabyte size logs.
Have -stdout call log_init() but imply --nolog. See #1774.
NT & mscash OpenCL: Avoid double auto-tune for "-test -mask".
--list=opencl-devices fix. Do not print SIMD width as "stream
processors" (although we count it as such for other purposes).
PBKDF2-HMAC-MD4/5: Allow VLIW5 to use native vector width. The
register pressure limitation doesn't apply to these formats.
krb5tgs format: Bump MAX_EDATA_SIZE, see #1838.
Implement a function that creates a "struct db_main" from test
vectors, and a destroyer for it. See #1835.
Allow changing or dropping the UTF-8 degree sign in GPU temp
printouts.
Bug fix for the shared check_pkcs_pad() function. This surfaced in
form of false positive in dmg (header version 1).
AMD bug-workaround for SHA-512 formats when using bitalign for
ror64(). Closes #1840. Also closes #1834, adding a related pragma
needed for older drivers.
AMD bug-workaround for oldoffice-opencl. Closes #1497.
Mask mode: Change ?a to frequency-ordered, from Rockyou data. Closes
#1842.
Mask mode: Change ?l, ?u, ?d and ?s to frequency-ordered, from Rockyou
data. See #1842.
Improve handling of UTF-8 "degrees" character in case terminal is not
UTF-8. If we know how to output it, do. Otherwise just skip that
character. Closes #1841.
Fix a cosmetic problem with speed figure calculations overflowing
32-bit integer under GPU-side mask mode. I already wrote code for it
but it was defined out.
Mask benchmark: Pure mask mode should send a NULL key.
OpenCL: Cosmetic. Move "speed index" adjacent to its source
parameters.
Copy/paste bug in raw/x-sha512-ng-opencl.
Tweak the use of setlocale(). We now query the "terminal locale" and
store it, but then reset LC_CTYPE to "C" so to not change behavior.
See #1841.
Implement a minor optimization for SHA-512 sigma's, from Intel's "Fast
SHA-512 Implementations on Intel Architecture Processors"
whitepaper.
Mask mode: Enable a sanity check that was disabled for
backward-compatibility.
RAR: Mute an irrelevant warning when loading pot entries.
Decrease log chatter from single mode, like was done with others.
Fix an autoconf problem resulting in the CPU detection in john.o
wasn't enabled when it should be.
Add CPU detection for AVX2.
Fixes for AVX2 detection.
Another fix for AVX2 detection.
Mask mode: Fix incorrect use to CTYPE macros, so ?W will case- toggle
according to our active encoding - not just ASCII.
--list=build-info: Show fallback binaries.
Bugfix for MPI vs. fallback binaries. We must not initialize MPI until
we are running the "final" binary.
OpenCL shared header bugfix: OLD_NVIDIA was defined on all non-nvidias
too.
Add cpuid tests for SSSE3, SSE4.1, SSE4.2, AVX512F and AVX512BW.
CPU detection fixes. We drop the check for SSE4.2 because only one
format needs it - that one should have a run-time check instead.
More CPU detection fixes. A bit more thorough.
Require AVX for CRC32C intrinsics (only SSE4.2 is needed but we don't
want to add another CPU fallback case). Closes #1856.
Minor fixes to Myrice's formats.
Salt cleansing bugfixes.
Bugfix for plugin_deps.pl, do not recurse autoconfig.h file. Closes
#1860.
oldoffice-opencl: Move benchmark logic out of salt.
pkzip: Fix a problem caused by the format changing the salt after
get_salt. We still do so, but leave that part out of the active
"dynamic salt" part for compare purposes. Otherwise we get problems
with pot sync and other things. Closes #1862.
Mask mode: Fix a problem with resume sometimes missing candidates.
This should now be 100% safe. Closes #1859.
Exclude Myrice's formats from Travis CI testing. It's got to be a
driver bug causing all trouble with them. Also ditch the
Formats_base thing, it was just confusing. Better learn the
difference between putting [Disabled:Formats] vs.
[Local:Disabled:Formats] in your john-local.conf! The former
replaces the original list while the latter adds to it.
Add some safety in dynamic parser code. See #1863.
Do not assume that char is signed. See #1863.
Mask mode: Ensure that things like ?l?u (eg. the default mask) only
produces a single case for case-ignoring formats.
AC: Do not use \b with sed, apparently it's not POSIX and it doesn't
work with OSX. Closes #1867.
Fixes for OSX El Capitan and CUDA 7.5. Closes #1868.
Enable showing memdbg summary with MEMDBG environment variable. See
#1872.
Fixes for our local stdint.h. Add intmax_t and uintmax_t as we now use
them in Jumbo. Also use compiler macros like __INT32_TYPE__ if
available. Finally, remove a clash in math.h from core.
arm32le.h: Do not define SIMD_PARA_* when we don't have NEON. See
#1874.
Assorted signed/unsigned bug fixes. See #1874.
Drop unused Blake2 functions. See #1874.
Move the nefarious LUKS test vectors to a separate file. #1861.
Split lines of luks_insane.tests.h so not to break havoc.
Add filename to cisco2john output, in id field.
Fix algorithm name of Azure AD format.
Fix algorithm name of Azure AD format, this time to the correct one.
Simplify OpenCL SHA-1 (unsplit) kernel a little.
Add a john.conf option AbortGraceTime for tweaking behavior of
 --max-run-time=N option. Closes #1638.
Add Truecrypt-opencl format. First cut, only RIPEMD-160 yet and for
some odd reason it's only a quarter the speed of Hashcat's as of
now. See #1344.
Optimizations for Truecrypt-opencl format.
pass_gen.pl: Honor -loops when generating bitcoin
Assorted hybrid resume fixes. Closes #1859.
Do not call wordlist's hybrid fix_state unless we are actually running
wordlist as parent mode.
Fix SessionFileProtect for batch mode. Closes #1774.
Consider the "Extra potfiles" not only when loading hashes, but also
for other things like -show and -loopback. Closes #1808.
Revert "Have -stdout call log_init() but imply --nolog. See #1774."
Revert "Hack fix for #1828."
List:Extra.Potfiles bugfix for --loopback (allow using loop_db).
john.c: Comment on a non-obvious line of code.
Log the command line used for session. It will be expanded from
abbreviations and unified to using '=' and not ':' as option
separator.
Finally, fix the Jumbo options-reset problem (when resuming)
canonically.
Fix a nasty bug in oldoffice-opencl where type 4 would never crack.
Closes #1881.
Change dyna salt formats to use fmt_default_dyna_salt(). Also assorted
fixes for dynamic salts, including OpenCL support.
AC: Test for hyperthreading, if available - define HAVE_HT. Set BF_X2
according to the result. Closes #1199.
AC: The HAVE_HT test was only done for AVX or higher, having bcrypt in
mind. But we might want to use it for other things too so this
commit makes it unconditional.
Set BF_X2 to 3 for non-OMP builds, regardless of CPU. See #1199.
Allow using OMP or CPU fallback without using SYSTEMWIDE.
Allow using CPU fallback without using SYSTEMWIDE (bug fix, the
previous commit only fixed OMP fallback).
Better fix for #1881. The previous fix worked fine but affected
performance.
Enable oldoffice's source() function, printing mitm hash in case user
wants to search for alternate keys. The use of MITM calls for using
dyna_salt. Closes #736.
Bugfix for dyna_salt in OpenCL: Bitfields are not allowed but we
actually only need the right size.
Show different output telling FMT_NOT_EXACT from --keep-guessing.
Fixes for CPU detection. See #1855.
[john.c] Clean-up from Jim's constant breaking of code style.
Disable cpuid checks if CPUID_DISABLE environment variable is defined
(at all). See #1855.
Bug fixes and easier-to-follow code for CPU detection.
Add 32-bit checks for AVX2, SSE4.1 and SSSE3. The AVX2 check is very
quick and dirty, not using leaf 7 nor extended flags but it should
be sufficient for a starter. This is UNTESTED but will hopefully be
Good Enough[tm] for #1855 for now.
Remove LENGTH from --markov mode documentation. Closes #1651.
Document PRINCE mode. Closes #1428.
configure.ac is not just a bash script.
Bugfix for 32-bit CPU detection. See #1886.
[options.c] Portability fix: Include jumbo.h.
Another 32-bit CPU detection bug fix. See #1886.
Safer CPU detect for cross compiles. See #1886. Closes #1855.
Code style fixes for Markov before doing actual changes. This is a
 no-op.
Delete Frank's diary from our source tree.
Code style fixes for Rexgen before doing actual changes. This is a
no-op.
regex fixes for -max-length.
Add hybrid regex support to markov mode. See #1445.
Add hybrid regex support to incremental mode. See #1445.
Add hybrid regex support to PRINCE mode. See #1445.
Add hybrid regex support to External [generator] mode. Closes #1445.
[mask.c] Fix for 32-bit builds not getting prince mode.
Tweaks for legacy builds.
Fix for 32-bit SSE2 CPU detection.
Yet another fix for 32-bit SSE2 CPU detection. How hard can it be to
get this right? See #1886.
Don't just "define" things like JOHN_AVX, set it to 1!
Remove a wall of 32-bit build warnings. The majority were due to a
silly macro that was sprayed all over the code base instead of put
in one place. Closes #1891.
Allow using external (generator) mode as hybrid parent. Closes #1889.
Allow using regex mode as parent to hybrid mask. See #1890.
Allow hybrid regex *and* mask, using a third mode as parent. Mask is
last since it can accelerate GPU. Closes #1890.
Update documentation of Jumbo cracking modes with regex, stacking and
some other details.
More updates on documentation and usage blob.
Mask mode: Avoid internal key buffer being thrashed by external
filter. Closes #1893.
PRINCE had same external filter bug as mask mode, see #1893.
john.conf typo.
Stupid bugfix for CPU detection. I did not notice that SSE2 flag was
in EAX while SSSE3 and SSE4.1 flags are in ECX, so I ended up
erroneously checking the xsave flag (and testing would not reveal
the problem when cross-compiled on newer machines...
CPU detection: Cosmetic (comments, and rename a confusing label).
Bugfixes for -test -mask.
OpenCL SHA-1: Concentrate an AMD workaround to within the SHA-1 header
only.
MPI bugfix for OMP fallback.
[pass_gen.pl] allow specified -salt shorter than 8.
Bugfix for vectorized use of OpenCL SHA-1 after recent change.
Much nicer code for previous commit: Use __typeof__() keyword.
WPAPSK-OpenCL: Revert to using AoS, for working around AMD bugs.
Closes #1905.
Fix a couple of problems with --pipe mode when using tty for input.
Windows fix for "label at end of compound statement".
HMAC-SHA256: Support up to 183 characters of salt. Closes #1904.
Reduce (default) chatter from descrypt-opencl.
Allow muting status output for GPU temp, utilization and fan
individually.
Fix a bug in loopback mode that caused a segfault in case the
assembled LM halves alone would crack 100% of the hashes.
SAP G bugfix. See #1917.
SAP G, add headroom for 4-byte UTF-8. Closes #1917.
[listconf.c] Move --list=opencl-devices to listconf_parse_late() so we
can get things from john.conf.
Bugfix in NTLMv1/MSCHAPv2 naive formats. Closes #1923.
Scrypt (Jumbo-specific input format) and PBKDF2-HMAC-MD* formats:
Avoid using dupe test vectors. They triggered false positives in the
Test Suite. Closes #1913.
Fix a rename in 9215a07e that ended up incorrect.
cryptsha512_kernel_GCN.cl was renamed to "unused" instead of moved
to "src/unused/".
Fix a clash between mscash2-opencl multi-device support and "Poor
man's multi-device support". Closes #1920.
Allow LDAP-style raw MD5. See #1613.
Drop nsldaps-opencl and unify salted-sha1-opencl with shared code. See
#1613.
Drop nsldap-opencl and unify raw-sha1-opencl with shared code. Closes
#1613.
Update CHANGES-jumbo.git and CREDITS-jumbo. See #1911.
Workaround for Intel for our use of typeof() in OpenCL.
Try not to fail if we can't install fglrx-dev.
Bump OMP_SCALE for PHPASS. See
http://www.openwall.com/lists/john-dev/2015/12/06/1
Slight change in -list=format-all-details output to avoid confusion.
Use a "fake db" for self-test and benchmark.
NT-opencl: Drop all special code for self-test, and use the new "fake
db".
Pass db to OpenCL auto-tune, use db->salts in all crypt_all() calls.
mscash-opencl: Drop special code for self-tests (using db from test
vectors)
Fail self-test if we can't find a test vector salt in the test db.
Bugfix and problem workarounds for ldr_init_test_db() "fields"
assembly.
Add a hack for muting messages from oldoffice-opencl's source() while
loading test db.
Fix a (benign) memory leak in cryptsha256-opencl. Closes #1945.
[unrarfilter.c] free() -> MEM_FREE(), closes #1942.
Unicode/codepage improvements for letters that doesn't have any case.
If one of -min-len or -max-len is given to mask mode, give the other a
default of format's min/max length. Closes #1947.
Add memdbg.o dependencies. Closes #1944.
Add a couple more memdbg.o dependencies. Closes #1944.
Correct fix for f3113f21
Tweak some macros in shared OpenCL code.
NT: Do not use local memory for OS X with nvidia. The actual criteria
is unknown, but this was seen better on a Macbook with GT 650M.
Driver bug workaround for GPU-side UTF-8 conversion on OSX with
nvidia. Unfortunately this halves the performance, but only when
UTF-8 is used and no internal codepage is active. Closes #1857.
[common-gpu.c] Increase a couple of printf buffers, and drop some
orphaned code. Closes #1950.
OpenCL sensor status: Increase another sprintf buffer, avoiding
possible overruns. See #1950.
hmac-SHA256: Move a test from split() to valid(), and other minor
fixes.
Add README-DISTROS for more detailed fallback build instructions. Also
tweak strip and install targets as well as autoconf handling of
--prefix option. Closes #1656, closes #982.
Usage blurb: Move -test-full to "hidden" options and update its
wording. Move -costs from "hidden" to standard options (along with
-salt et al).
wpapcap2john: Add option for manually specifying ESSID:BSSID pairs in
case there are "hidden" networks or your sniff file lacks beacons
for some other reason.
Wordlist: Speed up fix_state when memory buffered. See #1955.
Add some comments in unrar.c documenting a false positive from ASan.
Closes #1962.
loader.c: Do not prune test_db from --cost. Closes #1961.
loader.c: Do not prune test_db from --salts. See #1961.
Increase verbosity level needed for testdb output.
Avoid undefined behavior for signed integer in rpp.c. See #1963.
Reduce chatter from descrypt-opencl and also introduce some macros for
verbosity levels. Closes #1968.
Bugfix for some cases where self-test messages over-ran buffers.
HMAC-SHA384/512: Bump max salt length and support JWT hashes. Closes
#1898.
Update INSTALL-UBUNTU document not to use an older rexgen version
anymore.
Fix some whitespace that was thrashed by f3cd0e63
Allow $HOME to override the home directory (eg. ~/.john) in
JOHN_SYSTEMWIDE builds. Closes 1901.
The length sorting in RAR3 was ineffective unless we bump mkpc a lot.
OpenCL: Turn an Apple-specific fallback into a generic one. Some
clustering solutions may prohibit use of vendor-specific extensions.
--list=opencl-devices: Add AMD wavefront width.
Use realpath() for evaluating absolute path for OpenCL -I./kernels.
This is needed for some cluster environments like VCL but might
break other use cases. We may want a john.conf option for it.
Common OpenCL code: Decrease scope of platform struct, and some other
minor tweaks.
Common OpenCL code: Bug fixes and tweaks for VCL. Closes #1989.
Fix bug in default GPU selection introduced by 0a026d5. Closes #1995.
Mask.c, fix multiple problems with progress/ETA. See #1996.
doc/MASK: Document the special ETA behavior when iterating over
lengths. See #1996.
Conditionalize an AMD workaround. Closes #1994, see also #1497.
Change all use of __ARM_NEON__ to __ARM_NEON. See #1998.
RC4 bugfix for decrypting more than 256 bytes with RC4_single().
Unicode: Minor bugfix for detection of invalid UTF-8 sequences. Also
drop some local copies of valid_utf8() in favor of the shared one.
leet.pl: chmod +x
cracker.c: Remove sourcing of SSE2 header obsoleted by recent merge of
master branch.
loader.c: Unify ldr_init_password_hash() with John Proper (some
variable names differed).
Drop pristine ADL SDK files from our source tree and provide our own
header, supporting NVIDIA as well as AMD and free to distribute
under the "fair use" doctrine.
Support ADL on Windows (untested).
undrop bug fix: hash length is length 13 including the '+'. Closes
#2011, thanks to shrikus.
OpenCL: Generic workaround for OSX driver bug. Fixes many formats.
--list=opencl-devices, add SIMD width for AMD devices.
recovery.c: Avoid a race condition between closing a finished session
file (which releases our lock on it) and unlinking it.
Some tweaks for 1be1841: Support $JOHN path, set execute bit on
script, drop length limit and decrease score of variables.
Revert "Some tweaks for 1be1841: Support $JOHN path, set execute bit
on"
Oldoffice formats: Fix for --show vs. legacy MITM data in input files.
pass_gen.pl incorrectly required Crypt::ECB to be installed even when
not needed.
hmac-MD5: Bump max. salt length to 183.
Oldoffice: Optimization for dupe salts, similar to what we do for
WPAPSK but with a lot less gain.
Don't try to use typeof() with Intel OpenCL.
office2john: Handle old Word "0Table" vs. "1Table" properly.
Fix a problem with sequential dupe suppression in rules, when internal
codepage is used.
Fix for rules' consecutive dupe suppression of candidates containing
non-ASCII characters when using --internal-codepage. See 4faaba9.
Do not delegate hashes to generic crypt format unless specifically
requested. Closes #2069.
External mode filter: Support UTF-32.
External mode max-length fixes. See #2072.
External mode: Code style fixes and avoid unneeded differences from
core.
Drop support for deprecated -internal-encoding option (replaced by
-internal-codepage) except we still recognize it in restored
sessions.
Bugfix for --skip-self-tests. See #1967 and closes #2090.
CUDA fixes for macro-redefinitions. Problem likely introduced in
ad2e4727.
Pot sync performance fix: Use strnzcpy() instead of strncpy(). This
was found by comparing a run with LINE_BUFFER_SIZE at 0x400 instead
of 0x30000.
PRINCE: Allow a negative max element count. This will make the
effective max depend on each password length. For example, using
-prince-elem-cnt-max=-2 the effective max will be 3 at length 5, and
4 at length 6.
Mask mode: Tweak mask-stretching. Hopefully addresses #1983 a little
but does not fix all situations.
Wordlist mode: Ignore UTF-8 BOMs. Closes #2138.
Fix long standing bug in --pipe mode vs. encodings.
Implement --max-run-time-reset option that complements
--max-run-time=N. The new option resets the timer on each crack so
session aborts after N seconds of not cracking anything. Closes
#2139.
Drop the --max-run-time-reset option and instead include that
functionality in --max-run-time=N where N is a negative number. See
#2139.
OpenCL: Recognize nvidia Pascal series (ie. the cores per MP count).
Drop some noise from dlopen definitions.
Add number of PCI lanes to --list=opencl-devices.
fgetll() needs to use size_t for size, not int.
Quick-hack for rar2john so it always inlines.
Temporary (?) hack to speed up fgetll() considerably. We switch to
using realloc and we also double the chunk size each time. The con
is we may end up allocating almost twice the needed memory, worst
case. However, that's just virtual memory and the over allocated
memory would never be touched.
Minor tweaks to logger.c and loader.c: Do not depend too heavily on
LINE_BUFFER_SIZE and avoid undocumented arbitrary figures for
margin.
More formats fixed for truncated pot lines. This includes OpenCL
versions.
fgetll(): Safer realloc, and don't let the block size exceed 128 MB.
Also WAY faster (incremental) strlen.
Get rid of $SOURCE_BIN$. Pot entries are now always shorter than
LINE_BUFFER_SIZE (including the field separator and plaintext).
keepass2john & zip2john: Always inline. Also some clean-up of rar2john
that had a quick hack earlier.
Bug fixes for handling oversized sources.
Bugfix for zip2john for arbitrary length files. This is somewhat of a
hack but if the alloc fails we can't recover anyway. We'll just
 crash and burn.
rar2john bugfix
Replace the ZIP optimization that was lost in 528e6bc, for a 3x boost.
Also implements the same for OpenCL (we never had it there). Closes
#2141.
Apparently Pascal has 128 cores per SM, not 64. This explains why I
couldn't get the figures to sum up. Still, I see many references to
64 when googling.
Potsync fixes for truncated pot entries.
Revert "crypt: problems with #defines and strdup"
Avoid warnings if re-defining feature macros.
Fix CUDA build after e78d6ef, closes #2157.
Fix a long standing bug with -< rule reject when -min-length is not
used. This bug had little consequence until Jim implemented the
Hashcat rules recently.
Recovery.c: ASan "read" bugfix.
rules.c: Do not opportunisticly enable Hashcat rules mode. Use a safe
"nullstring" buffer for reading past (ASan "read" problems). And
define out a whole section that handles pre-processor stuff (should
never be encountered anyway!).
o5logon: Verify correct password, support Oracle 12 and early-reject
wrong-length candidates where possible.
kerberom.py: Move shebang-line to top (otherwise it has no effect) and
chmod +x.
Fix a silly bug affecting resumed sessions. A few values set from the
config file (notably options.reload_at_crack) was zeroed on
resuming. A similar bug was fixed in 6a543435 (reducing the memset
size) and I should have spotted these extra resets at the time.
Alas, I did not.
Re-enable some HC rules parsing without explicitly setting hc_logic,
and drop an obsolete section of code. This is a partial reversion
and partial continuation of e04c0c8.
VMS/UAF: Avoid portability problems with VMS macros containing '$'.
Closes #2154.
Add usage blurb and docs for providing rules at command line for
single mode. See #2105.
Revert "base64_convert.[ch] More work on size_t. commmon.[ch]
reverted"
Revert "base64 (and common) changed int into size_t where needed"
Optimization for --loopback mode when hitting old, very long pot
lines. See #2160. We now return a null string instead of a truly
useless portion of truncated cipher text.
loader.c: Use an existing macro instead of mimicking it.
Don't use abs() on return from hexlenl(). The latter now returns
size_t.
OpenCL: Don't try to enable cl_amd_media_ops unless it's actually
supported by the driver (eg. MESA does not).
OpenCL: Avoid using 'inline' with MESA in some cases. See #2204
Change algorithm_name of raw-md5u from "unicode($p)" to "utf16($p)" so
it matches other formats (eg. dynamic_29).
Briefly document the --format=@keyword syntax.
--show=invalid was no longer working correctly.
hmailserver and pkzip formats: Fix whitespace violations, some of
which bad enough to result in compiler complaints. Supposedly closes
#2217.
Recognize MESA even if it's written as Mesa. See #2204.
Drop some "tedious truths" from OpenCL common code.
WPAPSK format: Whitespace cleanup.
jumbo.h declared our own atoll() in a way that could clash with
 system's. See #2198, #2201.
CUDA wpapsk format had a borken adoption to new FMT_MAIN_VERSION in
9423af68. Even before that, the format was borken all the way since
f024c0a6 due to loader.h being inadvertently sourced (the rest of
the bogus changes were reverted but not the #include) and CUDA can't
cope with loader.h for some reason or the other. Closes #2198,
#2201.
zip2john: Add support for CRC and lengths only stored in the Data
Descriptor following the file data.
Blind try to fix a MinGW problem in CircleCI.
Makefile.legacy: added jumbo.o to genmkvpwd for non-atoll systems.
#2220
gpg2john: Drop dependency on LINE_BUFFER_SIZE. But we still have a
hard deck defined as BIG_ENOUGH (@kholia[tm]) which was also bumped
from 8192 to 64K. See #2218.
Ensure static unicode arrays are declared as external anywhere but in
unicode.o. See #2086.
Don't use well-known name DECLSPEC for other purposes than people are
used to. Changed to EXTATIC meaning either static or extern ;-I
DEScrypt-opencl: Get rid of a few compiler warnings from picky
run-time compilers (eg. OSX).
Drop an unused variable from salted_sha_kernel.cl.
ZipMonster format: Tried to get an aligned auto variable. We know that
isn't safe. Closes #2202.
OpenCL: Handle spaces in include path. Closes #2222.
Some no-op changes to unify raw MD4 and MD5 OpenCL formats for easier
comparison (trying to nail #2181).
Ensure no format ever processes > 4G keys per crypt (including
GPU-side mask) because the *pcount return can't hold it. Closes
#2181.
Mask mode: Fixes for "-test -mask", closing #1828 and #1835. Also
fixes some other glitches.
Argon2 formats: Do not print diagnostics unless verbosity was bumped.
argon2 files: Fix memdbg (plus whitespace violations that my emacs did
automatically while at it).
Drop the borken idea of $BIN_HASH$ yet again. Do not add it again,
it's NOT THE SOLUTION.
Use $SOURCE_HASH$ truncation of test vector's ciphertexts in
--list=format-[all-]details. Prior to this commit, they were simply
chopped to 256 or 512 bytes.
Fix for $SOURCE_HASH$ with formats having a binary that gets lost or
truncated when "pot-chopped".
Fixes to formats that may get their binary chopped by $SOURCE_HASH$.
See #2117. Note that some formats don't need fixes because their
binary comes early enough that it won't get truncated - one example
is the SXC format.
Fix pot sync for truncated pot lines. This code was missing from
#2196.
Revise verbosity levels. Closes #2230.
Fix Redmond whitespace violations in potcheck.pl, for crying out
loud...
potcheck.pl: Bugfix for long lines, enable verbose option, disable
undocumented/unimplemented options, add verbose output.
rar2john edge case fix, for RAR3.
OpenCL formats using "perfect hash tables" can't be used with single
mode cracking. Bail out with an error instead of potentially hanging
indefinitely. Closes #2232.
Alternate hybrid external mode 'HybridLeet' by Christien Rioux (See
#2099).
Revert a hideous bug Jim accidentally planted in my reference $0$
format in 8c48877. This wasted hours of my life searching for
apparent core bugs and the format is forever ruined for use with git
bisect (which was its primary use). Bad_day.mpg
Fix for MESA trying to evade detection. See #2204.
OpenCL: Device-detection fix for MESA. It gives us upper-case device
names. See #2204.
OpenCL: Refactor a reused codepage macro and move to opencl_unicode.h
OpenCL: Better shared-code support for nvidia Pascal.
Fix for another bug introduced in 8c48877.
Renamed oracle9i_fmt_plug.c to o3logon_fmt_plug.c (no other changes).
o3logon: Change format struct name. Fix some potentially expensive
code in valid().
Add a 'dump_stuff' debug macro for *char to opencl_misc.h.
ntlmv2-opencl: Preparation for gpu side mask. Drop vectorization
support. Reduce to a single kernel. Move the final hmacs to a
separate function.
ntlmv2-opencl: Implement gpu side mask & compare. Speed now more or
less on par with Hashcat, despite our format fully supports Unicode.
pass_gen.pl: Fix a UTF-8 bug introduced in 27ecb1d.
pass_gen.pl: Proper code for allowing user to say -codepage=UTF-8
instead of the short-cut -utf8 (which worked).
Autoconf: Fix a bug introduced by Jim in 154314c causing int128 to be
always disabled.
config.c: Avoid multiple warning/error output with fork/MPI
OpenCL: Minor tweak to GPU-side mask auto-tune output.
zip2john: Fix a bug introduced in 744cdb91 by yours truly: File magic
could not be used.
oldoffice-opencl: Some optimizations for good measure. Not much
difference.
GPU-side mask formats: Fix an edge-case bug.
pass_gen.pl: Again, don't require obscure libs unless actually needed.
This is a quick fix, the real solution is not to depend on these at
all. Problem introduced in 030e303.
OpenCL mscash & ntlmv2 formats: Drop a few redundant lines.
o3logon format: Use a truncated salt instead of segfaulting in case
user runs a codepage-encoded input file with encoding set to UTF-8.
See discussion in #2243. Closes #2243.
ntlmv2-opencl: Simplify code after recent change.
Many OpenCL formats: Change comparisons with constants according to
coding style. Nothing effectively changed, just easier to read.
pkzip format: Bugfix and improvement of SWF file magic.
john.conf: Update AMD driver info. Apparently we can't tell 15.11
(beta) from 15.12 (release) with just OpenCL queries. I presume the
OpenCL part of them is identical.
OpenCL: Fix a bug with Global_MaxDuration. See #2204.
ZIP formats may have their binary truncated in pot file, so can't use
hash funcs. Closes #2245.
Some more formats that may have their binary truncated in pot file, so
can't use hash functions. See #2245 et al.
Add a self-test for catching formats that may get their pot entries
truncated, but use get_hash(). See #2245, #2117 et al.
Last bunch of formats (that the self-test could detect) that may have
their binary truncated in pot file, so can't use hash functions. See
#2245, #2117 et al.
OpenCL RC4: Very odd way of gaining a 4x boost, nicked from Hashcat.
oldoffice-opencl: Prepare for GPU-side mask/compare.
oldoffice-opencl: Implement GPU-side mask. Speed now more or less on
par with Hashcat, despite our format fully supports Unicode. #1845
oldoffice-opencl bugfix: result was not cleared after MITM hit.
NSEC3 bug fixes. An iteration count of zero is valid according to the
RFC. Our crypt_all() would loop for ever if given zero. Closes
#2258.
Ensure reset(realdb) is called with --skip-self-tests. Closes #2256.
See also #2090 and #1967.
Fix some formats that used prepare() for things better done in
split(). Some of them even had the functionality in both. See #2254.
SAP B format: Don't output encoding warning with --list=format-tests
Fixes for prepare/valid encoding checks/conversions. See #2252.
as400-ssha1 format: Add a UTF-8 check in valid() since Convert()
depends on correct encoding being used.
Drop a spurious asterisk from AS400-DES format tag.
Add OPT_REQ_PARAM to a bunch of options that missed it. Closes #2260.
o3logon format: Drop the prepare. Closes #2254.
formats.c: Fix a problem with gpu-side mask formats, introduced in
1a666cb. Tricky business.
Opencl formats: Typos in comments.
krb5pa-md5-opencl: Prepare for gpu-side mask/compare.
krb5pa-md5-opencl: GPU-side mask & compare.
More optimized early-reject for krb5pa-md5-opencl. This changes our
shared RC4 to no longer hard-code length of decryption at compile
time which also affects the oldoffice-opencl format, but this had no
impact on performance.
Update copyright info in OpenCL SHA-1 formats (drop the GPLv2).
There's nothing left of Samuele's initial work in them nor is there
anything of significance from the projects he referred to.
OpenCL raw-sha1: Optimizations. Swap binary once before starting
crack, instead of wasting time doing literally 25 billion swaps per
second for the whole crack run. Also unify raw-sha1 with shared
sha1.
mysql-sha1-opencl: GPU-side mask & compare.
OpenCL: Drop the option USE_SHA1_SHORT (we always use it).
lotus5-opencl: Assorted changes trying to nail problems. #1726, #2242.
Refactor shared OpenCL primitives functions.
RAR-opencl: Use shared SHA-1 code.
Bug fix for memdbg builds: The exit checks were never made if session
ended with an abort or --max-run-time reached. This seriously
limited its use when debugging. Closes #2264.
Refactor shared OpenCL primitives functions (MD4, MD5).
On resuming, print brief info from main cracking mode.
c5c1fd3a broke RAR-opencl for OSX.
OpenCL shared SHA1: drop unused variables.
Add yet another "format class" --format=all that overrides anything
else (eg. [Formats.Disabled] in john.conf) so you know every single
format will be tested/listed regardless of conf.
Make --format:all more versatile. It now also support things like
all-gpu, all-@sha1 and so on.
as400-ssha1 Unicode test was made too early. No need to complain if
we're not seeing "this" format. See #2252.
office2john.py: Pure whitespace commit (spaces -> tabs).
office2john: Add handling of a specific problem, so we can continue
with other files instead of aborting with error. I'm not sure the
problem is with corrupt files or a bug in office2john.
pass_gen.pl: Errors and warnings should go to stderr.
pass_gen.pl: Drop/change some spurious old "print" which should now be
a return string to unified output function.
pass_gen.pl: Pure whitespace fixes (spaces -> tabs).
Mask mode: Correction for an ETA problem when running non-hybrid,
 fixed length, with node/fork AND with a mask that get truncated due
to max length. And only then, whoa.
Tweak verbosity levels again. We lacked a level between "normal jumbo"
and the lowest (1, which even suppress cracks). This new scheme is
more future safe: There is currently no level "> VERB_LEGACY" and "<
VERB_MAX" but we can easily just bump the max to 6 if we need one.
office2john: Revert 67d2d385 and also fix .editorconfig so any python
files are specified as having silly spaces-indentation.
Some tweaks for recent chmod change. Simplify code, use POSIX macros,
require rw for owner because eg. 0066 will make it unreadable by
yourself. Rename IgnoreLogChmodErrors to IgnoreChmodErrors since it
affects both.
logger.c: Code-style fixes, mainly over-long lines.
e50f748c missed related changes in john.conf.
logger.c: Portability workaround for Windows.
logger.c: Portability workaround for Windows, take 2.
wpapcap2john: Handle pcaps which lack beacons but do have probe
responses. They look the same as a beacon (for our purposes) so very
easy fix.
wpapcap2john: Minor cosmetic changes (diagnostics, comments).
MPI: Show node name with OpenCL device info.
misc.c: Fix a long-standing #ifdef-logic bug with MPI vs. fork causing
pexit() to segfault due to a dangling 'else'.
Fix alignment in usage blob (problem introduced in 347702b).
regex.c: Fix a new warning introduced by be39643.
AC: Add a test for sizeof(wchar_t). Note: If that type isn't available
at all, size will be defined as 0.
Unicode engine: Don't write *pointers as * multiplications, that
sucks. This commit is whitespace-only, no real change.
Regex mode: Fix for recent changes in librexgen. We now properly
convert to wchar_t. Closes #2296.
CUDA: Fix various problems with OSX/macOS.
Rexgen: Handle internal codepage correctly. See #2296.
Unicode/wcs bugfix for exotic systems.
Drop artifact from b64645f7.
When populating test_db, don't honor --user and --salt options. See
#2305.
Revert a change in mask.c from 6fb81e0 that introduced a bug.
mask.c: Bug fix for resuming iterated length. See #2306.
Replace our 7z2john.py with 7z2john.pl (pristine 7z2hashcat.pl)
Adapt 7z2john from pristine 7z2hashcat
7zip formats: Allow any data size (drop the idea of BIG_ENOUGH). See
#2314.
Bump SIMD max. salt length in HMAC-SHA256 format to accomodate for
some JWT tokens. See #2318.
Minor fix to office2john. See
https://hashcat.net/forum/thread-5739-post-32191.html#pid32191
Raw SHA-1 formats: Revert to unifying to hex, not Base-64. It had
several problems (could not --show old entries, not compatible with
dynamic_26, not compatible with Hashcat and so on).
Mask mode: Add ?h and ?H for hex digits (hashcat compatibility).
Closes #2324.
mask.c: Fix a silly bug introduced by c19f75fb9
Fix for rar2john buffer overrun problems. Closes #2337.
Revert "OpenCL formats using "perfect hash tables" can't be used with
single"
recovery.c: Re-position a comment that was pushed away from what it's
referring to.
Fix weird bugs introduced in c79fce1, messing salt resume up. You
 can't look at salt->count, that's the number of hashes for *this*
salt.
Drop the raw SHA-0 format. It's unlikely anyone will ever need it.
Closes #2342.
Missing update in autoconfig.h.in related to FPGA support.
Drop CUDA support. Closes #2348.
Some more cleanup after dropping CUDA (mostly cosmetic).
Some tweaks to incremental mode vs. --min/max-length options. See
#2316.
Fix a hideous bug in CPU version of WPAPSK format concerning new_keys
flag. It was only set in clear_keys() but not in set_key(). This is
not enough when running Single mode.
Fix two memory leaks in zip2john. See #2219.
Drop CUDA artefacts from doc/INSTALL*
Fix more memory leaks in zip2john. See #2219.
Fix more memory leaks in zip2john and stop using alloc_tiny. See
#2219.
Add new option --single-seed=WORD. This is incredibly effective for
eg. web dumps where some word/name is very common as part of the
passwords. Eg. for the LinkedIn dump you'd use
--single-seed=linkedin together with normal single mode.
Document the new --single-seed option.
encfs_common.c: Fixes for compatibility with OpenSSL 1.1. See #2355
Unicode/codepage fix for --single-seed option.
Bugfixes for Unicode/codepage support vs. single mode.
Change --single-seed option to only allocate global words once. Also,
never create initials or word pairs from seed words. See #497 and
#2357.
Add --single-wordlist option as well as a john.conf list entry for
static single mode seed words. Closes #497 and #2357.
Automatically bump word pair limit for --single-wordlist and
--single-seed options to accommodate the list of words. Also update
documentation.
Fix a bug in leet.cc format: You can't free a "tiny alloc".
Drop all use of ARCH_WORD_64 (introduced in 61cea42 for no clear
reason) in favor of uint64_t. Closes #2361.
Some code clean-up in loader.c, and fix potential issues with
truncated pot lines. See #2366, #2367.
7z2john.pl: Upgrade from upstream 7z2hashcat.pl. See #2314
7z2john.pl: Update from upstream 7z2hashcat.pl. See #2314.
Drop a whole bunch of obsolete stuff from "unused" directory. We
should be a bit more restrictive with archiving stuff there. Samples
should not be placed in the repo at all, we have a wiki page for
them. Docs should be in docs directory.
Add LZMA and LZMA2 decompression functions (from LZMA SDK).
7z formats: Add support for LZMA and LZMA2 decompression. Reduce false
positives to nearly zero and drop FMT_NOT_EXACT. Despite this we've
now got a number of optimizations for early rejection.
7z2john.pl: Update from upstream 7z2hashcat.pl.
7z: Bug fix for coder props parsing.
7z2john update from upstream.
Fix a recently introduced memory leak in 7z formats.
Memdbg fixes for new LZMA code.
7z formats: Avoid FP, ensure we round up to AES blocksize.
7z2john.pl: Update from upstream. Support added for SFX files. Closes
#2120.
Fix non-thread-safe code in enpass-opencl. Closes #2393.
7z2john.pl: Update from upstream.
Add generic OpenCL code for AES. Not used yet.
Enpass and Strip formats tweaks. They had unnecessary large buffers
and decrypted more than needed.
wpapsk2john: Drop hard-coded limit of ESSIDs supported in a batch
(realloc on the fly instead). Also search the database in reverse
order for quite some speedup on extremely large pcaps (eg. from
war-driving where the related data to a packet is more likely to be
near end). This was inspired by recent huge sample posted in a
Hashcat issue (955).
OpenCL AES fixes.
ODF-AES OpenCL format: GPU-side AES decryption. This was mostly made
just for testing and PoC but it turns out it made for a better than
4x boost on nvidia Maxwell (single CPU thread). AES code is still
not optimized at all and there's a couple of SHA-256's we could move
to GPU as well if we wanted to.
Add a mem_realloc() function just for good measure. Some uses of
realloc can't use this because they are in external *2john tools
that don't link memory.o. Others have their own special handling.
Closes #2365.
OpenCL AES: #ifdef out AES_cbc_encrypt so we don't get complaints for
unused functions.
7z2john.pl: Update from upstream.
Shared SHA-2 OpenCL code: Bugfixes. Protect macro arguments and ensure
second-level macro gets its W variable correctly.
plugin_deps.pl: Support OpenCL kernel sourcing other kernel(s).
OpenCL ODF-AES format. Try a new approach for using several kernels,
and now do *all* work on GPU (and drop OpenMP). This yielded another
near 4x boost and including recent 4x boost we are now near 15x the
original speed. Closes #2400.
7z-opencl: GPU-side early rejection (check AES padding). Closes #2398
7z-opencl: Minor fixes.
7z formats: Add padding size and compression type as "costs".
agilekeychain-opencl: Move all work to GPU. Closes #2403
Correction for agilekeychain algo name (missing in last commit).
bench.c: Bugfix for dyna salt when 1st salt is reused (would
segfault).
Enpass OpenCL: GPU-side AES and verify. Closes #2404.
Drop an unused variable in enpass kernel.
Fix for a segfault in dynamic prepare() during pot reload. I'm not
quite sure who's to blame but I fixed it in crk_reload_pot(). Closes
#2415.
Enpass kernel: Fix a race condition problem with the "any cracked"
bit.
7z2john.pl update from upstream. Support for split archives added.
Upstream c143795..fe800e0.
Globally fix about a thousand whitespace violations. Contrary to
popular belief, omitting spaces doesn't make your code faster. It is
actually a little harder than that. Also change a number of "char
unsigned" to the more friendly "unsigned char".
7z-opencl: 62cec9d5 introduced a false negative bug for the "many
salts" case. We did the early reject for all salts using data from
just first salt. Add a separate kernel for the AES stage and run it
per salt.
Fix race conditions and other bugs in recent AES OpenCL formats.
OpenCL agile/iwork/enpass formats tweaks / bugfixes.
7z-opencl: Drop an unused variable.
OpenCL AES: Add ECB interface.
Add volatile to variables handled with atomic functions.
OpenCL AES formats: Optimization just for good measure.
Enpass-opencl: Copy/paste bug
New Office OpenCL format joining the separate ones (#2424)
Allow building against OpenSSL >= 1.1. Formats that will be disabled
are currently GPG, GPG-opencl, KRB5TGS, PDF, PFX and WPAPSK (CPU
only). Closes #2355.
Add build-time warnings for disabled formats. See #2355.
Drop PFX format in favor of PFX-ng and rename the latter (but not its
format tag). Closes #2431.
Drop OpenSSL dependencies from WPA-PSK format. Closes #2434.
GPG formats: Support OpenSSL 1.1. Closes #2430.
AES fixes for internal AESNI-aware code vs. OpenSSL.
GPG format needs OpenSSL for AES_cfb128_encrypt
Add a self-test for -opencl suffix in label being lower case as
required. See #2436.
Zip-opencl: Drop AES from algorithm name, it's not used!
o5logon-opencl: White-space fixes (no-op).
o5logon-opencl: GPU-side AES and verify. This made a 7x boost.
OpenCL formats: Sod using atomics. There simply is no safe way to
initialize the variable to 0 other than to do it in a separate
kernel or transfer it from host side. These formats aren't fast
enough that it matters anyway. See #2439.
Various minor OpenCL optimizations and bug fixes.
ssh-ng bugfix for truncated pot entries. See #2457.
AS400-ssha1 format: Don't use identical test vectors with different
syntax. See #2457.
External HybridLeet mode: Case-toggle letters that lacks leet
alternatives. Closes #2461.
Do not allow a hybrid external mode applied to single mode. Closes
#2462.
Add a --max-candidates option.
Fix weird bugs introduced in 0ceedb2 including --make-charset not
working at all.
Let new -max-cand option also affect -stdout
Followup work for new -max-cand=N option. Closes #2475. * Change the
"Session aborted" message when session was stopped due to max.
reached. * Allow resuming such stopped job (for processing next set
of N cands). * Document the option. * Avoid silly portability
problems with %lu vs. %llu for uint64_t.
Some updates and white-space consistency for doc/OPTIONS.
-max-cand=N option can now take a negative number. That means the
"count" will reset on each crack (so it will terminate after N
candidates are tried since last crack). See #2475.
Set CRLF line feeds for john.asm in .editorconfig. See #2484.
Drop our local stdint.h
Drop obsolete typedefs.
unicode.h: Use stdint.h types
Makefile.in: Drop a stray "stdint.h" dependency for bitlocker2john
that probably got in when merging.
Travis CI: Allow OpenCL failures
Fix a bug where --show=left only showed statistics (but no hashes)
unless at least one hash was already cracked. Closes #2506.
keepass format: Move sharable stuff to a separate file, in preparation
for an OpenCL version.
zip2john: Support 64-bit sizes in zip version 4.5. See #2528.
Assorted 64-bit-size fixes for zip/pkzip formats. See #2528.
Fix a bug in fgetll. We somehow overlooked that fgets' size arg is an
int!
Fix a pkzip problem with sizes exceeding 32-bit signed int.
Do not trust AES padding as early-reject for 7z (unless that's the
only thing we can do due to truncated data). Closes #2532.
Oldoffice formats: Fix a bug caused by using source() in a salted
format. This is simply not supported. Add a self-test to stop us
from doing the same mistake in the future. Closes #2535.
Add some better information for password max-length in case of UTF-8.
We now express it in characters, not bytes.
DMG-opencl: Change max. duration to be less strict.
Bug fix for new length reporting. See #2529.
Tweaks for new length reporting. See #2529.
OpenCL: Whenever format have a notion of "cost" and we're auto-tuning
with (#2544)
SAP format fixes for max. length.
Bug fix for recent auto-tune-with-real-db.
Minor cleanup in 7z formats.
Length reporting tweaks for LM formats.
Hard code SAP/B length reporting for non-UTF-8 and set FLG_8_BIT
again. Closes 2545.
Don't start "the clock" until all loading, auto-tune etc. are done.
This ensures stable speed readings from start, and things like
--progress-every and --max-run-time will work better.
NT-opencl: nvidia bug workaround. Closes #2335.
Add a separate .editorconfig for src/ztex subdirectory, reflecting the
odd mix of line endings. Closes #2484.
pkzip format: Fix a bug in salt compare size and add a note about
"same password assumed within one archive". Closes #2542.
Fix a couple of obscenely promiscuous or slow prepare() functions.
There's many, many more to fix.
Apply nvidia bug workaround in three more formats. See #2335.
Add a format flag FMT_HUGE_INPUT telling us that a format supports
huge input lines (ie. longer than LINE_BUFFER_SIZE) and thus
may/will truncate its pot entries.
OpenBSDsoftRAID format: Drop a dupe test vector so we don't confuse
the test suite.
Update CREDITS-jumbo and CHANGES-jumbo.git
Revert some of changes from 272c664 for the net-md5 format. They were
needed for tag translation (I thought it was merely about supporting
bare hashes).
Git .mailmap for e-ago
Revert some of changes from 272c664 for the net-sha1 format. They were
needed for tag translation (I thought it was merely about supporting
bare hashes). Closes #2554.
Bug fixes and workarounds avoiding -Wformat-truncation warnings from
gcc-7. Closes #2533.
Style fix, no-op: Change all '# define' -> ' #define'.
Whitespace violation fixes, no-op.
Code cleanup: Drop a few unused macros (but not in borrowed code).
Serious performance bugfix for fgetll() in edge cases.
Add a makefile target for producing test files from all formats.
Bugfix for the fgetll bugfix in 07ddb4f7. Also never allocate less
than 4K.
Fix for uninitialized variable. Closes #2556.
Fix for uninitialized variable in fgetll(). Silly me.
Pot sync: Emit a warning if pot file shrunk. Merely seeking past end
does not detect that.
RAR OpenCL format was lacking FMT_HUGE_INPUT.
7z formats was lacking FMT_HUGE_INPUT flag.
OSPF format is FMT_HUGE_INPUT, electrum and multibit aren't.
Fix a bug introduced in 827df441.
Don't allow gcc 7.1.1 to complain about perfectly fine fallthrough
code.
Fixes for -Wold-style-declaration.
Various fixes for -Wextra
Add test files (from 'make testfiles') to .gitignore
DMG OpenCL: Use split kernel. Closes #2564.
Add SL3 CPU format. Closes #731.
Add a silly kludge for a silly test. See
https://github.com/openwall/john/commit/e4795d6#commitcomment-22477620
Buffer overrun fix for SL3 format in non SIMD builds.
Revert "Using the new multibyte character aware API from rexgen
(#2566)" (#2569)
SL3 ensure correct string termination in non-SIMD code.
Add SL3 GPU format. See #731.
SL3-opencl: Correction of format flags.
SL3 formats: Ensure we self-test prepare() function.
Ethereum format: Set FMT_HUGE_INPUT. See #2568.
SL3 formats: Share common code.
OpenSSL-enc format: Get rid of OpenSSL EVP calls. Apart from being
more portable we also got a 25% speed boost. See #1686.
Get rid of OpenSSL EVP in encfs formats. See #1686.
Drop artefacts from EVP and BIO high-level crap that we have gotten
rid of. Some of it were just comments now but they were removed or
modified to not hit a grep. Closes #1686.
mscash formats: Fix bugs for parsing salt containing '#'. See #2575.
ztex .editorconfig update after 0bfd564c
Fix some run-time OpenCL compiler warnings for ethereal-presale, plus
other insignificant changes.
PFX: Add a test vector with a non-ASCII password.
Ethereum-presale OpenCL: Auto tune tweaks. Aim for similar durations
of loop and post-process kernels. Avoid uint8_t and size_t in
kernels.
OpenCL SHA-1 (CTX version) clean-up.
PFX optimizations. Move the last HMAC-SHA1 to GPU (for crying out
loud?). Optimize/unroll some 8-bit operations. Use a quick raw SHA-1
for inner loop of PKCS12. Over 8x boost (that last thing made the
most). See #2176.
Update .gitignore with a couple of new entries.
Ethereum-presale OpenCL: Add a cast to pacify some runtime compilers.
OpenCL: SHA-2 AMD fix for OSX.
Optionally trust 7z padding for early rejection. See #2532 and
470766a.
Drop test for -Wno-implicit-fallthrough. For some reason, the cc
options check passes that even though it's not supported. Anyway
it's not needed unless we use -Wextra.
PFX formats: Add test vectors describing the need for #2606.
MS Money format: Correction for full UCS-2 support (strip 8th bit).
See #2607. Also some optimizations.
MS Money: #2607 8-bit fix: Assume ANSI codepage -> upper-case ASCII ->
strip 8th bit -> UCS-2
Regex mode now needs librexgen >= 2.0.6
Fix a problem with resuming a descrypt-opencl session. Closes #2662.
Fix a bug introduced in 0168a45a when using fork (not MPI). Closes
#2664. This might also affect #2547 somehow.
WPAPSK: Drop some dead code. There's little point in having a function
always return 0 and then test for other returns...
Improve WPAPSK valid() so we don't accept versions we don't support.
Closes #2678.
Fix a problem with possible AVX2 (et al) code before we
detect/fallback for CPU type. Closes #2679.
Remove some dead code from WPAPSK. Key is always 32 bytes.
WPAPSK formats: Add keyver as cost 1 = WPA (MD5), 2 = WPA2 (SHA1), 3 =
802.11w (SHA256)
WPAPSK-opencl: Add support (GPU side) for 802.11w / WPA-PSK-SHA256.
Closes #2648.
Add "PMF" to wpapsk algo names, indicating we support 802.11w
WPA2-PSK-SHA256
Sybase ASE format: Decrease max. length to 30. Closes #2686.
Bump RULE_RANGES_MAX from 16 to 30. See #2684.
Replace calls to abort(3) with our own error().
Fix a mismatch between (POSIX) sleep(3) and emulation of it for
Windows.
Replace guerilla usage of exit() to error().
QNX format: Move a valid check from crypt_all() to, wait for it,
valid()!
OpenCL: Bug fixes. The dreaded PGP-WDE bug that only showed up on
nvidias were caused by an optimized macro assuming data to be
aligned as integer. Most uses of that macro always involved aligned
data but the CTX versions of SHA-1 and SHA-2 may pass unaligned
data. The now added code to handle that will (hopefully) often be
optimized away.
OpenCL: SHA1/2 CTX also had alignment issues with output. Closes
#2710.
PGPWDE-OpenCL: 5% boost just be ensuring 'struct pgpwde_password' is
sized to end up aligned (124 + 4 == 128).
Oops, the previous commit obviously made a length 125 test vector
fail.
Tweaks for suppression of heavy logging. See #2717.
Bugfix for resuming a -stdin session.
OpenCL shared code optimizations (memcpy macro replaced with
functions).
Minor tweaks to pgpdisk-opencl. Avoid non-native widths like the
plague, reduce use of private memory.
pgpwde-opencl: Since host code hard-codes saltlen and key_len, do so
in kernel also. This allows the run-time optimizer to gain us some
5% boost.
Agilekeychain: Add the new test vector from f78f329 to OpenCL version
too.
krb5pa-md5-opencl: Add the new test vector from 4810534 to OpenCL
version too.
Electrum(-modern) OpenCL format: Correct algorithm name.
OpenCL shared code: Avoid build warnings on some platforms.
PGP OpenCL optimzation
Revert "Add support for Ethernet link type"
wpapcap2john: (Re-)add some diagnostic comments after previous
revertion.
eeacc1a merely hid an older bug instead of fixing it. Duh! Fix the
silly band-aid (add a warning print) and also fix the original bug.
wpapcap2john usage blurb: Clearify -e option.
wpapcap2john: Re-introduce support for Ethernet encapsulation. See
#2751.
Add hccap.h (preparation for #2753).
Add WPA-PSK PMK formats. Closes #2754.
wpapsk-pmk-opencl: Optimization for many salts.
wpapsk-pmk-opencl: Do not set FMT_CASE
hccap2john: Add support for reading hccapx files. Closes #2753.
Fix a problem where -max-run-time=N with -fork would loop signals so
some processes ended prematurely. See #2757.
Only add WPA 802.11w CMAC functionality if our OpenSSL has CMAC. This
does not apply to OpenCL formats, they do it on GPU. Closes #2763.
cracker.c: Ensure we never try to sync pot file if event_abort is
raised. This shouldn't happen anyway but was added as safety.
Remove dupe logic from x86-sse.h resulting in incorrect CPU_NAME for
anything beyond SSE4.1. See #2764.
Suppress OpenMP warnings for ztex formats. Closes #2771.
wpapcap2john: Improve ESSID gathering. Closes #2768.
wpapcap2john: Warn about silly tools like wpaclean.
hccapx2john: Update license/description & links.
wpapcap2john: Type led to build warning.
DEScrypt-opencl: Avoid printing silly message once per node.
Refactored some extremely confusing and misleading structure names in
wpapcap2john that had me make mistakes over the years.
wpapcap2john: Code clean-up (add alloc macros). Re-init more stuff
between each new file. Add some debug output in case of bad data
seen.
.mailmap update
Update CREDITS-jumbo and CHANGES-jumbo.git
wpapcap2john: Fix a long standing bug where too many outstanding
unverified auths wouldn't emit a realloc. Thanks to ZeroBeat for
finding this!
wpapcap2john: Cosmetic (diag) fix.
wpapcap2john: wpapcap2john: Add support for 802.11n HT field. Many
cosmetic changes (-v option for verbose output) and a few bug fixes.
Do not populate testdb if we're not going to use it. Also, fix for
repeated message using --fork. Closes #2792.
wpapcap2john: Warn about insufficient snaplen and speed up promotion
loop. Cosmetic bugfix for -v option: Some packets don't have a src
field.
wpapcap2john: Avoid double allocs/copies of same data, and refactor.
Fix indentation in hccap.h
wpapcap2john: Change bunch of "packetx" pointers to array "M[x]"
wpapcap2john: Detect and use M4 packets with intact nonce (in case we
missed the M2).
wpapcap2john: -r Ignore rep count
wpapcap2john: Safer handling of data, better handling of unverified
auths.
wpapcap2john: Bug fix for -c option (only use verified auths).
Loader and wordlist/PRINCE modes: Refuse to eat UTF-16 files. Wordlist
mode also had the UTF-8 BOM-skip extended to every line in the file
(for cases where luser concatenated several infected files). Also,
the checks were simplified/optimized, mitigating the performance
hit. Closes #2802.
OpenCL: Add a shared SHA-512 "CTX" (init/update/final style) kernel.
See #2801.
Do not suggest using --fork when user runs a non-OMP format with
-stdin or -pipe. Closes #2777.
Avoid running john-local.conf twice when CWD == $JOHN. Closes #2776.
wpapcap2john: Bugfix for -r (ignore replay-count) option. That whole
stuff is WIP. See #2773.
wpapcap2john: Bug fix and some refactoring.
wpapcap2john: Bug fixes and refactoring.
wpapcap2john: Minor fix for -v showing TS for hit.
wpapcap2john bugfix (cosmetic)
Mute a couple of OpenCL runtime build warnings. See #2810.
GPG-opencl: Add missing algorithms (cosmetic)
Make sure our definition of CPU name matches what was actually defined
with the cpuid tests. Closes #2764.
CircleCI tweaks. The log was completely unfathomable.
CircleCI bugfix
Fix a regression caused by 599a1674: The shared SWAP64 macro need to
be type agnostic (type may be vector).
No-op change: Unify whitespace.
wpapcap2john: Drop stray debug text.
Only bail on UTF-16 BOM's when input encoding is UTF-8. Closes #2812.
Travis CI seem to have a broken VM announcing AVX2 but not fully
supporting it. Disable AVX2 for now.
wpapcap2john: Correction of MAC address outputs in supplemental JtR
input file fields. We output STA MAC as uid and AP MAC as gid.
This way, user can select all hashes authenticating to a certain AP
using the --group option, or all hashes from a certain STA using the
--user option.
wpapcap2john: Fix the support for reading airodump .ivs files.
wpapcap2john: Switch to using stdint.h types.
wpapcap2john: Further tweaks to airodump-ng .ivs file support.
Relax the UTF-16 BOM check in wordlist and PRINCE modes to only warn,
not bail. The rationale is some low quality wordlists contain a mix
of encodings but likely mostly UTF-8. See #2812, #2802.
Tweaks for UTF-16 BOM check vs. MPI/fork (reduce repeated messages).
wpapcap2john: More tweaks. Output more unique handshakes, yet try to
avoid pure dupes. Fix an endianness bug that caused mishaps.
gpg-opencl: Get rid of build warning due to missing #include. Closes
#2816.
wpapcap2john, major rewrite: Keep state of AP/STA pairs instead of
just AP. This is needed for some elaborate attacks. Suppress
consecutive dupes from ivs parsing. Add EAPOL breakdown diagnostics
(-v -v -v) Improve option parsing. Improve checks of replay-count
(or not) and timing checks. Add support for outputting anonce-fuzzed
hashes where needed (happens automagically when needed, with
detected endianness). Add -d option (reduce dupe suppression, mostly
for debugging). Try to tell real AP's handshakes apart from the
attacker's. Lots of code clean-up.
wpapcap2john: Typo that for some reason only affected MinGW.
Move cygwin/ethernet.h to cygwin_ethernet.h because I love having
thousands of files in a single directory (not really).
wpapcap2john: Add -f <num> forced anonce fuzzing. This will fuzz +/-
<num> for both LE and BE (assuming incrementing last word of
anonce). Also tweaks for the auto-fuzzer (in some cases we can
detect fuzzing is needed even without -f option).
wpapcap2john: Handle (merely print in -vv) EAP Identity Responses.
Fix regressions after merge.
Use "-sj8" in all 'make' examples. See #2820.
wpapcap2john: Fix a cosmetic problem with -m combined with -vv. Fix a
BSSID source mixup with ethernet encapsulation. Fix handling of
handshakes with unknown ESSID (post-pone/warn and finally print
anyway, using MAC). Fix -e option that was broken for a while.
wpapcap2john: Indicate local (i.e. probably randomized) and multicast
MAC addresses (-vv).
doc/EXAMPLES: Update some old abbreviations the are no longer
unambiguous. See #2616.
wpapcap2john: Detect Ipv4 mcast MAC addresses.
wpapcap2john: Avoid wrong output of "WPA3" for WPA2 keyver 3.
Multibit format: tweaks for better OpenMP scaling (marginally
successful). See #2846.
Fix a silly bug where many formats used omp_get_num_threads() instead
of the intended omp_get_max_threads().
Fix a bug with non-OMP introduced by 00655ad63
Fix a few formats that failed to use the PH_MASK_* macros and thus
failed after recent core merge.
Fix a "bug" where dynamic did not adapt to new binary/hash sizes.
Fix about one ppm of the whitespace and/or code style violations.
Bump the default for mem-file-size to 2 GB.
Revert to using 1 GB -mem-file-size for 32-bit systems but keep it at
2 GB for 64-bit. Also update docs (and avoid future discrepancies).
OpenCL: macOS driver/runtime bug workarounds. Closes #2923.
Add JSON version of --show=types (as --show=types-json).
Mask mode: refactor a global variable name so we know its origin.
Mask mode: Bug fixes for min/max length and ETA/progress. Closes
#2882.
Markov mode: ETA/progress fixes. Closes #2799, see also #2779.
7z2john.pl: Update from upstream 7z2hashcat.pl (1.2, Nov 16 2017)
gpg2john: bug fixes for -DDEBUG builds.
--show=types-json: Output booleans and numbers correctly and start
numbering lines from 1 instead of 0. Mute fields that are empty
strings or boolean false.
Fix comments in john.c according to code style.
Register dynamic formats last. We certainly don't want it to have
precedence over optimized formats. The old comment about it needing
to be registered early (bc of thin formats) does not seem to apply
anymore.
Fix a build warning for status.c LLu vs. PRIu64 after core merge.
Avoid lots of build warnings for PRI[ux]64 on some systems.
cracker.c and wordlist.c bug fixes after core merge.
Travis conf: Do not allow ASan or macOS failures.
--show=types-json: Escape control chars (any 8-bit stuff is currently
assumed/required to be UTF-8 and not escaped).
Change run-time CPU detection for AVX2 to actually only check for AVX2
but neither of BMI1, BMI2, MOVBE and FMA3.
MPI benchmark: For MPI frameworks lacking reduce for uint64_t,
re-introduce the old bug of taking the sum of each half without
carry. Better than nothing and I'm not sure it ever hits us anyway
in a benchmark. If it does, well, it's just cosmetical.
--show=types[-json]: Ignore pot file(s) when --show=types is used, for
speed-up (it's not used anyway).
Fixes for new CPU detection in ./configure (eg. non-Linux) See #2966,
#2972
Make CircleCI builds silent again. The output is too large even
without it.
Move CircleCI-MinGW.sh from src to .circle See #2962
Bug fix: simd-intrinsics.h had an obsolete macro clashing with
non-SIMD builds.
Fixup: Last commit accidentally had things commented out instead of
removed.
.editorconfig entries for autoconf files (.ac, .m4)
Temporary bug workaround for PPC/ARM SIMD.
Bug fix for AVX512 CPU detection/fallback.
Autoconf: Add --enable-simd=foo option in a way that doesn't add it
when building john.o (which may break CPU fallback). And some more
clean-up. See #2862. Closes #3002.
Autoconf fixes: --disable-simd: Do not add simd-intrinsics.o for
non-intel.
AC cpuid fix: Enhance Claudio's excellent tests to use autoconf macros
instead of the less safe "bastard compile". Also adds it to AVX512*
tests in the same manner. See #2881, #3002.
AC: I was using mic.h where I shouldn't. It made no harm but was
confusing.
Avoid merge conflicts whenever core bumps version.
Refactor some file names with perv... sorry, reverse polish notation.
Fix a problem with cpuid checks in ./configure on x86-32. See #3041.
Temporary bug workaround: Do not probe 32-bit x86 for better than AVX.
See #3041.
Remove temporary workaround for 32-bit cpuid checks, and implement
them. Also fix missing stuff in x86-sse.h.
x86.S, some clean-up.
x86.S cpu detection bugfix for AVX2 and higher. See #3041
NT-old: Honor the --disable-simd configure option. Closes #3044.
Add INSTALL-FEDORA and also change all references to eg 'make -j8' to
'make -j4' because that seems to be a faster overall suggestion and
bumping it is documented.
Fix a major f*up with the last commit. We changed charset files.
The -D option to gcc defines the macro to 1, no need to say it.
INSTALL-FEDORA: Add proper name for libbz2
Bug fix for SAP-G format, problem introduced in 27c9bf17. SIMD_COEF_32
was tested before defined. Also down-tunes OMP_SCALE so it resembles
the multiplier used for length sorting. Closes #3050.
Autoconf changes for using NEON on ARM64.
Autoconf: Honor --disable-neon for aarch64 (actually cosmetic).
Autoconf: Add -Wno-error=cpp for --enable-werror
NEON autoconf tweaks.
Some manual clean-up after Dhiru's poorly reviewed grep|sed sessions.
Closes #3062
Test using --enable-werror for the bots
Autoconf: Avoid outputting "configure: configure:" in output from
AC_MSG_NOTICE.
Autoconf: Add -Wno-error=#warnings for clang when --enable-werror.
Autoconf: Unify comments (using dnl so they don't end up in the
compiled script), replace some comment that drifted away from what
it referred to and drop some obsolete comments. Also adds default
CFLAGS for when we can't test compiler flags (and this is the only
no-op change).
Further clean-up of autoconf scripts.
Drop some CUDA artifacts and fix bench.c diff against core for MinGW
timing.
SunMD5 would segfault at OMP_NUM_THREADS=1 after Dhiru's recent
clean-ups.
OpenMP: argon2 still used omp_t and scaling at single thread.
Drop some spurious OMP_SCALE stuff from OpenCL RAR format.
timer.[hc]: Code style fixes.
Loader bugfix for testdb cost ranges.
keystore format was freeing stuff in done() that was still pointed to
by the salt db. Change to mem_alloc_tiny (the alternative would be
using dynamic salt).
Add shared CPU autotune code.
Add optional autotune for all applicable formats.
NETLM format: Enable tuning for non-OMP builds. Add correct max keys
per crypt and scaling.
OMP autotune: Use threads == 1 for non-OMP.
Tweaks for MKPC tuning.
Tweaked MKPC and then OMP_SCALE for wpapsk-pmk (core i7)
Tweaks for MKPC tuning. We shouldn't go below MIN_KEYS_PER_CRYPT.
Fix for generic legacy builds.
OMP autotune: Cosmetics
WPAPSK: Tune MKPC and OMP_SCALE for core i7M, incl. non-SIMD.
Ignore any preset OMP_SCALE (ie. always autotune) if we're not an
intel arch. See #3091
Autotune: Tweaks/bug fixes.
KeePass and MySQL formats: Tune MKPC and then OMP_SCALE. See #3091
sha256crypt format: Tune MKPC and then OMP_SCALE. See #3091
sha512crypt format: Tune MKPC and then OMP_SCALE. See #3091
raw-MD5 format: Tune MKPC and then OMP_SCALE (including for non-SIMD).
See #3091
NT format: Tune MKPC and then OMP_SCALE (including for non-SIMD), see
#3091
mscash2: tune MPKC and OMP_SCALE for core i7 with and without SIMD,
see #3091
mscash: tune MPKC and OMP_SCALE for core i7 , see #3091
pkzip: tune MPKC and OMP_SCALE for core i7 , see #3091
phpass format: Tune MKPC and OMP. See #3091
SunMD5: Tune MKPC & OMP, see #3091
Revert omp_autotune() from crypt-md5 format. It's not ready for it.
Revert formats that currently crashes on --tune=auto, see #3091
AzureAD: Tune MKPC and OMP. See #3091
Autotune bugfix, null pointer dereference. Closes #3100.
Minor tweaks to autotune.
Argon2: tune MKPC then OMP_SCALE. #3091
Argon2: Avoid reading past buffer.
One of the Travis builds lacked --enable-werror
OpenCL: Add shared "CTX version" of MD5. Closes #3105.
RIPEMD formats: Tune MKPC and then OMP_SCALE for core i7
raw-sha1-ng: Tune MKPC and OMP_SCALE
HDAA bugfix for --enable-openmp-for-fast-formats
raw-md5-flat bugfix for --enable-experimental-code
raw-md5-flat: Tune MKPC and then OMP_SCALE.
CRC32 format: Tuned MKPC and OMP
HAS-160 format: Tune MKPC and OMP
Tiger format: Tune MKPC and OMP, and enable OpenMP for normal builds.
HDAA format: Tune MKPC and OMP, and enable OpenMP for normal builds.
PST format: Tune MKPC and OMP
32-bit bot-build have some problems with HDAA now that I can't
reproduce. Trying to decrease mkpc.
HDAA: Reverting all changes until I got it straight with the build
bots.
omp_autotune(): Remove some obsolete code.
Autotune bugfix and tweaks
chap format: Tune MKPC and OMP.
Clearquest: Tune MKPC and OMP
citrix NS10: Tune MKPC and OMP
DMD5: Tune MKPC and OMP
Dragonfly formats: Tune MKPC and OMP
EPI and EPIServer formats: Tune MKPC and OMP
Fortigate, tune MKPC and OMP
GOST: Tune MKPC and OMP
hmac-MD5/SHA1/SHA2 formats: Tune MKPC and OMP
HSRP format: Tune MKPC and OMP
IKE format: Tune MKPC and OMP
known_hosts format: Tune MKPC and OMP
krb5tgs: Tune MKPC and OMP
MDC-2 format: Tune MKPC and OMP
Money format: Tune MKPC and OMP
Mongodb format: Tune MKPC and OMP
Mozilla format: Tune MKPC and OMP
krb5pa-md5 format: Tune MKPC and OMP
Mssql12: Tune MKPC and OMP
Mysqlna format: Tune MKPC and OMP
Stribog formats: Tune MKPC and OMP
Pomelo format: Tune MKPC and OMP
Oracle12c format: Tune MKPC and OMP
Lotus85 format: Tune MKPC and OMP
krb5-3/17/18 formats: Tune MKPC and OMP
Krb5asrep format: Tune MKPC and OMP
SSH-ng format: Set to always auto-tune MKPC/OMP. The needs are way too
different between cost 1 and 2. Perhaps they should be two different
formats. See #3091
Ecryptfs format: Tune MKPC and OMP
BKS format: Tune MKPC and OMP
KDE KWallet format: Tune MKPC and OMP. This format needs cost
functions added (and a re-tune). It's probably best ran at
--tune=auto due to big difference between costs.
ZipMonster format: Tune MKPC and OMP
ZIP format: Tune MKPC and OMP
Whirlpool formats: Tune MKPC and OMP.
VNC format: Tune MKPC and OMP
VDI format: Tune MKPC and OMP
TCP-MD5 format: Tune MKPC and OMP
Xmpp-scram format: Tune MKPC and OMP
SSPR format: Tune MKPC and OMP
SNMP format: Tune MKPC and OMP
VTP format: Tune MKPC and OMP
Truecrypt formats: Tune MKPC and OMP
Snefru formats: Tune MKPC and OMP
Skein formats: Tune MKPC and OMP
Raw-sha3 format: Tune MKPC and OMP
Sha1crypt format: Tune MKPC and OMP
SAP-G format: Tune MKPC and OMP
Salted-SHA1 format: Tune MKPC and OMP
raw-SHA384: Tune MKPC and OMP
Raw-SHA224 format: Tune MKPC and OMP
raw-Keccak format: Tune MKPC and OMP
RAR5 format: Tune MKPC and OMP
RACF format: Tune MKPC and OMP
PWSafe format: Tune MKPC and OMP
Postgres format: Tune MKPC and OMP
Pgpsda format: Tune MKPC and OMP
PFX format: Tune MKPC and OMP
PDF format: Tune MKPC and OMP
PBKDF2-HMAC-SHA512 format: Tune MKPC and OMP
OpenVMS format: Tune MKPC and OMP
PBKDF2-HMAC-MD5: Tune MKPC and OMP
Panama: Tune MKPC and OMP
Padlock format: Tune MKPC and OMP
Oracle format: Tune MKPC and OMP
OldOffice format: Tune MKPC and OMP
O5logon format: Tune MKPC and OMP
O10gLogon format: Tune MKPC and OMP
Notes format: Tune MKPC and OMP
MD2 format: Tune MKPC and OMP
ODF format: Tune MKPC and OMP
Lastpass format: Tune MKPC and OMP
KRB5PA-SHA1 format: Tune MKPC and OMP
Keyring format: Tune MKPC and OMP
iTunes format: Tune MKPC and OMP
GELI format: Tune MKPC and OMP
Ethereum format: Tune MKPC and OMP, for slowest salt. Using faster
salts in a real run, --tune=auto might speed things up considerably.
EncFS format: Tune MKPC and OMP
EIGRP format: Tune MKPC and OMP
DPAPImk format: Tune MKPC and OMP
Django-scrypt format: Tune MKPC and OMP
Dashlane format: Tune MKPC and OMP
CloudKeychain: Tune MKPC and OMP
Blackberry ES10 format: Tune MKPC and OMP
Bitcoin format: Tune MKPC and OMP
axcrypt format: Tune MKPC and OMP
FDE format: Tune MKPC and OMP
AIX-SMD5 format: Tune MKPC and OMP
XSHA512 format: Tune MKPC and OMP
NETHALFLM format: Tune MKPC and OMP
NETLMv2 format: Tune MKPC and OMP
7z format: Tune MKPC and OMP
Strip format: Tune MKPC and OMP
ssha512 format: Tune MKPC and OMP
SL3 format: Tune MKPC and OMP
Ensure HMAC formats don't get precedence over eg. MSCash ones (the
former are very promiscuous since they don't really have a tag)
Common mscash code, fix warning message to reflect actual format.
Unicode fix for early use of conversions, eg. in valid().
RAR3 formats: Add self-test for #2899
Whitespace fixes after bfacbbfd (upstream has whitespace violations)
#2899
Further unrar tweaks: Drop a check we did that upstream doesn't, and
fix a cosmetic bug testing an unsigned variable for -1
Bugfix for 32-bit SSE4.1 and SSSE3 detection. Closes #3119.
SSPR OpenCL format: Optimizations, bug fixes. Run 32/64-bit code in
the inner loop instead of 8-bit PoC CTX-style. Add MD5 salt type.
Use a looped kernel. More than 12x boost - closes #3118.
Add CPU detection for SSE4.2, partly because it was trivial now and
partly because the CRC32 format actually uses it.
Enhancement for --show=types option vs. pot file truncated hashes. Now
shows full hash as 'canonHash' and truncated one as 'truncHash'.
Also adds a json '"truncated":true' when applicable.
Cosmetic fix for SSPR OpenCL algorithm_name.
Bugfix for scrypt format when converting Perl format hashes to
canonical. Spotted by chance when looking at #3127.
Add "Constant Buffer size" to --list=opencl-devices.
Revert Jumbo's old BENCHMARK_LENGTH -1000 hack (it predates Git
history), and instead *always* smash passwords when benchmarking -
and in a much better way for OpenCL formats in particular. They may
otherwise show an inflated benchmark speed due to way less thread
divergence than in real use.
Nick a hashcat workaround for some nvidia bug.
OpenCL: Much faster AES code. It can do tens of GB/s on a Titan X and
is needed for KeePass format.
Lotus5-opencl missed sourcing opencl_misc.h
More fixes after recent OpenCL commits, after build-bots' complaints.
I tested all formats on a few systems and they did not even warn.
Keepass OpenCL format. This includes new shared OpenCL code for
Twofish and ChaCha20 and uses the new fast AES code (which is the
heavy part).
Disable keepass-opencl from Travis Ci bot check. I'm pretty convinced
it fails there due to silly driver bugs.
Tighten self-test check for lacking FMT_HUGE_INPUT
GELI format lacked FMT_HUGE_INPUT.
Fix benchmark speed reporting of very fast formats (eg. LM-opencl w/
mask). This bug was introduced at core merge 844df7388 because the
 replaced int64 "add32to64()" function assumed unsigned whereas the
new code did not.
DES-opencl and LM-opencl: Add an implicit LUT3 that was missing. Also
source omp.h in files that need it.
Office OpenCL: Drop a redundant AES_set_decrypt_key(). See #3142.
Many formats: Drop silly redundant memsets and useless error checking
when calling AES set_key.
Fix for recent LUT3 substitution. This bug had no impact because the
macro is normally not used.
truth.pl: Add some tweaks and usage blurb bits. Even I did not
remember how to use it.
Keyring-OpenCL: Do everything on GPU-side (AES and MD5 was CPU-side).
Closes #3147.
ZIP-opencl: Do everything on GPU-side.
Drop FMT_OMP from Zip-opencl
DMG formats: Add "version" as a second cost. Loosely related to #3129.
Add shared OpenCL DES code.
DMG OpenCL: Do everything on GPU and drop OpenMP. Massive boost,
especially for v2 low-iteration-count hashes (more than 35x with
Titan X using non-OMP). Closes #3129.
Keepass-opencl fix for some less rigid devices.
Enable keepass-opencl in Travis CI.
Revert "Enable keepass-opencl in Travis CI.". It was a good change,
but not enough for ol' Travis.
Replace shared OpenCL HMAC-SHA1 with a more versatile one.
Truecrypt-OpenCL: Do all work on GPU.
PGPWDE-opencl: Do all work on GPU.
Openbsd-softraid-opencl format: Do all work on GPU.
bitlocker-opencl: Fix a buffer overrun (ASan fault).
Drop the many JUMBO_JTR clauses in formats.c. The idea had some merits
but it ended up utterly confusing.
formats.c: Fix a problem with self-test vs. certain highly-optimized
formats that simply can't have set_key() called again with same
index. Problem seen in SL3-opencl, which was segfaulting.
OpenBSD-SoftRAID OpenCL: Work around an nvidia driver bug merely by
re-ordering stuff a little. The commit also includes some no-op
changes replacing constant numbers with macros, for safety and
clarity. Closes #3154.
OpenCL AES tweaks and bugfixes, better handling of different memory
types (avoiding copies).
OpenCL AES: Use void* for set_key so callers don't need to cast.
Add a couple more min/max length variables to options struct, to keep
apart actually requested vs. calculated limits.
mask.c: Typo and brainfart combined to a minor bug.
Minor fix for incremental: Allow -min-len=0 to override a longer
default. Options checking bugfixes. It's amazing how it keeps ending
up incorrect.
Markov mode: Honor a format's minimum length (bug introduced with
recent changes).
mkv.c: Simplify expression for min length check (no net change).
TACACS-plus format: Minor fixes while preparing for OpenCL version. No
point of storing full ciphertext if we only use 8 bytes of it.
Add NVIDIA LUT3 options for OpenCL MD4/MD5 kernels. They are just for
reference, not used. Also, stop using the ones that were already
defined for RIPE-MD-160 (TrueCrypt format) because the optimizer
ends up better without it.
Ensure we're not disabling LUT3 for SHA1 or SHA2 (where it does good)
when we do so for eg. MD4 or MD5 in multi-algo formats.
Twofish, GOST and Blake256: Omit some "sensitive data wiping" that we
 don't need or want in this application.
NTLMv1/MSCHAP formats: Add an atomic guard before or'ing into shared
bitmap. See #3160.
Fix a truncation vs. rejection bug introduced recently. Closes #3164
OpenCL: Add new options --lws=N and --gws=N for forcing local and/or
global work sizes. Unlike the good old environment variables
(LWS/GWS), the new options will be stored in the session file
ensuring a resumed session keeps same values.
rar2john.c: Fix buffer overflow w/ long path/file names. Closes #3114
Refactor common-opencl.[ch] to opencl_common.[ch]. Otherwise no-op.
Fix cosmetic bugs in size output from --list=opencl-devices.
Monero format: cn_slow_hash() had a 2 MB stack-variable that caused
many systems to segfault (esp. with OMP). Changed it to alloc/free
since it wont affect speed anyway. Also some minor tweaks to Monero
itself. See #3156.
Bitwarden format: Do not decrypt more than needed. See #3156. This
probably fixes the segfaults as well - we were reading past the end
of salt->blob - but I could never reproduce that segfault. The
OpenCL format had it right already.
CircleCI script: Use '#!/bin/bash -e' for catching any error.
Keepass formats: Bump MAX_CONT_SIZE to 16 MB. Some kernel rewrite was
needed for handling that. Closes #3169.
Mask mode: Never stretch a hybrid mask. Closes #3189.
Add a FMT_MASK flag, telling us the format supports internal mask.
Warn user if such format is used without mask acceleration. Closes
#3046.
OpenCL LM and DEScrypt: Add FMT_MASK flag.
cracker.c: Do not use a copy of db->fmt->params, it's volatile in
Jumbo at least after soon coming mask mode changes.
Revert "OpenCL: warn if --dev lists more OpenCL devices than fork"
OpenCL: Bail out if --dev lists more OpenCL devices than fork or MPI
nodes. Closes #3176
Tweaks for poor-mans-multiple-GPU vs. fork/MPI. See #3176.
Add --list=format-all-details output for internal mask generation. Add
format group --format=mask, eg:
Prepare a bunch of OpenCL formats for having reset() called again
during a crack (eg. after changing internal mask).
OpenCL formats bug fixes. Due to the way things are freed, we need to
use calloc() or we may get into trouble. Also drop some artefacts.
LM-opencl: Revert all recent changes until I figure out what the heck
is wrong with it (build bots fail, double free despite using
MEM_FREE macro)
doc/MASK: Document the "--test --mask" feature and some other internal
mask details.
Remove dead code and superflous zero-initializations of
globals/statics.
oldoffice-opencl format: minor tweaks.
Revert some "dead code removal" that wasn't dead code >.<
OpenCL SHA1 kernel: Bugfix for mysql-sha1 when using internal mask.
Due to double non-CTX SHA-1, initial key is thrashed and need
rebuild for each internal mask candidate. Closes #3198
Internal mask formats: Prepare for better mask mode support.
De-obfuscate some of Sayantan's code. Various fixes.
Drop many assert()'s from Sayantan's code. We've had them for years
and they never triggered. A few were replaced with proper code
(message and error).
Mask mode: Enhance mask-stretching, plus other significant
enhancements and bug fixes. We can now stretch a mask like "?dword"
and iterate lengths (eg. --min-len=5), for zero or more digits
 prepended to "word".
Mask mode, fix session resume.
Update best64 rules from upstream (which is hashcat). Closes #3202.
Shared OpenCL pbkdf2-hmac-sha1 format. Allow not defining OUTLEN
(salt->outlen will be used instead) as long as you instead define
MAX_OUTLEN to whatever the biggest length is.
OpenCL PBKDF2-HMAC-SHA256: Add support for multi-limb output (ie.
output length beyond 32). Closes #3172.
Add "hashcat logic" flag when sourcing base64.conf, and fix a bug in
rules.c for handling that. Closes #3208.
Ansible CPU format: Use "skip bytes" correctly. 2x boost just like
that. See #3171, #3172, #3211. GPU format fix coming soon.
Add "skip bytes" functionality to OpenCL PBKDF2-HMAC-SHA256, for a 2x
boost in Ansible format. See #3171, #3172, #3210.
New Ansible kernel with on-board HMAC-SHA256 for the last 5% boost.
Now we're close to optimal. See #3171, #3172, #3210.
Add mask info to the "Proceeding with..." messages printed when
resuming.
Mask mode: Allow stretching a mask like PASS?dWORD --> PASS?d?dWORD.
We will find the first range and repeat it.
Benchmark tweaks: Show GPU utilization even with 1-second benchmarks,
and add a "mask accel" notice inside the algo name brackets when
applicable. Also move the note that we are benchmarking with UTF-8
or a codepage into the brackets.
bench.c: Move the "GPU util" figure from benchmark to the first speed
output line, to cut a little on line lengths.
More tweaks for benchmark output: When benchmarking with a codepage,
include its name in output. Also fix generic legacy build.
KRB5-ASREP CPU format: Bug fixes. The magic spell "// XXX" is proven
ineffective, it actually does not fix things. See #3213.
OpenCL shared AES code: Bugfix for (unused as of now) AES_cbc_encrypt,
and add CTS mode functions. This in preparation for KRB5-ASREP
format enhancements. See #3213.
OpenCL shared HMAC-SHA1 code: Bug fix for a use-case we don't have
yet. This in preparation for KRB5-ASREP format enhancements. See
#3213.
KRB5-ASREP CPU format: Don't produce 32 bits of key when we only need
16 (2x boost for etype 17). And fix a cosmetic bug.
memory.[ch] and opencl_misc.h: Drop most of the ridiculous number of
"dump_stuff()" functions and add macro shortcuts for dumping an
array or variable:
KRB5 AS-REP OpenCL format: Do everything on GPU (2x boost) and drop
OMP. Don't produce 32 bits of key when we only need 16 (another 2x
boost for etype 17). And fix a couple of bugs. Closes #3213.
NVIDIA bug workaround for krb5-asrep-opencl. An alternative
workaround is to use --force-vector=2, See #3214.
rar2john was using dump_stuff() stuff without sourcing memory.h
KeePass OpenCL format has its own memcmp_pmc() and can't use the
shared one.
There were artefacts left from older "GPU util" reports when
benchmarking, producing a wierd output with several-formats --test.
Also, do the sensor readings a bit earlier so we don't get lowered
figures.
Some quick fixes until I understand WTH is wrong with Windows builds.
Ethereum-presale-opencl: *Use* the PBKDF2 kernel instead of bloody
copying it! Closes #3215.
macOS fixes after changing the PBKDF2-SHA256 kernel.
Ethereum Presale formats: Exploit the fact they did not use a proper
salt for PBKDF2. Huge boost when attacking many salts.
Bug fix for Ethereum Presale no-salt-exploit.
Add a little safety around Presale no-salt-exploit stuff. Some oddness
with self-tests (I think) could trigger problems.
Allow SAP B and G formats to crack "half hashes" taken from
RFC_READ_TABLE. To enable this, set john.conf "SAPhalfHashes = Y".
Fix a misplaced line (after using patch -F3) in fvde-opencl.
zip2john bug fix for certain files. See john-users ML:
http://www.openwall.com/lists/john-users/2018/03/26/15
JtR normally does not apply rules to an empty word. Hashcat does, so
this commit makes us do so too if "hashcat logic" is turned on.
Mask mode: Start iterating from given mask's length if no --min-len
was given. Also fix a bug where we sometimes failed resetting the
format although GPU-side mask changed.
Add current device's "internal mask target" to
--list=format-all-details.
Fix a problem with !OS_TIMER systems where sig_timer_emu_tick() was
called before we had any handler in place for it. Closes #3226.
bt.c: Do not use alarm(3) because it may clash with our other uses of
timers. Adjust the figure for where we print a warning about many
loaded hashes. See #3226.
Fix a tpyo in bt_hash_type_*.c
Mask mode: When iterating but no min-len given, do not consider a
default mask's length for min-len.
Cosmetic bug fix: After recent changes, status lines would sometimes
include a "current mask length" even though not iterating over
length.
krb5pa-sha1-opencl: Do everything on GPU (similar to
krb5asrep-aes-opencl). This rendered a 50% boost and GPU util now
99%. See #3216.
KRB5PA-SHA1 formats: Revert shrinking of a few buffers. I really can't
see why but ASan tells me it's needed (single-block AES so 16 bytes
should do, but 32 is needed).
Drop a shared pbkdf2 macro that should be local to formats. Drop some
redundant artefact from PBKDF2-HMAC-SHA512 OpenCL formats.
OpenCL HMAC-SHA256 bugfix (copy/paste bug from SHA-1, never surfaced).
OpenCL GELI format: Do all work on GPU-side. This commit also adds
shared OpenCL code for HMAC-SHA512. About 3x boost non-OMP and GPU
util now 100%. See #3216.
GELI-opencl, fix algo name after post-processing moved to GPU-side.
zip-opencl: Avoid a warning on some platforms.
GELI-opencl: Work around an AMD bug.
Shared HMAC-SHA-2 OpenCL code: Fix bugs that never surfaced yet
(alternative code path, still not tested at all).
PEM format: Cosmetic (code) bug fix.
OpenCL PEM format: Do all work (3DES/AES and ASN.1 decoding) on
GPU-side. Boost is in the order of 10x and GPU utilization went from
N/A to 100%. See #3216.
Minor completion of PEM formats' algorithm name.
ODF format: Post-process on GPU (Blowfish, SHA-1) and join with
ODF-AES so we only have one format just as for CPU (cost added).
See #3216.
Rename ssh-ng format to just ssh, and sshng2john.py to ssh2john.py.
Closes #3231.
Documentation fix after renaming ssh-ng format. See #3231.
Have autoconf check for function instead of trusting OpenSSL version
number. Closes #3163.
SSH format: Tweak the "cost" stuff so we can tell 3DES from AES.
Closes #3229.
SSH format: Drop a sub-percent optimization and instead optimize the
 generic function and stop initializing large arrays to zero right
before filling them with other data.
OMP/MKPC auto-tune bugfix. See #3229
mask.c: Avoid dupe output with fork/MPI.
unicode.h bugfix (not used right now).
Correction of hex representation of a new line character in doc/RULES.
This little "bug" has been with us all the way since 2012.
Keystore-opencl: Move work from set_salt to get_salt. Do not
re-transfer keys when not changed. Use packed key transfer (GPU-side
mask still needed) and bump max password length to max. Bump GPU
salt size, there was never any reason to cripple the format in that
regard. We (now) only transfer actual used size. The OpenCL 1.2 spec
mandates a minimum size of 64 KB so that is what we currently go
for. We could also opt for dynamically switching to global memory if
(and only when) a larger size is needed.
Shared OpenCL code for HMAC-SHA256 fixes.
PEM formats: Add 2nd cost "cipher".
Assorted OpenCL macOS tweaks/bug workarounds. This makes PEM-opencl
work with the Intel(R) HD Graphics 630, and oldoffice-opencl with
AMD Radeon Pro 560 Compute Engine (and the latter fix may help other
formats too).
Adjust core count (which in turn affects speed index) for Intel GPUs.
Closes #3238.
Assume an unknown AMD is a GCN, since most others should have died
from age by now. This is a temporary workaround for #3237.
Correction of 2nd cost description for PEM formats. Closes #3236.
Fix some bogosities in blockchain CPU format.
Blockchain OpenCL: Move CPU post-processing to GPU. Some 13x boost and
GPU utilization (with single-thread CPU) up from 5% to 99%. See
#3216.
KeePass formats: Add 3rd cost "algo" for testing purposes.
OpenCL shared AES: Clean up, separate basic AES from cipher modes and
OpenCL memory type handling. Introduce alternative code (the one we
used prior to 1a0704f) for use by buggy OpenCL runtimes in some
cases. See #3235.
OpenCL: Improve shared debug stuff.
OpenCL: macOS+AMD bug workaround for TwoFish on GPU. See #3235.
Work around a macOS runtime bug for AMD GPU. See #3235.
Revert the disabling of bitselect for OSX. There was regression
(sha256crypt format now failed w/ CPU). Narrowing it to only
disabling bitselect in swaps did not help.
Add support for knowing core count of nvidia Volta. This is untested.
Move the AES-XTS code from Truecrypt kernel to the shared AES code.
Reduce transfer size a little for md5crypt-opencl.
Fix a few stupid ass bugs sneaking in while I looked the other
direction.
Truecrypt OpenCL: Bugfix after recent moves of code.
OpenCL: Move check_pkcs_pad() from DMG kernel to opencl_misc.h, and
drop the copies in agile and pem kernels.
Keychain-opencl: Post-process on GPU. See #3216.
Shared OpenCL: Fix a compiler warning. See #3239.
STRIP OpenCL: Post-process on GPU. See #3216.
Lastpass OpenCL formats: Post-process on GPU. See #3216.
STRIP and Lastpass OpenCL: Don't re-transfer same key data.
Allow using custom mask placeholders along with --test
OpenCL: Ensure --force-scalar always equals --force-vector=1.
Change phpass-opencl from hard-coded vector width 8, to variable width
and use whatever the device announces as preferred width. Also bump
max. length from 15 to 39 and get rid of ALL use of 8-bit data types
 in the kernel (NOTE: test vectors changed so boost is even better
than the looks of it). See #3216.
A couple of issues found when researching candidates for #3242: Drop
"Blowfish" from encfs formats' algo name. It was incorrect. Drop old
now unused 'any_cracked' variable from keyring-opencl.
phpass-opencl: Packed key transfer.
Fix an ASan warning seen in WPAPSK-opencl (benign, read beyond stack
buf).
Revert "phpass-opencl: Packed key transfer". It was prematurely pushed
upstream.
Bug workaround for shared SHA1-CTX for macOS w/ AMD GPU. This makes a
few more formats pass self-test, at least PGPdisk, OpenBSD-SoftRAID,
krb5pa-sha1 and krb5asrep-aes. See #3235.
Add 2nd cost "algorithm" for PGPdisk, for benchmarking purposes.
Revert shrinking a buffer in pgpdisk (CPU). It was needed due to a
stoopid function using more than requested of it.
EncFS OpenCL: Post-process on GPU (AES-CFB, HMAC-SHA1, checksum).
Addresses #3242.
PGPdisk OpenCL: Post-process on GPU (AES/TwoFish/CAST). Addresses
#3242.
PGPdisk-opencl: Separate kernels per salt, for better optimizations.
PGPSDA OpenCL format: Post-process (CAST) on GPU. See 3242.
Ethereum OpenCL format: Post-process (Keccak) on GPU. See #3242.
Ethereum-presale OpenCL: I was confused and introduced a bug in recent
commit. GLOBAL_SALT_NO_INIT means the PBKDF2 salt is __global but
not the "salt" struct!
RAR5 OpenCL: Post-process on GPU. Closes #3242.
7z-opencl: Correct case in algo name.
Shared OpenCL MD5: Avoid using "volatile" driver bug workaround unless
needed. The Intel(R) HD Graphics 630 took a huge performance hit
while it worked fine without it (the workaround is needed for AMD
Radeon Pro).
Jim decided to put a test vector with "1 iteration" in
pbkdf2-hmac-md5. That's OK but it shouldn't have been the first one.
Benchmarks were off and even worse, auto-tune was f*cked too.
Drop the pull-request template (I see no need for it) and trim the
issue template down a bit. The more text, the less chance Joe Blow
will read it.
Add/remove/tweak workarounds for macOS OpenCL runtime/driver bugs. See
#3235.
De-optimized OpenCL kernel to some extent for office 2013, works on
macOS now (and shouldn't really affect better devices/drivers). The
runtime optimizers are so buggy it's a mystery they work at all. See
#3235.
More of add/remove/tweak workarounds for macOS OpenCL runtime/driver
bugs. See #3235.
Change uint in host code to uint32_t. See #3246
More of add/remove/tweak workarounds for macOS OpenCL runtime/driver
bugs. See #3235. Also drop/rework a dangerous (in terms of
potential bugs) nvidia optimization and replace with a safe one (for
any arch).
Finally found the problem with macOS/AMD crashing while building
md5crypt OpenCL kernel. The check for support for amd_bitalign was
flawed and macOS never supports proprietary stuff. See #3235.
OpenCL: Re-implement more optimizations for aligned buffer in shared
MD4/MD5/SHA1/SHA2 code (and fix a bug). See af150a18.
Fixes for MINGW OpenCL builds, kudos to BotPass. See #3246.
Fix warning message when running OpenCL format with CPU
post-processing and OpenMP is disabled. Closes #3248.
opencl_common.c: The buffer for build options were allocated as
PATH_BUFFER_SIZE instead of a more logical LINE_BUFFER_SIZE. The
former is only 256 bytes on some systems (POSIX has it as 256).
Closes #3251.
Revert a Makefile change (#3246) that broke MinGW buids on build bot.
Re-apply the Makefile fix but now also fix the ever-broken CircleCi
script. Closes #3246.
A couple more MinGW fixes from #3246 (from BotPass' 2nd patch).
bcrypt-opencl: Enforce "CPU kernel" for OSX with Intel HD Graphics.
Closes #3245.
Keystore-opencl: Do not allow a GWS larger than 'count'. Closes #3254,
see also #3223.
OpenCL Lotus5, oldoffice and o5logon: Use new scheme of LWS=NULL for
less-than-full batches instead of rounding GWS up to LWS multiple.
See #3223.
OpenCL Office: Use new scheme of LWS=NULL for less-than-full batches
instead of rounding GWS up to LWS multiple. See #3223.
OpenCL GELI: Use new scheme of LWS=NULL for less-than-full batches
instead of rounding GWS up to LWS multiple. See #3223.
OpenCL Keyring: Use new scheme of LWS=NULL for less-than-full batches
instead of rounding GWS up to LWS multiple. See #3223.
OpenCL KRB5PA-MD5: Use new scheme of LWS=NULL for less-than-full
batches instead of rounding GWS up to LWS multiple. See #3223.
OpenCL NTLMv2: Use new scheme of LWS=NULL for less-than-full batches
instead of rounding GWS up to LWS multiple. See #3223.
Minor cleanups in a few OpenCL formats.
OpenCL PGPdisk: Use new scheme of LWS=NULL for less-than-full batches
instead of rounding GWS up to LWS multiple. See #3223.
OpenCL PGPSDA and PGPWDE formats: Use new scheme of LWS=NULL for
less-than-full batches instead of rounding GWS up to LWS multiple.
See #3223.
OpenCL shared check_pkcs_pad() was slightly too promiscuous; it let
anything with a trailing null through (possible false positives, so
no sweat).
A couple of minor FORMAT_NAME vs. FORMAT_LABEL fixes.
Debug function tweaks, and minor fixes in int128.h. This is a no-op.
cprepair: Avoid a conflict with memory.h
Many OpenCL formats: Use new scheme of LWS=NULL for less-than-full
batches instead of rounding GWS up to LWS multiple. See #3223.
GPU mask: Decrease verbosity needed for seeing auto-tune mask
multiplier.
The "spinning wheel" cursor sometimes left an artifact after use.
Bitlocker formats: Drop the fake test vectors that messed with
reported benchmark speed, and set a better autotune target that will
actually make the format perform good (although a bit unresponsive)
IRL.
Stop piggy-backing arguments and drop the notion of "max crypt_all
time" from OpenCL autotune, it was an artifact. Closes #3278.
wpapcap2john: Handle QoS packets with four addresses. Closes #3282.
Internal mask: Fix a cosmetic bug with MaskLengthIterStatus=Y where
we'd output the old length instead of the new one, and incorrect
ETA/progress.
Mitigate a benchmark regression introduced in e5b031e. See #3289.
Autoconf: Add a way to check if -Wno-foo is supported. This has to be
implemented by checking if -Wfoo is supported because the "no-"
forms will soft-fail.
OpenCL PBKDF2-SHA1: Support longer candidates.
OpenCL DMG format: Support longer candidates. Test vector needed!
dynamic.conf clean-up. Sort entries and update the "synopsis". Closes
 #3346.
dynamic.conf clean-up. Change a few ($p,$s) to ($p.$s). See #3346.
Define out (unless -DDEBUG) three very slow test vectors from DMG
formats. See #3344.
Ethereum bugfix. There was a race condition with new_keys variable,
it must be reset outside the parallel section. Closes #3344.
WPAPSK and -PMK formats: Add support for PMKID. Closes #3356.
Avoid a problem with some internal-mask formats reading stale data in
get_key(). Addresses #3358.
wpapcap2john: Support reading TZSP encapsulated files directly. Closes
#3282.
WPAPSK/PMK formats: Drop some duplicated code. Closes #3359.
WPA formats: Add a missing salt_hash() function, greatly speeding up
loading of many hashes.
wpapcap2john: Add support for getting PMKID hashes. Closes #3364.
Meta data tweaks for wpapcap2john output.
Usage blurb fixes for wpapcap2john
wpapcap2john: Tweaks for edge cases and lack of (or late) ESSID clues.
wpapcap2john: Support reading pcap-ng format. Closes #3371.
Brainfart fix in wpapcap2john. Not sure why some compilers accepted
it.
wpapcap2john: Handle mac filtering of addr3 and addr4, and fix some
edge- cases where BSSID was possibly picked from wrong field. Closes
#3373.
wpapcap2john: Replace some repeated code with a more robust function.
mask.c bugfix, see #3381.
OpenCL: Avoid anonymous unions. Some buggy AMD/clang runtime seems to
have problem with it. Closes #3387.
AC: Ignore PERL_UNICODE=SDA for plugin dependency check. Closes #3380.
Avoid calling init() for --list=(...) options with ZTEX formats. See
#3379.
AC: Find OpenSSL on macOS Homebrew without any special options. To
achieve this I had to disable pkg-config by default, because it's
broken. Actually I can't remember having seen a non-broken
pkg-config in my entire life so it's probably a good thing in most
cases but it might break some other system so you'd have to add
--enable-pkg-config to the configure options. This hopefully
addresses #2347.
There's something wrong with our scatter intrinsics (FLAT_OUT option)
for AVX512 and the MIC, we just disable using them for now. See
#2273.
simd-instrinsics.c: Code style, no-op.
Bug fix for our 'vscatter' pseudo-intrinsics, and re-enable it
(currently affects AVX512 and MIC). Fixes #2273 properly.
Drop john.conf boolean option WordlistMemoryMap and replace it with a
numeric option WordlistMemoryMapMaxSize defaulting to one terabyte.
Multibit format: Add FMT_UTF8 flag, and disable a test vector that is
encoding-specific (it will fail for any encoding not having this
character). Closes #3393.
listconf.c: OpenSSL deprectated SSLeay functions for version
enumeration. Handle both situations. Closes #3396.
Ensure we show AVX512 "level" (eg. AVX512F, AVX512BW) in algorithm
names.
Alignment bug fix for VS build of argon2_core.h, found by coincidence.
Fix comments in john.conf - eg. "there" vs. "their" and over-long
lines.
Travis CI: Include -list=build-info (CircleCI already got it).
We go out of our way to safely fetch a silly run-time version figure
from OpenSSL despite them being complete bastards, renaming the very
 function that could otherwise have been used to work around their
crazy decisions. Closes #3402.
Fix incremental mode's ETA calculation when CharCount (in john.conf)
is lower than actual number of characters available. See #3404.
Change our vswapxx pseudo-intrinsics so they just return the result,
without changing input. This sometimes allow optimizations. Neither
our OpenCL macros nor the johnswap.h CPU macros modify the input.
pseudo-intrinsics.h: Cosmetics
Enable AVX-512 for bitslice DES in simplest possible way. Closes
#2274. See also #3399.
AVX-512 adds rotate instructions. Using them makes for a terrific
boost (25-50%) for all hash functions! See #3399.
05272574c broke AMD NEON, this fixes it (and more).
getopt.c: Do not consider an abbreviated option to be ambiguous when
the first alternative is a prefix of all others. See #3404.
Internal: Command-line override of CharCount,
--incremental-charcount=N. See #3404.
Pseudo-intrinsics: Alternative vswap32() for AVX-512F and others
having non-emulated vcmov and vroti.
Only use the alternative s0/s1 functions for SHA-512 if we emulate
vroti (this is another 20% boost for AVX-512).
SIMD code, cosmetic changes/future logic.
Alternative SHA-2 sigma functions for OpenCL. The ones for SHA-256
tested 2% faster on nvidia. The ones for SHA-512 are slower because
we have native rotates.
zip2john: Try old pkzip function when "strong encryption" clause
fails. See #3411.
Optimize vroti16 for AVX2 (used in MD5). We've had this optimization
for SSSE3 up to AVX but missed it for AVX2 until now.
john.c: Tweak the "Use the --show option..." message for formats that
have a prepare() function (suggesting to use --format as well).
Closes #3412.
SHA-2 Sigma/sigma alternatives for SIMD and OpenCL code. 3% boost for
AVX2.
SHA-1 optimization with ternarylogic, another 5% boost for AVX-512.
Add a vternarylogic emulator for AVX2 (only for debugging).
SHA-2 SIMD and OpenCL shared code: Add yet another set of alternative
sigma functions. These use vternarylogic or LOP3.LUT. This was again
a regression for nvidia so not used there, but it made another 10%
boost for AVX-512 and a compound boost over last month of 70-77% for
SHA-2. See #3399.
Support correct speed calculation on nvidia Turing (untested).
AC: Change positive output of HT test to read "maybe". Closes #3418.
README.md: Change from travis-ci.org to travis-ci.com
--loopback mode: Add a notice about what permutation rules are in use.
pseudo-intrinsics.h: Fix a bug affecting eg. ppc64, introduced in
05272574. See #3428.
rar2john: Fix a bug where we would pick a zero-byte file as target
despite a comment saying we wouldn't. Closes #3431.
Dynamic: Change an erratic unsigned long to uintptr_t. See #3122.
Add a self-test for salt_hash() returning zero with zero-size salt.
Autoconf: Fix for OpenCL detection problem with macOS Mojave. Closes
#3429.
OpenCL cloudkeychain: Drop an unused variable.
OpenCL: Disable kernel binary caching for macOS - some bug of theirs
make it problematic. Closes #3434.
OpenCL DMG & krb5-sha1 formats: Do not use bitslice AES for macOS. See
#3435.
OMP auto-tune fix for sha256crypt et al, who uses a multiplier in
 init(). See #3436.
Pseudo intrinsics: Drop union members that are not used.
sha2crypt CPU formats: review OMP tune after fixing the bug in
23bff737.
Autoconf: Change -Wno-stringop-truncation to
-Wno-error=stringop-truncation and add -Wno-error=format-overflow.
Update doc/INSTALL-UBUNTU, kudos to @junquera. Closes #3439.
Bump SIMD max. salt length in HMAC-SHA256 format to accomodate for
some JWT tokens. Also see #2318. Closes #3443.
Bug fix for previous commit. See #3443.
Rules dupe suppression was saying it removed "!! hashcat logic ON/OFF"
duplicates. I'm not 100% sure it actually had any ill effect but for
peace of mind I changed the code so it doesn't remove them for sure.
Suppress over-detailed logging of dupe rule suppression unless
verbosity is bumped to the max.
Add a rules subdirectory under run and put more hashcat rule files in
there. We already had best64, which was merely renamed.
Shared Unicode stuff: Add strlen32() for future use.
RAR formats: Add a padding check for very small files, to mitigate
false positives due to CRC32 collisions. Closes #3451.
rar2john: Bump file name buffer size to ensure it can hold worst-case
UTF-8 as well.
zip2john: Fix a few potentially uninitialized variables after a42fd75
#3447
Update CREDITS-jumbo and CHANGES-jumbo.git files
Update Unicode stuff from 7.0.0 to 11.0.0. A bug was also fixed in the
script I use to process data, so some dupe entries were pruned.
RAR formats: Change the logic for method 0x30 padding check so we get
early rejection for large files if they have any padding at all.
Add a build #warning when Electrum is not built because of missing
zlib. Closes #3456.
It seems that MESA defines cl_amd_media_ops but then doesn't support
it. That must be the dumbest bug ever. Closes #3454.
RAR bugfix after recent changes (we had false positives).
zip2john: Add another warning, when needed, about the assumption that
all files have the same password. Also mutes some erroneous partial
diagnostic output. Closes #3464.
Debug stuff and workarounds for #3450, #2457. General audit of shared
OpenCL code, for uninitialized variables and unified error messages.
Do not blindly trust OpenCL drivers' compliance.
Refactor john-mpi.[ch] to john_mpi.[ch] and move the #if HAVE_MPI to
the header. Closes #3463.
zip2john tweaks. See discussion in #3462.
zip2john: Fix a silly bug and a memory leak introduced by previous
commit.
Fix a bug where loopback mode would interfere with --max-len option.
pp.c: Sync with upstream as of bffda8c (Thu Jul 5 11:53:15 2018
+0200). No significant changes.
Update .gitignore
Fast subsets cracking mode, with full Unicode 11.0.0 support.
Add a debugging verbosity level, --verbosity=6 and have "plaintext"
format print its candidates at debug verbosity.
Single mode: Split on upper-case initials when loading words.
Legacy build fix.
Another legacy build fix.
Colorize output for cracked administrator passwords.
Rules: Add WN rule command, which is like TN except it toggles shift
instead of merely case. That is, it does what TN does but it also
toggles eg. 1 <-> !, 2 <-> @ and 3 <-> #.
Enhance the default loopback ruleset: Use the new ShiftToggle instead
of the alphabet-only NT toggler. Also add three really basic rules,
suitable for loopback.
OpenCL: Numerical error codes were sometimes printed twice because
get_error_name() already adds it to its output string.
OpenCL: Print node number on error.
New WN rule can't be used with -c rule rejection so could produce a
humongous number of duplicates for a case insensitive format (eg.
foo and FOO). This had me figure out a canonical way to fix that
and similar problems without using a rule flag. It's effective on
most rules that use conv_* arrays.
Add "toor" and "sa" as admin names (exact match except for case).
Tweak the handling of --max-len=N option vs. rules, for a huge
performance boost in many situations. See #3467, #3468.
--rules=NT: Add rule reject flags and length reject commands, for some
boost in general and when using --max-len=N option in particular.
Expose min_length constant in rules (#3471)
OpenCL: Use a min. KPC better optimal for Single mode. See #3467.
Ensure the capping of min. KPC doesn't roll over to negative. See
#3467
Ensure a large salt count doesn't result in too much memory use. The
default limit is set to 4 GB but it's a john.conf setting.
Wrap opencl_common.h in #if HAVE_OPENCL so we can source it even for
non-OpenCL builds.
Drop #ifdef guards around '#include "opencl_common.h"' now that the
header itself is wrapped.
Single mode: Fix bugs with --max-len handling.
Minor fixes for warning messages.
Fix bugs in human_prefix() and have it return a tiny-alloced string.
The latter meant I also had to move it to misc.c.
Add rule constants for min_length-1 and min_length+1. See #3471
Editorconfig: Give up on using length 8 tabs for single.c
Change single mode's key buffer's index and offset type to 32-bit for
OpenCL ansd ZTEX builds. 16-bit is not enough for them. Also improve
the way we try to adopt to the buffer size limit: First decrease
max. length (if larger than 16). See #3467.
Rules logging tweaks and bug fixes.
Tweaks for 32-bit single key buffer size/index.
7z2john.pl: Update from upstream.
rar2john: Avoid compressed files shorter than 4 bytes, to avoid false
positives. See #3477
rar2john: Avoid compressed files shorter than 4 bytes, to avoid false
positives. The previous commit was incorrect. See #3477
rar2john: Minor enhancement in file choice - we now pick the file with
smallest *packed* size of all files that have a larger *unpacked*
size than the minimum (which depends on method - for stored files we
can use 1-byte files because there is reliable padding but for
compressed files we need at least four bytes for avoiding false
positives from CRC-32 collisions in case both early rejection and
actual uncompress happens to pass (seen once, with 1-byte compressed
file in #3477). If we're seeing two files with same packed size we
pick the one with smallest unpacked size (as long as it's larger
than minimum).
Single mode bugfix for when we had to decrease max_length in order to
fit in max. size buffers. We need options.eff_maxlength to be
updated in order not to read past buffer later. See #3467
Bugfix: --max-cand option wasn't working with Single mode.
OpenCL: Tweak kernel binary caching and fix some weird bugs with it.
We now make/use cached binaries for anything except: - nvidia unless
 on macOS (nvidia has their own caching but not on macOS) - CPU on
macOS (it's somehow b0rken in their runtime).
rar2john: More minor tweaks for picking "best" file from archive in
terms of performance, data size and immunity against false
positives.
Temporary security fix. See #3482 and #3484
Rename common-gpu.[ch] to gpu_common.[ch].
rules.c (almost no-op): Move stuff around in rules.c so they appear
under correct comment labels such as "Crack rules", "Added in Jumbo"
and "Hashcat specific". Also update doc/RULES a little.
rules.c (no-op): Add most of codepage-specific stuff to new header
files to get it cleaner and easier to overview or compare to upstrem
john proper.
Add new early-reject rules, see #3468
Single mode: Reset max. length and min. KPC to original values when
done, in case we're running batch mode.
Change .editorconfig for a few core files where I officially give up
maintaining a tab width of 8.
Single mode tweaks for better OpenCL performance.
Single mode: If we're running forked, divide the max. buffer size so
we don't end up eating more memory than was intended.
Bugfix for toys unable to provide fork.
Warn about too few candidate passwords. See #3386
The 'bench_running' timer flag has long been mis-used for other
things.
Add min/max length limits (if given) to the "Proceeding with..."
message that goes to screen when session is resumed. Also enable
that message for batch mode, even if not resuming.
cracker.c: Don't mute KPC warnings when iterating mask. Also fix an
MPI logic bug in single.c.
Bugfix in OldOffice-opencl after 9cc4e0193.
Fix bugs in Jumbo-specific part of ldr_split_string() (and improve it
to boot!), and fix a performance bug with list_add_global_unique().
Closes #3494.
Our shared enc_isupper/lower functions were stupidly slow. Fix them
and add enc_isdigit(). Using this, vastly improve the Jumbo-specific
pass in ldr_split_string() but make it optional (should have been
from the point of introduction).
Reduce pot sync log messages, only show them for mother process. Also
drop some old debug info, the pot sync functionality is well tested
now.
Turn off self_test_running flag earlier in case of --skip-self-tests.
Drop the separate GET_NEXT_MULTIPLE_VW macro and have
GET_NEXT_MULTIPLE vector-aware instead.
Comment tpyo in rules.c
Single mode: Dynamically increase SINGLE_WORDS_PAIR_MAX if we have a
large key_count, currently using log2(key_count).
Add john.conf option RelaxKPCWarningCheck, defaulting to true. If set,
we only warn if count is less than (min_kpc - min_kpc / num_salts).
This means we don't warn at all for single-salt/unsalted. For 10
salts, we allow 10% less than min_keys without a warning, for 100
salts we allow 1% and so on. See #3386
Update docs with warnings about MPI. Also fix some over-long lines
and miss-spellings of the terrific word "automagic" ;-) Closes
#3493.
Update docs with information on single mode operation with GPU. Closes
#3504
Change the default for RelaxKPCWarningCheck to false, but mention the
setting with the warning output (the 10th/last line, and only for
 main process). See #3386.
Single mode on GPU prioritizes speed over resumability. Add a user
setting to change that. See #3467.
Bug fix in the "marie -> mary" rule stemming from john proper. See
https://www.openwall.com/lists/john-users/2018/12/06/1
Single mode with OpenCL/ZTEX: Log SinglePrioResume setting.
OpenCL "speed index" function: When resorting to using vector width as
a factor, use sizeof(long) instead of sizeof(int). See #3502
Revise the workaround and debug stuff for OpenCL + MPI race condition.
We now only ever do it for MPI and only when actually running more
than one process. Also, we don't create any lock file - instead we
lock the kernel source file for writing (although we don't touch
it).
Prince mode: Ensure "last" buffer for rules_apply is aligned.
Print LWS/GWS info at normal verbosity, just not for benchmarks.
Optionally agree on LWS and GWS for whole MPI cluster.
OpenCL: Optionally synthesize -lws -gws to session file if applicable.
Rules: Add a hack for recognizing Hashcat's '-' command even if not in
"hashcat logic" mode..
Add some rules optimizations (mostly rejection flags/commands).
Add optimization flags to KoreLogic rules.
Stacked rules. New option --rules-stack=(SECTION[,..]|:rule[;..]) adds
a second pass of rules, applied after normal processing.
Document the important difference between --stdout=LENGTH and instead
using --stdout --max-len=LENGTH.
Change john.o optimization from -O0 to -O1 due to this:
Refactor FMT_UTF8 --> FMT_ENC. This is a no-op change, binary should
stay identical. See #3509.
Use the new FMT_UTF8 flag to ensure some format gets correct input
encoding. Closes #3509
Update .gitignore with some common test/temp-files patterns.
Introduce a few UTF-8-32 functions in unicode.c, and use them in the
Subsets mode. See #3510.
Halve the memory use for Single mode buffers, for OpenMP builds
without OpenCL support. We should make the choice at runtime
instead! See #3467
Mute stringop-truncation warnings from gcc. We believe they're all
false positives. We do NOT mute them for ASan or memdbg builds so
when we want to see them for auditing code, they'll be there.
Closes #3499
Replace some strncpy's where we require padding, with a function that
does the same but shouldn't trigger gcc warnings - and that has a
name telling a reviewer it's deliberate. See #3499, #3127
Single mode: - Bugfix: params.h was using HAVE_OPENMP where _OPENMP is
correct. - Let ZTEX-builds (non-OpenCL at that) use 16-bit index and
32-bit offsets just like OpenMP, for a usable 32K KPC and roughly
half the memory footprint compared to OpenCL which has 32-bit
indeces as well.
--list=build-info: Show effective limits for Single mode.
Fix "Proceeding with..." messages for lengths when hybrid mask is in
use.
Add compiler directives for suppressing some gcc format-overflow false
positives. See #3521
OpenCL: Drop some obsolete bug workarounds. See #3525
UTF-8 to UTF-16 conversions in shared code and OpenCL kernels: Drop a
few redundant checks that apparently wasn't optimizaed away.
Add a shared function for converting UTF-8 to UTF-8-32.
Self-test: Add a few sanity checks for format flags.
SAP H format: Drop an erroneous FMT_ENC flag as well as a
 re-implementation of fmt_default_split().
SAP H format really expects UTF-8 so we set that flag (with it's new
meaning as of recent changes) to get warnings if target encoding is
something else.
Update CREDITS-jumbo and CHANGES-jumbo.git
Fix undefined behavior, similar to eefe9047
Change nice(20) to nice(19), see #3513
Portability fix for idle.c
All perl scripts: Use perl from PATH instead of hard-coded
/usr/bin/perl
Cosmetic fixes to -list=build-info
Cosmetic changes to single mode warning texts.
Add a free() in cracker.c to aid in finding real leaks. Closes #3543
cracker.c: Change last commit so it's like upstream john proper
instead. The free was removed by Jim in de7b7c63 back in 2013 as he
changed the alloc to mem_alloc_tiny for unclear reasons. Then at
some point that was reverted by a core merge without the free
getting its way back.
ETA/Progress fixes (some were b0rken after --rules-stack).
Drop an accidentally committed debug line.
Tweak the human_prefix function for more correct output. Drop a
similar function in opencl_common.c and use shared one.
Single mode: Call crk_done() after we're finished. Closes #3554
WPAPSK salt_hash() improvement: Use the last four bytes of MAC address
instead of first four. Since the first three are vendor-id, they are
much less unique.
Add run/oui.txt as of 2018-12-26 11:04Z and a Makefile target for
manually updating it. See #3567
--stress-test option: Avoid printing warning more than once.
--stress-test option: Add stats.
All perl scripts: Drop '-w' when using 'env perl' and instead ensure
all scripts have 'use warning;'
Single mode "proceeding with" message showed wordlist rules intead of
single rules. Closes #3571.
pass_gen.pl: Stop using OpenSSL stuff where not needed.
Improve Single mode restore w/ --rules-stack. Closes #3541.
rules.h: Add missing "external" to a variable declaration.
single.c: Fix a hard-to-see typo introduced in 5be3ed4908
rules.c bugfix: Word pair rules with implicit '+' would not update
length so could erroneously fail min/max length checks.
Update misc.c
Slight change of logic for automagically increasing
SingleWordsPairMax.
unit-tests target: Fix backwards dependencies. (#3586)
AC: Do not enable unit tests when --disable-native-tests is used.
Closes #3601
UTF-8 tweaks for WPAPSK formats. Remove a ridiculous hard-coded test
in formats.c that complained about the change. Closes #3593.
WPAPSK-PMK formats: Bail out if user clearly has misunderstood the
format. Closes #3585.
Add a --single-retest-guess=BOOL option, overriding john.conf
equivalent. See #3594.
Fix an academic memory leak (only happens at exit) in single.c. See
#3594
Add a limit for recursion depth with SingleRetestGuessed. The default
limit is 10000 but can be overridden in john.conf. Closes #3594.
Bugfix in leet format's valid(). Closes #3584
Re-print the single mode warning about disabled recursion at session
end. (#3604)
unique: Add support for large buffers/hash tables, and other tweaks.
unique: Improve options parsing (any option can now come before or
after the output filename).
Unify hccap2john output format with wpapcap2john (don't dash the
MACs).
Add a safe-guard against logins/plaintexts that are printed to the
screen with some weird encoding upsets your terminal.
Fix Progress/ETA bug introduced with stacked rules.
Fix a division by zero in unique.c
cracker.c: Detect when a reset() call lowered min. kpc, adjust
warnings accordingly (#3674)
Fix for "make clean" on eg. OS X w/ proper gcc after unit-tests
changes. Closes #3675.
No-op re-order of some john.conf sections before core merge.
axcrypt2 OpenCL kernel: Drop unused variable.
Decrease compiler optimization level for unicode_range.c and add a
dependency on it for subsets.o. Closes #3636
unicode_range.c: Work around brain-dead clang claiming to be
compatible with gcc despite it isn't. Closes #3691
Avoid printing device list errors more than once under --fork
OpenCL auto-tune tweaks. Add backward GWS runs.
Revision of x86-sse.h and x86-64.h after core merge. Also drop some
obsolete stuff from DES_bs_b.c, see #3696
Change auto-tune password length, as well as length of default mask
for "-test -mask", to 7.
Update CHANGES-jumbo.git and CREDITS-jumbo
cprepair: Add -P option, don't output unprintable ASCII (and with -n
also suppress any invalid UTF-8). Also some minor bug fixes.
Bugfix after adding VERB_DEBUG. Closes #3730
Minor fixes while working with #3712
Bugfixes for hybrid mask vs. eff_maxlength. Closes #3712
Cosmetic fixes, #3718
If default (john.conf) mask is used, print it. Closes #3728.
More hybrid mask length fixes. Closes #3731
Print a warning when -verb:1 suppresses -stdout candidates. Closes
#3729
Print a notice of default input encoding if != ASCII and --pipe.
Closes #3717
AC: Fix intel 32-bit issues. Closes, I hope, #3678
Mask mode: This change closes #3735, found by lucky hip shot. I hope
it doesn't end up with regression in other cases - testing needed.
Show default input encoding for wordlist mode as well, see #3717
Make us sending host's sizeof(size_t) to OpenCL less confusing.
Work around idiot Beignet driver defining cl_amd_media_ops despite not
supporting it. Closes #3750.
Send -D__BEIGNET__ or -D__POCL__ to OpenCL runtime compiler when we
suspect platform is one of those, for easier future bug workarounds
in kernels, just like we do with Mesa. See #3750.
Mask mode bug fixes. Reverts previous bugfix, which wasn't complete.
Instead adds a correct fix for hybrid mode, correcting a change I
did in c1d7a2f for incrementing lengths (which we never do in hybrid
mode).
Do not disable self-tests when running "-test -mask". Instead do a
normal non-mask self-test, then reset format and benchmark with
mask. Closes #3249
AC: Do not auto-enable rexgen if library found. See #3753.
Tweaks for mask-benchmark w/ proper self-tests. See #3249
Some mask mode fixes for non-mask self-tests followed by mask
benchmarks. Problems seen with raw-md4-opencl followed by
 raw-md5-opencl using -test -mask -form:raw*opencl
Revert "Some mask mode fixes for non-mask self-tests followed by mask
benchmarks." This reverts commit
b2aba18f4ad7cfb86a9093e14df48527ef97d329.
Rename --nolog option to --no-log. The old name will continue to work
for some time but is deprecated.
Tweaks to ldr_split_line. See discussion at
https://www.openwall.com/lists/john-dev/2019/04/07/1
HDAA: Drop b0rken OpenMP support, since it was poor anyway. Closes
#3107.
7z formats: Use atoi() instead of atou() if CRC field is negative,
then cast to unsigned int. Also change atou declaration to
"unsigned int" as opposed to just "unsigned". Closes #2998.
hccap2john.c: Drop an unused function
Add a warning about self-tests currently being disabled with -test
-mask.
Revert part of 4b6929031ad. See discussion at
https://github.com/openwall/john/commit/4b6929031
Revert "Many OpenCL formats: Use new scheme of LWS=NULL for
less-than-full batches" This reverts commit
0d3ea928cbdb2c2a861c0ecd3c00fc2113213387.
Jumbo changes according to recent core changes ("NEON"->"ASIMD" for
aarch64). Also test for __aarch64__ in case it doesn't already
define __ARM_NEON.
OMP tune, a bunch of formats. See #3091
Various updates to install instructions. See #3513, #3753
Update relbench
Fix copyright blurb in showformats.[ch]
Fix a type for SHA-512 using 128/128 ASIMD: "4x" should be "2x".
Change RAR3 formats to benchmark length 5, just like hashcat. See
#3697
Mask mode: For -test -mask, give format a mask corresponding to its
requested benchmark_length. See #3697.
Major bugfix for -test -mask. That functionality wasn't added to the
newly merged code path. See #3697.
Final (hopefully) fixes to mask-mode benchmark after merge.
Mask-mode benchmark: Add/replace benchmark_comment with "(length n)"
if a custom mask is being specified. Also separate used mask from
requested mask (so we can see whether mask is default or not).
OMP_SCALE & MKPC tuning, many formats, Closes #3091
More doc updates for regex / librexgen.
Update CHANGES-jumbo.git and CREDITS-jumbo
Add the john.conf [Debug] section for legacy benchmarks.
mask.c: Export a function for evaluating mask_add_len from a mask
without calling mask_init(). Use said function when benchmarking
with mask, for getting a proper (length n) message. See #3780.
Add cmp_all() to auto-tune, and have all formats calling OpenCL
functions from cmp_all profile it. See #3779.
Autoconf: Add -fno-strict-aliasing to NEON and ASIMD builds. This
pacifies numerous compiler warnings from DES_bs_b.c
Fix some GPU device number reporting problems. Closes #3639
If building BE, print a notice that OpenVMS format is disabled. Closes
#1474
Fix the last few OpenCL formats that didn't set GWS to next LWS
multiple larger than count. Closes #3489.
Fix a silly bug in new autotune code. We were trying to call binary
with a "ciphertext" that was already a binary. Closes #3788.
Cosmetic bugfix for node/fork OpenCL.
Add multi-word rules, and include them in "jumbo" rule-set. Closes
 #3789
Un-comment a number of config options, so they're seen with eg.
--list=parameters:options:opencl (see #3598)
Revert all core parts of 15163f4. We need to regression test the
changes better before releasing it. See #3517, #3468. We'll
revisit this after 1.9.0-Jumbo-1 release.
Mute some diagnostics from yescrypt, at least for now. Also pacify a
strange warning from clang stopping us from doing sanity checks,
WTF?
Disable monero format for BE builds. See #3513
Fix salt hash size in drupal7
BF-opencl: Workaround for Sayantan's strange keyboard. Oh, and a
salt_hash() fix that was missed until now for that reason.
nsec3 format salt_hash fix (was using ancient fixed salt hash size)
Update CHANGES-jumbo.git and CREDITS-jumbo
params.h tweaks for future Jumbo
Fix some algorithm names that were incorrect. See #3513
Lots of lgorithm_name fixes, and mute another yescrypt build warning
(notice). See €3513
A few more algorithm_name fixes. See #3513
Work around some weird problem with dynamic cmopiler format, #3796
SIPDump and eapmd5tojohn: source <getopt.h> instead of declaring
extern optind. See #3799
Fix a -Wdiscarded-qualifiers in opencl_common.c for SYSTEMWIDE
Revise INSTALL-UBUNTU. Add OMP fall-back.
--show=format[-all]-details fixes for new benchmark_length
Minor bugfix for special many-salts benchmark --test=-1
Don't suggest setting RelaxKPCWarningCheck when it's already set. Also
make message a little shorter, avoiding a line wrap decision.
7z formats: Move the compile-time TRUST_PADDING macro to a john.conf
option, so user can decide. Have it default to true, since we've
only had a single report of false negative and that was an archive
of dubious origin. Closes #3806.
Disable the generic crypt(3) format on CircleCI, it seems to always
fail now. Closes #3867
Relbench: Avoid division by zero, closes #3878
RACF: Adjust OMP scale after testing on super, see #3091
Re-tune krb5-db and o5logon for super. See #3091
Avoid inconsistent messages from -test -mask, closes #3875
Update CREDITS-jumbo and CHANGES-jumbo.git
loader.c: Only sort salts for real db's. See #3091.
OMP auto-tune: Print any messages to stdout instead of stderr.
formats.c: Add a missing space in error message.
Fix a logic bug in self-tests.
7z formats: Add support for "deflate" compression, kudos to philsmd.
SL3 and salted-sha1 opencl formats: Revert to using LWS=NULL when
count is lower than GWS. I just can't find whatever bug is triggered
by not doing this. Closes #3880.
Mute warnings from gcc 9 and from new OpenCL headers.
CRC32C: Report AVX if applicable Stribog: Report AVX if applicable
Pomelo: Report AVX/AVX2 if applicable
Drop raw-sha1-ng format. It's obsolete. See #3881
Drop raw-sha256-ng format. It's obsolete. See #3881
Drop raw-sha512-ng format. It's obsolete. See #3881
Drop Alain's good old NT format. It's finally obsolete. #3881
Drop Pomelo format, it's obsolete and probably not used anywhere. See
#3881
External mode: Fix a problem with UTF-32 -> UTF-8 length, so we behave
like the comments in eg. dumb32.conf says we do.
 Suppress "Many salts" benchmarks where appropriate. Closes #3795.
Update CHANGES-jumbo-git and CREDITS-jumbo. See #1911.
Just demonstrating why even I should always go through bot checks :-/
Bail out with message if running legacy benchmarks with --mask.
Fix legacy builds, see #3875
Avoid a problem casting size_t to negative and then to off_t (#3904)
Bugfix (again) for UTF-32 vs. UTF-8 length issues with external mode.
This needs to be fully reworked post release of 1.9.0-Jumbo-1, it's
still messy and we've got catch-22's.
john.c: Avoid printing many of same error message under MPI.
Add the debug code for #3795 as a config option.
Move bench debug code to correct position
Fix and complete one/many salts debugging
Unsplit test vectors that were split through use of backslashes inside
the string constant. See #3896.
Fix one/many vs. raw benchmark output for a number of formats (#3906)
Repurpose and refactor aix-smd5 to md5crypt-long. Closes #3872.
Drop doc/BUGS
Move NEWS from GitHub wiki to doc/NEWS
Check OpenCL headers version and bail out if 1.2 is not supported. See
#3918.
Update CHANGES-jumbo.git and CREDITS-jumbo, see #1911
Re-unify raw-md4 and raw_md5 formats, fixing bugs in the former.
Closes #3916.
Fold long lines in doc/NEWS
Ensure we never benchmark less than two salts for "Many salts".
(#3922)
Save the script used for #3795 under unused
Save my template bisect script under unused
Update CHANGES-jumbo.git and CREDITS-jumbo, see #1911
Fix salt alignment in a bunch of formats, see #3932
Better error message when some rule is missing in a list, eg.
--rules=one,two. Closes #3935.
Fix non-SIMD blake2 aliasing violations leading to failures. Closes
#3079, See also #3932.
Disable STRIP & Enpass formats on S/390. Bod knows why they fail.
(#3940)
Comment out some unused stuff from ed25519-donna code
memory.c: Move the dump_stuff* debug functions out of #if SIMD...
blocks, it was breaking "make debug" on non-SIMD targets.
Minor fixes to doc/INSTALL and doc/INSTALL-FEDORA
Rename CHANGES-jumbo.git to CHANGES-JUMBO and update it. The older
CHANGES-jumbo contained information already in other files (but
shorter, more like our new NEWS file). Closes #1911.

JimF (1545):
configure: removed many function checks (for #910)
phpass: non-OMP builds were no SIMD, fix #916
configure: Added arm-NEON instruction detection logic, #911
change execute bits
configure: removed many size checks per #910
bleeding-jumbo version. Set to 1.8.0-jumbo-1+bleeding-jumbo
bleeding version: changed per magnum
configure: added LARGEFILE_SUPPORT to many 32 bit systems which were
missing it
configure: LARGEFILE_SUPPORT, added a check for 32 bit systems
configure: broken case statement
wrap all include limits.h which were not prior wrapped
wrap all include locale.h which were not prior wrapped
haval: failure to build after HAVE_TYPES_H code added
sph hashes: failure to build after HAVE_TYPES_H code added
configure: header wrappers for termios.h and sys/params.h
autoconf: updated logic for _LARGEFILE64_SOURCE, added -DEBUG to ASan
added -BSD to 32 bit freebsd ASFLAGS
autoconf: Removed the header includes that were not needing wrapped
MODES: document. Added Markov/Mask. Fixes #904
base64_conv: fixed length is get_len where mime has trailing =
pass_gen: added sapH
New POC format, sapH, still very slow
pass_gen: some debugging stuff left in sapH, removed it
sapH: added OMP, still only oSSL
sapH: added cost and did some small speedups reusing buffers better.
pass_gen.pl: added SIP hash
sip: added new test vectors made with pass_gen.pl
SIP: changed to use proper binary, vs the cracked way.
loader: fixes #930 dynamic hex not being removed from .pot load
Makefile.in some defines needed. Problem showed up in #933
cygwin: Fixed #932 fseeko link fail GetFileSizeEx
SIP: Fixes #923 Format now actually works
SIP: a bit faster, and smaller salt. Preload CTX
SIP: wrong buffer size used
listconf: format-tests. If nothing can be output, then do not output
loader: incresed number of parms being loaded from file, for scrypt
scrypt: ScryptKDF.pm data was NOT loadable from a file. #938
scrypt: fixed 938
scrypt: changed ScryptKDF signature to be a single field
truecrypt: fixed unified format. Fix #941
rsvp: fix #937 Caching had to be for each type
rsvp: use array for new-keys, and made more extensible
rsvp: depending upon clearkeys does not work for self test code
pass_gen: added RSVP hash
dyna_salt: Fix salt_smash_check. Fix #920
Bugs in split and FMT_SPLIT_UNIFIES_CASE fixed
self-test, always call clear_keys before adding new keys
gost: needed split-unifies-case
Added FMT_SPLIT_UNIFIES_CASE to these are formats with hex hashes
places ishex() and other helper functions into common.[ch]
moved include from .h to .c after 4770e4c
blackberry-es10: added UC only validation of hash
HDAA: added proper valid
IKE: improved valid to only accept lc hex
LUKS: improved valid to only accept lc hex
PDF: improved valid to only handle lower case hex
rawsha512_opencl: fixed split non-case issue
base64_convert: added upcase/locase to length of hex. Reformatted .h
file
removed execute bits
HDAA: fixed dupe salt detect logic. fix #951
HDAA: stop silly memory usage, and free things up, during salt()
autoconf: added a couple echo statements for plug processing
autoconf: added probe and check for perl before usage
formats.c: salt dupe logic working for dyna_salts also
kwallet: fixed salt dupe problem (#953)
formats.c: some helpful commentted out code for tracking down salt
dupe problems
base64: reduced memset overhead. Only clear bytes when needed
dynamic_21: removed this format hash totally
Build: hopeful fix for aes.a always built, #957
pfx: added comment about salt
build: fixed circular autoconfig-stamp-h problem, got auto-autoheader
working again
dynamic: disable a lot of dynamic hashes
formats: allow OMP_SCALE modification at runtime, in john.conf
OMP_auto_scale: I had not left the multiplier at 1.0, my bad.
blockchain: OMP_SCALE 2.3x increase on AMD, please check on Intel
crc32: OMP_SCALE 42% increase on AMD, please check on Intel
crc32: scale not set on last commit
EPI: added OMP, and OMP scaling
Fortigate: better handling of OMP_SCALING
gost: OMP_SCALE tuning
dahua: OMP_SCALE tuning
hmacSHA2: OMP scaling
mongodb: OMP scaling
mssql12: OMP scaling
mysqlna: OMP scaling
postgres: OMP scaling
racf: OMP scaling
sapB: OMP scaling
ssh_ng: OMP scaling
omp_scaling: added comments on a couple formats
sapH: changed OMP scaling computation for later in SIMD
oldoffice: OMP Scaling
VMS: OMP scaling
tcp_md5: OMP SCALING
gost: OMP_SCALE, was wrong scale
mss1l12: wrong OMP_SCALE used in ad8faae
mysqlna: wrong OMPSCALE used in 2221cbb
postgres: wrong OMPSCALE used in 3524196
IPB2: changed how OMP_SCALE is built in init, and adjusted scale a bit
EPI: forgot to add FMT_OMP to flags
sapB: valid() fix #959
sapG: valid() fix #960
base64_convert: fix #968
base64_convert: pure formatting difference, no code change
mediawiki: fix #961
added element to gitignore after fixing makefile
formats.c: handle possible floating point rounding issue
unicode: overflow, fix #963
zip format: enhanced valid, fix #966
7z: fixed valid() bug: fix #962
base64_convert: handle to_len. Fix #965
configure: plugin_deps.pl skip warning about autoconfig.h fix #970
plugin_deps.pl: autoconfig.h was not output if not there. File needed
closed also
base64_conv/lastpass_sniffed: last buffer overread problems fixed.
Fix #964
base64_conv: finished up mods. Now all converts honor to_len
encfs: fix bug #971
HDAA: fix #972
pkzip: fix #973
pdf: fix #974
office: valid fix #975
KRB4: bug fix (realm buffer too small), and valid() fix #976
pbkdf2-hmac-sha512: prepare bug: fix #977
KRB4: removed a warning
pbkdf2-hmac-sha512: fix #977
office and opencl: created a office_common set of functions
office-common: removed execute bit
office-common: comment and eol at end of file. NO code change
encfs and opencl: created an encfs_common set of functions
md5ns: Converted to thin dynamic format. OMP/SIMD benfits
md5ns: warning of unused variable
cryptmd5: created an cryptmd5_common set of functions. Added {smd5} to
GPU, but not working
luks: Here is a version that is twice as fast, and does not have ASAN
errors
LUKS: added SIMD code
md5ns: changed so that native hashes get written to .pot file
md5ns: improved ALGORITHM text
bcrypt / opencl: created a BF_common set of functions / data
dragonfly: OMP_SCALE adjustments
sha1crypt / opencl: added sha1crypt_common functions and data
dyna_salt memory leaks, reported on #980'
memdbg: added 4 new information getters for memory blocks
dyna_salt: Updated. Changed dyna_salt_create to actually do something,
in memdbg mode
cuda_cryptmd5: fix #984
dyna_salt: removed redundant code
pass_gen.pl added dyna types: 1013 1016 1400 1401 1501 1502 1503 1504
dynamic: larger hashes. Fixed max PLAINTEXT length, as found by test
suite
dynamic_2010/2011: fix #987
dynamic flat hashes. Same issue as fixed in 3494a53
cryptsha512: was not cleaning its salt buffer
dynamic: flat SIMD buffer cleaning bug fixed
ssh-ng: add #debug compile switch to new type2 bcrypt hash. Fixes
-test time to execute
zip: retuned OMP_SCALE, and -test wallclock is much better, and c/s
are same
rawsha256-cuda needed a wrap
cryptsha256: added tests into common code. Fixed a couple of failure
to zero out salt issues
pass_gen: a couple fixes after last change adding all the dyna, to get
-tstall to work properly again
cp737 encoding fix for greek lower final epsilon. Fix #993 fix #994
cp737 lower. There is a LOW_ONLY field. That is what should have been
used, and now is
additional fix needed after d33e727
rar: propre fix for #1000 and #1001 ASAN read issues
rar: removed debugging line
removed comments no longer required, and set allocation back to what
it was
autoconf: check buildability of asan. fix #1006
list=build-info: output memdbg, DEBUG and ASAN status
rawmd5u: fixed overflow bug, found by ASan
small edits listing user number on hash
mscash1: changed to allow standard OMP_SCALE adjustments
mscash2: changed to allow OMP_SCALE, improved performace a bit
pass_gen: allow unicode input and base64 output, by splitting out
unicode input as a new function
dynamic: better base64 handling. Added peoplesoft hash
pass_gen: added dyna_1032, peoplesoft hash
dynamic: fix to get phpass working. It uses CryptBS layout
mediawiki: thin dynamic, now store hashes in the native format into
the .pot file
phpass: changed thin format to store native hash in .pot file
mediawiki: small clean up after .pot output format change
phpass: cut/paste coding bug
PHPS: thin format now outputs raw original hashes
base64_conv: problems handing NULL data, for crypt conversions
rar: ASan bug, #1010
wordlist: unfreed memory block in -pipe mode
bench_cost: got dyna_salt remove working properly
dynamic: now fully handles base-64 of MIME, Crypt and CryptBS types.
Fix #1007
dyna_1033: changed expression shown on screen output
dynamic_1033: hide this format. It is only a fake testing format
dynamic documentation: #874. There may still updates to be made
IKE: fixed salt-dupe failure problem, #953
SSH: added dynamic salt. Removed salt-dupe problems. #953
PFX: dyna_salt. removed salt-dupe problem. #953
dyna_salt.h: needed a #define wrapper
pkzip: converted to dyna_salt. Removes dupe-salt problem. #953
pkzip.h: had some code left for an earlier tried method, that was not
needed
relbench: [handle the new speed for cost] message ignored. Fix #1013
PRINCE: make sure to disable in john.c if no GMP
ecryptfs: error, valid not getting only the $1$ signature
ecryptfs: completed fix #1017
dyna_salt warning correction. Fix #1021
removed DOS line endings
prince: removed warning, AND better handle this in non-memdbg and
memdbg modes
pass_gen: added 2 new pbkdf2-sha1, fixed line wrap problem in base64
code
pass_gen: changes for rsvp processing
pass_gen: added aix-smd5, and some -mode=xx to allow certain formats
to share salts
wowsrp: This format was broken. It only handled even 64 byte binary,
but for modular math, this will NOT be the case
pass_gen: added sha1crypt, tiger, whirlpool. Modified pp_pbkdf2 to
also do pbkdf1
pass_gen.pl: pbkdf1 caused bug in pbkdf2. Fixed
chap: enhanced valid
pass_gen.pl: added, hsrp known-hosts chap and dynamic_1010
wowsrp: small changes. There should have been cases still not
correctly finding values, even though they did not show in the TS.
Lack of memset for nonGMP, the todo length was wrong on GMP, and
likely would have caused ASan overread errors.
pass_gen.pl: added blackberry-es10, citrix-ns10 and clipperz-srp
hashes
clipperz: Fixed this SRP format. Fix #1028
wowsrp and clipperz: added split and proper canonization code. Fix
#1027
dahua: it is a raw format, but we were treating it salted in tests
added dahua. Some cleanup on clipperz-srp
pass_gen.pl: added fortigate, lp and lastpass
lastpass_sniffed: salt-dupe code was broken. #1031
lastpass_sniffed: Wrong (WIP) code left in non SIMD section
scrypt: salt-dupe problems. #1031
truecrypt: #1031. Only a comment, since this format does not have
duplicate salts.
Added hashes for: rawmd2, mdc2, mongodb, mysqlna, o5logon, postgres.
Added get_username() but not heavily used yet
pass_gen.pl: added pst format
pass_gen.pl: added raw-blake2
luks: fixed dupe salt problem (dyna salt). #953
added keccak (512-256), siemens_s7, skein (256-512, BUT they are not
compatible), ssha512 and tcp-md5
prince: portability stuff
pass_gen.pl: fixed small bug found in dupe-salt script running
mozilla: fixed dupe salt problem. #1031
pass_gen.pl: some changes needed help in the salt-dupe finding project
SKEY: fixed salt dupe bug. #1031
pass_gen.pl: added strip format
pass_gen.pl: ns-md5 was nto working any more. Fixed
wpapsk: salt dupe removal fix, remove binary from salt. #1031
pass_gen.pl: code changes to allow some formats to use the same salt
values
pass_gen.pl: added stub functions for ALL current JtR formats not yet
handled
Salt Dupe removal code broke format due to salt hashing
dynamic_fmt: big bug in NON SIMD builds. Found from warning messages
on a MIPS-64 build
dynamic: changed location of min-plaintext-length
sha1crypt_common.c: Error compiling on BE systems
hmacSHA384/512: Fixed failure on 64 bit BE systems
formats.c: added some additional debugging information when we have
hash/binary failures
krb5-18: BE64 port, and some simplification
dynamic_17: fixed be issue on mips64 build: #1032
unicode.c: E_md4hash fixes on BE allowing unaligned access, mips64be
EPI: fixed core on unaligned systems (salt hash accessed 4 byte align)
md5ns: fixed core on unaligned systems
truecrypt: fixed BE problems with this format
cloudkeychain: BE failures fixed
LUKS: BE fixes
EFS: fixed BE problems
added strrev function to configure and supporting code
mpz_int128: removed recursion from in128tostr and about 4x faster
PRINCE: problems when building using GMP
configure: ARCH_ALLOWS_UNALIGNED moved to static headers. Fix #1034
pass_gen: comments only
scrypt: corrected pointer alias issues causing scrypt to fail on
MIPS64. #1032
formats.c: Added some debugging code, to help in porting to BE, or
finding other errors
pp.c: 32 bit issue, and GMP issue fixed
removed spurious file. This header is in escrypt, and somehow an old
version was put into src root
sybase-prob (feal8), fixed on 64 bit BE, #1032
another spurious header from escrypt folder
formats.c: inadvertant code removed in 131064f
formats.c: re-enabled extra debugging for get_hash failures
office_2007: salt dupe logic. We decrypt now, vs encrypt.
pass_gen: phase1 of rewrite, to unify output. #1039
Makefile.legacy: added a couple newer objects
pass_gen.pl, updated to allow Lophtcrack type hashes to work with
unified output
pass_gen.pl: added bitcoin
dynamic: some length problems in the preloads. x86 shorter than SIMD
Dynamic: changes made to thin formats. Set plaintext length(if thin
sets it to 0). Also, salt_compare function was missing
mediawiki: upgraded thin format for better linkage to dynamic
dynamic.conf: comment about dynamic_1033 missing
pass_gen: dynamic_1033 was missing
md5-NS: upgraded thin format for better linkage to dynamic
formspring: upgraded thin format for better linkage to dynamic
asaMD5: upgraded thin format for better linkage to dynamic. NOTE,
broke before. plain-len was set to 32, but max should be only 12
osc: upgraded thin format for better linkage to dynamic.
pass_gen.pl: dynamic_17 was totally busted, trying to set specific
salts
phpass: upgraded thin format for better linkage to dynamic.
pixMD5: upgraded thin format for better linkage to dynamic.
pass_gen.pl: recent change, the unified output, broke PHPS
PHPS: upgraded thin format for better linkage to dynamic.
options/listconf: added asAN and unified version string. Fix #1005
crypt-descrypt: for length-13, we never disable, since they are in ST
code
added UID to crack outputs (optional). Issue #517
wpapcap2john: placed bssid into uid field, we had the connecting mac
in that field before
hmailserver: converted to dynamic_61 thin format. Now has OMP, and
SIMD usage
sapG: improved OMP scaling (slight, only 10% or so improvement)
XSHA512: improved OMP scaling
logger: helps with -fork multi process screen overwrites. DOES NOT
fix, just makes better
changed _asAN to _asan for JtR version string
dynamic: FLAT and not-sse2safe salted formats extended length of salt
and passwords
john.c: in cygwin, some aborts leave tty input in unstable state.
CYGWIN only
cygwin: -fork updates. Also improved logger.c LOCK_DEBUG logging
cygwin: removed prior jtr stty echo at exit.
logger.c: added fcntl locking (ONLY for cygwin at this time), vs flock
pp: mute some warnings
dynamic_1505: was crashing on travis for clang builds
dynamic_1505: input length fix, (again)
dynamic_1505: converted to flat format only. Non flat could only have
11 byte password, with 12 byte salt. Just too small
john_bashcompletion: Update to handle new usage screen that does not
list formats
options: remove format list from normal usage string
options: usage string slightly changed
pass_gen: added agilekeychain
pass_gen: agilekeychain code was not properly saved. a few changed
pass_gen: added blockchain
blockchain: improved detection and eliminated FMT_NOT_EXACT
blockchain: first check was looking over too much data.
blockchain: Now handles the V2 hashes. We may want to work more on
this, and use native mime hashes, etc
blockchain-opencl: Added v2/iteration changes to opencl version
cryptsha512: changed comment to match actual code
cryptsha256: same comment problem as seen on cryptsha512
cryptsha256: got SIMD code working
memory: added dbg funtions for 8 byte SIMD not doing byte swapping,
i.e. SHA512, but flat buffer
cryptsha256: some tweaks done when porting code to cryptsha512
cryptsha512: added SIMD. Also moved test vectors to common code
cryptsha512: bug in non-OMP builds
cryptsha512: error, any 2 limb hash was broken. All passwords over 15
 bytes were failing
cryptsha256 and 512: fixed for 32 bit cygwin OMP builds (worked around
compiler bug)
cryptsha: removed debugging code. These formats are working fine now
cryptsha512: added more test vectors, and found bug for length 32 and
48 hashes
hmac-SHA256/224: added SIMD
hmacSHA256/224 some small fixes, found when porting SIMD to sha512
algorithms
hmacSHA512: added SIMD
hmac-sha512/384 would not compile in non-SIMD mode without this change
hmac-SHA384: added SIMD. Some debug code still commented out, for now
hmac-sha256/224, use PAD_SIZE for the prep_ipad. It is the correct
size
hmacSHA512: removed some DOS line endings. Switched to using PAD_SIZE
for size of prep_[io]pad
hmac-SHA2 stuff: Removed debugging commented out code. Changed
OMP_SCALE for sha512 formats, SIMD
xsha512: added SIMD
removed DOS line endings
hmacSHA384: bug fix and a couple changes trying to find the OMP bug
#1063. Still a problem
ssha512: added SIMD
ssha512: error building/running NON SIMD
hmac-md5: change to prior commit. #1067
configure: removed memdbg-ex from configure. That mode was written as
a last resort, hail marry attempt to find memory bugs. It does not
work for all formats, and should not be used that way.
XSHA512: inc needed added to pragma for non-SIMD. Not sure why it did
not complain on SIMD builds, but I added it there also
XSHA512: fixed bug for passwords under 4 bytes, in SIMD code. Key was
not cleaned properly
base64-conv: fix for input text length being more than actual
available input text. Fixes #1071. A new -l test added also
base64-conv: some fixes for last commit
base64-conv: some fixes for last commits
base64-conv: switched from strlen to strnlen
drupal7: Added SIMD logic
drupal7: minor text editing
office_2007: SIMD code added
office-2007: removed the crypt buffer, was not really needed
office-2007. Changed interface to non MMX function to simplify
crypt_all code. Same type logic will be used for 2010 and 2013
office2010/2013 added SIMD. Debugging code still there, commented out
office2010/2013 debugging stuff removed
office simd formats. Some simplifications
office: lengthened plaintext length and removed a couple of warnings
office: added max length password test cases
ecryptfs: added SIMD (debugging code still there, commented out)
ecryptfs: cleaned up comments
pwsafe: Added SIMD code. 4x improvement
keyring: added SIMD instructions
SAP-H: added SIMD logic
sse-intrinsic: SHA formats, added SSEi_FLAT_RELOAD_SWAPLAST flag. Fix
#1082
Sybase-ASE: added SIMD logic
blackberry-es10: added SIMD instructions
pass_gen.pl: added ability to do shorter salts for sha256crypt
cryptsha256: handle shorter salts on SIMD code.
pass_gen: special code got left in
sha512crypt: added SIMD ability to handle shorter salts
cryptsha2: added code to help easily build withing SIMD (for testing)
net-[md5|sha1]: Salt align problems on clang. #1081
cryptsha2: mute warnings. Fixes #1080
sybase-ase: unify case, and improve valid. Fixes #1087
dynamic: SHA384/SHA512 added SIMD instructions
(magnum:) Re-applied Jim's "bcrypt / opencl: created a BF_common set
of functions / data" (but updated to current core). Original commit
8c0a72f.
siemens-7 had 2 $ in the hash.
siemens: only validate a 1. Since we now have split, a 0 should not
be valid any more
dynamic: in a debug build, we do NOT disable any dyna format. #956
dynamic: calls dynamic_FIX_SALT_TO_HEX if hash is dyna on --show=left
fixes #986
MMX_COEF_SHA[256][512] changed to SIMD_COEF_[32][64]. Also changed
PARA for SHA2 functions
cryptsha[256][512] a little cleanup of MMX name tags, also goes with
#1099
dynamic:sha384. Some typo corrupton
SIMD: #1099. Switched from (SIMD_COEF_32>>1) to SIMD_COEF32_BITS.
Added that define to x86-sse.h and x86-64.h
SIMD: #1099. Changed define name of MMX_TYPE to SIMD_TYPE_STR
after const added to count, gcc at times in the pragma omp
blockchain: fixed promiscuous valid issue once we change to strtokm
django_scrypt: issue seen when switching from strtok to strtokm
SKEY: do not use strtokm at this time.
gpg: change to work with strtokm, skipping over prepended delimiter
kwallet: fix for promiscuous valid when switching to strtokm
mozilla: fix to skip prepended delimiter, strtokm
ssh-ng: fix to work properly with strtokm
strip: Fix to work with strtokm
memdbg: Try to maintain 16 byte alignments, since we are using allocs
now for crypt/input buffers
dynamic: Regression problems (32 bit and memdbg) in dynamic after
3b6727f
dragonfly: we missed a done
dragonfly: we missed a done
net-md5/sha1. Quasi thin format. We have to call dynamic.methods.done
by hand here, since we overloaded it
memory.c: alignment versions of mem_alloc and mem_calloc
dynamic: redone, to use the aligment versions in memory.c
dynamic: redone, to use the aligment versions in memory.c, prior patch
was incomplete
dynamic: redone, to use the aligment versions in memory.c, prior 2
patches were incomplete
dynamic: redone, to use the aligment versions in memory.c, prior
patches were incomplete
mssql12: buffer alignment with new mem_calloc for SIMD code
many formats. Fixed 32 bit builds for SIMD after moving to calloc
many formats. Fixed 32 bit builds for SIMD after moving to calloc
many formats. Fixed 32 bit builds for SIMD after moving to calloc
memdbg: got this working with new mem_aligned code. Also got VC ported
over
memdbg issues and a memory leak found in VC port
cygwin. Do not add _XOPEN_SOUR=600 define for _posix_memalign. It
causes tons of warnings, and cygwin works without it
memdbg.h missing from these 2. Also a memory leak found and fixed once
 memdbg was there, in office
raw-sha512-ng: fixes crash in 32 bit builds, alignment
VC port, without memdbg with new _malloc_aligned
VC port, _malloc_aligned. Hopefully last fixes
VC portability issue (_malloc_align)
misc.[ch]: added new strtokm function, and a #define to switch all
strtok to use it
Mute warnings about different parameter types in function declaration
vs definition
memdbg: mute warning, size_t vs long param issue.
VC compiling in jumbo.h
plugin.c: added a comment about very suspect looking code
config.c: switched to using strtokm
markov: use strtokm and now start and end can be empty, with just the
: chars
raw2dyna switched to using strtokm
raw2dyna: revert strtokm change
markov: level is now an optional param
Some SHA512 formats were getting failures on VC debugging builds
memory.c: bug in non-aligned arched for alter_endianity_w64.
mask_ext: added missing memdbg.h include
dynamic: binary align to 8 bytes
cygwin build problem fixed. #1250
dynamic: fix #1208 warning about pointer bool conversion.
regen-lost-salts problem. Fix #1154
dynamic format honors --disable-omp-for-fast-formats. Fix #1334
dynamic disable OMP was not right. It always was disabling. Now it is
controlled by configure, and can be turned on
-stress-test problems for thin dynamic formats. Fixes #1135
required changed to get MSVC to build again with SIMD
dynamic: work to get SIMD_PARA working for dyna with multiple hash
types, where the para do not match
dynamic: fix problem where including dyna headers could turn off OMP
in john.c
dynamic PARA problems fixed core crash for PARA=4 md4. #1127
rar2john. Removed warnings seen on 32 bit builds
dynamic. After 3e0b41d 32 bit builds where we had MD5 asm were failing
at link. #1334. NOTE, dyna 1401 and 1503 core and I do not know why,
yet
MSVC issues with OpenMP build.
dynamic: honor --disable-omp-for-fast was broken.
dynamic: removed MD5_Go code. #1298
changes needed to at least compile on my sparc
pufferfish. jumbo.h required for VC to build (no snprintf)
has-160: fixed BE failure
dominosec8: ported to work for BE systems. #1345
dominosec8: memdbg found a memory leak
stribog: memory leak
PO: removed MD5_go logic, replaced it with oSSL
dynamic: Dos line endings
md5_go: removed this module. Fixes #1347
strcasestr: added to autoconf for systems without this function. Fix
1366
small fixes to b383434
has160: required memdbg.h being added
many source files missing memdbg.h
pomelo: after adding memdbg.h, the format was crashing in memdbg_on
builds
memdbg: allow greater than 16 byte alignment. Fix 1200 Never align
 better than requested Fix 1195
aix_ssha: This would not build on non x86 systems
memdbg: added memory.h include to the memdbg header
memdbg: improved test code in john.c by adding the memdbg checks on
exit
hmacSHA384: fixed core on non-aligned 32 bit systems
XSHA512: would not build on non x86 systems
memdbg: bug fixes in EXTRA checks, after recent changes
XSHA512: fixed failures in non SSE builds
dynamic: updated todo comments with items which have been completed
memdbg: the dereference of uint32 data for fence post checking is no
longer done, so remove those unused defines
sse-intrinsics: wrap the sha256 and sha512 in #define logic
x86-64 non-SIMD build help. #1370
x86-64 added -mno-mmx to the ignored list. #1370
john.c: setvbuf portability issues. Likely there are other
portability issues using that function
final status line in --wordlist mode did not contain percentage. Fix
#1372
markov: fix core if force run where level is too small
PHPS: address sanitizer found bug. This format requires 3 byte salt
(legacy code build). Fix #1261
dynamic: removed some older todo comments. Allow better debugging
within VC
dynamic_1506: remove the StartInX86Mode flag
memdbg: mute warnings seen on 32 bit compilers from recent changes
pseudo_intrinsics: pure SSE2 could not be built
oracle12: Fixes to make work on BE system
Some changes needed to get current MinGW64 to compile
testparas.pl: perform the testing longer.
testparas.pl: added seconds param, and extra information to final
table
Mingw64: strip needs to use the proper cross compiler strip
django-scrypt: if N too large, the kdf aborts, but we were not
trapping it. Fixes 1413. This probably should be in valid also
pbkdf2-hmac-sha1: stack overflow possibile: Fix #1411 and #1412
pbkdf2-hmac-md5: added proper valid() Fix #1388
configure: removed warning messages about Expect not found
scrypt lib code: validation checking. Fix #1385
ssh: Improved valid, checking whether we can get the salt properly or
not
vtp: buffer overwrite. Fix #1391
vtp: domain_name length check in valid. Also, fixed overwrite problem
where we had an array of pointers, and not an array of chars.
gpg: improvement to valid(). Fix #1417
gpg: Handle a default algorithm. Fix #1418
keepass: fixed valid(). Fix #1419
luks: improved valid. Fix #1386
base64_conv: handle binary or whole files with the utility. Fix #1423
crc32c: changed test vector to be proper for ripper12345 starting
from ripper
crc32: added crc32c to the format. Fix #1422
crc32c: improved performance of non-SSE4.2 code. crc32, reduced
complexity of crc32c setup code
base64_conv: mute warnings
crc32: Unified all CRC32 / CRC-32C function into crc32.[ch]. This DOES
change the core files (crc32.[ch]), but was given the go ahead to do
this. Fix #1379
pass_gen.pl: Fixed dynamic parsing that uses a negative salt length.
 Added dynamic_1017
pufferfish: added proper valid checks. Fix #1390
oldoffice: Improved valid to fix ASAN found buffer read overflow. Fix
#1410
office: improved valid. ASAN buffer read overflow. Fix #1409
wbb3: Segment fault. Fix #1434
opencl_pkbdf2_sha1: Same fix as done to core. Fix #1411 Fix #1412
opencl_gpg: Same fix as done to core format. Fix #1417
oldoffice-opencl: same fix as on core. Fix #1410
Add support for more NTP hash types. Partly reverts 4556eb4
django: buffer overflow. Fix #1408
clipperz: buffer overflow. #1407
MinGW64 warnings. MinGW64 can no longer handle %I64 modifier for long
long
MinGW64. %I64 reverted.
bitmap-perf-test: attempt to correct MinGW64 build on CirclCI
MinGW64 warnings. and modification to all %ll* to LL* and %zu to Zu
within printfs. #1439
opencl-rawmd4: fix warnings for sprintf parms #1441
MinGW64 warnings muted. #1439
--list=format-all-details, added more details to A $ format line
MinGW install doc. Missing package
Small debugging change to the build script to see Makefile
CircleCI debug take 2
CircleCI removed debugging for now
pbkdf2-hmac-md5: attempt to quiet CircleCI until we figure out problem
Missed another ctx reference
MD[45].[ch]: make the structures appear closer in binary layout to
oSSL
md4.c: fix is a global cut/replace screw up
configure.ac: fixed oSSL search for cross compile
CircleCI-MinGW.sh: no longer requires OPENSSL_LIBS after fixes to
configure
dynamic-thin: Enhanced so a temp dyna script can be PART of the thin
format file
dynamic_thin updates. A couple fixes and warning muting
thin-dynamic: made changes to allow native test hashes in test script
dynamic: added 2 new flags, for pre-compute in sha1/sha256
dynamic.conf: added missing comments, and set coldfusion salt-len to
64
pass_gen.pl: updated with 3 dynamic formats from dynamic.conf
configure.ac: OpenSSL issues between no-pkg-config and cross compile,
etc. #1451
NETLMv2: Buffer overflow. #1405 #1406
django: fixed core dump. Fix #1397
NETNTLM buffer overflow. Fix #1405 Fix #1406
uint_xx cleanup: Fix #1457
MIN/MAX code cleanup. Fix #1456
SKEY: proper valid. Fix #1396
salted_sha1: Improved valid. Fix #1395
pomelo: improved valid() Fix #1389
dynamic: #1453. Added all hash types to the crypt() and crypt()
optimizations
dynamic: some optimization of types md5(md5(....)...))
dynamic_1501 optimization
dynamic: Added true dynamic expression building format
updated copyright notifies in dynamic_compiler
pfx: fixed memory leak
pfx: fixed memory leak fully. Fix #1246
dynamic_compiler. Remove the @ chars from expression. added memdbg.
Added a testing main() function
gpg: mem leak. fixed #1245
gpg: missed a big-num memory leak
pass_gen.pl: updates to get_token function, to match what is being
done in the dynamic_compiler code
hmacMD5: fixed core. Fix #1472
dynamic_compiler. Some rename, and moved a function into
dynamic_compiler_plug.c. #1477
renamed dynamic_compiler_plug.c to dynamic_compiler.c. Fix #1477
dynamic_compiler: real working. Still much work to do, but many
hashes ARE fully working
dynamic_compiler: error, md4/md5 salt-as-hash were getting 40 byte
length for the salt
pbkdf2-hmac-sha1: buffer overflow in prepare. Fix #1478
dynamic_compiler: Handles more expressions, still more work to do
dynamic_compiler: added gost, ripemd, tiger and whirlpool types
dynamic_compiler: had to add MGF_INPUT_xx_BYTE for hashes not 16 byte
long
pass_gen: fixed a bug in gost processing for dynamics
dynamic_compiler: added unicode. initial gost tables properly. logic
for salt length
dynamic: unified $salt/$pass to be $s/$p
dynamic_compiler: added todo list, changed salt logic. Some pure
whitespace editing
pass_gen: removed all _64e hashes and replaced with _64c. _64c will
use crypt alphabet
-list=build-info: lists SIMD information
opencl and cuda wrapper defines not present
load_one_format (john.c). Do not lower case format name IF it is a
dynamic= hash. They are case sensative
pass_gen: fixed some errors in constant validation checking. Removed
all md5u() types. Now utf16() must be used instead. It gets same
results, but removes this ambiguity
dynamic_compiler: many fixes and enhancements
dyna_compiler: static added to many functions. Some md5u() stuff was
still left and removed
dyna_compiler: removed or commented out unused or not yet used static
functions. Fixed a couple bugs in ripemd256
dynamic: normalized unicode() -> utf16() for dynamic expression
strings
dynamic: fixed wrong comments. Fixed bug in dyna_85. Fix #1488
dyna-big-crypt: removed spurious comment (not implemented hash)
dynamic_compiler: Starting work to hook in the compiled dyna back into
existing dyna hashes
dyna-compiler. SALT_AS_HEX was not properly building test strings.
f6aef52
dyna-comp: got the padding functions working properly
dynamic: Fixed problem loading test strings (example was dyna-29) if
the array of test strings had any types not loaded, then all the
rest were not being used. Now we properly pack them into the array,
and use them
dyna-comp: fixed 2nd-salt not working
dynamic: bigcrypt-md4 was broken. Fixes #1479
dynamic: got uc() and lc() working properly in both the format and
compiler
dyna-preloads: incorrect comments fixed
dyna-big-crypt: Fixed issues in many types. Wrong length for
md5-overwrite for outputs other than hex. Key clean problems
 switching between BE/LE formats (or 32bit to 64 bit). Issues
mentioned in #1463 and #1491
dyna-compiler. Fixed mem leaks and SHA512 was not working (mentioned
in #1463)
dynamic-preloads: comments about flags used
dynamic: Fixed big problem caused by new self-describing thin dynamic
which is a LATE loading format. They get loaded and initialized
AFTER john has called init(), and the working set of data for the
dyna format was mangled. This patch keeps it. Also, looking at
buffer over flows for dyna-39 Asan problems (#1489 #1490)
dyna-compiler: we have only consts from 1 to 8, not 1 to 9 Always
output max key-length, so that we get 110 bytes. hexify constants
(convert : into \x3a)
dyna-compiler. 2 script output issues. Buffer overflow on
convert_old_to_new. #1490
ishex() and other hex stuff from common.c Changed to only work with
even number of characters. #1494
ishex() additional fixes. #1494
dynamic_big_crypt.c: this file has been REMOVED, and is now
autogenerated by ./configure. #1491
dyna-big-crypt: some cosmetic changes to last checkin
dynamic_big_crypt: Fixes build problems
dyna-big-crypt: still build problems on clang and cirleci
dyna-big-crypt: echo some debugging, to try to figure out why they can
not compile
dyna-big-crypt: hopefully this will fix circleCI build. I really do
not know why it can not build/run dynamic_big_crypt_chopper.c file
dyna-big-crypt: last change to try building early, was not building a
new dynamic_big_crypt.c at all
dyna-big-crypt: changes to the .c and .sh to hopefully at least get
Travis functional again
dynamic: added haval256_3 and haval128_4 hash types to dynamic
dynamic: haval added to compiler, and some other bugs fixed
dynamic: modifications to get dynamic-big-crypt.c building properly
even on cross compiled systems.
PHPS2_fmt_plug.c: chmod to 640
2 more source files with wrong mode (set to 640)
dyna-compiler: Now ALL internal formats that can work with
dyna-compiler, DO properly work
dynamic: LARGE change. Added all HAVAL hashes (15 varient) Converted
large hash type linkage code into #define macros (in all but 2 spots
in dynamic.h) Removed deprecated *_base16 flat hash functions
dynamic: some more define macro repetative code reduction for large
hashes
dynamic: haval fully integrated, and fully working. Found numerous
other areas where I could use macros to reduce source size using
macros.
dynamic_big_crypt.c: produce this file in Makefile #1491. Still work
to do. The chopper file will be ported to Perl
dyna-preloads.c quiet 2 unused var warnings. Added new format type:
-format=dynamic-all That type will test all dynamic, including any
disabled dynamic formats
dynamic_big_crypt_chopper: ported to perl. Changed make process some
dynamic-big-crypt-gen: changed how script is executed in makefile.
removed a comment left in from debugging development
dynamic_big_crypt.c generation, changed commenting
Dynamic: Added MD2, Panama and Skein hash algorithms
dynacomptest target added (builds the thin test engine for dynamic
compiler this tool is a decent parser test engine, and I also use it
 to create dynamic large hash test strings
dynamic_compiler: added lc($p) and uc($p) syntax
dynamic: added lc() and uc(). Better format label
dyna_compiler format: label string had last byte missing
dyna-compiler: space pad string pasting macros. Added lexical error
handling for mixing p, uc(p) and lc(p) (and u)
Compile fix for nt2/rawmd4 for 1032054 for any non SIMD build
dyna-compiler: Fixed upper case hex hashing in inner functions like
md5(MD5())'
some plugged hash files unplugged. Fix #1525
base64 length for valid usage. Returns -1*len if string not pure
base64. last change had bug in the check for pure string logic
dyna-compiler: added symple function exponentation such as
md5($s.$p)^4 which is md5(md5(md5(md5($s.$p))))
silence a couple warnings
dyna-compiler: set BENCHMARK_LENGTH properly
dynamic_compiler_fmt: was not showing proper label if expression ended
with exponent
pix/mdc2. Fixed benchmark length. Fix #1536
fixed salted dyna hashes listing no salt len. Fix #1535
dynamic salt length. Needed to be negative in this one
configure: target for dynacomptest needed CPPFLAGS
dynamic: bare hashes with but no additional salt work if user in
field 1
sse-intrinsics: recent changes in 9783278 totally broke my VC builds
md4.c: undid some changes in abe2cdb. #1526 references this also
dynamic: fix salt crash problem. Fix #1545
dyna-compiler: handle both and (and , , )
pufferfish: fixed bug in valid. Fix #1546
sapH: fixed bug where there were spaces in the hash. Fix #1547
base64_convert: bug in new return -len for non pure strings. Fix #1551
dynamic: salt buffer read of invalid data. Fix #1554
wow-srp: handle odd length hex and better validation. Fix #1553
openssl-enc: valid() problem. Fix #1550
BE issues (missing johnswap.h) and problems building with HAVE_SHA256
false, but using oSSL
Fixed crash on !ALLOW_UNALIGNED systems. Change required when dyna
flags went from 32 to 64 bit integers
dynamic problems on BE due to recent optimizations
dynamic a couple tweeks for big crypt building
removed link for pwsafe2john from legacy makefile
dynamic problems under BE systems with the set_input2_len32 function
dynamic: had to add set length_32 with clearing back into the command
list, crashes on SSE 64 bit (but not on my AVX2 64 bit??)
dynamic_compiler. magnum fixed $u handling on raw hashes. Fix #1541
dyna-compiler: change prepare to always call expr_normalize
dyna-compiler: rolled back 1c9f842
dyna-compiler: corrected prepare always normalizes hash
dyna-compiler: removed commentted out code
dyna-compiler: prepare adds dyna= signature to all valid raw hashes
dynamic: increased buffer count for MIC
dynamic_types.h: auto scale size of arrays based upon with of
SIMD_COEF
sha3: fixed Asan overflow in input string. Fix #1492. Whenever using
Keccak functions we need to round up to next even by 8 offset
dyna-compiler: some salt_as_hash types were missed
dynamic: big crypt 64 bit SIMD hashes had key cleaning problems. Fix
#1563
pass-gen: dynamic expression was not handling upper case hashes
 properly
dynacomptest removed from default target. It still can be made, but
must be explictly requested. (request from @magnum)
dyna-big-crypt: 64 bit SIMD cleaning for more cases
Added -format=dynamic-all to tests for the CI builds
dyna-compiler: Computing max length of input password was not taking
into account hash type, or length of sub hashes.
dyna-compiler: improved test shell. fixed a couple bugs turned up by
the test shell
dyna-compiler documentation. VERY thin, but it is a start
wrong file attributes
dynamic_big_crypt.c in unused was way out of date
dyna-compiler: Track length of both input buffers independantly, to
compute max input length allowed. Also, distinguish between 64bit
SIMD, 32bit SIMD and non-SIMD in length computations, because they
each have a different max length allowed
100% white space cleanup. no code change
dyna-compiler: added a library of pre-built scripts. It is sparse ATM,
but I will be able to add formats which the compiler does not
optimize well
dyna-compiler: added a ,nolib flag to the expression to force NOT
using the lib scripts
dyna-comp: added debug dumping of scripts read in from the compiler
lib
dyna-compiler-test.sh: switched to ksh. Ran on my cygwin before, but
when I tried on ubuntu, it was no go
dyna-compiler: Added some hashes to the optimized lib. Fixed a couple
bugs
dynamic: Added dup flags, Tiger and TIGER
dyna-preloads: Removed some depricated flags
dyna-parser: added TIGER flags, fixed bug in MD5 flags
dyna-compiler: additional optimizations. Speed same or better for
almost all dyna types
dynamic was not building properly unless building SIMD
dyna-compiler: Had some bugs for RIPEMD128 and the HAVAL hashes in
picking right key-hex and salt-hex length
dyna-compiler: pretty print the script in ,debug mode
fixed trailing whitespace problems. #1567
dyna-compiler: Fixed ASan error, when we walked backwards past our
buffer
simd build problems under VC
dyna-comp-test: changed from /bin/ksh to /bin/bash. Use local .pot
file so the logic will work properly
dyna-compiler: added O=# flag to control optimization
dynamic: renamed some flags. Fixed bug where pass to offset other than
32 was failing
dynamic: added a specific comment at each point where we need to edit
the dynamic files when adding a new digest type into the code
added a .editorconfig file (see editorconfig.org). This is just a
start. We may want to change this
.editorconfig. LOL, Saved before I updated Notepad++ with the plugin
and had no trailing lf
dyna-compiler: added first -O2 optimization (post script creation
pinhole optimizations) Currently only 1 optimization, but there will
be more (and the 1 is a pretty good one). Optimization is
MGF_KEYS_INPUT, and will only be done if the script allows it
adjusted file attributes
dyna-compiler: added O=3. This optimization level tries to build a
script in intermixed SIMD. It is not perfect yet, but does work
dyna-comp-test.sh had rm commented out for some debugging
dyna-compiler. Better mixed SIMD code. Parse more complex expressions
properly
dyna-big-hash: added 4 temp vars. had to adjust simd code also. This
is early WIP code. Right now, there is no way to use these vars,
just write to them.
crypt: failing to build right, and coring on legacy build. Fix #1471
Removed all FMT_MAIN_VERSION defines. Fix #1185
stray source got inserted in last update
pkzip/zip: switched internal code to use strtokm and ishexlc. Cleaned
up a bunch of crap because of using unsigned chars all over the
place. Fix #1123
more formats changing mem_calloc_tiny to calloc/free. #1110
more formats changing mem_calloc_tiny to calloc/free. #1110
more formats changing mem_calloc_tiny to calloc/free. Fix #1110
64 bit raw sha formats were not aligned properly for 32 bit builds
x86-sse.S: changes from last core merge which needed to be mirrored
in our jumbo specific code
raw-sha384: fix buffer overwrite. Fix #1577
depricate john.local.conf. added new include syntax and a new
john-local.conf file can be made, both in the $JOHN directory and in
the local directory. #1495
.gitignore updated with john-local.conf found anywhere in the tree
dynamic: MGF_SET_INP2LEN32 was not working since adding the @dynamic=
hash type. Fix #1579
more work deprecating out john.local.conf file. #1495
I had C on the mind, and used wrong include type for john.conf
mssql05: buffer overflow (ASan error) found with jtrts
putty: buffer overwrite. #1582
files would not compile in non-SIMD builds
bug in non-SIMD builds
non-SIMD compile failure
dyna-big-crypt: fix crypt_only code. Fix #1585
trip: would not build in non-SIMD mode" ;
removed last traces of kwallet2john.c
dynamic was failing for any MD5_X2==1 build, i.e. many non-SIMD
added non-SIMD build to Circle-CI. #1598
added non-SIMD build to Circle-CI. #1598 (try 2)
added non-SIMD build to Circle-CI. #1598 (try 3, likely to fail,
since sudo could not run)
added non-SIMD build to Circle-CI. Fix #1598
added non-SIMD build to Circle-CI.
circle-CI. Added 32 bit-any and 32 bit-sse2 build/test
CircleCI. I think the script was not properly exiting upon errors for
many cases
configure: if fmt_externs.h or fmt_registers.h fail, then bail and
reset makefile back to the stub. #1590
changed link for windows pre-build
dynamic: problems building on non LE systems
build on sparc failing. ar -s by itself was not valid. But -s can be
combined with other options (looking at gnu ar)
dynamic: commented out a slow hack, as it appears to no longer be
needed
make process had problems under Solaris
mask.c: Added ?W which is ?w with case switched for all characters.
#1612
mask: updated document to match new behavior
mask: ETA was 1/x the correct time in -fork=x mode
dynamic: base64 input was not properly validating in all cases. Fix
 #1625
dynamic: fix problem where salted dyna format requires saltlength to
be set by the designer of the script, but it is not set. Fixes #1611
pass_gen: output error message if a NON supported or mistyped hash
name is tried. Fixes #1639
pass_gen: if testing just 1 format, do not output the ^D message.
config: Could not .include [section] which did not have a colon char,
such as [Options]
HAVE_BIO_NEW: disable some items if that function is not available:
#616
Fixed busted Makefile.in from last commit
Makefile.in: extra space left when moving PROJ stuff out of main
asignment
pseudo_intrinsic.h: failed to build with MSVC after 3884f8f
recovery.c: created config var to protect restore files. Default is
off. Fix #1635
external: The error file was using the global file name, not the file
name for the external object. Fix #1650
dynamic_compiler: bug, first character of constants was not being
picked up
dyna_compiler: fixed core if an empty constant value provided in the
expression
dyna_compiler: Building test strings before validating the hash fits
in our SIMD buffers could cause a crash
loader.c: ldr_in_pot var useful for more than just builds with
HAVE_CRYPT
base64_convert.c: some stack overwrite conditions fixed
has-160: rename the fmt_main structure so it is found after raw-sha1
Merge of raw-sha1 and nsldap, and start of common raw-sha1 code for
GPU
non-thin phpass-md5. This needs LOTS of work, but will compile/run
phass-MD5 thick: Rewrote. Now with SIMD and OMP. #1672
phpass: scrubbed thin format stuff. Will rename fat file when I know
things ae happy.
phpass-MD5: made format label, and output string compatible with older
thin version
dmg2john.c: Removed headers (ossl/evp and hmac, and gladman). These
were not needed, code not in this file
cq: busted cmp_all
added new file, just a header, to easily do hmac-sha1. For removing
gladman code. #1686
hmac_sha1.h: added digest length, so that a shorter than 20 byte
buffer can be used
all gladman code REMOVED. The only thing it was there for was
hmac-sha1. We now have that function of our own making. #1686
Opps, missed this for gladman code removal. It was done just was not
in last git commit
failures on travis? Does not have problems in cygwin, but hopefully
this fixes the problem
hmac_sha1: new code was failing on 32 bit builds
bitcoin: was not building on my sparc box
luks2john: removed oSSL BIO_base64 junk. #1686
Luk2john was not building correctly
Legacy build: luks2john
hmac_sha1: port for VC
openbsd-softraid: removed evp_hmac_sha1, replaced with hmac_sha1
changed name of hmac_sha1.[ch] to hmac_sha.[ch] to make it more
generic and so I can add more sha* hash types
2 files missed in header name change
fixed some warnings in certain builds. Also added wpapsk-opencl to
self_test_full_body function check
hmac_sha: added sha256 and sha512 hmac
encfs: removed oSSL hmac-sha1, replaced with JtR code. #1686 still
have AES to work on to complete
rawSHA1-ng: added new common code, including {SHA} handling
7z, recent changes needed fixed for VC to compile
raw-sha1-linkedin: updated to use common interface and {SHA} signature
into .pot file. #1613
formats: added some commented out code to help find out why a source
would be different
salted-sha1: created common code for use in CPU/GPU code to keep
validation code the same
attributes
dmg: replaced oSSL hmac-sha1 with our own code
.pot: allow multiple .pot files to remove already cracked items. Fix
#1623. Also document john-local.conf and removed references to
john.local.conf
encfs_common: Obviously chainedIV is not being tested. We need to add
a test vector for that type
encfs: right or wrong, this sets the logic back to original. The h[]
array is not preloaded in the ChainedIV now. Not sure if that is
right or wrong way
new single mode rules found
hmac_sha: if a system did not have SHA256/512 in openCL it would fail
to build
single rules: improvements to the recent group of new rules
luks: fixed non-inline processing. Was reading 0 bytes from file. Fix
#1696
single: added configuration for guessed_keys->count extra work.
Default is to do this work (like core). but we can now override it.
#1706
single: added logging of two configuration set items
.pot extra load. Ported to mingw and msvc
MSVC, .pot reload. Changed to a do{}while
agilekeychain-opencl: added test vectors: #1637
added test vectors to bbes10, luks and sha3-512. #1637
pass_gen: simplified the large hash dynamic macros. Will be adding
more soon so wanted it simpler
dynamic: added 4 SHA3 hashes and 2 Keccak hashes
dynamic: Moved disabled dynamic formats over to their own conf file
dyna_compiler: Bug in keccak-512 outer function
pass_gen: added sha3 and keccak dynamic formats, and added these
hashes to the expression parser
pass_gen: added MD2 to the dynamic hash types
dynamic: lowercase for -list=subformats. #1717
created error_msg() function. #1714
fixed usage of pexit to error_msg. Fix #1714
missed file in last commit (pexit to error_msg change)
dyna-compiler: invalid parsing. using *_b64 in function search. Should
have been *_64
dynamic: fixed issues found with fuzzer, base-64 encoding and re-crypt
problems
pre-built dyna-big-crypt.c: base64-recrypt and keccak fixes
dynamic: normalized case of all hash functions. Fix #1717
pass-gen: normalized case of hash names of unused dynamic expression
hashes
mem_align. Fixed globally, and no performance hit, Uses proper define
macro
formspring: force valid to honor the 2 byte salt.
several formats needed format flag bits set
added FMT_SPLIT_UNIFY_CASE
format.c. Calls to free() changed to MEM_FREE(). VC was crashing,
since all allocs are done using the _aligned_malloc, thus the native
free() can not be used
formats: make fmt_case_unify tests happen against every test case
removed deprecated code. This was only there for VC, and now we handle
this differently in memory.[ch] for VC
PHPS/PHPS2: fixed to properly unify case in split
formats.c: Added additional logic in the unify-case function which
will be required to properly handle compiled dynamics
formats.c: eliminate false positive failure to unify case happening in
mysql
dyna-compiler: for now, disable FMT_SPLIT_CASE. In the end, this
format may need to be setup to handle split_case, but until I figure
it out, this allows it to pass -test-full
nuked clan (nk) input hash case bug (#1472)
fixed split-case bugs. Some for #1742
dynamic_big_crypt: better handle larger SIMD alignment issues
fixed split-case bugs. Some for #1742
split case bugs. #1742. added upcase hex to common
revert 56d5b15
common.c: Added some common defines for types. Added a length variant
to the ishex*() functions. Added u and l variants to hexlen
IPB2, fixed bug. We can not treat itoa16 as a string. Fixes #1766
Start using the common defines for HEXCHARS, base-64, etc in common.h
dynamic: removed deprecated, unused internal function. Fix #1768
recent calloc change (9e100b4) boned VC builds
fixed new common hexlen function. DMD5 format, properly ignore case.
#1742
dynamic: sph_ripemd spelling error
krb: adjusted for case/split issues
added a --show=invalid command. #1773
some tweaks to --show=invalid and dox.
dynamic_IS_VALID now preloads on first call. subsquent calls are
instantaneous
dynamic loading. Improved the isValid function to FULLY cache, and
only do the real work 1 time
enabled display of count of valid and invalid hashes for a
--show:invalid usage
several formats: corrected many split/case issues. #1742
numerous formats. Fix salt-case bugs. #1742
rawsha512-ng. New problems on my VC builds. THis code moved into
psuedo-intrinsics, but was left in this file also.
configure: changed != xno to = xyes for some display results (crypt,
mmap)
many formats. Fixes for split-case bug. #1742
many formats: fixed salt-case bug. #1742
enhanced check for FMT_SPLIT_CASE problems. There still are formats
left to be found but this patch finds a large class that can be
found automatically
split-case bug. Last push had issues in opencl-o5logon format
simd-intrinsics: removed non-portable debugging crap
fixed numerous compiler warnings from recent split-case improvments.
#1785
memory leaks fixed in new -test-full code (dyna salts)
mssql-* fixed split/case bugs. These changes must have been missed
with committing all the files last week
split-case bugs: The last of the known bugs are fixed. Also 2 new
methods added for finding these, but they still need more work to
get right
travis-CI and CircleCI. Changed from -test to -test-full. Not 100%
sure this was done right, we will see
-test-full fixed memory leak in salt for a test
some -test-full missed on circle. Add a john-local.conf to have no
formats disabled
some changes in handling of Disabled:Formats, to make easier better CI
testing
problems with travis.yml, not sure what the problem is, possibly echo
can not be used ???
jumbo.h: we had a VERY busted header wrapper. OMG, failure to set
the define, lol. I think this was my F-up
add jumbo magic define, so we KNOW when code is being built jumbo or
not. This allows better merging of john master code, whichc we do
not control
merged -test and -test-full into 1 function. Removed flag for
-test-full. It was not needed. -test-full is now a required numeric
param that tells the level to test. formats.c reworked, to make a
simpler merge from master branch, and it documents differences
between master branch and bleeding. #1792 and #1795
jumbo.h. Recent proper header wrapping was causing problems with VC
builds
4th split-case bug test, full generic. But has some bugs and
whitelisting to do. Only runs at -test-full=1 level
rar (fixed split-case bug). bcrypt, white-listed split-case check 4,
#1742
EPI, chap, openssl-enc. split-case problems. #1742
wbb3, putty, lotus5-opencl. split-case problems. #1742
revert changes from 399217f
last of the split-case bugs. Fix #1742
split-case test-4 is now to be run at level 0
fixes and code clean up for inline vs _inline for _MSC_VER builds
formats.c: memory leak in -test-full mode, found with memdbg
foramts.c: fixed asan read problem, and memory leak for -test-full
mode
keystore. remove memdbg warning by switching to tiny_memory
allocation
asn1 changed mode to 644
pem: remove salt clean warning.
asn1_plug: include required for VC build
PEM: core crash found by fake CI on my sparc system
rawSHA1. Fixed BE problems. #1861
nt, raw-md5, raw-md4: Fixed BE problems. #1861
nt: BE problem. #1861
Keccak: BE problems: #1861
dyna-big-crypt: put SIMD wrapper around a couple functions. #1874
Azure AD added to pass_gen.pl
AzureAD format added. (CPU only)
AzureAD execute permissions
Azure AD: removed FMT_OMP_BAD. this one is not bad
luks: BE issue. #1861
pass_gen.pl: start to normalize the -loops param
rollback changes from da7d26a
pass_gen.pl normalized adjustable loop counts for all formats, using
-loop= value
pass_gen.pl: -loops= default for django was wrong. Changed it to 10k
pass_gen. Help screen for dynamic was not always being displayed
dyna-compiler. Auto-Escape constants with colon char in them
dyna-compiler. Handle ) char in constant strings
dyna-compiler. Allow @ char in constants
bcrypt performancce improvement on AVX/AVX2 systems. Fix #1199
john.c: correctly handle * char for dynamic= format: Code by
@magnumripper. #1747
add config var to disable runtime-CPUID checks
revert john.c changes to CPU_detect code location
reverted other CPUID-ignore code. Left as commented out
revert ignore-CPUID logic
asa-md5: handle salts from 1 to 4 bytes, not fixed at only 4. Fix
#1887
john.conf: removed left over traces of the IgnoreCPUID stuff
dynamic-self describing documetation update
hmac-SHA256: New larger salt logic was busted for salts of varied
sizes
hmac-SHA256: changed the SIMD salt structure
added onlyhex salt type for dynamic. This will use salt dictionary
with only low case hex
dynamic_generic_expressions: Got working with re-gen salts logic.
#1463
makefile.in: memdbg.o this one should not have been added
hmac-SHA256: Added JWT hashes to format. #1904
hmac-SHA256: a little more safety in the JWT convert logic.
hmac-SHA256: still more validation checking in split.
rawsha1-linkedin: binary function not declared static
vdi added to pass_gen.pl (still have 128 bit to do)
aes-xts: moved to its own file, to make it easier to use by multiple
formats
wrong create bits
VirtualBox .VDI encrypted disk format
VDI: added aes-128-xts code and vectors
VDI: compile problems on non SIMD systems.
vdi2john: new conversion program for Virtualbox encrypted disks
made AES-XTS into more generic function. Now there are params for
128/256 bit AES, and a param to do decryption vs encryption
VDI: Ported properly to BE systems.
base64: hex2raw conversion was null terminating???. Caused buffer
overflows. Raw data should never had this done to it.
verbose message removed \n char. Fixes #1958
dynamic (many formats), fixed problems with OMP runs
some porting issues for MSVC
dynamic: Fixed threading problem.
dyna: fixed other OMP failures
revert a7af331
fix build problem for CircleCI (mingw in general) after 0e235c7
last patch would not compile. malloc vs mem_all0c_tiny use different
params :(
jumbo.c: removed memory.o dependancies
Many formats. Removed undefined shifting of signed ints. #1957
postgres: failure to load with user names of variable lengths. Problem
found with Test Suite
postgres. Longer user name added to self test vectors. Without the
salt cleaning, the ST will fail
john.c: avoid loading test db in -skip-self-test mode. Fix #1967
bt.c: switched setitimer to getitimer. Fix #1966
pass_gen.pl. Added QNX hashes
qnx: cpu, non-SIMD format. #1970
change mode bits
qnx: added fmt_split_case, founnd by the CI
qnx: moved split to common code
qnx: removed split, and valid() fail hashes with upper case
qnx. Bug in valid, and added max length sha512 hash that works
currently
sha512: added QNX hashing bug to generic sha512 source
Fixed numerous warnings on my sparc system. Possible uninitialized,
missing headers, etc
qnx: improved speed of sha512 (for 128%pwlen==0 passwords). Also fixed
memory leaks in get_salt/valid
base64_convert: major overhaul. Now buffers adhere to max size
requirements, INCLUDING null padding. NO extra bytes needed
opencl_rawmd5: ASAN overflow fix. Fixes #1979
pass_gen.pl Added special code to get SHA512-qnx hashes to work. The
pod is in unused, and the code is currently commented out within
pass_gen.pl
Found out a local perl mod does not need execute bit. Thought it did
pass_gen.pl: fixed bug in get_salt() where a charset would not be used
unless there were 3 args. Changed qnx hashes to use get_salt with
hex-low
.editorconfig ending \n missing from the new .pm file
axcrypt: fixed asan overread bug. Fix #1987
dynamic: added McAfee master unlock password
dynanmic-expressions. Handle NULL bytes in constants. Fix #1985
fuzz: cygwin port problems fixed
fuzz: remove files upon successful fuzzing test. #1969
bench: make -test=0 deterministic in the number of times things are
run
bench: commented out debugging code was not right
fuzz: rolled back file removal code. That change was likely very wrong
thing to do
legacy generic build fixed for usage with new test_db logic
config: emit warning when a List.*** section is replaced with another.
#1988
bench.c: generic build. New ldr_* functions could not be static
stribog: #warning is not portable
config.c: warn for multiple List.* sections if -verb is > 3
gost3411-2012-sse41.h: use a john header, so systems without
inttypes.h work
stribog: warn for __GNUC__ and _MSC_VER
hmac-sha384 and 512. Salt not aligned properly. Fix #2000
dynamic-compiler.c: ASAN error. #2004
dynamic-compiler.c: Possible buffer overflow cases. #2004
qnx-sha512. avoid div by 0
raw-sha1-axcrypt: handle raw 32 byte hex hashes
gpu_sensors.h: made adjusments to compile under cygwin (win32)
hmac-SHA*: Salt alignment was wrong
formats: ST failures. provide additional help when -verb>3 is used
(failure hash, etc)
formats: bugs in new printf strings for debugging
QNX-sha512. Figured out some issues, and now all hashes should match
the buggy QNX implementation
sha2, remove execute bit
formats: better self test failure verbage under higher verbosity
settings
pass_gen: added SXC hash creation
raw-sha: common code was WAY too promiscuous in prepare/split
converting any 40 byte line into an SHA1 hash
pass_gen.pl: added VNC protocol
vnc: moved key loading into get_key. Use ECB instead of CBC mode.
About 20% improvement in speed
vtp: improved multi-salt speed tremendously
vtp: attempt to get OMP working on the CIs
VTP: Fixed OMP bug also improved speed some by not double buffer
copying
pass_gen: added VTP hash
cloudkeychain. Fixed issues found with jtrts
pass_gen: added cloudkeychain
pbkdf2_hmac_sha256.h: muted compiler warning
AzureAD: moved MD4 code into OMP section of format
wowsrp: cache first SHA1 for multi-salts
wowsrp: rolled back 3a94817. It was busted even though ST passed
hsrp: doubled speed (or slightly better)
racf: simplify encryption. Speeds it up a bit
racf: improved multi-salt speeds by caching schedule setup. #2023
SybaseASE: improved multi-salt speed about 3.5x. #2023
wbb3: improved multi-salt speed some (about 20%)
dynamic: large update. Added ability to crypt to an offset in the
buffer. Fixed several bugs
mask: problems expanding ?d inside groups. #2029
mask: replacement skipping in groups had to handle escaped [ and ]
mask: handle non set custom placeholders
mask: abort on undefined placeholders, and abort on empty groups.
#2034
mask: proper abort logic
unicode: expose utf8_to_utf16_be to the world
keystore: cache password work. Setting up for SIMD conversion
keystore: added SIMD logic
keystore: fixed ASAN error, and hopefully ST failures on CIs
keystore: fix for overwrite, hopefully fixes CIs
keystore: disable SIMD until I can figure this out
formats: change to sprintf did not match params. Warning came from
circle-CI build only?
AxCrypt-Auditing-HOWTO.md: removed UC2 and DOS eol problems. Fix #2041
configure: enable a couple of compiler warnings which were not being
enabled
keystore: removed warning, changed cmp_all logic to do the real work
for SIMD runs
remove clang warnings from john.c and cracker.c. Created new itoa
functions
pass_gen: added keystore (not 100% sure it is correct, but I think so)
hopefully fix errors on MinGW build for CircleCI
krb5tgs: mute compiler warning
pass_gen.pl: made a tweak on keystore, to make it mimic ITW hashes
better
keystore: got SIMD code corrected. buffer overwrites
keystore: hopefully working properly with PARA_SHA1 > 1
keystore. adjusted copywrite
ax-crypt: fixed ASAN overread. Fix #2042
pass_gen.pl: still a nit in keystore
editorconfig: set utf-8 to be default encoding for all edited files
gitignore. Added *.tmp to files to be ignored
keystore-opencl: Fixed bug for binary passwords. Improved performance
about 2.5x. Created common code to share with CPU version
bug in bench.c. When testing multi-salts, the salts were NOT being
allocated with approved alignment
bench.c: last checking had problems. Now simply using mem_alloc_tiny
remove -x bit from file
bench: reusing mem_alloc_aligned. Changed memory.c to handle
alignments < sizeof(void*)
ssh: mem leak. #1247
dyna_salt: The init function now returns the format that was linked
in, so we could undo if we need to
ssh: removed problem with dyna_salt leaking. Changed loader.c to call
dyna_salt_init at an earlier time (prior to calling valid)
memdbg.c: added code (commented out) in checkShapShot_possible_exit()
so that all memory in the list, even freed is shown
zip: emit 1 time warning when older hash signature seen. Fix #2061
logging: added optional date signature to start of log lines (default
is NO date signature)
memdbg: would not build under Makefile.legacy
logger: give option to log datestamp in UTC
hmacMD5/SHA256. Found bug in computation of bit count. Still does not
fix all problems
hmac-MD5/SHA256. Same bug as prior, but still does not fix things
hmac-md5: Possibly fixed.
hmac-SHA256: same corrects as provided by hmac-md5
pass_gen: hmac-md5 and hmac-sha256 now generate random length < 184
byte salts
hmac-md5 and sha256: normalized buffer defines based around PAD_SIZE
(and PAD_SIZE_W for when used as offset into ARCHWORD_32 arrays)
pass_gen: made long salts for hmac-sha224, 384 and 512
hmac_*: got long salts all working properly. Changed code so each
format is much more similar to all others. We may merge files at
some time
hmac-SHA224/256: Merged into a single format file
hmacsha224 file delete was missed
hmac-SHA384/512: merged into a single file. Some small changes to
hmac-SHA256 also
hmac-sha384: bug in non-SIMD logic.
logger: place fork# after date. Also added date logging when using
--log-stderr command line switch
memory.c: Failure allocating aligned 0 byte. Showed up on sparc
old_office: fixed memory leaks
pbkdf2_hmac_md5 and md4: These were not working on BE system. Now
they do
removed BE swapping from pbkdf2 code. Greatly simplified BE stuff
last BE change missed a few files
dyna-compiler: fixed core when using constant data under certain
situations. Fix #2067
zip: was not processing zip files with external linked files. Fixes
#2063
opencl-zip: same bug as was in the CPU version
XSHA512: stop strcpy of data past end of buffer. #2065
memory.c: return null on 0 size, and added some asserts
memory.c: cosmetic changes
john.c: In cygwin, when using -fork, the tty is screwed upon exit (no
echo). This simply fixes it for cygwin builds
cygwin: proper fix for screwed TTY after -fork mode run
SHA512 formats (cpu/opencl/cuda). Use more of the common code in
rawSHA512_common_plug.c
zip/openclzip: created common code for tests, valid and binary
phpass. Made common code for CPU, Cuda and OpenCL. #978
phpass: common code failed on non-SIMD systems
pass_gen.pl: added pbkdf2-hmac-md[45] hashes
pbkdf2-hmac-md4: common code for CPU and OpenCL. Fixed bug in opencl
and increased max salt length to 115 bytes for both CPU and GPU code
exe bits set, and cleared
opencl-pbkdf2-md4 kernel simplify code a bit
pbkdf2-hmac-md5: common code for CPU and OpenCL. #978. Saltlength set
to 115 bytes
pass_gen: increased max salt length possible for other pbkdf2-hmac
formats
editorconfig: added .cl to normal handled text file group
pbkdf2-hmac-md[45] opencl kernels. Reworked code to avoid 2nd limb
array. When adding long salt to sha1, I was not able to get it
working until I removed that 2nd array
sha1-crypt: added 115 byte max salt logic
pbkdf2-hmac-sha1: made common code for CPU/GPU. Extended salt length
to 115 bytes
pass_gen: pbkdf2_hmac_sha256 was not putting out longer length salts
properly
pbkdf2-hmac-sha256: created common code between CPU/GPU #978. Extended
salt max length to 115
sha1crypt: reverted max salt length to 64 bytes. pbkdf1-hmac-sha1
still allows 115 byte salt, this format simply validates 64 byte max
salt length
some tweaks missed on common code for pbkdf2-sha1 and sha256
pbkdf2-sha512: common code added for CPU/GPU formats. #978
mscash/mscash2. Common code changes, cuda/openCL/cpu. #978
gpg: common code created for GPU/CPU formats. A LOT of common code.
Still probably more to do
recent common code changes. Missed memdbg.h include in some files
gpg: improved performance on multi-hash for sha1/ripemd160 by
properly aligning data in 64 byte blocks. There was about a 10-20%
improvment due to stopping of buffer copying. However, this change
does not appear on the -test=x line, since none of 2 test hashes use
multi-hash length. For sha256 and sha512, it should never end up in
multi-hash mode, unless larger keys are needed
gpg/gpg-opencl. Improved sharing. Added all formats to openCL format
(reverting to CPU code). Other improvements and bug fixes
SHA512 common code. Setting wrong alignment in binary. Only affected
32 bit builds
gpg-opencl: issue warning when running and there are hashes which
would use CPU code. Added some comments
opencl SHA1 CTX model. Code moved from gpg kernel code, into a header
file so it can be used by other kernels
wrong exec bit set
gpg-opencl. Kernel code now handles > 16 byte keys. Also updated CPU
code to better match GPU code for this function
added a .git pre-commit hook file. Everyone could use this local hook,
but it has to be linked from JtR root to .git/hooks/pre-commit
before it will do anything
pre-commit hook. Edit was missed
pre-commit hook. Added additional txt file checks
pre-commit. Was getting exit 1 on certain conditions
cleaned up a couple of file problems, mode and DOS lines
pbkdf2-hmac-sha2.h Missed adding git commit -m "pbkdf2-hmac-sha2.h
Missed adding defined(OPENCL_FORMAT) in the opencl-pbkdf2-common.c
file. This was triggering warning about static code not being used.
Fixes #2070
pre-commit. Slight changes again.
pass_gen. removed -vector option, added -outformat=s option. options
can be normal, raw, user, vector. Since we have a common output
function, this was easygit add ../../run/pass_gen.pl !
pass_gen. updated usage screen (names now format to width of console).
 If -vector mode, then there must be only ONE hash listed
cuda builds: include johnswap.h. Fix #2076
typo in cuda_cryptmd5_fmt_plug.c
v1 changes for external-hybrid mode
recovery. File locking problems in cygwin worked around. #2068
external:hybrid documentation
fixed problems in external_hybrid_example script
external.c: fixed a bug in hybrid external mode. Now working
some edits to doc
external-hybrid. Changed some variables, and fixed bug in fix_state
for hybrid runs
restore testing format added.
restore-tester, got OMP working for OMP_NUM_THREADS > 8
restore-tester format. A fake format that can be used to help fix
issues in restore, or pause, making sure john misses no candidates,
and does not rerun words
hybrid-external. Handle bug found by magnum, where a filter failure
was also aborting the entire crack loop
recovery. Hack fix for cygwin, now done for all runs-over-win32
systems
external-hybrid. Got restore working 100% correctly
external-hybrid. Added logic (I hope) for utf32 processing
external-hybrid. Changes for utf32 were not right
restore_tester format. Proper porting to VC. 20ms delay for each
candidate check
restore_tester format. Proper porting to VC. 20ms delay for each
candidate check
removed this file from the fork
external-hybrid. Corrected a couple issues pointed out in #2027
wordlist: Fixed restore problem if wordlist was fully preloaded in
memory. #2082
wordlist: restore. Fixed issues between restoring mem_map and
non-mem_map builds. #2082
unused/restore_tester format. Added multi-salt format
salted hashes, proper restore for the salt, #2082
Revert "salted hashes, proper restore for the salt, #2082"
multi-salt save-restore logic. Code added to validate salts are
exactly the same upon recover as they were when the .rec file was
saved
deterministic salt order within salt sorts
dynamic salt sorting. Wrong logic before
restore_tester format: added 3rd type, salted-hashed. Also casecmp in
valid, so that multiple hashes can have same salt
salt-resume logic, using deterministic sort order salts and unique
hash salt detection
dynamic md5 salt generation was using wrong data
loader: removed comment about experimental sorting, since now we
always sort
restore-tester format: moved to TestSuite project
file added to gitignore, and .pre-commit.sh hook was incorrectly
flagging a deleted file as not having an EOF
loader. Removed sort by salt count, so that salt sorting is
deterministic
base64_convert. Problems in base64 to base64 convert if last block was
not full length. Fixes #2088
pass_gen.pl: added racf
rar5-opencl. Recent common code changes broke this format. #2093
pbkdf2-hmac-sha1: small issues not properly common code converted.
common-opencl: split() was not being called during auto-tune running.
 Caused common code pbkdf2-sha1 to core
memdbg: removed tiny logic from realloc. Removed some strange code
from header
memdbg: reworked MEMDBG_realloc. Fixed #2062
wpapsk: sort salts deterministically
oldoffice: assure full salt is used in salt-sorting
oldoffice: removed redundant calls to dyna_salt_init within
salt_compare
korelogic.conf: updated Rules:ReplaceSpecial2Special go handle more
characters. Fixed some bugs. Then replaced entire rule list with 2
rules
korelogic.conf: merged more, using the rules PP
external: restore problems. #2099
external-hybrid restore: The ext-v1 record did not have proper \n
ending the record
recovery.c: changes writing .rec file for cygwin/mingw/msvc. May
relate to #1949
recovery.c. Improper mingw define used
common.h / common-opencl.h Moved the get_power_of_two macro into
common.h. Fix #2091
command line rules. Fix #2105
elite external-hybrid mode added to john.conf
updated extern:leet script.
external:case mangler (hybrid external) added to john.conf
extern-leet: Removed some conversions which were pretty rare in real
world test cracking
external-hybrid restore. Always chomp the line, not just on success
external-hybrid resume was not right
cracker.c: multi-salt restore records were often being written for
non-salted hashes
rexgen: case alphabets. Renamed to cased, so it works, now that rexgen
has removed the case option internally
single: fixed memory leak if externs are used
regex: port issues for VC
leet external-hybrid. Some additional max candidate checks
john.conf: external-leet, more adjustment to total work
dynamic: improved performance for single mode work. Does use more
memory, but its at least 2x faster.
mask.c: vc and mingw can not handle %hhu
Changes required to incorporate PasswordPro/HashCat rules
rules: rules_count was not counting rules properly, nor doing dupe
removal in HC mode
Rules Hashcat. Added documentation
exe bit removal
dynamic: allow valid() to not fail on salts with embedded null bytes
(which can not have the $HEX$ removed from the salt string)
hash-cat-rules. Add some warnings about the git add
doc/RULES-HashCat HashCat ON logic
symlink: remove warnings about undefined execv function
new hashcat rules, changed HashCat into hashcat (text strings users
will see)
external. Allow enumeration of external hybrid words
config: recent change leaving trailing space on read lines impacted
parameters. They were not being found properly
rules: Changed HashCat into hashcat.
rules: hashcat logic. Proper handling of code in rules_reject. Only
skips the - reject code, and does all other
Rules: Added ONM which omits a range of characters
rules: added eC and e?C which are extended title case rule
make debug was failing to link (missing assert)
rule x rule stacking (perl script). Also john config processing in
perl module. #2122
jtrconf.pm: removed experimental warning, by dereferencing the arrays
jtr_rulez.pm: Moved this common module here, was in test. I plan on
using this for other john utils, so wanted it in a general location
jtrconf.pm: needed hard copy, or we were pulling back the reference
later
perl config code, better default basepath
rulestack perl script. Improves finding the other modules
RULES-hashcat. pure white space change
sha2.h: updated for older oSSL, so that include of just sha2.h was
enough. gpg_common and other code just includes that header
added HC p command. Added Onm command Fixed numerous bugs in new HC
type rules. Fixed E and added 'e' command. Handle spaces properly
interspersed in rules as valid character data.
unicode.c Would not compile if UNICODE_NO_OPTIONS was used
rules.c: removed undocumented McXyz handling in c and C rule
operations
rules.c McXyy change for c was not correctly done
config: in HC mode, we were NOT processing [ leading rules. We were
silently ignoring them but not using them
dynamic big crypt: Wrong years in copywrite notice
POC for generic code to handle VERY LARGE non-hash types
keepass was crashing after we set LINE_BUFFER_SIZE to small value.
This is a hack fix, but gets things working for now
super long hashes now truncated in the .pot file
korelogic 1 line leet line had some issues
gost: sse4.1. changed code so it builds in 32 bit mode with or
without -O2
potcheck.pl. Right now, just a skeleton script. #2161
potcheck.pl: -validate and -longline_fix should be working now
loader: increased file line buffer, to handle max sized value, AND the
password
potcheck.pl: line buffer should have been 1k, not 4. Also, lines are
trimmed to 256 bytes, not max size, BUT max size is used for
-validate switch
potcheck.pl. Constant changes does at request of Magnum. This should
now be done for -long and -validate switches
loader.c: revert commit 7856266
base64 (and common) changed int into size_t where needed
configure: added potcheck.pl logic to configure script. It will abort
if there are any LONG lines in ../run/john.pot
base64_convert.[ch] More work on size_t. commmon.[ch] reverted
base64: changed int into size_t
fuzz.c: returned buffer size to 0x30000. Fix #2166
fuzz.c: permissions
rar2john. Fix for 31-32 bit size, and FIXME comment about getting 64
bit length buffers
rar2john: Handle > 32 bit lengths. Fix #2170
added .log message directive code into rules scripts. #2140
common.ch: removed unused base64 functions. From comment in #2132
added .log processing to prince and single modes
radius2john better handling of very long salts
radius2john.pl: long salt was WRONG dynamic type
radius large salt documented
base64_convert.c: pure white space changes.
dynamic.conf: added mongodb, salted, MiTM
pkcs12: added SIMD logic. #2176
pfx: added SIMD sha256. improved validation. Added hmac-algo to
formats cost
pfx-ng. Hopefully fixed problems with SIMD crashes for SHA256
pfx-ng. disable SIMD for SHA256 until the problem is found
palshop: improved performance
pfx-ng: added SIMD for sha512 and corrected SIMD for sha256
scam: added SIMD code. Fix #2186
bks: non-simd was failing
rar2john: size_t can not be used for 32 bit builds. Must be a 64 bit
value
scram: adjusted header include order, so that we can turn off SIMD for
speed testing
pkcs12: would not build properly on non-SIMD systems (sparc)
XSHA512: was still returning binary align warning for sparc-32. Now
no warning returned
rawSHA512_common: quiet silly compiler warning
o5logon: fixed crash on requires alignment systems
phpass: fixed core on requires align CPU
pbkdf2-hmac-sha1. Fixed failure on BE systems
gost.h: some builds were having macro redefine problems for
bswap_32/64 macros. Now gost.h uses johnswap.h macros instead of
trying to recreate them
.pot hash trim. Added $ for any format with binary_size>0. #2188
pfx_ng can ONLY handle 30 byte passwords, at this time
pfxng. Some debugging code slipped in on last commit
krb5tgs: fixed already cracked hash removal logic. Fix #2188
hmac-sha: added hmac-sha224 and hmac-sha384 to the easy jtr interface
pfx-ng: added non-simd sha224 and sha384
pfx-ng: bumped PLAINTEXT_LENGTH to 48, which seems to be the max that
openssl can generate
SSHA512: Fixed ASAN buffer overflow bug
zipmonster format: added SIMD logic. Could still be improved some
zipmonster: fixed crash in some SIMD builds
radius changes in dynamic conf, due to very large salt changes
zipmonster: 25% improvement in speed for SIMD code
zipmonster: another 25% improvement in speed for SIMD code
zipmonster: small improvements in ossl, a bit in SIMD, and removed
original commented out SIMD hex-upper code
zipmonster: some changes when I was trying to track down the SIMD
core problem was left in. Now removed
testing code was inadvertantly pushed
chopped hash .pot support added to certain questionable formats. #2117
.pot chopped file valid() processing moved into common code in
loader.c
additional work normalizing FORMAT_TAG type defines
Additional normalization of FORMAT_TAG defines. #2196
Additional normalization of FORMAT_TAG defines. #2196
Additional normalization of FORMAT_TAG defines. #2196
Additional normalization of FORMAT_TAG defines. #2196
Bugs in last normalization push. #2196
Additional normalization of FORMAT_TAG defines. #2196
Additional normalization of FORMAT_TAG defines. #2196 (all completed)
loader.c: auto-pot chop logic. One condition was failing. Needed
shorter length
regex 1.4 integration, and full restore logic
rar2john: improved performance quite a bit for large rar files
line endings
common: change hexlen*() functions to all be size_t
rpp: reset refs, so that rpp can be run more than 1 time (needed for
 rulez-toolz projects
improved XSHA512. Part of changes made tracking down core on sparc
systems
cisco-asa: added a short salt
wrong format tag length used
atoll for systems without atoll. #2220
Makefile.in: added jumbo.o to genmkvpwd for non-atoll systems. #2220
pass_gen.pl: added zipmonster, asamd5, mongodb_scram, has160, fgt,
iwork, palshop, snefru hashes
pass_gen: added mdc2, efs and keyring hashes
memory.c: fixed crash in argon in memdbg mode. #2226
argon: some changes to build properly on certain systems
memdbg.h: fix build problems for mingw on circleCI
loader changes needed for -show to work properly. chopped pot issues,
and chopped pot with BIN_HASH data also.
mscash: wrong length used for signature check. #2225
off by one error in strnzcpyn() found with jtrts.pl, when adding
palshop and eigrp hashes
radmin: avoid buffer overread (asan) bug
argon2: merged 2 formats into 1. Put tags on extern functions, and
made many others static. #2229
argon2i: removed the older format file
oracle-9i sniffed hash (new format)
oracle9i. This one is actually CASE SENSITVEgit add
oracle9i_fmt_plug.c even though oracle is not. This is due to the
password being encrypted. I am not 100% sure it is case sensative,
OR if the encryption is alway the lc of the password. We need more
research on this onegit add oracle9i_fmt_plug.c
oracle9i-sniffed. Used asan to find some bugs causing CIs to fail in
-test-full. Did not core on cygwin
oracle9i: fixed bus error on Sparc. Hopefully this will fix clang
crash on TravisCI
oracle9i: removed const from iv (causing core on clang build), and
simplified sha1 key building function
oracle9i (o3logon). Added OMP, fixed overflow in prepare. Changed
format label to 03logon
oracle format: Some straggler FORMAT_TAG normalization
whirlpool. Fixed buffer write-overflow error in split. (found with
ASAN) #2237
whirlpool. Improved valid, to keep certain non whirlpool hashes from
passing. #2237
hexlen?() functions, added a return if the string was solid hex or
not, for validations
oldoffice: remove warnings caused by hexlen change
o3logon: change buffer size to right size, and added SHA1 to algorithm
name
hexlen?() functions had off by 1 bug detecting extra bytes.
pem: bug in valid, which recent changes to hexlen function made show
up
pem: re-add proper validation
validation enhanced on several formats (some cores and ASAN errors).
#2239
validation enhanced salted-sha1. #2239
validation overflows and cores. keepass and o3login. #2239
formats.c --test-full=0 was too agressive on salt-unifies. It was
even calling the code for formats with salt being fmt_salt_default.
That is now fixed
dynamic: added new dynamic string type: utf16-be with function
DynamicFunc__setmode_unicodeBE
configure: fail-abort if --enable-foobar but foobar can not be
loaded/used/found. #2241
salted-sha1 ASAN overflow. #2246
winzip: better valid, and split to canonize the magic field
salted_sha1: better validation, and fixed ASAN problems (salt buffer
overwrite)
wbb3: fixed overflow error caused by sub par valid function
wbb3: valid was not quite right. The salt is used as is, and not
converted back to raw from hex
base64_convert.h: adjusted header comment about negative return, since
all returns are not size_t so the negative logic is no more
salted-sha1: More work to fix overflows. #2246
winzip: common split code for both opencl and CPU. #2245
dynamic_1590 created. This added to handle IBM AS/400 salted sha1
hashes
keepass: improved valid, to avoid buffer overwrite. #2249
pass_gen.pl changed episerver to use ../run/pass_gen.pl[1] (the
unicode ready buffer). Added eigrp hash
keepass: revert valid change, to get CIs working again
keepass: removed non-inline file code within valid. It was causing
crash problems. #2249
as400-ssha1 thin dynamic format
as400-sha1: fixed compile warnings, and warning about not fully
cleaning the binary buffer
local changes should not have been committed
pass_gen.pl: commit 4223342 was incomplete and broke the usage page
unrarcmd: we add num items, but were only clearing 1. If more than 1
item added, undefined data would be present
pass_gen: Removed Basename lib requirement
pass_gen.pl: Removed Term::ReadKey as a required module to print the
usage screen. If found it will be used, but the script does not carp
if the module is not found. #2251
pass_gen: removed String::CRC32 and wrote my own pp crc32 function.
#2251
fuzz: Some constants changed to sizeof. Output a completed after each
format test during --fuzz run
o3logon: adjusted code to make --fuzz work with this format
o3logon: reworked changes to valid for fuzzing, as per @magnumripper
suggestions
o3logon: fixed bug in last commit
mdc2 is not always available. It now has a 1 time check
hdaa: patched to fix ASAN read error in new short format
SRP formats had resource leaks for non GMP builds
HDAA: better fix for memory overread ASAN for the new short hashes
recent changes with work around for invalid UTF-8 during valid,
changed to not error(), but to print a warning and ignore the
hashes. error() call in valid is a bad-thing (tm) to do, especially
exiting when no format specified, when jtr is searching for proper
format to match the hash
Corrected if condition boolean logic when fuzzing
HDAA: added split to canonize the shorter hashes (trailing or no
trailing $ char
wbb3: created a dynamic format compatible with wbb3. Can be much
faster. No OMP (usually), but good with fork
dynamic: removed speed hit to several formats caused by changes needed
for dynamic_1592. Added a new flag and only dyna_1592 uses it at
this time, so it is the only format that takes the hit
legacy generic build was not working
prince: changes needed to match john.c, so that legacy builds work
pass_gen: removed requirement of Digest::MD5. Handle interface change
introduced in v2.00 for Crypt::ECB. Fix #2266
gpg2john: allow multi-part message blobs (i.e. large file support).
#2109 #2218
gpg2john: each file processed gets its own large enough hash buffer
allocated. Prior code could crash on a multiple files run
gpg: full large file support. #2109 #2218
gpg: bugs found in -test-full=0 using ASAN
pass_gen: added as400-ssha1 and dynamic_1590
logger. Recent changes were not VC compatible
dynamic.conf. A tweak to 1598 and comments.
dynamic: work to improve utf8 handling
logger: Still issues with VC
gpg2john: buffer overflows fixed. gp fixed if bad file name given.
changed code so both -d and -S can be used on same run
Handle almost all of GPG. Still missing twofish hash, but added:
SHA224/384 hashes, crypts: Camellia-128,192,256
bench.c: message buffer not large enough. Bug when more gpg crypt
types added
gpg: added twofish. Now gpg format (cpu) handles ALL types
gpg: after testing all type and all max bit sizes, these sizes are
adequate for the big num variables to avoid overflow
bench: better fix for cost message buffer overflows
twofish: fixed new cfb mode to not output more bytes than requested
gpg2john: better error handling. #2255
gpg2john: fix hang problem in bz2 code. #2255
gpg2john: added warn_exit calls when truncated compressed files are
seen
misc.c: fixed a speelling error in comment (pure no code change)
rules: now multiple rulesets can be specified on command line
pbkdf2-hmac-sha1: fix ASAN buffer read overflow
TravisCI: try to hook in libASAN into one of the build steps
TravisCI: try to hook in libASAN into one of the build steps, take 2
Travis. Trying to find the right package to install
rpp.c: rule dupe removal code was broken for new multi rule list
handling
as400-ssha, commenting the hash format properly
o3logon: a couple functions needed static, and max buffer length was
too small
New format: oracle 10g sniffed protocol
oracle: added OMP support
oracle: ASAN problem fixed. Needed buffer large enough for UTF-16
data
o10glogon: adjustments to OMP_SCALE
wpapcap2john: fixed ASAN (and crash) problem, and found memory leak
zip: when there were multiple hits in the same SIMD block, later hits
would not get properly found. #2281
dynamic_big_crypt: Was not using openSSL version of Whirlpool if it
was present
leet: added compile time usage of openssl whirlpool if available. It
usually is faster than the SPH
pass_gen: added leet hash
leet: added additional test hashes
leet: added SIMD code for the SHA512 part. About 1.3x faster
leet: the non-SIMD had a syntax error in it.
pass_gen: removed errant comma added to list of formats handled
leet: trying to pacify the CI failure. I can not replicate the
problem, so sort of shooting in the dark
leet: that was problem found in travis-ci. Fixed in a documented way
 now
ssha512: fixed ASAN bug in 32 bit linux builds
#2363 sed s/ARCH_WORD_32/uint32_t/g In all source, other than core
john files
Porting issues for VC
salt resume bug fixed. New v2 salt record in .rec file stores KPC and
if this changes upon resume, it is handled properly Fixes #2817
argon2: alignment problems, caused crashes on machines requiring
alignment
enpass: Alighnment problems
fvde. Alignment problems and failure on BE systems
itunes-backup Alignment problems and failure on BE systems
Fix leet format on BE systems
BE buffer out of bounds warning when building BE systems
sha2 generic. Fixed stict pointer warning when building on PPC64
mscash1: memory access (crossing 16 bit and 32 bit pointer access) on
some BE systems was causing failures of 0 and 1 byte passwords.
Using proper union fixed problem
formats.c: make the testing candidates stir the long candidates a bit
better. When getkey and setkey do not match, print out what was
returned
raw-MD5: enhanced format so that SIMD works on a BE machine. #2888
raw-MD4: enhanced format so that SIMD works on a BE machine. #2888
raw-SHA1: enhanced format so that SIMD works on a BE machine. #2888
(small change for raw-md5)
raw-SHA24/256: enhanced format so that SIMD works on a BE machine.
#2888
raw-SHA384/512: enhanced format so that SIMD works on a BE machine.
#2888
raw-MD4/5: enhanced format self tests to make sure all byte
boundaries are handled in key loading. #2888
pbkdf2_hmac_sha1.h: ixed this core file to run properly on BE
machines. #2888 MANY formats will be fixed by this.
raw-MD%u: fixed this format to run properly on BE SIMD machines.
#2888 (changes to misc.[ch] also)
raw-SHA1-linkedin: fixed this format to run properly on BE SIMD
machines. #2888
BitCoin: fixed this format to run properly on BE SIMD machines.
#2888
BKS and Blackberry-ES10: fixed formats to run properly on BE SIMD
machines. #2888
zopmonster: fixed format to run properly on BE SIMD machines. #2888
dynamic, keyring, keystore, leet: fixed formats to run properly on BE
SIMD machines. #2888 (Note, dynamic was simply forced to not be
SIMD for BE systems)
dynamic: Missed some dyna files. #2888 SIMD BE porting
cryptmd5, cryptsha256, cryptsha512, and simd-intrinsics. Updated SIMD
code to work fr BE. #2888. Flat intrinsics needed fixing
leet: Port for #2888 moved into simd-intrisics.c flat loading, where
it should have been. removed from the format crypt_all
more changes to simd-intrinsics.c (BE port). Also ports for
blackberry, and citrix. #2888
episerver: port to BE SIMD #2888. Tested for all encodings
ecryptfs: port to BE SIMD #2888.
IPB2: port to BE SIMD #2888.
BKS: port to BE SIMD #2888.
hmac-*: port to BE SIMD #2888.
xsha xsha512: port to BE SIMD #2888.
sl3 ssha512: port to BE SIMD #2888.
salted-sha1 raw-sha*-ng: port to BE SIMD #2888.
Fix alignment problem in "dynamic"
phpass: salt alignment problem
android FDE: salt alignment problem
HDAA: salt alignment problem
dynamic (hmail server). salt alignment bug
Fix alignment problem in sha256crypt format
sha512crypt. Fixed alignment problem
Fix alignment problem in panama format
oracle11: Ported to BE #2888 Fixed alignment problem #2868
mysql-sha1: Ported to BE #2888
mssql mssql05 : BE porting #2888
mssql12 : BE porting #2888
pwsafe : BE porting #2888
RAKP : BE porting #2888
mscash2 : BE porting #2888
"mschapv2 and netntlm : BE porting #2888 (Hard format!)
mschapv2 and netntlm : Alignment fixing caused bug in compiling
non-SIMD code.
phpassMD5 : BE porting #2888
sapB, sapG, sapH, sunMD5, HDAA. Ported to BE SIMD #2888. All of
these formats ignore SIMD, and revert to oSSL code if built on BE
systems
SybaseASE : BE porting #2888
rar : BE porting #2888
office (all 3 variants) : BE porting #2888
office: Bug in cbfc4d4 fixed. Bug was when SIMD_COEF_32 was > 4
fixed mixed DOS eol. Fix #2919
NT2 : port SIMD to BE. #2888. Tiny code clarifications to raw-md5u,
since we do not BE change the GETPOS macros. Changed to the word
version which are BE independant
wpapsk: BE porting. Disable SIMD building on BE. #2888
7zip: BE porting. Disable SIMD building on BE. #2888
sapB, sapG, sapH, unalignment requirements met. #2868
added encoding and utf8 tests to Travis CI
Drupal7 : port to BE SIMD. #2888
HACKING.md Added additional information in sources.list generation,
along with a site to auto-generate
pass_gen.pl : added sapG algorithm
sapG : Now working for BE-SIMD builds. #2922
pass_gen.pl : added sapB algorithm
sapB : Now working for BE-SIMD builds. #2922
strnzcpy, using dynamic size for copy. #2928
oracle11 : ST failed after 09ccc79 for non SIMD builds. saved_key was
not allocated, but a single flat variable, in non-SIMD builds
added strnzcpylwrn to misc.ch. Same as strnzcpylwr but returns length,
not pointer to the buffer
Use strnzcpy and strnzcpyn in more places #2928
configure/arch.h. Enable UBSan alignment checking. Forces
ARCH_ALLOWS_UNALIGNED=0. Fixes #2908
HDAA: Salt alignment problems on 32 bit builds
configure : avoid any opencl_*.o and copying kernels unless building
for OpenCL. Fixes #2939
argon2. Normallized buffer usage, bringing more into standard usage
Makefile.legacy. Added a gomp enabled cygwin-x86-any target (for
testing non-SIMD builds)
Simd helper code #2915 GETPOS and set/get key logic (#2949)
Added common code for get_hash (#2953)
Damn. DOS linefeeds did make it in. Removed here
configure: Added a FULLY working, unified UBSan. alignment sanitizer
now works for all CPUs supported
formats.c : error exposed by UBSan. memcpy was getting passed NULL
for orig_salt in certain cases
configure : Add PowerPC64 building (#2942)
makefile.legacy, cygwin-x86-anyomp fixed
base64_decode replaced with base64_convert (#2969)
configure: Reworked the summary section
citrix_NS10. Removed ASAN problem from OMP builds
Couple of formats where salt alignment not right
rar: disable for CPU requirement alignment
base64. Whitespace change (eol)
powerpc: Simd Port fixes. Simd working for 64 and 32 bit AltiVec on
power
ppc configure port. Should be fully working now
configure: Added new --enable-ubsantrap. Like UBSan, but crashes on
any problems
rar disabled on aligned required systems, warning emit reversed
configure. A bad version got in. Fixed
rar: quite format not used warnings during configure run
Powerpc CPUID checking logic added to configure. #2861
Prior commit omitted calling autoconf before checking it in.
ppc configure fixed
ppc_cpuid. Added copyright. Changed logic getting auxv data (on
stack). Using caps2 for testing for 2_07git add ppc_cpuid.c
ppc_cpuid.c Make this work on non-linux PPC systems (for configure).
Simply return tha all needed flags are there
phpass. Salt align bug on non-simd 32 bit. #2956
ppc_cpuid.c would core dump on non linux systems if run with no
command line options
BE non-SIMD re-porting for XSHA512 and zipmonster
bks: BE Port #2888
BE report for hmacMD5 mssql12 mschapv2 netntlm ssha512. #2888
Autoconf: Add --disable-simd option. This will not disable any default
SIMD features enabled in the compiler but will also not enable any
more of them. And most importantly, it will define JOHN_NO_SIMD
macro which in turn will force using no-SIMD code paths in JtR.
ppc_cpuid : cleaned up error message shown when built on non-Linux
system
fmt_split_case issue dyna_compiler and --test-full=1 (#2997)
Port of wpapsk to use proper pbkdf2_hmac-sha1.h file. The great side
effect is this fixes the BE problem, and it is faster (#3005)
non-intel SIMD fixes. #3010 (#3011)
sapH, Port to BE SIMD code. #2922
HDAA. Ported SIMD to BE (ugly format). #2922
mute VC warnings
pkzip. Add warning for builds without zlib. #3033
Argon2. Disable SIMD logic in --disable-simd builds. #3044
pomelo. Disable SIMD in --disable-simd builds. #3044
scrypt. Disable SIMD in --disable-simd builds. #3044
pass_gen. Increased pbkdf2-hmac-sha1 to 179 byte salt. Started on
bitlocker (more to do)
expanded pbkdf2-hmac-sha1 opencl to 179 bytes, and kernel is unlimite…
(#3052)
pass_gen. Increased pbkdf2 hashes to allow salts up to 179 bytes
pass_gen.pl added msmoney format
x86-64.h. moved define of NT_X86_64 to inside prior JOHN_NO_SIMD
block
3limb salt handling for all 32 bit opencl pbkdf2-hmac-* #3035
 PO format. changed MD5_word to uint32_t. #3103
dynamic. Changed MD5_word into uint32_t. #3103
Star/Libre Office. Handle their buggy SHA1 logic (optional)
updated spelling error, removed testing code
Merge StarOffice stuff into LibreOffice
required changes to get VC to build again
logger.c has exe bit set per 679ae3c
Removed ALL memdbg code
updated the unused pre-built version of this file to its proper state
fixed dynamic overflows in single mode
dynamic. Perform a full-clean if m_count is less than max keys
dam exe bit set
do not force require full clean if m_count is MIN_KEYS size. also
added comment for later
dynamic_types: fix non-SIMD compile error caused by c31b66d
Added dynamic-compiler RDP format when script builds incorrectly
(#3568)
fgetl and fgetll both handled lines with NULL incorrectly. fgetll also
memory leak on long lines with NULL
Changes to common.[ch] and misc.[ch] Jumbo.[ch] triggered by code from
unit-tests project.
sap_pse require small change in valid after changes to misc.c
first version of unit testing code. Some changes needed in common.c/…
(#3580)
fixed bullet-proof code to not core on 32 bit builds
using void* in arithmetic is not valid, even though gcc extension
allows it (#3608)
uncommented all john.conf Option section params. There were a couple
source changes required to keep the default behaviors
default wordlistrules back to empty rule behavior

Dhiru Kholia (532):

Add jtrTestSuite as a submodule
Update MinGW stuff
Drop a couple of old formats from the unused/ folder
blatant attempt to increase my patch count
Support TGS-REP hash extraction, based on Kerberoast
Don't ignore non-inlined keepass hashes without giving a reason
Use sane inlining defaults in keepass2john
Update olefile to 0.41, add ACCDB support
Add support for cracking Inno Setup hashes
Revert "Add support for cracking Inno Setup hashes"
Add sample Inno Setup hash
Add support for md5($p.$s.md5($p.$s)) scheme
Add support for cracking HSRPv2 hashes
Add some HSRPv2 test vectors
A blast from the past!
Make SSH format actually crack Superfish key
Give credit where it is due
clean up /doc, #1
Stop wasting time in maintaining the legacy build system
Revert "Stop wasting time in maintaining the legacy build system"
Stop wasting time in maintaining the legacy build system
Emit a scary message when Makefile.legacy is used
Respect PEP8 a bit
Recommend PGPCrack-NG for cracking symmetrically encrypted PGP files
Add support for Notes/Domino 8+ H-hashes
Add support for SHA3 hashes (unify with Keccak stuff)
Fix the build failure caused by the recent SHA3 patch
Add support for hmac_pbkdf2_md5 hashes, v2
Add credit for SHA3 stuff
Remove unused function from the SHA3 format
Fix description in dominosec8 format
Add support for HAS-160 hashes
Add HMAC-GOST (currently broken for unknown reasons)
Add a test vector for HMAC-GOST
Add support for POMELO hashes
Add non-SSE implementation of POMELO as backup
Add support for Pufferfish hashes
Add support for Stribog hashes, v6 (SSE4.1 enabled)
Tweak README.MinGW a bit
HMAC MD5 cleanup, fixes #1324
Fix ASan crash in the new Keccak code
Add support for new Oracle Database 12c password hashes
Add support for Oracle 12c "H" hashes
Port truecrypt_volume2john.c to Python
Add support for keyfiles in TrueCrypt format, #1
Add support for keyfiles in TrueCrypt format, #2
truecrypt_volume2john.c is dead now
Create index on aes.a library, MinGW fix #1
Stop overwriting AR variable, MinGW fix #2
Simplify linking for luks2john.c, MinGW fix #3
Fix uaf_encode.c compilation, MinGW fix #4
Fix sshng2john.py, fixes #1374
Restore a new, robust version of krbpa2john.py utility
Remove broken krbpa2john stuff from pcap2john.py
Generate new hash format in krbpa2john.py
Make krbpa2john.py a bit more robust
Restore salt extraction "feature" in krbpa2john.py
Unify, and cleanup keystore2john stuff
Make krbpa2john.py a bit more robust, #3
Make krbpa2john.py a bit more robust, #4
Add support for MinGW builds, with <3 from JimF!
dynamic_1016 never did md5($s,$p), it does md5($p,$s) instead!
Use CircleCI for MinGW builds instead of Travis CI
Make CI testing configuration saner
Show CircleCI build status in README.md
Add support for more NTP hash types
Fix copying stuff in MinGW builds
Suppress wine warnings in CircleCI environment
MinGW build filenames now include short commit hash
"download" badge now downloads latest MinGW build
Start cleaning up *2john tools
Add support for vBulletin > 3.8.5
Use "dynamic_2006" for vBulletin!
Fix issue #1595
Update olefile to 0.42.1 (released 2015-01-24)
Make CircleCI testing more robust
Update blockchain2john.py to handle v2 hashes, fixes #1050
Make blockchain2john.py work with Python 3 too
Simplify kirbi2john.py utility
Add a test vector in blockchain format
Remove a debugging statement from the HSRP v2 parser
Improve handling of "special" blockchain v1 wallets
Remove testing dictionary, WTF?
[ssh] advertise ability to crack PEM files
[ssh-ng] add support for EC private keys
[ssh] add a PEM test vector
Fix downloading of Windows builds
Fast PEM cracker
[pem] fix errors in pem2john.py
[pem] make the format more robust
[pem] false positives shouldn not happen now
[pem] add support for dsa keys
[pem] add a better format description
[ssh-ng] advertise support for new "OPENSSH" format private keys
[keychain] tested on OS X El Capitan
Add support for cracking files using GPG symmetric encryption
make gpg format slightly more robust
Add support for cracking Apple iWork '13 files
Add support for iWork '09 files
GPU support for cracking Apple iWork files
Add partial support for KWallet 5 files
Fix out-of-bounds write in raw-md5 format, caught by ASan
ssh format is dead, use ssh-ng format instead
Fix bogus line endings in vdi2john.pl
Advertise npdf2john a bit
Update instructions for MinGW builds
[vnc] fix cracking of short passwords, fixes #2021
Add documentation for cracking PDF files
Add documentation for cracking PST files
[iwork-opencl] Remove unused identifier, noticed by Lei Zhang
Improve VTP packet parsing a bit
Add a comment in keystore2john.py
[keepass] KeePass 2 does not support Twofish by default
Warn instead of aborting when using keyfile with KeePass 2 databases
Document known formats built using the dynamic compiler
LUKS only supports sha1 as the hash function currently
Document support for cracking JBoss AS 7.1 hashes
enfcs: warn for unsupported ciphers
Minor fixes in DYNAMIC_COMPILER_FORMATS.md
Ugly hack to get pfx2john working again after LINE_BUFFER_SIZE changes
Add support for "BKS" (BouncyCastle) format keystore files
pfx-ng format
Document the correct hash format for MongoDB hashes
Add utility to extract MongoDB hashes in JtR format
Faster way to crack MONGODB-CR system hashes
Fix a misleading indentation compiler warning
Add support for cracking MongoDB SCRAM-SHA-1 system hashes
Rename MONGODB-MiTM system hashes to MONGODB-CR network hashes
Add support for sha256 in pfx-ng format
Add support for sha512 in pfx-ng format
Add support for MD5(Palshop) hashes
Add a sha512 test vector in pfx-ng format
Add a usage note for gpg2john utility
Add support for cracking SocialEngine hashes
Add support for cracking MD5(ZipMonster) hashes
Add some more MongoDB network authentication test vectors
Add a pfx-ng test vector from CMIYC 2013 contest
Fix TravisCI failures by disabling some OpenCL formats
Fix a buffer overflow warning at rawSHA512_common_plug.c:333
Fix a unused function warning in zipmonster format
Add support for cracking Poison Ivy "hashes"
krb5pa2john: Fix for broken user extraction due to field name change.
Closes #2299.
New format for cracking leet.cc hashes
Add support for cracking FileZilla Server hashes
leet: avoid crash while processing dynamic_82 type hashes
Add dump-apex-hashes.sql script to README.apex
Add support for Enpass Password Manager (CPU + GPU)
Add support for modern KWallet databases
Add pdf2john.pl tool for extracting hashes from PDF files
Use Fedora 25 in CircleCI
HSRP: remove an old joke
Remove remaining traces of ssh2john
Fix rename-to-exe logic for the MinGW build
Fixes and improvements for iwork2john.py
Add support for Oracle PeopleSoft PS_TOKEN hashes
Fix the link to LICENSE file
Fix a typo in iwork format
Add SIMD support for cracking modern KWallet files
Fix a thinko in enpass format
phpFox v3 hashes can be cracked using dynamic_13
Add support for iTunes Backup (#2410)
iwork-opencl: move all work to GPU, v2 (#2417)
s/erros/errors (fix typos)
Add support for cracking FileVault 2 hashes
FVDE (FileVault 2) OpenCL support (#2436)
Fix error handling in luks2john.py
Remove now unnecessary BIO_ checks
Remove pdf2john.py, use pdf2john.pl instead
gpg2john.c: fix compilation error in case of missing zlib-dev
Improve salt and hash validation in EPiServer format
Support handling of modern Bitcoin Core wallet versions
Various efs2john.py enhancements
Add support for cracking Padlock databases
Initial support for Jetico BestCrypt 8.x / 9.x (#2448)
dmg2john.py: handle encrypted_blob_size with value 64 properly
Fix a minor memory leak in BestCrypt format (#2459)
Fix a minor memory leak in Padlock format
PEP 8 fixes for pcap2john.py
Fix a typo in spelling out "JWT"
Add support for cracking MS OneNote (.one) files
Add support for cracking Windows BitLocker passwords
Add support for auditing XMPP SCRAM-SHA-1 hashes
Increase PLAINTEXT_LENGTH in Bitcoin format
MSDP MD5 password hashes are also supported
Misc OpenBSD disk encryption related improvements
Fix padding checks in ssh-ng format
Add support for bcrypt PBKDF in OpenBSD-SoftRAID format
Various 1password2john.py cleanups
Add another test vector in Oracle12C format
Add another test vector in pomelo format
Add support for IPsec Authentication Header (AH) hashes
Add support for cracking FreeBSD GELI hashes
Add support for ArubaOS hashes
Add support for cracking SNMPv3 hashes
Fix ALGORITHM_NAME in BitLocker CPU format
Various bitlocker2john.c cleanups
Various BitLocker related documentation fixes
Use OS X support in Travis CI
All FreeBSD GELI versions (v0 to v7) are supported now
Be more informative when benchmarking itunes-backup format (#2527)
Fix import problems in kerberom.py
Add support for cracking Ethereum wallets (#2525)
Add support for cracking Electrum wallets
Add support for cracking MultiBit wallets
Improve and extend support for IS-IS hashes
Add support for more OSPF and IS-IS authentication algorithms
Add support for Ethereum presale wallets
Limit output bkp field length in ethereum2john.py
Partial OpenCL support for Ethereum format
Fix a harmless out-of-bounds read in ethereum format
Fix sneaky false positives in ethereum format
Fix typos in blockchain formats
Blockchain formats: Cleanup code and support My Wallet v3
OpenCL support for Ethereum presale wallets
Add a real-world Ethereum presale wallet test vector
Reduce false positives in MultiBit format
Ethereum format: Tune OMP_SCALE value
Ethereum format: Move AES decryption + Keccak to GPU
MultiBit format: Improve performance by doing delayed decryption
OpenCL support for cracking PFX files
OpenCL support for cracking FreeBSD GELI
Add support for EAP-MD5 hashes
Add support for cracking Electrum 2.8+ encrypted wallets
Electrum format: remove build hacks and add SIMD
OpenCL support for cracking Electrum 2.8+ wallets
Warn about unsupported configurations in keepass2john.c
Add support for MS Money
Add support for ChaCha20 in KeePass format
Add support for Access 2007 files using CryptoAPI
Add support for SunShop hashes (#2605)
Fix handling of GPG symmetric encryption without MDC
Reduce false positives in MultiBit format (take 2)
Add FMT_NOT_EXACT in GPG format to handle files without MDC
Fix multiple problems in sshng2john and SSH-ng format
Use Fedora 26 in CircleCI
Make uaf2john more robust in input handling
Make keepass2john.c more robust in handling input
Make dmg2john.c more robust in input handling
Make vncpcap2john.c more robust in input handling
Few enhancements for openssl2john.py
Make SIPdump more robust in handling input
Make eapmd5tojohn more robust in input handling
Make wpapcap2john more robust in input handling
Enable usage of libFuzzer with JtR
Add support for SHA-256 in openssl-enc format
Enable fuzzing of bitlocker2john using libFuzzer
Enable -fstack-protector-all for a CircleCI build
Assorted PuTTY cracking enhancements
Add WPA2-PSK-SHA256 (802.11w) support
Add support for Ethernet link type
Add support for Dashlane password manager
Few fixes for the LastPass format
Add support for LastPass CLI
Enable fuzzing of putty2john using libFuzzer
Few enhancements for the LastPass formats
Report tunable costs for ssh-ng format
Use the proper stdint.h file
Cleanup old putty2john build stuff
Report tunable costs in OpenBSD-SoftRAID format
Get rid of EVP_BytesToKey from Dashlane format
Add support for truncated Electrum hashes
Fix a whitespace error in electrum format
Report tunable costs in electrum format
Fix a typo in bestcrypt2john.py
Add support for cracking PGP WDE encrypted drives
Eliminate warnings about unused functions from secp256k1 code
GPG format supports cracking PGP Zip files too
Add support for cracking PGP Virtual Disk images
Add support for EME2-AES in PGP Disk format
Fix a problem detected by ASan in PGP WDE format
Enable fuzzing of racf2john using libFuzzer
Enable fuzzing of hccap2john using libFuzzer
OpenCL support for PGP Disk format
Show limitations of pcap2john.py explicitly
OpenCL support for PGP WDE format
OpenCL support for LastPass formats
Add support for cracking Kerberos AS-REP responses
Various enhancements for the TrueCrypt format
Minor updates for 1Password stuff
Make error messages more informative in keepass2john.c
Add more test vectors in PGP Disk format
Add support for cracking PGP SDAs (self-decrypting archives)
Generate 32-bit MinGW builds with AVX2, AVX and SSE
Add another test vector in PGP SDA format
Cleanup DMD5 format a bit
OpenCL support for cracking PGP SDAs
Make luks2john.py compatible with Python 3 too
Fix support for RAR 5.x SFX files
Remove CPU fallback from GPG OpenCL format
Relax length_id check in PDF format
Use brute-force in office2john.py to discover RC4 Encryption Header
Fix and improve valid in krb5asrep format
Fix and improve valid in krb5tgs format
Bail out for SSH1 keys in putty2john
Improve error handling in gpg2john
Fix a regression in gpg2john for multi-key armored ASCII files
Add OpenCL support for SHA-256 based S2K
Add OpenCL support for SHA-512 based S2K
Initialize Twofish correctly in GPG OpenCL format
Support etype 17 and 18 in KRB5 AS-REP format
Add support for extracting TGS-REP hashes
Update documentation to reflect removal of krb5-23 format
Fix reliability problems in KRB5 AS-REP format
Minor updates for KRB5 TGS stuff
Add support for cracking etype 17 "native" Kerberos hashes
Add support for cracking Kerberos des-cbc-md5 hashes
Add support for cracking Kerberos credential cache
Add missing null-termination in KRB5 AS-REP format
Add support for cracking Neo wallets
Minor speed improvement for krb5-3 format
Cleanup the bundled libsecp256k1 code a bit
Rename krbpa2john.py to krb2john.py
Fix parsing for WinZip AES encrypted archives
Add information about auditing Openfire
Add support for PKWARE's SecureZIP
Fix endianness issue in BitLocker format
Add varied test vectors in GPG format
Notes on testing JtR on big endian systems
Fix salt alignment in known_hosts format
Reduce false positives for GPG files lacking MDC
Disable POMELO on big-endian machines
Improve parsing of WinZip style AES encrypted archives
Further reduce false positives for GPG files lacking MDC
Disable POMELO correctly on BE machines
Fix cracking of GPG files using 64-bit symmetric encryption
Add AVX2 version of POMELO
Improve robustness and correctness of zip2john
Assorted fixes for POMELO format
Fix alignment problem in config.c
Fix alignment problem in OpenVMS format
Re-fix alignment problem in OpenVMS format
Correct casing for webEdition word
Fix alignment problems in ethereum format
Fix a typo in dynamic format
Correctness fixes for GPG symmetric mode cracking
Fix alignment problem in pkzip format
Refactor krb5asrep format to facilitate OpenCL implementation
Fix alignment problems in various OpenCL formats
Improve HACKING notes based on feedback from JimF
Add support for cracking NetIQ SSPR hashes
Treat various values as unsigned integers in 7z2john.pl
Relax CRC validation check in 7z format
Various fixes for alignment problems
Add EmulateBrokenEncoding feature
Add support for cracking TACACS+ hashes
Improve scalability in tacacs-plus format
Use strnzcpy and strnzcpyn in more places
Clear salt structure before usage
Add support for cracking WLCCP LEAP hashes
Fix salt alignment in tacacs-plus format
Add support for extracting iSCSI CHAP hashes
Rename $ to _ in UAF code to avoid compiler issues
Add support for RADIUS CHAP hashes
Fix OpenSSL API usage, and hash validation bugs in DPAPImk format
Unify all Lua scripts into a single file
OpenCL support for cracking PEM files
Add OpenCL support for cracking AS-REP hashes
Assorted fixes for tacacs-plus stuff
Fix salt alignment in phpass and ethereum formats
Fix warnings in MinGW builds (#2964)
Update licensing terms for tacacs-plus stuff (#2976)
Add a test vector with negative CRC value in 7z format
Use Fedora 27 in CircleCI
Add support for cracking DHCP OMAPI hashes
Add support for cracking BIND RNDC hashes
Add a note about PrestaShop 1.7
Fix JtR-TestSuite failure for RSVP format
Add new_keys logic for some hash types in RSVP format
Add support for cracking Apple Notes databases
Assorted code cleanups
Update README.BitLocker to reflect current state
Fix warnings by using correct data types and format specifiers
Expose kdf type as a tunable cost in ethereum format
Rename "omp_t" to "threads"
Minor cleanups for multibit format
Various fixes for ODF format
Minor cleanups for OpenBSD-SoftRAID format
Refactor OpenBSD-SoftRAID format
Assorted code cleanups, part 2
Add OpenCL support for OpenBSD-SoftRAID stuff
Cleanup bitcoin format a bit
Add support for older bitcoin wallets
Cleanup kwallet format a bit
Cleanup money format a bit
Assorted cleanups, part 3
Cleanup net-ah format a bit
Cleanup ospf format a bit
Cleanup sspr format a bit
Cleanup putty2john a bit
Add support for cracking 1Password for macOS databases
Cleanup dashlane format a bit
Cleanup EncFS format a bit
Add support for cracking MultiBit Classic .wallet files
Cleanup vtp format a bit
Cleanup Siemens-S7 format a bit
Cleanup VDI format a bit
Update copyright year
Cleanup xmpp-scram format a bit
Report tunable costs in multibit format
Rename sxc2john.py to staroffice2john.py
Rename the sxc format to staroffice
Unify common code for StarOffice format
Add varied test vectors for StarOffice format
Make staroffice2john.py slightly more flexible
Rename and refactor the ODF formats
Minor cleanups for openssl2john.py utility
Fix copyright text in gpg-opencl format
Minor cleanups for encfs2john.py utility
Remove the broken ssh2sshng.py utility
Fix license text in bitcoin2john.py utility
Fix license text in aruba2john.py utility
Fix license text in krb2john.py utility
Fix license text in htdigest2john.py utility
Fix license text in cracf2john.py utility
Add attribution text in LibreOffice format
Add OpenCL support for cracking NetIQ SSPR hashes
Document the process of cracking AuthMe hashes
Add support for cracking Bitwarden hashes
Add GPU support for cracking Bitwarden hashes
ethereum2john.py - add newline in the warning message
Add support for cracking BitShares wallets
Add documentation for RACF stuff
Add support for cracking BitShares backup files
Cleanup tacacs-plus format a bit
Cleanup bitwarden format a bit
Add support for cracking Monero wallets
Add support for AES-192-CBC in SSH-ng format
Add support for AES-256-CBC in SSH-ng format
Enhance symlink resolution feature in MinGW builds
Add support for cracking Ansible Vaults
Tune MKPC and OMP_SCALE in net-ah format
Tune MKPC and OMP_SCALE in wbb3 format
Tune MKPC and OMP_SCALE in tcp-md5 format
Tune MKPC and OMP_SCALE in sspr format
Tune MKPC and OMP_SCALE in pgpdisk format
Tune MKPC and OMP_SCALE in pgpsda format
Tune MKPC and OMP_SCALE in pgpwde format
Tune MKPC and OMP_SCALE in vtp format
Tune MKPC and OMP_SCALE in securezip format
Start using CircleCI 2.0 platform
Document cracking of TACACS+ hashes
Tune MKPC and OMP_SCALE in PEM format
Cleanup SNMP format a bit
Tune MKPC and OMP_SCALE in RSVP format
Add support for cracking Telegram (Android) passcode hashes
Add support for cracking DNS TSIG hashes
Add support for extracting HTTP Digest Authentication hashes
Cleanup the old Android FDE format a bit
Add support for more ciphers in PEM format
Calculate correct key length in PEM-OpenCL format
Add support for tunable costs in PEM formats
Cleanup space-before-comma instances
Add OpenCL support for cracking Ansible Vaults
JtR works for iWork 2018 files too
Add support for cracking Adobe AEM hashes
Add support for cracking andOTP backups
Skip the unrequired decryption of data
Add support for cracking Android Backups
OpenCL support for cracking Android Backups
Document the process of auditing Kerio Connect hashes
Python PEP 8 and other trivial fixes
Cleanup BestCrypt format a bit
Add support for cracking Signal passphrases (#3271)
Add support for cracking RADIUS authentication hashes
Add support for cracking DHCPv6 authentication
[GPG] Avoid incorrect processing of very old GPG keys
Add support for cracking DHCPv4 authentication (#3279)
Add support for cracking older BitShares wallets (#3283)
JtR supports PRiVCY wallets (#3286)
Generate less revealing hashes for Bitcoin wallets (#3290)
Fix a typo - can't trust autocomplete always (#3291)
Add support for cracking Telegram Desktop passcodes (#3295)
OpenCL support for cracking Telegram Desktop passcodes (#3296)
Bunch of trivial fixes (#3298)
Improve auditing documentation a bit (#3299)
Parsing fixes for telegram2john.py
Add support for cracking SAP's PSE files (#3303)
Some assorted trivial fixes
Add another tunable cost in PFX formats
Add support for cracking Tezos keys
Add SIMD support for cracking Tezos keys
Add OpenCL support for cracking SAP's PSE files
Add OpenCL support for cracking Tezos keys (#3317)
Add another test vector to SIP format
Fix the size of salt buffer in Tezos kernel
Fix algorithm name in Tezos CPU format
Tezos: Give credit where it's due
OpenCL support for cracking SSH keys - Initial Cut
Make GCC 8 happy in various ways
Add SIMD support for cracking Ethereum Presale wallets
Fix more GCC 8 warnings
Use Fedora Rawhide in CircleCI
Fix recent incorrect buffer resizing in charset handling code
Ignore GCC 8's -Wstringop-truncation warning
Disable -Wstringop-truncation more generally
Partial revert of commit bf2048be86 - part 1
Drop redundant memset zero padding before strncpy
DMG: Add long self-test vectors
 Install zlib1g-dev package explicitly on Ubuntu (and Debian)
Add a note on cracking ZooKeeper hashes
Cleanup and enhance axcrypt2john.py utility
OpenCL support for cracking AxCrypt 1.x
Minor SSH-OpenCL tweaks
Style fixes for bitlocker2john.c
Add support for cracking AxCrypt 2.x
Add support for cracking IBM 4690 ADXCRYPT hashes
Add APFS formatted DMG self-test vectors (#3362)
Make pcap2john.py easier to deploy (#3384)
Add support for cracking APFS FVDE volumes
Add one more self-test vector in agilekeychain format
Share more code between STRIP formats
Share more code in Apple Keychain format
Share more code in Password Safe format
Share more code in 1Password Agile Keychain format
Relax parsing in pdf2john utility
Add OpenCL support for cracking 1Password Cloud Keychains
Fix up our issue template language
Enable ADXCRYPT to run on BE (e.g. mips64) machines
[ZIP] Fix processing of multiple extra fields
Add support for cracking SolarWinds Orion hashes
Fix comment styling a bit
[CHAP] Accept longer challenge strings
JtR works with Office 2019 just fine
Support Python 3 in ml2john.py utility
Add support for QAS vas_auth
[MAINT] Rename ml2john.py to mac2john.py (#3491)
Add an alternate implementation of mac2john.py script
[MAINT] Fix an informational message regarding KRB stuff
Clean-up and enhance pem2john.py
OpenCL support for cracking AxCrypt 2.x containers
Make DiskCryptor format functional on BE machines
Add support for cracking encrypted VMware VMX files

Claudio André (318):

OpenCL: AMD devices have some weird bug when using pinned buffers.
Forgotten file: see ec080119f45ffef07f7f1dd36e120ee56d5d6734
OpenCL: Updates to get_processor_family().
Inform the user if (s)he is using a recommended driver. Also, it is
possible to warn if a crappy driver version is detected.
Be more user friendly using a better README file in Github main
project page.
Change the comment of sha512crypt-opencl PLAINTEXT_LENGTH.
Cosmetical changes to OpenCL SHA256 formats: PLAINTEXT_LENGTH
Update copyright and contact information in SHA2 OpenCL.
Remove any OpenCL output when running JtR --show.
Mute a TestSuite error message caused by an OpenCL message.
Mask mode implementation in raw-sha256-opencl.
Correct mask mode behavior when GWS is supplied. We'l use the figure
as-is but we still print a message about internal generation. See
5980618088329ea3820c226887e62a4b6c779c5b.
Small tweaks (clarify a few comments and remove a TODO).
OpenCL share code with CPU formats in RAW-SHA512.
Move OpenCL auto tune code to reset() in raw-sha256.
Mute auto tune warnings on default verbosity. Closes #1146.
Try to make things clear to the OpenCL compiler.
Adjust c/s for mask mode benchmark.
This changes might produce better binary code for GCN (unfortunately,
 the ancient drivers on super cannot produce a final prove).
Keep this senseless change in sha256 OpenCL in history, but remove it
from source.
Used shared code between CPU and GPU formats for raw-SHA256. (No
static method in header).
Use only mem_calloc_tiny inside shared code.
Used shared code between CPU and GPU formats for raw-SHA512. (No
static methods in header file).
Mask mode implementation in [raw/x]-sha512-opencl.
Handle the salted format for the GPU mask mode.
SHA512 OpenCL 130% performance gain on Tahiti (I see no reason for it,
but AMD does).
SHA256 OpenCL follows the SHA512 recent changes (> 1Gc/s on Tahiti).
SHA512 crypt OpenCL follows the recent pattern.
Share more code inside OpenCL driver detection.
I forgot a define for other platforms.
Refuses to run the kernel if the driver is known to be a disaster.
Optimize SHA512 crypt OpenCL for Nvidia.
Fix a possible bug in xsha512 OpenCL format.
Small tweaks.
Fix for the intermitent mask error on CPU. - see:
https://github.com/openwall/john/issues/1037#issuecomment-71936052
OpenCL raw-SHA256 and raw-SHA512: Do not compile kernel in init(). It
slows things down.
OpenCL SHA256crypt and SHA512crypt: Do not compile kernel in init().
It slows things down.
Reinitialize formats after any execution.
Add new AMD drivers to common-opencl.
Initialize OpenCL environment on init(), otherwise john's output
became a mess.
Protect macro parameters.
Fix to get the Omega driver performance using 6770.
Deprecate the 'dual' autotune_run interface.
OpenCL sha2crypt: Drop the separate crypt_all_benchmark().
Remove a duplicated initialization function call.
Improve error output when no OpenCL devices available.
Move Claudio's SHA2 file to be a '-ng' format.
Use a maskless version for raw-sha512.
Small tweaks in SHA2 OpenCL files.
Fix for sha256crypt running on AMD 15.5 driver.
Previous fix for sha256crypt is necessary for Catalyst 15.7 too.
Remove unnecessary code.
Send a sorted plaintext list to GPU in sha256crypt.
Remove the unnecessary stress applied to OpenCL device in sha256crypt.
For recent AMD drivers, select the new OpenCL sha512crypt kernel.
Fine tune the OpenCL sha512crypt kernel for GCN on super (15.7
driver). It is 2x-3x faster now.
Fine tune the OpenCL sha512crypt kernel for Nvidia sm_5x on super. It
is 50% faster now.
Fix: while testing (tuning) LWS values, it is not possible to use a
GWS value bigger than the buffer memory available.
Send a sorted plaintext list to the GPU device in OpenCL sha512crypt
Further tune the OpenCL sha512crypt kernel for AMD driver >= 15.5.
Fix: waste of memory by forked OpenCL runs. Closes #1707.
For sm_5x, vector operators does a better job in OpenCL sha512crypt
(on super).
OpenCL: Use bitalign for ror_64() in shared code in Claudio's formats.
See #1819.
Tune raw SHA2 OpenCL Claudio's formats and follow the new pattern.
Bugfix in ldr_sort_salt(). Last salt did not get hash updated.
Use lut3 when available.
Fix a Ctrl+C - Ctrl+V bug.
Seems to be better to revert to previous statemet (AMD driver still
surprises me).
Adjust Claudio's raw SHA2 formats to deal with GPU mask tune and
benchmark.
Inser a #pragma for old AMD compilers. Closes. #1834.
Tune only once in GPU mask mode.
Clean the value of a variable to avoid the error below: gws:
3072 1512Kc/s 1512695 rounds/s 2.030ms per crypt_all()+
gws: 6144 272547Kc/s 272547 rounds/s 22.542ms per
crypt_all() gws: 12288 1507Kc/s 1507300 rounds/s
8.152ms per crypt_all()
Split OpenCL SHA512crypt a little bit more.
New OpenCL SHA256crypt kernel.
Remove some duplicated code.
Tweak OpenCL SHA512crypt a little.
A few tweaks.
Use MAYBE_CONSTANT for other SHA2 GPU kernels.
Remove duplicated code.
Run indent and astyle ONLY.
Remove unnecessary code in OpenCL sha512crypt.
Update driver detection to new Crimson.
Include the last [1] WHQL certified driver for the 2nd generation of
legacy hardware [2]. [1] From now on only beta quality software will
be provided (if provided, of course). [2] Radeon™ HD 8000 – HD 8400,
HD 7000 – HD 7600, HD 6000, and HD 5000 Series.
Improve recommendations about OpenCL driver versions.
Allow to calibrate SHA512crypt kernel to any GPU. Gain seen on super:
> 3k c/s on GCN > 15k c/s on NVIDIA
Add some new configuration values to john.conf.
If exists, use the configuration saved in john.conf.
Get rid of the specific kernel for GCN (no longer recommended).
Allow to calibrate SHA256crypt kernel to any GPU. Gain seen on super:
> 10k c/s on GCN
Move drivers and definition of "recommended" to john.conf.
Fix the misformatted device identification/position message.
Remove the platform" notation in JtR.
Select the most powerful GPU as the default one.
Fix for singularis vs pluralis.
Fix a memory leak in sha[256/512]-opencl.
Fix raw-sha512 to follow the fakedb patterns.
Fix raw-sha256 to follow the fakedb patterns.
Merge AxCrypt in-memory secrets with Raw-SHA1.
Fix AxCrypt-SHA1 on non-SIMD. Closes #2008
Improve find_valid_opencl_device() detection.
Improve Raw-SHA2-opencl scalability.
Improve auto-tune controls.
Prepare MASK_KEYS_GENERATION macro to future improvements.
Fix a bug introduced in the scalability improvement.
Use shared code for NSLDAP.
Tune memory used by raw-SHA2-opencl.
Mute a compiler warning.
Shrink the hash/spread function. False positives < 0.05% (for 100,000
and 3,000,000 test cases).
Tune OpenCL kernel.
Re-format a header file.
Fix a memory overread. Closes #2049.
NVIDIA driver is doing something really strange. This implements a
workaround.
Do nothing if db is not set.
Tune SHA512crypt-opencl further. Improvement 1-2 kc/s.
Improve Raw-SHA256-opencl scalability.
Synchronize a debug message in raw-SHA512-opencl.
raw-SHA512: - Use the proper GWS size in prepare kernel. - Sync a
comment.
Small tweaks for raw-SHA256-opencl.
Tune memory usage for non GPU mask mode.
Correct a misplaced label in --list:opencl-devices.
GWS should be multiple of processors count not multiple of CUs.
raw-SHA512 [OpenCL]: - Sync with raw-SHA256. - Use a k=2 Bloom Filter.
Do not use a #define as a function. - see #2091.
Improve the Ubuntu OpenCL documentation.
Always save the commit used to build JtR. - Nothing will change if the
user has a regular git tree.
Improve OpenCL error handling.
Allow an OpenCL system wide build. - save compiled binary kernels
inside JtR private home folder.
Fix XSHA512-ng-opencl (see #2208).
Tune gpg-opencl auto-tune max duration.
Re-enable mscash2-opencl format on Travis CI.
Fix and document sha512crypt-opencl problems with NVIDIA OpenCL driver
361.
Rearrange [raw/x] sha512 formats
path_init() must be called prior to CPU fallback mechanism
Fix some typos
Fix an ld warning seen on CI
Fix a out-of-range compare bug in gpg2john
TravisCI: add ASAN into gcc build step
regex: fix for a warning seen when compiling against
OpenCL: a platform could have no devices
CI: test JtR (also) on a fresh environment
github: add guidelines to github users when they create an issue or a
pull request
github: improve issue guidelines trying to avoid CUDA and OpenCL known
issues
CI: Add a TS check
CI: test a non OpenMP build
CI: change the recent OS image to be used
CUDA: more cosmetic cleanups
OpenCL: make errors non-fatal for a "--test" run
encfs_common.c: fix a leak
axcrypt: fix prepare to keep input fields unchanged
loader: fix the --show
lotus: fix a macro name
fix: add a missing include in common.h
fix: change an "#include" in nsec3 format
circle: fix the build issue introduced in
a63b819bfc8a1559ace20b2824eb7f2e9014b5ea
build: clean up some 7-zip tmp files (see #2383)
minGW: always use aligned allocs
minGW: workaround for (wpa/pcap)2john tool
minGW: fix for a memdbg bug introduced in b701bc0
build: add a configure option to treat build warnings as errors
OpenCL: be more resilient on --list=opencl-devices
OpenCL: fix for a non SIMD build
general: mute a warning and a possible truncation (#2486)
OpenCL: improve error handling in --list=opencl-devices (#2489)
readme: fix markudown (#2487)
racf2john: mute a compiler warning (#2519)
PO: fix a bad conversion in a signed char buffer (#2520)
maint: add a static code analysis tool (#2561)
maint: remove one unused function in ssh_ng format
descrypt-opencl: warn about the slow startup at many salts
OpenCL: mute ptxas info (from build log) in default verbosity
OpenCL: add the failure code to the error message
CI: update Travis environment
maint: fix MAYBE_INLINE macro for GCC 5+
maint: mute a compiler warning [-Wmacro-redefined] (#2746)
maint: add pocl as a valid OpenCL implementation (#2750)
maint: rename 'gpu_devices' to be adequate to any accelerator
OpenCL: allow to duplicate a device on --devices list (#2781)
maint: tweaks to incremental vs. --min/max-length options (#2796)
maint: mute a compiler warning [-Wformat-extra-args]
maint: remove a comparison of an unsigned < 0
maint: remove a comparison of an unsigned < 0
docs: add Debian and derivatives to Ubuntu installation notes
docs: specify the path to john binary on issue template
2john: fix a tautological compare in wpapcap2john
docs: explicitly state it's about user-local build
incr: allow -max-len to bump Incremental's maxlen beyond the conf
CI: disable AVX2 for all Travis builds
OpenCL: rename to follow the file naming conventions
maint: remove a duplicate #include
mask: a MACRO was defined in the wrong place
OpenCL: fix the guard name in opencl_mask_extras
core: fix a bug introduced in 38d20bb
maint: remove a comparison that is always false
Revert "maint: remove a duplicate #include"
modes: increase MaxLen if MinLen is specified
CI: run the encoding tests on its own job
CI: test encoding inside OSX job
maint: remove all OpenCL references in a non OpenCL build
maint: use runtime CPU detection in ./configure
CI: use clang with ASAN
CI: let the new SIMD detection do its job
maint: autoreconf to update the build process
maint: add more checks to ppc_cpuid detection
maint: style and unterminated '#if' fixes
maint: mute warnings in ppc_cpuid
maint: fix for non SIMD builds
maint: remove the usage of 2017q3 worker
maint: handle a build outside git and archive
maint: partial revert of 551e468
maint: fix for uninitialized variable
maint: fix for uninitialized variable
maint: remove any tag from the JtR version
maint: remove .gitmodules
autotools: add a make target to copy OpenCL code
autotools: group optional libs at the end
OpenCL: do not use realpath(3) on CygWin
CI: link the Travis badge to the default branch
maint: enable fallback for CygWin
doc: proper case OpenCL acronym
OpenCL: warn if --dev lists more OpenCL devices than fork
mask: add FMT_MASK flag to raw-SHA512-opencl
OpenCL: add workaround for '-I' on CygWin
OpenCL: fix for uninitialized pointer in bitwarden
sha512crypt-opencl: Disable vectorizing for macOS. See #3234
docs: add guidelines for contributing
docs: add license and code style to guidelines
docs: announce the project relevance
OpenCL: rawSHA256 and rawSHA512 new scheme of LWS=NULL
OpenCL: SHA256crypt and SHA512crypt new scheme of LWS=NULL
maint: refuse --device if the format is not device-aware
OpenCL: fix AMD driver info
OpenCL: refactor the acquisition of the OpenCL environment
OpenCL: JtR should work after a lock fault
docs: update AMD OpenCL instructions
OpenCL: rawSHA256 and rawSHA512 set GWS to LWS multiple
OpenCL: SHA256crypt and SHA512crypt set GWS to LWS multiple
OpenCL: rename the get_multiple macro
OpenCL: fork was over validating `--dev`
OpenCL: add devices in the preferable order
Sort OpenCL devices (#3507)
OpenCL: warning if there are more devices available
OpenCL: check beforehand if there is any OpenCL device
OpenCL: postpone OpenCL initialization on fork
OpenCL: xSHA512 lacks proper binary_hash()
OpenCL: use NULL for lws (raw-SHA256/512)
OpenCL: rawSHA256/512 review autotune procedure
OpenCL: comment about rawSHA256/512 rounds reversing
maint: fix an undefined-behavior
maint: fix DiskCryptor packing for MinGW
maint: fix an overflow in keepass2john.c
OpenCL: tune sha512crypt for Intel
OpenCL: fix AMD driver info
OpenCL: fix sha512crypt on Radeon RX Vega
OpenCL: tune sha512crypt for GTX 1080
OpenCL: tune sha256crypt for GTX 1080
OpenCL: tune raw-SHA256/512 on super
maint: make sha512_reverse() available for non-SIMD
OpenCL: refactor opencl_load_environment()
OpenCL: fix for a possible buffer overrun
OpenCL: refactor the gpu_device_list[] variable
OpenCL: refactor the device_list[] variable
OpenCL: use a constant as devices list terminator
OpenCL: use 1-based OpenCL device numbers
OpenCL: improve the --device error message
OpenCL: use a constant as "no GPU selected"
maint: drop the unit tests for a default make
maint: remove unit-tests on `make clean`
maint: build unit-tests on `make check`
maint: move LCM(a, b) to shared code
maint: stop using isspace() in common_opencl
tests: add unit-tests for the trim() functions
maint: remove debug code in VMware VMX
zip2john: it was not using the proper buffer
maint: clarify the error `fopen: $JOHN/john.conf`
maint: stop duplicating the path in path_expand_ex()
OpenCL: add const qualifier
maint: modify cfg_name inside cfg_init() cautiously
OpenCL: add const qualifier to SHA256crypt/SHA512crypt
docs: OpenCL fixes for 1-based OpenCL devices
maint: fix a warning in sunmd5 in gcc 9
 OpenCL: fix the device name not printed
OpenCL: mute a warning on Windows
OpenCL: autotune must set GWS as multiple of LWS
mask: free resources only after use
docs: add a recipe on how to build on Windows
Make the windows target the default when running on Windows (#3745)
docs: minor fixes on how to build on Windows
OpenCL: remove unused variable from vmx
docs: warn about unsupported OpenCL drivers
OpenCL: tune for the length a run will be using
OpenCL: tune for the length a run will be using
OpenCL: tune using code that matches benchmark more closely
docs: missing full JtR directory
fuzz: add const qualifier to fuzzing (#3820)
7z: stop to modify the caller's key in set_key()
OpenCL: rawSHA512 force auto-tuning
SIP: general improvements supplied by nanosonde (#3836)
SIP: fix for fails cracking qop=auth (#3840)
mask: add event to free (destroy) mask objects (#3843)
Handle GPU overheating gracefully (#3838)
packaging: upstream package stuff
hash_table: ocl_hc_128_prepare_table uninitialized allocation
OpenCL: print a message when a GPU wakes up from overheating (#3853)
OpenCL: rawSHA256 force auto-tuning (#3855)
Filter ztex formats when --format=cpu (#3854)
ztex: fix an overflow in sha256crypt (#3857)
OpenCL: if SleepOnTemperature = 1, print sleeping message once (#3861)
maint: move regex to experimental (#3860)
docs: drop non-updated jumbo-specific documentation files (#3865)
docs: cosmetic change in badges (#3866)
OpenCL: hash_ids uninitialized allocation (#3868)
docs: update the link for the Windows Build
CI: revamp all automated testing
CI: refactor and small tweaks
OpenCL: SHA256 and SHA512 force tune after fmt_done()
autotools: fix AVX512BW detection
autotools: fix AVX512BW detection
CI: run some tests only after merge
autotools: remove references of HT

Sayantan Datta (266):

DEScrypt OpenCL: Use a "safer" kernel for some devices (drivers) that
could not cope with the goto's.
mask.c: fixed stretch_mask buffer overflow bug
descrypt-opencl: limit max LWS to 64
descrypt-opencl: Limit LWS to 64.
descrypt-opencl: proper handling of cpu devices.
descrypt-opencl: Force LWS to 1 for cpu devices on AMD platform.
descrypt-opencl: remove stray output.
descrypt-opencl:set default LWS to 64 for Intel CPU on Apple.
relocate reset() after mask_init()
device mask
work-in-progress:mask mode internal candidate gen
work-in-progress:mask mode internal candidate gen: #2
work-in-progress:mask mode internal candidate gen: #3
work in progress: gpu mask commit 4
Raw-md4-opencl: partially working gpu password generation
Raw-md4-opencl: password generation on GPU(in alpha stage)
self-test: reworked
self-test tweak
Raw-md4-opencl:unified kernel for self-test, mask mode and other
modes.
Raw-md4-opencl: Fixed incorrect atomic update.
Raw-MD4-opencl: better get_key()
Raw-MD4-opencl: increased internal key count.
self-test: re-enable max-length test.
clean debuuging outputs.
fixed self-test for bfegg, HMAC-SHA1 and mysql.
Raw-MD4-opencl: reduce max kpc for mask mode.
fixed broken mas-mode
descrypt-opencl: reorganized code for new self test.
descrypt-opencl: properly release used kernels.
descrypt-opencl: leaner code
descrypt-opencl: better compare on gpu
descrypt-opencl: removed unnecessary code.
descrypt-opencl: better compare on gpu
descrypt-opencl: efficient hash loading on to GPU
descrypt-opencl:safer get_key, better hash loading code.
descrypt-opencl: efficient key transfer.
descrypt-opencl: remove unnecessary parameters form key struct
descrypt-opencl: faster set_salt()
descrypt-opencl: process keys only when it is changed
descrypt-opencl: seperate kernel file for finalize keys kernel
descrypt-opencl: process keys only when changed
descrypt-opencl: remove redundant and unnecessary code
descrypt-opencl: removed extra memory allocation
descrypt-opencl: consolidate redundant lines
descrypt-opencl: cleanup
descrypt-opencl: cpu-fix
descrypt-opencl: Successful kernel compilation on MIC(self-test still
fails).
descrypt-opencl: performance optimizations.
descrypt-opencl: save hardcoded kernels on disk and cache them
quickily on next run.
descrypt-opencl: hardcoded kernel Nvidia/Intel optimisations.
descrypt-opencl: passes -test=0
descrypt-openc: all kernels tests for max usable LWS.
descrypt-opencl: select kernels at runtime
descrypt-opencl: reduce static memory use.
descrypt-opencl: cosmetic changes
mask mode: Fixed memory corruption by GPU mask.
Avoid byte addressable store on AMD gpus.
Use private memory for Nvidia GPUs.
raw-md4-opencl:Use host defined macros in kernel.
Fix progress reporting when GPU mask is used.
raw-md4-opencl: fixed potential memory bug.
mask.c: fix template key buffer overrun.
Build tables.
raw-md4-opencl: prepare for new hash checking
raw-md4-opencl: new hash checking.
raw-md4-opencl: use perfect hash table.
raw-md4-opencl: optimize when gpu mask locatations are fixed.
raw-md4-opencl: bitmap performance test
build-table:fix openMP issues for older comilers.
raw-md4-opencl: fix issues with opencl 1.1
raw-md4-opencl: fix issues for Juniper.
device info: add check for gcn 1.0, 1.1 and 1.2
raw-md4-opencl: hash checking optimized upto 10k hashes.
raw-md4-opencl: finalize.
raw-md4-opencl:cosmetic changes.
raw-md4-opencl: reduce GPU memory use.
raw-md4-opencl: proper use of multi_profilingEvent[] and update war[].
raw-md5-opencl: new hash checking.
raw-md4/md5-opencl minor performance improvements.
raw-md4/5-opencl: remove leftover debug code.
raw-sha1-opencl: new hash checking.
raw-sha1-opencl: compute hash without changing key.
generate hash tables: suppress warnings.
raw-sha1-opencl: add license info.
raw-md4/md5/sha1 -opencl: bypass pinned memory allocation when unable
to allocate page locked memory.
build tables: use more latest openMP.
raw-md4/md5/sha1-opencl: fix constant cache use.
build_table:resolve memdbg issues.
fix memory leaks and memdbg issues with new hash checking stuff.
raw-md4/md5/sha1: fix bug where mem object is queried before it's
created.
nt-opencl: rearrange kernel.
nt-opencl: use key transfer similar to md4 kernel.
nt-opencl: gpu mask update.
new format:mscash-opencl and bug fix for other gpu mask enabled
formats.
mscash-opencl:gpu mask update.
gpu mask format use constant cache more aggressively.
mscash-opencl: pass parameter using kernel args instead of -D
mscash-opencl: remove unnecessary parameters.
mscash-opencl: new hash checking.
mscash-opencl: seperate create_clobject
opencl-mscash: new auto tune.
mscash-opencl: finalize autotune.
mscash-opencl: remove warnings.
opencl-mscash: fix multiple memory object release.
mscash-opencl: fix opencl compiler warning.
mscash-opencl: more stringent limit checks.
nt/raw-md4/md5/sha1-opencl: auto-tune update.
use better alternative to clFillBuffer conforming with opencl 1.0.
nt-opencl: bugfix.
lotus5-opencl: probably should fix cpu device errorsgit add
opencl_lotus5_fmt_plug.c
nt/raw-md4/md5/sha1-opencl: do not use local memory when device is
cpu.
loader.c: fix sequential_id calculation.
loader.c: additional checking for sequential_id calculation.
mscash2-opencl: new multi-gpu code.
mscash2-opencl: delete previous files.
mscash2-opencl: add copyright info and fix include braces.
mscash2-opencl: add opencl flags.
mscash2-opencl: improved multi-gpu scaling.
mscash2-opencl: autoTune chooses LWS more accurately.
mscash-opencl: fix valid bug.
Temporary Bug fix large hash collisions.
lm-opencl added.
lm-opencl: prepare for new auto_tune().
lm-opencl: cosmetic changes.
build-tables: bug fix for large hash collisions.
lm-opencl:new auto tune.
lm-opencl: device autotune.
lm-opencl: accurate LWS tune.
lm-opencl: auto tune finalize.
common-opencl: add missing macros.
lm-opencl: auto tune bugfix.
lm-opencl: more bug fixes and cleanup.
lm-opencl:verbosity control.
descrypt-opencl: better alternative to clearing buffers inside kernel.
bt(build_table):add 64bit hash type and qualified file names.
lm-opencl: new hash checking.
lm-opencl: use hash table.
lm-opencl: remove extra kernel parameters.
silence compiler warning for non-omp builds.
re-introduce changes removed by commit 5bad3f1a203
lm-opencl: add bitmaps for hash checking on GPU.
lm-opencl: bug fix.
lm-opencl: extract only required number of bits.
mask-mode resume problem bug fix.
lm-opencl:full unrolled kernel.
lm-opencl: faulty amd opencl compiler workaround for full unrolled
kernel.
common-gpu.c: fix memory leak.
mscash2-opencl:allocate memory only MAX_GPU_DEVICES.
mscash/nt/raw-md4/md5/sha1-opencl: auto tune bugfix.
mscash-opencl: many salt optimization and auto tune bugfix.
nt/raw-md4/md5/sha1-opencl: more auto tune bugfix.
lm-opencl: merge finalize keys with main kernel.
lm-opencl: finish merging finalize keys kernel with main kernel.
lm-opencl compiled only when opencl 1.2 build is detected
add ifdef CL_VERSION_1_2 before cheking its value.
lm-opencl: mask-mode test
lm-opencl:gpu side mask supports only one placeholder.
lm-opencl: gpu side mask bugfix.
lm-opencl:mask mode enabled.
lm-opencl: optimization when mask placeholder positions are static.
lm-opencl: use const cache to store gpu side mask keys.
lm-opencl: select full unroll kernel for gpus.
lm-opencl: patch missed with commit 04da2a
nt/raw-md4/md5/sh1-opencl: bugfix.
rebuild tables when 10 percent of hashes are cracked.
lm-opencl now conforms to CL 1.0 standards.
lm-opencl: remove remaining cl 1.2 stuffs.
lm-opencl: fix constant cache issue.
lm-opencl: add end of file LF.
lm-opencl:autotune bugfix.
lm-opencl: more autotune bugfixes.
lm-opencl: minimize use of global variables.
mscash2-opencl: init kernels and autotune in reset.
lm-opencl: 40 percent performance improvent on AMD GPUs with cat 15.5.
lm-opencl: do not release uninitialized buffers.
mscash2-opencl: do not release uninitialized buffers.
lotus5-opencl: more stringent limit checking.
lotus5-opencl: bugfix, doesn't fix #1505.
lotus5-opencl: possibly fix #1505
lotus5-opencl: some optimizations and cosmetic changes.
added new format nsldap-opencl.
descrypt-opencl: optimizations.
prepare to adjust nsldap with raw-sha1-opencl.
mask-mode: ?W bug-fix. It didn't update mask_num_qw used by other
modes.
mask.c: fix when mask contains characters > than 0x7f.
added new format: salted-sha1-opencl.
salted-sha1-opencl: bugfix in loading hashes.
build tables: signal handler bugfix.
mask.c: fix bug in truncate_mask/save_restore.
descrypt-opencl: changes required to support better auto-tuning.
descrypt-opencl: more changes.
descrypt-opencl: new host code
descrypt-opencl: more leaner code.
descrypt-opencl: code cleanup
descrypt-opencl: seperate kernel for hash checking.
descrypt-opencl: prepare for new hash checking.
descrypt-opencl: bugfix.
descrypt-opencl: merge functions.
descrypt-opencl: unify hash checking for all kernels.
descrypt-opencl: put common functions in separate file.
descrypt-opencl: common hash checking for all kernels.
descrypt-opencl: cosmetic changes.
descrypt-opencl: new hash checking.
descrypt-opencl: new hash checking.
descrypt-opencl: support for very large number of hashes.
descrypt-opencl: added bitmaps for hash checking.
descrypt-opencl: put key processing in common file.
descrypt-opencl: restrict frequent buffer updates.
descrypt-opencl: mask-mode init.
descrypt-opencl: support more placeholders on GPU side.
descrypt-openc: non-static gpu side mask bugfix.
descrypt-opencl: correctly report c/s rate.
descrypt-opencl: suppor mask mode for all kernels.
descrypt-opencl: use constant cache if possible.
descrypt-opencl: get key mask mode.
descrypt-opencl: build kernels in reset instead of crypt_all
descrypt-opencl: more compact code.
descrypt-opencl: use -D build opts to hardcode salt.
descrypt-opencl: rearrange kernels.
descrypt-opencl: improve basic kernel.
descrypt/lm-opencl: bugfix and other modifications to support auto
tune.
descrypt-opencl:auto-tune for fully unrolled kernel.
descrypt-opencl: save LWS parameters.
descrypt-opencl: re auto-tune only when using meask mode..
descrypt-opencl: use device name instead of gpu_id and include all
kernels to determine lws limit.
descrypt-opencl: new auto-tune bugfix.
descryp-opencl: hardcode only kernel autotune.
descrypt-opencl: prepare basic kernel for auto-tune.
descrypt-opencl: auto-tune for basic kernel.
descrypt-opencl: get key bug fix.
descrypt-opencl: set hash_ids[0] to 0 before self test.
descrypt-opencl: more auto-tune bugfixes.
descrypt-opencl: code cleanup.
descrypt-opencl: remove stary output.
descrypt-opencl: format details bugfix.
descrypt-opencl: basic kernel tuning.
descrypt-opencl: bugfix.
descrypt-opencl: fine tune hardcoded and fully unrolled kernels.
descrypt-opencl: auto-select suitable kernel.
descrypt-opencl: match suitable kernels with devices.
update amd device codenames.
 lm-opencl: add more debug code.
lm-opencl: limit bug fix.
descrypt-opencl: use basic kernel for all apple.
add FMT_REMOVE parameter for formats that access the salt->list during
cracking.
lotus5-opencl: fix possible issues.
descrypt-opencl: implement FMT_REMOVE.
build tables: throw overflow warning instead of error and show file
and line.
build-tables: reduce memory consumption for upto 50 mil hashes.
descrypt-opencl: fix auto-tune for non-amd/nvidia gpu.
thread safe opencl_build() and opencl_build_from_binary() assuming
certain conditions.
fix issues missed in 0dbaf3a5bacbd34.
opencl_read_source is now thread safe.
descrypt-opencl: bugfix.
descrypt-opencl: fix thread safety issues for parallel build.
descrypt-opencl: parallel build.
descrypt-opencl: parallel build bugfix.
descrypt-opencl: make enc_salt thread safe.
descrypt-opencl: remove all critical sections.
unify hash checking for rawmd4/rawmd5/nt-opencl.
bugfix for last commit.
put create/release object in common file for 128 bit hash checking.
use qualified names to prevent name clashes.
more common functions for 128bit hash checking.
reduce number of global extern variables for common 128bit hash
checking.
bugfix for d2a0668

Frank Dittrich (248):

korelogic.conf: fix month name in [List.Rules:MonthsFullPreface]
korelogic.conf: split some rule definitions for dupe suppression
submodule update
opencl_sxc_fmt_plug.c apply Alexander Cherepanov's sxc format fixes
from 2013 to the opencl implementation (improved valid()) Without
these fixes I was able to trigger segfaults with fuzzed input files.
asan documentation fix
office_common_plug.c: add a comment from the CPU version that got lost
when separating code that is common for CPU/opencl
Add tests formatted in exactly the same way as the hashes in john.pot
May be I'm too paranoid, but I wanted to make sure each format
"understands" the hash format it uses for john.pot
osc format: use a hash without trailing spaces for the canonical hash
representation ($dynamic_4$) that is used in john.pot. The original
hash caused trouble because of the trailing spaces.
dynamic: use $HEX$ for ciphertexts ending with ' ' or '\t' when
storing them in the pot file.
sha512crypt: unify test hashes for CPU and GPU formats
osc format: add one more self test
sha256crypt: unify test hashes for CPU and GPU formats
relbench: avoid "Use of my $_ is experimental at ./relbench line 82."
Bash completion for file name in --prince=*, fixes #1020
john.bash_completion: fix typo in a comment.
bash completion: enable the --prince=FILE workaround for -prince=
Fix bash completion for --format=
siemens-s7: unify hash "versions" 0 and 1
Revert "siemens-s7: unify hash "versions" 0 and 1"
Revert "Revert "siemens-s7: unify hash "versions" 0 and 1""
 siemenss-s7: valid() did still accept invalid hashes like
$siemens-s7$
$1$599fe00cdb61f76cc6e949162f22c95943468acb$002e45951f62602b2f5d15df217f49da2f5379c
b
instead of
$siemens-
s7$1$599fe00cdb61f76cc6e949162f22c95943468acb$002e45951f62602b2f5d15df217f49da2f537
9cb
gpg and gpg-opencl: rename 1st tunable cost, iteration count ->
s2k-count
Fix legacy builds for 32bit Linux
Fix oracle format breakage caused by commit
eaaf00625c2af421736b1e69dfaad3939cd4fb0f cmp_one cannot be a dummy
function (returning 1) if cmp_exact is a dummy.
Fix legacy build after some plugins have been converted into
non-plugins
Makefile.legacy: change object file name sequence to match Makefile.in
and keep the core john files at the top of the list
Corrext a stupid typo I made in commit 682d60e, fixing legacy builds
Avoid some scan-build false positives by adding
__attribute__((__noreturn__))
misc.h: reformat changes made in commit
05754f018aa785a2f646861be38556ec7cd3ad65
Sybase-PROP format: fix undefined behavior (signed integer overflow)
benchmark-unify: sort -f of mappings (for better readability)
Fix memory leaks for ./john --test=0 --format=tc_*
Fix memory leaks for ./john --test=0 --format=Snefru-128
Fix Memory leaks for ./john --test=0 --format=HAVAL-128-4
Fix memory leaks for ./john --test --format=whirlpool*
Fix a problem with "salt dupe checks" for unsalted formats. Closes
#1229.
benchmark-unify: add mapping NT-old -> NT
Report glibc version with --list=build-info
listconf.c: define _BSD_SOURCE and _DEFAULT_SOURCE for OpenCl builds
Fix some memory leaks for ./john --status[=NAME]
Add build timestamp to --list=build-info
Add Microsoft compiler version info to --list=build-info
Revert second part of commit 03404dfd88c6fcc8aa8fa7dd31b72bbc060cc200.
Otherwise, --list=build-info would would produce a segfault.
Report glibc version with --list=build-info
WoWSRP format: fix memory leaks (GMP) detected by ASan
Clipperz format: fix memory leaks (GMP) detected by ASan
john.c: replace exit(0) with error() for invalid encoding options in
john_load_conf_db() and in john_init()
opencl_rawmd4_fmt_plug.c: replace exit(0) with error() in case of
error
regex.c: replace exit(0) with error() in case of error
pkzip_fmt_plug.c: Fix issue #1232 (signed int overflow) for real
pkzip_fmt_plug.c: fix -fsanitize=undefined runtime errors
For FMT_NOT_EXACT, printwarning for main process only
Improve hdaa format's valid(). Closes #1262
Fix scrypt's prepare(), dont pass NULL pointers into
base64_valid_length Closes #1259
Fix scrypt's prepare(), dont pass NULL pointers into isdec()
pfx format valid(): call atoi(p) after isdec(p), avoid undefined
behavior
Fix "binary() returned misaligned pointer" for HMAC-SHA384 (32bit
Linux clang)
Fix "binary() returned misaligned pointer" for HMAC-SHA512 (32bit
 Linux clang)
keepass_fmt_plug.c: avoid "Warning: salt() returned misaligned pointer
pbkdf2-hmac-md5_fmt_plug.c: Fix reporting of tunable cost (iteration
count)
pbkdf2-hmac-md5_fmt_plug.c: fix syntax error caused by typo
bsdicrypt: report iteration count as tunable cost
Fix bash completion for --show= after --show=types had been added
For --regex=alpha, '.' and '?' need to be escaped
regex.c: escape backslash inside words.
truecrypt_fmt_plug.c: fix memory leak for ./john --test=0
--format=tc_*
truecrypt_fmt_plug.c: fix whitespace error
ssh_fmt_plug.c: fix memory leak in valid(), fixes #1247
Revert "ssh_fmt_plug.c: fix memory leak in valid(), fixes #1247"
Fix segfault in dyna_salt_remove_fp() when called from valid()
Revert "Revert "ssh_fmt_plug.c: fix memory leak in valid(), fixes
#1247""
mkvlib.c: improved sanity checking of Markov stats file, fixes #1467
mkvlib.c: adjusted sanity checking of Markov stats file
Add version.h.new to .gitignore and `make distclean`
detect.c: fix for `make -s -f Makefile.legacy generic
Remove detect.o build target from Makefile.in It is not needed for
autoconf builds.
unrarhlp.c: include "jumbo.h" to fix legacy build
genmkvpwd.c: inclcude "jumbo.h" to fix legacy build errors
wpapcap2john.c: include "jumbo.h" to fix legacy build
wordlist.c: fix legacy build warning
pp.c: fix error: assignment to expression with array type which, for
some reason, only occurred in legacy builds
john.c: fix legacy build warning
Makefile.legacy: add the JTR_GIT_VERSION logic
Makefile.legacy: `make clean` now removes version.h and version.h.new
XSHA512_fmt_plug.c: fix "warning: unused variable" for legacy generic
build
listconf.c: fix "warning: unused variable" for legacy generic build
Enable OpenVMS format for legacy little endian legac builds
Enable pkzip format for legacy builds
Makefile.legacy: fix librexgen detection (new path of librexgen.h)
Replace use of rm and mv in Makefiles with $(RM) and $(MV)
Adjust rexgen build instructions
options.c: avoid warning for legacy opencl builds
mscash2_fmt_plug.c: fix memory leak Found when testing ./john --fuzz
from https://github.com/loverszhaokai/JohnTheRipper/tree/fuzz_option
with an --enable-asan build
Add --enable-ubsan (UndefinedBehaviorSanitizer)
Makefile.legacy: fix -j handling for generic (best.sh) and other
targets
Finally fix memory leak in ssh format (get_salt) #1247 for real
listconf.c: add UBSAN_STRING to --list=build-info output
RAR3 (CPU and OpenCL): warn just once for pack_size or unp_size > 1 TB
(helps reducing noise when fuzzing these formats)
gpg-opencl: warn just once about currently unsupported hash algorithm
(helps reducing noise when fuzzing this format)
Add UBSAN_STRING to john's usage output
Replace "-autoconf" with "-ac" in JOHN_BLD definition
Adjust scope of some recently introduced warn_once variables
mozilla format: improve valid()
dmg format: improve valid()
vtp_fmt_plug.c: improve valid()
7z_fmt_plug.c: improve valid()
encfs_common_plug.c: improve encfs_common_valid()
gpg_fmt_plug.c and opencl_gpg_fmt_plug.c: improve valid()
opencl_7z_fmt_plug.c: apply recent CPU format fixes to OpenCL format
Improve valid() in agilekeychain and agilekeychain-opencl
Add run/custom.chr to .gitignore, fixes #1501
opencl_rar_fmt_plug.c: Fix valid() after ishex() changes
blockchain_fmt_plug.c and opencl_blockchain_fmt_plug.c: improve
valid()
cloudkeychain_fmt_plug.c: improve valid(), fix undefined behavior
androidfde_fmt_plug.c: improve valid(), fix undefined behavior
keystore_fmt_plug.c: improve valid(), avoid undefined behavior
kwallet_fmt_plug.c: improve valid()
putty_fmt_plug.c: improve valid()
rar5_common.h: improve valid()
ssh_ng_fmt_plug.c: improve valid()
bitcoin_fmt_plug.c: improve valid()
chap_fmt_plug.c: improve valid(), avoid undefined behavior
improved valid() for pwsafe (CPU/GPU), avoid undefined behavior
django_fmt_plug.c: improve valid(), avoid undefined behavior
efs_fmt_plug.c: improve valid()
eigrp_fmt_plug.c: avoid undefined behavior in valid()
Makefile.in: specify path for dynamic_big_crypt_generator.sh avoiding
this error: /bin/sh: dynamic_big_crypt_generator.sh: command not
found
bench.c: fix segfault introduced in commit ac6a0ea
keepass_fmt_plug.c: improve valid(), avoid undefined behavior
Fix segfault for ./john --list=subformats caused by commit ac6a0ea
Generate more test output for `make check`, for Travis build clues
Makefile.legagy: avoid --make_check usage because it fails in jumbo
More undefined behavior fixes in keystore format's valid()
LUKS format: improve valid(), avoid undefined behavior
mongodb_fmt_plug.c: improve valid(), avoid undefined behavior
office_common_plug.c: improve valid(), avoid undefined behavior
openbsdsoftraid_fmt_plug.c: improve valid()
wbb3_fmt_plug.c: avoid undefined behavior in valid()
add run/dynacomptest to .gitignore
With isdec(p), atoi(p) will not return values < 0, so drop some tests
Makefile.legacy: define HAVE_LIBSSL While legacy builds always used
-lssl, autofonf changes made including openssl includes depend on
HAVE_LIBSSL. This should also fix
https://github.com/openwall/john/issues/1526#issuecomment-120008446
Fix comments in ike-crack.h: s/HAVE_OPENSSL/HAVE_LIBSSL/
sapH_fmt_plug.c: avoid memleaks on invalid hashes
Adjust sap2john.pl comments
Make dyna-compiler-test.sh work with dash and some cleanup
.gitignore: add some dyna-compiler-test.sh temp. files that might
remain when dyna-compiler-test.sh was interrupted.
Bash completion: complete --regex= to --regex=alpha: and provide
completion of --regex=alpha: (using the existing
[List.Rexgen.Alpha:*] section names.
regex.c: make debug output depend on #if DEBUG and use stderr instead
of stdout.
Move "plaintext" format further down the list of formats
Move dummy format down the list
Makefile.legacy: repeat the Makefile change done in af07c69
Makefile.legacy: fix for opencl builds
options.c: Fix compiler warning for legacy opencl builds
doc/MASK: improve ?W example to make it clear that it toggles case
sapH format: set FMT_CASE and FMT_UTF8 flags
Fixes in comments: s/definded/defined/, s/defind/defined/
Bash completion: For --restore= and status=, ignore .[1-9][0-9].rec
Fix build without plugins (renamed some Keccak*_plug.c files)
Makefile.legacy failed after making some Keccak* iles non-plugins
dynamic_compiler.c: avoid segfault for expressions that neeed user
name or constants, but no salt. Example:
--format=dynamic='md5(lc($u).$c1.lc($p)),c1=::'
Add a SingleSkipLogin config variable
fuzz.c: fix a copy&paste error introduced in aeff562
Add LogFileProtect config variable
logger.c: remove extra empty line introduced in previous commit
bash completion for --show=invalid
blake2b-ref_plug.c: drop unused blake2b_clear_lastnode()
Fixes for legacy generic builds (make -f Makefile.legacy generic)
options.c: Fix compiler warning for legacy non-opencl builds
regex.c: fix a few remaining memleaks
include "misc.h" in several files using error()
skip john_register_all() for ./john --stdout
regex.h: check for rexgen version >= 1.3 instead of >= 1.1
doc/MARKOV: Don't abbreviate option names
doc/RULES: some minor adjustments
doc/DYNAMIC* and doc/MARKOV: use utf-8 instead of utf8 as encoding
name
doc/EXTERNAL: don't abbreviate --max-length and --max-length
Document that --min-length and --max-length values mean bytes, not
characters.
doc/DYNAMIC: Fix typos introduced in bec0a5
doc/ENCODINGS: avoid abbreviated option names and use = instead of :
to separate option names and values Also, use --internal-codepage
instead of the deprecated --internal-encoding.
doc/NETNTLM_README: use --format= instead of -format: and $ instead of
% as a command prompt
doc/PRINCE: Don'r abbreviate option names and s/crash-cource/crash
course/
doc/DYNAMIC_SCRIPTING: adjust format options
Fix configure breakage (UbSan) caused by commit 0fd8af10
Warn about disabled Stribog formats due to missing SSE 4.1
stribog_fmt_plug.c: One warning instead of three for missing SSE 4.1
Avoid clang wanrings for legacy builds
Bash completion: basic support for dynamic expressions
(--format=dynamic=...)
Bash completon: support completion for --fuzz=
options.c: upper case placeholders (--fuzz-dump[=FROM,TO])
fuzz.c: adjust screen output for --fuzz-dump
fuzz.c: add file size to --fuzz-dump option's screen output
fuzz.c: flush stdout before generating file.
chmod a-x src/escrypt/scrypt_platform.c
HDAA_fmt_plug.c: s/mem_calloc_tiny/mem_alloc_tiny/ because of memset
office_common_plug.c: s/mem_calloc_tiny/mem_alloc_tiny/ because of
memset
ssh_fmt_plug.c: drop memset after mem_calloc_tiny
Fix syntax error introduced in
e75c71c4056c5cc2ebcbc8283604e5839045fa11 and avoid some warnings
(-Wdeclaration-after-statement and "comparison of distinct pointer
types")
Two more instances od s/equivelent/equivalent/
doc/DYNAMIC_EXPRESSIONS s/--format=/--format=dynamic=/
doc/OPTIONS: document the meaning of negative --max-run-time values
 Fix my typo in doc/OPTIONS
/qnx_common.h: avoid segfault in hash validation, fixes #2165
dynamic_parser.c: Avoid segfault
./john --list=format-methods: call fmt_done() for all formats
Revert "pem: bug in valid, which recent changes to hexlen function
made show up"
PEM format: avoid undefined behaviour of atoi()
PEM: There were some more atoi() calls which needed an isdec() check
KeePass: improve valid() to avoid ASan errors, see #2239
pem_fmt_plug.c: Add isdec() check for number of iterations
KeePass: Instead of error(), just tread non-inlined data as invalid
ssh-ng: reject unsupported ciphers as invalid
Bash completion: add completion for --fuzz-dump=
qnx: improve valid() to avoid segfaults
qnx: improve valid(), don't allow additional "fields" at end of hash
fuzz.c: add 8bit replacement character, add @ to list of separators
base64_convert.c: fix memory leak when using -r and -w options
Some more changes due to oclHashcat/HashCat being renamed to hashcat
Spelling and grammar fixes in documentation files, comments, and
string literals
Remove all references of CUDA from doc/BUGS
Increase Raw-SHA1-Linkedin's PLAINTEXT_LENGTH for non-SIMD builds
(#2530)
Add some known hash collisions to sapB format tests
Fix for make -f Makefile.legacy clean
Fixes for make -f Makefile.legacy generic
Legacy generic build: Test BF_X2 = 3 if BF_X2 = 1 was faster than
BF_X2 = 0
Legacy builds: adjust plugin format sequence (match autoconf seq.)
remove john.local.conf from Makefile.legacy
Makefile.legacy: fix dependencies
Makefile.in: fix dependencies
Reduce sizeof(saved_key) to PLAINTEXT_LENGTH + 1
Enhance Argon2 tunable cost reporting (#2742)
Enhance and unify tunable cost descriptions (#2745)
./john --list=ext-modes should only list standalone external modes
(#2775)
Add missing split() functions to convert hashes into canonical form
Format test: split() must add format tag to raw hashes
Cleanup binary() for some more formats
Fix bash completion for ./john --show=types-json
Ignore some files/directories when creating archives
Unify john version string logic, based on ideas of Claudio André
Makefile.in: workaround vim's broken syntax highlighting
Fix legacy build warnings introduced with commit 136a3e8
rar_common.c: Fix warning for !HAVE_MMAP case
wpapsk/wpapsk-pmk: add 802.11w. support for legacy builds for CPU
formats
Unify version strings for legacy builds
Remove duplicate #include preprocessor directives
Fix legacy generic build
Avoid strcasestr -Wimplicit-function-declaration for legacy builds
relbench: fix benchmark output parsing for changes made in commit
7187208
benchmark-unify: adjust mappings to current format names

Zhang Lei (108):

Resolve issues encountered when building for MIC.
Patches needed to build some libraries for MIC.
Update mpz_int128.c to refer to the bug of icc
Restore the lost two lines in config.sub
Use tab instead of spaces as indentation.
Put variable binding after './configure'
some fixes in the building system
syntax correction
OMP_SCALE tuned for MIC
Correct some indentations
some bug fixes
revert a mistaken fix
Add special compiler flags for MIC
use LibreSSL instead of OpenSSL for MIC
fix MIC intrinsics in sha256-ng & sha512-ng
distinguish the SIMD_TYPE name of MIC from AVX512
remove some trailing spaces
Add MIC support in raw-sha1-ng
Remove some trailing spaces
some minor fixes for MIC
fix sha256crypt & sha512crypt for MIC
fix other non-dynamic formats on MIC
minor error fix
further tune OMP_SCALE with MIC
Use gather instruction to optimize some code
Refactor code that uses __AVX512__
Some optimization with gather/scatter
append 'ULL' to some int64 literals
tune OMP_SCALE in SybaseASE
unify the interface of vcmpeq_epi8
fixing typos
some small fixes
several bug fixes for _mm512_cmpeq_epi8
remove the emulation of _mm512_cmp_epi8_mask
wordlist.c: a minor fix
mic.h: tune interleaving factors
add OpenMP support to sunmd5
sunmd5: simplify the OpenMP clause
sunmd5: rearrange code for OpenMP support
sunmd5: include <omp.h>
sunmd5: add OMP_SCALE
add interleaving to SHA512
Add interleaving to SHA256
fix interleaving in SHA512
set SIMD_PARA_SHA256/512 to 1
sse-intrinsics.c: fix a few indentations
mic.h: add missing help macros
fix interleaving bug in sapH
sse-intrinsics.c: use scatter instruction for AVX512 & MIC
sse-intrinsics.c: a few fixes for the use of scatter
README-MIC: add instructions for building OpenSSL
mic.h: conditionally define interleaving paras
configure: no need to set LIBS for MIC
tune a bunch of OMP_SCALEs for MIC
pbkdf2_hmac_sha1.h: disable vectorization of a loop
byteorder.h: replace strange int types with normal ones
fix configure.ac for 64-bit Power
Add AltiVec intrinsics support.
Add AltiVec intrinsics
ppc64.h: a few minor fixes
sse-intrinsics: clean up a few temporary changes
 A few fixes for AltiVec intrinsics
pseudo-intrinsics: a few optimizations for AltiVec
pseudo_intrinsics.h: fix errors in last commit
pseudo_intrinsics.h: fix some indentations
fix an error introduced in commit 9783278
pseudo_intrinsics.h: optimize vtesteq & vtestz
dynamic: hopefully fixed MD4 problems on MIC. #1437
ignore *.dSYM files on OS X
rename sse-intrinsics to simd-intrinsics
mic.h: tune SIMD_PARA_MD4 & SIMD_PARA_MD5
Add ARM NEON intrinsics (untested).
Refactor pseudo_intrinsics.h
Add further support for NEON
Polish NEON's int64 intrinsics
Fix a few intrinsics for NEON
Include simd-intrinsics.o in a ARM build
Use NEON's vbsl to implement vcmov
Disable raw-sha1-ng for ARM due to a compiler bug
Optimize vroti implementation for NEON
mic.h: tune SIMD_PARA_MD4 & SIMD_PARA_MD5
Use SIMD SHA1/2 functions for episerver
Fix bugs in episerver
Add UTF8 support to episerver's SIMD build
Fix a bug in episerver
Add SIMD support to 7z
Revise ALGORITHM_NAME of episerver
7z: optimize SIMD performance
7z: optimize SIMD performance
7z: fix a bug that makes trouble when para > 1
7z: fix ALGORITHM_NAME
7z: cleanup after merging in magnum's code
7z: fix a bug introduced when merging code
Avoid unaligned access for some archs
7z: fix a bug in SIMD code
7z: alter PLAINTEXT_LENGTH for SIMD build
Avoid strcpy if key is already aligned in set_key()
Replace some `uint64_t`s with `ARCH_WORD_64`s
Add SIMD support to rar
7z & rar: use global array as vector buffer
Optimize MD5_I() for NEON and AVX512
Introduce more ternary logic instructions
Fix indentations in the previous commit
Fix a few untested code for AVX512
Use AVX512's ternlog instruction to emulate vcmov
Tune interleaving paras for ARM
Rename vtesteq_epi32 to vanyeq_epi32
Unify the format of NEON and Altivec's intrinsics.

Kai Zhao (84):

Update plugin.c
change 'int count = *pcount;' to 'const int count = *pcount;'
add check for hex string and lower case
Add siemens-s7 hash check in valid(). See #1094
Add check for get_binary()
Update siemens-s7_fmt_plug.c
fix valid bug for 7z_fmt_plug.c
Dominosec: Fix a typo leading to slight over-allocation.
Update kwallet_fmt_plug.c
Update mongodb_fmt_plug.c
Update nukedclan_fmt_plug.c
Must check cfg_database->list before cfg_add_line()
fix bug of androidfde_fmt_plug::valid()
fix valid() function in src/blackberry_ES10_fmt_plug.c
fix get_salt() in cq_fmt_plug.c
fix segfault: --field-separator-char=c
fix segfault: --markov::default:
report error message and exit when --input-encoding= a long string
Return CP_UNDEF when NULL == encoding
remove the useless parameter for sevenzip_decrypt() function
remove additional blanks
Fix spell mistake
Close #1242
Close 1230
StripZeros: Copy as much as ct2_size to ct2 to avoid buffer overflow
Fix SSH Format get_salt()
Fix SKEY Format get_binary()
Fix: 32bit build with GMP fails
Fix: RAR5 common valid(), change atoi to strtol
update coding style
Fix: ssh2john double free or corruption
Fix: ssh2john.c double free
Fix: gpg2john.c buffer over flow
Fix: gpg2john global-buffer-overflow
Fix: gpg2john global-buffer-overflow
Fix: gpg2john global-buffer-overflow
Fix: keepass2john heap-buffer-overflow
Remove assert(count == 1) by check the return of fread()
Check the buffer size before fread()
Check the buffer size before fread(): keepass2john
Fix: keyring2john remove assert by checking the return value
Fix: keystore2john remove assert and check buffer size of fread()
kwallet2john.c: Check the return value of fread()
luks2john.c: Check return value of jtr_fopen()
pwsafe2john.c: Remove assert by checking the return value of fread()
pwsafe2john.c: Remove unused variable
rar2john.c: Remove assert and check the buffer size before fread()
Use header.caplen instead of header.len
Use: warn_exit("%s: Error: read failed: %s.", argv[0],
strerror(errno));
Remove build warning
Fix: wpapcap2john.c buffer overflow
Use printf("%zu") to print size_t
Report error if config file does not exist when '--config=somefile'
Fix spell mistake, there is no BF_common_plug.c
Fix misspelled words
Fix typo: initailize -> initialize
loader.c: fix segfault introduced in commit ac6a0ea
Implement --fuzz options to fuzz formats' functions
Add doc/README.coding-style. See #350 and #1567
Add coding stlye: Add a blank line after variables declaration
Add --test-full option which crack max passwords at the same time and
can detect FMT_* flags
Remove build warning. Close #1605
1. Add FMT_8_BIT: has-160, pomelo, pufferfish, Stribog-256 2. Remove
FMT_8_BIT: VNC 3. Do not check FMT_8_BIT of crypt
1. Add a new test vector for stribog-512 2. Add FMT_8_BIT flag for
stribog-512
Fix test_fmt_8_bit() to handle all the cases
 Add more test vectors for skein-256 and skein-512
Add test vector for vms
Move the new test vector to the end
Add comments for vms
Fix: is_key_right() ignores the result of get_key() when invoked by
test_fmt_case()
Fix the detects of FMT_8_BIT for crypt and wpapsk
Fix fmt_self_test_full_body() for dominosec8 and one grammatical
mistake
Add FLG_TEST_FULL_CHK in some cases which need FLG_TEST_CHK
Add FMT_8_BIT for dominosec8
Change s_size to err_buf in is_key_right() and fmt_self_test_full_body
of formats.c
Add FMT_OMP_BAD and FAST_FORMATS_OMP
Add test vectors for agilekeychain and lp
Add test vectors
Add test vectors
Add test vectors
Fix keyring crypt_all() which causes the cmp_all() always return 1
Fix the crypt_all() of vnc
Detect FMT_SPLIT_UNIFIES_CASE error
Change to 'FAILED' when there is a FMT_SPLIT warning or error

Solar (75):
Fix a bug in truecrypt for non-x86 archs. Closes #912.
Early (contest) version of AVX2 support for DES.
Fix a compile error in dynamic_compiler_plug.c
Relbench: Fix over/underflow when calculating geometric mean.
Fix benchmark_length for cq format
Fix OpenMP performance issue in Sybase-PROP. Closes #1673
OpenCL md5crypt: Optimizations.
OpenCL auto-tune improvement: Re-tune GWS with the tuned LWS and
considering the queried number of compute units.
Fix the "max. length in index 1: wrote 55, got 55 back" self-test
error seen for raw SHA2 formats after auto-tuning.
Intrinsics: Use vcmov (if not emulated) for MD5_I and MD4_G. Disable
support for SSEi_FLAT_OUT for MD4 & MD5 (currently not used anyway).
Optimize OpenCL MD4_G.
md5crypt intrinsics: Optimization for mmxput3().
md5crypt intrinsics: Use switch in mmxput3 (unless XOP).
Some fixes for huge input lists.
http://www.openwall.com/lists/john-dev/2015/09/15/5
Avoid ldr_remove_marked() when there's nothing marked for removal.
Implemented better prefetching, john-huge-prefetch.diff
john-huge-largehash.diff: Prepare us for larger bitmaps and possibly
hash tables and cleaning up our use of integer types a little bit in
general.
john-huge-loader-bypass.diff: Loader optimzations.
john-huge-loader-mt.diff, splitting initialization of bitmap and hash
table into 3 threads.
raw-md5: Optimize valid() and split().
Rules: Optimizations for repeated simple commands.
Correct an allocation bug for the "ShowUIDinCracks" feature.
john-loader-mem-save.diff
john-loader-pw_size.diff: "struct db_password" allocation.
john-huge-prefetch2.diff: Prefetch tweaks.
Bump unique's default buffer size.
Speed up --show.
john-huge-show2.diff + errata from
 http://www.openwall.com/lists/john-dev/2015/09/20/3
john-huge-prefetch3.diff: Fix problems with prefetch. Closes #1771.
Spelling fix in a comment: equivelent to equivalent
Assorted spelling fixes
Fix for argon2. Closes #2650.
Fix several issues with OpenMP in multibit format. See #2846.
fuzz: suppress collision warnings
md5crypt-opencl optimizations favoring same-length passwords
md5crypt-opencl: choose slow path on len >= 10*4*8 instead of on x[9]
md5crypt-opencl: workaround AMDGPU-PRO libamdocl64.so segfault
Pack altpos and ctx_buflen into 3 registers
Enable explicit use of bitselect on AMD and of LUT3 on recent NVIDIA
Further specialization of the MD5 compression function
Keep the MD5 result in registers on every other iteration
Disable specialization to password length if it varies within the
group
md5crypt-opencl: Further specialization for password length 8
md5crypt-opencl: only detect divergence in 64 lid's near current
Add pristine upstream yescrypt
Force SKIP_MEMZERO for use in JtR
Upgrade from escrypt to yescrypt
Added ULL to the 64-bit YESCRYPT_ROM_TAG* constants
Updated copyrights, dropped obsolete comment in scrypt_fmt.c
Fix some unaligned accesses, enhance some salt_hash()
Have most "Raw" benchmarks of salted formats behave like "Many salts"
Documented the benchmark_length flags
Make "gost" format's "Raw" benchmark behave like "Only one salt"
Added the missing #include "arch.h" to "haval" to let it use OpenMP
radius: just output some recovered passwords when we can (#3862)
yescrypt: Explicitly use huge pages starting with 12 MiB, not 32 MiB
JtR hack: don't use OpenMP inside (ye)scrypt
scrypt: Support *_keys_per_crypt above thread count
Use yescrypt_kdf() instead of crypto_scrypt()
Fix many format-specific benchmark oddities (issue #3795, PR #3873)
aix-smd5: Trivial optimizations (#3876)
Workaround gcc 9 change for const vs. OpenMP shared() (#3884)
eapmd5tojohn: Use proper signal handler prototypes
[skip ci] Add usage examples to doc/MASK
ztex: suppress "Many salts" benchmarks (#3899)
Pass correct salt into crypt_all() in benchmarks (#3898)
[skip ci] Update README*
Fix "make -f Makefile.legacy generic" again, was broken by #3894
(#3914)
Update NEWS: mostly adjust magnum's added credits/years for style
(#3913)
Don't report virtual c/s if virtual time didn't advance (#3926)
md5crypt-long optimizations (#3930)
doc/RULES-hashcat corrections
Update more mentions of unreleased 1.8.0-jumbo-2 to 1.9.0-jumbo-1
OpenBSD-SoftRAID: drop benchmark comment (have tunable costs), limit
to first test vector
Re-enable 2x interleaving for SHA-1 on x86-64 XOP
doc/NEWS update (#3951)

Apingis (58):
descrypt-ztex format; includes library for operating ZTEX hardware and
bitstream (4MB). Requires libusb-1.0. Autoconf support.
HDL sources; Copyright and license information; small corrections
Improvements in ztex library (#2577)
ztex-bcrypt format (#2582)
error introduced in online conflict resolution - corrected (#2584)
bugfixes in ztex-bcrypt trigger another build attempt by CI.
Built new descrypt-ztex bitstream from existing design. - Added
programmable clocks - Minor improvements in the design - Fully
compartible with previous version
Added FMT_TRUNC
Rename for more consistent directory names; added
fpga-descrypt/README.md and summary
fpga-descrypt\ztex_inouttraffic_summary.html
Оn bad input packet, board correctly removed from operation.
If the only board fails, don't exit, wait until it's up
Ztex improve (#2711)
bugfix in bcrypt-ztex bitstream; $JOHN; --devices (#2756)
ztex-improve (#2760)
Fix typos (#2762)
Removed limit of (2^18-4) keys/board per crypt_all() (#2766)
doc/README-ZTEX; Simulation testbench cleaned up and extended (#2803)
sha512crypt-ztex format (incl. Verilog sources) (#3287)
Create README.md (#3288)
ztex formats with --test fix (#3294)
ztex_drupal7.c based on unified sha512crypt+Drupal7 bitstream (#3314)
FMT_TRUNC corrected (#3318)
firmware upload on a board with user-defined SN (#3321)
Separate keys_per_crypt setting for self-test (#3323)
docs update (#3320)
sha256crypt-ztex (#3374)
Improved ztex documentation (#3376)
Improved ztex documentation - 2 (#3377)
md5crypt-ztex, phpass-ztex
improvements to md5crypt-ztex, phpass-ztex (#3421)
handle limit on input (to the board) packet length; minor corrections
(#3422)
Checked, corrected occurences of PKT_MAX_LEN (#3423)
firmware version control (incl.new firmware), drupal7-ztex
PLAINTEXT_LENGTH increased to 64 (#3424)
sha512crypt-ztex improvement (incl.new bitstream) (#3425)
adjusted default frequency to match new sha512crypt/Drupal7 bitstream
(#3426)
--test works (also with --mask), displays correct numbers (#3427)
various ztex improvements (#3432)
sha256crypt-ztex new version (#3588)
default frequency; documentation (#3589)
README update
Improved sha512crypt-ztex (#3628)
README corrections (#3629)
[List.ZTEX:Devices] (#3637)
ZTEX device numbering: Minor changes, coding style corrections (#3643)
ZTEX: allow to specify SN in cmd-line if board is listed in config;
drupal7 min_keys value to reflect new bitstream (#3646)
bcrypt-ztex new bitstream (#3661)
new md5crypt-ztex bitstream with clock gating (idle current
consumption 0.4A); README update (#3684)
improved descrypt-ztex (#3702)
descrypt-ztex: Ensure updated pcount < 2^31; const qualifiers (#3710)
Don't register ztex formats, skip boards initialization with --show
(#3713)
Numbering of individual FPGA on the board from 1 to 4 (#3716)
{sha256|sha512|md5}crypt-ztex: salt_length=0: print warning, don't
 process (#3742)
src/ztex/Makefile.in: added distclean target (#3774)
bcrypt-ztex handles > 512 hashes/salt (#3807)
bcrypt-ztex: fixed memory leak, correction in docs (#3828)
ztex improvements (#3833)
bcrypt >512 hashes/salt fix (#3839)
sha512common.h fix (#3845)

Fist0urs (30):
Initial commit.
Fixed bug in self-tests (thanks to @jfoug and @magnumripper)
Cosmetic changes
Added check while validating hash (thanks to @jfoug)
Added support to parse self-decrypting binary
Removed useless imports
Cosmetic changes
Added format for 16bytes-RawSHA1 AxCrypt in-memory secrets
Added doc to audit AxCrypt and its secrets
Added an important detail concerning cracking raw-sha1-axcrypt hashes
Removed unused variable
Fixed -mno-sse tests failure
Cosmetic changes in doc to fit .md style
Improvments of krb5tgs Add support to different ciphertexts input
format Get rid of max data len concerning edata Get rid of RC4
rounds Get rid of last hmac-md5 Performances increase
Fix split salt Change "$krbtgs$23$" to "$krb5tgs$23$"
Add verification of checksum if potential pass were to be found
Changed input format from edata1$edata2 to edata2$edata1 Thus had to
use dyna_salt (and put back edata max len definition)
Changed input format to fit upcoming oclhashcat one Get rid of max
edata2 Performance increases
Changed instructions to fit new format validation
Fix misnamed variable in parsing self-decrypting PE
keepass2john.c: incorrect parsing of keyfile within Keepass 1.*
Modified keepass2john and keepass_fmt_plug.c - keepass2john now handle
all different kind of keyfile within Keepass 1.x and produces an
unique type of output - keepass_fmt_plug.c was modified so that it
could handle all kind of keyfiles within Keepass 1.x + removed a
SHA256 of keyfile that is now pre-computed in keepass2john
Cosmetic changes in keepass2john and keepass_fmt_plug.c
Added support of keyfile for Keepass 2.x, both in keepass2john and
keepass_fmt_plug.c. Remaining only support of xml' keyfile
Added support for xml keyfile within Keepass 2.x
Add tool "kerberom" to /run This tool is aimed to retrieve ARC4-HMAC
encrypted tickets to be cracked with john
Add hash extractor + support for cracking DPAPI masterkey file from XP
to Win10
kerberom: new version adding Windows support and Windows implicit
(#2557)
Remove CRLF and trailing whitespaces in run/kerberom. Addresses #2559
(#2562)
Remove "kerberom" tool (obsolete)
Add documentation on Kerberos 5 TGS attack

Elena Ago (15):

BitLocker (OpenCL version): BitLocker password cracking tool (#2427)
New hash format (#2516)
Bitlocker-opencl: Various fixes, closes #2512. Closes #2546.
BitLocker: Extend OpenCL password length limit and use common tests
 bitlocker2john code reworked (more check added). code cleanup and
comments in main attack kernel
Comment fix
Format flags
getopt_long removed (getopt instead), VMK comparison improved with
version check (should help to avoid false positives)
Add support for new BitLocker attack modes
Support new attack modes in BitLocker format
Various enhancements for BitLocker format
Autotune problem fixed: warn labels, split events, loop number.
Profiling only kernels and final transfer
bitlocker2john: file_length must be a long int to support large files
bitlocker2john can find recovery password even without user password
Code reworked - added two different salt and aes offsets

Aleksey Cherepanov (10):

added --show=types option; renamed m_allow_rawhash_fixup into
dynamic_allow_rawhash_fixup and made it global
removed unused variable in --show=types code
Add support for "lineage" hashes
move Aleksey's and magnum's code for --show=types[-json] from
ldr_split_line() into showformats.c
replace showformats_nis() and showformats_lonely() calls by unified
showformats_skipped()
rename --show=types-json into --show=formats; rename fields:
showtypes_json -> showformats, showtypes -> showformats_old,
meanings are changed too
replace "consistencyMark" with "skipped" in JSON, add rowFormats:[]
for skipped hashes; combined code into showformats_skipped()
document --show=formats option (#3773)
documentation for --show=formats: update after fixes for ztex formats,
added about ad-hoc format, fixed minor typo (#3858)
fix buffer overflow in HC rule p (duplicate word N times) cutting
length (#3856)

Jan Starke (6):

Update regex code for working with latest rexgen library from
https://github.com/teeshop/rexgen. Older versions of the lib are no
longer supported.
compiles cleanly with rexgen 1.3.1, contains version check for rexgen
removed error messages in case rexgen is not available
updated configure script
Using the new multibyte character aware API from rexgen (#2566)
Using the new multibyte character aware API from rexgen (#2571)

András Veres-Szentkirályi (5):

Raw-MD{4,5{,flat}}: handle uppercase hex digits (#2845)
added memcpylwr, first for rawMD5 only
merged further memcpy + strlwr calls
added strnzcpylwr for strnzcpy + strlwr combos
strnzcpylwr: copy zero terminator as well

Ivan Freed (4):

Utility to help with cracking DiskCryptor
JtR format for cracking DiskCryptor
Fix structure packing in DiskCryptor stuff
OpenCL support for cracking DiskCryptor

Christien Rioux (3):

 Bugfix for path_expand_safe() returning offset from allocated buffer
(segfaulting when freed).
OpenCL: The HANDLE_CLERROR macros evaluated its input twice on error
fix for unroll bug

Sylvain Pelissier (3):

Add new version of eCryptfs file
Copyright Date changed
New copyright

Stuart Morgan (3):

Adapted the regex matching to cover the example /etc/security/passwd
file provided by IBM
The parser code did not permit this script to be executed because of
the way argparse works. When this was removed, the 'username'
matching code was not reliable in the event of trailing spaces or
linefeeds. I have replaced this code with a regex and otherwise left
it alone (with the exception of swapping .lstrip() and .rstrip() for
.strip())
Fixed command line arguments & added more informative help

akos.sereg (3):
Adding extra config parameter: shell script to be executed on
successful password crack
Concat safer with snprintf
Adding sample script to configuration file

campeador (3):
Change optparse to argparse
Change optparse to argparse in efs2john.py and openssl2john.py
Delete comments in efs2john.py

walkerk1980 (2):
Added LogFilePermissions, PotFilePermissions, IgnoreLogChmodErrors
Made suggested changes to logger.c, add temporary umask change to
allow log file creation perms, change to bitwise check from strcmp()
on LogFilePerms and PotFilePerms

Christian Mehlmauer (2):

update OSX compile instructions
Update .gitignore

Rob Schoemaker (2):

Add support for cracking AS/400 DES hashes
Update information about AS/400 DES hashes

Rémi Dubois (2):

Adding dynamic_1034 for PostgreSQL MD5 support
Additional comments for PostgreSQL

Sanju Kholia (2):

Port luks2john.c to Python
Remove luks2john.c (replaced by luks2john.py)

Shinnok (2):
Ignore .DS_Store files on OS X.
Properly format the .gitignore file.

Dennis Addo (2):

 Rewrite common.c: isdec*() #2951 (#2979)
Update README.md #1972 (#3003)

Louis Granboulan (2):

configure: detect openssl installed by MacPorts on macOS (#3577)
HDAA: remove check that cnonce is lowercase hex

arcctgx (2):
Make sure wpapcap2john initialises allocated memory with zeros.
(#2429)
add support for more recent Gadu-Gadu login packets

chipx0r (2):
PKCS#8 DER Compatibility
try/catch certificate

Catalin Patulea (2):

radius2john: support for LINUX_SSL (cooked capture) containing IP
radius2john: preserve unknown vendor extensions when constructing salt

Michel Normand (2):

set ARCH_LITTLE_ENDIAN for powerpc64 little endian in src/ppc64.h
add src/escrypt/tests in .gitignore

philsmd (2):
pdf2john.py problem, commit efc315e926f521cc41c002742a994776edbacc1d
broke it
pdf2john /Encrypt /Length problem for PDF versions 1.1 - 1.3

smarteam (2):
Update INSTALL-UBUNTU (#1)
encodings file name

Alexander Cherepanov (2):

fake_salts.h: change cp1252 dash to '='
episerver: remove gratuitous memset in hash_type

Will Dormann (1):

Make input parsing more robust in sshng2john.py

[email protected] (1):
Add support for parsing AIX hashes in unshadow

antnks (1):
Wrong comment statement which may confuse: NONCE_COUNT should go prior
to CNONCE RFC: https://tools.ietf.org/html/rfc2617#section-3.2.2.1
request-digest = <"> < KD ( H(A1), unq(nonce-value)
":" nc-value
":" unq(cnonce-value)
":" unq(qop-value)
":" H(A2) The code which actually concatenates the values however
is correct: snprintf(salt.static_hash_data, STATIC_HASH_SIZE,
":%s:%s:%s:%s:%s", login.nonce,
login.nonce_count, login.cnonce,
login.qop, static_hash);

bigendiansmalls (1):
Add support for RACF KDFAES hashes (v0.16)
c0re (1):
pdf2john: Fix AttributeError

clavoillotte (1):
Fix preferred master key display in DPAPImk2john.py

cpsw (1):
Suggestion for code style (#2985)

jagotu (1):
Support DPAPI masterkeys from Windows 10 1607+ (#3419)

rofl0r (1):
use SHA1_Init() consistently with other SHA1_* functions

rschoem (1):
as400. added docs, updated copyright, added python script

terrybwest (1):
keystore-opencl format

Ryan Govostes (1):

Add support for hashes hidden by DeepSound

Albert Veli (1):

keystore2john.py: Python2/3 compatibility

Alex Stanev (1):

Add new tool hccapx2john.py

Alexander Haase (1):

Added support for HDAA as defined in RFC 2069.

AlzoxX76 (1):
Update INSTALL-UBUNTU

Austin Walker (1):

Fix error due to unassigned variable

Bernhard M. Wiedemann (1):

sort input files (#2560)

Bernhard Spil (1):

Replace deprecated des_ methods and calls that were available in
openssl/des_old.h with the DES_ counterparts.

Bigendian Smalls (1):

racf2john update to dump KDFAES hashes (#2510)

Chick3nman (1):
ethereum2john.py - warn users about risks associated with sharing
hashes

Christian Inci (1):

Fix for 31-character passwords on PKCS12 (off-by-one error)

Dhirendra Kholia (1):

Make ssh-ng saner by using a tiny ASN.1 decoder
Dmitri Lapin (1):
Fixed wrong header (was immintrin.h - AVX header)

Enter Name (1):

decodestring vs. decodebytes: Confirmed compatibility with Python 2.6
& 2.7 (#3347)

Francois Pesce (1):

Add a cracking rule that converts words in t9 (#3230)

Ioannis Profetis (1):

autoconf: Replace deprecated symbol check for OpenSSL >= 1.1.0 (#2558)

Ivan Novikov (1):

Add support for cracking ColdFusion 11 hashes

Julian J. M (1):
format openssl-enc. Implement ascii percentage check. (#3452)

Leorize (1):
idle: add Haiku support

Liu Yihao (1):

rar2john edge case fix. See #2209.

Lukas Odzioba (1):

Added salt length check

Mathieu (1):
Fix crash when receiving CTRL_C_EVENT on Windows from parent like
Johnny.

Matlink (1):
Rename HashCat to hashcat (#2463)

Maxime Goyette (1):

added a working python script to convert one or many radius pcap files
into john friendly hashes

Michael Kramer (1):

Kerberoast format added

Mickey Sola (1):

UnRAR code, update unrar code from upstream:
https://github.com/Cisco-Talos/clamav-devel/tree/master/libclamunrar
as of 2fe4942f7: * mass update of copyright headers * fix possible
out of bounds stack read. * bb11601 - check array boundaries in
unrarvm rarvm_getbits(). * bb11601 - revise buffer limit check due.
* unrar - adding proposed changes to fix RAR VMSF_DELTA Filter
Signedness error * Eliminating warnings, converting iterator
variables to size_t when used to compare against sizeof(). added a
couple of missing #includes. * adding back changes to eliminate
warnings from unrar files other than unrarppm. * Various other
manual fixes for our fork. See #2899.

Miha Zupan (1):

Improve path detection in telegram2john (#3301)

MrTchuss (1):
 Support for FortiGate SHA256 format (#3297)

Noah Luddy (1):

Fixes command line prompt ambiguity

R. Brenton Strickler (1):

Explain DBError exception.

Ralf Sager (1):

Add support for cracking NSEC3 hashes

Agnieszka Bielec (1):

newest argon2 CPU version, no idea why virtual speed is 8 times slower
than real

Steven Noonan (1):

OpenCL: Add a missing space after -D__MESA__

TROUNCE (1):
Update geli_common_plug.c (#3864)

Tonimir Kisasondi (1):

Added keyring2john.py

Warren MacEvoy (1):

Update INSTALL-UBUNTU (#2625)

The following changes were made between 1.7.9-jumbo-8 and 1.8.0-jumbo-1:

magnum (1889):
formats.c: Only check binary/salt alignment if not using fmt_default_*
formats.c: Deliberately use misaligned ciphertext in self test
formats.c canary for binary/salt buffers
bench.c fix for salts that copy the pointer in set_salt()
bench.c: clean up that salt fix a little
bench.c: call set_salt() before bench_set_keys() (hopefully fixing the
last issues with formats having pointers to freed memory)
Fix compiler warnings for void pointer arithmetic
trip_fmt: trivial fix for declarations at block starts, for VC builds.
This is originally from Jim's patch d91bf42 but I dropped it in
bleeding at first due to a merge conflict and because the
offending lines are also declarations. However it turns out when
this macro is not defined, the lines end up as spurios ';' so it
upsets VC nonetheless.
Revert the last revertion. We only actually wanted it reverted in
magnum-jumbo
Loader.c fix for get_source()
Fix a silly typo in loader.c
mscash bugfix for codepage encoding
GOST fmt_default_get_source
Enable get_source() to raw_SHA1_LinkedIn_fmt_plug.c, and add
fmt_default_get_source to cuda_pwsafe_fmt.c
Bug fix for the previous patch
raw-sha1_li: Move key_count update to cmp_one to regain original
speed.
Revert "raw-sha1_li: always store complete (repaired) hashes"
fmt_default_get_source in Tavis' format
Fix get_source() for reduced binary in nt2, raw-md5 and raw-sha1.
 Needed to move the get_source() self-test to after crypt_all()
test.
Added fmt_default_get_source to odf_fmt
Revert "reduced binary size" from formats that use get_source()
Revert "Fix get_source() for reduced binary in nt2, raw-md5 and
raw-sha1. Needed to"
Use the "repairing" version of raw-sha1_li.
LinkedIn: Off by one from ditching key_cound in favor of NBKEYS
formats.c: Tidy up the binary canary stuff and fix (my) indentation
-Wdeclaration-after-statement in default CFLAGS
RAR: make self-test comments more clear about what is what
params.h had a bad merge at 501c9626, this patch fixes it.
Fix for fake_salts.c that regains 52MB of BSS
define REPORT_OPENCL_WARNINGS in bleeding after it was undef'ed in
magnum-jumbo
get_source for radmin_fmt
Add MGF_FULL_CLEAN_REQUIRED to dynamic_1003
Damn mis-merge of params.h again. Is it meld or git doing the Wrong
Thing?
Add pfx2john to .gitignore
Add fmt_default_get_source to a couple of new formats
Enable source() for NT, NT2, raw-sha1, raw-sha1-linkedin, raw-md5 and
dynamic.
Enable source() for raw-md4
john-1.7.9.5-align-1.diff
loader.c: For Jumbo, we must populate up to fields[5] for l0phtcrack
to work
Merge fix for odf_fmt, and new format struct for agilekeychain_fmt
New format structs for truecrypt formats
New split() style declaration for hmac formats
v10 format struct for Dhiru's new formats
Make sunmd5 handle bleeding FMT_MAIN_VERSION as well as current. Also
fix some Linux warnings.
loader.c: The master.passwd test ate the pwdump format so uid was not
loaded properly for pwdump. Also, we accept empty LM but present
NT hash as pwdump.
opencl_gpg lacked forward-compatible format struct
Dual-version format structs for new cryptsha2 formats
loader.c: Bugfix for pwdump files when uid is re-inserted in source.
NOTE: This bugfix applies to core 1.7.9.6 too!
loader.c: Honor field_sep_char when rebuilding pwdump format.
OSX fix for rawSHA512_fmt
Suddenly, an is_aligned() macro in common.h start to clash with an
inlined function in formats.c. This patch undefs the macro
before declaring the function.
Merge branch 'unstable-jumbo' into bleeding-jumbo
Merge fixes for bleeding
Bump FMT_MAIN_VERSION to 11
Temporary move plugins to unplugged/ and fix all non-plugs.
formats.c Jumbo self-test: use crypt_all(count, salt) prototype.
Remove all atexit(done)
Move plugins back from unplugged directory
Have pwsafe-cuda default to one GPU.
New interface crypt_all() for remaining formats.
Update crypt_all() ABI to FMT_MAIN_VERSION 11 in x86-64.S.
mschapv2 bug fix. All CPU formats working now.
All CUDA and OpenCL formats fixed. Now builds fine.
Makefile: Add CUDA header search path
Regression bug fixes to keychain-opencl and agilekeychain-opencl.
More regression fixes, mostly to done() functions. We must basically
do a normal release_obj() and also release all kernels. Nothing
else. Also fixes to SALT_ALIGN.
More regression fixes.
More done() fixes to OpenCL formats.
Regression fixes. Automated editing is error-prone :-P
Comment out debug prints in pwsafe-cuda
Fix a bogus merge. In bleeding, we'll show OpenCL warnings.
Drop all #ifdef FMT_MAIN_VERSION except in dynamic. The latter should
probably benefit from new ones added instead, so it can be moved
between unstable and bleeding at will.
pkzip: SALT_ALIGN
Documentation update (bleeding-jumbo specific)
Enhance the --mkv-stats=FILE option so it can find a file residing in
john's home directory. Plus some other little fixes.
Regression fix for the valid-killer self-test. It's normally not
active so I missed it until now.
MPI and krb5-18/23 merge fixes.
SybaseASE salt alignment fix.
Minor OpenCL/CUDA tweaks. Enable advance_cursor() for CUDA-only builds
too. Also some typos: git grep -il tunning | xargs sed -ri
's/([tT])unning/\1uning/g' git grep -il plataform | xargs sed
-ri 's/lataform/latform/g'
OpenCL: Change the -DAPPLE for OpenCL build options to -D__APPLE__.
Bugfix for "make generic".
Commit 04cf5f1 hurt performance for dynamic because data locality
suffered. This patch reverts it and use a union instead, to
satisfy the compiler.
Relbench: Ignore the new "All nn tests passed self-tests"
Single mode: Optionally load the full GECOS field (before splitting)
as one extra candidate. This is controlled with PristineGecos in
john.conf (and defaults to disabled).
Use list_add_unique() for the optional PristineGecos loading.
Add a john.conf option for over-riding SINGLE_WORDS_PAIR_MAX.
truecrypt_volume2john had CRLF line endings.
clang debug target fix.
Salt/binary alignment fixes.
BFEgg: Don't convert to base64 in crypt_all(), instead implement a
proper binary() that does the reverse. Conflicts:
src/BFEgg_fmt_plug.c
Fix a logic error for some only-once warnings.
Use correct value of BINARY_SIZE: GPU formats, and a couple Dhiru
missed (?)
Revert "clang debug target fix."
Alignment fixes in xsha512, nethalflm and sabB.
formats.c: Never mind binary/salt alignment if size is 0.
Alignment fixes to WPAPSK/ZIP formats (including GPU versions). And
yet some tweaks to XSHA512 (MEM_ALIGN_WORD is wrong for 64-bit
when we return ARCH_WORD_32).
Revert "OpenCL: Change the -DAPPLE for OpenCL build options to
-D__APPLE__."
xsha512-opencl: Alignment.
formats.c: Salt/binary size self-tests (if -DDEBUG).
Use -DDEBUG for the clang-debug targets!
dummy.c: Unify MAYBE_INLINE with core CVS to reduce differences.
Office OpenCL formats: Better valid(), with CPU format as template.
RAR formats: Implement a real bastard of a valid(). I will soon start
denying having written it.
xsha512-opencl bugfix. My bad when fixing alignment.
Allow running an OpenCL binary without having any OpenCL platform
active.
Bwall's pwsafe patch broke builds using CommonCrypto. This fixes it.
Fix a hideous bug in rar2john introduced by in 216cc57. A variable was
declared as just "unsigned" (taken as int) instead of "unsigned
char" which lead to incorrect output. Not sure why the compiler
did not warn.
Correct ALGORITHM_NAME of mssql-12, pbkdf2-hmac-sha512 and the
Truecrypts. Use sha2.h for a number of formats that depended on
OpenSSL 0.9.8 (all but Truecrypt). Bump max length to 125
for pbkdf2-hmac-sha512 format. There's absolutely no point in
limiting it to 15.
Fixes for builds using Jim's homebrewn SHA-2.
Drop a lot of crap from the unused/ directory. Old files can be
fetched from git if needed.
mscash2: Drop no-op HAVE_OPENSSL lines.
OpenCL bugfix for legacy parsing.
Move --platform to "hidden" options with a note it's deprecated.
options.c: Print formats last.
Warnings from valid() should be suppressed if ldr_in_pot is set.
Fix for ldr_in_pot.
Introduce hash functions to hmac-md5.
Introduce hash functions to hmac-sha1.
Try to automagically detect and locate NSS, GMP and KRB5 libs, and
enable corresponding formats if successful. This is using Posix
find/grep and backticks. And pkg-config, if available.
MPI: Limit scope of an internal variable.
BLAKE2: Do not "secure zero memory".
SIPdump fixes (CppCheck). Also revert using MEM_FREE() and mem_alloc.
That is not a good idea for a stand-alone.
(Ab)use bench_running variable for keeping opencl_process_event() from
processing events while self-test is running. Otherwise a slow
self-test in combination with a short Save interval will thrash
the .rec file during start of a resumed session.
WPAPSK formats has a minimum password length of 8. This patch adds
code to john.c that enforces this. We may want to add min_length
in the format struct later on, and remove this hack.
WPA-PSK had a silly ESSID length limit of 14 that was already bumped
to 32 (the actual real-life limit) for the OpenCL version.
Moreover, it segfaulted on longer names. This patch bumps it for
CPU & CUDA too. For some reason the hccap buffer size is 36
(from hccap2john) so I kept that. Perhaps it's because it ends
up aligned including room for a NULL.
Decrease PLAINTEXT_LENGTH for all WPA-PSK formats to 64, because that
is the spec. limit. More interestingly, the spec. says the
characters set must be 0x20-0xfe (printable ASCII) so we also
unset FMT_8_BIT.
Add two WPA-PSK test vectors, including one for WPA (as opposed to
WPA2).
Get DMD5 straight, and add OMP support. This format was seriously
broken.
dmg: Make _memmem() static inline (will hopefully optimised away the
switch), enable 64-bit tests on 64-bit arch and disable
unaligned checks on platforms that does not support it.
dmg-opencl: Same fixes to _memmem(). ssh-ng: Provide an own
static inline copy of _memmem() instead of using one from dmg.
Bump OMP_SCALE for DMD5.
Format struct bugfix for DMD5 vs bleeding.
Assorted BINARY_ALIGN & SALT_ALIGN fixes.
More BINARY_ALIGN & SALT_ALIGN fixes. There a lot still to do.
Unify Kerberos names.
DMD5 format should not lowercase nonce hashes. They are used as-is!
hmac-sha1 bugfix for get_hash().
Cosmetical changes.
SAP formats: Minor fix mostly for satisfying Valgrind.
Fix an off-by-one in dynamic.
Office: Use calloc instead of alloc, just for muting Valgrind.
memory.c: When -DDEBUG, we turn mem_alloc_tiny() into a normal alloc,
to better track problems.
Drop some #ifdef DEBUG, or change them to specific ones like
RAR_DEBUG.
Wow-srp: This format is not case sensitive so unset FMT_CASE, and also
make the upper-casing in set_key() encoding-aware.
Wowsrp has a minimum password length of 8, so add it to the hack we
did for wpa-psk.
Get the "weird_fmt" back to unused/.
Fix V11 interface for the "weird" format. And optimise it a little
(lol).
Adjust a stack buffer after the recent bugfix to Office 2007. This
doesn't matter but it makes the code less confusing.
Get rid of all use of alloca() and dynamic size arrays. Also fixes a
bug in keyring format: The "crack array" was not always reset
correctly. Makefile: Add OMPFLAGS comments for OSX llvm-gcc.
Truecrypt format: Use signed int for OpenMP loop, as required by
spec.
DMG: Another alloca() found and changed to fixed size stack array.
Dynamic: make sure valid() is looking at de-hexified data if
applicable.
Pwsafe-cuda: Adjust BLOCKS according to count argument. Shaves a
minute from self-test and helps Single mode. Verified with the
Test Suite. Conflicts: src/cuda_pwsafe_fmt.c
CUDA formats: Adopt to count argument to crypt_all().
ZIP: Implement a proper cmp_all() instead of returning 1 and relying
on cmp_one(). This should be faster due to less function call
overhead.
Office OpenCL formats: avoid false sharing of crack array.
getopt.c: Allow an option named --foo to coexist with an option named
--foobar.
Minor fixes suggested by CppCheck. Nothing exciting.
Comment out unused functions in md5_eq.c
Unify dynamic_fmt.c among branches
bf-opencl: SALT_ALIGN fix.
OpenCL: Now that we always init OpenCL, try to be more tolerant
against device failures. This is not perfect but it works fine
for a situation my MBPr sometimes get stuck in - one device
stops working correctly but the other work fine. This is
probably not complete, especially if it's device 0 that get
stuck and device 1 that works...
Move WPAPSK test vectors to wpapsk.h and add a length 63 test.
wpapsk-cuda: Some compilers did not like having the test vectors in
wpapsk.h so we define it out when building kernel.
git grep -l '<stdint.h>' | xargs sed -ri 's/<stdint.h>/"stdint.h"/'
Fix a bug in dmg formats, and add debug output.
WPAPSK (CPU and GPU): Exploit same-essid weakness. Huge speedup when
applicable. See
http://www.openwall.com/lists/john-dev/2013/01/29/13 and on for
discussion.
DMG debug bugfix (duh!), and a new test that seems pretty generic.
dmg2john error message.
dmg2john: Temporary fixes to handle large files and iteration count.
DMG: Support iteration count. DMG-opencl: A bunch of fixes. This
format was not up to par with CPU format. Serious flaws fixed.
Add a known plain test for 8 consecutive nulls.
SIMD support for NTLMv1 and OMP tweaks for non-SIMD.
NTLMv1: Very effective hint to the optimiser in cmp_all(). And
benchmark using crippled plaintexts, so we measure real-life
performance.
NT2: Bugfixes to experimental OMP support (defaults to disabled).
DMG formats: Disable all but the NULL test, but bump that one to 16.
DMG: Re-enable the other plaintext checks (but not Apple).
DMG: properly re-enable plain checks, for OpenCL too.
NTLMv1: Bugfix, and compact the hash functions (before merging the new
stuff to mschapv2)
MSCHAPv2: SIMD support and exploitation of weakness just like NTLMv1
(the formats are very similar)
NTLMv1 and MSCHAPv2: reject hashes in valid() as hinted by Solar.
Bugfix for Jim's patch. Some debug functions were not static, and they
were included several times for GPU builds so the compiler
bailed. Also wrap that debug stuff in WPAPSK_DEBUG.
MSCHAPv2, NTLMv1: After the 3rd DES block check in valid(), cache the
indexes so we don't have to loop again in binary().
loader.c: Support "late reject" from binary(). If binary returns NULL,
the line is rejected.
NTLMv1 and MSCHAPv2: Make use of new late-reject feature, so it gets
tested. This means we no longer test the 3rd DES block in
valid() but instead return NULL from binary() if it fails.
Mute some over verbosity in OpenCL, and make the pwsafe format default
to auto-tuning.
pwsafe-opencl: revert a change by bwall that broke auto-tuning for
weak devices.
Truecrypt: replace a dynamic array with alloc/free. The format is so
slow this does not hurt performance at all.
OpenCL: Tweak find_best_gws. It gave up to early: We need to take into
account how much work was done. If it took 10x more time than
minimum but we had a 1000x higher GWS, we should definitely not
stop! This code needs more tweaking.
pwsafe-opencl: Now that common code doesn't give up too early, lower
max time for GWS enumeration to 2.4 seconds (should avoid
watchdog problems).
Revert "NTLMv1 and MSCHAPv2: Make use of new late-reject feature, so
it gets tested."
Revert "loader.c: Support "late reject" from binary(). If binary
returns NULL, the"
Add some self-tests for functions returning NULL.
NTLMv1 & MSCHAPv2: Use multiple blocks for the SSE version (still
single threaded) for bumping "many salts" speed. Actually we
bump it to the ceiling of what JtR can report - the output is
capped at 4294M c/s.
bench.c: Support benchmark speeds over 2^32.
NTLMv1/MSCHAPv2: Use memset for bitmap if more than 200 keys per
crypt.
NT2, NTLMv1, MSCHAPv2 tweaks for experimental OMP+SSE2.
Dynamic: Bugfix for MMX (I think it's right, I can't link it).
Raw MD4/MD5: MMX fixes.
More MMX fixes, including a cludge in the max-length self-test, for
MMX formats that use total_len.
dynamic.c: Bugfix.
(Alain's) NT format: Adjust x86-sse.S to new ABI.
Use DO_ALIGN macros in Jumbo .S files.
Optimizations for keystore format. Sub-percent speedup :-P
Portability fixes from building on ancient Sparc Solaris 32-bit.
MSCHAPv2 & NTLMv1: Emit a warning after loading 100 c/r pairs,
informing about slow loading and alternative formats.
Re-introduce the old versions as mschapv2-naive and
netntlmv1-naive. Also fix the DES caching in NETNTLM: it may be
called from prepare() so the cache must be checked not only in
binary() but also in valid().
Siemens-s7 format: Trivial improvements, and separate key setup from
salts. This made a 30-40% boost.
o5logon: Minor optmizations.
CUDA device list: Add PCI id to output, and replace a deprecated
parameter with a new, getting more detail.
Add any_cracked to a bunch of OpenCL formats that use crack array.
Conflicts: src/opencl_agilekeychain_fmt.c
src/opencl_dmg_fmt.c src/opencl_gpg_fmt.c
src/opencl_keychain_fmt.c src/opencl_zip_fmt.c
Add "naive" to the ALGORITHM_NAME for old load-fast-run-slow versions
of MSCHAPv2/NTLMv1, as in [32/64 naive]
Add wpapcap2john to .gitignore.
GPG: Optimizations, and bump max length to 32.
Endianness & alignment fixes from testing on a Sparc32.
More BE and alignment fixes. Some problems were in shared code in
unicode.c and the problems has been fixed backwards until now.
Most unicode.c functions convert/process UTF-16LE *regardless of
arch* and any endian fixes must account for that (normally none
is needed).
para-best.c fix for core changes.
Make itoa64 and the likes const.
Crypt-SHA formats: Use a ROUNDS_MIN of 1.
pbkdf2-hmac-sha1: Support OSX 10.8 from "the Dave Grohl tool" as well
as GRUB hashes, without converting them. We support hashes
longer than 64 bytes but we don't calculate/verify past 64 even
in cmp_exact() - the chance for such a collision is
slim-to-none.
pbkdf2-hmac-sha1: Do a complete calculation & verify for full binary
size, just for good measure.
pbkdf2-hmac-sha512: Improve the warning messages given for partial
matches.
Endian fixes for BFEgg, wpapsk, pbkdf2-hmac-sha152, pwsafe and shared
pbkdf2-hmac-sha1. Also a bugfix for openvms but it still
segfaults for me. This is now the only format that doesn't pass
self-test.
Drop run-time version checks for NSS that would be bad if run-time lib
is older than build-time one.
Mscash: Get rid of compiler warnings on linux-ppc32. Conflicts:
src/mscash1_fmt_plug.c
Alignment fixes.
Rules.c bugfix for -max-length. Format's max length is already
adjusted down to max-length, and the RULE macro would do an
unwanted incrementaion.
Use sha2.[ch] from unstable branch.
Revert the previous patch and use the SHA2 version that was in
bleeding instead, but properly indented. Also try to mitigate a
bogus strict-aliasing warning.
Mute harmless compiler warnings if MIN() already defined.
Better fix against that alleged strict-aliasing violation.
Move md5_eq.c to unused.
Remove md5_eq.o from Makefile.
formats.c: Stop trying to use system's strnlen(), instead always use
our supplied fmt_strnlen(). We could move it to misc.c too at
some point.
Revert SHA2 strict-aliasing fixes. They did not work at all for BE.
We can force Jim's generic SHA-2 with -DFORCE_GENERIC_SHA2. This patch
adds a way to force OpenSSL over CommonCrypto, using
-DFORCE_OPENSSL_SHA2.
Alignment fixes (bleeding only)
Alignment fixes
Mute a warning for generic target.
Merge the current MSCHAPv2 naive format (with FreeRadius support) to
Deepika's bitsliced format. Then drop the original format and
use BS format as "naive". Lots of whitespace changes, view with
-w.
DES BS plain SSE asm: Properly underscore P if applicable. It works on
OSX anyway but I'm not sure it will on others that
-DUNDERSCORES.
Alignment fix for mschapv2-bs
MSCHAPv2-BS: Merge two loops into one. This made a 5% boost here.
MSCHAPv2 BS: #ifdefs for FMT_ flags
Update CHANGES-jumbo.git & CREDITS-jumbo.
I always end up confusing crypt(3) base64 with MIME Base64: Add some
comments.
Fix a slight bug in sha512crypt binary() that could never surface. It
was processing the trailing NULL. Also drop some copy-paste
artefacts.
Some Kerberos libs are not thread safe, but fortunately we can find
out using the krb5_is_thread_safe() call. Conflicts:
src/krb5-18_fmt.c src/krb5-23_fmt.c
Decrease OMP_SCALE in a number of formats. This number needs to be
tweaked for each format, not just copied from a template.
Alignment fixes to Sybase & pkzip.
Sort load order of GPU formats so they are easier to find within a
full --test.
Merge branch 'bleeding-jumbo' of
https://github.com/bwall/JohnTheRipper into bleeding-jumbo
listconf.c: Rewrite the formats list function. It was overly complex
and buggy.
Add format groups "opencl", "cuda", "cpu" and "gpu" for use with
--test as well as with things like "--list=formats" and so on.
Also allow wildcard in the end, as in wpapsk* or krb5*.
listconf.c: Suppress leading space in benchmark_comment.
Document --format group aliases/wildcards.
pwsafe-opencl: Some platforms will complain unless OpenCL functions
are static, inline or both.
Put Sayantan's format last in registration, until they are fixed.
PWSafe-opencl: Fix a bug where device max sizes were not queried for
all kernels (now that we use a split kernel).
OpenCL DES: Honour device limit for LWS.
Shave 35 seconds off pwsafe-cuda self-test. I did this before but it
was recently reverted by some other patch.
Revert some previous "CppCheck fixes" that might affect performance
(and is not really needed anyway). Not sure what I was smoking
that day.
Drop bogus BE code from MMX path in nt2. It would never be used.
Documentation updates.
Add OSX clang-debug target.
Mute MSCHAPv2 warnings appearing due to valid-killer self-test if
-DDEBUG.
Previous patch made for warnings unless -DDEBUG o.O
Typo in fix for fix. Time to sleep?
Various constant arrays made const.
NT formats: Five percent boost for codepage encoding, from
optimizations in set_key().
raw-md5u: Three percent boost for codepage encoding, from
optimizations in set_key().
ntlmv1/mschapv2: Five percent boost for codepage encoding, from
optimizations in set_key().
mssql05: Optimizations in set_key() for codepage encoding.
DES-opencl: Safer LWS capping (ensure GWS is still multiple of LWS).
ODF formats (all three): Handle content.xml sizes less than 1024 bytes
correctly.
ODF formats: Make valid() a little more picky about hex length.
Add macosx-x86-sse2-opencl (ie. 32-bit) build target.
More uint64_t fixes.
Use "stdint.h" in pbkdf2-hmac-sha512 format.
DMG formats bugfix for "0 iterations" (assume 1000).
Truecrypt: Simplify cmp_all(), unfortunately for no performance gain.
dmg2john: bugfix.
dmg2john: Use base name of file name in output.
DMG formats: Serious bug fixes for 3DES v1 images. This fixes the
problem with false positives as well as an even worse one: false
negatives o.O
uaf2john: Do not support writing to outfile, only to stdout. And add a
make target.
wpapcap2john: Support multiple infiles. zip2john: Use base name of
filename for login field.
MSCHAPv2 & NTLMv1: Smaller hot array. 25% boost for many salts, a
couple for one salt. Also enable OMP for SSE2.
MSCHAPv2 & NTLMv1: Again disable OMP for SSE2 - it works fine on some
CPUs but not on others.
Merge MSCHAPv2_fmt_plug.c and NETNTLM_fmt_plug.c to one file:
ntlmv1_mschapv2_fmt_plug.c, sharing much of the code.
Make odf2john work with python 2.6 (as well as 2.7).
Change odf2john so we fall back to the deprecated function only if the
newer one is not supported.
Unicode fixes for NOT_JOHN
unicode.c: Support re-init of encoding.
mscash2-opencl honours LWS and GWS env variables, des-opencl and
bf-opencl honours GWS environment variables, added functions to
detect optimal GWS for bf-opencl and des-opencl
Fix SAVE= timer similar to Costin's patch.
Fixes for the recent timer changes.
If OS_TIMER, we only call status_get_time() if --max-run-time or
--progress-every was given.
Update CHANGES-jumbo.git and CREDITS-jumbo.
Document --request-vector and --request-scalar options in
README-OPENCL.
mscash2-opencl: Fix for forced LWS larger than kernel's max.
Document the OSX problem with OpenCL kernel header files.
Wordlist bug fixes: Reject wordlist containing NULL bytes, and handle
inconsequent line feeds (eg. spurious CR within a line). This fix
will do for Jumbo-8 so we don't introduce new problems.
Import cisco2john.pl from unstable.
Fix the postgre format: Rename it to postgres, including the tag. The
old format is still supported. Also improve valid().
Some format name changes to clarify what they are.
s/Staroffice/StarOffice/
Add CXX variable to Makefile. Add separate PROJ_ sections for C++ as
well as pcap, so John does not fail building even if they do.
Also add uaf2john and truecrypt_volume2john to the main targets
and move office2john to unused.
Bugfix in keystore2john.
uaf2john: Fix an aliasing problem.
Fix another warning in wpapcap2john (fread return value).
keystore2john: Fix another fread warning.
raw-md5-opencl: support max. length of 55 (the max that fits in one
round) while minimizing transfers to actually needed size. This
is a winner, although the boost is not quite as good as I hoped
for.
raw-md5-opencl: Limit GWS to what is supported by the index size.
Clean up and simplify the max-length self-test. The previous version
was confusing when debugging problems so we now do it after all
other tests.
Revert "Experimental: Let Incremental and Markov modes call
format->methods.clear_keys()". This doesn't seem to be of much
value, and if it turns out it is, we'll re-introduce it then.
Fix generic build.
Fix generic build after the timer fixes.
Move wpapcap2john from PROJ_PCAP to PROJ - it does not depend on
libpcap! Conflicts: src/Makefile
rawmd5-opencl segfaulted due to common-opencl missed calling
clear_keys().
formats.c: Minor Jumbo-specific bugs fixed.
raw-md4-opencl: support max. length of 55 (the max that fits in one
round) while minimizing transfers to actually needed size.
Revert 8e992e4 and instead bump RULE_BUFFER_SIZE in params.h by 4. We
should look into this more closely but this will do for now.
Makefile: Add a "Done." after building the pcap targets.
Experiment with error-tolerating crypt_all_benchmark().
Replace 4 duplicate definitions of _memmem() with a faster
implementation in misc.c. I believe the old code was O(M*N)
while the new is "somewhat like O(M+N) for most cases" yet is
shorter. Also, it is declared so that it's inlined where
applicable.
misc.h: include string.h for memcmp.
Unicode re-init bugfix.
NTLMv2-opencl: Use packed key buffer as recently introduced in
raw-md4/5. This also enables full-length UTF-8 support (up to 81
bytes of UTF-8 for representing up to 27 Unicode characters).
NTLMv2 kernel: Bugfix for no-byte-addressable code path.
pass_gen.pl: Support UTF-8 for -salt argument when -utf8 option is
given.
Stop user from trying to use any encoding other than iso-8859-1 with
formats that lack support for it. It would just end up in false
negatives.
Drop bogus FMT_UTF8 flag from Truecrypt.
NTLMv2 OpenCL: Use length 8 for benchmark unless -DDEBUG.
NTLMv2 OpenCL: Early transfer of keys.
Change MPI reduction for benchmarks for accurate average speeds. Also,
reduce the int64 properly (only affects 32-bit or BE systems).
Update README-mpi.
CUDA formats: Tweak default THREADS*BLOCKS so they work on weaker
devices.
NTLMv2 OpenCL workgroup size enumeration tweaks.
Dynamic: Change name of "salted Cisco PIX" to "Cisco ASA"
Add thin format for Cisco ASA.
Copy the improved ZIP valid() to the OpenCL version of same format.
Refuse to translate \x00 into a null in user classes.
Add external mode filters for UTF-8 validation. This is slow though.
Change self-test use of set_key("", index) into something that
actually passes a large enough buffer for any use.
Change self-test use of set_key(current->plaintext, index) into
something that actually passes a large enough buffer for any
use.
Bleeding fixes/unifying MSCHAPv2 & NETNTLM (naïve bit-slice versions).
Sparc fixes. Latest optimization of mschap/ntlm formats broke BE.
PDF format still used the Bad Idea[tm] version of valid().
Purely academic bugfix in common-opencl.c. kernel_source was not freed
in clean_opencl_environment().
Add missing deallocations in OpenCL DMG & RAR formats' valid().
formats.c: self-test tweaks and source formatting.
SAP F/G: Bump maximum length. We over-ran a buffer with the latest
self-tests.
Fix recently introduced OMP flaws in keyring format. This lifts the
limit of number of threads and reduces needed memory per thread
by a factor of 128.
Make raw-sha256-ng a non-plugin
Another 10% boost in raw-sha256-ng from set_key() optimizations.
Self-tests that make raw-sha256-ng fail. It doesn't support plaintext
lengths beyond 55, and something else is wrong too.
Workarounds for bugs in raw-sha256-ng. Until we get better fixes, we
decrease max. length to 55 and introduce a workaround for
cmp_all().
Bug fixes to thin dynamic formats PHPS, formspring, mediawiki, osc and
phpassMD5. They over-ran buffer when given bad input files.
More robust valid() in salted sha1 / nsldap formats.
Do some actual validation in SIP valid().
Actually, while at it we should just as well validate it is lower-case
hex.
WOW-SRP fixes. Not sure I found all, this is a mess.
Makefile tweaks for JOHN_CFLAGS etc.
rar2john big-endian fixes for Unicode filenames in RAR.
Canonical fixes for new tcphdr.h (failed on Sparc)
Revert "JtR-Dynamic_validation_raw_hashes-2.patch". It made things
worse.
Macros for prefixing MD[45]_(Init|Update|Final) with "john_" for
avoiding clashes with OpenSSL library.
Endian stuff.
Change all uses of <stdint.h> to our local "stdint.h".
Drop un-needed header in ssha-opencl.
OMP support for raw-sha512-ng.
Tweak OMP_SCALE for AMD.
Add macosx-x86-64-native build target. To use it, you need to copy
osx_as_wrapper.sh (from the src directory) to /usr/local/bin/as or
to some other directory (but the script MUST be called 'as') and
ensure this directory is first in your $PATH.
keystore2john: Use basename of filename for login field, and convert
to symlink to john.
Bugfix for keystore2john target.
Precedence: MSCHAPv2 should pick sploit version over BS one.
Generic crypt format bugfix.
Testpara vs. HAVE_OPENSSL fix.
kwallet2john: Handle multiple files, and strip suffix and path from
 login.
truecrypt_volume2john: Use filename as login field, and convert to a
john symlink.
raw-sha512-ng: faster set_key().
raw-sha256-ng: OMP support.
Use -march=native for Linux 32-bit GPU targets.
Makefile fixes for OSX 32-bit cross-compile.
Bugfix for memory.c under -DDEBUG. In order to easier track bugs,
mem_alloc_tiny() is changed to a normal alloc. But it was supposed
to still maintain requested alignment!
Dynamic bugfix for $HEX$.
Change the way -DDEBUG affects mem_alloc_tiny(). The outcome is
practically the same but this is cleaner.
Revert "Change the way -DDEBUG affects mem_alloc_tiny(). The outcome
is practically"
Final canonical change for -DDEBUG versus mem_alloc_tiny.
Do not misalign SSE2 buffers when --save-memory=3 or above. The
previous fix did not catch all cases.
Mute self-test warnings for misaligned binary/salt if --save-memory
made the alignment smaller.
pbkdf2-hmac-sha512: Store canonical representation in pot file.
Keccak: header tweaks that make it build on SSE2, AVX or XOP. Maybe
not 100% but does work. Emits wads of warnings on AVX and XOP
though, and AVX is not faster than SSE2.
formats.c: Move max-length tests to before cracking tests. We must
verify that we can still crack stuff after using max-length
keys.
raw-SHA512-ng, fix a bug introduced in 14a5985
Max-length self-tests fixes. The current version still hangs BSDI
forever, I have absolutely no idea why. Working on it.
Fixed max-length self-test.
OMP_SCALE tweaks. Just picking a number out of thin air is actually
not really recommended.
More OMP_SCALE tweaks.
Relbench tweaks, for ignoring warnings.
OMP_SCALE tweaks. There are massive gains to win in some formats. All
OMP formats should be revised, but there are so many :-/
OMP_SCALE for raw-sha1-ng.
Copy the SSE2+ SHA-256 and SHA-512 from epixoip's formats to
sse-intrinsics.c before we start optimizing things away :-)
Revert "Revert cmp_all omp parallel, performs poorly on AMD"
raw-sha1-ng: Bump OMP_SCALE to 1024.
Make our usual MD5 format handle AIX's {smd5} hashes too, and drop the
much slower separate aix-smd5 format.
The pre-built (with-icc) .S files are now invalid, we need to generate
new ones.
Now that our MD5 format supports cracking AIX {smd5} hashes too, the
"FreeBSD" output is more confusing than ever. I changed it to
crypt-MD5.
I missed the "please keep this list sorted" :-)
Drop the intermediate encoding from aix2john.py and support the native
hashes in aix-ssha instead. We now lack some test vectors.
Minor fixes to aix-ssha.
aix-ssha bugfix for new binary().
Fixes for a couple of make targets.
Drop inconsistent paths in the auto-conf hack.
Re-worked valid() for aix-ssha. This version tests every field for
valid amount of valid data, where "valid amount" depends on the
tag. Also, bumps max. salt length to 24 as that is the actual
 maximum. If Alexander can smash this, I give up :-)
No longer any need for passlib python module in aix2john.py.
Unicode fixes for office2john.py.
Dynamic: Move a definition to avoid compiler warnings.
Netscreen: Better valid().
DMD5: Better valid().
Drop the wchar stuff from gpg2john. 8-bit GECOS now works fine.
Putty: drop an unused variable.
Add updated sse-intrinsics-xx.S files. Built with -O2 using icc
13.1.1. -O3 (as we used with 12.1) took 47 minutes and was a lot
slower.
Merge branch 'master+patches' into bleeding-jumbo. NOTE: The charset
files are gone for now. The old ones are incompatible. For
testing, create new ones from the Rockyou list, or something.
Drop deprecated Incremental modes like "All6". You can now use
-min-len and -max-len instead. Also add an entry for
"custom.chr" which will not be supplied - it's convenient for
one-off charsets.
Add external mode filters Alnum-case (a-zA-Z0-9 and space) and ASCII
(0x20-0x7e).
Add interim chr files to bleeding. We now have -inc:utf8 and
-inc:ansi, while -inc just produces 7-bit ASCII like before.
Update CHANGES-jumbo.git and CREDITS-jumbo.
Update CHANGES-jumbo.git and CREDITS-jumbo for bleeding.
Merge branch 'master' into bleeding-jumbo
Merge bugfix for status.c.
Default to ISO-8601 style ETA output, in order to make it shorter.
Rename -inc:ansi to -inc:latin1.
Use a different ETA format string when less than 24h.
More restrictive valid() in nsldap, salted-sha1 and ssha-opencl.
Comment out dynamic_1028. It was using un-implemented functions...
Improve valid() in all MSCHAPv2 and NETNTLMv2 formats.
MPI vs. node/count fixes.
Bugfix for MSCHAPv2.
Node support for Markov.
doc/MARKOV note about UTF-8 filter.
Replace a dropped MPI header from wordlist.c.
Mute some warnings JimF were seeing about unintialized variables. They
are bogus from what I can see but maybe some compilers get dizzy
from the gotos.
Wordlist.c bugfix after core merge. --pipe was trying to fseek stdin.
Wordlist.c bugfix after core merge. --pipe must be treated as a
per-thread memory buffer.
sse2i_winfix.pl: Sort underscore macros, as they otherwise come in
random order from the hash, which is confusing when inspecting
diffs.
Minimize unneeded differences between Jumbo and core.
Wordlist fixes.
Status.c post-merge fixes for Jumbo.
Wordlist.c fixes after merges.
Wordlist.c fixes.
Node/fork/MPI fixes.
Post merge fixes. Some unifying of MPI vs. node/fork.
Do not close stdout under MPI. Until now, all nodes except root node
did (legacy from original patch).
Mute Keccak warnings on AVX builds.
Mute Keccak warnings on XOP and other builds.
Hopefully mute last Keccak warnings.
Unify MPI with fork. Still not tested at all.
MPI vs -fork fixes.
MPI fix. Now hopefully, "mpirun -np x" works the same as "-fork=x"
when used with -nodes option.
Warn about MPI save/restore being busted for now.
MPI vs. -fork fixes. Now might actually work, including resuming.
MPI documentation updates.
Fix for -fork vs. -max-run-time.
MPI: Read/write .rec file compatible with --fork.
MPI documentation: We can now start a session with MPI and resume it
with -fork, or the other way round.
MPI: Ensure that a resumed session runs on the correct number of
nodes.
Avoid #pragma warnings from clang et al.
MPI vs. fork fixes, and documentation.
Fix GPU builds after recent core merges.
MPI: Drop obsoleted output.
Revert "SIP valid() fix."
MPI: If OMP_NUM_THREADS is set, we assume the user knows what he is
doing. Here's how to pass it to remote hosts: mpirun -x
OMP_NUM_THREADS=4 -np 2 -host ...
Tune a couple of "FIXME" OMP_SCALE macros.
Add lanman.chr made from Rockyou (repaired, upper-cased in Unicode
stage, converted to CP858 [superset of CP437] and split at
length 7).
Wordlist: Fix a division by zero bug in get_progress.
Copyright blurb.
Minor cosmetical fixes to bench.c.
Bugfix in agile.
Fixes for generic make target (more needed).
raw-sha512-ng: Fix for 32-bit builds.
Update benchmark-unify. Some formats are not very comparable, eg.
Office, that got support for 2013 which is much slower. But I
added them.
Drop some no longer used MPI stuff.
Raw-sha1-ng: Fix for clang.
Typo in john.conf, and an added fflush(stderr) for MPI.
VMS: Better valid().
Drop unused variable from VMS.
Merge recent sxc fixes to sxc-opencl.
Merge a whole bunch of fixes made to CPU formats, to the OpenCL ones.
memory.h fix for generic builds.
Fix "testpara" make targets.
MPI fixes. After latest merges we had the MPI vs. OMP checks made
twice.
Tweak SHA1_SSE_PARA for latest ICC. 3.5% faster.
MPI fixes (output only on root node).
MPI: non-root processes now lock main rec-file in shared mode at first
read.
testpara32 fixes.
MPI: If more than one node, all must use LOCK_SH.
Merge branch 'master' into bleeding-jumbo
Document the new argument handling in rec_lock() and
rec_restore_args().
MPI: We no longer close stdout for non-root nodes, so we can output
cracks to it just like other builds do.
Override crk_help() message under MPI when applicable.
Do not use CommonCrypto for SSE2 builds.
Jumbo code audit: Remove unnecessary differences from core tree. Drop
some EOL whitespace. Update some documentation. Fix an OpenCL
 problem introduced by latest core merges. Use UTF-8 encoding for
doc/RULES as it contains 8-bit characters. Drop some deprecated
information from doc/MARKOV (though we'll keep supporting it for
a while).
Bugfix for the avoid-CommonCrypto patch.
Variable changes in mscash formats, to avoid function name clashes
that seem to be the result of defining _DARWIN_C_SOURCE.
Variable changes in PDF, Tiger and Whirlpool formats, to avoid
function name clashes that seem to be the result of now defining
_DARWIN_C_SOURCE.
Updated doc/BUGS for bleeding.
Rename AIX format so they reflect their well-known tags.
Move a Jumbo tweak from signals.h to common-opencl.c in order to mess
as little as possible with core.
Raw-sha256: Support Cisco type-4 hash format. This is made in
prepare() so that any .pot entries are compatible between the
formats.
Bug fixes for previous patch.
Do not include os.h from signals.h. Instead, ensure it's included
earlier.
Make all definitions in EPI_fmt static.
Make MD5 helper functions in sse-intrinsics.c static and MAYBE_INLINE,
and drop unused debug/sse_debug() stuff.
New intrinsics .S files after recent change. 4-5% faster MD5crypt.
Re-enable NT format, that was MIA since 7112446.
Typo in README-OPENCL.
Fix a mistyped #error pragma, although it sort of worked as intended
anyway.
Hand-edited sse-intrinsics-32.S to fix an issue the sse2i_winfix.pl
does not catch yet: Labels that have ".." in them. Apparently we
can't use ".." in a macro name (for -DUNDERSCORES) so these
labels need to be renamed throughout the file. TODO: Fix
sse2i_winfix.pl so can do this in a generic way.
Add a missing linefeed in status output for --stdout.
listconf.c: Do not check for fmt_default_valid. It does not exist
anymore.
bash-completion: Drop trailing whitespace.
bash-completion: Redirect dynamic.conf errors to /dev/null.
The StatusShowCandidates config option showed "combinations" until
now, this changes it to use "candidates" in the format "43210p"
as we now have that figure handy.
The StatusShowCandidates line was accidentally dropped from john.conf.
Optionally reject printable binaries.
MPI fix for -reject-printable
Add --verbosity=N option. N is 1-5 and defaults to 3. For now, level 1
will mute printing of cracked passwords to screen. Level 4 will
add some extra output from OpenCL and level 2 will mute some
other. More to come.
Mute --stdout candidates with --verbose=1
ShowCandidates fix (cosmetic).
DES-opencl: Rename to descrypt-opencl, and new-style algorithm name.
Plus some trailing whitespace fixes that emacs did for me as soon
as I touched the file.
Dynamic: missing paren in BE code.
OS X specific build instructions.
Incremental: Correct progress for multiple nodes.
MPI: Tweaks for file locking problems during restore.
raw-SHA256-ng: Fix OMP bug in vanilla SSE2 code path.
Add ULL to large numbers in SHA512 code, muting loads of complaints
 when building 32-bit.
Update .mailmap for git shortlog.
Update CHANGES-jumbo.git and CREDITS-jumbo
New charset files (because charset.c changed). Also a new Jumbo-only
"Alnum-case" incremental mode, that is upper+lower case and
includes the space as well.
Enable OpenMP for NT2 format. Works fine on intel i7, not so good on
AMD.
Add a linux-X32-native make target (for X32-ABI). For this to work we
also change ARCH_WORD to "long long" in x86-64.h.
Rename linux-X32-native target to linux-x86-X32-native.
Fix a problem when resuming forked or MPI sessions that use the
--encoding option.
Move crypt(3) format to lowest precedence.
Document some requirements in doc/INSTALL.
Possible bug fix for
http://www.openwall.com/lists/john-dev/2013/05/27/20, thanks to
Claudio.
MPI: More consistent exit handling.
Update MPI docs. Also, MPI was accidentally enabled by the last commit
so this is fixed too.
Drop all uses of alloca() and variable size arrays.
Fix bugs in LUKS format. Allocations did not account for padding.
Drop HAVE_ALLOCA_H from os.h since we are not allowed to use alloca()
at all.
LUKS: bug fix in valid().
Oracle11: Alignment fix.
hmac-sha-2 formats: Alignment fixes.
Document the OSX krb5-23 problem.
Avoid using GCC-specific pragmas on compilers that don't grok them.
100% white-space patch. I omitted dynamic as Jim's working on that
one.
100% white-space fixes (GPU kernels).
Change name of bf-opencl to bcrypt-opencl, like the CPU format.
Adopt to new incremental modes (incomplete).
Adopt to new incremental modes.
Use official lm_ascii.chr from core John. The rest of the
non-Jumbo-specific were already identical.
Reduce OpenCL formats verbosity, if requested.
xsha512-ng-opencl: Alignment.
Change --raw-always-valid (et al) to --bare-always-valid to avoid
confusion.
Documentation updates.
Add a line to --list=hidden-options:
Fix a cosmetic bug when listing formats with wildcards that match
dynamic.
Update doc/MARKOV. The examples are updated for version 1.8 output of
John (which incidentally simplifies the calculation part a lot as
we now get a proper p/s figure directly).
doc/MARKOV, change s/cracked/tried/ in a couple of places. They are
not synonyms...
doc/MARKOV: Yet some minor adjustments.
Even more changes to doc/MARKOV: We can use the -max-run-time option
instead of trying to find a proper markov level for measurement.
mkvcalcproba: Support reading from stdin.
genmkvpwd: Change %lld to %llu for correct printing of large numbers.
Fix progress calculation for wordlist mode without rules, when
memory buffer is in use.
Fix a couple of problems with raw-sha1-opencl. And while at it, bump
 max length to 55 and use "compressed" key buffer for a nice
boost.
NT2 format: Only use OMP+SSE2 for AVX or XOP. Older CPUs won't do
well.
Dynamic OMP bugfix.
Wordlist.c: Fix for status overflow w/ millions of rules.
MPI: Fix cosmetic problem - help message printed several times.
Drop the word "intrinsics" from algorithm names.
FORMAT_NAME vs. ALGORITHM_NAME fixes.
Fix ALGORITHM_NAME for mscash2_opencl again, after Sayantan ignorantly
reverted it.
Fix alignment for all formats that had my name in them for any reason.
Lots of algorithm_name vs format_name changes.
DES-opencl bugfix. It was using atexit() instead of registering
done().
Use fmt_default_binary_hash_N() in more formats.
Fix a problem when using -fork with an MPI-enabled build. We
initialize MPI before even parsing options, so if/when we are to
fork later on, we need to tear that "1-node MPI session" down
before forking, or the forked MPI world will result in all sorts
of funny business. Tricky stuff.
Wordlist.c bugfix: Do not try to split-load a too small file into
buffer.
Add repeats16 and repeats32 external modes (from the obsolete contest
branch).
More use of fmt_default_binary_hash_N().
Even more use of fmt_default_binary_hash_N().
Correct MPI node number in pexit() message.
Minor cosmetic fixes and code cleanup.
Replace a check in gpg-opencl valid() that I accidentally dropped in
ddb17ef
Replace a check in gpg-opencl valid() that I accidentally dropped in
ddb17ef. Hopefully proper this time.
Update CHANGES-jumbo-git and CREDITS-jumbo
Add a comment in ext_init explaining a confusing bail-out.
Warn when resuming an external mode lacking resume(). For modes
that actually doesn't need any restore handling, add a no-op
resume() function to mute the warning.
Add a restore() function for KDEPaste. Very slow but better than
nothing.
External mode now refuses to resume unless a restore() function is
present.
Minor optimizations and corrections in 7z.
Endianness fixes in Blake and Keccak.
Set FMT_UNICODE in 7z formats, so we don't produce false negatives.
Bad wordlist.c bug fixed, triggered when running -loop with fork/MPI.
Make Incremental and Rules optionally less chatty.
Mitigate compiler warnings from last patch.
Proper Unicode (and codepage) support for 7-zip format.
Rename scrypt to django-scrypt, as we have more generic scrypt
inbound.
NT-opencl: Copy prepare() from CPU format. Without it, it did not
handle pwdump format files.
All NT formats: prepare() tweak for bare hashes.
WPAPSK-opencl: Change auto-tune so we target a loop kernel duration of
max. 200ms instead of a total crypt_all() duration of max. 10 s.
This makes for a 16% boost on my laptop, 1% on GTX570 and no
change on 7970.
KWallet bugfix for long keys. Kudos to Private2@GitHub.
Change django-scrypt to use the new scrypt code enhanced by Solar.
NOTE there may be more changes needed, I just made it build and
test OK. Are we using SIMD now as-is? Should we use escrypt_r()
instead?
Fix generic build after introducing fmt_default_binary_hash_N
functions.
Revert changes to DES, BSDI and LM formats from 831abd7. Never change
core files unless you really know what you are doing!
Drop incrementals "try" counter, which was naively updated at every
call to crk_process_key(), in favour of the now existing
status.cands which is better implemented. This also means I
could drop Jumbo-specific code for restoring progress -
status.cands is already handled by core.
Bugfix: The "3/3" batch mode output was mistaken for a percent figure.
The OpenCL initialization called from john.c is made passive, so it
doesn't wake up GPU devices unless needed. Queue and context
creation is postponed until opencl_init_opt().
Refactor some function names of common-opencl.c (prefix with opencl_).
Add some comments and drop some obsoleted stuff.
Do not initialize OpenCL devices more than once.
Fixes for mscash2-opencl. The existing code was bogus in several ways.
Bugfix for pbkdf2 unsplit OpenCL kernel. Bug may have affected several
formats where salt length was larger than 23. Thanks to Marcus
Desto for reporting.
Rename the super confusing SHA2 macro in pbkdf2-hmac-sha1 to
"SHA1short". Also apply the recent bugfix to CUDA and CPU code
too.
Fix CUDA pbkdf2 too.
raw-md5-opencl bugfix: You can't release buffers that are not
allocated.
No OpenCL initialization whatsoever until opencl_init_opt().
Fix for opencl_done() when not initialized. I thought it would work
as-is.
Add a 'hidden' build target macosx-x86-64-native-gpu.
Workaround for crappy OpenCL drivers (that don't understand compiler
options that are mandatory).
pbkdf2-hmac-sha256-opencl: Simplify cmp_all(). We don't need to test
more than 32 bits, the rest are checked in cmp_one() and this
will happen very infrequently.
Warn when b0rken device is skipped.
Various minor fixes for Sayantan's OpenCL versus OS X.
EFS bugfix: This is a Unicode format. Also make a local function
static.
Proper --encoding support for EFS. Also fixes a bug with uninitialized
data.
Improve comments in unicode.h.
Comment on a weird line in options.c that looked like some code fell
off.
Jumbo uses a shorter tab stop than core for the usage blob.
Revert "Jumbo uses a shorter tab stop than core for the usage blob."
Revert "added mask mode interface"
Revert "nt-opencl: fixed bug causing inefficient keys transfer."
Revert "mscash-opencl: compare on gpu"
Revert "Various minor fixes for Sayantan's OpenCL versus OS X."
Revert "mscash-opencl: optimization patch #3"
Revert "mscash-opencl: optimization patch #2"
Revert "mscash-opencl: optimization patch #1"
Revert "mscash-opencl: optimization patch #0"
Revert "New format added, mscash-opencl: ported directly from
 mscash-cuda."
Revert "raw-sha1-opencl: cmp on gpu"
Revert "raw-md4-opencl:cmp on gpu"
Revert "nt-opencl: use opencl_init instead of opencl_init_opt"
Revert "nt-opencl: cmp on gpu"
Revert "raw-md5-opencl bugfix: You can't release buffers that are not
allocated."
Revert "Raw-MD5-opencl: status check bug fix"
Revert "Update opencl_rawmd5_fmt.c"
Revert "Raw-MD5-opencl: fixed memory cleanup"
Revert "Raw-MD5-opencl: compare on gpu"
Fix after reverts.
Revert "nt-opencl: no branching in coalasced loading loop."
Fixes and bug fixes for CommonCrypto builds.
pbkdf2-hmac-sha256-opencl: Immediate response to keystroke.
pbkdf2-hmac-sha256-opencl: Rename kernel since it's no longer
"unsplit".
Move 7z-opencl to unused/
Update CHANGES-jumbo.git (CREDITS-jumbo needed no change).
Never use CommonCrypto for mscash2. Fixes issue #362.
First draft of {SSHA512} (LDAP-style) format. OpenSSL only for now.
ssha512 format: Add OMP support and a couple more test vectors.
Add missing FMT_OMP flag to ssha512 format.
Fix a bug where -DDEBUG would cause an infinite loop in
mem_alloc_tiny() because MEM_ALLOC_SIZE is set to zero
(Jumbo-specific) and we ended up never breaking out of the while
loop. Thanks to bytebieger for noticing the problem and
patiently convince me.
Fix another potential problem with -DDEBUG vs. mem_alloc_tiny().
Print human readable OpenSSL version string with --list=build-info.
Update CHANGES-jumbo & CREDITS-jumbo.git.
Use the recent mem_alloc_tiny() bugfix for size zero even when not
-DDEBUG because returning a pointer might be asking for problems.
Add session_start_time external variable that maps to time(NULL) at
session start.
Another tweak to the mem_alloc_tiny() vs. -DDEBUG issue.
Bare minimum of fixes to bwtdt so it doesn't break too much.
Add some comments to bwtdt format.
Unicode bugfix for OpenCL Office formats.
Stack smash bugfix for putty format. Bug introduced in 79c871d, using
sizeof len fields (ints) instead of actual fields (arrays),
causing a buffer to be potentially 8 KB too small.
raw-sha1-opencl: Fix bug where obsolete code that only handled up
to length 23 was accidentally used.
raw-sha1-opencl: Drop obsolete code for short length.
Bugfixes due to format labels no longer lower-case.
Fix bug in formats.c where old names for descrypt, bsdicrypt and AFS
were used in strcmp().
wpapcap2john: Add SSID in "user" field for Single mode use.
hccap2john: Add SSID in "user" field for Single mode use.
Always -D_OPENCL_COMPILER when building OpenCL kernels.
Avoid a compiler warning for signed vs unsigned integer.
Whitespace-only in common-opencl.c
Allow calling opencl_get_user_preferences() with NULL argument.
OpenCL formats: Use more shared code from common-opencl.c
Tweaks for LWS/GWS environment vs auto-tune vs defaults.
Fix a couple compiler warnings about "array subscript has type ‘char’"
and other minor things.
Make Oracle 10 format more robust (allow lower-case salts even when
 prepare() is not used).
Fix issue 328 using the new num_internal_keys format parameter.
Issue #328 revisited.
Issue #328 revisited again...
Revert "Issue #328 revisited again..." Revert "Issue #328
revisited."
Bump FORMAT_MAIN_VERSION due to 018a5ac.
Issue #328 hopefully settled.
Under --verbosity=1 we don't print passwords to screen when cracked,
so also mute the warning about them not being the complete list.
Add new masks ?s (specials), ?a (printable ASCII) and ?A (everything
between space (0x20) and 0xff).
NTLMv2: Bump max. salt size (username + domain) from 27 to 59
characters.
OpenCL fixes. Work around silly Intel driver issue with
-cl-strict-aliasing and some other weird issue with late nvidia
drivers not supporting CL_DEVICE_COMPUTE_CAPABILITY_MINOR_NV et
al.
OpenCL bugfix (common code): sequential_id was used where platform_id
was expected. Also a workaround for Apple's brain-dead driver
that apparently dies on dual spaces eg. between -DTHIS and
-DTHAT. That is just ridiculous.
Minor tweaks to common OpenCL code.
Added a function for tracing memory allocations in OpenCL.
Limit GWS in OpenCL formats using packed buffer, so we don't over-run
the 26-bit index.
Bump HASH_LOOPS for RAR OpenCL format to one optimal for Tahiti and
Titan.
RAR OpenCL: Add more profiling output.
RAR OpenCL: Back to 64x buffer
RAR OpenCL: Do blocklen sha1's at once.
RAR OpenCL: Get rid of HASH_LOOPS. Use a 64x __private buffer. Merge
that and IV into the loop kernel. Finally a significant speedup.
Various OpenCL formats: Use byte addressed stores if allowed (though
sometimes not if AMD). For RAR this gave 33% boost on AMD.
raw-sha512-opencl: Obey device/kernel limit for LWS.
Drop old junk from OpenCL kernels.
OpenCL RAR kernel accidentally got max length limited to 16. Bump to
22 again, it does not affect performance now.
When starting an external mode that lacks restore(), emit a warning.
Yet an optimization for RAR OpenCL, and a minor fix for done().
Minor tweak to RAR OpenCL. We now do over 20K on 7970.
MPI: Enable poor man's multi-GPU support. This should be available for
-fork too but it's very tricky to accomplish.
No-op MPI tweak + comments.
Add a self-test that ensures FMT_OMP_BAD is not used without FMT_OMP.
We could consider defining FMT_OMP_BAD so it includes FMT_OMP.
Fork: Enable poor man's multi-GPU support. This is experimental. It
postpones format init and that might have side effects. To mitigate
unnecessary problems, everything is just like before as long as
you don't use both --fork and --device options at once.
Don't cache CPU kernels. There were problems seen on Well's AMD
driver.
Incremental mode may get >100% progress because of uneven splitting.
Show that as 99.99% instead.
Inc progress: For some odd reason we sometimes ended up showing 99.99%
instead of DONE so cap it at 100.0% instead of 99.99%.
Document the "poor man's multi-GPU support".
Bugfix for "poor man's multi-device support" for --fork.
Cosmetic tweak for MPI + OMP notice.
Drop the use of OS_TIMER=0 for AMD OpenCL builds, but add instructions
for how to re-enable it.
wpapcap2john: Ensure we don't overrun SSID buffer with overlong tags,
and print a debug message. Thanks to bytebieger for reporting
this issue. We still do not know the cause. See
https://github.com/openwall/john/pull/366 for
discussion.
Mute a compiler warning for rakp-opencl.
rakp-opencl: Whitespace fixes and rename file according to standard.
rakp-opencl: Bug fixes, optimizations and packed key buffer.
rakp-opencl: Optimizations for HD7970.
Add macosx-x86-64-native-opencl target because latest Xcode broke
CUDA.
OpenCL: Fix for the fork/MPI "poor man's multi-device" stuff, so
formats that support multiple devices natively does not get
screwed up.
Export get_platform_id() from common-opencl.
Fix OSX build targets for CUDA for Xcode 5.0 and CUDA 5.
NVCC tweaks (mostly comments, but add -fmad=true) in Makefile.
pass_gen.pl: Only require libs that are actually needed for specific
run.
Fix an obscure bug in mscash where an overlong key would truncate
to 28 instead of 27 in UTF-8 mode. NT already had a fix for it so
unify the code for them.
Cosmetical fixes in common-opencl.c.
Protect a CUDA macro from being expanded to something not intended.
I'm not sure there was a problem, just saw the dangerous definiton.
Change malloc() to mem_alloc() in a bunch of formats. The latter sits
in memory.c and has error checking.
Align fix to CUDA md5crypt.
Update CHANGES-jumbo.git and CREDITS-jumbo.
Update CHANGES-jumbo.git (the unstable-jumbo part of it).
Add ?h mask. Hashcat defines it as [\xc0-\xff] but we use all 8-bit.
Mask mode: Add progress indicator and node/fork/MPI support.
Mask mode: Minor mods to get_progress().
Mask mode: Add support for restoring session.
Mask mode: A little faster node/fork splitting, avoiding modulo op.
Faster node/fork/MPI distribution for External mode. This changes the
session file format so it will be incompatible, so until we get
that sorted out it's ifdef'ed out and we use the core scheme.
Drop some contest edition artifacts from wordlist.c
External.c: Revise the "no modulo" node distribution patch so it
doesn't break session file compatibility.
Mask.c: Change node distribution so it matches external's new one.
mask.c: Remove redundant initialization.
Fix UTF-8 bug in mscash2 introduced in 42912b39.
sapG: Add FMT_UTF8 even though we are not FMT_UNICODE.
RAKP formats: Improve valid().
Fix buffer overrun in KWallet.
Fix a bug in eleven formats where cmp_one() and/or cmp_exact()
erroneously relied on cmp_all() having been called first. This
is *not* always the case, when bitmaps are used we go directly
to cmp_one(). This leads to FALSE POSITIVES but only likely for
broken hashes that are uncrackable anyway so it wasn't that much
of a problem.
Disable the "cmp_one() assumes cmp_all()" self-test. It needs to be
replaced with something more reliable, or enabled manually at
times.
Add FMT_OMP_BAD to nt2, md4/md5 and tweak Single mode performance for
nt2.
Bugfix for RAKP formats.
Replace Dhiru's RAKP format with a new one based on hmac-sha1 so
inherently SSE-aware. 3x speedup.
RAKP: Add OMP support.
Mute warning about lack of OMP support if format is OpenCL or CUDA.
Bugfix for 087fe65.
Allow use of alter_endianity() on archs that doesn't allow unaligned.
This function was originally only for MMX/SSE2 archs.
Fix a bug in RAKP for interleaved SHA1. Dang, that was hard to nail.
I literally had ONE BIT wrong %-)
RAKP: Optimization for "many salts". About 65% boost.
HMAC-SHA1 format: Optimizations and OMP support.
Add support for RAKP to pass_gen.pl
Fix bugs in pass_gen.pl HMAC generation for lengths over pad size.
RAKP formats: Minor fixes for very long usernames (salts).
HMAC-MD5: Optimizations and OMP support.
Changed SHA1_SSE_PARA to 2 for any gcc >= 4.5.4. Until now it was
1 for gcc > 4.7.0 but that seems to be slightly worse.
pass_gen.pl: Fix b0rken whitespace (tabs vs spaces) most of which was
my bad.
Revise format list in README-OPENCL, fixing issue #376.
Explicitly state that wpapsk formats do support WPA and WPA2. Also
prevent future bugs when doing strcmp() of format.params.label.
Improve valid() in netlm, nethalflm and mssql12.
Activate a modified (and temporary) valid-killer self-test. Move a
bunch of offending formats to unused.
Just because I used RAKP as a fool-proof example, it wasn't.
Improve valid() in krb5pa formats.
Chicken out on the ARCH_WORD change for X32 - make it conditional.
Never use DES BS assembler code for X32 (until it's fixed).
Minor tweaks to krb5pa-md5
UTF-8 bugfix for ntlmv2-opencl.
Add yet another debug function in memory.c, dump_text(ptr, len)
Bugfix for raw-sha1-opencl.
Include missing header for newly introduced isprint() in memory.c
krb5pa-md5-opencl format added.
Makefile bug, I used .c instead of .o which had strange results.
krb5pa-md5-opencl optimizations.
Some tweaks to RC4 to no avail.
krb5pa-md5-opencl: Support devices that can't do byte addressable
store.
Update CHANGES-jumbo.git & CREDITS-jumbo.
krb5-pa-md5 formats: Check another byte of known plain.
Bugfix for --skip-tests, fixing problems with numerous OpenCL formats.
Fix the bugfix in 06094f1b.
krb5pa-md5: Tweak RC4 on GPU, unrolling and working on 32-bits at
a time when possible. Some 50% boost seen here and this should also
make it runnable on devices that can't do byte-addressed stores.
Improved shared LWS test - use unique keys in case it matters for
thread diversion. For krb5pa-md5-opencl this is a 45% boost IRL
while the benchmark gets a hit. Also do a similar thing in local
GWS enumeration of some formats.
krb5pa-md5-opencl: Some more RC4 unrolling.
DMG formats: Merge recent change in CPU format to OpenCL. Use new
shared dump_text() for debugging plaintext.
Revert to using 4x for SIMD SHA-1, for gcc >= 4.7 unless AVX or XOP.
Show preferred vector widths with --list=opencl-devices. How the heck
 did I miss this until now!? It's important information and we
can use the same call to default to vectorization - or not - at
runtime for some formats.
Make office2007-opencl choose vector width according to what device
says is best (up to 8 currently supported).
Make office2010-opencl choose vector width according to what device
says is best (up to 8 currently supported).
Make office2013-opencl choose vector width according to what device
says is best (up to 8 currently supported).
Make wpapsk-opencl choose vector width according to what device
says is best (up to 8 currently supported).
Make krb5pa-sha1-opencl choose vector width according to what device
says is best (up to 8 currently supported).
Deprecate --request-vector option and rename --request-scalar to
--force-scalar. OpenCL formats that support vectorizing will now
automatically detect vectorizing opportunity (by asking device)
and enable it with the recommended width. For benchmarking and
debugging purposes, --force-scalar will disable vectorizing.
Cosmetical changes to ntlmv2 & krb5pa-md5 kernels.
Revert to outputting to stdout (not stderr) for dump_stuff*()
functions.
Vectorize rakp-opencl and choose vector width according to what device
says is best (up to 8 currently supported). Also drop old
crappy find_best_kpc() function. We should use the shared one
but this format is not very critical on GWS anyway.
Update README-OPENCL regarding vectorized formats.
Add some device limits considerations in various OpenCL formats.
Add even more vector details to --list=opencl-devices.
Fix cosmetical bug in ntlmv2-opencl & krb5pa-md5-opencl.
Add __attribute__((vec_type_hint())) to vectorized kernels.
RAKP-opencl: Use partial results transfer.
Total revamp of mysql-sha1-opencl. Most notably we now use a packed
key buffer. Also sync test vectors between CPU and OpenCL
formats. And finally give them both a split() that unifies case
(to uppercase) which fixes a long standing bug.
MPI vs. --fork bugfix: When running forked, we erroneously called
nice() even when configured not to.
NTLMv2, krb5pa-md5 and MYSQL-SHA1 OpenCL formats: Change a workaround
for self-test inconsistency, and comment it.
Better description of MySQL formats vs. versions.
Add ciphertext example output with --list=format-[all-]details. And
change a bunch of formats so they produce that output without a
tag, fixing some bugs while at it.
Add "DCC" to description of mscash formats.
Assorted bugfixes from one-off temporary self-tests. Alignment and
split-unifies-case problems in phpass-cuda, mscash-cuda and raw
Keccak formats.
Drop unused declaration from NT formats.
mask.c bugfix when restoring string containing whitespace.
More effective vectorizing in wpapsk-opencl (avoiding conversions in
inner loop).
More effective vectorizing in krb5pa-sha1-opencl (avoiding conversions
in inner loop).
Add a global bool "ForceScalar" in john.conf that turn off vectorizing
globally in case you have a weak device that give "improper"
hints for our use (eg. if a side-effect of vectorizing is
register spilling).
Add a --force-vector-width=N option (for debugging) and add support
for all sizes to the vectorized OpenCL formats.
Shorten the format name of krb5pa formats.
Fix an over-allocation in krb5pa-sha1-opencl when running vectorized.
WPAPSK-opencl: Vectorize last bits of kernel. Write vectorized
keybuffer already in host code. Much nicer code.
Protect from future bugs using snprintf() instead of sprintf().
WPAPSK-opencl: Minor cosmetically changes unifying with krb5pa-sha.
krb5pa-sha1-opencl: Vectorize last bits of kernel. Write vectorized
keybuffer already in host code.
RAKP-opencl: Clean up code using macros.
Don't even try to support a vector width of 3. I can't see it being
worth the effort.
OpenCL vector code cleanup using macros.
NTLMv2-opencl: Vectorizing support.
Update README-OPENCL.
More OpenCL code cleanup using macros.
NTLMv2-opencl: bugfix for device limit consideration.
Common-opencl: Change an ambiguous function name.
Re-enable vector width 3. Most problems with it not working is because
our LWS/GWS enumerations pick unsuitable figures for that width.
OpenCL formats: Use shared functions.
RAKP: Use lower default KPC so weak devices don't die on us.
RAKP OpenCL tweaks.
common-opencl.c: Make it use its own helper functions w/ available,
plus a bunch of whitespace fixes.
RAKP OpenCL: Implement proper GWS tuning.
OpenCL formats: Use size_t instead of int in a bunch of places because
of problems where eg. a GWS of 32M x buffer size of 64 ends up
in 2G which doesn't fit in a signed int.
Duration limit tweaks to some OpenCL formats.
Add opencl_get_vector_width(device_id, size) function to
common-opencl.[ch] and have all vector-capable formats use it.
New OpenCL LWS/GWS enumeration scheme: 1. Enumerate GWS using *LWS
== NULL. 2. Enumerate LWS using the now known GWS.
md5crypt-opencl: Shrink arrays a little, they were not fully used.
And do not rely on optimizer taking away the redundant copy in
md5_digest(). 1-10% boost seen depending on device.
Make generic split pbkdf2-hmac-sha1 kernel more generic: - Support
arbitrary output length unless -DOUTLEN. - Support run-time
iterations count unless -DITERATIONS (a fixed one will make
for a little faster code).
Encfs-opencl: Use the generic split pbkdf2-hmac-sha1 kernel and
auto-tune to device.
mysql_netauth: Add a split() that unifies case, fixing issue #394.
Speed up self-tests significantly for OpenCL formats.
A couple of OpenCL formats needed workarounds for self-tests not
doing things like IRL.
Tweak for Intel HD Graphics in --list=opencl-devices. Apparently they
don't have a notion of SP, just EU.
More fixes after changing self-test scheme for OpenCl. Some of these
fixes only work around self-test inconsistencies but some were
real bugs.
Speed up self-tests for CUDA too.
Add -DJTR_RELEASE_BUILD which can change some defaults for releases.
Currently it's only used to disable the hacky backtick-auto-conf
stuff in Makefile.
Add self tests that ensure min/max keys per crypt are sane.
OpenCL raw MD4/MD5 formats bugfix for overtaxing key index.
OpenCL raw-sha256 & raw-sha512-ng workarounds for self-test
inconsistency.
raw-sha1-opencl: Bugfix for key index max size.
Fixes for building CUDA on OSX Mavericks. Hopefully it doesn't break
other versions/OS's (it shouldn't).
CUDA fixes, in hope it will work on both OSX and Linux now.
Very silly bugfix: OSX Mavericks' OpenCL drivers crashed and burned
when descrypt kernel sourced this header file, just because its
last line lacked a line feed!
Rename ripv2 to net-md5 and add a very similar net-sha1 format.
See issue #404
Makefile CUDA: Add natural path to nvcc with an NVCC macro, and
make the OSX -ccbin quirk a macro too.
CUDA fix for when nvcc is not in /usr/local/bin.
Fix bfcrypt-opencl for devices with too little local memory (auto tune
down or fallback to using CPU kernel).
Change some ambigous OpenCL function names:
get_current_work_group_size() -> get_kernel_max_lws()
get_max_work_group_size() -> get_device_max_lws()
Fix openssl-enc valid(). Also includes Awesome Debug Macro[tm] :-P
More fixes for bcrypt-opencl. It couldn't handle things like LWS=7.
DES-opencl: Get rid of hard coded stuff and fix device auto-tuning.
Update doc/BUGS.
bcrypt-opencl: Ensure GWS is multiple of LWS.
mask.c get_progress() bugfix: It was sometimes reading uninitialized
data, resulting in random progress output (typically 0.00%).
mscash2-opencl: Proper Unicode support.
des-opencl bugfix: Do not release kernel unless it was initialized.
Not sure if this was a driver bug here, but it won't harm
anyway.
Fixes and added info to --list=cuda-devices.
Cosmetical fix for --list=cuda-devices: Clock rates are base 10, not
base 2.
This is the very definition of a cosmetical fix.
Make rar-opencl count accelerators as GPU not CPU.
rar-opencl: cosmetical.
A small step towards a working mingw64 target.
mscash-cuda: Proper Unicode support, and bug fixes.
mscash2-cuda: Proper Unicode support, and bug fixes.
Revert a terrible change Sayantan made in ebc6a5ee. mscash2 supports
variable iterations count. Instead of b0rking the CPU format you
should fix your GPU formats.
mscash2-opencl: Minor Unicode bug fix, and some other fixes.
Add ALIGN_FIX in x86-64.S for MinGW win64 builds.
Typo in asm comments.
mscash2 formats: Add test vector for non-standard iterations count.
Minor change for pwsafe-opencl for use with Intel OCL SDK 1.5.
rar-opencl: Avoid using global memory in inner loop, and coalesce
state buffer.
wpapsk-opencl: Use a coalesced state buffer. 2% boost.
Drop ALIGN_FIX from win64 target and add it in a comment instead. It
is supposed to be applied manually if your build crashes in
DEScrypt or NT.
Minor tweaks and optimizations to new RIP/OSPF/BGP formats.
Fix some weird code in NT_fmt.c (purely cosmetical).
I made some curious testing with the classic "swap bytes without a
temp" in OpenCL RC4. It seemed slightly faster on Kepler but
slower on others.
Apparently Microsoft uses a different x64 calling convention. This is
an attempt to fix x86-64.S. We now use _WIN64 instead of
__MINGW64__ in case someone tries to do things with MSVC or
 anything else.
Unify MAYBE_INLINE definition with core john.
http://www.openwall.com/lists/john-dev/2013/10/29/2
Drop the -mstackrealign from win64 build. It was not a correct fix.
And change 'strip' to $(STRIP) for easier disabling (define it as
'dir' or 'echo' or whatever when making a debug build)
Makefile: Update build alternatives descriptions.
Win64: One more non-volatile register was used. We redefine it for
_WIN64 to avoid preserving it.
x86-64.S: More redefinitions of registers because the previous patch
introduced a register clash on Win64.
Avoid use of "long", and similar fixes for Win64 and X32.
This might be what is missing for Win64 in DES_BS_ASM.
Linux-X32: Add an SSE2-only build (for testing) and finally solve
the assembler problems after I warmed up with the Win64 stuff.
Also more cosmetical tweaks to the make target list. Oh, and we
rename the targets back to linux-X32 as opposed to linux-x86-X32.
This passes the Test Suite.
Another try for Win64 vs. BSDIcrypt.
x86-64.S: Recent changes gave a theoretical performance hit due to
using r10 as tmp2. This patch rectifies that.
Revert "x86-64.S: Recent changes gave a theoretical performance
hit..." That was totally bogus. R8 and up have the extra
penalty. Add some comments instead.
x86-64.S: Fix a bug in Win64 handling where we redefined 'iterations'
(defined as %edi) without changing a corresponding push/pop of
%rdi.
x86-64.S: Code cleanup, revert to using unprefixed temp registers,
and overall less deviation with core.
Fix for 913e2fce - it broke 32-bit MMX/SSE2 builds.
RAR (CPU): Bump plaintext_length to max as this does not affect
performance.
Revert 40f5a93 and instead add a test for icc.
Move broken (to be fixed) formats from unused/ to broken/ and
delete some other old crap from unused.
Citrix Netscaler 10 format added.
Minor fixes to raw md4/md5/sha1 formats.
Improve Single mode seed from rar2john
Common OpenCL: Move variable declarations to C file and use external
declarations in header.
Proper vectorizing support for shared OpenCL functions.
Various OpenCL bug fixes for running vectorized.
OpenCL: drop the "show_speed" logic in favor of options.verbosity.
OpenCL formats: Use options.verbosity instead of other strange
mechanisms.
Rename "kpc" to "gws" in RAKP OpenCL to avoid confusion (maybe it just
adds to the confusion but at least it's in line with other
formats =).
JtR coding-style fixes.
OpenCL, shared auto-tune bugfix for vectorized runs.
OpenCL: Last case (I think) of dropping legacy magic in favor of
options.verbosity
OpenCL: reset opencl_v_width in case we load another format later.
OpenCL md5crypt bugfix.
RAKP-OpenCL: Always enumerate work sizes.
Use shared auto-tune for raw-sha1-opencl.
OpenCL: Faster auto-tune.
Add profiling timer resolution to --list=opencl-devices.
OpenCL autotune tweaks.
Better wildcard support in format names. Mostly for testing. Examples:
-format:*office -format:office* -format:raw*ng
Trivial optimizations for ssh-ng, 20-25% boost for AES, less for 3DES.
Multi-device bugfix for pwsafe-cuda.
pwsafe-cuda: Fix a compiler warning.
Silently ignore a missing john.local.conf.
Mute [incorrect] speed figures for --test=0.
Cosmetical change in mscash2. Code was hard to follow due to reuse of
a variable that the optimizer will take care of anyway.
mscash2-cuda: Support non-standard iteration counts. Funny thing, I
unknowingly fixed it in b7023df - it was a problem in prepare().
Use CPU format code in CUDA mscash2 - and fix a bug in the CPU format
that prevented it from loading certain format hashes. This
affected the OpenCL format too.
Fix a really nasty bug in mscash2-opencl introduced in e3289d3 which
made all salt lengths of exactly length 22 result in false
negatives. Test vector added to prevent this in the future.
Fix a bug in mscash2-cuda where UTF-8 salts would produce false
positives. We should to enhance the Test Suite to test this. The
CPU and OpenCL formats were manually tested too and they are fine.
Document recent --test=0 and wildcard features.
Drop -cl-strict-aliasing from OpenCL build options. It's deprecated.
Revert "added new format param"
Revert "Bump FORMAT_MAIN_VERSION due to 018a5ac."
Adjust unused RAKP format for FMT_MAIN_VERSION 11 in case someone
needs it.
Enable all test vectors for OpenCL and CUDA md5crypt.
Cosmetical typo in common-opencl.c.
Generic crypt: Allow setting actual salt with --subformat, and create
test vectors on the fly.
Generic crypt: Add support for SunMD5.
Enable use of SIGUSR1 for non-MPI builds too. It's weird having
different functionality. The need for SIGUSR1 with MPI comes
from mpirun's ability to pass SIGUSR1, but not SIGHUP, to the
processes.
Suppress "Press 'q' or Ctrl-C to abort.." message if we are reading
from stdin.
Print a warning when FMT_NOT_EXACT.
WPAPSK and other PBKDF2-SHA1 OpenCL kernels: Try to support old
devices (OpenCL 1.0). I don't have any so this is not tested.
Revert "WPAPSK and other PBKDF2-SHA1 OpenCL kernels: Try to support
old devices". This reverts commit f7189d8. It did not help.
Fix a typo in previous merge
Call common_init() before listconf_parse_late(), fixing a problem with
upcoming --list=format-tests.
Crypt format: re-add format list for bash completion.
Fix a (Jumbo specific) off-by-one bug in crypt format.
Crypt format bugfix (portability issue).
Use crypt_r for self-made test vectors in crypt format, if applicable.
Bugfix in raw SHA384/512 formats: Sign extension problem.
Use memset() instead of bzero() in Lotus85. bzero() is deprecated.
Crypt format: More workarounds for varying crypt() implementations.
I should get myself a Linux box too. Blind fix for Linux OMP in c3.
Better behavior for eg. "./john -t=0 -form:dynamic -enc:utf8" (mute
complaints).
Support wildcards in dynamic formats, eg. -form:dynamic_3*
Print PCI device id's in hex.
Add generic pbkdf2-hmac-sha1 format, variable rounds and lengths.
Due to a bug somewhere (else) you don't get good SSE speed unless
 building for OMP (the latter is much faster even when running just
one thread).
Add ios7tojohn.pl and docs for it.
pbkdf2-hmac-sha1: Bump OMP_SCALE for better multithreading.
pkzip: Bugfix for #467. Two early rejection tests had to be disabled.
Hopefully they can be re-enabled by JimF after some tuning.
Makefile fixes (drop PROJ_CXX), and remove old C++ code.
pkzip format: Add a test vector with the hash from issue #467 for
future enhancements.
--list=format-all-details et al: Handle formats that lack test
vectors.
Add run/luks2john to .gitignore
Bugfix and tweaks to gpg2john
gpg2john: Use "Real Name" from key user data for login field, and the
rest for gecos field. As last resort (no user data) use base of
filename as login field.
Add note in BUGS about Intel OCL SDK 1.5, fixing #400
Update CHANGES-jumbo & CREDITS-jumbo
Make a warning() function in common_opencl_pbkdf2.c static.
mscash2-opencl: rename warning() to dcc_warning to avoid present and
future clashes.
self-tests: When valid() fails, print corresponding ciphertext.
Apparently, Windows does not provide SIGUSR1 so check for it before
using it.
Change description for pbkdf2-hmac-sha512 from "OS X 10.8" to "OS X
10.8+"
Update doc/INSTALL. OS X Mavericks can build macosx-x86-64-native
without any external tools or tricks.
Typo in doc/INSTALL
bench.c: Avoid a silly compiler warning on OSX that I won't bother
getting to the bottom with.
Prefer COMPUTE over DISPLAY (for AMD OpenCL). This means you can
eg. export COMPUTE and then run "ssh -X" without hassle. Also,
assume :0 if lacking both.
OpenCL: stdlib.h needed for setenv() on some systems.
OpenCL: _BSD_SOURCE needed for setenv() on glibc.
Fixes krb5 formats for issue #495.
Move (win)zip formats back in business. We now have warning output in
place for the false positives (04514d7). This does not solve
issue #434.
crypt(3) format: Only allow --subformat for benchmarks. Fixes #475.
Mask mode: Since we do not yet support --min/max-length, error if
used. Fixes #380.
gpg-opencl: Sync with CPU format. Fixes #482.
EPI alignment. Fixes #491.
Fix an old bug with CPstoreUTF8 vs. split hashes and --show. Some
cracked hashes with passwords containing 8-bit characters would be
output as partially uncracked in case CPstoreUTF8 was enabled.
Add a shared function testing a string for valid UTF-8.
Add a john.conf option for default input encoding.
Better detection of alternate help message for MPI.
ETA/progress fixes for incremental and mask modes when we are doing
more than one node's part of workload.
Minor fixes for ETA/progress for Incremental & Mask modes.
luks2john.c, use %zu for sizeof(). Closes #500.
Dominosec format: Add hash functions, partially fixing issue #495.
loader.c: Add ifdef'ed hash table distribution debug output. Closes
#496.
Dominosec: Use prefab binary hash functions.
EPI format: Add proper hash functions. Closes #495.
Move unused/aix_smd5_fmt_plug.c into normal source directory - it
supports length up to 125.
SAP B: Atom found a suspected over-run of sum20. This triggers that.
SAP B (comments only): The sum20 overrun is AOK, we do it like SAP
does.
Various changes for DMG. Avoid false positives. Hopefully we're not
introducing false negatives (None of our sample hashes produce any)
because now we're only checking for eight consecutive nulls,
either in main blob or secondary one.
Bugfix for multi-node Unicode init.
wpapcap2john: Print client BSSID to uid field, and AP BSSID to gid
and GECOS fields (in dashed and compact formats). Add WPA or WPA2
to the shell field and filename to last field. Add missing glue for
dupe suppression. Finally add a license blurb. Closes #516.
wpapcap2john: Add some sanity checks. Closes #518. Also cosmetical
fixes (license, whitespace) in WPA source files.
hccap2john: Add same extra output that was recently added to
wpapcap2john (BSSID's, filename etc) and handle multiple files
better (don't abort on error, print filename among warnings).
Closes #521. Also, support multiple entries in each file. This
was a weird limitation but it might have been due to Hashcat
legacy behavior (it initially only supported one per file).
Anyways it's lifted now.
wpapcap2john: Add support for Radiotap encapsulation. Closes #377.
wpapcap2john: Add support for Prism & PPI encapsulations.
wpapcap2john: Add a note in output about unsure authentications
(still crackable but might be an incorrect password for that AP).
Closes #519.
Cherry-pick 2475121 from Dhiru - Travis/Coverity support. Closes #512.
White space fixes (fixing over-long lines) in core files.
"Secure Mode" implemented. When this is enabled in john.conf, no
cracked password is ever printed, logged or stored in plaintext.
Instead you get an indication of length and use of classes. For
example, the password "Ignit3" would output as L6-?l?d?u telling
you that it's six characters long and consists of lowercase, digits
and uppercase characters. This closes #505.
The wrong default was used for SecureMode when not found in john.conf.
Sybase PROP fixes. Seriously, you can't expect a char to be aligned at
4 and an "XXX" comment is not a valid workaround.
Have --fork use SIGUSR1 for status, just like MPI always did. Ditch
some old MPI code as it can now use the same. This is to free up
SIGUSR2 for upcoming "restart" patch.
DMG formats: even more debugging
Always start a resumed session by emitting status.
Reload upon SIGUSR2 or --reload-every=N seconds. A reload is currently
the same as stopping a job and then resuming it, it's not very
optimized. Still, it can be invaluable eg. for forked sessions
with salts. However, the current implementation (execv) is not
supported under MPI, which is a major problem.
DMG-opencl add missing variable from 74ac62a.
Minor bug in "reload" patch: Some output to stdout was supposed to
go to stderr.
Cleanup of old weirdnesses: JimF has/had the habit of misusing the
loader options struct for options totally unrelated to loader,
and/or have applicable entries duplicated in that as well as the
normal options struct. Also fixes some over length comments.
Drop FMT_OMP from wpapsk-opencl: It doesn't use (nor need) OMP
anymore.
If we are a release build, only output OpenCL build log if there was
a fatal error (or --verbosity was increased) - not just for
warnings.
Fix torrents of warnings seen on recent OSX drivers for various OpenCL
kernels. This also works around several driver bugs!
Various OpenCL formats: Auto-tune bugfixes when LWS is supplied by
user.
Work around an OpenCL bug in OSX 10.9.2.
Fix for crude auto-tune picking a far too high value on Iris (Intel
HD Graphics 5000).
OpenCL: Add HD5000 info.
ARCH_WORD: Always use "long long" for LLP64 and "long" for LP64. We
could currently use "long long" for all of them but the future may
be different.
WIP: Add GPU sensor readings to --list=cuda-devices and
--list=opencl-devices as well as to status output, if required
library can be dynamically loaded.
Change the "reload" feature to actually just load what other processes
wrote to pot file, instead of aborting and restarting
completely. This is a whole lot faster and enables support for
MPI as well.
Add an ifdef around flock() for MinGW. This shouldn't result in
reading of an incomplete last line and even if it does, other
code should safely handle it anyway.
Coverity stuff update.
Various signal.c tweaks for MPI: Avoid signal loops when signaling
parent for reload. If running Cygwin w/ fork or MPI, save session on
key press in case a following termination does not save
properly. Also clarify MPI requirements in doc/README-mpi.
MPI: Add signaling of cracked hashes. There is also various changes to
the handling of --fork sessions in this regard.
Drop the experimental --reload-every=N option. We always reload at
session save (unless totally disabled in john.conf).
Do not re-read pot file lines that were processed by initial loading.
Drop an unneeded strnzcpy() during reload of pot file. Also adds debug
stuff to crk_reload_pot().
Pause/abort on filesystem stimuli. This defaults to
/var/run/john/abort and /var/run/john/pause but can be
reconfigured to any other files.
Recent structural changes and new features introduced some terrific
bugs. This hopefully fixes most of them.
Speed up pot reload (a lot) by using bitmap.
"Reload" code cleanup and optimizations.
Less timid --max-runtime: We now give a 3 sec grace period and then
force quit as if ctrl-c was pressed twice.
Keep loader's salt hash table for pot reload (updating it w/
required).
The documentation of --save-memory was duplicated in doc/OPTIONS.
Don't keep salt hash table if --save-memory=2 or greater, or if the
format doesn't have a salt_hash function anyway. Update salt hash
table after sorting (if applicable). Add some safety measures
to pot reload.
Add some OSX building tips to docs.
Pot reload: In Single mode there may be dupes, so in that case we
search for them.
Add options to john.conf for defaults and batch mode - Single mode
rules - Wordlist rules - Incremental mode
Various minor tweaks, and Jumbo code cleanup in loader.c (overlong
lines).
Optimizations (some were really bug fixes, but they only affected
performance) for pot reload.
Trigger a reload between batch mode passes.
Loader.c bugfix
Fix confusing mixup of variable names in wpapcap2john, and add ivs
(v2) support.
Loader.c bugfix bugfix :-(
Fixes for generic build.
Bugfix: Dominosec failed get_hash() w/ OMP. Closes #536.
OpenCL: override X11-forwarding.
Reload: Quirk for RAR, because it returns a slightly different salt
for each call to get_salt().
Disable OMP in Dynamic unless all PARA's are same. Closes #385.
wpapcap2john: Bump max handshakes limit, plus cosmetically.
NTLMv1/MSChapv2 "naïve" formats: mute a loader warning in OMP
benchmark.
Dynamic: Move the OMP disable workaround so it actually works after
a make clean :-/
Drop a test for --encoding that is problematic in case DefaultEncoding
is set in john.conf.
Reload bugfixes.
Fix for --encoding vs. DefaultEncoding and --test.
Set a label for the "dummy" stdout format.
Add default rules for -wo and -loop
Show "x password hashes cracked, y left" even for --show=left
Rename regex to rexgen everywhere, it's less confusing.
Bugfix: --show=left showed crack counts backwards.
Revert "Rename regex to rexgen everywhere, it's less confusing."
Add a separate --hashed-encoding option. Example: Your input files
(ie. login and GECOS fields) are UTF-8 and your wordlists are
too, but you are attacking LM hashes originating from a system
running CP850.
Add ability for --loop to join LM pieces automagically, and processing
them before the normal loop.
Mute encoding message if we're in default ASCII mode.
grub-opencl hotfix for "Joe".
Loopback bugfix for multi-node.
Change name of --hashed-encoding to --target-encoding.
Mute a message about intermediate encoding when not applicable.
Fix options flags for all old Jumbo options, catching incorrect usage.
Also allow -input-encoding as an alias for -encoding.
Fix a bug where the Unicode system was left undefined, and another
one that made our two Unicode-on-GPU formats fail for certain
encodings due to a brainfart.
vncpcap2john fixes (mixed declarations and code).
Encoding bugfix.
Mute intermediate encoding output when it doesn't really apply.
Add a verbosity config option.
Mute some warnings unless john_main_process.
Log what ruleset is used, and log "loopback" (not "wordlist") when
applicable.
More minor changes to "loopback" mode log output.
Allow defaulting an alternate incremental mode when encoding is UTF-8.
Allow size_t for -mem-file-size
Fixes for loopback reassembly.
Another bugfix for Loopback mode.
Mute a compiler warning.
Mute a compiler warning
64-bit bugfix. raw2dyna used %* supplying a pointer diff.
Fix for generic build.
Korelogic rules added as a separate config file. Not sure why we did
not include it earlier. This is the enhanced and bug fixed
version.
Korelogic rules: Shorter names for the components, and use .include
for the aggregate.
Pick john.conf default for LM formats more often.
Unicode: Best-effort conversion for invalid codepage, instead of
producing guaranteed garbage. Another option would be to truncate
at first invalid character, but for input conversions this approach
might be better.
Bugfix: Intermediate encoding was not triggered in batch mode.
Various tweaks & bug fixes, mostly for encoding.
Notification fix for intermediate encoding. It's incredibly hard
to get all details right. So many moving parts.
mssql12 will allegedly also crack SQL Server 2014 hashes.
xsha512 format will not crack OS X 10.8 or higher (those are
pbkdf2-hmac-sha512).
Print the cracked totals to stderr for --show=left, in case normal
output is piped somewhere. This addresses main part of issue #556.
Add a new super rule-set "All", that is Jumbo + KoreLogic...
Reduce chatter
Reduce more unnecessary output (encoding stuff).
Typo in usage for -costs. Untabify and add missing paren.
Rexgen update. This is for SVN rev 113 of rexgen, WIP.
Bugfix for tunable costs. You must not put anything after salt->keys
due to optimizations. Solar should have commented on that in the
header.
Optimize mask-mode order for 8-bit characters with ?A and fix some
overlong lines.
Slightly more OpenCL output at -verb:2
Add cprepair - a tool that fixes double-conversions to UTF-8 and/or
mixes of UTF-8 and a legacy codepage into clean UTF-8 (best
effort).
cprepair bugfixes and docs
Add last-line EOL for a bunch of files.
Add session file entries for implied options (default rules, wordlists
etc).
Add --mkv-stats to the storing of "implied options". Also fixes
inconsistencies with --mkv-stats option - it could not be used
together with --markov whereas I believe it should be the other
way round.
Add aes.a to .gitignore
Default salt sort to "most used first" for possible speedup in some
situations.
cprepair bug fixes and improvements.
cprepair: clarify an option in the usage blob. The purpose of -f
is to be able to eg. choose CP866 or CP737 over CP850 in case all
7-bit characters on a line is non-ascii letters. This will pick
the right decoding (out of two given) for cases like "Müller" and
"Стандарт" but for many words it will still just be a guess.
If we have reused salts, log when a salt is removed from list. This
is debug-only for now.
Fix finding librexgen under OSX.
Rexgen headers define UTF32 empty so workaround needed in unicode.h
Bail out if --regex used with --wordlist does not contain \0
Slight adjustment of rexgen logging for wordlist
Wordlist.c: More natural order of logging.
memdbg fix for cprepair
Fix problems with gpg2john that came (back?) after memdbg merge.
Rexgen "autoconf" fix for Linux.
Drop "no rexgen" log message unless verbosity was bumped.
Posix mmap() for wordlist mode - without dropping the old array
buffer. This gives up to a 3x boost, especially for --fork. The
buffer is read using a function similar to fgetl() but that
reads from the memory map and that works with up to 16
characters at a time.
Blind Windows fix: we should probably munmap() the same size we mapped
for good measure.
wordlist.c: mmap() bugfixes for edge cases (page boundaries, super
long input words, files not ending with LF and so on).
Fix some debug output that went to stderr while other of same kind
goes to stdout.
Add autoconf stuff to .gitignore
Fix for Travis/Coverity scan now that we use ./configure
Add autoconfig stuff to .gitignore, and add some macros we will
need later (ax_opencl.m4 and ax_prog_cc_mpi.m4).
Allow for NSS to be missing.
Rename configure.in to configure.ac - the former has been deprecated
since 2001.
Change final message from configure to something less confusing.
Rename HAVE_LIBCRYPT to HAVE_CRYPT for several reasons, mainly because
it's what core John uses. One other reason is that we no longer
really depend on the library per se, but the function (it might
be present in libc).
Whitespace fixes (a dnl comment was split by a linefeed)
Change final message from configure (again) to something less
confusing.
Autoconf fix for pcap.
Add CUDA and/or OpenCL stuff to JOHN_OBJS if applicable.
Decrease required version of autoconf to 2.68, for Travis CI to work.
Autoconf: Use proper OpenCL test macro instead of just checking for
lib. Also adds [suddenly] required autohell stuff.
Add real copies of compile, config.guess and config.sub (they were
symlinks).
OpenCL autoconf tweak.
Fix an OMP bug in krb5-18 and krb5-22. Closes #577.
Use double for ETA calculation, for much better estimate early on.
CUDA autoconf tweak
Fix redefinition warnings from blake2-config.h
The -DUNDERSCORES test needed '-' option to strings(1) because
"Unless the - flag is given, strings looks in all sections of
the object files except the (__TEXT,__text) section". It also
lacked placing a space at end of -DUNDERSCORES
Test disabling the automatic autoconf stuff in Makefile, in case
that's what make the build bot fail.
Adjust Travis for autoconf (do not specify make target).
Mute a red herring compiler warning in wordlist.
Correct some warning typos from configure
Avoid warnings from non-existing CUDA lib directories.
Rename "list alternatives" target (old default) to "help".
Cosmetic changes to configure.ac, and change a few [main] in
AC_CHECK_LIB() to something relevant.
OpenMP and OpenCL are defined as features (as opposed to packages) by
the prefab macros, so change MPI and CUDA accordingly. ICC-asm
is kept as a package.
Drop unused AX_OPENMP macro
Add a space after -lpcap, and add -lcudart if found/used.
Revert changes in SIPdump and vncpcap2john
An old PROJ_PCAP entry was still in Makefile.in, producing an
infinite loop.
Fix a bug handling default GPU device set in john.conf
Implement a method for soft-disabling of formats, in john.conf.
Add support for selecting OpenCL device as --device=accelerator
(can be abbreviated down to "acc").
Mute a compiler warning for possibly undefined variable in cost range.
Fix logic error in device selection abbreviation.
Add a stub Makefile, overwritten by ./configure but replaced by
distclean.
PCAP format fixes for legacy Makefile.
"make -f" is more portable than "make --file"
Documentation updates for autoconf.
Silence warnings from when CONFIG_MAINT is unset. It would confuse
users. Git never preserves timestamps. There are good reasons
but it's a nuisance.
Autoconf: Add a proper check for ARCH_INT_GT_32.
Autoconf tweaks: Use the HAVE_MMAP in wordlist.c. Enable fork for any
system that pass AC_FUNC_FORK.
Add AX_LANG_COMPILER_MS inlined in configure.ac (it was replaced
in the GNU archive by a more generic macro, but I don't want to
hack ax_opencl.m4).
Use -O0 during feature tests. Using -O2 is begging for trouble.
Add missing header for exit(3) in most autoconf AC_LANG_SOURCE tests.
Add a separate test for -march=native, closing #583. Also drop remains
of obsoleted JOHN_CFLAGS et al.
Hack workaround for Makefile vs. .gitignore
Conditionally include some standard -L and -I directories.
Fix MPI check. AC_PROG_CC_MPI replaces AC_PROG_CC so we must not run
the latter first.
Remove obsoleted stuff from para-best.pl
Fix X32 build. There were several issues but nothing major - except
maybe we can't add '-m32' or '-m64' from configure. That sort of
thing must be left to the user.
Add shorthand "--with-arch=opt" which adds opt to CFLAGS, ASFLAGS and
LDFLAGS. The best example might be "--with-arch=-m32" altough
many other uses are possible.
AC_USE_SYSTEM_EXTENSIONS must be run *after* MPI test, otherwise the
latter is screwed.
Fix for "light" cross-compile (eg. X32 or 32-bit, on a 64-bit host).
We must preserve CFLAGS but if we put -O0 in its tail, we override
any -O2 that come before it.
Autoconf: Portability fix. Note to self: Always exit() from main in
configure tests, never return. Do not include <stdlib.h> but
define "extern void exit(int);" instead. This is portable while
not producing garbage in config.log.
Start supporting cross compiles better. Add a --disable-native-tests
option but also disable them automatically if cross compiling
(as told by autoconf). This addresses some of #585.
Support more arch.h targets from target triplet, and get endianness
from it too. We can actually tell ARCH_ALLOWS_UNALIGNED from it too
but that is not implemented yet.
Support $2b$ hashes. Thanks to Kent Fritz for reminding us.
wpapcap2john fix. For some reason it still fails on BE.
Major autoconf arch.h tweaks. Separate generic tests from x86-specific
ones. Most targets should now have basic buildability...
Autoconf: Fixed reversed logic for icc asm.
Autoconf: Disable -march=native if we cross compile, dude.
Add autoconf_arch.h. Sooner or later we'll not need any other one
(which means we could drop it altogether and write configure output
to it instead).
Drop __ILP32__ from the "64-bit test". That macro does NOT tell us
anything about CPU width. It just show a data model that could be
X32 (which is 64-bit) or just about any normal 32-bit system.
Add a bunch of 64-bit compiler macros to test for in the 32/64-bit
test. And remove another incorrect one (_M_IX86).
Default to x86-sse.h for any 32-bit x86. We'll eventually pick the
right one depending on MMX/SSE2 etc. But this is the right one in
99.99% of cases anyway.
Autoconf: Fix reversed logic for pkg-config. It was enabled when cross
compiling, instead of the other way round.
Autoconf: Minor cross compile fixes.
Autoconf: Typo in comments.
Fix a minor bug in [Disable:Format] that made it impossible to
disable the first format (descrypt).
Implement the intended logic for 32-bit x86: First pick x86-any.h,
then if at least SSE2 is detected we change it to x86-sse.h.
Add cross compile versions of SSE/AVX/XOP tests.
Add AVX2 support for cross compiles.
Re-order non-native AVX2/XOP tests same as native ones, and do not
omit looking for XOP if AVX2 was found. Change output slightly
in non-native so you can tell the difference. Decrease
cross-compile to AC_CANONICAL_HOST - we are not building a
compiler (actually we are, but that one does not count)!
Fix for single mode segfaulting if pot reload deleted last salt. I'm
not quite sure why this is needed (it should already be handled)
but it fixed the problem.
Only try intel-specific tests on compatible CPUs.
Support <gmp.h> located in <gmp/gmp.h>.
Automatically set HAVE_NO_SSL_WHIRLPOOL if the lib lacks it.
Minor fix from Sparc/Solaris tests. The case of ntohll being undefined
should probably be fixed in some better way (handling BE). My
problem was that it already *was* properly defined beforehand.
Handle non-native OpenSSL check. This is an obvious start, more to
come.
Handle the case where pkg-config is not available.
Even better non-native OpenSSL check. We now don't just check for the
library (which almost anyone has) but also the headers.
Check for pcap headers, not just the library. Many people lack the
former but have the latter.
Autoconf: Fix for non-native builds. When using 'action-if-defined'
for AC_CHECK_LIB we don't get the default action so we need to
do it manually.
Add alpha.S when applicable.
Add support for x86 w/o SSE2. We should add a notice too, telling
user it might be time to buy some new gear ;-)
Stupid bug fixed in unrarhlp.h
Add AC_PROG_* and AC_PATH_PROG stuff, for aes/Makefile.in in
particular.
Print a summary after ./configure
Summary tweaks. And make -> $(MAKE) within Makefile.in
Makefile portability fixes.
Only try AESNI if intel family CPU :-P
Bugfix for OpenCL report detection (cosmetical).
Report AES-NI support
Portability fix for AES-NI support.
Drop obsoleted comments.
Missing rexgen and/or sse4.1 warnings no longer needed.
Crypt format OMP fix for Solaris.
vncpcap2john portability fixes.
wpapcap2john BE bugfix. Note to self: you can't do just
"JOHNSWAP(foo)" - you must do "foo = JOHNSWAP(foo)"!
wordlist.c: initialize a variable that gcc was worried about.
AES-NI: Pure whitespace fixes, for next patch to be readable.
Finally, add run-time detected AES-NI support for OS X too.
Fix AES-NI for 32-bit OS X. Also fixes some cross-compile issues.
Bugfix for forked resume.
Document the OpenCL device list round-robin behavior with -fork
and MPI.
Change all "test this && test that" to "test this -a that". And
replace "==" used within tests to a single "=" for portability.
Protect a test argument for including spaces.
Protect more test arguments just for good measure and fix the
ax_jtr_utility macros for DOS linefeeds and vast amounts of
spurious whitespace.
Auto detection for HAVE_SKEY. Not tested.
Handle CUDA 64-bit libs. We now test for CUDA after OpenCL because
the latter may find something that benefits the former.
Improve 32/64-bit CUDA and OpenCL library search. We now defer all
of it to the point we now CPU width for build.
Add some AC output for CUDA and OpenCL path enumeration.
Bump Jtr Jumbo version. Improve CUDA/OpenCL path enumeration output.
Cosmetical glitch fixed.
More summary output: Cross compiling, fork, MIT KRB5, PCAP.
Fix CUDA after f9711be broke it.
Bugfix for CUDA autoconf.
Include johnswap.h in a bunch of files that lacked it.
BE fix in ZIP format broke strict aliasing rules.
Drop "missing library" warnings during configure now that they are
listed in the summary.
Autoconf: Comment out many tests that we do not use the results from.
Just cherry pick them back if/when we actually need them.
AC_JTR_FLAG_CHECK and AC_JTR_SET_NORMAL_INCLUDES broke MPI builds
because they were called too early: The MPI/CC test *must*
preceed anything else that uses/checks the compiler and/or the
CC and CFLAGS variables.
Fix HDAA format for non-intel archs when OpenMP is in use.
Document Sparc build workarounds.
NSS ended up being required. PKG_CHECK_MODULES need an
action-if-not-found be it just a no-op.
Update docs for the fact MPI is not auto-enabled.
Add c3_fmt to summary.
Add missing entry in Makefile.in after merge.
Drop Mozilla format warning when not built - we already got it from
autoconf.
Support unconfigured "make clean" with a kludge in Makefile.stub.
Autoconf: Bugfix for native XOP test.
Document the fact that passing CFLAGS to ./configure will stop it
from adding "-g -O2" automatically.
Remove another improper adding of -O2. We just can't do that, it
breaks things.
Bugfix for ETA/progress output from wordlist mode.
Fix wordlist ETA for memory map mode.
Minor tweaks for ETA/progress calculations.
246f945 introduced a bad bug when -loop is used with a file name other
than the active pot file.
Fix AES-NI for X32 builds. This requires a new enough yasm, so we
should test for that - until then, just disable it.
Autoconf: Warn if we end up with no -O optimization option in CFLAGS.
Closes #592.
Deprecate openssl_local_overrides.h using autoconf. We keep supporting
openssl_local_overrides.h too for now, for legacy builds.
Stop using AC and AX as prefix for our local macros, those prefixes
are preserved. Add JTR_LIST_ADD macro for adding unique values to a
variable (think CFLAGS). Add a test for CommonCrypto and use it.
Autoconf: Improve robustness of JTR_LIST_ADD.
Hopefully fix S/Key format.
Autoconf: Recognize more optimization options (for the warning).
Drop bogus execute bit on a handfull of files.
Include arch.h header in a number of files using AC macros. Re-enable
a bunch of tests that were disabled for not being used (so
developers are able to start using them without changing AC).
Break some particularly long lines in configure.ac. And fix
cross compile issues.
Pacify clang compiler warnings.
"make install" now strips binaries.
Mute silly compiler warnings on OSX. @jfoug I LOVE this macro!
Add --disable-pcap to configure.
Autoconf. Drop the --with-arch option before someone starts depending
on it.
Drop all remaining use of += in autoconf macros, it failed on OpenBSD.
Fix bugs in SKEY introduced in a4ba81f. Now actually tested too,
on OpenBSD 5.4 (x86-64). Well, it builds - but it fails self
test. More to come. Closes #597.
More bugfixes to SKEY format. Did no-one use this format in years?
Bugfix in ODF format.
Pacify a compiler warning in wpapsk.h.
Pacify compiler warnings in pbkdf2_hmac_sha512.h.
Add ACX_HEADER_STRING macro (from GNU folks but not in archive).
Use it for C3_fmt to pacify compiler complaints on OpenBSD.
Quirks for OpenBSD autoconf.
Quirks for autoconf on OpenBSD: SIPdump and vncpcap2john.
Minor fixes for legacy builds vs. PCAP.
Workaround for fantastic systems not having grep -E.
Minor fix for AC c3_fmt.c
Add specific build instructions for Ubuntu Linux.
Cygwin workaround for vncpcap2john.
Do not add CPPFLAGS to CFLAGS, it doesn't make sense.
c3 fix for OpenBSD warning
Add a debug target. We always build with -g (stripped in 'make
install') but this target turns off optimizations and defines
DEBUG so code can adopt.
Add a fix for llvm-gcc-4.2 in raw-sha1-ng and make clashing structs
in lotus85 & vms static.
If -DDEBUG, warn when benchmarking (in case non optimized, or debug
code slows things down). Also "mark" debug build in the version
string.
Fix a missing '$' on a variable within OpenCL detection.
Fix for FreeBSD. Closes issue #566.
Fix a typo in cracker.c (HAVE_SYS_TIME_H vs HAVE_SYS_FILE_H).
Drop repeated info from o5logon-opencl format_name (vs.
algorithm_name).
The fix in 2080fc0 need to be applied to os-autoconf.h too :-/
Regex mode: Print a warning that we can not yet resume this mode.
Move CUDA tests to its own macro JTR_CUDA. Check for nvcc too, as
 well as llvm-gcc-4.2 on OSX.
AC: cosmetical bugfix: JTR_LIST_ADD_RESULT did not show all added
items.
Mark a couple of variables as precious. That means they get included
in --help and we don't need to AC_SUBST them.
Since we list OPENSSL_CFLAGS and OPENSSL_LIBS as precious, we must
honor them also when pkg-config is not used. If they are not, check
for /usr/local/ssl (just because legacy Makefile did).
Use strip with no options. POSIX defines no options so this ought
to be most portable. Also, list S/Key in summary, not SKEY.
AC: Various Makefile fixes.
Use supplied flags for NSS if given, even if pkg-info is not used.
Closing #603.
Autoconf: Change some if;then;fi to AS_IF()
AC: Tweaks for allows_unaligned test and some cosmetical changes.
If OpenSSL flags was specified, trust them without checking. Also
fixes a type in check for ws2_32 lib.
AC bugfix for CUDA under OSX.
Bugfix: Move definition of DEBUG_STRING in options.c. Some compilers
choked on its misplacement.
AC: Improve CUDA test. nvcc can't use a gcc newer than 4.6 as backend.
AC: Improve librexgen test. It detected positive on too old versions
and failed during build.
AC: Don't bother checking for AVX2/XOP unless we have AVX.
Bugfix in wordlist.c: mmap() checked for error the wrong way.
If -DDEBUG, be more verbose about mmap() problems.
Only add -lpcap when linking files that actually need it. Closes #604.
Only add extra libs (including OpenSSL) where needed - often only
to john itself. Closes #606.
Improve AC handling of CommonCrypto. If we have SHA2 in CommonCrypto
but not in OpenSSL, we'll now detect it.
Add --with-commoncrypto and --without-openssl options. JtR can be
built with either or both.
Makefile: Fix linker flag order, some hosts are picky.
OSX 32-bit fixes. Our ALIGN() macro clashed with something on OSX.
The ALIGN clash was just too bad. Rename all instances to JTR_ALIGN.
Two formats were using openssl/sha.h instead of our sha.h.
Add a missing CFLAGS backup. They were restored to pre-CUDA...
Turns out *all* modes do an emma within get_progress() so this was
redundant anyway.
Yet another idiotic feature macro needed, this time for getting
memmem() on Linux.
Show used extension macros in AC output.
AC: Fix for Linux quirk.
AC: Mute red herring warnings.
Wordlist.c: cast printf arguments to mute warnings.
Drop -ldl from all targets but john itself.
Fix a bunch of mostly not-really-problems, and one or two real ones.
Detected by cppcheck.
list-build-info now shows basic info about seek, ftell and memmem.
And also shows if we are running OpenSSL, CommonCrypto or both.
Minor fix in Makefile.orig. We could end up defining eg. HAVE_KRB5
instead of HAVE_LIBKRB5 and overall it was a confusing mix. Same
for GMP.
S/Key format was listed as 'xyes' on OpenBSD.
SAP B passed salts to loader with possible garbage in the end
(variable length). This lead to loader potentially seeing
"different" salts even though they were the same. Closes #608.
Add GMP, KRB5 and yasm to Travis "build bot" config.
Try to not DOS the Travis build hosts %-)
Add a check for BIO functions in crypto lib.
Fix for legacy Makefile using X32 and AES-NI (needs bleeding-edge
yasm, untested)
See if we can test both with and without OpenMP in Travis CI.
Revert "See if we can test both with and without OpenMP in Travis CI."
We already do one build with clang, which is non-OMP.
Check whether yasm supports our target (needed for X32).
Solaris fix for large file support.
AC: Don't bother checking for nvcc backend gcc unless nvcc is found.
AC: Drop erroneous use of CFLAGS_EX
SKEY_jtr.h did not have an LF (not even a b0rken Redmond one) at EOF.
Fixes for systems lacking <stdint.h>, including adding SIZE_MAX and
UINT32_MAX needed for escrypt.
Workaround for old gcc having problem with unrarppm.c
AC, large file support bugfix: Move size check for size_t and off_t to
after we have decided on feature macros. They might very well
change!
Sparc bugfix (alignment) for mssql12.
AC: Linux 32-bit large file support bugfix.
SKEY valid() fix. It was WAY to promiscuos, locking on header lines in
TS.
I had to add a backslash to get this list working over two lines.
'dnl' did not do the trick.
Wow, that was clever. I tested everything except the format I fixed.
Bugfix for 1f341404. You can't use HAVE_ macros unless including
autoconfig.h...
The change for krb5 in 41969aec had the side effect LIBS was polluted.
Don't add OpenCL paths if --disable-opencl. Revert bogus changes for
OpenCL/CUDA vs HAVE_DL and handle it properly in common-gpu.c
instead.
I wondered where all crazy libs checks came from. They are
OS-specific! So let's do them only when appropriate.
Silly typo
Revert the redundant "=1" for -D. Drop a large block of comments
that does not belong in source. Change more 'dnl' to backslashes.
Revert to a somewhat safer krb5 test (hopefully working for MIT as
well as Heimdal now). If we do not even have the
krb5_is_thread_safe() function, disable OpenMP for krb5-18/23
formats to be safe.
Don't try to use HAVE_LIBKRB5 as an aggregate - create a new one.
Better Kerberos 5 detection. Detect MKShim and Heimdal.
Add krb5support to list of libs to search for Kerberos 5 functions.
Revert last patch and simply disable krb5-18/23 formats if Heimdal
was found (until we know, and handle, the differences).
Fixes for Heimdal Kerberos.
Add support for specifying AS and LD to configure.
Update doc/INSTALL for OSX builds.
configure re-built after 0cd5e47. Also, move x86-mmx.h back from
unused, and fix MMX targets so they use core assembler code +
Alain's NT and Deepika's DES.
Drop HAVE_SSE2 et al from blake2 code (in favor of standard ones like
__SSE2__). Also drop JOHN_AVX and JOHN_XOP from autoconf - they
were never needed.
AC: On OSX, detect when 'as' can't assemble AVX instructions and
inform about the workaround.
Workaround dynamic's requirement that MD4_SSE_PARA == MD5_SSE_PARA.
Add `--param allow-store-data-races=0` if supported. Closes #632.
Minimize diff against core.
 Drop a redundant -DAC_BUILT
Drop a redundant ';' from mozilla
Add more granularity in SIMD algorithm name.
Dynamic: Hide OMP-only function when not building OpenMP.
Add a detail to OSX build instructions in doc/INSTALL.
Suppress some clang warnings.
Debug build fix for systems with no CLK_TCK. Closes #642
New plugin format, capable of optional plugs.
Convert non-plug formats to new-style plugins.
Convert all OpenCL formats to plugins.
Convert more optional formats to plugins
Plugify pfx, rar, ssh, wpapsk and zip.
Plugify mozilla format.
Fix for legacy Makefile vs. new plugin format.
Plugify HMAC-MD5, HMAC-SHA1 and raw-SHA0.
Plugify krb5-18/23 formats.
Plugify CUDA formats and add some missing glue for other formats
as well.
Minor fixes for new plugins.
Move raw-sha224/256 CUDA formats back to being non-plugs and fix other
issues. All known problems gone now.
Initialize the Unicode system before calling listconf_parse_late()
and speed up --list=format-tests for GPU formats.
listconf.c: Handle the case where field 0 is given but no others.
Example:

{WOWSIG"6D00CD214C8473C7F4E9DC77AE8FC6B3944298C48C7454E6BB8296952DCFE78D$73616C74",
"PASSWORD", {"SOLAR"}}
Some --encoding issues fixed (see #664).
Make it possible to build x86-64 without SIMD, for example like this:
./configure --disable-native-tests CFLAGS="-O2 -g -mno-sse2
-mno-mmx"
Add a note in Unicode external formats about them being generated from
Unicode version 6. Unicode v7 is pending publication, planned
October 15.
Some more or less no-op changes to SAP F/G format while investigating
a problem.
Minor changes to --encoding (et al) usage output.
Fix for x86-64 no-SIMD build, vs. new plugin interface.
OpenCL speed-ups for --list=format-[all-]details
pwsafe OpenCL kernel overwrites input buffer, so the "new_keys"
optimization introduced in 6a2e122a breaks the format unless
only one salt is attacked. Closes #640
pwsafe CUDA kernel only does one limb of SHA256 and its salt size
is 32 so max. plaintext length simply CAN'T be more than 23.
Closes #641
EPiServer: Fix a bug in binary() as well as some Unicode bugs.
Closes #643
CUDA SHA-224 was given a wrong binary size. Closes #645.
SAP F/G format: Support up to 40 characters of ANY length Unicode
for SIMD too. Fix a buffer-selection bug. Issue #662 is still yet
to solve.
sapG bug fix. Closes #662.
HAVAL fixes. Closes #652.
Reorder test vectors so an untagged one come first if applicable.
Minor Unicode/encoding fixes.
Move the recent --list=format-tests changes to a new function, and
use it also for --list=format-all-details.
Listconf: Drop trailing ':' from example cipher text.
Fix peculiar bugs introduced by 5bd568d1
OpenCL: Make sure our COMPUTE/DISPLAY tweak works for
--list=opencl-devices
Fix a bug in KRB4 that made it unreliable unless used with a
single afs salt or with des salts only. Closes #653.
It turns out krb5-23 are just NT hashes. Drop the krb5-23 format
(it was a lot slower) and change kdcdump2john.py so it outputs
$NT$ tags for those.
Office2007-opencl: Some debug stuff for future use. And drop a segment
of code if not needed (depending on HASH_LOOPS figure).
Add a check for -Q in compiler argument tests. Closes #682.
krb5pa-md5: Unify input formats (always write new style format to
pot). Closes #654
krb5pa-sha1: Drop a test vector that confused us. Closes #655
Revert 58a02ba and implement a better scheme: We only let USED
fields through split(). This way unused fields can differ and it
won't be a problem. This enhances solutions to issues #654 and
#655.
Revert "Revert 58a02ba and implement a better scheme: We only let
USED"
Proper version of f8ced75 that was reverted earlier. There was an
off by one error. Also fixes the OpenCL versions of the formats.
Hopefully this settles it for #654 and #655.
Add the example from
http://www.openwall.com/lists/john-users/2014/06/23/1 as a test
vector in the known_hosts format.
Make full Unicode (4-byte UTF-8) optional for fast MS formats,
because it means a performance hit of 10% or so. Default to
disabled.
Cosmetical changes to ZIP source from merging changes to OpenCL.
ZIP-opencl: implement Jim's changes from CPU format.
ZIP formats: Bugfix for (unlikely) bad salts caught late. Actually
I think we should bail out with error in get_salt() instead but
this bugfix is needed as long as we handle it this way. Any real
problems are very unlikely though.
Drop some obsoleted variables from zip-opencl.
ZIP format: salt size fixes. Pot sync would not work without the
same quirk in cracker.c that RAR already had. And salt_hash()
should iterate over the entire salt.
bash completion: Format class enumeration missed stripping the commas.
zip and pkzip formats does not unify case, so should only accept lower
case.
Minor corrections after compiling with -pedantic.
"Enable" the 'cpu' format class for non-GPU builds. Actually it was
just not mentioned in the usage blob for non-gpu builds.
Autoconf: Cosmetical, crypt(3) is a format, not a mode!
Mute bogus OpenMP warnings when running MPI.
Revert to using rexgen if found, but only if it's the version we
support. We can expand this to recognize newer versions once we
support them. You can still use --disable-rexgen to manually
disable it.
Drop the SHA2 ng-i formats. They were interim ones before the
generic ones got SIMD support. Closes #677
RIPE-MD128/160 bug fixes. Closes #675
Oracle, unify case in split(). Closes #657.
For FMT_NOT_EXACT, --show=left should only output hashes we did
not found *any* candidate for. Closes #671.
"Found no" or "did not find any", pick one.
OpenVMS bug fix. It would not even load its test vectors when put
 in a file.
Change self-test so it detects problems like the OpenVMS one that
was fixed in 84ff14a (valid failed unless init was called first).
Fix obscure bug in wordlist.c caused by 11e5191. It would make nodes
get out of sync when dupes was found after truncation. Closes
#699.
7c76d2c had some problems with multi-field test vectors. This
should do the trick.
Move the function null pointer checks to after fmt_init() where
applicable.
Further shuffling in formats.c - more tests should take place after
fmt_init().
Workaround for a thin dynamic problem resulting from recent self
test fixes.
Fix another case of wordlist w/ --fork getting out of sync. This
happened when --min-len or --max-len was used. See also issue
#699.
Change a couple of allocs into callocs in common-opencl.c, possibly
mitigating effects of buggy drivers.
Call prepare() and split() during pot reload. This should not be
needed (or so I thought) and has some performance penalty in some
situations but the dynamic format requires it for hex/dehex of
salts. Closes #702.
Initialize a variable introduced in c882813 so we don't parse any
garbage it might contain.
Pot reload: Move redundant initialization out of hot loop.
Whitespace fixes in formats.c and loader.c
Do not print candidates in status lines when in secure mode.
Closes #713 although we could improve this by printing the
"components" just like in a crack, eg: `1g 0:00:00:00 DONE 2/3
(2014-08-12 21:46) 100.0g/s 101200p/s 101200c/s 101200C/s
L6-?d..L6-?l`
Quick and dirty optimization for RAR5 format, for a 90% boost.
Fix OpenMP for previous commit.
When executing a "partial" john command during bash completion
enumeration, do NOT include any input hash files. This seems to
fix #701.
Fix a problem with --skip-self-test
Drop the "alleged" from RAR5 algorithm name. It actually is
PBKDF2. Also fix some bugs and comments in valid().
RAR5 valid() had a test done twice.
Add some alternative addresses for Dhiru to .mailmap so Git knows
he is one same person
Bugfixes and improvements for -external:dumbdumb
RAR3 OpenCL need init() before benchmark, in order to output
correct length of test vectors.
RAR5: Move some functions to a header file for sharing with OpenCL
format.
RAR5-OpenCL: Initial implementation. Quick'n'dirty.
pbkdf2-hmac-sha256-opencl bug fix and auto-tune, from rar5 format.
Do not add implicit target-encoding to session file for --stdout.
Autoconf: Enforce CPU formats' precedence over GPU ones. Closes
#724.
RAR5 and PBKDF2 OpenCL formats: Autotune tweaks and optimizations.
bench.c: init all OpenCL formats early, to avoid assorted problems.
RAR5 OpenCL: Minor optimization for post processing. We could do this
on GPU but it wouldn't make a difference. This way we use
PBKDF2-HMAC-SHA256 kernel unmodified.
Use shared code for auto-tune in pbkdf2-hmac-sha256-opencl.
Improve OpenMP performance of Siemens-S7 format w/ many-salts.
Mute some warnings in sha1crypt
Speed up OpenCL auto-tune in rar5 and pbkdf2-sha256
RAR5 OpenCL bugfix - I picked the wrong line in last commit.
RAR3 OpenCL: Use shared autotune.
crypt-sha1: Use correct name of function, for less confusion. 100%
cosmetic.
Add alternative address for Claudio to .mailmap so Git knows he is
one same person
Autoconf: Ignore case when sorting format list.
Ignore case when sorting formats for the usage blob.
Autotune bugfix in RAR5 OpenCL.
Add loop count output to shared autotune statistics
RAR5 & PBKDF2 OpenCL: minor tweaks.
WPAPSK-opencl: replace autotune with shared code, including fixing the
latter to handle vectorized kernels.
wpapcap2john: Fix a bug when calculating eapol size. Closes #728.
wpapcap2john: Fix bug when stripping encapsulations. Optionally do not
output incomplete (unverified) authentications.
Swap Claudio's addresses in .mailmap
Fix bogus line endings in configure.ac (it had a disgusting MIX of
LF and CRLF!)
Fix bogus line endings in jtr_local_conf.m4 (it had a disgusting
MIX of LF and CRLF!)
Our current HMAC formats are "salt is message, password is key". I
always forget which it is, so add this to algorithm name. We
could/should implement "salt is key, password is message" too.
Bump some artificial limits in our generic PBKDF2-HMAC-SHA1 format,
and add WPAPSK DK example (test vector) to it. Also a minor bug
fix.
Add OpenCL format for generic PBKDF2-HMAC-SHA1.
pbkdf2-hmac-sha1 opencl tweaks.
pbkdf2-hmac-sha1 opencl bugfix for weak devices.
OpenCL: Autotune tweaks.
OpenCL: More autotune tweaks.
Drop the FIXME comment in wpapcap2john. EAPOL packets have a 16-bit
length field excluding itself and the two 8-bit ones preceding it.
Fix some format/algorithm names
Fix some OpenCL algorithm names when running vectorized.
Rename pbkdf2-hmac-sha1-opencl source file
Minor fixes to Lotus5-opencl. Format is still b0rken - it is hard
coded for a LWS that not all devices support. It also does not
obey LWS/GWS environment variables (#601).
Lotus5-opencl bug fixes. Closes #601.
Lotus5 OpenCL bugfix when bumping GWS from default.
Various formats: OpenCL AMD bug workaround.
8-bit fixes for pass_gen.pl. The issues were not really caused by
recent dropping of Authen::Passphrase, but found due to it.
Obvious optimization for EIGRP format: Move some work to get_salt,
where it's done once instead of millions times per second.
Support meet-in-the-middle hint for Office 97/2003 hashes. Speedup
on CPU is about 2x but it doesn't show in benchmark. See #736.
Fix plugin headers dependency in Makefile for john.o.
Add critical section pragmas to all use of "any_cracked" variable, and
fix some bugs while at it.
OpenCL: Show source file name when a kernel build fails.
Move DEScrypt-opencl format to broken/
Switch to Lukas' (faster) SHA512 kernel for Office2013.
Workarounds for AMD bugs in Catalyst 14.6beta.
Change the message "Press 'q' to abort (...)" for -pipe and -stdin
modes.
Fix bugs and add some WLAN stuff to cisco2john. Still not much
tested.
Vigenere bug fix in cisco2john.
Various OpenCL fixes.
Move DEScrypt-opencl back in business, but make it disabled in
default john.conf instead. This way it's out of the way unless
you call it by name.
7z-opencl: Move back in business but default to disabled in
john.conf
Drop argument "arch sm_10" to CUDA compiler. It's been deprecated
and now doesn't work anymore on newest versions.
Grub kernel: Mute compiler warnings about unsigned vs signed int.
Keyring-opencl bugfix. A trailing zero was put in a buffer not
made for it, writing past end.
7z-opencl bugfix. A trailing zero was put in a buffer not made for
it, writing past end.
pwsafe-opencl and rar-opencl: Handle devices that limits LWS in
some OTHER kernel than crypt_kerne. There will be future
problems with this unless we fix all multi-kernel formats.
Show in --list=format-all-details whether format is disabled in
john.conf. Also mute a SAP F/G warning when running --list.
Various OpenCL formats: Honor device limits.
CUDA raw SHA224/SHA256 formats were broken since some time.
Change format struct name of raw-SHA0 so it sorts after SHA1.
Closes #741.
Loopback bugfix for 8-bit characters.
mozilla2john was not being built in an autoconf build. Closes #743.
pass_gen.pl: Fix a bug in the try/catch for LM encodings.
More encoding fixes for pass_gen.pl
OpenCL: Ensure AMD device topology output despite outdated headers.
OpenCL: Add a hardcoded ".0" after PCI string for nvidia, and fix
indentation.
OpenCL: It turns out PCI id's are 8-bit bus, 5-bit device and
3-bit function. I presume this is the correct way to parse what
we get from the query.
OpenCL: Fix stray output.
Unify some more format names in pass_gen.pl.
pass_gen.pl: Sort format names when listing them.
Add a --keep-guessing option, that sets FMT_NOT_EXACT for any
format and thus lets us search for plaintext collisions.
OldOffice format: If we cracked a hash and are in FMT_NOT_EXACT
mode (ie. --keep-guessing option was used), store a meet-in-the-
middle value for faster cracking.
OpenCL list devices: Show platform/device extensions if verbosity
was bumped.
getopt.c: Alternate dupe-options check for options which are
FLG_NONE and OPT_REQ_PARAM. This frees up options flags.
Trust the new options dupe-check to free up 15 flags for Jumbo.
Pacify a compiler warning in wpapsk.h. I think it was latest CUDA
nvcc that started complaining about the "deprecated conversion
from a string literal to "char *"
Stop mentioning krb5-23 format - it doesn't exist anymore since it
was really just the good old NT format in disguise.
Typo. Apparently I should use a larger terminal font.
OldOffice: Add intermediate key to .pot file. Functionality is
added but disabled for now. We need a core change first, or
--show will not work properly. See #736.
Using FLG_NONE does not work for the recent dupe checking, use
flg_set == 0 and req_clr == OPT_REQ_PARAM instead.
Add a FLG_ZERO for (in a sense) better readability of the Jumbo
option dupe-checking.
Drop DEFAULT_ALIGN from dmg-opencl.
CUDA formats salt cleaning fixes (sha256crypt and sha512crypt).
OpenCL formats salt cleaning fixes (agilekeychain, bcrypt, gpg,
keyring, rar and sxc).
Salt cleaning and other bug fixes to 7z and SIP formats.
Some tweaks for the new salt-cleaning self-test. We clear the
buffer again after our tests, mitigating possible issues.
OpenCL formats need a postponed init() for "poor man's multi-
device support". This caused problems for a few CPU formats so
we now ensure it only happens for actual OpenCL formats. Closes
#755.
Cosmetic: Status help message now says SIGHUP if SIGUSR1 is not
available.
Binary buffer cleaning self-test. See #757.
Fix bugs in Skein and Snefru formats, #757.
Fix a bug in IKE format, #757
Fix bugs in AIX, DragonFly4, eCryptFS and Dummy. See #757.
Support -encoding for -stdout. And tweak mask mode's encoding handling
a little: For consistency, it now honors --intermediate-enc if
used. Closes #756.
Drop the now obsoleted mask-mode code from rpp.c and rpp.h
mask.c: Re-add encoding support, and ?h ?H ?A place holders.
Minor changes to doc/ENCODINGS
sha1crypt-opencl format added. Split kernel is WIP.
Found a slightly enhanced version of that sha1crypt-opencl among
all my local topic branches %-)
Update CHANGES-jumbo.git and CREDITS-jumbo.
Jumbo options dupe-check bugfix.
I reverted too much in 945be4a... some Jumbo-specific stuff re-added
(namely, support for using \xHH notation in rules and ranges), now
with comments.
Deprecate "ansi" in encoding list output. We still support it as a
shorthand for ISO-8859-1 although it's ambiguous. Add support
for "latin1" as another alias for the same, even though that is
still somewhat ambiguous :-P
WIP for ADL/NVML mapping to our device id (#750)
ADL debug output fix.
Change --intermediate-enc to --internal-encoding. The old name is
deprecated immediately, sorry 'bout that! To my defense I call
the fact we haven't had any release with the old option...
Mask mode: Do not produce mixed-case candidates for case-
insignificant formats like LM. Closes #764.
Mask mode: Code cleanup.
Bugfix for bash completion of encoding option name change.
Mask mode: Start implementing custom mask (WIP, #762).
Autoconf: Add --disable-openmp-for-fast-formats (some CPUs can't
handle it well so it ends up slower than one core).
Add -<N (rule reject for --min-length=N) and U (reject unless valid
UTF-8).
Merging super old topic branches exploded in my face. Build bots
to the rescue.
Bugfix for resuming with an implicit internal encoding.
Mask mode: When !FMT_CASE, produce lower case candidates instead of
upper case ones. There's a slight difference with some
encodings.
mask.c: Always parse \xHH notation. Closes #772.
mask.c: Handle command-line arguments given in UTF-8 when internal
encoding is some codepage.
Minor tweak for UTF-8 handling in mask mode. Better safe than sorry.
Move a variable declaration to avoid compiler warnings.
mask.c: Allow escaping \ as \\. So \\x40 means literally \x40 and not
the character with ascii code 0x40.
Allow custom masks given in UTF-8 on command line.
Loopback mode: Improved handling of assembled LM halves.
Mask mode: Parse hex notation in custom masks.
Add support for a default mask in john.conf. The default one (same
as Hashcat) is is not too useful until we implement -min-len and
-max-len support for mask mode (#380).
Mask.c: Do not touch empty strings during de-hex.
Mask mode, resume bug fix. Closes #780.
mask.c: Allow using static placeholders within custom placeholders
?1..?4.
Mask: Do not touch empty strings during expansion.
AC: Add AC_PREFIX_DEFAULT([../run]) as part of trying to fix #612.
OldOffice: Revert to using core's salt copy, so a found "middle"
value does not get lost in set_salt(). See #736.
mask.c: Preserve existing escapes during pre-processing. See
discussion in PR #776.
"memdbg" fixes for a bunch of OpenCL formats. See #739.
MEM_FREE bugfix for rar5-opencl. A buffer was freed in done() but
should be freed in release_clobj()! See #739
Add some missing inclusions of memdbg header. See #739
Drop some debug output from mask.c, and only talk if we're the
main process.
Mask mode: Fix progress count for node/fork/MPI. See #771
mask.c: Cosmetic changes.
Mask mode: Fix progress reporting.
Make it possible to build 64-bit "any" or MMX-only builds, for
testing purposes.
Split the PBKDF1 kernel for sha1crypt-opencl.
Minor fixes to PBKDF kernels.
OpenCL: Drop some debug output now that the NVML problem is fixed.
Also move the ADL output to opencl_list_devices() so it's not
printed when emitting a status line (although this is currently
defined out by default).
CUDA: Ensure mapping between CUDA id and NVML id, using PCI id.
Add NVML device id to --list=opencl-devices.
Re-create ADL main control for each call to get_temp(). This is
probably just a bug workaround, it should not be needed. Closes
#784.
Fix "generic" legacy build. Closes #782.
OpenCL: Add possibility to override build options using environment
variable or john.conf. Also add custom build options capability to
sha256crypt-opencl and sha512crypt-opencl, as a PoC. If given,
they will be concatenated with global options.
Add clReleaseProgram() to Sayantan's formats.
Add a --stress-test[=TIME] option that is just like --test except
it loops the test(s) forever.
Memory leak bugfixes to mscash2, bcrypt and lotus5 OpenCL formats.
Office2007-opencl: Use shared auto-tune.
Office2010-opencl: Use shared auto-tune.
Office2013-opencl: Use shared auto-tune.
encfs-opencl: Use shared auto tune.
krb5pa-sha1-opencl: Use shared auto tune.
krb5pa-md5-opencl: Use shared auto tune.
ntlmv2-opencl: Use shared auto tune.
mysql-sha1-opencl: Use shared auto tune.
Minor fixes for the last eight commits.
Allow 10s total for encfs-opencl. It's a lot of iterations!
Drop some unneeded code from raw-md4/5 OpenCL formats.
Drop obsoleted code from OpenCL formats, and fix a bunch of little
issues.
OpenCL: Ensure unmapping of buffers.
OpenCL: Bugfix for reporting utilization from ADL OD6 devices.
Optionally include GPU sensor readings when printing status line.
GPU sensor status: Some GPUs report a utilization of 0% when they
actually do not support that query, mute that. Also, tweak output
a little for multi-device (as in mscash2, not -fork).
GPU: Abort job in case we hit the temperature limit set in john.conf.
Log GPU sensor readings at start/stop of session.
Allow 30 seconds grace time for a timer abort (-max-run-time) so we
don't lose the session save for slow/split-kernel formats.
Avoid overhead in inner loop of split kernel OpenCL formats.
Reduce overhead in gpu_check_temp().
Clean up (cosmetic) of GPU code common for CUDA/OpenCL.
Show sm_5x (Maxwell) core counts correctly in --list=opencl-devices.
Ensure initial MPI/GPU logging in -fork children.
Fix arch link issue for 32-bit SSE2 or better. See #793.
Cross-compile version of the fix in 087383c. See #793.
ZIP_fmt: non-inline data length bugfix. Closes #792.
#792 bugfix for the OpenCL version of the format.
Pure whitespace fixes in cryptsha512_common.h
Copy-paste bug in 8cd6cf53.
Copy-paste bug in 73246ae (duh!)
More fixes for building (test builds) x86-32/64 without MMX or SSE2.
pdf2john, fix problem introduced with 292b35c. Closes #789.
Resource cleanup fixes in various OpenCL formats. See #739.
Perform some obscure Jumbo self-tests only if --test. Closes #799.
Stress test bugfix: sig_done() must set SIGUSR2 to SIG_IGN if used.
Mitigate race conditions in SIGUSR2 handling. See #798.
Improve error reporting to log file: Eg. "Terminating on error:
john.c:144" Closes #800.
Add sm_50 SP multiplier for CUDA too (I did not remember we had to
calculate it from a table like in OpenCL).
Further mitigate race conditions in SIGUSR2 handling. See #798.
RAR OpenCL: Implement dynamic salt. cracker.c: drop the kludge we
had for RAR and ZIP formats. It's now taken care of by the
dynamic salt stuff. See #752.
ZIP OpenCL: Implement dynamic salt. See #752.
Add .gdbinit to .gitignore.
Fix a silly bug in CUDA-id to NVML-id mapping. Closes #801.
In last commit, a paren escaped from where it should be and ended
up where it should not. That's what you get when you have to
dry-run testing fixes :(
Fix -DDEBUG builds for a couple of OpenCL formats.
RAR formats: Use fmt_default_salt_hash_dyna_salt().
ClearQuest: Add some parens that Dhiru tried to do without.
RAR: Never include data blob in salt compare size.
Makefile.stub: exit with error if not configured.
Experiment with using mmap() for large RAR blobs.
RAR mmap: Stop using offset, because it's too hard to determine
page size in a portable way. And this has no real drawbacks.
Add debug info to dyna salt compare.
pkzip: Improve OMP scaling. It was hard-coded with a ceiling.
Dynamic salt: Adjust SALT_CMP_SIZE macro, it ended up wrong due to
struct padding.
RAR-opencl: Get up to speed with CPU version (ie. mmap() support).
Dyna salt debug.
Drop broken custom hash functions in pkzip and TC. Closes #791.
Add missing hash functions (and/or use default ones). Closes #790.
EFS: Bugfix for iterations count. BAD BUG, possibly false
negatives. Shame on @kholia. Closes #563.
AC: Create a GNUMakefile alternative. The old stuff is still there
too. See #805.
Case fix for GNUmakefile. We had it as GNUMakefile.
Drop the GNUmakefile if "make distclean", or users will be very
confused.
WPAPSK-opencl: Auto-tune fixes.
Revert "dyna_salt. Use 4 param COMP_SIZE macro" Revert "rar_fmt:
removed numerous warnings, and added comments where we can not
handle large file size"
rar2john: missing linefeed in usage blurb.
See #794. Add MAX_INLINE_SIZE to params.h and have keepass2john
honor it, with an optional -i override.
keepass2john fixes.
keepass2john bug fixes.
rar2john: adopt to #794, using -i for setting inline threshold and
defaulting to MAX_INLINE_SIZE.
Minor fixes to keepass2john and rar2john.
zip2john: Implement getopt (sorry, had to change some option names)
and MAX_INLINE_SIZE, see #794.
luks2john: Implement getopt and MAX_INLINE_SIZE, see #794.
Tiny optimization for RAR formats: Drop an assertion that is now
well tested. We keep it for -DDEBUG.
Unify RAR & RAR-opencl. Some minor fixes were not implemented in
both.
RAR formats: Use 64-bit fseek() if possible. We still can't fread()
larger than 4 GB but we can now seek 5 GB into a file for fetching
(hopefully smaller) data.
rar2john: Use 64-bit fseek(), just in case.
Mute a warning in legacy generic builds, addressing #807.
Makefile tweaks.
Cosmetic auto tune fix in NTLMv2-opencl.
ClearQuest format failed on 32-bit due to sign extension. Closes #806.
If we have gmake, use GNUmakefile and do not overwrite Makefile.
Closes #805. This also adds a missing @SET_MAKE@ that was likely
intended to be included in every makefile since we've been using
AC_PROG_MAKE_SET for long.
Fix numerous memory leaks in OpenCL formats.
ssha-opencl: Use shared auto tune.
Flush log file before signaling children we're done. Closing #798.
AES-NI code: Change a __declspec to something more portable.
Change name of
s/fmt_default_salt_hash_dyna_salt/fmt_default_dyna_salt_hash/g.
Refactor function names of opencl_autotune.h. The common_* names
always confused me.
With a fast enough GPU, auto-tune would overflow signed 32-bit
integers.
OpenCL: Allow auto-tune with a single kernel-invocation limit instead
of total crypt_all() limit.
Use new alternate auto-tune scheme for a couple of formats.
Fix a bug in Office OpenCL formats where cracked array was not
 properly reset when running vectorized.
Salt cleaning in OpenCL Office formats.
pbkdf2-hmac-sha1 opencl bugfix.
Revert a memleak fix in OpenCL formats. Something goes terribly
wrong unless we do it this way.
Drop the Makefile stub. I causes too much problems. We can symlink
Makefile -> Makefile.stub in release tarballs though.
Office 2013, tweak auto tune
Fix an issue with --skip-self-tests and dynamic salts.
CUDA: cosmetic, device list.
OpenCL raw-md4/5 bugfix.
Keyring-opencl bugfix.
pbkdf2-hmac-sha1-opencl: Report iterations as tunable cost.
AC: Do not add -march=native in case we are OSX with broken as.
Add salt_hash, binary_hash and get_hash functions to ClearQuest.
Closes #810.
WPAPSK-OpenCL: Minor optimizations for GTX980, for a 5% boost.
Add stacked mask mode to Incremental, Markov and Wordlist modes.
It's not actually handled in mask.c yet. Also, resuming a stacked
session may (or may not) need work.
Autoconf: Assert that we don't have a stray GNUmakefile still in
the tree, after some of magnum's poor ideas in #805.
GPU: Initialization tweak and other minor fixes.
Benchmark: Print GPU utilization (if available) after benchmarking
OpenCL or CUDA formats more than one second.
In hybrid mask mode, do not use candidate alone. While it could
sometimes be a feature, it could also produce up to 50% unwanted
candidates.
Bug fix for hybrid Markov + Mask mode.
Optimization for short masks: Avoid repeated strlen's that the
optimizer would not dare taking away as written. Also make key a
const char* so we don't screw up.
Bugfix for non-hybrid mask mode.
Bugfix for hybrid mask w/ node/fork: For now we always let the
parent mode distribute the work. For large masks it might be
better to let mask distribute but that is a more intrusive
change (to all modes).
Fix non-GPU builds for bug introduced by 9a5e3ae
Fixes for stacked (eg. hybrid mask) session save/restore.
Fix progress/ETA reporting for hybrid mask/stacked modes. Wordlist
mode still has coarse reporting in some situations (mmap) when
wordlist is small and mask is big.
Fix some missing pieces in the Regex mode.
Cosmetic bugfix: mask_done() was called a tad too early so final
status would be empty for short runs.
Bugfix for save/resume in Hybrid mode. If we are about to abort,
we must not call parent's fix_state(). Closes #823.
Further tweak for #823. In extreme test cases using -max-run=1 we
sometimes did not get any progress for a (one second) run.
Change default custom masks (which resembles Hashcat's) to use eg.
a-z instead of ?l so they are not affected by internal encoding.
Add mask placeholders ?L and ?U which is non-ASCII lower and upper
case characters.
Reject ?L and ?U placeholders unless an 8-bit codepage is in use.
pkzip format: Mute warning about missing files if we're just reading
pot lines.
Remove some #ifdef DEBUG, or change them to something more specific
like POTSYNC_DEBUG or DYNA_SALT_DEBUG.
Max length fixes in rules.c for internal encoding.
Fix handling of invalid UTF-8 in mssql05. Closes #830
Ensure key cleaning on invalid UTF-8 in same fast formats. No
problem seen with these but #830 hinted we might get problems so
we play safe.
Mask mode: Add support for format's max length. See #834
Mask mode: Fix bugs for using default mask (bare --mask option).
Mask bugfix for default mask. See #834
Mask: Cleanup of previous changes. This is a no-op except we now
keep a pointer to the actual original mask in case we'd need it
for debugging.
Use a separate default mask for Hybrid mode.
Hybrid mask: Bail out with error if [the used portion of] hybrid
mask does not contain a ?w placeholder. See #834
Mask mode: Ensure max length is not exceeded after conversion to
UTF-8 too, if applicable. Hopefully this is the final fix for
closing #834
Drop CP/M linefeeds from Jim's files.
Oracle: Address #832 (overlength salt segfault)
Common OpenCL: Do not fail if we can't get PCI info. It will only
happen when we don't need it anyway. Addresses #836
OpenCL, cosmetical: Strip leading whitespace in device names.
OpenCL: Improve handling of --fork vs. (lack of) --device.
Mask: Better handling of max length when internal encoding is used.
Fix non-OpenCL builds. Missing #ifdef.
Minor fix for using default mask.
Wordlist: Unicode length fix when internal encoding is not UTF-8
but target encoding is.
Fix for john.o in debug build target.
Add a 'memdbg' build target which can be used without editing
anything.
Mask mode: Use mask_add_len in hybrid parent modes. See #837
Mask mode: Use mask_num_qw in hybrid parent modes. See #837
Comment out -DMASK_DEBUG that was accidentally committed.
Mask mode: Various tweaks. See #837
Mask mode fixes. See #837
Hybrid Mask: Fix detection of missing ?w when mask was not
truncated. See #837.
Mask mode: Do not allow ?w unless running hybrid/stacked.
Mask mode: Change meaning of ?l, ?u, ?d and ?s to always mean 7-bit
ASCII, and add ?S (high bit specials) and ?D (high bit digits).
Mask mode: Escape fixes. Some things were lost by recent commits,
others were not really good to begin with: We must never look
backwards for detecting escapes, only forward. See #776 for older
discussion.
Mask mode: Escape fix for \xHH hex. See #841.
Despite generated from the "Unicode database", encoding_data.h
contained a dupe greek character.
Include ASCII in list of encodings, mostly for tab completion to
include it.
Add a "make clean" for Travis CI, just to be 100% sure. See #711
pbkdf2-hmac-sha256 bugfix for buffer overrun, see #711.
Mute the "Using default input encoding ..." if we actually do not
use any input.
Mute the "Press 'q' or Ctrl-C to abort" when using -stdout and
output is a terminal. If you could read it, it wasn't needed...
Mask mode: Warn if \x00 is used in mask.
Overhaul of all formats that use base64decode(). It adds a NULL
string terminator that the destination buffer must be sized for.
Mask mode: Allow ?w in pure mask mode, just warn that it doesn't
 have any special meaning.
Mask mode: Support -min-len. It will call mask_crack with each
length between min-len and max-len.
Bodge for bash completion of eg. "john -stdout -list=...", see #842
Mask mode: Fix for -min-len=0.
OpenMP optimizations: Use 'atomic' instead of 'critical' when
possible. In oldoffice, keep using critical but make it a block.
Mask mode min-length session save/restore bugfix. See #845
Mask mode initial documentation. Closes #824.
Mask mode: Do not "use" min-len in hybrid mask mode. Parent mode
will honor it after taking mask_add_len in account though. See
#848
Fix OpenCL build after 38218fe3
Mask mode: Progress reporting fixes. See #837
Signals.c edge case fixes.
Mask: Cosmetic: Print some warnings/errors in main process only.
Mask mode: Drop redundant code, fix max_add_len for truncated
hybrid masks.
MDC2 format: Fix build error. And indentation style..
Mask mode -max-len fixes. See #837
Mask mode: Warn if mask is shorter than supplied -max-length
parameter. See #837 for discussion.
Add base64conv to .gitignore
Rules.c: Do not convert from internal to target encoding in case we
are running hybrid mask. It will happen in mask.c after applying
the mask.
Mask mode: Reject 8-bit "codepage" placeholders with UTF-8 encoding.
Mask mode: Cosmetic/whitespace.
For pure mask mode using internal encoding, skip conversion if our
mask (after current truncation) can't actually produce any 8-bit
characters. This can make for up to a 2x performance boost.
Fix bug when using -min-len with pure mask mode.
OpenBSD-SoftRAID: Show correct ALGORITHM_NAME. Also drop the
FORMAT_NAME as it's identical enough to the FORMAT_LABEL.
Add memdbg info to --list=build-info
Mask mode: When iterating lengths in pure mask, show progress for
current length. See #854
OpenCL: Rename opencl_autotune.h to opencl-autotune.h to prevent
it being copied to the kernels directory.
OpenCL: Move GPU-side Unicode stuff to a shared header.
DEFAULT_ALIGN. See issue #317. Fix for double-format source file.
Break out common OpenCL kernel stuff to headers.
Oldoffice-opencl format added. Closes #719, see also #736.
OpenCL: Optionally use local memory for RC4.
Shared OpenCL headers: loop protection, dependencies and other
things.
oldoffice-opencl fixes. MitM has to be disabled during autotune
and benchmarking. Also some RC4 tweaks.
oldoffice-opencl: Workaround for AMD bug.
OpenCL RC4 tweaks.
OpenCL: More stuff moved to shared header.
Mask mode: Resume progress/ETA fix for -min-len. Purely cosmetic
bug.
AC: Mention --disable-pkg-config in doc/INSTALL (#864).
OpenCL: Fix a silly problem with Intel compiler for vectorized
bitswap.
Fix new unneeded lib dependencies.
OpenCL RC4 tweaks. Mainly no-ops for easier profiling.
OpenCL RC4: Use local memory if device is GPU. Actually this is
 extremely detrimental to performance on some (weak? integrated?)
GPUs like my laptop GTX 650M but I'm not sure what to base the
choice on and it's up to a 290% boost on flagship cards. We
have to work around AMD bugs as usual though.
OpenCL RC4: Regression fixes (driver bug workarounds) for Macbook
(nvidia GT 650M & Intel HD4000).
OpenCL RC4: Fix some macro names before they cause trouble.
OpenCL: Fix some header includes using brackets instead of quotes.
keyring-opencl: Optimize for 2.5x boost, add auto-tune, bump length
limit.
OpenCL oldoffice & krb5pa-md5: Fix --test=0 (they failed because we
required an LWS of 64 - but we can do with anything up to 64).
Add auto-tune to pbkdf2-hmac-sha512-opencl.
EIGRP format: Add cost support.
OpenCL: Fix a bunch of OCL_CONFIG names.
pbkdf2-hmac-sha512-opencl has a split() that unifies case but
missed the format flag telling about it.
raw-md5u and raw-blake do unify case in split() but lacked the
format flag telling about it.
Mait for events typo.
7z-opencl: Split kernel, auto-tune and use shared SHA2. This format
is so extremely slow that we still need to have it disabled in the
default config. It can takes minutes even with --test=0. Closes
#754.
7z-opencl: Modify algorithm name, reflecting that we do any AES on
CPU as opposed to GPU.
OpenCL shared SHA256: Optimize memcpy for 30-50% boost of 7z-opencl,
plus some device-specific tweaks. An alternate memcpy is defined
out right now but it may be better IRL (or once adding some test
vectors of varying length to 7z).
"Enable" the 7z-opencl format in john.conf.
pdf2john: Unicode fixes, and lots of fixes/fuzz for parsing files of
various origins/versions
MYSQL: Fix a bug with OMP for single-thread, plus some no-op changes
that makes for easier testing.
7z-opencl: Proper Unicode/codepage support.

JimF (657):
get_source() supporting salts too. Added to crc32 and sapG.
raw-sha1_li: always store complete (repaired) hashes
Loads of minor fixes for Sparc (endianness, alignment)
Sparc_Porting_and_OtherStuff-bleeding.diff
Use first two salts when benchmarking instead of top one.
wpapsk get_source()
Portability_issues_bleeding.diff
Use our own SHA-2 unless we have it in OpenSSL. This enables all SHA-2
formats even for users that only have OpenSSL 0.9.7. This also
means we can have the formats as plugins.
Reflect what SHA-2 we use, in ALGORITHM_NAME
Enhancements to cryptsha256 (17% boost)
--list=format-methods[:WHICH] option added
Updates to the dynamic format
Fix for disabled dynamic_7 leading to all dynamic >7 busted
JtR-Bleed-make-generic-fixes.diff
Jumbo-Bleed-ssh2-and-other-small-changes.diff
JtR-Bleeding-Linked-in-MMX-fix.diff
Dynamic-Sha256-addition.diff
Added numerous types to dynamic. New fake-salt code and format.
formats.h version update. pass_gen.pl update
JtR-Jumbo-dyna-fixes-after-source.diff
loader.c: Fix for loading bare hashes.
JtR-Bleeding-Dyna-SHA2-size-fixes.diff
Dyna-bleed-fixes.patch
Added OMP, removed a few compile warnings, depending upon compile
conditions, oSSL/GMP builds
Bug fixes for dynamic union crypt_key
SSE2 support for WPA-PSK. The code was taken and modified from my code
in mscash2. They are pretty different, due to how the main
format logic is (mscash2 split out just the inner function, this
one did not break out the crypt). But they do about the same
things. Also, in SSE2, mscash2 BE's the binary data. I do not
do that here, since the wpapsk.h is common code to
CPU/CUDA/OpenCL, and that is where that functionality is. So, there
simply is a call to alter_endianity_to_BE when the results are
stored at the end of crypt. There really should be no speed
impact, due to this fact.
wpapcap2john utility by Jim. C port by Dhiru.
wpapcap2john fixes from Jim
Here are a couple more fixes
Implement ldr_sort_salts(), currently hard-coded to only apply to
wpapsk formats. In the future we will probably want a format
struct entry for format->methods.salt_compare().
Some porting items. The typecasts are due to void* having an undefined
size (on some compilerrs). error C2036: 'void *' : unknown
size
Dynamic: Remove many more of the typecasts, putting them into anon
unions.
listconf.c: Ensure we have a proper version of NSS lib before calling.
bench.c: Get around a crash in VC (and mingw), in -test=0 mode.
These only use clock(), so if the bench is fast, there are 0 clocks
used. I simply put a 1 in the time slot, if it is 0.
mingw port issues
More mingw port issues
Another version check for NSPR in listconf.c
Some warning pasifications
pass_gen.pl: Support creating WPA-PSK test hashes!
dyna_17_BE.patch from Jim
Portablity fixes for VC.
basename portability 'fix'.
fix wpapcap2john warnings
Dynamic bugfix, spotted by new warning in gcc 4.8.0.
Fixed mem leaks, found by mshell.
Minor fixes.
Fix portability problem with "extern inline" stuff in misc.h
Bugfix for NT format, 32-bit x86-sse.S.
rar fmt in bleeding (VC), mem leaks, and trailing WS.
JtR-jumbo-dynamic_1014_fix_non_sse.patch
Move alloc/free out of parallel region in putty format
Move alloc/free out of parallel region in Gnome Keyring format
OMP fixes for FGT & RAR
JtR-Dynamic_validation_raw_hashes.patch
JtR-Dynamic_validation_raw_hashes-2.patch
Re-apply JtR-Dynamic_validation_raw_hashes-2.patch but with some
missing stuff added.
JtR_Bleeding_Dynamic_Salt2_UserID_Salt_Bug.patch
JtR_Bleeding_Dynamic_MEM_ALLOC_SIZE.patch
Dynamic_OMP.patch
JtR_bleed_Dynamic_bare_hash_fix.patch
JtR-bleeding-dynamic_SSE2_OMP_12_13_fixes.patch
JtR-bleed-dynamic-OMP-preformance-increase.patch
JtR-SunMD5-SSE2-scale.patch
JtR-Dynamic-MMX-Fixes.patch
Disable OMP for MMX in wpapsk format
Dyna21-crash.patch
dynamic-valid.patch
JtR-bleeding-dynamic-OMP-fix.patch
JtR-Unstable-crash-fix-dyna_21.patch
JtR-Bleeding-Dyna-big-hash-fixes.patch
Portability issues fixed. fork/kill are not standard. They are *nix
specific. I have taken it out of VC builds, and also removed the
options from DJGPP builds. I would imagine that mingw also
would have problems, but I do not have a build env under that
any more, so did not add defines for it.
SHA-256 incorporated into sse-intrinsics.c - and a test format for it.
Raw-sha-224/256: Use new shared intrinsics.
JtR-bleeding-SSE2-pbkdf2-headers-v0.patch
JtR-bleeding-agilekeychain-fix.patch
JtR-bleeding-encrfs-any-pbkdf2-fix.patch
bleeding-vcport.patch
testpara32 fixes for bleeding branch.
JtR-bleeding-pbkdf2-256-upgrade.patch
JtR-sha512-pbkdf2-and-aix-ssha-enhancements Conflicts:
src/aix_ssha_fmt_plug.c
JtR_OS_H_improvements-bleed.patch
JtR_OS_H_improvements-v3.patch
JtR_Dynamic_statics.patch
JtR_CC_pbkdf2_sha2.patch
../JtR_bleeding_Dyna_enhancements.patch
JtR_Dynamic_1300.patch
JtR-bleeding-Dyna-add-SPH-Tiger_Whirlpool.patch
A small patch to get the new rar5 to build under VC
Enhancement to RIPEMD
Dynamic enhancements
Dynamic speed increases, RIPEMD speed up and new Dynamic functions
missing parentheses in dynamic_fmt.c
JtR_bleeding_pbkdf2_sha512_bug.patch
JtR-bleeding-regen-lost-salts-rewrite.patch
JtR-rules-avoid-double-include-char-classes.patch
JtR-bleeding-defines-cleanup.patch
small changes to regen-salts
SHA512 into intrinsics, normalizing SHA1 SSE2i interface to be same as
SHA2
SHA512 into intrinsics
removed password len restrictions from pbkdf2_hmac_sha*.h files
improved speed of scrypt, about 2x
more changes to scrypt and correct valid
missed 2 files on last scrypt update
SSE added to raw-sha384/512 and some fixes to other raw SHA formats
SSE added to raw-sha384/512 and some fixes to other raw SHA formats
use a var, to allow for easier step debugging, to not have to walk
into the longcand function
some small changes in SSE SHA2 code
fixed bug in recent changes, if compiling in oSSL mode
Added OMP to raw MD4 MD5 SHA1 formats
remove OMP in MD4 MD5 SHA1 formats if built with 32 bit mmx-*.S
file(s)
Merges some of the macros roti now handles ROTR as well as ROTL.
 Removed some if XOP defines, since we now have the cmov instruction
memicked in non XOP builds
added aix-ssha* pbkdf2 hashes
added 125 byte password self tests (max size). Removed old oSSL
commented out code. Fixed issue where a SHA256 constant was used
instead of a SHA512. Should have caused no crashes, since the 512
constant is smaller, but waste of memory
Use code from pbkdf2_hmac_sha512.h. 2x speedup for oSSE, and gets
about 4x+ speedup for SSE2 (XOP gets more)
Allow the oSSL pbkdf2-sha* code to also be included in SSE2 builds.
Now pbkdf2-hmac-sha512_fmt_plug.c can fully use the header
functions, even when built with SSE. No reason to keep the internal
hmac function in this fmt file any more. It was used in cmp_exact.
Added max length password and salt to test cases. Fixed serious bug,
doing strlen on salt, vs using salt->length. The static salt buffer
is not memset (probably should be) in the get_salt, and thus, a salt
record 'can' have extra crap after the salt. Does not hurt anything
AS LONG AS you only look at salt->length amount. The recent patch
to use pbkdf2_hmac_sha512.h file lost the usage of salt->length and
instead used the strlen, which was bad busted. Found when a shorter
salt followed a longer salt in self tests. Things started to fail
unexpectedly, and were hard to track down as why they were failing.
added new file to help within pass_gen.pl
Fixed compile issue in XOP from recent changes
added loop count to sha*Crypt formats
Added an ifdef in format.c to allow development work to temp turn off
validations. Found useful in testing theories in format changes,
and testing speed, without having to fully get the format working
Added SSE intrinsics to SHA256Crypt. Still more work to do, but
currently this is working and faster
updated the sse-intrinsic .S files, removed memdbg.h from cryptsha256,
and fixed possible overwrite isse, and fixed rawSHA256_ng_i code
that eliminates final crypt adds. We were using SHA512 IV's and not
SHA256, boom
sha256-crypt: Turn off SSE for OMP builds, until OMP crash bug found
raw-md5: modified OMP threading computation, to hopefully improve AMD
scaling
Added max length passwords and max length salts to all values in
dyna_preloads. Also fixed numerous items within pass_gen.pl which
were for building dynamic hashes
Reverted 01a920146, raw-md5 OMP scaling attempt
Added max length passwords etc to dynamic in dynamic.conf, Updated
DYNA docs and a fix or 2 in pass_gen.pl
generic was broken, needed to use salt->length and not strlen of salt
Added SIMD code to SHA224/256 within Dynamic, and started some
significant core changes to dyna, to help use SIMD code for dyna
large formats
fix compile problems on x64 builds
moved the rounds= #define into the header
Major modifications to dynamic. SHA1 re-written from ground up. Now
uses 'flat' input buffer method, the same as SHA256. This improved
perforamnce of ALL sha1 formats, except sha1() (raw-sha1). That
format lost 15% speed or so, due to not being able to use
specialized key loader. However, it was already 20% or more slower
than the hand tuned raw-sha1* formats, so losing this speed race
within dyna is not a major concern, when all other formats that have
more complexity DID increase in speed, some up to 200%. Also, we
are NO LONGER limited to 55 bytes on SSE for SHA1. This patch also
removes the depricated formats dyna-27 and dyna-28. These were
 depricated, and only ran in sse builds, and were 20% slower than
cryptmd5, so were not of any use, and had a LOT of code spacer (1000
or more lines of code), and that code was VERY buggy. It simply has
been removed totally
increased max length of non MMX password input from 80 to 110 bytes.
all formats other than MD5/MMX layout have been increased. Also
formats NOT MMX safe will get 110 byte inputs even when the build is
an SIMD build. Before, these subformats were also honoring the 55
byte max length, even though they did not need to. Also fixed a bug
in parser which was not computing test counts properly if a TestM=
or TestF= was being used.
Added large hash FLAT sse code for MD5/MD4. Also added some native
FLAT format md4/5 to dynamic, and started adding these as 2nd re-gen
hashes for dynamics where the older mixed layout is faster. The new
FLAT is not limited by short max length passwords
new dyna conf file missed, and some problems with dyna generic builds
missed checking this new dyna conf file in
failed to build in OMP builds
Fix build problems on XOP systems
Fix build problems on XOP systems
increased PW length of mscash2 from 27 to 125 and username length from
22 to 128
OpenCL mscash2 password length bug fixed, and PW length increased to
125 byte JtR max length
mscash2-opencl: fixed bug in new 125 byte password code
Changed dynamic format names. Moved the expression to the algorithm
section
fixed algorithm name
Portablity issues, VC, and some other fixes
add some useful dynamic sub-formats
Better OMP for whirlpool. Use oSSL Whirlpool functions if oSSL
version high enough (much faster)
pass_gen.pl, add a -tstall switch. If that switch is used, then
it calls all of the functions, crypting 'password' with each.
Fix debug builds for dynamic formats with TestD= test cases
Cygwin64 make target!
Cygwin64 / MinGW64 self-test fix
#ifdef out some select() stuff in tty_getchar() that was used for
Cygwin but isn't needed (anymore?).
Update to hi res timer code.
Add a build target for win64-cygwin-x86-64-opencl and drop -static
from Cygwin64.
Dyna fixes.
Band-aid for Dynamic 1300 in generic build. Closes #361.
VC fixes
scrypt fixes
Add Jan Starke's rexgen cracking mode (depends on external library,
so default disabled in Makefile)
regexp now possible w/ wordlist
Dynamic salt fixes, salt sort and enhacements to raw2dyna
Dynamic "flat buffer" formats salt length bump
Fix for the new AESNI stuff vs. Redmond Operating System.
Fix VC build
Rexgen fixes.
More rexgen fixes.
Rexgen fixes (patch 3).
Memory debugging stuff.
add missing "memdbg_defines.h" file
Rexgen update 4
JtR-bleed-rexgen-updates-5.patch
pass_gen.pl update for dynamic 2000's formats
mmap() fixes for Windows
Disable Kerberos 5 18/23 formats for Cygwin
JtR-all-pass_gen_pl-2.patch
HMAC-sha256, allow longer salt. Original patch provided by Colm
O'Flaherty but this is reworked for current tree.
Autoconf, initial work.
JtR-autoconf-2.patch
JtR-autoconf-3.patch
JtR-autoconf-v3.patch
JtR-autoconf-v3a.patch
added use icc-S intrisic files fixed code in md5.[ch] and md4.[ch]
where they were not detecting we had oSSL. They were always
compiling the john_md5 stuff, even if ssl was there. modified
x86-64.h and x86-sse.h (arch.h) so that they include autoconfig.h
AND started to comment out things hadled in the autoconfig.h file
Modifications to sha2, to hide name clash problems, and MANY BE bugs
fixed
Starting work on getting sparc working with autoconfig
Added ability for setting env var CONFIG_MAINT=1 to re-enable the
auto-reconfigure logic in Makefile. Normal users will never set
this, so will NEVER autoconfig. Developers working on configuration
scripts can set this env var, to make sure they always have proper
changes in autoconf handled properly
Added better logic to SSEx code probes. Added a couple comments on
some should be done stuff (zlib and openssl). Fixed LITTLE_ENDIAN
setting logic, using proper variable
Updated CPU probes again. Looked at compiled ASM this time, and I
think this is it
Missed updating configure
Added rm -f to arch.h before we link it, so that it will be ln-s
properly.
Stripped out most of the targets from Makefile.in Now Makefile
will not contain the fat targets which we had in the original
makefile. The original makefile is still present for these fat
targets. make --file=Makefile.orig target will still work,
producing the output.
Added a tiny comment, to list end of the default target. Let it
'show'
Allows this to compile on sparc (generic sha2). It still fails, but
now compiles
Still had a few alignment issues to clean up
Changed HAVE_PCAP to be a proper define, in the makefile
Changes to makefile for cygwin/mingw building. Also added -DAC_BUILT
listing this was an autoconfig build. We will have to edit c and .h
files to allow them to do special things for non-autoconfig builds,
i.e. Makefile.orig builds. Not all of that work is done yet, but
now we CAN do it
Got make -f Makefile.orig working properly. autoconf stuff still
works properly also.
added size checking for all common elemental data types. Changed the
ARCH_INT_GT_32 check to use this information
Revert changes to makefile
Try to ignore Makefile in src, by explictly specifying that one
src/Makefile as a true file (cp Makefile.orig Makefile). Trying to
get rid of git seeing Makefile, even though it is in .gitignore
added config.cache to .gitignore, and set Makefile back to a ln -s.
NOT SURE HOW to fix Makefile to have something there, BUT be ignored
moved x86 special logic into a macro. PUtting this into macro cleans
up configure.ac, AND it makes it much easier to start adding logic
for other systems, and then within configure.ac, simply have a
switch on the system type, to call the right one. Also, moved the
ax_lang_compilerr_ms into a macro to clean house a bit, and a few
other small cleanups
Moved logic to build fmt_*.h and john_build_rule.h from the
Makefile.in, into configure.ac (configure). Also, these files are
NOT cleaned up by a make clean. They are cleaned by a make
distclean
Removed last legacy target for plugin stuff, it was not needed
Found a bug, where a john.local.conf was not loading properly, if last
line did not contain \n char. Now, adding the blank line and the
comment in john.conf AFTER the include of john.local.conf, it does
not matter. JtR will load the john.local.conf file no matter what
it ends like
Missed a need alignment core. Hopefully we are all 'good' now
Fixed BE failures
sha1 and sha512 were not quite right. Rewrote pbkdf2-hmac-sha256.
The inline code was a nightmare. Now all of these should be BE safe
Added AVX2 probe logic to x86 detection
The C source for apple was busted. The closing } was in the wrong
place. Also reduced some line counts in configure. Configure not
checked in, because I have another change coming also in a minute
Changed the openssl probe from simple lib probes, to using
PKG_CHECK_MODULES. Now we get proper cflags and libs required for
oSSL. This depends upon pkg-config 0.9.0 being installed. This now
[almost, some includes and unrarppm.c still need help] allows my
sparc box to properly work with ./configure && make
This file goes with prior commit (the moving oSSL logic into
PKG_CHECK_MODULES changes)
Fixed BE bugs in these 3 formats. Tiny bit hackish, but since they
were 3 combined formats, it sort of had to be that way.
Some ifdefs. YES, this breaks (possibly) the rar for sparc. But
without this, the compiles HANG on this file. We can fix it
properly when we have time. Right now, lets at least have this file
fully compile. As a bonus the self test still passes! Do not ask,
but it does
removed legacy stuff, which was causing an unused function warning on
SSE builds. Legacy format did not do SIMD code and needed this flag
still had a core in this one on BE systems
Fixed BE failes for dyna_17 and phpass
Fixed BE bug in format
Fixed BE bug in format
Fixed BE bug in format
Fixed BE bug in format
Fixed BE bugs in these formats
Fixed BE bug in format
Fixed BE bug in formats
Fixed BE format. This one sucked, due to only 2 bytes not even
containing enough to byte swap. NOTE, pbkdf2's have to end on
'even' wordsize for the pbkdf2 so we can swap them
Revert prior hack. Now, on some OLD sparc gcc compilers, this can be
built with -O0, and it builds. This still does not find the
problem, BUT we can not turn the proper code back on on a sparc
system in general
Fixed BE problem with this format
Fixes problems with BE OMP in this format
Removed the LIBS = for rexgen. This was blowing any install that has
 liblibrexgen.so, and causes almsot ALL ./configure compiles past
that location to fail, and it gives an unbuildable end result
Fixed for andriod FDE format, AND got it working on a BE system
Fixed BE problems (this one SUCKED to find)
Alignment not assured, but needed to be word sized
Added new utility macro m4 file Added macro to detect usablity of
gcc command args. Now CFLAGS extra args, OPT_NORMAL and
OPT_INLINE flags detected using this macro. Moved the include
and lib path resolver code into this utility class Added x86_64
and x86 lib path into possible OpenCL lib paths (after we detect a
32 or 64 bit build) Fixed --disable-openmp still showing
openMP=yes on status. Fixed sometimes blank GMP on status
rolled back this change. Having a CPU that does not align, is not
critical. Most aligned needed was in MMX_COEF. We will have to do
this on a case-by-case basis, and when we DO have alignment needed
on systems which do not support aligned.h, then we will have to do
the work in other ways.
With the order change of ossl, I was getting horrible failures and
cores on cygwin. This corrects the problem. NOTE, we may have to
readress this issue, this may not be the final solution
changed some test logic. Was not testing right on all bash
Added logic to check if the GCC argument testing code actually works.
If it does not work, we emit a warning, and revert to using only
-O2. Some additional cannonization of sh test clauses. Renoved
quotes from RHS, if they are only a small constant.
Commented out old CUDA logic in Makefile.in. Magnum, please validate
we can remove this, and then remove it
Added logic to add defines to CFLAGS and ASFLAGS for CYGWIN64
added win32 .exe to ignore
use x86intrin.h exclusively in XOP probe in configure script
Issues found in BE builds
Pacify cygwin warnings
Removed malloc/realloc which put BAD code out there when we have
configure compiling problems. Removed openssl header and linb auto
checks, moved them to their own macro and then put a commented
call to them in the non pkg-config probing of ossl Removed the
bogus check using -Wbogus Sparc compiled that one, so spurious -W
warnings might be passed in. The -f and -m seem to trigger the
compiler to sqwak and not compile (which is proper).
Added code to add AES into openssl, for the aes/aes.a IF oSSL does not
have aes. This was tested on my sparc system. It has oSSL, but
not aes. Now o5logon format works, and builds properly
added oSSL header comment to the CBC128 functions borrowed for AES
Set FULL_UNROLL on by default for AES. Now same speed as my oSSL
build
CFLAGS_EXTRA was needed, to get things like the defines for Cygwin64
handle string.h/strings.h wherever strings.h was being used. This is
now AC proper code
Added lower level AES functions to aes/aes.a These are oSSL
compatible. NOTE, we need to do this for the SIMD code
Updated the oSSL function checks This should work properly when a
system has pkg-config with some very non-standard oSSL settings.
It still should work if not using pkg-config, by doing the search
for the lib ssl and lib crypto.
added aesni asm for cygwin32 and cygwin64 builds! about 180% speed
improvement to o5logon
added the type string for WHICH type of AES code is being used. We
have to set this in the init() function, getting the algorithm
element, adding this to it, and then resetting it. This is
 because this is a RUN TIME check, not a compile time check
changed AES type strings
changed logic for cygwin/mingw .exe vs ln -s building of *2john
helpers
added quotes to AC_SUBST variables which might contain spaces. In
the AC_SUBST for CFLAGS_EXTRA we now pull in existing . Now with
this, a user can pass in CFLAGS_EXTRA, get those put into CLFAGS,
and NOT lose the configure handling of the CFLAGS variable. This
may help on some systems that are not cross compiling, BUT which
need to have some unique CFLAGS data entered. NOTE, I do not
know how to add this to the help screen. That would be VERY nice to
inform the users of. Also, we might want to add ways to do ASFLAGS,
and LDFLAGS without messing with the REAL variables and having
configure not auto probe them.
Correct fix for FreeBSD. Created a new m4 class for system specific
probes. Moved cygwin stuff here also, and streamlined cyg64
checking (no compile needed
the emms kills ETA on cygwin. I am not sure why it was added, but I
bet it kills MANY other system's ETA
Added 64 bit file handling for 32 bit systems. Moved 2 functions from
misc.[ch] into jumbo.[ch] which are new files
removed spurious execute bit from these files
Only do the LARGEFILE64_SOURCE if compiling 32 bit versions of these
OSs
added warning for 32 bit fseek/ftell
was not building on my sparc system, with ossl. The flags were not
there any more.
Some configure tweeks. aes Makefiles not quite right, moved some
system_specific stuff around in the macro
Added host os to the john BUILD string
64 bit on 32 bit compiler changes, required for solaris.
fopen64/_fopen64 options added. Also, _POSIX_SOURCE can NOT be set
prior to stdio on sparc if we expect any 64 bit file mechanics to
work properly
Patch needed to compile in VC
Fix for open issue of TS failures. What was happening was the
length of the input buffer was being cleaned, BUT the input buffer
was left dirty. This would cause subsquent runs to fail.
Since this is a MMX_COEF build, we needed to keep these buffers
clean, and losing the length of how much has been dirtied was
NOT a good thing.
Some fixes found in my alias code project. The alias still has work
left to do, but these bugs fixes can be pushed now
Fixes problems with krb5 not being detected and not showing in status
of configure run
Added AC_BUILT define wrapper to 32 bit fseek warning
Made Sybase-PROP BE compatible, AND changed non-static functions to
static or made them feal_ specific
set the krb5 check back to using AC_SEARCH_LIBS, instead of 2 separate
lib tests. Now we also ONLY set HAVE_LIBKRB5 instead of 2 of them.
This should make incorporation fo HEIMDAL easier, I hope
Still more krb5 issues (when krb5 was found, it was not showing yes
Fixed bug where OMP builds (x64), were failing with multiple defined
DES_bs_crypt functions
Fixed problems building with legacy Makefile.orig
Added SKEY algorithm into JtR as native code.
changed status to list skey now being built using internal code
Overflow bug fixed, now using temp var for crypt results. Using
unsigned int as parm type, to work around silly compiler warnings.
added the 4 SKEY formats
fixes for SKEY formats
Improved SKEY jtr native speed about 20%. Still just oSSL, but faster
fully handle 64 bit files, using autoconfig functions found (if using
autoconfig). If not, then origina fseek/ftell used. Also, we were
calling free on a mem_alloc_tiny object (that would be a core)
Needed to add jumbo.o to luks2john. Only needed for some builds, but
is needed
bare minimum MinGW port to build john. There will be more to do
moved sleep() for Win32/Mingw into jumbo.[ch], and added sleep() to
autoconfig
"Moving non-portable functions into AC detection. Handle !HAVE_LIBDL.
Move things from misc.[ch] into jumbo.[ch], and use AC, and non-AC
detection logic. There is still a LOT of functions to do. Some
are still not even in autoconf.ac. I simply did not want this patch
to grow too large. Better to put it in now, than later. Also,
there were several targets in Makefile.in where -DAC_BUILT was not
being set. Now that we are using more AC logic, we NEED to know
that. There was also a failure on non-standard path for one util,
needing openssl cflags
wow, huge problem in internal strcasecmp, BUT there probably will not
be many systems using that code
Fixes legacy builds
allow wildcards. I use config-cygwin.cache or config-mingw.cache in
the same dir
Bug fixes in the setenv jtr built function
Allow disable usage of pkg-config
Moved a few more non-portable functions into jumbo.h, and a few blind
function defines for _MSC_VER from misc.h
Added check for -ldl OR windows.h and if neither, then do NOT add any
GPU code. There is a warning emitted in such case for intel chips.
Also changed common-gpu.c to properly use the WINDOWS_H code. This
still should work on Makefile.orig, since HAVE_LIBDL is defined for
GPU targets
Fixes for HAVE_LIBDL or HAVE_WINDOWS_H for work. Also added to
plugin.c
Set AC_BUILT to 1. Also, added @PTHREAD_CFLAGS@ to uaf2john. It would
not link on mingw without this
Wrapped all unistd.h in wrappers. Made a wrapper for other headers
(fcntl.h, etc). Switched to using -std=c99 in MinGW, which
eliminates most of the printf formatting issues, BUT which caused
some porting issues
Reduced this OMP_SCALE a lot. This was taking a LONG time to run even
-test=0
Fix for legacy build issue with LLd
Fixes for broken krb5 systems, like cygwin. Removed the not HEIMDAL
defines. Changed ifdefs to #ifs for some HAVE_XYZ defines. WE
should do that across the board
added lbz2 to gpg2john. Code was there, just enabled it in autoconf
Removed 32 bit MMX/SSE asm. I still have Makefile.orig and dynamic
left to do
Added logic to fall back to MMX, prior to x86, IF SSE2 native test
fails
Changes needed to properly build a MMX, now that 32 bit mmx.S are gone
Reduced the OMP scale.
Removed legacy mmx.S code, mostly from dynamic formats
This should be the last file needed for the 32 bit *-mmx.S removal
Removed last traces of mmx MMX_COEF code. This was causing serious
problems in dynamic, and other formats. Even things were having
 probems building. NOW, DES/BF, etc still use MMX, but all MD[45],
SHA1 do not
Renamed Makefile.orig to Makefile.legacy. The .orig was a BAD idea,
as this was a special extension and prone to be overwritten and lost
Made a little more robust. Had some old projects with HAVE_REXGEN
defined. By adding this simple logic, it will work either with
HAVE_REXGEN or HAVE_LIBREXGEN being defined. It will still unset the
logic IF the lib is found and too old
small warning removal for MMX build code
pkzip is better, but still not 100% fixed
fixed pkzip format (old pkzip). Now we store both crc and timestamp
checksums, and test BOTH inside the format. If either matches we
assume the checksum is 'ok'. There will be a few more false
possitives, BUT there is no other way, since there is no idication
of what zip compression tool built the zip, and WHICH of the 2
fields were used. Also fixed bug where if the decompressed item was
128 bytes or less, we would fail
Fixes bug #245. The fix was also done in some of the fix of prior
commit
fixed many warnings from cygwin build, due to this function NOT being
defined in a header, even though it is exported and usable.
changed if statmenents to allow dynamic to properly use OMP
Removed MMX_COEF==2 logic from dyna, and output sph_ripemd for the
ripe_md functions
Not a 100% fix for dyna OMP, but shows what MUST be done. I can fix
this, I just have to add some logic to detect pure MD5/MD4/SHA1, and
make appropriate inc computations. WE ALSO have to deal with para
sha2 formats. The formats are ok if they evenly divide the para
used. THat is why the SHA1_para == 1 worked fine for MD5_PARA 3, but
SHA_PARA==2 would not
Fixed issues in Dyna OMP, for SSE types. Now all builds should be OMP
happy
Got the problematic dynamic formats working on OMP builds. These are
the ones that can NOT be done with OMP. They now are built, but run
without OMP, while all the other dynamic DO run with OMP in the same
build
Removed code that was commented out (OMP run of non-OMP-compliant
formats)
2 new dyna functions, 39/40. net-md5 and net-sha1
Fixed bug in new formats
Added a hybrid fat/thin dynamic into the net-md5 format. All salts
less than 230 bytes are handled by dyna. The others are handled
by the original fat code in net-md5-fmt.c
fixed silly compiler warnings
Bug fix for #644
make sure these dyna formats are allowed to use max length passwords
the hash will allow
changed prepare to NOT add 5768U and 5768Fx if the data is not valid.
We were making invalid hashes pass valid by doing this before.
This function should also be calling prepare
rolled back prior forced prepare change. Added code to output user
name IF it is seen in the fields. We may want to do this for other
fields also
If salt bytes had high bit set, then char would sign extend, and put
out fffffff0 instead of f0. Typecasting to unsigned char for
each char avoids the sign extend
Added hybrid partial thin Dyna_40 to net-sha1. Changed net-md5 to
show dyna_39 in algorithm. Changed net-md5 to use a non-numbered
variable (made it easier to port net-sha1 to dyna).
Added proper prepare to haval. Also split up to have 2 valid
functions
Added matching test string as we have for the raw hashes
added prepare function and matching test string to these formats
Added prepare where needed, multiple valid functions (where needed),
and duplicate test cases (where needed)
Some changes spotted on sparc system
Added code to create FULL passwd compatible strings in
-list=format-tests. These formats now generate these types (GPU
formats not tested): dynamic (15,35,36,37,1015,1401), MSCHAPv2
mschapv2-naive, nethalflm, netlm, netlmv2, netntlm,
netntlm-naive, netntlmv2. They probabaly should be retestd
Changes for msvc building
Additional test on dynamic thin encoding
Added longer salts to net-md5/sha1 formats
Some more VC ports
Make sure when we remove HEX from dyna, that we do NOT remove it for
unsafe characters (line feeds, colon, etc). This is for .pot display
changed utf-8 encode size for UTF_8 formats
Added call to prepare to .pot file loader
Rolled back f89acf9 and added split forwarding into dyna thin formats,
so they call dynamic.split when needed
re-wrote length function to deal with 4 byte utf characters in a JTR
correct manner
Enable 4byte utf-8 characters. Surprisingly, I saw a very slight
performance gain (in -test mode)
Handle 4 byte utf-8 characters #685
prepare normalize short hashes, part of bug #650
Cannonized MSCHAP. The cannonical format is the short format, with no
extra fields
removed dependancy on openssl
Removed openssl dependancy
Fixed long standing memory buffer overflow issue, listed in bug #690
zip-aes: This format did not actually work
zip2 format. #434 #691 Removed FMT_NOT_EXACT. Now fully detects
passwords.
Comment chagnes in zip2john, and OMP tuning in the format)
Removed PLUGFORMATS_SRCS from makefile. This was a legacy requirement
for knowing when to make fmt_structs.h headers. Now we do this in
configure, so this variable is totally unused (and HUGE). This is
cleanup
Better information given for buffer over/under flow issues. If block
listed as not being freed, then a memory dump (some of the start and
some of the tail of buffer) will be output. This helps track down
just WHAT was the problem, and is helpful, especially when a COMMON
allocation function was where the data was allocated.
rexgen is now disabled by default. If the user wants it, and has a
working version, then --enable-rexgen is required to get it used
added defined simple dynamic formats to the -tstall function
added split to static linked methods in dyna thin formats. Added
FMT_DYNAMIC to all dyna thins. Did some other work on dyna thins to
make sure they were linked properly
cannonical issue (embedded nulls). Bug #705
document trailing WS in dyna salt can be a bad thing, where core JtR
will strip it. raw2dyna now detects this, IF the user uses it
Fixed problems with thin dyna. Should fix #704 and corrects 6cbc306.
6cbc306 was needed, but moving the function linkage out of the thin
formats init, and always calling dynas own init was wrong. This is
now corrected
Removed Gladman HMAC code, choosing a faster saved state model, like
we do in PBKDF2
wrap the intrinsics.h file in #define fence posts, so it is only
included one time
Modified pbkdf2_hmac_sha1.h to also do PBKDF1 algo, and modified
crypt-sha1_plug.c to use it
Added SSE2 logic to crypt-sha1. Proper salt handling in crypt-sha1 and
fixed the get-iteration function. Removed crypt-sha1_plug.c that
code was not needed at all
added copyright, and changed algo string from PBKDF2 to PBKDF1
(copy/paste bug)
Fixed crash bug
better validation. Salt up to 64 bytes. use strnzcpy where needed
added Local config processing from john.local.conf. The documentation
was changed also, and configure will create a commented skeleton
file. Any non List. section can be handled in this way, updating
the param values to our local value. This keeps the user from having
to edit the john.conf file, which is a BAD thing to do, since it is
owned by JtR developers and user changes get smashed
Added code to handle HSRP hash, and removed Crypt::PBKDF2, by writing
better inline pure perl PBKDF2-hmac code, and using it
Added -genall and -rgenall functions
fixed a bug in LM and mschapv2. Compute rand salts by hand for all
Authen::Passphrase stuff, so that -nrgenall logic works. Added
non-rand for -nrgenall into the random generation functions. This
version now fully mathces (regression test), my version that drops
Authen::Passphrase usage.
Removed dependancy on Authen::Passphrase
added keychain to pass_gen.pl, and generated some extra test strings.
NOTE, the data which has been encrypted is the EXACT same data, it
is just done with differnt password, salt and iv values
2 pass_gen.pl updates. Added additional encoding tries in LANMan
function. setup_des_key now generates a deterministic key,
setting all parity bits to 0.
pass_gen.pl: Added word_encode as param #2. Changed output on
messages to STDERR
Updated to v1.20
mschapv2: improved speed of cmp_exact by elimination of very costly
call to binary() for each found word
mschapv2 - ntlmv1: Fix to cmp_exact needed to work for both formats
pbkdf2-hmac-sha256: bug fix. prefix check did not work at all
pass_gen.pl: some name normalization. Fixed salt smash bug in pbkdf2
code. added pbkdf2-hmac sha1 and sha256
pass_gen.pl name normalize for dynamic (now dynamic_xxx works for
all)
pbkdf2-hmac-sha256 validation bug. Same in the CPU format
rar5 SSE: Work started. This code change now uses
pbkdf2_hmac_sha256.h It does NOT yet do SSE2, but is starting to
set things up. NOTE, we will have to do the last 32 hmacs
outside of SSE2 (probably)
Got RAR5 working with SSE2. NOTE, OMP is broken. I have looked, but
see nothing wrong
rar5_fmt: Found my OMP bug (stupid me)
pass_gen.pl: fixed sub name broken by 742f99c
Dyna: Some debugging logic, dealing with seeing what was going on at
setup time for split
net-md5/net-sha1: Fixed so they properly process dynamic_39/40 format
files
net-sha1/md5 still did not properly handle dyna_39 input files
salt() self_test for proper cleaning
fixing salt() not properly being cleaned. Many formats to go
fixing salt() not properly being cleaned. Many formats to go
fixing salt() not properly being cleaned. Done with Cygwin64
Mozilla format: was getting memory overwrite error. Not sure WHERE it
was, but it's gone now.
krb5-18 has salt non-clean on ubuntu-64
formspring, proper binary size and binary/salt align for dynamic
Fixed binary size, binary align, salt size and salt_align for all
dynamic thin formats
a few MSVC specifc changes
Fixed memory overwrite bug
a couple changes to memdbg
bcrypt_pbkdf2 portability issues
RAKP: fixes the pre-clean buffer seen on sparc #763
sha1crypt: BE fixes, #763
hsrp: fixed BE problems #763
sha1crypt: small fixes to BE code
rar5: BE fixes for #763
mozilla: fixed comment
config.c: fixed bug #767
Config.c: bugfix for last change. appends option properly, and works
with an empty section without core
crypt: fixes bug #779. Adjustments made for crypt format, to properly
work with subformat or salt being passed in.
First draft of dynamic salts. Inserted into zip format. Not used yet
by John core processing
Fixed warning about salt alignment issues
dyna_salt. zip format now compares salts properly
dyna_salt Fixed possible bug, if salt1 size was larger than salt2
dyna_salt: Made a fmt_default_salt_hash specially for dyna_salt
formats
dyna_salts: added fmt_self_test salt smashing logic for dyna_salts
VC bug, strdup defined for memdbg builds was undone by this define
dyna_salts: added cleanup of salts dynamically allocated
dyna_salt, added defines for easier debugging
rar: added dyna_salt code. Also removed 2 unneeded vars from salt
struct, filename and pos. They are temp vars
rar dyna_salt size problem. Stupid bug
rar2john: Removed static sized HUGE inline buffer. Now it is
variable, and can be controlled on command line. For -p type 1 files
rar2john: fixed overflow and logic that would not find values if
maxinline set too small
cracker.c wrong data type used. Both linux and Win32 man page lists
clock_t as type
dyna_salt. Use 4 param COMP_SIZE macro
dahua: removed warning, properly typecast char* to int, and just use
single isalnum function/macro to do the work
rar_fmt: removed numerous warnings, and added comments where we can
not handle large file size
legacy makefile needed dyna_salt.o. Fixes #807
cygwin-64 Makefile.legacy build fix
dyna_1300. Fixes #786
Wrote myself some TODO notes. All set length function may need to
clean buffers for md5_go builds
rar2john: added trivial getopt for VC
dynamic_format: Bug in MMX raw md5 code
getopt: Added a missing_getopt.[ch] set of files for sys without them
msvc: getopts ports, many external processes now compile also
correction of 339052e and getting #825 working. Prelim and not the
final fix I want, but better
dynamic: finalized #825 fix
net-md5 handle core for --skip-self-test. #828. Bug much worse than
just the skip-self-test case. It could have given many false
negatives
net-sha1 handle core for --skip-self-test. #828. same bug as net-md5
oracle: Correct max length for user name
oracle: increased to 30 character user name
oracle: added a user name length manifest, to clear things up a bit
opencl pbkdf2_sha512: removed warnings about using char values in
array access and operations
opencl-hmac-sha512: changed typecasts to ARCH_INDEX calls
dynamic fmt. pure comment change
dyna salt. debugging flag was in wrong place.
common-opencl use ARCH_INDEX for char value to avoid warnings
dyna_salt fix (memleak) in opencl formats
pass_gen.pl: added cisco4, and added hooks for cisco8/9 when I find
out how to do them
pass_gen Added cisco8 and cisco9 hashes
cisco8 hash added to CPU format
cisco8: added to GPU. Fixed a few things on CPU side also
pass_gen: added scrypt and django-scrypt hashes
oracle8, trying to correct crash on travis
oracle8, bug in last push
memdbg: correct 64 bit issues. make memdbg_extra NOT be default
pbkdf2_hmac_sha256: Travis fix (I hope)
pbkdf2_hmac_sha256: try again, Travis (my last try)
cisco8: prepare, no need for allocation. Flat static buffer is just
fine
memdbg: stop huge error dumps. Also catches crossed pointers (infinite
loop), and shows start of memory
cisco9: Modified scrypt_fmt.c to also handle cisco9 hashes
pure text formatting changes
escrypt: use functions from JtR memory.c
removed a dupe comment line added in some prior checkin
memdbg: had OMP commented out. Fixes #843
pass_gen.pl: removed scrypt include from global, and put into subs
makefile.legacy: Added OMPFLAGS to link for extra programs, since
they link memdbg.o built by john build
Numerous files, removed trailing whitespace
base64 conversion changes. Only minimally hooked into Jtr right now
removed execute bit on base64_convert source
scrypt, fix stack smash after base64_convert changes
base64_convert: adjusted some memory size values
memdbg: ex mode, fix for #843
base64_convert: changes buffer size logic. Added tests for overflow
logic to exe file
scrypt: added ScryptKDF format, #368 fixed
snprintf: Fixes #851
scrypt: allow scrypt hashes built with more than 32 bytes
base64_convert: Enhancments and bug fixes. Added flags to contol
processing
base64_convert: missed flag
base64_convert: missed changes to opencl
oracle: fix crash on garbage file
scrypt: fix #852
SKEY: valid() must pass checks in salt() also
base64_convert: full pass test suite. NO memory allocation required
 any more
base64_conv: added len functions. Added mime with dot handlng. Removed
inline base64 code from scrypt and pbkdf2-sha256 formats
Fixed broken dyna40 as required by 01c44fd
dyna40 removed commented out broken hashed
net-sha1: needed to extend plaintext length
oracle: Partial fix for names > 16 <= 30 bytes
pcap2john: merged all pcap files. #838
cleanup of pcap2john.py
pcap2john: fixed var bug, commented out a couple for now
pcap2john.py: all handled now. Exceptions caught and ignored
configure: wpcap lib added to get cygwin working with pcap
pcap python, moved orignal files to unused
memdbg: moved from targets in Makefile.in into configure --enable
values
base64_convert: added flags to converts and flags to command line tool
pass_gen: added ODF, cleaned up many commented out code blocks. Added
some stubs
memdbg: added back as make targets. Removed memdbg_defines.h
pass_gen: allow -conten=xxx for ODF format
configure: cygwin with latest compiler release requires _GNU_SOURCE
also
NSS removal, some extra stuff still was left
base64_convert: fixed some bugs in base64-hex, added new mime alphabet
pbkdf2-hmac-sha1: added 2 new hash types in a prepare()
change signature to show this hash should get a utf-8 password
pass_gen.pl: added mozilla format hashes
mozilla: optimizations, removal of gladman hmac code
opencl_pbkdf2_sha1: add new passlib types in prepare
ODF: memset missed in salt function (found by TS)
pass_gen: put in wrong value into iteration for ODF (found by TS)
pass_gen: bug fixed in odf-aes code
pass_gen: wrong quote used
DEFAULT_ALIGN. Worked on cleaning up some of these (issue #317)
fixed BE core caused by f5366a2. NOTE all fmt_default_binary_hash_x
requires this alignment
openbsd-softraid: split long test string.
openbsg-softraid: fixed compile issue on certain versions of oSSL
openbsd-softraid: Fixed BE issues, reduced bin size
openbsd-softraid: wrong spelling
openbsd-softraid: reduced the binary size
zip: fixed crash on aligned systems, sparc, etc
zip, fixed BE and alignment issue
zip: one of the BE changes were missed
DEFAULT_ALIGN. Worked on cleaning up more of these (issue #317)
gpg: core fixed in BE caused by 6c1b599
DEFAULT_ALIGN. Worked on cleaning up more of these (issue #317)
DEFAULT_ALIGN. Worked on cleaning up more of these (issue #317)
DEFAULT_ALIGN. Worked on cleaning up last formats (issue #317)
DEFAULT_ALIGN. Removed define from formats.h. Fixes issue #317
dynamic BAD_OMP. from issue #375
rar: On vc, pthreads is likely not there
salted-sha1: added OMP support, fixes isssue #424
gpg: better valid. Will be a fix for #499, when ported
gpg-opencl: ported valid from CPU, with caveat
VMS: disable from BE builds
configure: BE output small bug
SIPdump: remove array index of char warning
tgtsnarf: warning about no trailing end of line fixed
EPI: sped up some, removed numerous punned pointer deref warnings
aix-ssha: removed some punned pointer warnings
several formats, removed punned pointer warnings
several formats, removed punned pointer warnings
configure: add ARCH_LITTLE_ENDIAN=1 to CFLAGS
configure: missed autoconfig.h.in in last commit
john: allow command line switch vs name of exe for extra utils (VC
only for now)
gpg2john: Fixed #477
gpg2john: mute subkeys (unless -S is used). Static added to many
globals
gpg2john: bug in last code. Algo 16 was broken by my recent changes
wpapcap2john: removed unverified key output (fixes #729 )
opencl-GRUB: add LWS/GWS output in verbose mode, as listed in #839
rar2john.c: POC code to add RAR5 support
crypt: problems with #defines and strdup
rar2john: switched vint handing from 32 bit to 64 bit in rar5 code
rar2john: wrong value in hash line, removed some comments, split
defines into header
removed exe bits from header
rar2john: Completed, merge rar5 into rar2john
rar5tojohn.py: moved to unused
dynamic_40: updated comments and screen output listing pad20, vs pad16
sse-intrinsic: SHA512 interface was 64 bit, but implementation was
using 32 bit../run/john -test=0 -form=bitcoin Now both use 64 bit
bitcoin: OMP bug in 32 bit linux, caused buffer to not align 16 bytes.
New helper functions created
memory.h: fixed spelling error
pass_gen.pl: odf making null padding on content too large. It worked,
BUT was more than needed.
rar2john.h some DOS eol in the file removed
rar2john: handle rar files where main header record is longer than 13
bytes (seen on rar files with ascii comments
benchmark-unify: updated to v1.8.0 formats
benchmark-unify: missed an older md5crypt
rar_fmt: removed allows_unaligned stuff in crypt all, and speed is
faster (5% or so)
rar_fmt: additional simplifications, simplifying prior to SIMD port
rar_fmt: even more simplifications
pass_gen: added rar4 -hp mode, -p mode should be doable
pass_gen.pl: added rar -p stored file processing
pass_gen.pl: added -p for compressed data
pass_gen: merged all 3 rar4 types into a single rar function
rar: put a hash of the data object into the salt structure
pass_gen: some cleanup
rar: -hp was also having same salt problems
opencl-rar: same issue (same salt) in this format as seen in the CPU
pass_gen: odf-1, found a bug in Crypt::OpenSSL::AES, and worked around
it
pass_gen: work around AES decrypt bug in a different way
pass_gen: hopefully last fix for perl bug on ODF AES
pass_gen.pl: added zip (AES) format
zip: Speling eror in comment, lol
opencl-pbkdf2-sha512: working on #539 issues
pbkdf2-hmac-sha512-opencl: Added code to allow all input types, per
#539
pbkdf2-sha512-opencl: issue #539, failing on some GPUs. Hopefully this
helps
opencl-pbkdf5-sha512: converted 8 to 64 bit from password and salt
opencl-pbkdf2-sha512: disable $ until we can get it working better
configure: --help message, work towards #612
configure: Fixed #612
configure: removed a couple stray \n from --help
configure help macro: added copywrite to edits, and some comment
changes
ecryptfs: better valid, and custom salt was not working
ecryptfs: added another non-default salt test string
pass_gen: added ecryptfs handling
pass_gen: added Office_2007. functionized get_salt, get_iv,
get_content
BFEgg: adjusted OMP_SCALE, removed a FIXME
bitcoin: improved valid() and removed many FIXMEs
pbkdf2-sha512-opencl: switch uint64(32)_t to ulong/uint, some other
cleanup changes
7z: improved valid(), removed many FIXMEs
cloudkeychain: use better pbkdf2, updated valid(), removed many FIXMEs
cloudkeychain: Added SIMD code to the PBKDF2-sha512
androidfde: Added SIMD code for pbkdf2-sha1, and removed a FIXME
blackberry: set OMP_SCALE and removed FIXMEs
missed one of my comments
cloudkeychain: reduced size of salt object and removed a FIXME
eigrp: improved OMP scaling
dmg: Added SIMD and removed a FIXME
dmg: MMX code should continue and not return
Makefile.legacy: removed old no longer c project mozilla2john
base64_convert: quite compiler warnings
base64_convert: bugs in base64_valid_length, flags not being used
properly
django: better Valid() and removed a few FIXMEs
EFS: better valid, ported to SIMD, cleaned up code
EFS: small changes to valid
gpg: improved valid even more
gpg2john: removed a couple fixmes
hsrp: Removed FIXMEs, OMP tuned
EPI and hsrp: removed some fixmes and OMP tuned hsrp
hsrp: Oops. last commit had code being copied the wrong way and was
lost.
mozilla: improved valid()
Some fixmes in KRB4, md2
sip: changed benchmarks to get 1salt/many-salt tests
net-md5/sha1: fixed salt length bug, and removed a FIXME comment
twofish plug: removed fixme and changed code to not crash on
REQUIRE_ALIGN machines
removed a few FIXMES
tcp_md5: fixed salt size bug, remvoed a FIXME
keepass: added twofish test hash. We will want to test on Sparc also,
since twofish may have issues
VTP: better valid(), OMP scaling, improved speed a lot. removed FIXMEs
eigrp: improved valid()
dmg-opencl: comment removed same as in CPU format
rsvp: improved valid()
dmg-opencl: comment removed same as in CPU format
efs: format renamed
lastpass: improved valid()
odf: improved valid()
pbkdf2-hmac-sha512: improved valid()
rawSHA256: added cisco4 and base64 processing where needed
rawSHA256: left DEBUG turned on.
 gpg-opencl: improved valid()
pkzip: removed a spurious fixme comment
pbkdf2-hmac-sha512: better valid()
new atou() function added to misc.c
7z: change to usage of atou() in valid and in getsalt */
7z: opencl fix to mirror CPU version
pass_gen.pl: added office-2010 and 2013 formats
pass_gen.pl: simplified the office formats with functions
OMP scaling for several formats which had OMP_SCALE=1 and FIXMEs
haval: OMP scaling (45x and 35x improvments seen)
nukedclan: improved OMP scaling (about 80% gain)
radmin: improved OMP scaling some
SIP: improved OMP scaling
openssl_enc: improved cygwin OMP speed 64x. Improved non-omp speed a
tiny bit
truecrypt: for sha512, switch to internal pbkdf2 header. SIMD for
sha512. Moved logic from cmp_all/cmp_one into crypt_all
truecrypt: salt align, fixed failure in not turning off sha512 and
ripemd160 in the whirlpool init

Dhiru Kholia (356):

Add support for cracking KeePass 2.x files.
Get rid of BOM.
Add support for cracking PKCS #12 files.
Use BIO_write instead of BIO_puts.
Add one more test vector in pfx format.
Fix OMP build.
Bump LINE_BUFFER_SIZE. Needed for pfx format.
Rename pdfdump to pdfdump_old and add new working pdfdump file
Add OMP support to TrueCrypt formats
Use correct value of BINARY_SIZE
Remove debugging code (linux-x86-64-clang-debug was giving warnings)
Fix warnings when building linux-x86-64-clang-debug target
Add support for cracking Siemens S7 authentication protocol hashes
Fix multiple bugs in sshng2john.py
Document usage of ssh formats
Fix problem in office format found by magnum by using valgrind
Fix problem in office OpenCL format found by magnum by using valgrind
The "cracked array" was not always reset correctly. Fix it.
Integrate dmg2john with john
Remove dmg2john target and stop confusing people
Fix warnings in wpapcap2john
Add wpapcap2john target in Makefile
Add sample test vectors for IKE PSK format
Add support for cracking Java KeyStore files (Ladghar special edition
patch)
Fix running of KWallet format on big-endian systems
The new split() in pbkdf2-hmac-sha512 did not use new interface
prototype in bleeding.
Add support for WinRAR SFX files
Fix warnings in dmg2john
Add ssh2sshng.py utility for converting ssh hashes into sshng hashes
Information on cracking Oracle APEX hashes
Misc. fixes for putty2john utility
Add support for handling multiple files in uaf2john
Add support for handling multiple files in truecrypt_volume2john
Dynamic type for md5($p,$s) for long salts
Remove redefinition (issue #233)
Move office2john.c to unused directory
Fix ASAN crash in pkzip format
make valid() "stronger"
Fix ASAN crash in dmd5 format
Document known ASAN related crash in ipb2 format
Python port of dmg2john utility
Commit another weird and custom format
misc. fixes
fix resource / memory leaks in dmg2john.c
Remove dead code
Remove dead assignment
Remove unused code
Fix ASAN related crash in nk format
Fix compilation failure with Clang (trunk)
Fix memory leak + "invalid read" in pwsafe formats
Fix "invalid read" in keepass format
Fix "invalid read" in ike format
Fix misc. "invalid read" problems
make valid() stronger in clipperz format
make valid() stronger in radmin format
make valid() stronger in pst format
make valid() stronger in mozilla format
fix potential leaks in dmg format
fix memory leaks in zip2john
fix typos
fix warnings in gpg2john
misc. small fixes in dmg2john
Fix ASAN crash (without any performance loss)
Fix problems pointed out by Frank
Fix ASAN crash in oracle format
Fix ASAN crash (still needs better valid)
Add support for cracking bitcoin wallet files
Fix problems found by Alexander Cherepanov in mozilla format
Fix ASAN crash and (get|set)_salt methods
Fix multiple ASAN related crashes
fix clang-analyzer warning in mozilla DES code
Fix mistakes made in commit 23dc44df
fix potential memory leaks in rar2john
Hopefully the "right" fix for the zip crash
misc. fixes for zip stuff
Fix a typo in keepass2john
Fix base64 decoding bug and add support for Cloud Keychain files
Add support for cracking Cloud Keychain files
Make agilekc2john work under Python 3.3 too
Make agilekc2john work under Python 2.4 to Python 3.3
Rename agilekc2john.py to something more correct
Agile Keychain optimizations (thanks atom!)
Calculate only what we will use (thanks atom!)
Even more optimizations (courtesy of atom)
Add support for SHA-3 (Keccak) hashing function
some less-than-ideal fixes for keyring2john
misc. portability fixes
more portability fixes
Make dmg2john work under Python 2.4 to Python 3.3
misc. Python portability fixes
even more portability fixes
Remove unused code from ml2john.py
Make odf2john work under Python 3.3 too
Make office2john.py work under Python 3.3 too
Make sshng2john work under Python 3.3 too
Make sshng2john work under Python 2.4 to Python 3.3
portability fixes
Fix crash for "qpdf" generated file and misc. PEP8 fixes
Add error handling in pdf2john
Fix various crashes found by Alexander Cherepanov
Document that mozilla format fails on big-endian boxes
Don't be too greedy. Fix failures caused by 07a3e05.
Add support for cracking AIX smd5 hashes.
Add aix2john utility
Add support for "standard" smd5 hashes
Add support for "SSHA variants" in aix2john
Add support for cracking AIX ssha* hashes.
Add "well-known" test vectors
Add one more hash posted on john-users as test vector
misc. fixes for AIX stuff, recommended by magnum
Fix aix2john to use existing MD5 format
Add more test vectors in MD5 format
Add more test vectors in AIX format
Fix build for older versions of OpenSSL
Add variant of aix-smd5 format to handle longer password lengths
fix crash in ike format
fix crash in dmg format
borrow valid from the corresponding CPU format
fix crash in keyring format
tighten check in episerver format
Add support for cracking HAVAL hashes
minor tweaks
Add support for cracking Tiger hashes
Add support for cracking ripemd hashes
Add support for cracking LUKS
fix self-tests in LUKS format
Add support for cracking Whirlpool hashes
fix yet another crash in dmg format
check maximum hash length in episerver format
check maximum username length in mongodb format
fix crash in putty format
New dynamic types from Alexander Cherepanov and "Waffle"
fix crash in clipperz format
fix crash in office format
fix crash in sxc format
fix crash in wbb3 format
fix crash in pdf format
fix leaks and a crash in LUKS format
pbkdf2 usage optimization in zip format (thanks solar and sc00bz)
Cracker for files encrypted using "openssl enc" command
Add support for cracking Android FDE
fix name ;)
Add support for cracking RAR 5.0 "hp" mode archives
minor fixes for rar5 format
make androidfde2john run under Python 3.3.x
Update OFFICE file
OMP and binary_hash fixes for bfegg format
OMP and binary_hash fixes for dominosec format
dominosec optimizations
try to fix build on old Mac hardware
try to fix build on old Mac machine
Add support for cracking MD2 hashes
Add support for cracking PANAMA hashes
Add support for cracking Snefru hashes
Add support for cracking Skein hashes
bitcoin format also works for litecoin wallet files!
Add support for cracking scrypt stuff
minor fixes for scrypt format
Add support for SHA3-256 hashing function
get rid of DEFAULT_ALIGN from panama format
get rid of DEFAULT_ALIGN from radmin format
get rid of DEFAULT_ALIGN from blake format
get rid of DEFAULT_ALIGN from Keccak formats
get rid of DEFAULT_ALIGN from ripemd format
get rid of DEFAULT_ALIGN from skein format
get rid of DEFAULT_ALIGN from haval format
get rid of DEFAULT_ALIGN from scrypt format
ensure salt alignment
fail a bit more gracefully for -p mode RAR5 files
get rid of DEFAULT_ALIGN from snefru format
get rid of DEFAULT_ALIGN from whirlpool format
get rid of DEFAULT_ALIGN from tiger format
get rid of DEFAULT_ALIGN from ripemd format (again)
get rid of DEFAULT_ALIGN from putty format
get rid of DEFAULT_ALIGN from putty format, this time for real
get rid of DEFAULT_ALIGN from md2 format
Add support for cracking blockchain "My Wallet" files
blockchain format actually works now ;)
Use SIMD in blockchain format
blockchain format optimizations
make use of IV in blockchain format
GPU support for blockchain format
fix build problem caused by blockchain format
fix function prototype in mscash2-opencl format
fix OMP build problem
some "dangerous" optimizations in the blockchain format
copy optimization made in blockchain CPU format
small optimization in ssh-ng format
add GPU support for cracking GNOME Keyring files
add support for cracking 7-Zip files using header encryption
add GPU support for cracking 7-Zip files (aka GPU killer format)
Fix Issue #311 (use Base64 instead of hex encoding)
remove unused code from bitcoin2john.py
remove broken feature from bitcoin2john.py
add support for cracking IPMI 2.0 RAKP (RMCP+) hashes
fix slow hash loading problem in RAKP format
somewhat better valid for RAKP format
ipmi_dumphashes2john utility is no longer required
fix whitespace errors
Cracker for BlackBerry Enterprise Server 10 hashes
avoid naming conflicts with OpenSSL
add documentation for cross-compiling MinGW builds
MinGW portability fixes # 2 (for Makefile)
MinGW portability fixes # 1
GMP is not worth the trouble for MinGW builds (revisit later)
a format for CRAM-MD5 cracking
CRAM-MD5 format actually works now ;)
format for cracking eCryptfs
add ecryptfs2john utility
add more test vectors in ecryptfs format
add another test vector in md5ns format
fixes for linux-ppc64 target
fixes for OMP linux-ppc64 target
ecryptfs format is actually correct now
early preview of EFS format for cracking M$ EFS
commit bwtdt format before I lose it!
move "bwtdt" format to unused/
pdf2john fixes for bugs triggered by CMIYC 2013 files
add more test vectors in pdf format
add another test vector in keepass format
add another test vector in odf format
add another test vector in pkzip format
some KWallet format "improvements"
more KWallet fixes, thanks to gaganpreet!
fix math in kwallet format
commit broken MD5 format
get the build target right
building gpg2john explicitly is no longer required
fix typo in "bitcoin-qt" string
document steps required to crack bitcoin wallet files
add a note for building JtR-jumbo on FreeBSD
misc. ASAN related tweaks
fix stack-buffer-overflow detected by ASAN
use our standard license wording in 7z format
"fix" licensing in blockchain formats
remove a "critical" function from ecryptfs format
"fix" licensing in agilekeychain format
fix licensing in agilekeychain OpenCL format
fix licensing in LastPass format
fix licensing in cloudkeychain format
fix licensing in 1password2john.py utility
fix licensing in office2john.py utility
fix valid() in fde format
fix valid() in blackberry-es10 format
fix valid() in cloudkeychain format
fix valid() in blockchain format
fix valid() in blockchain-opencl format
fix valid() in rar5 format
fix valid() in eCryptfs format
fix valid() in 7z format
fix valid() in efs format
clean up dmg stuff a bit
"temporary" format for cracking RIPv2 MD5 hashes
add more test vectors in net-md5 format
add OSPFv2 test vectors in net-md5 format
Start using Travis CI
add "win64-mingw-x86-64" build target
make "win64-mingw-x86-64" build target work
build putty2john.exe properly
add support for cracking TCP MD5 signatures (BGP RFC 2385)
stop using "bzero" function
make debugging of "win64-mingw-x86-64" builds easier
fix broken MD5 handling in gpg format (thanks Lukas!)
add support for cracking "SYB-PROP" hashes
fix ASAN crash in "sybase-prop" format
RAR 5 "-p mode" support
handle multiple encrypted files in a RAR 5 archive
add "-p mode" test vectors in rar5 format
change "join" to "john" ;(
Add support for Siemens S7-1500 PLCs
handle "SPEC_SALTED" case properly in gpg parser
handle "SPEC_SALTED" stuff properly in gpg format
add lotus2john.py utility
Add support for cracking Redmine hashes
fix issue #456 (lotus85 format is seriously broken)
eliminate false positives for PGP keys using 2 byte checksum
assorted fixes for GPG format ;(
add support for "IDEA" in gpg format
we have a local "IDEA" implementation now
partial fix for issue #472
more fixes for gpg format (issue #472)
remove multiple definition of "idea_options" (fix MinGW build)
add support for cracking XenForo hashes
slightly better support for XenForo hashes
tweaks required for building on FreeBSD 10 (32-bit)
enable static win64 MinGW builds
fix overflow in clipperz format (fixes #493)
fix building of luks2john with ASAN enabled
fix building of luks2john (again!)
whitespace fixes for dmg2john.c
LUKS fixes (format change!)
add python port of keychain2john utility
turn off debugging by default in "grub-opencl" format
make kwallet2john.py executable
Use existing LDFLAGS for vncpcap2john.c too
add support for NetBSD's sha1crypt
fix memory leak in krb5-23 format
add documentation about kerberos auditing
fix a possible typo in formats.c (promiscuous)
add support for cracking known_hosts hashes
Old habits die hard ;(
fix typos in keychain2john.py utility
add "known_hosts2john.py" utility for processing known_hosts file(s)
fix "crypt_key not specified in enclosing parallel" compilation error
SecureMode tutorial
add support for handling multiple files in rar5tojohn
RAR 5 hashes from CMIYC 2014
advertise rar5tojohn.py instead of simply bailing out
Fix algorithm name in RAR 5 format
Respect iterations in RAR 5 format. Hash re-extraction is required.
Add documentation for cracking Apple DMG files
rename DMG cracking tutorial
fix a very bewildering false positive scenario in o5logon format
Guard OpenMP block properly in o5logon format
Add support for cracking HSRP MD5 authentication hashes
add license in hsrp2john.py
add license in rar5tojohn.py
get the dedication bit right
Support cracking EIGRP hashes. Highly speculative!
Minor fixes for ssh-ng format
misc. minor tweaks for recent cisco formats
add support for cracking NTP authentication hashes
slightly better NTP packet parser
fix a weird zip overflow bug
Add support for cracking VTP (Cisco L2 protocol) hashes
add support for cracking sha1($salt.$pass.$salt) hashes
Add support for cracking Dahua hashes
add more dahua test vectors
Add support for cracking RSVP authentication hashes
Fix and enable SHA-1 support in RSVP format
slightly better valid for vtp format
 Add support for cracking IS-IS authentication hashes
remove exec. bit from a bunch of files
Support cracking BFD authentication hashes
Support SHA-1 hashes in BFD authentication
slightly saner EIGRP format
more EIGRP fixes
Enable cracking of more EIGRP hashes
More EIGRP related fixes and improvements
Add support for cracking MDC-2 hashes
disable the old and crappy mozilla format
clean rewrite of the mozilla format
mozilla2john.py helper script
restore the partial hash pre-computation trick
Add support for HMAC-SHA-256 hashes in EIGRP format
Handle custom salt in ecryptfs2john.py [1/2]
Handle custom salt in ecryptfs format [2/2]
Add a test HMAC-SHA-256 hash in EIGRP format
Fix memory clobbering in EIGRP format
Clean up unused/ folder
Clean up weird comments in dmg format
Clean up weird comments in blockchain format
Clean up comments in pcap2john.py
Clean up kwallet2john.py (PEP8 fixes + a nasty logic fix)
More EFS clean ups
make little things right (TM)

Claudio André (192):

Selectively pick stuff from Claudio's experimental tree.
Get some nt-opencl fixes from Claudio
Rebase on bleeding and copied 'my' formats.
Fix small tweaks after merge.
Remove any (#if FMT_MAIN_VERSION > 9).
Small tweaks.
Another small tweaks.
Missing parameter.
Fix the misaligned in xsha512.
Change spaces to tabs.
Space to tabs in john.c (only at lines I put there).
Fix the misaligned in xsha512.
Change spaces to tabs.
Space to tabs in john.c (only at lines I put there).
Create comments that document new shared functions.
Use sha.h and sha2.h as recommended at
http://www.openwall.com/lists/john-dev/2013/01/21/20.
Replace spaces to tabs (only).
Fix some misalignments.
Reintroduced some missed code during bleeding merge.
Fix a bug in benchmark.
Print the result transfer speed.
Move the duration config control to opencl_get_user_preferences().
More BINARY_ALIGN & SALT_ALIGN stuff.
Put copyright information inside common-opencl files.
Remove some trash from comments.
Small tweaks.
pwsafe-opencl: Auto-tune global work-size.
Move some printouts to common code.
Move more code to common.
Cache compiled kernel for CPU too.
Event more code moved to common.
Applies the same idea of 84fda0e04d414a77bdaebf58997dfa119768c794 to
pwsafe.
Mute some over verbosity find_best_lws (as done in
016f973d34b7ca34981acb7f078a0750a6d15d29)
Revert some bad changes i did.
Forgot to declare an extern function.
During OpenCL initialization start only requested devices (not all
available ones).
As defined in guidelines, include the 'new_keys' test.
To be more error tolerant, only working devices will be listed as
ready to use (it makes sense). Well, if every requested device
fails, there is a new check that should make the experience nice.
Use shared code in OpenCL raw-md4 and raw-md5 files.
Remove a comment that does not apply.
Include the duration check for GCN and stop auto-tune if 200ms limit
was reached.
Use shared code in OpenCL md5crypt.
Revert the 200ms kernel execution check.
Now it runs an OpenCL build on a host without *any* working OpenCL
device or platform.
OpenCL formats will be listed only if they can be used.
More than 25% performance gain on sha512crypt on VLIW5.
Small tweaks.
Fix a misalign.
Great improvement on VLIW5 (buggy but passes self test).
Clean-up.
Hopefully the bug is gone.
Tested with TS and 'real' cases from
http://openwall.info/wiki/john/development/GPU#Acceptance-Tests.
Everything seems to be Ok and on VLIW5 (at least mine) it runs at 2x
faster.
Better bounds detection Or I fixed a bug (i haven't found) or I
create a new one.
Now opencl_find_gpu is called only if there is (at least) one valid
OpenCL device (previous patchs). So opencl_find_gpu must point to
one valid OpenCL resource (even if it is a CPU). It means
opencl_find_gpu will point: - to an OpenCL CPU. - or better,
to an OpenCL GPU (if any).
Allow sha512crypt to be used on no_byte_addressable hardware.
Refactor opencl_find_gpu() in order to show its real functionality.
GCG! Ah GCN! Are you kidding me?
Change a comment about binary kernels generation.
Fix a problem on office2013 kernel on AMD.
Try to detect if the OpenCL library exists on runtime.
OpenCL SHA2 formats have to follow CPU get_binary code.
Better auto-tuning setup (for split kernels).
Wrong profile event was in use.
Revert "Try to detect if the OpenCL library exists at runtime."
Fix for Sayantan's formats.
Use default uint64_t definition (stdint.h) for non-OpenCL code.
Include uint64_t definition (stdint.h) in common-opencl.h.
Add FMT_SPLIT_UNIFIES_CASE to a bunch of SHA-2 formats.
Fix bf-opencl after a bogus merge conflict resolution.
Fix des-opencl for bleeding
Move salt transfer to set_salt() for sha256crypt.
Move salt transfer to set_salt() for sha512crypt.
Move salt transfer to set_salt() for xsha512.
Proper event usage for LWS auto-tune.
Remove the sha256crypt-ng: it is not necessary anymore.
My branch becames the reference to the next merge.
Small tweaks in sha512crypt OpenCL code.
A proper fix to the incapacity to deal with passlen=24 AND saltlen=16.
Previous commit can handle it, but since it hurts performance a
litte, this is a best solution.
Remove unused blank lines.
Memory flags reviewed at NVIDIA OpenCL Best Practices Guide (chapter
3.1).
Fix a memory leak.
Mute some warnings on OSX.
Create a cisco type 4 OpenCL format (re-using raw-sha256).
Fix a bug inside -DDEBUG code.
sha256 (raw): support max. length of 55 and minimize transfers to
actually needed size.
Inform GWS auto-tune routine if the format has a maximum
global_work_size it can handle.
Transfer partial data while JtR is still creating keys.
Compute and show index transfer time for sha256 (raw).
Merge raw-sha512 and xsha512 in order to use only one file.
Fix a bug on GCN at bb06e7e589fa0e74c566f95861513c29f540f7cf.
Mute a warning on OSX.
strlen() was an important bottleneck.
Initialize all important GPU buffers.
Use a better default LWS for sha256 (raw).
Small performance tweaks for sha256 and sha512.
Remove the '__local' version of sha512 (raw) kernel.
sha512 (raw): support max. length of 55 and minimize transfers to
actually needed size.
Transfer partial data while JtR is still creating keys (sha512 [raw]).
Mute (again) a warning on OSX.
Cisco type 4 fix
Remove unused xsha512 file.
Unify valid() and binary() for crypt-sha512 formats
Finish 0b7a3f5efbbbad94c421e34855b7168f04d4d92d (unify common
functions).
Improve error handling and messaging while at --list:opencl-devices.
Call OpenCL related functions only at format init(). - also, use make
sequential_id an int.
Remove unnecessary OpenCL init() functions.
sha256crypt-opencl fix for opencl_preinit().
Revert to do a full check (device is really working) while
initializing a specific device.
Unify OpenCL raw-sha256 and cisco-4, just like the CPU formats.
Follow naming convention specified at: a77b867.
Unify valid() and binary() for sha256crypt CPU and OpenCL formats.
Remove (old) debug code.
Remove fake multi-GPU support from sha256crypt.
Improved shared LWS/GWS test (see 261d7b1).
Document opencl_find_best_lws()
Try to fix a regression on 13.4 Catalyst driver (revert performance to
13.1 situation).
Unify common/shared code in order to try new LWS/GWS enumeration
schemes. See: 24e0565.
Continuation of 24e0565. In order to apply the new LWS/GWS scheme,
some shared code is moved to a new function.
Remove duplicated (auto tuning) code.
Use common auto-tune functions for RAKP-opencl
Move duplicated code from OpenCL formats to autotune.h file.
OpenCL bug fixes and tweaks.
Move auto tune functions to its own source file.
Band aid fix for Intel HD Graphics auto "tune".
Use real auto-tune for Intel HD Graphics.
Start using pointers to OpenCL events. So, from now on,
crypt_all_benchmark() is not mandatory for formats using shared
auto-tune code.
xsha512 format will not crack OS X 10.8 or higher (see
6947648242ca51260945d2dab8e1af129362fbc2).
Use auto-tune always (at least, for this fast formats).
Replace spaces with tabs in common-opencl.c file.
User should be able to --list=opencl-devices -dev=0. See:
cb4d82165f6257fc710e1b6c2179514db83e9e2e
Replace spaces with tabs in common-opencl.h file.
Fix: select a proper name for a function.
Deprecate opencl_rawsha512_fmt_plug.c and opencl_xsha512_fmt_plug.c.
See #676.
Missed one file in 0287d486a8f21888ec9f111a74a790d00da6b5fa.
The get rid of the "-ng" part in OpenCL formats raw-sha512 and
xsha512.
Small tweak.
Fix a typo.
Grammar fixes.
Use shared code for auto-tune in rar5.
Small tweaks for LWS auto-tune (rar5 and PBKDF2-SHA256).
Fix a few errors in shared auto-tune code.
Refactored get_multiple macros. Improved auto-tune default behavior.
Small tweks.
Document the usage of john.local.conf for OpenCL settings. Remove
unnecessary examples.
John.local.conf should be used to keep OpenCL settings.
Shared OpenCL autotune tweaks.
Optimize raw SHA512 to be ready to latest driver (14.6beta).
Move shared macros to the header file.
Small tweaks.
Remove more unnecessary code.
Shrink the kernel a little bit more.
Remove one unused var.
Clean raw SHA256 code.
Rewrite some comments.
Remove one unnecessary math operation.
Comment about a pending task.
Fix #392 (at least on AMD driver 14.6beta). No regressions (at least
on bull).
Avoid unnecessary SWAP in cryptsha512-opencl. - 8% gain on GTX 570
(now it runs at 14.2K c/s) - Also, on HD 6770 (driver 14.6beta)
achieve the original performance (from 12.6 driver).
Clean up lotus5-opencl initialization.
pbkdf2-hmac-sha256-opencl off-by-one buffer overflow fix.
Optimize sha256-opencl further.
Fix an error in sha256crypt-opencl. Now it passes JtR-TestSuite.
Small tweaks.
Increase HASH_LOOPS for sha256crypt-opencl.
Remove commented code.
Autotune for sha256crypt-opencl (it is fast enough).
Fix a formatting problem in sha256 OpenCL kernel file.
On --list=opencl-devices show PCI information for NVIDIA cards.
Last but one improvement for sha256crypt OpenCL file.
Revert to use a safer memory access patern. Need to investigate what
is wrong with the program.
 Changed the API of OpenCL auto-tune init(). Make a relation between
the ADL/NVML device to "our" device number, using PCI ID (#750)
Clean code in SHA2 fast OpenCL formats.
Fix a bug in auto-tune seen in NVIDIA boards.
Get the NVIDIA compute capability information.
Optimize sha512crypt-opencl kernel.
Fix a compilation problem on OSX.
Minor clean-ups on sha512crypt (OpenCL) prepare.
Shrink prepare binary code. On AMD driver 14.9 they reintroduced a
bug, workaround have to be applied again.
Well, use this setup to fool the AMD driver madness.
Fix a couple of comments.
Shrink sha512 prepare further.
Use __constant if possible (sha512crypt-opencl).
Do a sane job during OpenCL initialization.
Optimization for NVIDIA SM 30 and beyond . Gives 2000 c/s on GTX
Titan. - For future usage.
Fix to #739.
Ops: I missed one cast.
'Improve' Raw-sha1 initialization too.
Performance improvement on HD 6770 (4%). - also, apply magnum's
request to auto-tune always.
Better for auto-tune on NVIDIA.
Small tweaks for cryptsha512.
Fix for auto-tune using Intel GPU.
Avoid any type conversion between int and uint.

Frank Dittrich (166):

Add get_source to --list=help:format-methods output
Add binary and salt alignment to --list=format-[all-]details
Fix typo in doc/README-OPENCL
Fixed some more typos
Add binary_hash_[56] and get_hash_[56] to 16 formats
mozilla_des.c and .h: #if 0 unused functions
sap2john.pl: improve coding style (tabs vs. spaces)
doc/OPTIONS: remove the only [tab] which is not part of a heading
Remove dead code from mozilla_fmt.c - used #ifdef DEBUF
Fix grammar
Adjust benchmark-unify to jumbo-8 format name changes
Benchmark-unify: map KRB5 format names
Fix a bug when using --stdout, introduced in e14ac00.
benchmark-unify: adjust format name mappings Add "Mozilla SHA-1
3DES" mapping Adjust "pdf" mapping
Fix bash completion for formats if --list=hidden-option has
--format=CLASS
Bash completion: Workaround for --format=CLASS parsing
Let --list=format-methods handle new format methods
benchmark-unify: sort format name mappings
chmod a-x for several files that don't need to be executable
Rename formats: "Office 20xx" -> "MS Office 20xx", "M$ Cashe" -> "MS
Cashe"
Enhance doc/README.bash-completion, mention prerequisites
dynamic.conf: Move the dynamic_1501 format definition (Redmine)
chmod a-x run/dynamic_flat_sse_formats.conf
dynamic_flat_sse_formats.conf: adjust algorithm names
dynamic.conf: avoid nested brackets [ [ ] ] in john --test output
Add a space in dynamic_18 format name
Add --list=format-tests.
Ignore "reserved" dynamic formats for --list=subformats
Makefile: Detect which address sanitizer option to use for clang debug
builds
Revert "Makefile: Detect which address sanitizer option to use for
clang debug builds"
doc/Option: Add --list=format-tests description
Some bash completion improvements
Fix typos in comments (run/repeats16.conf, run/repeats32.conf)
doc/OPTIONS: "-example ciphertext" added to --list=format-details
Grammer fix in doc/EXTERNAL
In core doc files, use GNU-style double dashes for long option names
1,$s#-for=#-form=#g in run/DYNAMIC, because -for= is ambiguous
doc/DYNMIC_SCRIPTING: replace -form=$dynamic$ -sub=dynamic_1030
with -form=dynamic_1030 etc.
--list=format-tests: Handle tests that contain line feed characters
Improve valid() for clipperz, fixes issue #457
Improve valid() for openssl-enc, fixing #498
Improve valid() for django format, fixing issue #460
Build ../run/luks2john (luks2john.c and Makefile fixed)
Improve valid() in rar format, fixing issue #459
Django format: document undefined behavior of atoi in valid()
...and reduce risk of triggering that undefined behavior.
Improve valid() of NETLMv2, fixing issue #461
Fix --list=format-tests: for empty ciphertexts, call prepare()
Fixed some more bugs in Django format (valid)
Fixes for KeePass format's valid()
Fixes for Bitcoin format's valid()
Fixes for 7-Zip format's valid()
Fixes for 1Password Cloud Keychain format's valid()
Fixes for gpg format's valid()
Format LastPass, sniffed sessions, additional check in valid()
Fixes for ODF format's valid()
Minor improvement of PBKDF2-HMAC-SHA512 format's valid()
Replace 2147483647 with INT_MAX in valid()
Even more checks in gpg's valid()
Fix warning caused by ffc7812
Add poor OpenMP scalability flag to --list=format-all-details
dynamic_THIN_FORMAT_LINK: use PASSWORD_HASH_SIZES ... to make sure
all the binary_hash[i] and get_hash[i] methods get passed to the
thin formats.
benchmark-unify: add some mappings to make it work with older
versions at least as well as unstable does...
benchmark-unify: add/change mappings for changes in core 1.8.0 --test
output
benchmark-unify: adjust logic for dynamic_n and old md5_gen(n)
some relbench improvements and adjustments
Fix -Wdeclaration-after-statement compiler warnings in dmg2john.c
c3_fmt.c: support ./john --list=format-tests --format=crypt
--subformat=md5crypt
c3_fmt.c: Print errors to stderr instead of stdout
luks2john.c, use %zu for sizeof(), to fix compiler warning on
linux-x86
bash completion: reuse --pot= logic for --loopback=
bash completion: support --intermediate-enc= and --target-encoding=
sh completion: support --mkv-stats=
bash completion: support --bare-always-valid and --verbosity=
.mailmap: make sure git maps my commits to my new email address
Fix issue #494 (some log info was printed once per forked process)
fix typo in comment
Re-add an "if (john_main_process)" that is needed for MPI
formats.h:define FMT_MAIN_VERSION earlier than before
define a FLG_COST for new option --cost
First revision of testing for different tunable costs among hashes
Let 9 more formats report tunable cost values
Tunable cost values for 7z and keepass
Tunable cost values for skey format (S/KEY)
Tunable cost values for agilekeychain and aix-ssha* formats
Android FDE format: remove unneeded and misleading components from
salt definition
Tunable cost for bitcoin format
blackberry-es10: remove component "iterations" from salt definition
7z-opencl: prepare for FMT_FORMAT_VERSION > 11 even if broken
Tunable costs for cloudkeychain and Django
BF_std.c: Fix a typo in a comment
cloudkeychain: tunabme cost name "iteration count" instead of
"iteration_count"
Report tunable cost values: dmg_fmt_plug.c drupal7_fmt_plug.c
encfs_fmt_plug.c
episerver: report hash type as tunable cost (1: SHA1, 2: SHA256)
Revert "blackberry-es10: remove component "iterations" from salt
definition"
Revert "Android FDE format: remove unneeded and misleading components
from salt definition"
doc/OPTIONS: better --cost= examples
Minor adjustments to john's usage and --list=hidden-options output
doc/README.librexgen: use --regex instead of --rexgen as option name
commit b6b18713 broke KRB5 support for make -f Makefile.orig, fix it
Make sure -j option works for make -f Makefile.orig
Fix gcc warnings for non-autoconf 32bit Linux build
Wordlist.c: Fix clang compiler warning (or even bug) in
restore_state() caused by commit 0dcff5d8
wordlist.c: let all regex specific code depend on #if HAVE_REXGEN
Fix AES-NI support for legacy (non-autoconf) 32 bit linux builds
Format django-scrypt: report tunable costs N, r, p
Log tunable costs even if they don't differ among salts
ike format: report hash algorithm used for hmac as tunable cost
keepass: report version as 2nd "tunable cost"
keyring: report tunable cost (iteration count)
LastPass, sniffed sessions: report iteration count as tunable cost
odf: report "iteration count" as tunable cost
office: report "MS Office version" and "iteration count" as tunable
costs
oldoffice: report "hash type" as "tunable cost"
pbkdf2-hmac-sha1: report iteration count as tunable cost
pbkdf2-hmac-sha512: report iteration count as tunable cost
pbkdf2_hmac_sha256: report iteration count as tunable cost
pwsafe: report iteration count as tunable cost
rar5: report iteration count as tunable cost
sxc: report iteration count as tunable cost
adjust some formats in src/unused to FMT_MAIN_VERSION > 11
sha1crypt: report iteration count as tunable cost
broken/opencl_7z_fmt.c: tunable cost related fix
scrypt format: report N, r, p as tunable costs
Use "$(CC)" instead of "gcc" in Makefiles
some more Makefile changes
Makefile.legacy: fix two errors for generic build
Makefile.legacy: avoid make warnings for generic builds
Makefile.legacy: make sure we really use the intended compiler for aes
subdir
 chmod a-x hmacMD5_fmt.c and aes/openssl/ossl_aes_crypto.c
wordlist.c: avoid compiler warnings for debug builds
doc/ENCODINGS: 2 grammar fixes
Skip tunable cost reporting if no hashes were loaded This fixes a
possible segfault
unicode.c: s/ansi/ANSI/
john.bash_completion: improve completion for encodings
listconf.c: call fmt_init for --list=format-[all-]details ...to
report an adjusted max. password length:
reorder ripemd-128 tests (untagged hash first)
Sync PBKDF2-HMAC-SHA256 CPU format tests with OpenCl format
Sync OpenCL ODF format tests with CPU format tests
Sync OpenCL RAKP format tests with CPU format tests
Sync OpenCL Raw-MD4 format tests with CPU format tests
Sync Raw-MD5 format tests (OpenCL / CPU / dynamic)
Fix debug build warnings in opencl_pbkdf2_hmac_sha256_fmt_plug.c
Raw-MD5: fix debug build breakage caused by 9aa03202
Sync raw-SHA256 hashes
opencl_keychain_fmt_plug.c: fix debug build warning
Sync Raw-SHA512 format tests (CPU/OpenCL/CUDA)
Bash completion: fix comletion for format classes
Bash completion: Fix issue # 687 for real.
Bash completion fix (format list still had "dynamic,")
Report librexgen version for ./john --list=build-info
listconf.c: '\t instead of ' ' between regxen's build time and runtime
version
Re-add reporting of load time versions of NSS and NSPR libraries
Fix a bug "use of logical '&&' with constant" in bench .c
Fix generic build (-f Makefile.legacy) breakage which was caused
by commit 7ec88f81a3826006e667ff8b2972ec89d87612fa.
Fix two compiler warnings for non-sse OMP builds
rsvp format: report hash algorithm used for hmac as tunable cost
Mute warning ‘sig_handle_reload’ defined but not used if SIGUSR2
is not defined
Report some more tunable costs
rsvp format: fix valid() to avoid segfaults
Make sure --list=format-methods also reports source()
Use "Disabled in configuration file" instead of "Disabled in john.pot"
in --list=format-all-details.
One .include [List.Rules:OldOffice] in [List.Rules:Jumbo] is enough.
Adjust bash completion bodge from commit a15e12e for --fork=
Fix gcc warning introduced in 172e933
OpenBSD SoftRAID: report iteration count as tunable cost and add
BENCHMARK_COMMENT
s/varient/variant in comments

Sayantan Datta (73):

mscash2-opencl:added multi-gpu interfaces,bug fixes
opencl-mscash2:no segfault when mixing devices with large difference
in speed
mscash2-opencl:code cleanup
mscash2-opencl:code cleanup
mscash2-opencl:code cleanup
bf-opencl:code cleanup
bf-opencl:code cleanup
bf-opencl:code cleanup
descrypt-opencl:code cleanup
mscash2-opencl:ciphertext format changed, no trailing white spaces
descrypt-opencl:cmp on gpu
shared valid,split,prepare for cpu and gpu mscash2
descrypt-opencl:improved compare
bcrypt-opencl: fixed AMD 7970 ASIC hang
mscash2-opencl: supports 128byte salts
mscash2-opencl: fixed warnings and other issues
mscash2-opencl:multi-gpu fix
mscash2-opencl:fixed quick benchmark
Raw-MD5-opencl: compare on gpu
Raw-MD5-opencl: fixed memory cleanup
Update opencl_rawmd5_fmt.c
Raw-MD5-opencl: status check bug fix
nt-opencl: no branching in coalasced loading loop.
nt-opencl: cmp on gpu
nt-opencl: use opencl_init instead of opencl_init_opt
raw-md4-opencl:cmp on gpu
raw-sha1-opencl: cmp on gpu
New format added, mscash-opencl: ported directly from mscash-cuda.
mscash-opencl: optimization patch #0
mscash-opencl: optimization patch #1
mscash-opencl: optimization patch #2
mscash-opencl: optimization patch #3
loader: added sequential id for salts
mscash-opencl: compare on gpu
nt-opencl: fixed bug causing inefficient keys transfer.
added mask mode interface
added new format param
mscash2-opencl:adhere to device limits.
mscash2-opencl: fixed zero min_keys_per_crypt bug.
mscash2-opencl: improvedpreprocess kernel.
mscash2-opencl: supports non-standard iteration count.
mscash2-opencl: expected to fix problems with weaker gpus.
Patch from the Hash Runner 2014 contest. Lotus5-opencl format.
descrypt-opencl:fixed memory leak. An unrealesd Event object, although
innocuous in most cases turned out to be the root cause of this
issue.
Updated mask mode. Now faster than incremental, as it should.
new mask mode ?H placeholder bugfix and some other optimizations.
duplicate check before inserting character following an escape
sequence.
improved performance when using node/fork with mask mode.
Generates skipped chunk using session/restore.
mask mode resumes accurately without any loss of work.
fixed compiler error introduced in 4b0612750841044
fix ETA calculation with affecting work division among nodes.
split mask mode functions.
Hybrid-mask enabled.
mask mode bug fix and performance improvement in hybrid modes
mask mode now supports --max-length
Hybrid mask: Bail out with error if [the used portion of] hybrid does
not contain ?w
Hybrid-mask: leaner code for supporting --max-length param.
Hybrid mask:added varibles mask_add_len and num_qw
Hybrid-mask:fix mask_add_len
mask mode escape sequence fix
mask mode add_mask_len fix following the escape sequence fix
mask mode escape sequence fix
mask mode leaner code for max/min len iteration
mask mode: fix redundant key generation after exhausting all
placeholders when using min-length
 mask mode fix: 9 does not gets dropped with--mask=[112010-998-0]
mask mode fix: sequence of characters in a placeholder now conforms
with rules.
mask-mode: fix distribution of candidates when using -min-length
mask-mode: fix session/resume when using -min-length and fork.
mask-mode: leaner code.
mask-mode: update mask_tot_cand in truncate_mask()
stretch mask with last placeholder
mask-mode:fix issues with mask stretching.

Lukas Odzioba (27):

CUDA multigpu first draft for pwsafe
gpg2john: Handle multiple keys per in file, as well as multiple
infiles.
Add OpenCL 1.2 error codes to our debug routines.
Include OpenCL version information into --list=build-info
(re-)enable agilekeychain format
gpg2john fixes
CUDA versions in --list=build-info
Basename helper function in misc.c, also capable of stripping
extensions.
gpg2john GECOS support
fix crash in gpg format
SIP valid() fix.
SIP valid() fix.
Implement shared fmt_default_binary_hash_N() for the common case.
Last bunch of formats making use of shared binary_hash() functions.
Commit PBKDF2-HMAC-SHA256 format
Commit PBKDF2-HMAC-SHA256 OpenCL format
tool to convert gadu-gadu auth packets to JtR format
phpass OpenCL kernel tweaks
Add binary_hash() functions to phpass-cuda and md5crypt-cuda formats.
PBKDF2-HMAC-SHA256 Patch for splitted kernel.
label test vectors in gpg format
Fix some broken valid().
Fix keyring valid()
Multi-device bugfix for pwsafe-cuda.
MD5crypt-opencl optimizations.
pbkdf2-hmac-sha512-opencl
Workaround for buggy CUDA memcpy() of length 0 in md5crypt.

ShaneQful (20):
Added prepare method which checks for the old style pdf hashes and
converts them to the new style pdf hashes if necessary
Cleaned up comments and added self to credits
Fixed bug Dhiru found in pdf2john.py. There was an issue with one of
my regexs so that it didn't cover all situations
Remove FIXME(s) (Don't scare Frank and magnum anymore)
Removed assert which caused bugs in some documents generated with word
for the mac (cherry picked from commit
ddbbc65639305473a3311a339501da82473b1d77)
Update CREDITS-jumbo
Added License to pdf2john
Unicode fixes for python 3 in office2john
Added Python 3 support for pdf2john
Fixed spelling error
Changed is_meta_data_encrypted to return a boolean rather than a
string
Changed get_encrypted_object_id to get_object_id in order to get any
 object id from the trailer
Impemented parse meta method, just get values methods left
Fix for checking meta data encrypted in python 3.2
pdf2john now gets the title and year created from xmp metadata
pdf2john now gets all relevant metadata when metadata left unencrypted
Fixed bug in pdf2john where wrong objects could have been used
Small fix to get_xmp_values method in pdf to john
Added ability to odf2john to generate a gecos field for meta data of
it exists
Removed pointless fix to pdf2john

Solar (20):
Fix breakage caused by commit aab646b42afe93ca909b
Bleeding Backports from john-1.7.9.6-c6.
john-1.7.9.6-selftest-1.diff
john-1.7.9.6-done-1.diff
john-1.7.9.6+done-reset-1.diff
john-1.7.9.6+reset-crypt_all-2.diff
john-1.7.9.6-LM-1.diff
john-1.7.9.6+crypt_all+LM-crypt_all-pcount-1.diff
PoC optimization for NTLMv1. Maintain a table for the third DES block.
2nd version of NETNTLM patch
Try to speedup cmp_all() by use of a bitmap. This only makes sense
when cmp_all() is called multiple times per crypt_all()'s actual
processing - that is, when cracking two or more C/R pairs at
once. There's a check for that in the code. Moreover, use of
the bitmap is automatically disabled when the C/R pair count
reduces to 1.
raw-SHA512-ng, trivial optimizations.
Licensing files
john-rawSHA1_ng_fmt-omp-reduction.diff
Add 'dokuwiki' external mode from
http://www.openwall.com/lists/john-users/2013/06/08/14
Added scrypt format
Mask mode for GPU
john-descrypt-opencl-section0-fix.diff
7z2john.py removal of some broken code
Patches from the Hash Runner 2014 contest.

Alexander Cherepanov (16):

Just a small patch to fix strncmp in NETLM format.
WPAPSK bugfix.
Some typos.
Typo in dynamic.conf
Replace unicode em-dash in putty files.
Replace non-ascii char by hex code in pass_gen.pl.
Replace several non-ascii copyright signs by "(c)" string.
Fix typos in README.bash-completion.
Fix some typos in README-OPENCL.
Get rid of strncmp with n==1.
Add some band-aid to valid() in sxc format.
Improve robustness of valid() in pkzip format.
Make valid() in sunmd5 format robust.
Fix a crash in dynamic with an overlong salt.
undrop: limit the length of password. (Fixes a crash.)
krb5: ensure that username and realm are non-empty. (Fixes SIGFPE.)

rofl0r (15):
factor out struct tcphdr
 Makefile: allow JOHN_LDFLAGS
tcphdr.h: remove unneeded typedef
tcphdr.h: special-case endian.h for mac/bsd
timer.h: remove unneeded include (sys/timeb.h)
timer.h: re-add sys/timeb.h to windows section
rename vncpcap2john.cpp to vncpcap2john.c
turn vncpcap2john.c into C
vncpcap2john.c: fix memleaks
vncpcap2john.c: initialize pointers
vncpcap2john.c: remove unneeded check
fix Makefile for vncpcap2john
vncpcap2john: add license blurb for own contribs
Makefile: add proper dependency information for vncpcap2john
vncpcap2john: remove bogus NUL check in makehex()

bwall (13):
Initial pwsafe optimization
Removed OpenSSL tag from pwsafe_fmt_plug
Updated copyright
Optimized the iterated SHA256 in pwsafe-cuda
Optimized the iterated SHA256 in pwsafe-cuda
Added copyright information
Applied optimizations to pwsafe-opencl and cleaned up copyrights
Cleaned up and optimized pwsafe cuda and opencl
Updated some copyrights of updated files
fixed the multiple gpu revert issue
Implemented split kernel on pwsafe-opencl
Pushed for more optimal settings
Increased the plaintext size to 87 in pwsafe-opencl

Harrison Neal (7):

Add OpenCL plugin for RAKP
Have RAKP OpenCL plugin be compiled and used
SHA-1 OpenCL and/or AES-NI support for o5logon format
Fixes for OMP support
Initial centralized AES support (OpenSSL, AES-NI)
Consolidate o5logon AES-NI and non-AES-NI formats
Fixes for Mac OS X and parallel compilation

Tavis Ormandy (7):

The second transpose in crypt() is not necessary, we can do it during
set_key().
Move a multiplication in set_key into crypt() to vectorize it.
Fix comments.
testing omp support
Use MEM_ALIGN_SIMD for message buffers.
make cmp_all parallel in omp builds.
Revert cmp_all omp parallel, performs poorly on AMD

Spiros Fraganastasis (6):

add support for new openssh key format
initial keyfile support in keepass format
initial support for twofish in keepass
add support for cracking VRRP hashes
minor misc. fixes
add support for cracking GLBP hashes

Aleksey Cherepanov (4):

add --skip-self-tests option
 fix to repare build with defined BENCH_BUILD after add of
--skip-self-tests option
Rexgen2rules: Convert of rexgen-like syntax into john rules.
PoC rule generator for leetspeak

Jeremi Gosney (4):

sse/xop implementation of raw-sha256
More efficient loading of plaintexts
sse/xop implementation of raw-sha512
fixes for rawSHA512_ng format

Michael Samuel (3):

Add External:KDEPaste. This takes advantage of CVE-2013-2120 to find
seeds that KDE Paste applet uses to generate passwords.
Add license notice to KDEPaste external mode.
Add external mode that replays the Awesome Password Generator 1.3.2
RNG.

Sanju Kholia (3):

add instructions for cracking KWallet
Added python code for kwallet2john.
Added Python code for KeyStore2John and help file for cracking
KeyStore files.

Costin Enache (3):

Fix for clock err in timers.
rules.c: Provide enough space for \xHEX encoded lines (worst case, ie.
x4).
Silly bug in sunmd5 busted the rounds parsing.

Thiébaud Weksteen (3):

Add support for cracking OpenBSD disk encryption
Add script to extract OpenBSD softraid hashes from disk image
Use internal PBKDF2 implementation for OpenBSD softraid hashing

Deepika Dutta (3):

Add bit-sliced MSCHAPv2 implementation by Deepika. This is added as a
new separate format: We now have mschapv2, mschapv2-bs and
mschapv2-naive.
Add MMX version of DES_bs_crypt_plain().
netntlm bitsliced conversion

junmuz (3):
CUDA Multi Device Options Parsing Support Added
Minor fix for Checking device id
GPU DEVICE LIMIT log

Michael Ledford (2):

Adds support for multiple keys existing in an encrypted DMG. When
multiple keys are found it selects the first password key header.
Adds support for DMG sparseimage files. Previously the first chunk
selected could be empty. Now chunk number one is chosen for DMG
sparseimage files and a new heuristic is added to the DMG format
plugin to look for 'Apple'.
Adds support for DMG sparsebundle files.

Nicolas Collignon (2):

McAfee ePO passwords converter
enhance mcafee_epo2john.py and add license blurb
Dániel Bali (1):
Add OpenCL target for Win32/Cygwin.

Vlatko Kosturjak (1):

Expose markov stats file as "hidden" command line option
(--mkv-stats=FILE)

atom (1):
office2john: Patch needed for cracking CMYC 2014 challenge 3.

bartavelle (Simon Marechal) (1):

Updated copyright information

rejsmont (1):
warning() should be static also in the header

Christopher Gurnee (1):

For Bitcoin format, use JtR's SSE variant of SHA-512 instead of the
ossl one for an almost 2x speed improvement. Also a small speed
boost to the non-SSE variant by avoiding EVP_BytesToKey().

Oleksii Shevchuk (1):

Add clWaitForEvents first

Péter Kasza (1):

add support for IBM ClearQuest hashes

Sergey V (1):
make blockchain-opencl non-plug

Lord_Rafa (1):
fixed the arm profile to compile in parallella

Konrads Smelkovs (1):

Perl utility to convert AIX's /etc/security/passwd to JtR format

Guth (1):
pdf2john, validate trailer

Sébastien Kaczmarek (1):

Lotus Domino 8.5 format added

The following changes were made between 1.7.9-jumbo-7 and 1.7.9-jumbo-8:

magnum (731):
Add a "Section [%s] not found" before outputting the catch-all
--list=?
Merge fix
raw-md5-opencl: Report human readable errors
opencl_mscash2_fmt.c did not declare clear_keys static.
Fix mis-use of CL_DEVICE_MAX_WORK_GROUP_SIZE in a number of OpenCL
formats.
Revert "Version change for Jumbo-7RC, and undefine
REPORT_OPENCL_WARNINGS" as this was only for the "jumbo-6-fixes"
branch.
Add Alain's Truecrypt format as posted to
www.openwall.com/lists/john-users/2012/07/18/3. Make target for
 truecrypt_volume2john is missing.
Use maxlen in dumb16 and dumb32 external modes.
Make sunmd5 check FMT_MAIN_VERSION, so the same file works for
non-bleeding as well as bleeding. Also fix some Linux warnings.
FIXME: This version still fails self-test under OMP. FIXME:
Jim and Simon, add your license statements!
Fix new sunmd5 format struct so it works with bleeding too. Also fix
the ALGORITHM_NAME enumeration to new-style. This now says it's
using SSE2 but apparently it is not, on my gear.
Sunmd5: fix SSE2 for 64-bit
wordlist.c: fix segfault reported in
http://www.openwall.com/lists/john-users/2012/08/01/5
wordlist.c: fix segfault reported in
http://www.openwall.com/lists/john-users/2012/08/01/5
Define MMX_COEF in x86-64.h and drop the workarounds in SSE2 formats
that were needed cause it was missing.
Use clear_keys() for SSE2 buffer cleaning in a couple of formats,
instead of doing it in set_key() if index == 0.
Backports from john-1.7.9.6-c6.
Replicate Vlatko's fix in 487c44ce to build_kernel_from_binary() in
magnum-jumbo and downstream.
pass_gen.pl sync between branches
wow_srp: Use the OpenSSL version, revert the -lgmp from Makefile.
TODO: put some (default commented out) HAVE_GMP stuff in Makefile.
Add HAVE_GMP to Makefile, default commented out
.gitignore update
sed -ri 's/\bfree\(/MEM_FREE(/g' *[ch] (and then selectively
commit most changes)
Bugfix for previous commit
Limit External:Strip to 16 characters even if cipher_limit is higher.
Do not use cipher_limit directly in External:KnownForce, because the
length is fixed - ie. no shorter lengths will be tried!
Change a couple more ISO-8859-1 files (not in -fixes branch) to UTF-8
mssql12: drop obsolete comments about SSE2. Also drop the
UTF-8-specific version of set_key() as we're not working against
vector buffers anyway and enc_to_utf16() can handle utf-8 too.
External:Subsets: Revert defaulting to cipher_limit because it's often
too long. Instead, use cipher_limit to cap the used length.
RAR: #ifdef galore
dmg_fmt: #undef HTONL before defining it (warnings seen on OSX)
mssql12: Unicode fixes
Oldoffice: Proper Unicode support
Make opencl_DES_fmt.c compatible with bleeding's format struct
des-opencl: Use .h files for OpenCL includes (.c did not work on OSX)
Fix a benign warning in OSX
The new Kerberos formats depend on Kerberos 5 libs and headers. This
patch make them non-default - you have to uncomment HAVE_KRB5 to
build them.
Make new Kerberos formats' structs forward compatible
Use ASCII copyright signs
Makefile: change a couple of hard-coded gcc to $(CC).
Update some build targets with custom CFLAGS and LDFLAGS
sunmd5: bugfix and miniscule optimization
EncFS: Backport some fixes that was committed only to bleeding-jumbo
OSX: Only link against krb5 if HAVE_KRB5 is defined.
Office2john: Support multiple files in one run.
Office format simplifications. Avoid a lot of alloc/free and
memcpy/memmove even though they did not hit performance much.
Office: regression fixes for last patch
Office: Get rid of the rest of malloc()'s + some other misc fixes.
Office: Move Unicode conversion to set_key() so we don't have to
convert the same password several times. Add all sample hashes
we've got, as test vectors.
OldOffice: Move Unicode conversion to set_key() so we don't have to
convert the same password several times.
Ooops. Fix pbkdf2-sha512 format struct for bleeding.
Special wordlist rule for cracking old office documents truncated at
15 characters.
If ran under MPI though not built for it, complain!
Improve the "OldOffice" wordlist rule
Show SHA-1 in oldoffice FORMAT_NAME
OldOffice: Inner-loop optimisations and much smaller memory footprint
per hash. Unfortunately the gain was smaller than I hoped for,
but it may be better IRL.
Office: Very minor optimizations. OldOffice: For "salt-only"
formats (non-hashes), BINARY_SIZE should be 0
Fix for sunmd5 for non-intel hardware, submitted by Sam Novak.
RAR: gws_test() made static.
New format office-opencl. This first version handles only Office 2007
and a max plaintext length of 19.
Office-opencl: Fix some compiler warnings.
Office: Fix some over-long test vectors that was due to a bug in
office2john.
Fix format structs for new formats so they are compatible with
bleeding
.gitignore update
Office-opencl SALT_LENGTH bugfix
Office (CPU version), optimizations for 2007/2010, 2x boost!
Re-introduce APPLE in opencl_device_info.h
Office-opencl renamed to office2007-opencl, and office2010-opencl
added. Both use split kernels, so durations should not exceed
200ms (by far).
Added a CHANGES-jumbo.git which is (almost) raw output from "git
shortlog -n --no-merges", and placed (the changes part of)
Solar's announcement of Jumbo-6 in CHANGES-jumbo (mostly as a
placeholder, it might need editing).
Update CREDITS-jumbo
hmac-sha1 bug fix: Certain optimisations did not take SHA_BUF_SIZ in
account. Very few builds pick the old SHA-1 implementation that
uses 320 bytes instead of the now normal 64 bytes but native OSX
llvm now does, so triggered the bug.
Add a version of our shared find_best_workgroup() that supports
vectorized formats.
Bugfix for d708f98 (find_best_workgroup_vector())
Revert recent find_best_workgroup() changes; they were no-ops. Also
remove some older no-op code.
Merge 1.7.9-jumbo-7 as released, and bump the "unstable" version
string accordingly
OpenCL: Read the LWS environment variable in the shared find_best
function.
Fixes for new opencl_device_info.h names
Add "Rules:None". I have added this as temporary a dozen times. It's
useful in some cases for Single mode and it really doesn't harm
having it in place.
Office OpenCL formats: Run vectorized on some platforms (including
CPU).
RAR kernel rearrangement, preparing for a split. This alone made for
some boost.
OpenCL Office formats: Mitigate GPU hogging with much shorter kernel
durations. Barely any performance impact but much better desktop
response during cracking.
Add some license stuff to RAR format, just in case. The wording was
already present in clamav's unrar sources.
Forward-compatible format structs for sha256crypt-opencl and
postgreSQL
RAR OpenCL: Finally, split kernel into quicker subkernels. Lousy speed
so far but passes self-tests. We use a 64x buffer, and run a
non-destructive SHA-1 over it instead of copying and flogging
bits around all the time. This eats loads of memory but we are
forced to use global memory anyway after splitting.
RAR OpenCL: Use byte-addressed stores in some cases, if supported.
RAR OpenCL: Back to 1x buffer and use private memory in inner loop.
Limit sub-kernel duration to 200 ms for GCN. Use local memory on
nvidia.
RAR: Tweak GWS enumeration. Also a workaround to get the OpenCL kernel
running on Mac MBPr with GT 650M. This does not make any sense
at all, but it works. It is clearly a bug in OSX.
Office 2013 OpenCL format added with fair performance given the task.
Currently fails self-test on HD7970 for no apparent reason.
Office 2013-OpenCL: Vectorize (but only for CPU).
Drop some bogus logic from Office OpenCL formats. I confused local
work size with global ditto.
Office OpenCL formats: Add benchmark comments (iterations count)
Add option --status-every=N
Add a comment about Kepler (sm_30) in Makefile
Unify CUDA and OpenCL "human_format" functions.
RAR: Fix bugs with OpenCL profiling. In gws_test(), we did not take a
changed value of HASH_LOOPS into account. Worse, in crypt_all()
we had the global profilingEvent sent with
clEnqueueNDRangeKernel() even though we did not use it. This
lead to a memory leak and a slow but certain death.
Office OpenCL formats: Remove unused code for shared
find_best_workgroup()
Fix an inner loop memory leak in most (22 of 24) OpenCL formats that
use the shared find_best_workgroup() function. NOTE:
opencl_cryptsha256_fmt.c and opencl_cryptsha512_fmt.c are NOT
fixed here. They need some other fix, for example reverting to
private functions or a fixed (semi-fixed) value.
Revert patch 4f12506 and implement a cleaner solution: We now use a
pointer for profileEvent, and it's set to NULL after
find_best_workgroup() so no more events will be created.
Office OpenCL formats: Now that I got profiling straight, use the
shared find_best_workgroup() for a little boost on some
platforms.
Add missing FMT_OMP to Office OpenCL formats.
Another tiny (and benign) cl_event memory leak fixed in
common-opencl.c
OpenVMS format: forward-compatible format struct.
RAR: Prepare to use the shared find_best_workgroup() function - but
don't. For some reason, it returns a suboptimal figure. Also, it
takes time.
Office OpenCL formats: Revert to fixed LWS yet again. Auto-conf is
simply too slow and unreliable.
--list=cuda-devices: show number of SP, if known.
Some added information in doc/BUGS - that might be candidate for
moving to some other documentation because it's not really about
bugs.
common-opencl.c: Support split kernels with
opencl_find_best_workgroup(). RAR & Office OpenCL formats:
Implement the above but still, use hard-coded figures.
RAR & Office OpenCL: Enumerate LWS if explicitly asked to.
common-opencl.c: Add a way to pass parameters when building kernel.
RAR & Office OpenCL formats: Use this (more to come).
Make RAR (OpenCL) much more responsive, doing less per kernel call.
Print Office kernel durations while benchmarking (GWS=0).
Office2013-opencl: Fix minor bug in enumeration.
RAR: GETCHAR/PUTCHAR macro optimisations (vs. arch)
RAR: Tweaks.
Silly bug: The new --status-every option clashed with the original
--status option. Rename the former to --progress-every.
RAR: Fix bugs in auto-tuning to device. Office2013: Less X
hogging.
NETLM/NTLM/v2 formats: Fix a signed vs unsigned char bug, and make
case shifting UTF-8 aware (as opposed to just codepage-aware).
Also bump some limits to accommodate for UTF-8 encoded user and
domain names.
common-opencl.c: Add get_global_memory_size(), and tweak
find_best_workgroup().
Add NTLMv2-opencl. Well behaved but I/O-bound. Vectorized on CPU,
passes Test Suite. Full Unicode support.
NTLMv2-opencl: Change hard-coded challenge length figures to
SALT_SIZE_MAX and fix a bug in salt_hash() (our salt format is
different from the old).
NTLMv2-opencl: Only run vectorized on CPUs.
Encfs: Trying to nail an OSX problem. Change a dynamic allocation to
using alloca() explicitly. Some optimisations while at it.
Problem went from segfault to FAILED (cmp_all(1)) but it
actually seems to be a problem with whatever OpenSSL version I
happen to use here.
Drop an erroneous linefeed from --list=hidden-options.
Revert "Encfs: Trying to nail an OSX problem. Change a dynamic
allocation to using"
Fix for common_opencl_pbkdf2.c that hid the actual error output on
"SYNC FAILED"
Fix a common mispeling
Add missing error handling in all CUDA kernels. Also add some
information in README-CUDA about too high THREADS settings as
well as a specific hint from HANDLE_ERROR for the "too many
resources requested" error. Finally, add some missing stuff in
Makefile so a header file change will cause a rebuild of
wpapsk-cuda or pwsafe-cuda.
ssha-opencl: Tweak the device auto-tuning, fixing several problems.
Bugfix in md5crypt-cuda. If we clear any_cracked we must also clean
the outbuffer.
ssha-opencl: Further tweaks for work size selections
encfs-opencl: Make it possible to specify GWS. LWS can already be
forced due to the shared find_best_workgroup function honoring
it.
Respect "iterations" parameter in odf (sxc) PBKDF2 kernel
For some reason the OpenCL kernel for OpenPGP/GnuPG was using unsigned
long for 32-bit integers. Fixing this made for a 20% boost on
Tahiti but it did not help the nvidia, which fails with the
cryptic message "Module-scoped variables in .local state space
are not allowed with ABI". Maybe that is just another driver
bug.
Some added information in GPU docs, regarding temperatures, watchdogs
 etc.
Rename doc/README.opencl to README-OPENCL for consistency with the
CUDA docs.
Update CHANGES-jumbo.git and CREDITS-jumbo.
Manual corrections of CHANGES-jumbo.git and CREDITS-jumbo
Move some less used options from main usage output to "hidden"
(shown with --list=hidden-options)
Add description of --progress-every=N to doc/OPTIONS.
OpenCL: Apple does not expose get_compute_capability() so we need to
find out SP count using mory hacky approaches.
Add Max. GPRs/thread block to --list=cuda-devices
Add more info to --list=cuda-devices.
Support for codepages: CP852, CP1250 and ISO-8859-2.
Update doc/ENCODINGS with newly supported encodings.
Split kernel and some optimizations for wpapsk-opencl. And device
auto-tuning.
wpapsk.h: The prf_512() function was doing four times more work than
needed. This change boosts OpenCL and CUDA formats a lot,
decreasing the CPU post-processing. HD7970 went from 66197 c/s
to 89468 c/s.
Trying to work around AMD bugs in ntlmv2 OpenCL format, or help the
optimizer, or both. The RAR and Office OpenCL formats were checked
too but did not contain such code.
OpenCL kernels RAR, Office, NTLMv2, WPA-PSK: Drop all pragma unrolls.
The compiler unrolls anyway where sensible.
wpapsk-opencl: Move all post processing from CPU to GPU or otherwise
out of the inner loop. Great boost! Also bump max plaintext
length from 15 to 32.
Add progress indicator during GWS enumeration in all my formats.
ntlmv2-opencl: Implement clear_keys() for faster buffer cleaning.
Update .gitignore
opencl_rawsha256_fmt.c: Add missing stuff for FMT_MAIN_VERSION > 9
opencl_rawsha256_fmt.c: Add more missing stuff for FMT_MAIN_VERSION >
9
Add null-hash test vectors to some unsalted formats that lacked them.
Add max. memory alloc. size to --list=opencl-devices
ntlmv2-opencl: Use up to 1/4 of GPU memory
ntlmv2-opencl: Map buffer instead of enqueing transfers. This can be
changed with a #define to use clEnqueueRead/Write. Until now, it
was mapped *and* enqueued transfers, which was a bug!
ntlmv2-opencl: Don't try to account for clear_keys in gws_test().
ntlmv2-opencl: Revert 65071a1. Documentation as well as google results
are contradicting but apparently it was correct as it was.
Use CL_READ_WRITE_INVALIDATE_REGION if available (OpenCL 1.2)
OpenCL formats: more complete release_clobj() and other tweaks.
Extend opt_flags to 64-bit and move all Jumbo-specific flags to
the high 32 bits. Core now have 7-8 free flags, Jumbo has 24.
The previous patch reset a FLG_NONE behaviour that Jumbo did not have
before. This had unexpected consequences so this is now reverted
(FLG_NONE is 0 again). Also a cosmetic (code) fix for loopback.
New GPU options: --request-vectorize and --request-scalar. These are
only supported by a few formats. Obviously they are mutually
exclusive.
ntlmv2-opencl and office-opencl formats: Never run vectorized unless
asked to. There's just too many problems (driver bugs).
OpenCL formats Office/rar/ntlmv2/wpapsk: Add a clFinish in
release_clobj() for safety. With a little luck this fixes some
problems and I'll regret blaming the drivers :-P
cryptmd5-opencl: Selective use of bitselect, use pinned memory and
 auto-tune to device.
OpenCL: Shared find_best_workgroup() tweak for vectorized formats.
office2010-opencl needed the same workaround for OSX that office2007
had.
Regression fix for 11ecdb0: loopback mode did not work because
wordlist.c mis-read the 64-bit options flags.
cryptmd5-opencl: Allocate/map buffer per nvidia's best practices for
pinned memory. This involves creating an extra buffer that we
have missed so far.
ntlmv2-opencl: Allocate/map buffer correctly. Quite a boost on some
devices.
office2007-opencl: Allocate/map buffer correctly. Also fixed what I
thought was an Apple bug. Silly me.
office2010/2013-opencl: Allocate/map buffer correctly.
RAR & wpapsk-opencl: Allocate/map buffers correctly.
Re-work some OpenCL error handling and add a margin for max gws
ntlmv2-opencl: Use macros for MD4 and MD5 (because they are same
whether vectorized or not)
Office2007/2010 OpenCL: Use a macro variant of SHA1, that is also has
an optimized version for the short-length SHA-1's (idea from
wpapsk).
wpapsk-opencl: Use Milen's SHA1 on AMD, and Lukas' on nvidias.
ssh-ng: Benchmark with crippled plaintexts, so we get real speed
reported: It's about 10x faster than the old ssh format.
OpenCL raw-md4/md5/sha1: Fix buffer allocations per nvidia's
recommendations
Revert "OpenCL raw-md4/md5/sha1: Fix buffer allocations per nvidia's
recommendations"
EncFS: Reduce effects of an OSX problem.
Update .gitignore
EncFS: Fix a bug that for some reason only surfaced on OSX.
Fix format structs for krb-ng, lastpass-sniffed and mongodb to work
with version 10.
Add ULL to the 64-bit options flags definitions.
Move the "hidden" options printout to options.c. Move --plugin to
"hidden" and move --save-memory above format list.
krb5ng: Various optimisations, including: Check for known plaintext.
Always use fast PBKDF2 for short keys instead of choosing at
compile-time.
krb5ng: Make a couple of functions static.
Update .gitignore
krb-ng: Remove (#if 0...#endif) an unused function
xsha512-ng: Fix a format struct problem for v10. The split() function
prototype has changed!
Fix for "testpara" and "generic" make targets
mskrb5: Use Dhiru's approach from krb5-ng, giving us proper binaries
instead of being a "salt-only" non-hash format. The known
plain-text trick is now used as a short-cut within crypt_all().
The benchmark may indicate a slight regression for single core
but IRL this is faster and scales better.
New format: krb5ng-opencl. Core copied from WPAPSK but now made
generic so future formats using pbkdf2-hmac-sha1 will be easy.
Various tweaks: Remove the "MS" from krb-ng format name. Bump limits
for wpapsk-opencl up to what is actually supported by WPA-PSK.
Remove some redundant code from wpa-psk kernel.
wpapsk-opencl and krb5ng-opencl: Support vectorization. Great for
OpenCL/CPU and probably VLIW AMD. Currently default to off, use
with --request-vectorize
OpenCL formats wpapsk/ntlmv2/office/krb-ng: Revert to always vectorize
 on non-GCN AMD GPUs. Not on CPU, because I see some drivers
doing a very poor job of compiling it although it should be a
no-brainer and produce SSE2.
Vectorized OpenCL formats: Add parens against ambiguity, and fix a
whole bunch of kernel bugs that crawled in there while I was in
WAT mode.
Last vectorizing bug hopefully fixed.
krb5pa-sha1: Last change to input format (hopefully). krb5ng-opencl:
rename to krb5pa-sha1-opencl, incorporate CPU format changes and
support etype 17 as well.
Rename mskrb5 to krb5pa-md5 and support our new input format from
krbpa2john.py. The old input format is also still supported
although I am not aware of any tool outputting it.
valid() tweaks in a couple of formats. If we print a warning about
salt length, we better a) be sure the hash is otherwise valid,
and b) include the FORMAT_LABEL in the printout.
Add a self-test that fails if a format claims SPLIT_UNIFIES_CASE
despite using fmt_default_split. This finds several offending
formats.
Implement a split() in krb5pa-sha1 formats
KRB4: More robust valid()
krb5pa formats: Make the new split() functions fmt struct v10
compatible.
KRB4 valid() fix
WPAPSK valid() fix
krb5pa-sha1 valid() fix (use the one from krb5pa-sha1-opencl)
Include some AMD extensions to OpenCL device list.
Self-test "valid() killer" that unfortunately will produce a segfault
instead of FAIL when it hits a bug. (only active with -DDEBUG)
Rearrange the AMD OpenCL device-list items.
OpenCL device-list fix: Need to init device_info[d] earlier.
OpenCL device-list fix: strip leading whitespace from buggy device
names, and other list-device tweaks.
OpenCL device list: Mute a compiler warning on OSX. Print PCI stuff
with leading zeros.
Unicode: Ensure that our 'UTF32' typedef does not end up 64-bit.
ntlmv2-opencl: Partial binary transfer, for a little boost.
Speed up iterated split-kernel formats auto-tuning by not doing all
the iterations.
Speed up RAR OpenCL format's auto-tuning by not doing all the
iterations.
OpenCL device list: Do not print "cache size" if zero (it's usually
not true).
pass_gen.pl: Change deprecated 'utf8' encoding to 'UTF-8'
NTLMv2: Do Unicode conversion on GPU. Support for short lengths
(faster), honor the --length option. Drop support for
vectorization.
Rename --length to --max-length and also add --min-length. Make Markov
and Incremental modes honour them. Add req_minlen and req_maxlen
variables to External mode for accessing them. Change a bunch of
existing external modes to honour them. TO DO: --min-length
support for Wordlist and Single. And regression testing!
Add some sanity checking to --min-length and --max-length. Support
--max-length in Single and Wordlist. Also, add some MPI stuff to
fprintf calls that lacked it.
Support --min-length in Rules (includes Single mode) and Wordlist
without rules. This needs to be checked for performance
regression, especially for the case where we do not use the
option.
Bugfix: Do not list request-vector/scalar in list of --list options.
Enable crypt(3) for 64-bit OSX targets.
rules.c: Skip (as opposed to truncate) words longer than maxlength.
wordlist.c: Implement --min-length and --max-length in buffer loading
when rules are not in use. Move handling of unbuffered
--min/max-length away from dummy_rules_apply(), and into the
main loop. --max-length will skip words, not truncate them.
Minor fixes for --min/max-length
wordlist.c: Implement --min/max-length for pipe mode too. And a bug
fix for unbuffered/stdin reads.
ntlmv2-opencl: If max length is decreased, reflect it in benchmark
comment.
Silence a bunch of compiler warnings emitted by newer gcc (the ones
that were my bad)
Drop SPLIT_UNIFIES_CASE from clipperz format, because, well, it
doesn't.
formats.c bug fix: We must call clear_keys() after self-tests,
otherwise first batch of keys to crypt_all() in the real crack
might be thrashed. This made for failures in hmac-md5, hmac-sha1
and mssql but I just couldn't find the culprit until now.
Silence a compiler warning from the OpenCL binary-caching patch
Quick-response to "events" (like key presses) for my loop-kernel
OpenCL formats. I'll let Claudio and Sayantan decide for
themselves whether to use it or not. Note that the clFinish() is
needed within the loop for this to work correctly.
Use OpenCL's "spinning wheel" cursor during self-test, so we can see
where it ends. We could have it within benchmarking too but it
would affect speed. It's not 'erased' until after benchmarking,
so we can still see when a real crack really starts.
Make "my" OpenCL formats adjust global worksize to the count argument
to crypt_all(). On my laptop, this decreases self-test time for
Office 2007 from 3:30 to 0:41. This is also good for Single mode
since we can now set min keys per crypt to local worksize.
Fix a bug introduced in ef133e4 where a system with OS_TIMER=0 would
run into problems when the formats called sig_timer_emu_tick()
during benchmark. To fix this, I made bench_running a global
variable. Maybe there are better ways. This patch also fixes a
minor bug in erasing the "spinning wheel".
When StatusShowCand option (in john.conf) is true, add the figure to
log as well.
Add a john.conf option NoLoaderDupeCheck, for skipping dupe-checking
while loading hashes. This is for testing purposes only.
Sayantan's formats: errors/diagnostics to stderr, please.
Fix .gitignore after 6b66504
SAP formats: Add a salt length check in valid()
RAR format: Since the OpenCL code no longer falls back to CPU in
Single mode, break out the OpenCL code into a separate
rar-opencl format just like the rest of them.
Sayantan's formats: errors/diagnostics to stderr, please.
Fix .gitignore after 6b66504
SAP formats: Add a salt length check in valid()
RAR format: Since the OpenCL code no longer falls back to CPU in
Single mode, break out the OpenCL code into a separate
rar-opencl format just like the rest of them.
Drop "data_info" from raw-md4-opencl and raw-md5-opencl. It was
unneeded and just wasted time. Plus other tweaks.
Use cached binaries for Apple platforms too.
Bugfix for OpenCL binary caching. The old code depended on a null
terminator that wasn't guaranteed. Also, I'm not sure you can
 pass a buffer to sprintf() both as destination and as one of the
arguments with defined results.
Bugfixes for v10 format struct in opencl-encfs_fmt.c
Fix a bug in options.c print_usage(), reported by scan-build.
unicode.c: remove some dead code as indicated by scan-build.
Remove dead increments in sapG, found by scan-build.
Remove dead increment in sha256crypt, found by scan-build.
Reset a "dangling reference" in single.c as found by scan-build. This
was not really a bug but we can reset it to NULL for good
measure.
Comment out all "rar_dbgmsg()" from unrar sources, to silent some
tools.
Fix a minor memory leak in rules.c userclass_expand() reported by
scan-build.
Fix a non-issue in calc_stat.c just to silence scan-build.
cuda_xsha512: reduce BSS with 70 MB or so and provide a done() embryo.
opencl_xsha512: reduce BSS with 15 MB or so and provide a done().
opencl_rawsha512: reduce BSS with 15 MB or so and provide a done().
cuda_rawsha512: reduce BSS with 15 MB or so and provide a done().
opencl_mscash2: reduce BSS with 8 MB or so and provide a done().
opencl_bf_fmt.c: Reduce static footprint. More is needed but it's in
opencl_bf_std.c and can't be allocated in format's init() just
like that.
OpenCL formats: rename all "release_all()" functions to "done()" in
order to prepare for upcoming core changes
OpenCL: Change opencl_init_opt() to use the binary caching feature.
common-opencl.c: Output "build" time even for cached binaries, in case
they take over 2 seconds.
Makefile: Copy the OpenCL kernels using -p flag, so we don't
invalidate cached binaries unless actually needed.
Bugfix: Cached binaries must include compile options as selection
criteria.
raw-md4-opencl & raw-md5-opencl: Honour the -max-length option, for
performance.
raw-sha1-opencl: Some optimizations, and honour the -max-length
option.
Fix a silly bug in my MIN/MAX macros. They were not protected enough.
Experimental: Let Incremental and Markov modes call
format->methods.clear_keys() after setting
options.force_maxlength to actual used length. This gives the format
a chance to optimize for that length if possible. No format
currently use this.
Change whitespace to '_' in binary file names.
OpenCL raw md4/md5/sha1 formats optmizations.
common-opencl.c: No need to call include_source() with a cached
binary.
common-opencl.c: Reflect that we are using a cached binary in case of
errors.
No-op workaround for false positive in scan-build. See
http://www.openwall.com/lists/john-dev/2012/12/23/12
formats.c: Add a self-test that puts maximum length candidates in all
buffer positions and then read them back to verify. This finds
incorrect claims of PLAINTEXT_SIZE as well as most kinds of key
buffer over-runs. It found 15 problematic formats right away.
sapB: The new self-test actually gave a false positive on this format.
This patch works around it.
raw-md4: fix a bug (SSE2 only) that made plain texts longer than 31
end up incorrect.
Fix the max-length bug in Truecrypt formats. While at it, bump max
 length to 64 because there is no gain in limiting it, and 64 is
the actual limit in Truecrypt according to
http://www.truecrypt.org/docs/?s=keyfiles.
Reduce risk of new self-test to segfault due to problems.
Fix the new max-length self test so it only runs once.
RAR formats: Max length fix.
sapG: Workaround for problems induced by the max length self test.
sapG: Regression fix for non-OMP builds after b15beb1.
DMG format still segfaulted from the valid() kill self-test. It also
made a strdup() before even checking the format tag, which is a
Bad Idea[tm].
PuTTY format still segfaulted from the valid() kill self-test.
formats.c: Only run the "valid() killer" once, even if we have a
larger keys_per_crypt than number of test vectors.
Fortigate: optimize order of tests in valid().
IPB2: optimize order of tests in valid().
Unified raw-md4-opencl and raw-md5-opencl again, and fix max length
bugs in them and in raw-sha1-opencl. The raw md4/md5 formats
should be identical except s/md4/md5/. The raw sha1 format could
also be very very similar but is not yet.
Fix a typo in numerous OpenCL formats.
Numerous bug fixes and optimizations in mysql-sha1-opencl.
Max length bugfix in ssha-opencl. This format can be optimized a whole
lot but I don't have the inspiration.
Fix max length bug in IPB2.
Fix max length bug in IPB2 (last patch did not really do it).
SIP: Remove some weird double work. More optimizations are possible.
Rar-opencl: Reduce self-test slow-down due to max. length test.
Office OpenCL formats: Reduce self-test slow-down due to max. length
test. SIP format: drop a now unused variable.
Max length bugfixes in dominosec, epi and krb4 (all just fence-post
bugs)
Oracle11 workaround for new max-length test problems (not really a
bug).
Make encfs-opencl a little more friendly. It still needs a split
kernel.
UTF-8 reverse bodge for the max-length self-test. Formats the take
UTF-8 and convert to UTF-16 use to triple the reported max
length, and truncate themselves. We must do the same, or we'll
get false errors.
mscash bug when using codepage, found with new max. length self-test
in conjunction with the almighty Test Suite!
BENCH_BUILD ifdefs for the new self-test, needed after adding
options.utf8 stuff.
Max-length fixes for dynamic.conf, fixing dynamic_1001..dynamic1013
Bugfix for a truckload of formats that did this:
Fix Lukas' pbkdf2-hmac-sha1 so it can handle a max. length of 64.
There is no gain in limiting it (actually the new code is faster
due to optimizations). Affected formats: agilekeychain, wpapsk,
dmg, encfs, keychain, krb5pa-sha1, strip - as well as CUDA and
OpenCL versions of them.
ODF: Use our own PBKDF2. Also, bump max length in OpenCL version - the
PBKDF2 is not made from the plaintext but from a hash of it.
Change the PBKDF2-HMAC-SHA1 in keychain.h to take an output length
argument and make ZIP and SXC use it instead of Gladman code -
2x boost!
OpenCL fixes: 2x boost for ODF and 4x boost for SXC. These format are
still very rough.
Improve max-length self-test so it better reflects real world.
Further improve max-length self-test to avoid false positives and
catch even more real problems. This also fixes an older problem:
Some formats depend on all keys being set before calling
crypt_all() so now we do call set_key() for indexes that are
otherwise skipped.
dynamic_fmt.c valid() bugfix: It did not reject overlong raw hashes.
More dynamic_fmt valid() fixes. Still passes Test Suite so should be
OK.
Dynamic bugfix. If DynamicAlwaysUseRawHashes = Y, we must disable it
for thin formats.
Reduce BINARY_SIZE for OpenCL formats that was transferring partial
binaries anyway.
OpenCL: Only replace whitespace with '_' in device name part, not
path.
Some updates to README-OPENCL. This whole file should be reworked but
I am not the man.
Reduce global work size to 'count' in more OpenCL formats, speeding up
self-test and helping Single mode a lot. Also some bugfixes. And
unify the various non-split pbkdf2 kernels we had, so there is
only one to maintain. RAR is currently b0rken.
s/Alocate/Allocate/g
More fixes to Dhiru's PBKDF2 OpenCL formats
OpenCL pbkdf2_unsplit fixes
More OpenCL fixes
Silly bug in SXC and ODF found by Dhiru
SXC bug tracked down, plus other minor bugs in SXC and ODF. Both work
now.
ssha-opencl: Check device max allocation size.
Make use of mem_calloc() in a bunch of GPU formats. Also change some
mem_alloc_tiny to mem_alloc and add MEM_FREE() in done().
Makefile: Ensure that a change of common_opencl_pbkdf2.h will result
in re-compile of common_opencl_pbkdf2.o and
opencl_mscash2_fmt.o.
More Makefile tweaking
gpg-opencl fixes. View this patch with "git diff -w"
gpg-opencl cosmetical fix.
Make it safe to call clear_keys() for --stdout
Drop an unused function in gpg-opencl
Change the plugin magic in Makefile so formats are loaded in
alphabetic order of format struct names. Also, change the struct
for raw-sha512 so it will take precedence over other 64 byte
(128 hex chars) bare hash formats (currently raw blake2).
Change name of blake2-512 to raw-blake2-512, so it doesn't get bullied
by the rest of the raw format gang.
Change calc_stat so that the -p option enables support for 8-bit
characters as opposed to just mute the warnings. The usage
output already claimed this was the case.
UTF-8 validity check in Markov mode.
Fix for UTF-8 validity check in Markov mode.
Full UTF-8 validity check in Markov. Markov mode now works pretty good
with UTF-8 provided you use -p with calc_stat and --enc:utf8
with Markov mode.
Revert "expose markov stats file as command line option"
Add usage output for mask mode.
Register GPU formats first (even before core formats - this is
experimental)
Revert "Mask mode bugfix"
Load raw-md5 and raw-sha1 OpenCL formats before others.
Do not trust the compiler to optimize away strlen of constants in
 valid()
Change the format tag of raw-md5-opencl to dynamic_0 just like the
others.
Rename raw-blake2-512 to raw-blake2 and reflect SIMD usage in
ALGORITHM_NAME
Fix lastpass format struct
Fix a bug introduced with e82b54e. When running --stdout, we have a
limited format struct, and params.label is NULL.
Fix a bug in rar2john - it tried to fclose() twice.
Add big-endian versions of our dump_stuff() debug functions.
Allow running an OpenCL build with no platforms detected.
Drop UTF-8 test from Markov. The performance impact is not worth it.
For slow formats we could use an external filter.
OpenCL crypt-md5: Get rid of global memory use in inner loop. More
effective buffer use and some other tweaks. Significant boost on
both nvidia and AMD.
A few more md5crypt-opencl tweaks, mostly cosmetical.
Shared (unsplit) PBKDF2-hmac-sha1 OpenCL kernel endian bug fixed (only
affecting non-word key lengths). And some other ZIP tweaks.
Revert "Load raw-md5 and raw-sha1 OpenCL formats before others."
Bugfix for Office OpenCL formats when running vectorized.
Cludge for a problem where benchmark_timer is reset (because benchmark
time was reached) but the benchmark timer is still active,
leading to SIGALRM with no handler. If we ever saw
benchmark_running, we keep away for good.
Revert f8bf37d and try a cleaner fix.
OpenCL Office formats: get_key() bugfix, affecting UTF-8 mode.
Unify output. "work size" and "work-size" are changed to just
"worksize". Not that it's more correct, but it's the shortest
version :-)
Turn off OpenCL warnings. And bump version string to
1.7.9-jumbo-8-unstable.
Drop an nvidia workaround that just affect 9600GT on 295.49. If
needed, we must make the selection criteria more narrow because
it makes krb5pa-sha1 fail on Bull.
gpg-opencl: Honour LWS and GWS environmental variables.
strip-opencl: Honour LWS and GWS environmental variables.
xsha512-opencl: Honour LWS and GWS environmental variables.
keychain-opencl: Honour LWS and GWS environmental variables.
nt-opencl: Honour LWS and GWS environmental variables, plus
optimizations.
pwsafe-opencl: Honour LWS and GWS environmental variables.
raw-sha512-opencl: Honour LWS and GWS environmental variables.
Ignore environment LWS in common find_best_workgroup. Formats that
handle that won't call this anyway. Fix a minor bug in
nt-opencl: This format does not honour the count argument to
crypt_all() so we better set min_keys_per_crypt same as max.
phpass-opencl: Honour LWS and GWS environmental variables.
mysql-sha1-opencl: LWS/GWS (environment variables) fixes.
Ensure local/global_work_size are reset in all OpenCL formats. When
running --test, we may otherwise end up using figures from a
previous format.
raw-sha1-opencl: LWS/GWS (environment variable) fixes.
Final fix to the SIGALRM problem. My previous attempt at installing a
restarting signal handler was correct but incomplete (only
worked on some platforms).
Office OpenCL formats: Once again revert to using scalar code for all
devices due to what seem to be driver bugs.
Common-opencl: do not release unused event. No harm done but it can
 produce confusing debug messages.
Wpapsk-opencl, krb5pa-sha1-opencl: Faster LWS enumeration (no
iterations).
Bugfixes for the new restart code in bench_handle_timer().
bench.c: For --test=0, make the OpenCL formats also do quick
benchmarking (for self-tests, not benchmark figures).
CUDA fixes. The --request-scalar/vector options does not apply to CUDA
(yet).
Less ambiguous usage text for --raw-always-valid
bench.c: Include stdlib.h for setenv
OpenCL: Print AMD-specific "board name" along with device name.
OpenCL: Bump cached file name buffer to PATH_BUFFER_SIZE after seeing
problems with very long names (long device name plus long
options made for 130 characters full path name).
Obey CL_KERNEL_WORK_GROUP_SIZE in a number of OpenCL formats.
bench.c: Add a linefeed after FAILED, and add output when all formats
pass.
More LWS/GWS vs. min/max-keys-per-crypt fixes.
The dreaded format struct stuff, for kwallet
raw2dyna: fgets wrapper macro for muting compiler warnings
README-CUDA: Now featuring magnum's "OpenCL vs. CUDA parlor", showing
how to tweak your CUDA formats using the values from OpenCL's
automatic tuning.
SybaseASE: Allocate key buffer using calloc (should not be needed
though)
Drop obsolete External modes (deprecated by the --min/max-length
options)
Fix the "fopen: john.rec: No such file or directory" bug. It was
caused by path_init() not being called early enough. Not sure
who did that.
Move the 500+ lines of --list logic out of john.c and into its own
file.
dynamic_fmt.c: Mute compiler warnings about breaking strict-aliasing
rules. listconf.c: Add JimF in copyright blurb.
Makefile: Add -Wno-attributes for muting loads of "warning:
always_inline function might not be inlinable"
Revert 8a1b909 and introduce another change suggested by Frank.
Fix a subtle bug in the OSX workaround for rar-opencl. Found by the
max-length self-test. Also bump max length to 22 and drop a couple
of obsolete "FIXME" in unrar code.
Re-enable a 9600GT workaround in ntlmv2-opencl and wpapsk-opencl, but
not in krb5pa-sha1. This should work on Bull as well.
phpass-opencl bugfix.
Make dynamic_fmt fully movable between unstable and bleeding.
Documentation updates.
Update CHANGES-jumbo and CREDITS-jumbo (and the .mailmap that helps
producing them).
Add Milen to CREDITS-jumbo.
Lastpass: Bugfix in valid().
dmg-opencl: valid() bugfix (segfaulted from the valid-killer[tm])
common-opencl: I believe "source size" is poor wording for binary
size.
Truecrypt formats: Change benchmark output to raw.
PuTTY, GPG and krb5-pa-sha1: Change benchmark to raw. SybaseASE: Move
stuff from crypt_all() to set_salt(). It did not make much
practical difference though.
Some more typos, and GPU README updates (supported formats list).
OpenCL: Change the -DAPPLE for OpenCL build options to -D__APPLE__.
Commit 04cf5f1 hurt performance for dynamic because data locality
 suffered. This patch reverts it and use a union instead, to
satisfy the compiler.
Relbench: Ignore the new "All nn tests passed self-tests"
Single mode: Optionally load the full GECOS field (before splitting)
as one extra candidate. This is controlled with PristineGecos in
john.conf (and defaults to disabled).
Use list_add_unique() for the optional PristineGecos loading.
truecrypt_volume2john had CRLF line endings.
clang debug target fix.
Add self-tests that fail if you have a binary size > 0 but are using
fmt_default_binary() - and a similar test for salt. I believe this
does not emit any false positives but I'm not 100% sure yet.
BFEgg: Don't convert to base64 in crypt_all(), instead implement a
proper binary() that does the reverse.
formats.c: Degrade the latest self-tests to just warnings, and they
are also completely disabled unless -DDEBUG.
Fix a logic error for some only-once warnings.
Use correct value of BINARY_SIZE: GPU formats, and a couple Dhiru
missed (?)
Revert "clang debug target fix."
NetHalfLM and XSHA512: Alignment fixes.
Backport OpenCL fixes that was only committed to bleeding.
Revert "OpenCL: Change the -DAPPLE for OpenCL build options to
-D__APPLE__."
Fixes to keychain-opencl, strip-opencl and agilekeychain-opencl that
were only applied to bleeding.
Use -DDEBUG for the clang-debug targets!
Office OpenCL formats: Better valid(), with CPU format as template.
RAR formats: Implement a real bastard of a valid(). I will soon start
denying having written it.
Minor self-test tweaks.
Bwall's pwsafe patch broke builds using CommonCrypto. This fixes it.
Fix a hideous bug in rar2john introduced by in 216cc57. A variable was
declared as just "unsigned" (taken as int) instead of "unsigned
char" which lead to incorrect output. Not sure why the compiler
did not warn.
Correct ALGORITHM_NAME of mssql-12, pbkdf2-hmac-sha512 and the
Truecrypts. Use sha2.h for a number of formats that depended on
OpenSSL 0.9.8 (all but Truecrypt). Bump max length to 125
for pbkdf2-hmac-sha512 format. There's absolutely no point in
limiting it to 15.
Fixes for builds using Jim's homebrewn SHA-2.
Warnings from valid() should be suppressed if ldr_in_pot is set.
Fix for ldr_in_pot.
Introduce hash functions to hmac-md5.
Introduce hash functions to hmac-sha1.
Try to automagically detect and locate NSS, GMP and KRB5 libs, and
enable corresponding formats if successful. This is using Posix
find/grep and backticks. And pkg-config, if available.
MPI: Limit scope of an internal variable.
BLAKE2: Do not "secure zero memory".
SIPdump fixes (CppCheck). Also revert using MEM_FREE() and mem_alloc.
That is not a good idea for a stand-alone.
(Ab)use bench_running variable for keeping opencl_process_event() from
processing events while self-test is running. Otherwise a slow
self-test in combination with a short Save interval will thrash
the .rec file during start of a resumed session.
WPAPSK formats has a minimum password length of 8. This patch adds
code to john.c that enforces this. We may want to add min_length
 in the format struct later on, and remove this hack.
WPA-PSK had a silly ESSID length limit of 14 that was already bumped
to 32 (the actual real-life limit) for the OpenCL version.
Moreover, it segfaulted on longer names. This patch bumps it for
CPU & CUDA too. For some reason the hccap buffer size is 36
(from hccap2john) so I kept that. Perhaps it's because it ends
up aligned including room for a NULL.
Decrease PLAINTEXT_LENGTH for all WPA-PSK formats to 64, because that
is the spec. limit. More interestingly, the spec. says the
characters set must be 0x20-0xfe (printable ASCII) so we also
unset FMT_8_BIT.
Add two WPA-PSK test vectors, including one for WPA (as opposed to
WPA2).
Get DMD5 straight, and add OMP support. This format was seriously
broken.
dmg: Make _memmem() static inline (will hopefully optimised away the
switch), enable 64-bit tests on 64-bit arch and disable
unaligned checks on platforms that does not support it.
dmg-opencl: Same fixes to _memmem(). ssh-ng: Provide an own
static inline copy of _memmem() instead of using one from dmg.
Bump OMP_SCALE for DMD5.
Unify Kerberos names.
DMD5 format should not lowercase nonce hashes. They are used as-is!
hmac-sha1 bugfix for get_hash().
Cosmetical changes.
SAP formats: Minor fix mostly for satisfying Valgrind.
Fix an off-by-one in dynamic.
Office: Use calloc instead of alloc, just for muting Valgrind.
memory.c: When -DDEBUG, we turn mem_alloc_tiny() into a normal alloc,
to better track problems.
Drop some #ifdef DEBUG, or change them to specific ones like
RAR_DEBUG.
Wow-srp: This format is not case sensitive so unset FMT_CASE, and also
make the upper-casing in set_key() encoding-aware.
Wowsrp has a minimum password length of 8, so add it to the hack we
did for wpa-psk.
Drop a bunch of old formats from the unused/ directory. We can always
get them back from git if needed, or from old tar balls.
Adjust a stack buffer after the recent bugfix to Office 2007. This
doesn't matter but it makes the code less confusing.
Get rid of all use of alloca() and dynamic size arrays. Also fixes a
bug in keyring format: The "crack array" was not always reset
correctly. Makefile: Add OMPFLAGS comments for OSX llvm-gcc.
Truecrypt format: Use signed int for OpenMP loop, as required by
spec.
DMG: Another alloca() found and changed to fixed size stack array.
Dynamic: make sure valid() is looking at de-hexified data if
applicable.
Pwsafe-cuda: Adjust BLOCKS according to count argument. Shaves a
minute from self-test and helps Single mode. Verified with the
Test Suite.
CUDA formats: Adopt to count argument to crypt_all().
ZIP: Implement a proper cmp_all() instead of returning 1 and relying
on cmp_one(). This should be faster due to less function call
overhead.
Office OpenCL formats: avoid false sharing of crack array.
Unify dynamic_fmt.c among branches
Move WPAPSK test vectors to wpapsk.h and add a length 63 test.
wpapsk-cuda: Some compilers did not like having the test vectors in
 wpapsk.h so we define it out when building kernel.
git grep -l '<stdint.h>' | xargs sed -ri 's/<stdint.h>/"stdint.h"/'
Fix a bug in dmg formats, and add debug output.
WPAPSK (CPU and GPU): Exploit same-essid weakness. Huge speedup when
applicable. See
http://www.openwall.com/lists/john-dev/2013/01/29/13 and on for
discussion.
DMG debug bugfix (duh!), and a new test that seems pretty generic.
dmg2john error message.
dmg2john: Temporary fixes to handle large files and iteration count.
DMG: Support iteration count. DMG-opencl: A bunch of fixes. This
format was not up to par with CPU format. Serious flaws fixed.
Add a known plain test for 8 consecutive nulls.
SIMD support for NTLMv1 and OMP tweaks for non-SIMD. (cherry
picked from commit f3c6622be4054faf8992112c4c5101d5731baff1)
Merge Solar's 2nd version of NETNTLM patch. (cherry picked from
commit 6ab65c5, manually merged from ea46591)
NTLMv1: Very effective hint to the optimiser in cmp_all(). And
benchmark using crippled plaintexts, so we measure real-life
performance.
NT2: Bugfixes to experimental OMP support (defaults to disabled).
(cherry picked from commit 884e277d25b7ae441ea1f1ee5aee335fd412ae47)
DMG formats: Disable all but the NULL test, but bump that one to 16.
(cherry picked from commit
17b2c582debddf2ee482c18516e9d03cdbae8901)
DMG: Re-enable the other plaintext checks (but not Apple). (cherry
picked from commit 5d47bdc00ac037456f53369099c860e1397946c4)
DMG: properly re-enable plain checks, for OpenCL too. (cherry
picked from commit 7be93725d5b8b0ad9ac6a42735ce16fa5ad334a6)
NTLMv1: Bugfix, and compact the hash functions (before merging the new
stuff to mschapv2)
MSCHAPv2: SIMD support and exploitation of weakness just like NTLMv1
(the formats are very similar)
NTLMv1 and MSCHAPv2: reject hashes in valid() as hinted by Solar.
Bugfix for Jim's patch. Some debug functions were not static, and they
were included several times for GPU builds so the compiler
bailed. Also wrap that debug stuff in WPAPSK_DEBUG.
MSCHAPv2, NTLMv1: After the 3rd DES block check in valid(), cache the
indexes so we don't have to loop again in binary().
Truecrypt: replace a dynamic array with alloc/free. The format is so
slow this does not hurt performance at all.
NTLMv1 & MSCHAPv2: Use multiple blocks for the SSE version (still
single threaded) for bumping "many salts" speed. Actually we
bump it to the ceiling of what JtR can report - the output is
capped at 4294M c/s.
bench.c: Support benchmark speeds over 2^32.
NTLMv1/MSCHAPv2: Use memset for bitmap if more than 200 keys per
crypt.
NT2, NTLMv1, MSCHAPv2 tweaks for experimental OMP+SSE2.
Dynamic: Bugfix for MMX (I think it's right, I can't link it).
Raw MD4/MD5: MMX fixes.
More MMX fixes, including a cludge in the max-length self-test, for
MMX formats that use total_len.
dynamic.c: Bugfix.
Use DO_ALIGN macros in Jumbo .S files.
Portability fixes from building on ancient Sparc Solaris 32-bit.
MSCHAPv2 & NTLMv1: Emit a warning after loading 100 c/r pairs,
informing about slow loading and alternative formats.
Re-introduce the old versions as mschapv2-naive and
 netntlmv1-naive. Also fix the DES caching in NETNTLM: it may be
called from prepare() so the cache must be checked not only in
binary() but also in valid().
o5logon: Minor optmizations.
CUDA device list: Add PCI id to output, and replace a deprecated
parameter with a new, getting more detail.
Add any_cracked to a bunch of OpenCL formats that use crack array.
Add "naive" to the ALGORITHM_NAME for old load-fast-run-slow versions
of MSCHAPv2/NTLMv1, as in [32/64 naive]
Add wpapcap2john to .gitignore.
GPG: Optimizations, and bump max length to 32.
Endianness & alignment fixes from testing on a Sparc32.
More BE and alignment fixes. Some problems were in shared code in
unicode.c and the problems has been fixed backwards until now.
Most unicode.c functions convert/process UTF-16LE *regardless of
arch* and any endian fixes must account for that (normally none
is needed).
Make itoa64 and the likes const.
Crypt-SHA formats: Use a ROUNDS_MIN of 1.
pbkdf2-hmac-sha1: Support OSX 10.8 from "the Dave Grohl tool" as well
as GRUB hashes, without converting them. We support hashes
longer than 64 bytes but we don't calculate/verify past 64 even
in cmp_exact() - the chance for such a collision is
slim-to-none.
pbkdf2-hmac-sha1: Do a complete calculation & verify for full binary
size, just for good measure.
pbkdf2-hmac-sha512: Improve the warning messages given for partial
matches.
Endian fixes for BFEgg, wpapsk, pbkdf2-hmac-sha152, pwsafe and shared
pbkdf2-hmac-sha1. Also a bugfix for openvms but it still
segfaults for me. This is now the only format that doesn't pass
self-test.
Drop run-time version checks for NSS that would be bad if run-time lib
is older than build-time one.
Mscash: Get rid of compiler warnings on linux-ppc32.
Alignment fixes.
Rules.c bugfix for -max-length. Format's max length is already
adjusted down to max-length, and the RULE macro would do an
unwanted incrementaion.
Fix indentation for sha2[ch] before merging it to bleeding. This is a
pure whitespace fix.
On second thought, use the SHA2 version that was in bleeding instead.
Also try to mitigate a bogus strict aliasing warning. We are
already using a proper union for all I can see.
Mute harmless compiler warnings if MIN() already defined.
Better fix against that alleged strict-aliasing violation.
Move md5_eq.c to unused.
Remove md5_eq.o from Makefile.
formats.c: Stop trying to use system's strnlen(), instead always use
our supplied fmt_strnlen(). We could move it to misc.c too at
some point.
Revert SHA2 strict-aliasing fixes. They did not work at all for BE.
We can force Jim's generic SHA-2 with -DFORCE_GENERIC_SHA2. This patch
adds a way to force OpenSSL over CommonCrypto, using
-DFORCE_OPENSSL_SHA2.
Alignment fixes
Mute a warning for generic target.
Merge the current MSCHAPv2 naive format (with FreeRadius support) to
Deepika's bitsliced format. Then drop the original format and
 use BS format as "naive". Lots of whitespace changes, view with
-w.
DES BS plain SSE asm: Properly underscore P if applicable. It works on
OSX anyway but I'm not sure it will on others that
-DUNDERSCORES.
Alignment fix for mschapv2-bs
MSCHAPv2-BS: Merge two loops into one. This made a 5% boost here.
MSCHAPv2 BS: #ifdefs for FMT_ flags
Add some formats to benchmark-unify.
Update CHANGES-jumbo.git & CREDITS-jumbo.
I always end up confusing crypt(3) base64 with MIME Base64: Add some
comments.
Fix a slight bug in sha512crypt binary() that could never surface. It
was processing the trailing NULL. Also drop some copy-paste
artefacts.
Some Kerberos libs are not thread safe, but fortunately we can find
out using the krb5_is_thread_safe() call.
Decrease OMP_SCALE in a number of formats. This number needs to be
tweaked for each format, not just copied from a template.
listconf.c: Rewrite the formats list function. It was overly complex
and buggy.
Add format groups "opencl", "cuda", "cpu" and "gpu" for use with
--test as well as with things like "--list=formats" and so on.
Also allow wildcard in the end, as in wpapsk* or krb5*.
listconf.c: Suppress leading space in benchmark_comment.
Document --format group aliases/wildcards.
OpenCL DES: Honour device limit for LWS.
Documentation updates.
Disable the max-length self-test unless -DDEBUG.
NT2: Five percent boost for codepage encoding, from optimizations in
set_key().
raw-md5u: Three percent boost for codepage encoding, from
optimizations in set_key().
ntlmv1/mschapv2: Five percent boost for codepage encoding, from
optimizations in set_key().
mssql05: Optimizations in set_key() for codepage encoding.
DES-opencl: Safer LWS capping (ensure GWS is still multiple of LWS).
ODF formats (all three): Handle content.xml sizes less than 1024 bytes
correctly.
ODF formats: Make valid() a little more picky about hex length.
Add OSX clang-debug target.
Add macosx-x86-sse2-opencl (ie. 32-bit) build target.
uint64_t vs OpenCL fixes. In OpenCL, long is always 64-bit while long
long (if supported at all) is 128-bit. On host side, we use
"stdint.h" (not <stdint.h>).
Use "stdint.h" in pbkdf2-hmac-sha512 format.
DMG formats bugfix for "0 iterations" (assume 1000).
Truecrypt: Simplify cmp_all(), unfortunately for no performance gain.
dmg2john: bugfix.
dmg2john: Use base name of file name in output.
DMG formats: Serious bug fixes for 3DES v1 images. This fixes the
problem with false positives as well as an even worse one: false
negatives o.O
Add wpapcap2john from bleeding-jumbo.
uaf2john: Do not support writing to outfile, only to stdout. And add a
make target.
zip2john: Use base name of filename for login field.
MSCHAPv2 & NTLMv1: Smaller hot array. 25% boost for many salts, a
couple for one salt. Also enable OMP for SSE2.
MSCHAPv2 & NTLMv1: Again disable OMP for SSE2 - it works fine on some
CPUs but not on others.
Make odf2john work with python 2.6 (as well as 2.7).
Change odf2john so we fall back to the deprecated function only if the
newer one is not supported.
Fix SAVE= timer similar to Costin's patch.
Fixes for the recent timer changes. Fixes a bug introduced for
--max-run-time where a resumed session would immediately abort
again. And if OS_TIMER, we only call status_get_time() if
--max-run-time or --progress-every was given.
Update CHANGES-jumbo.git and CREDITS-jumbo.
Document --request-vector and --request-scalar options in
README-OPENCL.
mscash2-opencl: Fix for forced LWS larger than kernel's max.
Document the OSX problem with OpenCL kernel header files.
Wordlist bug fixes: Reject wordlist containing NULL bytes, and handle
inconsequent line feeds (eg. spurious CR within a line). This fix
will do for Jumbo-8 so we don't introduce new problems.
cisco2john: Fixes after testing with lots of real config files. And
add some more stuff to seed output (hostname, communities).
Fix the postgre format: Rename it to postgres, including the tag. The
old format is still supported. Also improve valid().
Rename the postgre source file too o.O
Some format name changes to clarify what they are.
s/Staroffice/StarOffice/
Add CXX variable to Makefile. Add separate PROJ_ sections for C++ as
well as pcap, so John does not fail building even if they do.
Also add uaf2john and truecrypt_volume2john to the main targets
and move office2john to unused.
Makefile tweaks.
uaf2john: Fix an aliasing problem.
Fix another warning in wpapcap2john (fread return value).
Fix generic build after the timer fixes.
Move wpapcap2john from PROJ_PCAP to PROJ - it does not depend on
libpcap!
Revert 8e992e4 and instead bump RULE_BUFFER_SIZE in params.h by 4. We
should look into this more closely but this will do for now.
Makefile: Add a "Done." after building the pcap targets.
Fix valid() in ntlmv2-opencl. It counted salt length incorrectly.
pass_gen.pl: Support UTF-8 for -salt argument when -utf8 option is
given.
Stop user from trying to use any encoding other than iso-8859-1 with
formats that lack support for it. It would just end up in false
negatives.
Drop bogus FMT_UTF8 flag from Truecrypt.
Change MPI reduction for benchmarks for accurate average speeds. Also,
reduce the int64 properly (only affects 32-bit or BE systems).
Update README-mpi.
Comment out all atexit(done) in OpenCL formats and let the OS do the
clean-up. Otherwise we got segfaults in the end after a --test
run. The real fixes are already present in bleeding branch.
CUDA formats: Tweak default THREADS*BLOCKS so they work on weaker
devices.
Dynamic: Change name of "salted Cisco PIX" to "Cisco ASA"
Copy the improved ZIP valid() to the OpenCL version of same format.
Refuse to translate \x00 into a null in user classes.
Add external mode filters for UTF-8 validation. This is slow though.
Rename NETNTLM_old_fmt_plug.c to NETNTLM_bs_fmt_plug.c.
Change self-test use of set_key("", index) into something that
 actually passes a large enough buffer for any use.
Change self-test use of set_key(current->plaintext, index) into
something that actually passes a large enough buffer for any
use.
Sparc fixes. Latest optimization of mschap/ntlm formats broke BE.
PDF format still used the Bad Idea[tm] version of valid().
Add missing deallocations in OpenCL DMG & RAR formats' valid().
SAP F/G: Bump maximum length. We over-ran a buffer with the latest
self-tests.
Bug fixes to thin dynamic formats PHPS, formspring, mediawiki, osc and
phpassMD5. They over-ran buffer when given bad input files.
More robust valid() in salted sha1 / nsldap formats.
Do some actual validation in SIP valid().
Actually, while at it we should just as well validate it is lower-case
hex.
WOW-SRP fixes. Not sure I found all, this is a mess.
rar2john big-endian fixes for Unicode filenames in RAR.
Makefile tweaks for JOHN_CFLAGS etc.
Canonical fixes for new tcphdr.h (failed on Sparc)
Revert "JtR-Dynamic_validation_raw_hashes-2.patch". It made things
worse.
Macros for prefixing MD[45]_(Init|Update|Final) with "john_" for
avoiding clashes with OpenSSL library.
Endian stuff.
Drop un-needed header in ssha-opencl.
Add macosx-x86-64-native build target. To use it, you need to copy
osx_as_wrapper.sh (from the src directory) to /usr/local/bin/as or
to some other directory (but the script MUST be called 'as') and
ensure this directory is first in your $PATH.
Precedence: MSCHAPv2 should pick sploit version over BS one.
Generic crypt format bugfix.
Testpara vs. HAVE_OPENSSL fix.
kwallet2john: Handle multiple files, and strip suffix and path from
login.
truecrypt_volume2john: Use filename as login field, and convert to a
john symlink.
Makefile fixes for OSX 32-bit cross-compile.
Bugfix for memory.c under -DDEBUG. In order to easier track bugs,
mem_alloc_tiny() is changed to a normal alloc. But it was supposed
to still maintain requested alignment!
Dynamic bugfix for $HEX$.
Change the way -DDEBUG affects mem_alloc_tiny(). The outcome is
practically the same but this is cleaner.
Revert "Change the way -DDEBUG affects mem_alloc_tiny(). The outcome
is practically"
Final canonical change for -DDEBUG versus mem_alloc_tiny.
Do not misalign SSE2 buffers when --save-memory=3 or above. The
previous fix did not catch all cases.
Relbench tweaks, for ignoring warnings.
OMP_SCALE tweaks. There are massive gains to win in some formats. All
OMP formats should be revised, but there are so many :-/
Fixes for a couple of make targets.
Drop inconsistent paths in the auto-conf hack.
Drop the wchar stuff from gpg2john. 8-bit GECOS now works fine.
Netscreen: Better valid().
DMD5: Better valid().
Update CHANGES-jumbo.git and CREDITS-jumbo.
More restrictive valid() in nsldap, salted-sha1 and ssha-opencl.
Comment out dynamic_1028. It was using un-implemented functions...
Improve valid() in all MSCHAPv2 and NETNTLMv2 formats.
Remove unused variable from agilekeychain-opencl.
doc/MARKOV note about UTF-8 filter.
Revert "SIP valid() fix."
Copyright blurb.
VMS: Better valid().
Drop unused variable from VMS.
Merge recent sxc fixes to sxc-opencl.
Merge a whole bunch of fixes made to CPU formats, to the OpenCL ones.
Conflicts: src/opencl_agilekeychain_fmt.c
Updated doc/BUGS for unstable.
Typo in README-OPENCL.
Update .mailmap for git shortlog.
Update CHANGES-jumbo.git (CREDITS-jumbo does not need update).
Document some requirements in doc/INSTALL.
100% white-space cleaning.
Change name from 1.7.9-jumbo-8-unstable to 1.7.9-jumbo-8-RC
Change --raw-always-valid (et al) to --bare-always-valid to avoid
confusion.
Documentation updates.
Add a line to --list=hidden-options:
Fix a cosmetic bug when listing formats with wildcards that match
dynamic.
Fix a couple of problems with raw-sha1-opencl. And while at it, bump
max length to 55 and use "compressed" key buffer for a nice
boost.
mkvcalcproba: Support reading from stdin.
genmkvpwd: Change %lld to %llu for correct printing of large numbers.
bash-completion: Drop trailing whitespace.
bash-completion: Redirect dynamic.conf errors to /dev/null.
Update CHANGES-jumbo.git (CREDITS-jumbo needed no change).
Add repeats16 and repeats32 external modes (from the obsolete contest
branch).
Update CREDITS-jumbo & CHANGES-jumbo.git.
Add a comment in ext_init explaining a confusing bail-out.
Endianness fixes in Blake.
Bad wordlist.c bug fixed, triggered when running -loop under MPI.
Bugfix for pbkdf2 CPU/OpenCL/CUDA code. Bug may have affected several
formats where salt length was larger than 23. Thanks to Marcus
Desto for reporting.
Update CHANGES-jumbo.git (CREDITS-jumbo needed no change).
Fix a bug where -DDEBUG would cause an infinite loop in
mem_alloc_tiny() because MEM_ALLOC_SIZE is set to zero
(Jumbo-specific) and we ended up never breaking out of the while
loop. Thanks to bytebieger for noticing the problem and
patiently convince me.
Fix another potential problem with -DDEBUG vs. mem_alloc_tiny().
Print human readable OpenSSL version string with --list=build-info.
Update CHANGES-jumbo & CREDITS-jumbo.git
Use the recent mem_alloc_tiny() bugfix for size zero even when not
-DDEBUG because returning a pointer might be asking for problems.
Unicode bugfix for OpenCL Office formats.
raw-sha1-opencl: Fix bug where obsolete code that only handled up
to length 23 was accidentally used. And drop the obsolete code.
wpapcap2john: Add SSID in "user" field for Single mode use.
hccap2john: Add SSID in "user" field for Single mode use.
Make Oracle 10 format more robust (allow lower-case salts even when
prepare() is not used).
Another tweak to the mem_alloc_tiny() vs. -DDEBUG issue.
 wpapcap2john: Ensure we don't overrun SSID buffer with overlong tags,
and print a debug message. Thanks to bytebieger for reporting
this issue. We still do not know the cause. See
https://github.com/openwall/john/pull/366 for
discussion.
NVCC tweaks (mostly comments, but add -fmad=true) in Makefile.
Fix an obscure bug in mscash where an overlong key would truncate
to 28 instead of 27 in UTF-8 mode. NT already had a fix for it so
unify the code for them.
Protect a CUDA macro from being expanded to something not intended.
I'm not sure there was a problem, just saw the dangerous definiton.
Change malloc() to mem_alloc() in a bunch of formats. The latter sits
in memory.c and has error checking.
Update CHANGES-jumbo.git and CREDITS-jumbo
sapG: Add FMT_UTF8 even though we are not FMT_UNICODE.
Improve valid() in netlm, nethalflm and mssql12.
Improve valid() in krb5pa formats.
Explicitly state that wpapsk formats do support WPA and WPA2.
Update CHANGES-jumbo.git & CREDITS-jumbo
Merge recent change to DMG CPU format to OpenCL.
Use 8x for SIMD SHA-1, for gcc >= 4.7 if AVX or XOP.
Fix OSX build targets for CUDA for Xcode 5.0 and CUDA 5.
mysql-sha1 formats: SPLIT_UNIFIES_CASE bugfix and unify test vectors.
Better description of MySQL formats vs. versions.
Bugfixes from bleeding back-ported to unstable.
Fixes for building CUDA on OSX Mavericks.
Makefile CUDA: Add natural path to nvcc with an NVCC macro, and
make the OSX -ccbin quirk a macro too.
CUDA fix for when nvcc is not in /usr/local/bin.
Drop -cl-strict-aliasing from OpenCL build options. It's deprecated.
Print a warning when FMT_NOT_EXACT.
pkzip: Bugfix for #467. Two early rejection tests had to be disabled.
Hopefully they can be re-enabled by JimF after some tuning.
Update CREDITS-jumbo & CHANGES-jumbo.git
Fix an old bug with CPstoreUTF8 vs. split hashes and --show. Some
cracked hashes with passwords containing 8-bit characters would be
output as partially uncracked in case CPstoreUTF8 was enabled.
SAP B passed salts to loader with possible garbage in the end
(variable length). This lead to loader potentially seeing
"different" salts even though they were the same. Closes #608.
It turns out krb5-23 are just NT hashes. Drop the krb5-23 format
(it was a lot slower) and change kdcdump2john.py so it outputs
$NT$ tags for those.
Add some alternative addresses for Dhiru to .mailmap so Git knows
he is one same person

Dhiru Kholia (337):

BONUS: pwsafe format also cracks Password Gorilla databases.
Integrate psk-crack (from ike-scan) into john.
Add support for cracking Nuked-Klan CMS hashes.
Fix build.
Bump LINE_BUFFER_SIZE. Needed for pfx format.
OpenCL implementation of Keychain format
Keychain format is now 2.4x faster on CPU and scales almost linearly.
Thanks to Lukas!
Fix keychain format crash due to stack smashing
Add support for cracking 1Password Agile Keychain files
Reduce false positives in agilekeychain format
OpenCL implementation of Agile Keychain format
integrate keychain2john program with john
Add support for cracking ODF 1.2 AES encrypted documents
Update FORMAT_NAME in ODF format
Remove my hacks from ODF format
misc. minor fixes
Add AES-256 encrypted ZIP file test vector.
OpenCL implementation of ZIP AES format
Do not build Truecrypt unless OpenSSL >= 1.0.1
Add support for cracking M$ Outlook's PST files
GNOME Keyring cracker patch for JtR
Add keyring2john program (for now it is standalone).
Add support for cracking Office 97-2003 encrypted files
EncFS cracker patch for JtR
OpenCL implementation of EncFS cracker
OpenCL implementation of ODF format
SXC cracker patch for JtR
OpenCL implementation of SXC format
Respect "iterations" parameter in PBKDF2 code
Respect "iterations" parameter in keychain PBKDF2 kernel
add test vectors from CMIYC 2012
add test vectors from CMIYC 2012
add test vectors from CMIYC 2012
pdf2john fixes by JimF
Add OMP support in PDF format.
Detect PDF files using AES encryption and bail out early
Fix bug in PDF format exposed by a new test vector (added to pdfdump)
Detect unsupported PDF files and bail out
Implement npdf format which will eventually replace the existing pdf
format (once npdf2john is committed).
Remove ":" from npdf test vectors
Make npdf format compatible with core changes
Use our own SHA2 implementation
Fix breakage in keychain2john program
Add support for cracking PuTTY private keys
Rename ppk format to putty
Use our faster HMAC-SHA1 implementation
Add support for cracking OpenPGP / GnuPG private key files
Add support for more hashing algorithms
add test vectors from CMIYC 2012
OpenCL implementation of GPG format
Make NVIDIA OpenCL compiler happy
Integrate gpg2john with john
Add documentation for gpg format
Add OMP support in gpg-opencl format
Add support for cracking MS SQL 2012 hashes
Add support for cracking MySQL network authentication hashes
Make mysqlna format compatible with core changes
(v5) Add support for cracking DMG files
simplify DMG format
fix detection of plain DMG files
add more test vectors
Add dmg2john to .gitignore
Add OMP support in dmg format
OpenCL implementation of DMG format
Begin adding Office RC4 CryptoAPI Encryption support
Office RC4 CryptoAPI Encryption support is done for doc files
Office RC4 CryptoAPI Encryption support is done for ppt files
Office RC4 CryptoAPI Encryption support is done for xls files
Fix some correctness issues
Add support for cracking Office 2013 files
Add support for cracking STRIP Password Manager databases
ml2john.py extracts Mountain Lion hashes
fix permissions
fix shebang line
Fix OMP support in new KRB formats
Fix krb5-23 format on OSX
Fix krb5-18 format on OSX
Make Pyflakes happy
OMP build fixes for EncFS format
Add support for cracking GRUB2 and OS X 10.8 hashes. Coded by Lukas.
Remove my hacks from VNC format
Update for Office 2013
OpenCL implementation of STRIP format
Improve error handling
Add support for parsing newer Office files to office2john.py
Support cracking of iSCSI CHAP authentication hashes
dos2unix on chap_fmt_plug.c
Update OleFileIO_PL code
Use only filename instead of full path
Fix a correctness issue in Office format
Fix warnings and indentation in FGT format
Fix FGT format crash when built using linux-x86-64-clang-debug target
Add another test vector in CHAP format
Fix memory leaks in EncFS formats
Remove dead assignments
Fix license issues
Begin adding OMP-support to hmac formats
Add OMP support to hmac formats
Add support for cracking Oracle O5LOGON protocol hashes
PPC fixes
add more test vectors in o5logon format
Add support for PostgreSQL MD5 challenge-response cracking
Add usage instructions in VNC format
Speed-up o5logon format.
Remove my hacks from o5logon format
Revert "Remove my hacks from o5logon format"
Fix build when using ICC
Integrate OpenVMS support 2.4 patch by David Jones
Pull changes from v2 version of VMS patch
New version of office2john that rips keywords from document metadata
and puts them in GECOS field for use by Single mode. Also bump
some figures in params.h to cope with more words in said field.
Fix issue #100 gpg-opencl does not work
Fix static build on Owl-current systems
Add some more test vectors
Add ssh-ng format which is 4 to 5 times faster than ssh format
add DSA test vector in gpg format
Add another test vector (from MIT Kerberos 1.9 setup)
Add MS Kerberos 5 AS-REQ Pre-Auth aes256-cts-hmac-sha1-96 format
add crypt(md5(p),salt) format
Add support for cracking WebEdition CMS hashes
Add support for cracking sniffed LastPass hashes
Add support for cracking Clipperz databases
Add documentation for Keyring and Keychain formats
Add another test vector (from Windows 8 + 2012 AD setup)
Fix format flags
Cracker for both MongoDB system and sniffed network hashes
Add more MongoDB network test vectors
Add more MongoDB network test vectors
Add one more test vector (stolen from chapcrack)
Fix valid in gpg-opencl format
Add support for parsing rc4-hmac hashes and unify input format
make ciphertext parsing more robust in dmg formats
make ciphertext parsing more robust in gpg format
make ciphertext parsing more robust in npdf format
make ciphertext parsing more robust in chap format
make ciphertext parsing more robust in agilekeychain formats
make ciphertext parsing more robust in keychain formats
make ciphertext parsing more robust in keyring format
make ciphertext parsing more robust in mysqlna format
make ciphertext parsing more robust in krb5-18 format
Make dmg parser more robust
make ciphertext parsing more robust in RACF format
make ciphertext parsing more robust in ssh-ng format
Fix valid in RACF format
fix use-after-free bug
Make magnum and Clang happy at the same time
Add one more dynamic_1009 sample
make ciphertext parsing more robust in putty format
Fix build failure on FreeBSD 9.1
Add support for sha1($s.$p) which is used by SilverStripe CMS
Revert "Add support for sha1($s.$p) which is used by SilverStripe CMS"
Add another test vector in sha1-gen format
make ciphertext parsing more robust in keepass format
make ciphertext parsing more robust in encfs format
make ciphertext parsing more robust in encfs-opencl format
make ciphertext parsing more robust in strip formats
Revert "Add support for sha1($s.$p) which is used by SilverStripe CMS"
Add another test vector in sha1-gen format
Fix earlier breakage of odf-opencl format
make ciphertext parsing more robust in odf formats
Reduce weirdness in dmg format
make ciphertext parsing more robust in sxc formats
Fix potential memory leak in pwsafe format
Avoid pointless memcpy call in npdf format
Fix check (and avoid Clang warning) in ssh-ng format
Fix potential memory leaks and warnings in keepass2john
Avoid Clang warning in unrar library code
Fix warnings in gpg format
Fix potential leaks indicated by scan-build
Fix potential leaks indicated by scan-build in pdf2john
make ciphertext parsing more robust in pfx format
make ciphertext parsing more robust in ssh format
make ciphertext parsing more robust in o5logon format
make ciphertext parsing more robust in wbb3 format
Fix to pass new self-test for maximum length
Fix to pass new self-test for maximum length
Fix "Bad Idea" in agilekeychain formats
Fix "Bad Idea" in chap format
Fix "Bad Idea" in gpg formats
Fix "Bad Idea" in keychain formats
Fix "Bad Idea" in keyring format
Fix "Bad Idea" in ssh-ng format
Make ciphertext parsing more robust in LastPass format
Make ciphertext parsing more robust in MongoDB format
Get rid of Clang warnings for rar2john
Silence clang warning in KeyDBCracker.c
1. Fix valid in pdf format (patch from Narendra) 2. Avoid dynamic
memory allocation
Make agilekc2john more robust against bad data
Make encfs2john more robust against bad data
Make keepass2john more robust against bad data
Make zip2john more robust against bad data
Max length bugfix in pkzip format
Integrate keyring2john with john
Misc. updates for keyring2john
Get rid of some clang warnings
Add support for BLAKE2 hash function (https://blake2.net/)
Add another BLAKE2 test vector
Add some more "popular" BLAKE2 test vectors
Fix description and make Frank happy
Add support for cracking ODF files using AES encryption on GPU
Output ML hashes in gangnam (err... magnum) style
Fix badness in odf format
Add support for cracking LastPass's offline database
Implement fix suggested by magnum
Rename keychain.h to pbkdf2_hmac_sha1.h, add pbkdf2_hmac_sha256.h
Add npdf2john.py written by Shane
Get rid of older pdf format
Remove references to pdf2john which is now gone
Rename npdf to pdf
Use optimized implementations of BLAKE2 on supported platforms
Add htdigest2john.py utility. Thanks to Jim for help!
Fix generic build (BLAKE2 build problem)
Add OMP support to TrueCrypt formats
Rename pdfdump to pdfdump_old and add new working pdfdump file
Use correct value of BINARY_SIZE
Remove debugging code (linux-x86-64-clang-debug was giving warnings)
Fix warnings when building linux-x86-64-clang-debug target
Document usage of ssh formats
Fix multiple bugs in sshng2john.py
Fix problem in office format found by magnum by using valgrind
Fix problem in office OpenCL format found by magnum by using valgrind
The "cracked array" was not always reset correctly. Fix it.
Integrate dmg2john with john
Remove dmg2john target and stop confusing people
Fix running of KWallet format on big-endian systems
Fix warnings in dmg2john
Add support for WinRAR SFX files
Information on cracking Oracle APEX hashes
Misc. fixes for putty2john utility
Add support for handling multiple files in uaf2john
Add support for handling multiple files in truecrypt_volume2john
Dynamic type for md5($p,$s) for long salts
Remove redefinition (issue #233)
Add utility to convert Cisco type-4 hashes into raw-sha256 hashes
Fix ASAN crash in pkzip format
make valid() "stronger"
Fix ASAN crash in dmd5 format
Fix ASAN crash in oracle format
Fix ASAN crash (still needs better valid)
Fix ASAN crash (needs another look)
Commit another weird and custom format
Fix ASAN crash and (get|set)_salt methods
Python port of dmg2john utility
Fix ASAN crash in ipb2 format
Fix multiple ASAN related crashes
Fix ASAN related crashes in SAP formats
Revert "Fix ASAN crash (needs another look)"
Fix ASAN crash (without any performance loss)
Revert "Fix ASAN related crashes in SAP formats"
Document known ASAN related crashes
Fix mistakes made in commit 23dc44df
Revert "Fix ASAN crash in ipb2 format" due to speed loss
Document known ASAN related crash in ipb2 format
Fix ASAN related crash in nk format
Revert "Fix ASAN crash in dmd5 format" (better fix coming)
Fix ASAN crash in dmd5 format
Fix problems pointed out by Frank
fix clang-analyzer warning in mozilla DES code
misc. fixes
Remove dead code
Remove dead assignment
fix resource / memory leaks in dmg2john.c
Remove unused code
misc. small fixes in dmg2john
Fix a very weird compilation error with GCC 4.8.0
Fix compilation failure with Clang (trunk)
Fix memory leak + "invalid read" in pwsafe formats
Fix "invalid read" in keepass format
Fix "invalid read" in ike format
Fix misc. "invalid read" problems
make valid() stronger in clipperz format
make valid() stronger in radmin format
make valid() stronger in pst format
make valid() stronger in mozilla format
fix potential leaks in dmg format
fix memory leaks in zip2john
fix potential memory leaks in rar2john
fix typos
fix warnings in gpg2john
Fix problems found by Alexander Cherepanov in mozilla format
Fix valid in sip format
Fix various crashes found by Alexander Cherepanov
Fix crash in zip format found by Alexander Cherepanov
Fix various crashes found by Alexander Cherepanov
Revert "Fix crash in zip format found by Alexander Cherepanov"
Hopefully the "right" fix for the zip crash
misc. fixes for zip stuff
Fix crash in dmd5 format found by Alexander Cherepanov
Fix django crash found by Alexander Cherepanov
Fix a typo in keepass2john
Fix base64 decoding bug and add support for Cloud Keychain files
Make agilekc2john work under Python 3.3 too
Make sshng2john work under Python 3.3 too
Make sshng2john work under Python 2.4 to Python 3.3
Make agilekc2john work under Python 2.4 to Python 3.3
Fix crash for "qpdf" generated file and misc. PEP8 fixes
Remove unused code from ml2john.py
misc. Python portability fixes
Make dmg2john work under Python 2.4 to Python 3.3
Fix for django format crash (one of the many) :(
more portability fixes
Make odf2john work under Python 3.3 too
even more portability fixes
 portability fixes
Make office2john.py work under Python 3.3 too
Add error handling in pdf2john
some less-than-ideal fixes for keyring2john
Agile Keychain optimizations (thanks atom!)
Calculate only what we will use (thanks atom!)
Even more optimizations (courtesy of atom)
misc. portability fixes
Add ssh2sshng.py utility for converting ssh hashes into sshng hashes
Add support for cracking Siemens S7 authentication protocol hashes
misc. portability fixes
make s7 format compatible with unstable-jumbo
Don't be too greedy. Fix failures caused by 07a3e05.
Document that mozilla format fails on big-endian boxes
Changes from commit 7eecef982c5d458a22d8c9c3b870743edfc5ba76
Pick changes from bleeding-jumbo
Pick tweaks from corresponding CPU format
fix crash in ike format
fix crash in dmg format
borrow valid from the corresponding CPU format
fix crash in keyring format
tighten check in episerver format
tighten check in clipperz format
New dynamic types from Alexander Cherepanov and "Waffle"
fix yet another crash in dmg format
check maximum hash length in episerver format
check maximum username length in mongodb format
fix crash in putty format
fix crash in office format
fix crash in sxc format
fix crash in wbb3 format
fix crash in pdf format
clean up dmg stuff a bit
Start using Travis CI
tweaks required for building on FreeBSD 10 (32-bit)

Claudio André (157):

Improvement: pre-compute modulus values.
Improvement: multi-kernel mode created.
Improvement: allow to select at runtime the kernel to be executed.
Improvement: create a default/safe kernel.
Fix: allow user to run the software without
cl_khr_byte_addressable_store
Improvement: create a fast mode (more aggressive defaults).
Fix: inside get_processor_family (OpenCL) code.
Fix: missed some checks on common-opencl.c for disabled
cl_khr_byte_addressable_store.
Improvement: create a better way to select default global work size.
Improvement: a new AMD kernel that run in a set of stages.
Improvement: a new NVIDIA kernel that run in a set of stages.
Some incorrect comment changed.
Improvement: merge CPU and default kernels.
New OpenCL sha256crypt format.
Fix: code clean-up.
Merge with jumbo-6-fixes
Fix: more code clean-up.
Merge with jumbo-6-fixes
Fix: decrease memory usage.
Fix: code cleanup.
Fix: proper get_salt and set_salt.
Fix: proper get_salt, set_salt, valid and cmp_one.
Create an OpenCL sha256crypt for AMD hardware.
OpenCL kernels (cryptsha256) ready.
Fix: salt usage.
Fix: computation of GWS and LWS.
Fix: code clean-up.
Fix: detection of no_byte_addressable. Fix: new cryptsha512
working on CPU.
Fix: new cryptsha512 working on AMD.
Fix: code clean-up.
Include a referente to sha256crypt-opencl on DOC.
Fix: typos i made on kernel names.
Fix: new cryptsha512 working on NVIDIA.
Fix: some code clean-up.
Fix: use static when it makes sense.
Fix: remove commented code.
Fix: code clean-up.
Improvement: create a new raw sha512-ng.
Fix: unnecessary data conversion.
Fix: better (more correct) usage of John API.
Fix: get_hash error detected by TS.
Fix: code clean-up.
Use the fast approach to test on Jumbo.
Fix: code clean-up.
Applied: All known changes from JimF, along with unification of
differences between core-jumbo-magjumbo-bleeding
New control to PLAINTEXT_LENGTH. Check if magnum agrees.
Apply magnum technique of DEBUG usage to add tests.
Change type of large constants.
Version that uses __local memory. Keep on git to test different
hardware when possible.
Transfer less data between host and device. 5% faster.
Fix a compiler warning on 32-bits Set Plaintext size to 32.
Remove a develoment/debug function call from source code.
Code cleanup.
Use mem_alloc instead of malloc.
New validation of max global memory allocation size (very usefull to
fast formats).
Fix output format string.
Fix a typo.
Use dinamic allocation for the build log buffer.
Better control of auto-setting process.
Try on raw SHA-512 magnun's MaxRunTime setup.
As magnum said on list, this check is not necessary anymore (on JtR).
Improvement: 7% performance gain.
Avoid unnecessary data moving.
Code clean-up.
Revert name used.
Create a new version that uses __local memory. Good (at least) in
7970.
Remove copied code from everywhere. A new .h file has all necessary
code.
Fix format identification message.
Fix no_byte_addressable code.
Apply the same fix of no_byte_addressable to raw_sha512 and
sha256crypt.
Rename all those names to eg. DEV_UNKNOWN, DEV_NVIDIA and so on.
Go on and rename all those names to eg. DEV_UNKNOWN, DEV_NVIDIA and so
 on.
Fix an architecture misdetection.
Use magnun's MaxRunTime setup (256 and 512 SHA crypt). LWS command
line proper detection (Fix). Change name: from pFmt to self.
Mitigate GPU hogging with much shorter kernel durations (AMD GPU
only). Better desktop response during cracking. Some
algorithm clean-up (on AMD at least 20% performance improvement).
Better defaults.
Tried to implement a salt_hash function. It is failing despite the
hash calculus is working.
Remove some debug code.
Fix a problem with salt detection.
Fix: do not show an error message twice.
Split kernel execution (for NVIDIA).
Strategy change on AMD GPU.
Fixed a logical error.
Fix a typo.
Remove unused code.
Split kernel execution.
New names pattern applied.
Set plaintext size to 32.
Create a version of sha256crypt that uses __local memory.
Faster memory transfers.
Revert sha256crypt plaintext size to 24. With 32 the performance
impact on gcn is not acceptable.
Fix error reported on OSX.
Mute a compiler warning on OSX.
Fix memory leak, as reported by magnum.
Use firstEvent and lastEvent. Nothing bad happened, but less than
optimal values were selected some times.
Revert. Saw zero benefits.
Few ideas on LOCAL approach.
Fix to allow this format to be used on bleeding.
New OpenCL raw-sha256 format created.
Missing files to raw-sha256 format.
Fix: size of OpenCL buffer.
Some code clean up.
Some fixes on raw sha256.
Small optimizations.
Update ALGORITHM_NAME to the text pattern used in jumbo-7.
Fix a bug in GPU.
Clean up on raw-sha256 code.
Revert "Few ideas on LOCAL approach".
On fast hashes: implement clear_keys() for faster buffer cleaning.
Add missing stuff for FMT_MAIN_VERSION > 9
Revert "On fast hashes: implement clear_keys() for faster buffer
cleaning.". Hurts performance.
Use defines instead of numbers.
Add null-hash test vectors to "my" unsalted formats.
Links for wiki page on new formats.
Fix align as reported by magnum.
Mute a warning.
Improvement on LWS/GWS management. Fix a typo.
Set (min/max) KEYS_PER_CRYPT=1 and adjust to the real value on init().
New format xsha512-ng-opencl: a little bit faster than stock code.
On bull: Benchmarking: Mac OS X 10.7+ salted SHA-512 (pwlen <
32) [OpenCL (inefficient, development use mostly)]... DONE Many
salts: 67346K c/s real, 67346K c/s virtual Only one salt:
23130K c/s real, 26214K c/s virtual
Include a new observation about credits.
New wiki page created.
Remove duplicated code.
On raw-sha256: unroll 'block' and use defines when possible.
More code clean-up.
Fix a bug on NVIDIA.
Partial revert on raw-sha256. More code clean-up.
raw-sha512-ng: Fix a format struct problem for v10.
Fix a bug inside get_salt.
Fix a bug inside get_salt (not OpenCL files) .
Put Johnny information on README file.
Inside shared find_best_workgroup avoid creating new queues.
Revert "Fix a typo."
Shared code to allow to build and use binary OpenCL kernels.
Move all OpenCL files to a new folder.
Missed rar on 6b66504.
Quick-response to "events" (like key presses) for my loop-kernel
OpenCL formats. Adjust min_keys_per_crypt to local_work_size.
Adjust global worksize to the count argument inside crypt_all().
Rewrite find_best_gws to avoid any warning using sprofile.
Rewrite of auto-tuning (gws_test) code. Auto-tuning by doing
less iterations. Show (if required) execution times and detais:
- print: pass xfer: 10.01 ms, crypt: 3.46 ms, result xfer: 1.84
ms Clean code. Create new files to handle duplicated code. -
See magnum work at commit fff72afafcb0a502163e57d97491c6b57c7f8cc2.
In auto-tuning, adjust max run time for GPU and CPU accordingly.
sha256crypt, sha256crypt: use cached-binary OpenCL kernels.
Remove temp (__local) buffer on prepare (it produces no benefits).
Quick-response to "events" (like key presses) for "my" loop-kernel
OpenCL formats. Adjust min_keys_per_crypt to local_work_size.
Adjust global worksize to the count argument inside crypt_all().
Rewrite find_best_gws to avoid any warning using sprofile.
Rewrite of auto-tuning (gws_test) code. Auto-tuning by doing
less iterations. . See magnum work at commit fff72af. Show
(if required) execution times and detais: . print something like
this: pass xfer: 10.01 ms, crypt: 3.46 ms, result xfer: 1.84 ms
Clean code. Create new files to handle duplicated code. In
auto-tuning, adjust max run time for GPU and CPU accordingly.
sha256crypt, sha512crypt: use cached-binary OpenCL kernels.
Remove temp (__local) buffer on prepare (it produces no benefits).
Remove sha512crypt version that uses __local memory. No benefit using
it.
Adjust min_keys_per_crypt to local_work_size. Adjust global
worksize to the count argument inside crypt_all(). Treat the
new_keys condition. Include a new salt_hash fucntion. Fix
the bug reported on wiki JtR-valid-bugs. Proper done
(atexit(release_all)) method. Can handle md5crypt_MaxDuration in
config file.
Adjust min_keys_per_crypt to local_work_size. Adjust global
worksize to the count argument inside crypt_all(). Proper done
(atexit(release_all)) method. Can handle MaxDuration in config
file.
Include LWS/GWS information inside john.conf for formats that miss
that.
Fix: use the decreased global work size in cmp_all().
Use binary kernels on 'my' fast hashes too.
Fix: missing colon on DEBUG target.
Include LWS/GWS information for raw-sha256 inside john.conf.
More tests for DEBUG target.
 Fix a bug inside get_binary.
50% performance gain on GCN for sha512crypt. Recommended for (at
least) 12.08 Catalyst driver. From: Local work size (LWS)
64, global work size (GWS) 16384 Benchmarking: sha512crypt
(rounds=5000) [OpenCL]... DONE Raw: 8316 c/s real, 409600
c/s virtual
Small tweaks on default configuration. Local work size (LWS) 64,
global work size (GWS) 32768 Benchmarking: sha512crypt
(rounds=5000) [OpenCL]... DONE Raw: 13159 c/s real, 819200
c/s virtual
Fix: for md5crypt on CPU.
Fix: use a safer check to find gws.
Fix a bug using sha256crypt on OSX CPU.
Small tweaks on sha512crypt OpenCL file.
Include uint64_t definition (stdint.h) in common-opencl.h.
Add FMT_SPLIT_UNIFIES_CASE to a bunch of SHA-2 formats.
Mute a few warnings when using test=0 on OSX.
Mute some warnings.

JimF (64):
Add our own SHA-2, enabling 19 new formats for users stuck with
OpenSSL 0.9.7. OpenSSL or not, boost sha256crypt with 10-20%.
Fix radmin format under Windows.
Alignment fixes SHA2 on Sparc
Handle CommonCrypt in our SHA-2
CommonCrypto fix in sha256crypt
JtR-bleed-align-macro-fix-1.diff
JtR-Bleed-make-generic-unaligned.diff
JtR-New-Dynamic-35-36-SHA-types.diff
ike-crack.h: use our local stdint.h instead of global
JtR-MAG-UNSTABLE-ONLY-Dynamic-sha256-addition.diff
Updated dynamic adding sha2 formats, and GOST, also made a thin format
formspring to do regen-salts type 6
Dynamic name change (for display), and adding config option and
command line switch to cause dynamic to always use raw hashes
Missed this file in the dynamic raw-hashes-allways-valid patch
john-1.7.9.5-c6-sunmd5-v1.patch
New, SSE2 capable sunmd5
sunmd5 fixed for mmx, sunmd5 added to pass_gen.pl
Updated sunmd5 and added a couple of optimizations. Added finer
granularity memory dumping to memory.c/.h
many changes I had staged. many were performance changes, some were
porting changes. dynamic is now also updated. Also the POC wow_srp
format added
Added OMP, removed a few compile warnings, depending upon compile
conditions, oSSL/GMP builds
GMP fix for 64 bit
changes to dynamic, adding new methods to give more flexability to
larger formats, multiple output types, upon rehashing. Also updated
documentation
Portability issues fixed
stack overwrite bug fixed
added JtR tool to_dyna to the build. This tool converts hash:salt or
salt:hash into proper dyna format, converting salt to $HEX$ if
needed
Dynamic fixes, from Jim.
Here is a patch that removes any plug dependencies.
1. in formats.c, the 2nd loop in the 'max length' check, should start
at 0, not at 1.
 Bug fixes for mask mode.
Some changes to raw2dyna
Mask mode bugfix
Fix generic build (dynamic_generic_fix.diff)
Bug fixes for dynamic union crypt_key
SSE2 support for WPA-PSK. The code was taken and modified from my code
in mscash2. They are pretty different, due to how the main
format logic is (mscash2 split out just the inner function, this
one did not break out the crypt). But they do about the same
things. Also, in SSE2, mscash2 BE's the binary data. I do not
do that here, since the wpapsk.h is common code to
CPU/CUDA/OpenCL, and that is where that functionality is. So, there
simply is a call to alter_endianity_to_BE when the results are
stored at the end of crypt. There really should be no speed
impact, due to this fact.
Here are a couple more fixes
Implement ldr_sort_salts(), currently hard-coded to only apply to
wpapsk formats. In the future we will probably want a format
struct entry for format->methods.salt_compare().
Some porting items. The typecasts are due to void* having an undefined
size (on some compilerrs). error C2036: 'void *' : unknown
size
Dynamic: Remove many more of the typecasts, putting them into anon
unions.
listconf.c: Ensure we have a proper version of NSS lib before calling.
bench.c: Get around a crash in VC (and mingw), in -test=0 mode.
These only use clock(), so if the bench is fast, there are 0 clocks
used. I simply put a 1 in the time slot, if it is 0.
mingw port issues
More mingw port issues
Another version check for NSPR in listconf.c
Some warning pasifications
pass_gen.pl: Support creating WPA-PSK test hashes!
dyna_17_BE.patch from Jim
basename portability 'fix'.
Portablity fixes for VC.
fix wpapcap2john warnings
Dynamic bugfix, spotted by new warning in gcc 4.8.0.
Fixed mem leaks, found by mshell.
Minor fixes.
JtR-jumbo-dynamic_1014_fix_non_sse.patch
JtR-Dynamic_validation_raw_hashes.patch
JtR-Dynamic_validation_raw_hashes-2.patch
Re-apply JtR-Dynamic_validation_raw_hashes-2.patch but with some
missing stuff added.
JtR_Bleeding_Dynamic_Salt2_UserID_Salt_Bug.patch
JtR_Bleeding_Dynamic_MEM_ALLOC_SIZE.patch
JtR_unstable_Dynamic_bare_hash_fix.patch
JtR-SunMD5-SSE2-scale.patch
Disable OMP for MMX in wpapsk format
Dyna21-crash.patch
dynamic-valid.patch
JtR-Unstable-crash-fix-dyna_21.patch
JtR-Unstable-Dyna-big-hash-fixes.patch

Frank Dittrich (57):

Make --list= a little bit more user friendly
Bash completion: add support for --list=format-methods[:WHICH]
Silence -Wpointer-sign warnings in hmacSHA*_fmt_plug.c, take 2
Add --help option to john (prints usage summary)
Define new [List.External:Repeats_*] sections
New external mode variable "maxlen"
Rename external mode variable maxlen to cipher_limit
Rewrite Filter_ROT13 external mode to allow easier adjustment
Dummy format: warn about unsupported password length
Define new external modes to generate sequences of characters
Fix compiler warnings in dummy.c
Make use of cipher_limit in some more external formats
Fix warning in dummy.c, this time for real
Bash completion: add trailing space for --nolog etc.
bash completion fix
Fix bash completion for hidden options
Fix bash completion for --subformat
Fix bash completion for --option[=value]
Rewrite bash completion for automatic handling of new john options
Refer to "doc/OPTIONS" for help regarding --fix-state-delay=N
Fix ./john --list=parameters:SECTION (with .include [other])
Bash completion: support (src)$ cd ../run; ./john -[tab]
sunmd5_fmt_plug.c: Fix -Wunused-variable and -Wunused-but-set-variable
for generic builds (on a 32bit Linux system)
Adjust EPI format documentation
Rename doc/EPi.patch.README to doc/README.format-epi Reason: there
is no longer a separate patch
Fix broken links in source code
SAP CODVN B and F(G): be more restrictive in valid()
Fix "Compiler error in ./john.conf at line 612: Unknown identifier"
which occurred when using $ ./john --list=ext-filters-only
Separate function john_list_build_info
Add GMP library version to --list=build-info
Add even more version info to --list=build-info
Remove some obsolete FIXME comments from bash completion script
Fix a few typos in listconf.c comments
Fix typos in calc_stat.c
Fix more spelling errors
Fix spelling: then -> than
Fix typo in doc/README-OPENCL
Fixed some more typos
Add binary_hash_[56] and get_hash_[56] to 16 formats
mozilla_des.c and .h: #if 0 unused functions
sap2john.pl: improve coding style (tabs vs. spaces)
doc/OPTIONS: remove the only [tab] which is not part of a heading
Remove dead code from mozilla_fmt.c - used #ifdef DEBUF
Fix grammar
Adjust benchmark-unify to jumbo-8 format name changes
Benchmark-unify: map KRB5 format names
Fix a bug when using --stdout, introduced in e14ac00.
benchmark-unify: adjust format name mappings Add "Mozilla SHA-1
3DES" mapping Adjust "pdf" mapping
remove unused variable in putty_fmt_plug.c, valid()
Fix bash completion for formats if --list=hidden-option has
--format=CLASS
Bash completion: Workaround for --format=CLASS parsing
Fix bash completion for formats if --list=hidden-option has
--format=CLASS
Bash completion: Workaround for --format=CLASS parsing
dynamic_THIN_FORMAT_LINK: use PASSWORD_HASH_SIZES ... to make sure
all the binary_hash[i] and get_hash[i] methods get passed to the
thin formats.
 benchmark-unify: format name mappings sorted (sort --ignore-case)
benchmark-unify: add/change some mappings
.mailmap: make sure git maps my commits to my new email address

Sayantan Datta (25):

common-opencl.c:Added build from binary
mscash2-opencl:bitselect optimization
New opencl format:DES-BS
DES-opencl:various optimizations
des-opencl:use local memory
des-opencl:Performance improvement
des-opencl:performance improvement, broken ATI 4xxx
des-opencl:fixed ATI 4xxx
opencl-des: nv performance improvement
des-opencl:use bitselect for AMD gpus
bf-opencl:added cpu optimized kernel
mscash2-opencl: split kernel
des-opencl:runtime kernel preprocessing
des-opencl:slightly lowered memory usage
des-opencl: minor kernel improvements
mscash2-opencl:auto detect gws,apple cpu bug
des-opencl:fixed detection of matching salt
des-opencl:hardcoded kernel
des-opencl:added fall back kernels
opencl-mscash2: bug fix
mscash2-opencl:bug fixes
opencl-mscash2:bug-fixes
mscash2-opencl honours LWS and GWS env variables, des-opencl and
bf-opencl honours GWS environment variables, added functions to
detect optimal GWS for bf-opencl and des-opencl
bcrypt-opencl: fixed AMD 7970 ASIC hang
bf-opencl:fixed 7970 ASIC hang

Lukas Odzioba (22):

Make Truecrypt formats non-plugin and only build for OpenSSL >= 1.0.0
Fixed valid in pwsafe formats
Fixed valid() in pwsafe formats
removed rm/remove
Replace malloc() with mem_alloc() in many places.
I did some minor changes mostly in opencl/cuda formats. I also
added mem_calloc(size) to memory.c/h, not a big deal but helps
clean tables for candidates with just one line of code.
Changed names of fmt_main structs listed below, to be consistent with
naming convention we use most often: -name starts with
"fmt_" prefix, or "fmt_opencl", "fmt_cuda" for gpu formats.
More fmt_main name convention fixes
Rename CL_VERSION_1_0 macro to HAVE_OPENCL to clean the code a bit.
Some formats didn't respect -dev option given by command line.
gpg2john: Handle multiple keys per in file, as well as multiple
infiles.
Add OpenCL 1.2 error codes to our debug routines.
Include OpenCL version information into --list=build-info
CUDA versions in --list=build-info
(re-)enable agilekeychain format
gpg2john fixes
fix vncpcap2john exception handling
Basename helper function in misc.c, also capable of stripping
extensions.
gpg2john GECOS support
 GPG valid() fixes.
SIP valid() fix.
SIP valid() fix.

ShaneQful (18):
Added author comment at top of file & chmodded it to be executable
Added prepare method which checks for the old style pdf hashes and
converts them to the new style pdf hashes if necessary
Cleaned up comments and added self to credits
Fixed bug Dhiru found in pdf2john.py. There was an issue with one of
my regexs so that it didn't cover all situations
Remove FIXME(s) (Don't scare Frank and magnum anymore)
Removed assert which caused bugs in some documents generated with word
for the mac
Fixed spelling error
Update CREDITS-jumbo
Added License to pdf2john
Added Python 3 support for pdf2john
Changed is_meta_data_encrypted to return a boolean rather than a
string
Changed get_encrypted_object_id to get_object_id in order to get any
object id from the trailer
Impemented parse meta method, just get values methods left
Fix for checking meta data encrypted in python 3.2
pdf2john now gets the title and year created from xmp metadata
pdf2john now gets all relevant metadata when metadata left unencrypted
Fixed bug in pdf2john where wrong objects could have been used
Small fix to get_xmp_values method in pdf to john

Alexander Cherepanov (16):

sunmd5 coinflip speedup
Just a small patch to fix strncmp in NETLM format.
WPAPSK bugfix.
Some typos.
Typo in dynamic.conf
Replace unicode em-dash in putty files.
Replace non-ascii char by hex code in pass_gen.pl.
Replace several non-ascii copyright signs by "(c)" string.
Fix typos in README.bash-completion.
Fix some typos in README-OPENCL.
Add some band-aid to valid() in sxc format.
Improve robustness of valid() in pkzip format.
Make valid() in sunmd5 format robust.
Fix a crash in dynamic with an overlong salt.
undrop: limit the length of password. (Fixes a crash.)
krb5: ensure that username and realm are non-empty. (Fixes SIGFPE.)

Solar (11):
hccap2john fixes
RAdmin, SIP speedup
sunmd5: minor fixes
sunmd5: Get rid of modulo
Sunmd5: Eliminate some more uses of sizeof(string). Also replace one
use of MAX_KEYS_PER_CRYPT with count - should help with single
crack mode.
Merge Solar's Jumbo-7 changes selectively.
Merge Solar's john-1.7.9.6-selftest-1.diff. This was meant to be
committed to core -> bleeding-jumbo and we should have it in
1.8-unstable-jumbo by now but progress is so slow I (magnum)
 lost faith. Original comments follow:
PoC optimization for NTLMv1. Maintain a table for the third DES block.
Try to speedup cmp_all() by use of a bitmap. This only makes sense
when cmp_all() is called multiple times per crypt_all()'s actual
processing - that is, when cracking two or more C/R pairs at
once. There's a check for that in the code. Moreover, use of
the bitmap is automatically disabled when the C/R pair count
reduces to 1.
Licensing files
Add 'dokuwiki' external mode from
http://www.openwall.com/lists/john-users/2013/06/08/14

Narendra Kangralkar (10):

Make ciphertext parsing more robust in nk format
Make ciphertext parsing more robust in episerver format
Make ciphertext parsing more robust in IKE format
Make ciphertext parsing more robust in vnc format
Make ciphertext parsing more robust in django format
Make ciphertext parsing more robust in pbkdf2-hmac-sha512 format
Make ciphertext parsing more robust in office format
Make ciphertext parsing more robust in oldoffice format
Make ciphertext parsing more robust in zip format
Add support for cracking KDE KWallet files

Vlatko Kosturjak (7):

Fix: handle error handling better. Helps in debugging OpenCL errors.
For example, was getting OpenCL error
(CL_INVALID_PROGRAM_EXECUTABLE) instead of OpenCL error
(CL_INVALID_BUILD_OPTIONS) because program did not handle
previous error and continued to run even if build did not
succeed.
Fix: If Apple is in question, turn off build options as Apple OpenCL
does not like it. At least on Mac Book Pros having ATI Radeon HD
6750M.
Print to stdout without message for consistency
expose markov stats file as command line option
Initial import of ZSH completion for john
make targets for zsh completion
"polishing" changes

bwall (6):
Initial pwsafe optimization
Removed OpenSSL tag from pwsafe_fmt_plug
Updated copyright
Optimized the iterated SHA256 in pwsafe-cuda
Added copyright information
Applied optimizations to pwsafe-opencl and cleaned up copyrights

rofl0r (6):
timer.h: remove unneeded include (sys/timeb.h)
factor out struct tcphdr
Makefile: allow JOHN_LDFLAGS
tcphdr.h: remove unneeded typedef
tcphdr.h: special-case endian.h for mac/bsd
timer.h: re-add sys/timeb.h to windows section

Deepika Dutta (3):

Add bit-sliced MSCHAPv2 implementation by Deepika. This is added as a
new separate format: We now have mschapv2, mschapv2-bs and
 mschapv2-naive.
Add MMX version of DES_bs_crypt_plain().
netntlm bitsliced conversion

Myrice (3):
Bug fixed: xsha512-cuda long password bug
rawsha512/xsha512-cuda change thread/block number for better
performance
Add mask mode

Michael Samuel (2):

Add External:KDEPaste. This takes advantage of CVE-2013-2120 to find
seeds that KDE Paste applet uses to generate passwords.
Add license notice to KDEPaste external mode.

Costin Enache (2):

Fix for clock err in timers.
Silly bug in sunmd5 busted the rounds parsing.

Julien Perrot (2):

add support for sips uri (with explicit port number)
adds support for sips uri (with explicit port number) when converting
input from sipdump

Tavis Ormandy (2):

Avoid useless roti operation for R75, minor syntax cleanup.
manually unroll the cmp_all loop a few times, as gcc refuses to do so.

bartavelle (Simon Marechal) (2):

sunmd5.hack.diff
Updated copyright information

Linus Lüssing (1):

MSCHAPv2: Add support for freeradius-wep-patch challenge/response
format

alexandru (1):
Default value for number of iterations

tomrittervg (1):
fix typo in memory allocation check code

Mat G (1):
New format: Fortigate (FortiOS)

Diogo Ramos (1):

fix opencl kernel build on macosx

Michael Ledford (1):

Adds support for multiple keys existing in an encrypted DMG. When
multiple keys are found it selects the first password key header.
Adds support for DMG sparseimage files. Previously the first chunk
selected could be empty. Now chunk number one is chosen for DMG
sparseimage files and a new heuristic is added to the DMG format
plugin to look for 'Apple'.

Camille Mougey (1):

New formats: KRB5-18 and KRB5-23
The following changes were made between 1.7.9-jumbo-6 and 1.7.9-jumbo-7:

magnum (109):
define REPORT_OPENCL_WARNINGS in bleeding after it was undef'ed in magnum-
jumbo
Version notice for magnum-jumbo
RAR: make self-test comments more clear about what is what
Fix for common_opencl_pbkdf2.c for Mac (uint -> unsigned int)
options.c format list: Workaround for silly Windows bug at column 80
Brainfart in magnum-jumbo version: it's supposed to be 1.7.9-jumbo-
6+unstable, not 1.7.9-jumbo-6+bleeding
Remove the "unreliable, will miss guesses" from rawsha512-opencl and -cuda
after myrice's fixes. They pass the Test Suite.
Fixes for raw-md4, raw-md5, nt2 and raw-sha1 when *not* using reduced binary
size (ie. for bleeding-jumbo)
Drop the "unreliable, will miss guesses" from xsha512-cuda. It now passes the
Test Suite.
opencl_bf_std.c: replace uint to unsigned int
Drop the "unreliable, will miss guesses" from xsha512-opencl. It passes the
Test Suite now.
Add a "Section [%s] not found" before outputting the catch-all --list=?
Merge fix
raw-md5-opencl: Report human readable errors
opencl_mscash2_fmt.c did not declare clear_keys static.
Fix mis-use of CL_DEVICE_MAX_WORK_GROUP_SIZE in a number of OpenCL formats.
Version change for Jumbo-7RC, and undefine REPORT_OPENCL_WARNINGS
Drop a bogus run/pfx2john from the tree
Fix two declaration-after-statements in MPI code Add run/pfx2john
to .gitignore
Fixes after last patch
Make clear_keys in dynamic_fmt.c static
testpara-native test target added
Allocate one more byte for opencl source. Weird errors seen when not.
Use common find_best_workgroup() for a number of formats, fixing various
problems. Alloc/free kernel source memory instead of mem_calloc_tiny(). Make
global_work_size global. Always print diagnostics to stderr.
Fix a (possible) problem introduced in the previous patch
Pick a better fallback for optimal_work_group
raw-md4-opencl added (very near a copy of raw-md5-opencl except kernel)
common find_best_workgroup(): Use proper GWS when calculating LWS
Revert the "shared find_best_workgroup" from opencl_xsha512_fmt.c - it had a
special loop for calulating 10 samples and made worse without it.
Fix for global global_work_size
common-opencl.c: Fix for Apple CPU + do a warm-up run
Fixes for shared find_best_workgroup(): formats must use profilingEvent. Use
a loop targeting half a second. Again, use the shared function for
opencl_xsha512_fmt
Fix LOTS of output in OpenCL formats so they go to stderr (lmgtfy)
Fix a compiler warning
Fix sha1-per-key calculation
opencl_cryptsha512: Errors to stderr, please.
Use Sayantan's bitselect in RAR kernel (unless nvidia)
Add keychain2john to .gitignore
declaration-after-statement in wpapsk.h
Unify case in hmac formats, and set the flag. Also, fix bugs in rawSHA-2
formats, which claimed they unify, but did not.
raw SHA2 formats: accept upper- or lowercase hashes
Fixes after Jim's patch (new-style format interface mixed in)
 Ditch the last pFmt stuff (except in dynamic)
Add "LC_ALL = C" to Makefile, for better compatibility with some non-GNU
systems.
wordlist.c: fix segfault reported in http://www.openwall.com/lists/john-
users/2012/08/01/5
pkzip: Drop a now unused label, muting a compiler warning
Fix the self-test's clear_keys().
mscash formats: Warn when rejecting too long salts
rar_fmt: Move new CMIYC tests to the debug section so they don't skew
benchmarks
Update doc/BUGS and add "unreliable" to pwsafe-cuda
Fix pwsafe-cuda by limiting max length. Also move an endian-swap from
crypt_all() to get_salt() - and same fix for opencl format.
pwsafe-cuda: Bump PLAINTEXT_LENGTH to 32 again, this time with an array size
to fit.
bf-opencl and sha512crypt-opencl: use stderr for diag/error output.
pwsafe-opencl: merge fixes from -cuda version
bf-opencl: Workaround for AMD APP problems running this format on CPU. This
fix does not affect GPU at all.
sha1 OpenCL kernels: Fix compiler warnings (on OSX) and optimise a little
while at it.
Revert a buggy change to ldr_load_pot_line() in 1c637ba that lead to
segfaults. This may mean $HEX$ fixups for dynamic will be broken again, until fixed
some other way.
RAR: Drop the warning about device being CPU. I have seen OpenCL outperform a
CPU-only OMP build by 2.5x.
wordlist.c: We can't free the word buffer memory upon exit from
do_wordlist_crack() because some formats just copy the pointer in set_key() and the
final status line will call get_key(). Problem seen with oracle_fmt.
RAR: Add some early rejects. Huffman table check still TODO
OpenCL platform/device selection from john.conf was not working properly (it
was read too late)
RAR format: Use mem_alloc() and MEM_FREE().
RAR: Finally, quite effective early rejection! Verified with almost 70,000
test files, with no false rejects. And we are rejecting over 96% of the candidates
without resorting to a slow full check (which in turn may reject semi-early,
especially if it's a PPM block)
RAR: Correct benchmark output when running vectorized.
RAR: Smaller salt struct. Minor and benign bug fixed.
RAR bugfix. It's a wonder it worked as good as it did.
RAR debugging
Drop the "MaxOrder < 64" constraint - I have files with MaxOrder 165 that was
not cracked due to the limit. (-m4, -m5, -mct)
Change a bunch of ISO-8859-1 source files to UTF-8
Use LC_ALL=C for plugins' sed incovation in Makefile
Change all non-ascii copyright signs to (c) in source files.
RAR: Use mem_alloc_tiny() instead of mem_alloc()
Accidental spaces in Makefile replaced with TABs (it matters).
RAR: Memory leak fixed. It was introduced with early reject.
Update CREDITS-jumbo and CHANGES-jumbo
RAR: fix a compiler warning on 32-bit
RACF: Fixed a compiler warning and made format struct forward-compatible
VNC: Fixed a compiler warning and made format struct forward-compatible
memory.c: print some clues if alloc() fails. RAR: handle expected bogus alloc
sizes (bug introduced in 8f3e452)
RAR: User-settable max. duration (mitigating kernel exec. timeouts)
Simplify that last patch a little
Fixes for OSX's CUDA and OpenCL targets, and add macosx-x86-64-gpu.
Assorted OpenCL and CUDA fixes needed for OSX. - Always use "static" or
"inline" for non-kernel functions. - Avoid comparing integers of different signs. -
Change %Zd to %zd - Rename all local implementations of memcpy to _memcpy - change
"uint" to "uint32_t"
OpenCL: Do not add -cl-nv-verbose on OSX
MPI fix for loopback mode (cosmetical)
RAR: fixes from magnum-jumbo that should be in -fixes too
Makefile: Fix for linux-x86-sse2 target
OSX performance fixes
gcc 4.7 performance fix
EPIserver: Add proper Unicode support. A bunch of other formats: Unicode
fixes (no problems seen but better safe than sorry).
RAR: Make check_huffman() less greedy (I had a false assertion-fail, for an
invalid data stream)
RAR: Correct BINARY_SIZE (it should be zero!)
Raw-SHA1-ng: Set min_keys_per_crypt to 4 to avoid unnecessary performance hit
in Single mode.
Opportunistic copy of john.bash_completion to any (all) present directories
out of: /etc/bash_completion.d /usr/local/etc/bash_completion.d
/opt/local/etc/bash_completion.d
OpenCL auto-find GPU if no specific device requested. Also, an OSX OpenCL fix
of unknown importance.
OpenCL/CUDA: differentiate gpu_id -> ocl_gpu_id and cuda_gpu_id. This was a
necessity just waiting to happen.
A couple fixes for OSX. After this, ALL formats except some OpenCL ones
builds and works fine under OSX 10.8 - including with HAVE_NSS, HAVE_KRB5 and
HAVE_GMP. The separate make targets vncpcap2john and office2john are tested too and
works fine.
OpenCL tweaks depending on platform. Add a new function
get_platform_vendor_id().
GPU tweaks (cosmetically, if that)
Office 2007 leaked about 1 KB of memory per second
Office: Remove a no-op from DeriveKey() and protect us from segfault in
GeneratePasswordHashUsingSHA1() in case DeriveKey() returned NULL. Are there cases
we don't cover?
Common-opencl.c: It turns out strcasestr() is not standard, so cease and
desist.
pass_gen.pl: Print a notice if we stop due to --count limit.
The "arbitrary characters in rules, using \xnn" introduced by Jim in commit
ad56c30 did not work for preprocessor ranges. This is now fixed and tested.
Fix for wpapsk for big-endian, submitted by Sam Novak.
Drop radmin & pfx formats, and pfx2john. These will be in Jumbo-8.
Added a CHANGES-jumbo.git which is (almost) raw output from "git shortlog",
and placed (the changes part of) Solar's announcement of Jumbo-6 in CHANGES-jumbo
(mostly as a placeholder, it might need editing).
Update CREDITS-jumbo
hmac-sha1 bug fix: Certain optimisations did not take SHA_BUF_SIZ in account.
Very few builds pick the old SHA-1 implementation that uses 320 bytes instead of
the now normal 64 bytes but native OSX llvm now does, so triggered the bug.

JimF (25):
SKEY was lacking fmt_default_clear_keys in it's format struct
pass_gen.pl: Support for sha256crypt and sha512crypt
pass_gen.pl: add XSHA512
Dynamic format bug fixes post Jumbo-6, from bleeding
Fix radmin format under Windows.
GOST: Fixes for clang (bswap stuff)
Dynamic_Diff_Clear_Keys.diff
Fix for dynamic_1003
Jumbo-7RC-JimF-fixes.diff
 JtR-New-Dynamic-35-36-SHA-types.diff
JtR-Dynamic-Algorithm-Changes-35-36-37-fixes.diff
Add dynamic_27/28 to pass_gen.pl
JtR-Dynamic-27-28-fix-non-mmx.diff
JtR-J7-RC-ONLY-Dynamic-changes-NO-sha256.diff
fixes to dynamic split, fake_salt static removal and gost_plug multiple init
allow
fix to dynamic-27/28
Bugfixes to strnzcpy() and strnzcpyn()
All known changes from JimF, along with unification of differences between
core-jumbo-magjumbo-bleeding
dynamic: $HEX$ fixes
Fixed dynamic valid, removing all HEX data prior to validation of hash line
dyna_1008 had to have a couple tests commented out (NULL bytes). epi server
improved. pkzip format, old code left in, just in case, but in a #if 0 block
added some new formats to pass_gen
parser functions that were not properly in code
Added mssql-2012 format
Portability change

Frank Dittrich (23):

Adjust LDFLAGS for clang targets
Make --list= a little bit more user friendly
Bash completion: add support for --list=format-methods[:WHICH]
New --list=help instead of --list=?
Add --list=help:help and improve output in case of wrong method name
Bash completion: add support for --list=option[:value]
pdf2john.c: compare pointer to NULL, not to 0
Make some more --list= commands work despite a broken config
Drop unneeded (and buggy) memsets from KRB4
Add --help option to john (prints usage summary)
Fix dynamic_1010 format name
Fix salt size of dynamic formats in --list=format-details
benchmark-unify: make sure dynamic_NNNN is followed by a colon
john.bash_completion: improve comments
Fix salt size of dynamic formats in --list=format-details
Add OpenSSL version info to --list=build-info
Add FMT_MAIN_VERSION, CHARSET_V and MAX_MKV_* to build-info
New --list=[sections|parameters=SECTION|list-data=SECTION]
Bash completion: convert upper case section names to lower case
calc_stat.c: more verbose stderr output for non-ascii characters
Add [List.External:Filter_Rot13]
Make bash completion work for OS X
Bash completion: work around differences between GNU sed and OS X sed

Claudio André (22):

Fix: use the same tests available in cryptsha512_fmt.c Improvement: create
another test vector.
Improvement: better set_salt and set_key out of bounds management.
Better constants and control on find_best_gws.
Improvement: renamed AMD OpenCL file.
raw-md5-opencl: Do not use byte addressed stores
Improvement: use shared find_best_workgroup. Fix: invalid use of
__OPENCL_VERSION__.
Fix: remove commented code.
Fix: proper usage of profilingEvent in host code.
Improvement: use shared find_best_workgroup. Fix: invalid use of
__OPENCL_VERSION__.
Fix: remove commented code.
 Fix: proper usage of profilingEvent in host code.
Revert: usage of shared find_best_workgroup.
Fix: allow user to run the software (sha512crypt-opencl) without
cl_khr_byte_addressable_store
Fix: wrong if inside get_processor_family (OpenCL) code.
Improvement: create a default/safe kernel. Improvement: allow to select at
runtime the kernel to be executed.
Improvement: create a better way to select default global work size.
Improvement: code cleanup.
Improvement: better parsing of user input.
Improvement: change shared find_best_workgroup to allow cryptsha512-opencl
use it.
Improvement: change shared find_best_workgroup to allow cryptsha512-opencl
use it.
Better 'no_byte_addressable' detection.
Change Format-specific settings names (john.conf)

Dhiru Kholia (20):

Add support for cracking KeePass 2.x files.
Get rid of BOM.
Add support for cracking PKCS #12 files.
Use BIO_write instead of BIO_puts.
Add one more test vector in pfx format.
Fix OMP build.
Fix build.
Add support for cracking RAdmin v2.x hashes.
Add a fast test vector in pfx format.
Integrate pfx2john with john.
Remove my hacks from pwsafe format.
Complete Office format documentation.
Add documentation for pwsafe format.
alignment fix in radmin format.
BONUS: pwsafe format also cracks Password Gorilla databases.
Fix license in MD5 OpenCL kernel.
Update 1.7.9-jumbo-6-fixes
Update 1.7.9-jumbo-6-fixes
Fix build.
Drop "inline" from rar malloc stuff. It was probably done wrong.

Solar (9):
hccap2john fixes
RAdmin, SIP speedup
Optimized mscash2-opencl
Use "unsigned short" rather than simple "short" for single crack mode's
buffered key offsets.
We use "short" for buffered key indices and "unsigned short" for buffered key
offsets - make sure these don't overflow.
john-sha1-r3-bitselect.diff
wordlist.c truncation bugfix (from contest edition)
pwsafe-cuda: Bugfix (w[14] init) and optimisations
Final Jumbo-7 changes.

Lukas Odzioba (5):

Fix a multitude of declaration-after-statement warnings.
cudasha256speedpatchv3.diff
wpapsk-valid_fix.diff
Fixes for CUDA mscash formats.
opencl_cryptsha512_fmt.c: Change type of large constants
Myrice (4):
Bug fixed: xsha512 and raw-sha512 opencl/cuda IRL
Bug fixed: xsha512-cuda long password bug
xsha512-cuda: low min_keys_per_crypt and dynamic block number
Fixed xsha512-opencl, raw-sha512-opencl/cuda long time run. Extend password
length to 20 for these formats

Sayantan Datta (4):

mscash2-opencl:bitselect optimization
mscash2-opencl: automated bitselect selection
bf-opencl:simpler kernel code
mscash2-opencl:code cleanup

Knug Carl Gustf (2):

Speed up RACF encryption with a pre-computed lookup table
Speed up VNC encryption by pre-computing DES_set_odd_parity()

Vlatko Kosturjak (2):

Fix: handle error handling better. Helps in debugging OpenCL errors. For
example, was getting OpenCL error (CL_INVALID_PROGRAM_EXECUTABLE) instead of OpenCL
error (CL_INVALID_BUILD_OPTIONS) because program did not handle previous error and
continued to run even if build did not succeed.
Fix: If Apple is in question, turn off build options as Apple OpenCL does not
like it. At least on Mac Book Pros having ATI Radeon HD 6750M.

Tavis Ormandy (2):

test conservative storage of binary.
try redundant binary format.

Pavel Semjanov (1):

pkzip: Bugfix for check_inflate_CODE1()

Elijah [W&P] (1):

Fix CUDA targets for (future) Windows builds.

Erik Winkler (1):

macosx-x86-64-cuda build target added

The following changes were made between 1.7.9-jumbo-5 and 1.7.9-jumbo-6:

magnum (334):
This patch reverts part of the NT format to using an incorrect and no longer
needed length check. That beats me, but it will do until the reason is figured out.
Changes Makefile test targets for Test Suite 1.10
Correct truncation for NT format. This time with no performance loss
External filter now truncates at format's max length. Turn off wordlist
memory buffer if an external filter is used (a modification would trash the buffer
if a word got longer)
Reduced verbosity for normal test targets and added verbose ones
Replace length checks in nt2 too. Strangely enough it is faster with them.
XSHA intrinsics. Does not support OMP.
XSHA: added "option" to revert to OpenSSL for OMP builds. Also fixes 32-bit
MMX builds.
XSHA: Added support for OMP with intrinsics
pass_gen.pl added to run/ directory. This has only been available in separate
patches (like the test suite creation scripts) until now, and that version was not
up to date. This is a gem for testing stuff.
 XSHA: More optimised salt handling, better OMP scaling
mysqlSHA1 32-bit set_key() for SSE2
Bug fixes to lotus5 (OMP builds) and both mssql formats (MMX builds)
mysql-sha1 set_key tweaks
Salted-sha1: performance tweaks
Raw-MD5-unicode made thick again, using much faster code (2x dynamic)
raw-md5u final fixes (all linux builds pass test suite)
MD5_std vs intrinsics / arch.h PARA fixes. Never disable SSE for MD5, revert
to 4x instead.
mssql05 hash functions size 5 & 6
Oracle11: Added intrinsics
Oracle11 optimisations
drop cmpt_cp.pl and mkdumb16.pl (will be placed in the separate patch)
corrected a typo in the PARA gcc version tests
Adapt trip_fmt to Jumbo
Added MMX/SSE/intrinsics to sapB as well as lots of optimisation.
Fixed bug in unicode.c where specifically giving --enc=raw did not work as
expected
sapB: Various fixes. Max salt length 12 (changed self-test hash). Changed
cur_salt to struct pointer. Non-SSE builds fixes. Replace 8-bit characters with
'^'.
sapG: Added MMX/SSE/intrinsics. Decreased salt length to 36 (12 x UTF-8)
(changed self-test hash).
sapB and sapG: re-worked valid() for variable salt length. We still support
the old format (padded with spaces to 40 characters) but also non-padded.
make testpara target, for determining best SSE_PARA values
hmac-MD5 optimisations. 32-bit set_key().
RawSHA1: Optimised set_key() for some 17% speedup.
PARA tweaks from Solar's tests
testpara: decrease threshold for accepting a higher value from 5% to 1%.
sapB bugfix for 8-bit salts
sapG minor fix in get_salt() (only affected over-long salts)
sap_prepare.pl renamed to sap2john.pl and changed not to pad salt
sapB: Various bugfixes. Use enc_strupper() for proper uc of UTF-8, but init
the Unicode system in a way so it does not uppercase double-s
sapG: Various bugfixes. New cleaning mechanism. Seems to work fine now.
Intermediate key will never be longer than 192 bytes, decreased max number of limbs
to three. Dropped the internal UTF-8 conversion. It was never appropriate. sapG now
behaves like any other 8-bit format. If you want UTF-8, feed UTF-8.
sapG: Optimisations, 30% boost (now 2x compared to Jumbo-5)
HDAA: OMP support
sapG Bugfixes. Insufficient key cleaning in rare cases plus a really hideous
bug in "magic array" enumeration.
sapB: key cleaning bugfix. This was a mean one. Weird algo.
HDAA: get rid of a compiler warning
sapB & sapG minor optimisations
sapB & sapG: trim trailing spaces from (truncated) plaintext. Raise sapG
plaintext length (for UTF-8 encodings of long passwords) Warn if --encoding=utf8 is
used/not used depending on format: these formats are well defined in that aspect.
DragonFly BSD crypt-SHA2 formats added. Should work for 32-bit "versions" of
the bugs in them.
OMP support for raw-sha2 and DragonFly sha2 formats
DragonFly, cosmetic
Crypt-SHA2 formats using OpenSSL and with OMP support
Copyright and license statements overhaul
Fixes in a number of formats after failing new self-tests. A few of them were
probably real bugs.
Revert the not strictly needed fixes from last commit. Add a couple other
fixes.
 Deprecated raw-md5 thin (moved to unused) in favour of thick version
IPB2: fix in binary(), it was reading twice the size of ciphertext and
probably overwriting something.
Change #include "omp.h" to #include <omp.h> in a number of formats.
SHA-crypt-256/512: Change benchmark to only output raw figure since there is
no difference between one/many salts. Also output "rounds=5000".
Drop x86-ssei.h in favor of x86-sse.h, add tests that disable intrinsics for
gcc older than 3.4.6 (the oldest we know work at all) and fallback to asm. This
means the sse2/sse2i targets now work similar to 64/64i for newer versions of gcc,
that is, sse2i just means "precompiled intrinsics".
MSCHAPv2 salt() alignment
DragonFly SHA formats: Add 64-bit versions (in same format files: dual
structs)
loader.c: fix for --user=<uid> when loading pwdump files
bugfix for previous patch
loader.c: restore the defined-out block for DES bare hashes with whitespace
salt.
DragonFly formats: cosmetics
Make all sse2i targets use x86-sse.h. Add new test target linux-x86-64-32-
sse2asm.
sapG fix for long candidates (over 64 octets)
Jim's new SHA-1 intrinsics. Uses 16x4 buffer instead of 80x4. Improved "make
testpara32" to compare asm figures too. Updated SHA1_SSE_PARA for 16x4 code. New
pre-compiled intrinsics.
Dragonfly formats: Type-cast void pointer arithmetic
Adds --max-run-time=N option, for graceful exit after N seconds of cracking.
Makes --max-run-time disregard init & loading times
Correct a bug in the "double10" external mode
pass_gen.pl: Show usage blob when no options given
pass_gen.pl: Add salted sha-1 (of any length up to 16) and fix bugs in base64
encoding
Salted SHA-1: Support any salt size
Change MPI defaults to be more stable (no MPI_Barrier nor MPI_Abort)
sapG: add const for safety/optimisations
More randomness for HDAA in pass_gen.pl (lengths etc)
md5-mmx.S: new interface needed for multi-limb (>55 bytes) crypts (for HDAA)
HDAA intrinsics, 3x original speed
Wordlist.c hack for reading candidates from a pot file: 1. if --wordlist
given without a name, use default pot file 2. if wordlist name (after 1) ends
in .pot, parse it as such
Added hmac-sha1 and hmac-sha256 formats. Added wikipedia self-tests to sha-
md5
Added hmac-sha224/384/512 (and made hmac-sha256 a non-plug like these)
Changed benchmark type to raw for all HMAC formats
SAP B format: Optimisations
sapG: made "theMagicArray" static const. This alone made a 4% boost
sapB regression fix
Wordlist mode: Fix dupe suppression for non-buffered mode. Make the pot file
parsing mode aware of any custom field separator.
Replace the muting of "#!comment" lines when in memory buffer mode
sapB bugfix for generic builds
wordlist.c MPI fix for dupe suppression in non-buffered mode. We must
incrementent the line counter when skipping, because the different nodes will have
different views of what is a dupe.
sse2i_winfix.pl modified so it can handle any .S file
Copyright/license statements
More copyright/license blurbs
IPB2: OMP & intrinsics support, over 3x speed
IBP2: 32-bit mmx/sse fixes
 hmacSHA1: Support key lengths >64
All hmac formats: Support key lengths >64
All hmac-SHA formats: fix for salt containing '#' and self-tests for keys
larger than pad size (n/a for sha-384/512 as of now due to lack of support from
John itself for lengths gt 125)
New format: Drupal7 (non-plug, OpenSSL >= 0.9.8, OMP)
Copyright/license stuff
Add missing opencl-mysql declarations to john.c
Windows fixes for new sse2/sse2i scheme
Change Unicode tables to const - might help the optimiser
Unified make target linux-x86-64-gpu including both CUDA and OpenCL. Also
adds intrinsics to the 32-bit CUDA and OpenCL targets.
change -lcrypto -lssl to -lssl -lcrypto (helps static linking)
options.c: Sort formats list
New para-best.pl for "make testpara"
options.c: enable format name printing all the way to 80th column
options.c fixed erroneus output of \0
NT-OpenCL: Add FMT_UNICODE flag
loader.c: Do read gecos and homedir from pwdump-style input.
http://www.openwall.com/lists/john-users/2012/02/22/1
Makefile typo (cosmetic)
Manual sync with Samuele's tree's b890a51
Support specifying OpenCL platform on command line
Add a trivial platform/device lister (to unused/ for now). The plan is to
make JtR capable of outputting this list.
Mute compilation log if it's no more than a line-feed
Correct BENCHMARK_LENGTH for raw-sha1-opencl
Rename sha-crypt-256/512 to cryptsha256/512
OpenCL: rename -gpu to -device and support -platform=LIST (or -device=LIST)
Drop the separate listocldev.c
Auto memory size suffixes when listing devices
rar_fmt: 15% speedup if ARCH_LITTLE_ENDIAN && ARCH_ALLOWS_UNALIGNED. This
version can successfully crack non-hp archives with method "store" ("m0b") without
spawning unrar. Also fixed a bad memory leak. rar2john: Pick best candidate from
RAR file. Prefer stored files over compressed, and prefer smaller files over
larger.
Fix for gpu builds imlicitly using pre-built icc intrinsics
RAR format: Proper support for -p mode. OMP Support and optimisations.
RAR: Do not prefer zero-byte files. Use basename of filename for user.
Improve error output at salt loading. Clean up rar2john variable definitions.
rar2john: Handle Unicode file names. Less output unless -DDEBUG.
Fix for Unicode filenames. We need to zero the whole buffer between files.
Proof of concept patch for OpenCL options in john.conf
Make the RACF format a plugin
Fix for -subformat=list. The config file was not yet read so we could not get
info for user formats.
SIPdump added. Needs libpcap-dev so for now it's a separate make target "make
SIPdump"
Add LDFLAGS to SIPdump build target
Cosmetic changes to cryptsha256/512 (CPU)
ssha-opencl: If KPC was requested, use it when enumerating LWS
Revert "ssha-opencl: If KPC was requested, use it when enumerating LWS". This
did not work well.
Fix for cryptsha512-opencl in -gpu and OSX -opencl targets
Revert enabling of OMP in Makefile that was introduced in 0763f0d
Fix for OpenCL/CUDA targets w/ pre-built intrinsics
Fixed over-allocation in all OpenCL SHA-1 kernels. They used 320 bytes per
key buffer instead of just 64.
New pre-built intrinsics for latest sse-intrinsics.c icc version 12.1.0
 Add Local Memory Type, LE/BE and ECC info to OpenCL device list.
ssha-opencl: Move salt and datai buffers preparation away from crypt_all()
just for good measure. There might be a miniscule speedup for single-salt.
common-opencl.c: Add -DDEVICE_IS_CPU to the build options when applicable,
for self-adjusting kernels.
mozilla2john: Error (and usage) to stderr Makefile: revert OMPFLAGS (please
stop committing that!)
Mozilla_fmt bugfix: globalSalt array was smaller than needed, causing
segfaults. This should be forwarded to the FireMasterLinux project.
Mozilla_fmt: corrected updating of cracked[] array. Also set BENCHMARK_LENGTH
(possibly temporarily) for seeing how the upcoming get/set_salt() fix will affect
performance.
Enable atexit() in opencl_cryptmd5_fmt.c and opencl_phpass_fmt.c just to get
rid of compiler warnings
RAR format: OpenCL support added (in the same one format). Support for AES-NI
instructions by using OpenSSL EVP instead of low-level calls. Experimental support
for reverting to CPU in Single mode (when doing a lot fewer than max keys per crypt
in a batch). Currently #defined out. Added inline storing of RAR data in input file
when files are small enough. Added lots of self-tests. Default to use same length
tests as cRARk. Changed format tag from $rar3$ to $RAR3$ to enforce re-running
rar2john. Moved slow code from set_salt() to get_salt() and keep all allocations
during the whole run.
Set OS_TIMER to 0 in arch.h if AMD APP SDK in installed and building OpenCL,
or if mingw is used to build 64-bit Windows build.
Silly me forgot an #endif in x86-64.h
RAR: Mute warning about AES-NI engine (it may be false) fix comments
RAR: Re-introduce find_best_kpc() - only used when specifically saying KPC=0
sapB optimisations and sap2john overhaul
RACF: trivial fix to mute a compiler warning for non-OMP builds
RAR fixes for OpenCL. Better error handling (especially if -DDEBUG). Now
works fine on AMD Cedar (previously segfaulted).
RAR: minor optimisations. Enable fallback to CPU in Single mode (it's safe
now)
A couple more .gitignore entries
Make hmacMD5 a non-plugin in order to force it to load after mscash2. Now
only hmacSHA1 was a plugin of all hmac's so I threw it in as well.
Create doc/CHANGES-jumbo, generated from commit descriptions
Remove obsolete file opencl/PBKDF2.cl
Fix a typo in common-opencl.c
Cryptsha512-opencl Makefile fix for 32-bit build targets
RAR: various optimisations.
RAR (OpenCL): drop all use of byte_addressable_store. This also avoids lots
of endian swapping. Added four versions of RawPsw, one for each aligment, and copy
32-bits at a time whenever possible.
RAR: Tweak auto-tuning. Unroll first loop. Do endian swapping only once when
preparing RawPsw.
RAR: KPC enumeration tweaks
RAR: Made us of local memory optional (default disabled). Re-added
find_best_workgroup() and tweaked adjusting to device. Found a way to reduce
register use by 16, all else equal.
RAR: Optimisations for GTX580. Now 4400 c/s. Major overhaul of auto tuning.
Office format: Unicode support
RAR: Better granularity w/ KPC=0 (vs auto-homing). Auto-define NVIDIA from
cl_nv_pragma_unroll for architecture tweaks. Bump plaintext length for better
coalescion. Init lengths to 6 so the self-tests don't end up wildly diverging
threads (when it just sets a few keys out of thousands).
RAR: Fix LWS at 64, there is no point in evaluating it.
RAR: Decrease register use a lot, only one RawPsw buffer. KPC tweaks. Get rid
of the switch (for AMD) in sha_final() using a new bit flip macro. Optional fixed
length in kernel. Approaching 10,000 c/s on HD 7950
RAR: Minor optimisations verified on GTX580. Self-test length fixes. Unroll
inner loop by four for fixed-length kernels, dropping a switch statement.
RAR: Disable the 4x inner loop unrolling for AMD, it was detrimental
RAR: Use vectorized memcpy if AMD
RAR: Drop impossible conditionals for unrolled nvidia code.
RAR: Vectorize output copy for AMD. This too made an unintuitive boost.
Added hccap2john to .gitignore
common-opencl.c: Add -cl-nv-verbose if device is nvidia
Include a vectorized kernel in same file as non-vectorized. It is not used
yet. Add an opencl-rar.h file.
RAR: Disable fixed_len that was accidentally active
RAR: Remove a debugging notice unless actually debugging
Added a note about bash completion in the "Notes to packagers" in params.h
Fix a typo in bash completion
Fixes for bash completion's Makefile target
More fixes for bash completion make target
common-opencl.[ch]: Added some nvidia-specific stuff. Also ran indent on it
because it was a mess.
New (hidden) options: --list=[inc-modes|rules|externals|ext-filters|ext-
modes] to be used by bash completion as well as GUI's and other things.
New hidden option: --list=ext-filters-only. Unlike --list=ext-filters, this
does not include modes that also has generate()
New hidden options --list=build-info and --list=<section> Also includes
Solar's patch for compiler.c
--list: Missing CR for icc version output. Removed FLG_STDOUT.
Version string now reflecting magnum-jumbo (1.7.9-jumbo-5+unstable)
Add -list=hidden-options
bash completion now including incremental/external modes and rules
Tab vs space in doc/RULES
RAR: Separate FIXED_LEN from vectorizing (the latter needs the former but on
nvidias we may only want the former).
RAR: Fix EVP and unrar memory leaks. Early reject if rar_decode_number would
return > MaxNum. Experimental fuzzy early rejection found by Milen.
RAR: reverted experimental early reject, it produced false negatives. Added
lots of comments to unrar code, from original sources.
RAR: Added a reject test from Milen
Move some --list= options handling earlier, so we can show it even if we lack
a config file.
RAR: Another early reject found by Milen. This was already in place but we
might bail out just a little sooner with this test.
SHA0: use the basic function names since they're static anyway. This format
is good as a template for unsalted formats and sha1_gen_fmt_plug.c is good for
salted ones, as they don't contain any SSE2 or such complications.
Mute NSS/pkg-config warnings for LDFLAGS
Updated doc/CHANGES-jumbo and started to hand-edit it. More needed.
Updated doc/CHANGES-jumbo
Add run/opencl_bf_std.h to .gitignore
mozilla format: support new get_source() core
NT2: Simpler prepare() without mallocs, as Jim did to NT_fmt
config.c: fix for not breaking generic builds
get_source(): self test
Mute a compiler warning when building RAR in 32-bit
nt2 and raw-md5: Add get_source() functions
Raw-sha1: add get_source()
Make the last patch work on non-Windows
Revert get_source() function for now (it's still in bleeding-jumbo)
Move bash completion script to run directory in case a packager forgets about
it
 Fix a couple of compiler warnings
Revert hash and buffer sizes for unique, accidentally committed by Jim in
514c059
mscash: bugfix for codepage version of get_salt()
Alignment fixes in mysqlSHA1 and rawSHA0 binary()
Revert "Fix: improve build on multi-GPU environment."
Add new john.conf options subsections Markov, MPI and Jumbo and add a new
config item CrackStatus under Options:Jumbo. Note: If CrackStatus=Y in john.conf,
the --crack-status option disables it.
Drop the "Jumbo" subsection of john.conf, leave those items at the top
[Options] section.
Add --list=subformats and deprecate --subformat=list
List --subformat as a hidden option now (for --format=crypt benchmark)
Mute a compiler warning for raw2dyna.c
Make raw-sha (sha0) a non-plug so it always gets lower precedence than sha1
RAR: change KPC to GWS
Rest of OpenCl formats: Change KPC to GWS in john.conf and environment
variables
john.conf typos in comments (external:double)
c3_fmt.c: fix typos, raw-sha vs cryptsha
README.mpi minor updates
Updated doc/CHANGES-jumbo. We now have 516 commits since Jumbo-5 but this is
reduced to currently 172 lines in this file.
A couple minor changes to CHANGES-jumbo.
Source $JOHN/john.local.conf by default. This file is never overwritten but
will be created empty when building, if it does not exist. It must *not* be
included in any release tarball.
Create a rule section "Extra" for the previously commented-out rules for
Single mode. Also, add new rules "Single-Extra" (consisting of Single plus Extra)
and "Jumbo" (consisting of all rules we have, including NT). I preserved the name
"All" for local use, thus using "Jumbo" instead.
External:Keyboard: Fixes for 8-bit characters vs signed char constants.
http://www.openwall.com/lists/john-users/2012/02/26/2
Makefile: Never overwrite john.local.conf when building
Add --list=[encodings|opencl-devices]. Deprecate --encodings=LIST (but it's
still supported). We will later support --list=cuda-devices too. Drop --device=list
and --platforms=list (not supported from now on). For the catch-all of --
list=<subsection>, add a header "Subsections of...".
RAR: Support files larger than 2 GB.
Add --list=formats, listing all formats in the order they are registered.
Update doc/OPTIONS and doc/README.opencl
Some more .gitignore entries
wordlist.c: True dupe suppression for buffer mode, code based on unique.c
CUDA device lister: Handle error conditions.
john.c: Alter output from --list=? a little, working around a problem with
how bash completion was parsing it. Also add a function prototype for
cuda_device_list().
CUDA: Add some info to device list. Fix compiler warnings.
Added a stub doc/CREDITS-jumbo based on git author data, and manually edited.
Updated doc/OPTIONS and doc/CHANGES-jumbo
mscash2 sizeof() bug fix after the Sparc fixes.
wordlist.c: Fix a compiler warning on 32-bit
Revert "bf-opencl: performance and stability update". It broke building.
Fix line endings in a couple of files that were in DOS format.
mscash2-opencl: Fix a typo that merely led to a compiler warning
Fix line endings in opencl_mscash2_fmt.c again
mscash2-opencl: Fix a typo that merely led to a compiler warning
RAR_fmt.c & common-opencl.c: Fix compiler warnings on 32-bit builds
NT and mscash1: Fix strict aliasing breakage from Sparc fixes
 New option: --log-stderr
Closed-loop mode and dupe suppression fixes
Made Taviso's SSSE4 format a non-plug, depending on SSSE4
Add an experimental linux-x86-64-native build target
Fix compiler warnings in Tavis' SHA-1 format
When using --log-stderr, don't produce a log file
taviso_fmt.c: Replace memrchr with strrchr. Add $dynamic_12$ tag like other
raw SHA1 formats use. The format announced FMT_SPLIT_UNIFIES_CASE which was not
correct until now.
Updates to CHANGES-jumbo and CREDITS-jumbo
Add 32-bit "native" build targets
tavis_fmt.c: Fix a memory leak.
taviso_fmt: Fix for strrchr vs full-length keys.
Fix incorrect $dynamic_12$ tag in raw sha1 formats to $dynamic_26$
Do NOT delete ../run/john.local.conf on "make clean"
Fix 64-bit native target (we need -march=native in ASFLAGS too)
Rename rawSHA1_LinkedIn_fmt_plug.c to rawSHA1_linkedIn_fmt_plug.c so it
registers *after* the normal raw-sha1 format.
Another fix for john.local.conf
Fixes for -native build target
Updated CHANGES-jumbo
-march=native implies -mtune=native
raw-sha1_li format: Fix bug that would give false negatives from cmp_all()
Reduced binary for raw-md4, raw-md5, raw-sha1 and NT2. This decreases memory
usage with 3 bytes per loaded hash for NT/MD4/MD5 and 4 bytes for SHA1 and should
help keeping good stuff in cache.
Reduced binary size #ifdefs
XSHA_fmt_plug.c: Fix bug introduced by Sparc fixes
Closed-loop mode: Print name of pot file
Fix mySQLSHA1 and sha1gen bugs from Sparc fixes. All should be OK now.
Format/algo renaming of some of Lukas' formats that were missed due to even
stranger use of the macros
wordlist.c: Drop linefeeds in log_event() calls
Permit smaller hash size for wordlist dupe suppression, and prefix some more
log output with "- "
Drop email addresses from CREDITS-jumbo
RACF binary() alignment fix
Alignment fix in raw-sha1-linkedin
New (hidden) option --length=N for reducing plaintext length of a format,
mostly for testing purposes.
Add new options --loopback[=FILE] and --dupe-suppression (the former implies
the latter) and revert some stuff for backwards compatibility. NOTE: I had to free
up some option flags that was not really needed. But we are now OUT OF them!
Requested CREDITS-jumbo fix
Reflect new format names in CHANGES-jumbo
Move --field-separator-char to hidden option
Make GPU builds (for Linux 64-bit) -march=native
Make --log-stderr a hidden option
sha1_gen_fmt alignment fix (just to be sure)
phpass-cuda was inadvertently named phpass-opencl
GOST rename from "GOST 3411" to "GOST R 34.11-94"
Remove two unused variables from opencl_bf_std.c
Fix a (clang) compiler warning in MYSQL_fast_fmt_plug.c
Move MYSQL_fmt_plug.c to unused/, rename MYSQL_fast_fmt_plug.c to
mysql_fmt_plug.c and change its label to just mysql
CHANGES-jumbo updates. Hard to keep terse, we could drop more
CHANGES-jumbo don't repeat last names
Revert "Revert to older (working) phpass-opencl for now". This code was no
better.
 Revert to a non-vectorized phpass OpenCL format, hand-edited to reflect new
names/labels/etc and shared opencl_find_best_workgroup()
Set FMT_UNICODE for all mscash GPU formats
Move --fix-state-delay=N to "hidden" options. Does not seem to affect bash
completion.
Revise formats.h descriptions of FMT_UNICODE and FMT_UTF8
Add "unreliable" to mscash-cuda formats
64-bit GPU targets Makefile fix: -march=native vs. sse-intrinsics.o
--list=format-details: use hexadecimal output for the FMT_flags
Do not "advertise" linux-x86-native as the best target
Changed version to 1.7.9-jumbo-6-RC0
Fixes for --max-run-time
SSH: Fix a complaint about unused variable
Fix a cosmetical bug if you abort during loading, and loading is less than
one second.
RAR bugfix (though it would never show up)
Avoid a bunch of compiler warnings for keepass2john.c
Clean up a bunch of Dhiru's scattered variable declarations
Fix for fake_salts.c that regains 52MB of BSS
Fixed another bunch of Dhiru's scattered variable declarations, for the
HAVE_NSS case.
Add office2john and vncpcap2john to "make clean"
Remove UTF-8 BOM from keepass2john.c
clang targets: Add -Wall, and for debug targets also -Wdeclaration-after-
statement
Mute a couple of benign compiler warnings
CHANGES-jumbo update
Add a git .mailmap file to (cosmetically) unify the commit history
Do not define REPORT_OPENCL_WARNINGS for release. We'll still se build errors
but the output will be cleaner when all is OK.
Fix a -Wdeclaration-after-statement for MPI
Fixed my share of GPU formats' declarations not at block start.
Add another test vector to hmac-sha1
Make episerver, keepass, pwsafe non-plugs.
Added doc/BUGS which lists the three formats that does not work with big-
endian, as well as the problematic mscash CUDA formats.

Dhiru Kholia (126):

ssh2john: Detect if private keyfile has no password
Support rar's -p mode by spawning external unrar process.
rar2john: fix failure for RAR files with comment present.
RACF password cracker patch for JtR.
Add OMP support in RACF cracker patch.
Add OMP code to RACF based on magnum's changes to RAR format. Fix non-OMP
build of RACF.
Add .gitignore file.
Integrate SIPcrack 0.4 into JtR. SIPdump.c is still to be integrated.
Add CRACF.TXT (generated by CRACF program). Add cracf2john utility to convert
input CRACF.TXT file into JtR compatible format. racf2john is currently TODO.
Minor optimization in length calculation.
Add OMP support for sip format.
Remove a memory leak in RACF format.
Fix memory leaks in SIP format.
Get rid of Valgrind warnings from RACF format code.
Add one more SIP hash.
Integrate VNCcrack (http://www.randombit.net/code/vnccrack/) into JtR. JtR
can now crack RFB Protocol Version 3.8 authentication.
Minor fixes for RACF format.
Add keepass2john utlity for processing input KeePass 2.x database files into
a format suitable for use with JtR. KeePass format is in works.
Finally, the racf2john utility is here. Huge thanks to Nigel Pentland for
providing sample code.
Cleanup SSH format.
Fix thread safety issues in SSH format.
Make some functions static. Add a userid length limit check in racf2john.
Symlink racf2john to john.
Remove some minor compiler warnings.
Alternate and FAST method to ensure thread safety in SSH format.
Minor fixes for SSH format.
Fix build when using OpenSSL 0.9.7
Re-add OpenSSL version check for SSH format.
Fix compiler warning in VNC format.
Increase SALT_SIZE in RACF format.
Add VNC password decoder utility in unused folder. Works for TightVNC and
RealVNC products.
Add VNC passwd file support in vnc-decoder.c.
Add group information to racfdump. Increase SALT_SIZE in RACF format.
Add a patch for SSH format by Solar.
Improved vncpcap2john. Works for RFB 3.7 and 3.8 protocol versions. Works
with TightVNC and RealVNC servers. 3.3 RFB support is TODO.
Add make target for vncpcap2john.cpp
Use BSD version of tcphdr (patch by Erik).
Fast cracker for Mozilla Firefox, Thunderbird and SeaMonkey master passwords.
Fix a compiler warning in mozilla format. Add sample key3.db files.
Restore original DES code. Begin preparation for OMP support.
Fix worst of memory leaks (one leak in lowpbe.c:186 is still present). Make
code OMP friendly.
Fix remaining memory leaks by avoiding dynamic allocation.
OMP support for mozilla format. ~1.5x faster using 3 cores.
Remove requirement of cert8.db files for "no master password" check in
mozilla2john. Remove "no master password" check from mozilla_fmt.
Implement support for having test vectors and benchmarking in Mozilla format.
Simplify mozilla format by avoiding dynamic memory allocation.
Add support for cracking Office 2007 documents.
Fix 32-bit SSE2 build.
Add patch by Solar. Increase PLAINTEXT_LENGTH to 32.
Remove debugging code.
At least detect Office 2010 files.
Add support for Office 2010 files in office2john.
Fix "Too many open files" error by using proper free function.
Add support for *new* EPiServer's default hashes.
Minor cosmetic fixes.
Add few more self-tests in episerver format.
Implement proper set_salt and get_salt functions in episerver format.
Add speculative support for salted SHA256 hashes in episerver format.
Fix a silly typo.
Office 2010 support is now complete.
Implement proper get_salt and set_salt functions in office format.
Add OMP support in office format.
Add another test vector from RFC3211 (taken from extractkeychain-0.1's
sources).
Implement proper get_salt and set_salt functions in RACF format.
Implement keychain2john program.
Implement "fast" Mac OS X Keychain cracker with OMP support.
Wrong hacky fix for keychain format.
Implement proper get_salt and set_salt functions in pdf format.
Implement proper get_salt and set_salt functions in VNC format. Get rid of
unused version property.
 Ignore 2 more files.
Fix multi-salt performance in pdf format.
Cleanups and reduce global data in pdf format.
Keychain format is now FMT_NOT_EXACT.
Get rid of compiler warnings in Mozilla format.
"Easy fix" for compilation warnings on Lion.
Use CommonCrypto on OS X instead of OpenSSL for EpiServer format.
Use CommonCrypto on OS X instead of OpenSSL for RAR format.
Use CommonCrypto on OS X instead of OpenSSL for SybaseASE format.
Use CommonCrypto on OS X instead of OpenSSL for hmailserver format.
Use CommonCrypto on OS X instead of OpenSSL for raw-sha512 format.
Use CommonCrypto on OS X instead of OpenSSL.
Use CommonCrypto on OS X instead of OpenSSL in drupal7 format.
Use CommonCrypto on OS X instead of OpenSSL in cryptsha512 format.
Use CommonCrypto only on Lion.
Fix (hopefully) compilation problem on Snow Leopard.
Use CommonCrypto only on Lion.
Avoid warnings when nss is missing or pkg-config is missing.
Add help file for Mozilla format.
Fix help file.
Minor fixes for mozilla2john utility.
Fix size to match SHA1 requirements.
Add some sample Mozilla "non-hashes".
Implement proper get_salt and set_salt functions in Mozilla format.
Add support for cracking Password Safe files.
Implement fixes suggested by Solar. Integrate pwsafe2john with john.
Add SHA-256 OpenCL kernel.
Fix a crash in ssh2john.
Add another test vector in SSH format.
Implement changes suggested by Solar.
Add support for WoltLab Burning Board 3 hashes.
Add support for Django 1.4 hashes.
Remove executable flag.
Add support for GOST 3411. Authors: Jim, Sergey and Dhiru.
Add support for cracking ODF files.
Add test-case with duplicate salt.
Remove wrong usage of FMT_NOT_EXACT from ODF format.
Use proper license in mozilla format.
Remove my hacks from wbb3 format.
Remove my hacks from episerver format.
Remove my hacks from RACF format.
Fix license in mozilla format.
Fix RACF breakage.
Drop cracked array approach from django format.
Add more test vectors in vnc format.
Add more test vectors in django format.
Fix 32-bit episerver format builds.
alignment and size fixes for episerver format.
alignment fix for django format.
alignment fix for wbb3 format.
Add three more make targets. linux-x86-clang-debug is super good for
debugging.
Add support for KeePass 1.x files.
Fix salt size in SSH format.
Add support for cracking KeePass 1.x files.
Integrate keepass2john program with john.
Add more test vectors to KeePass format.
Fix warnings exposed by -Wdeclaration-after-statement flag.
Frank Dittrich (64):
bash-completion for JtR (to install: "sudo make bash-completion")
bash completion updates
Update and licensing statements for bash completion
bash completion updates http://www.openwall.com/lists/john-dev/2012/05/12/4
bash completion updates
bash completion update http://www.openwall.com/lists/john-users/2012/05/21/1
bash completion update http://www.openwall.com/lists/john-dev/2012/05/23/1
bash completion update http://www.openwall.com/lists/john-dev/2012/05/24/1
bash completion http://www.openwall.com/lists/john-dev/2012/05/25/3
bash completion update http://www.openwall.com/lists/john-dev/2012/05/28/9
bash completion update
bash completion update and documentation.
bash completion update http://www.openwall.com/lists/john-dev/2012/06/06/32
c3_fmt.c: improve --subformat= handling
bash completion: a few more improvements
Fix bash completion for unique
Adjust bash completion (--list=encodings and --list=opencl-devices)
Fix bash completion for input and output redirection (<, >, 2>...)
Move markov option handling to mkv.c (do_markov_crack)
use [Markov:Default] instead of [Options:Markov] for --markov
bash completion: avoid default completion for --opt=
Markov mode: move parameter validation into new function get_markov_options
Additionaly, the logic has been changed so that the max. values for level and
length are adjusted prior to checking min. values, to avoid adjusting min. value to
a max. value which is changed afterwards:
genmkvpwd should use MAX_MKV_LVL
Markov: different error messages for missing [Markov:Default] section and for
missing Statsfile definition in an existing [Markov:Default] section
Markov: improved logging
Improve error message for invalid incremental mode name
Implement --markov=MODE[:<options>]
Markov: Fix an old typo in an error message
Bash completion: avoid completion for --l= if --l= is ambiguous
Add --list=format-details
Change sequence of columns for --list=format-details
Add john.conf comment about [Options] section
Adjust copyrigth in usage output (1996-2012)
Fix --format=crypt --subformat=TYPE names in doc/OPTIONS
bash completion fixes for --loopback
avoid [ and ] in format names
Makefile: add comments for *-clang-debug targets
benchmark-unify: a Perl script to convert benchmark output for relbench
Several enhancements for relbench
--list=format-details: add number of test cases to output
Fix gcc warnings in --list=format-details code
relbench: important fix for duplicate format names in file1
relbench: update copyright, and change revision from 4 to 4.1
Some documentation fixes for dynamic formats
benchmark-unify: map "MYSQL" to "MySQL" and "NT v2" to "NT MD4"
Use a different format name for raw-sha1-ng, due to max. pwlen 15
print heading for --list=section only if subsections exist
Fix clang make targets for the HAVE_NSS case
fix hccap2john usage output
bash completion: bug fix for --list=
add --list=format-all-details
Markov: allow start and/or end to be specified in percent
Markov: first convert start % value, then end % value
Fix --list=? output (new-line/trailing space)
 Fix linux-x86-64-icc target for HAVE_NSS case
Markov: move mkv_start / mkv_end computation out of do_markov_crack()
doc/MARKOV: Clarify explanation START / END values
Adjust --list=format-all-details output to f61def05 changes
--list=format-all-details: fix indention for FMT_OMP flag description
Fix linux-x86-native build
Markov: fix segfault re-introduced with commit 2c5a95a3
Enhanced compiler version info for --list=build-info
--list=format-details: add benchmark comment and length and binary and salt
size
Fix another -Wuninitialized warning in keepass2john.c

JimF (36):
Update src/dynamic_utils.c
dynamic non contiguious number issues BE
Update src/dynamic_utils.c
Update src/hmacMD5_fmt_plug.c
Added MD4 to dynamic. 7 crypt functions were deprecated. These 7 'crypt'
functions were replaced with 'crypt_md5' functions. 4 new md4 crypt functions
added. Updated the dynamic conf file, and the dynamic documentation. Added 5 new
builtin types to dynamic (using md4).
Cygwin OMP builds were not building x86-sse.o properly, without this change.
Lotus5: Use the passed in format pointer, and avoid having to forward declare
the static global. Has added benefit of always appearing the same way in all init()
functions of all formats, no name differences.
Simplification of differences between SSE2 and SSE2i code. In nsldap, there
were problems with 32 bit SSE2.
Dynamic MD5_X2 md4 fixes
Dynamic md4 final fixes
Dynamic: Support null bytes in constants.
john-1.7.9-dynamic-unameonly-1.diff
LOTS of minor fixes for VC builds
Mscash and mscash2 formats prepare() fixes
Enhancement of extern:double
Make incremental mode restore ETA figures
-regenerate-lost-salts=N option, for cracking hashes where we do not have the
salt
Fix for the recent -regenerate-lost-salts=N patch.
Added a new rule, to reject a word unless it is a specific length. This could
be done in the past with 2 rejection rules. Updated the doc/RULES file, and
rules.c. Pretty trivial code change.
Arbitrary characters in rules, using \xnn
CORE CHANGES: Add a get_source() function, that (if present) can rebuild the
ciphertext from a binary
get_source() self-test more permissive (tags, case etc)
Win32 SharedMemory patch (only affects _MSC_VER, __CYGWIN32__ or __MINGW32__
builds)
raw2dyna added. This is a helper program that converts raw hashes to dynamic
format.
wordlist.c: Status bug fix
Raw-SHA1 "LinkedIn" format (partial hashes, 20 bit zeroed)
Loads of minor fixes for Sparc (endianness, alignment)
Format renames for dynamic (remove extra spaces)
Fix a cosmetical off-by-one bug in wordlist.c
Sparc_Porting_and_OtherStuff-unstable.diff
MMX_fixes_dynamic_and_some_other_patches.diff
Use first two salts when benchmarking instead of top one.
Portability_issues_unstable.diff
VC fixes to hccap2john.c and keepass_fmt_plug.c
 hmac-sha1 bugfix for SSE2. Now passes Test Suite.
pass_gen.pl updates for GOST, SHA2 etc.

Myrice (31):
Signed-off-by: myrice <myrice@myrice-laptop.(none)>
Inital xsha512-cuda
xsha512-cuda align implementation
CRLF->LF issued
Bug fix, Code clean
Password transfer optimized
cmp_all on GPU, bug fixed, rename gpu to cuda
Xsha512 patch from Solar
Optimize copy hash to host. Copy only when necessary
Makefile fixed
More itereations during one GPU call
Makefile for magnum-jumbo
Makefile updated for magnum
cmp_exact for xsha512-cuda
Reduce 80 rounds in sha512 to 77
Copyright fix. Static memory allocation in cuda_xsha512_fmt.c
XSha512 for OpenCL
xsha512-opencl auto find best local work size
xsha512-opencl cmp_all on GPU
auto find best LWS
xsha512-opencl and CUDA doc
xsha512-opencl: eliminate byte address store
Raw SHA512 CUDA format
Fixed: xsha512-cuda/raw-sha512-cuda: kernel_cmp_all() multiple result malloc
xsha512-cuda: Long password length support
SHA512 opencl implementation
BUG fixed: SWAP64 on AMD GPUs
code style adjust
Bug fix: xsha512-cuda: get_key() for long password
xsha512-cuda: avoid unnecessary hashes copy from GPU to CPU
Bug Fixed: xsha512-opencl incidentally comment out memcpy in set_key()

Claudio André (28):

Move src/*.cl to src/opencl/
OpenCL cryptsha512
Improvement: get the LWS and KPC from john's config file. Improvement: check
if group size is valid. Cosmetic: changes in benchmark comment. See commit:
9d3fe1414e. Performance: put all working variables in __local memory address space.
Fixed: allow temp buffer to use all the available local memory. Fixed: use small
KPC values when running on CPU. On real cracking tasks, big values behaves badly.
Fixed: format and algorithm names. Fixed: copyright information.
More error treatment in find_best_kpc and find_best_workgroup. Only transfer
keys to GPU is necessary (magnum idea).
openCL_crypt512.patch as of
http://www.openwall.com/lists/john-dev/2012/04/18/6
Changes in common_opencl.c
Improvement: Set john.conf LWS and KPC properly. Improvement: Unroll only
important loops. Performance: Use bitselect and bitalign when possible.
Performance: Created a different kernel to run on CPU. Performance: Created a
different kernel to run on NVIDIA. Fixed: Fix find_best_kpc() and
find_best_workgroup misbehaviour.
Improvement: disable #pragma OPENCL EXTENSION cl_khr_byte_addressable_store.
Improvement: use PUTCHAR macro (15% performance gain on CPU).
Improvement: failed to use PUTCHAR macro on GPU. More 3% performance on CPU.
Fix: Avoid memory override on CPU. Note: PUTCHAR fails in insert_to_buffer.
Cause is GPU memory misalignment access. Note: on CPU this is the final version.
Note: on GTX 570 this is (might be) the final version. Seems the compiler optimize
in a way i was unable to reduce register usage. Note: on 7970 this is going to be
the final version for a while. Not able to get useful profile information.
Fix: clear bad and commented code.
Fix: Avoid waste time on find_best_KPC.
Fix: Fix find_best_KPC to get the best configuration.
Put links on OpenCL readme file
Note: on GTX 570 this is the final version. Seems the compiler optimize in a
way i was unable to reduce register usage. Improvement: on Nvidia, use less local
memory and bigger LWS. Fix: more code cleanup.
Fix: more code cleanup.
Fix: even more code cleanup.
Improvement: change the code to be more multi-GPU friendly.
Fix: correct salt usage.
Fix: improve build on multi-GPU environment.
Fix: improve salt management.
Fix: local memory management done by host.
AMD better performance version. DO NOT merge with Jumbo
John jumbo official branch
Fix: use GWS instead of KPC.
Set my public email
Fix: declarations not at block start.
Improvement: more aggressive defaults on Nvidia.

Solar (23):
john-1.7.9-fastrip-1.diff This patch speeds up DES-based tripcode cracking by
making use of JtR's bitslice DES implementation, including its OpenMP
parallelization.
Bugfix in trip_fmt.c (from latest CVS version)
Drupal 7: Minor optimisation
Keyboard external mode fix (from CVS)
XOP for MD5/MD4/SHA-1
sse-intrinsics.c: Take advantage of some SSE2 and SSSE3 intructions if
available http://www.openwall.com/lists/john-dev/2012/03/21/4 (followup)
OMP Support for wpapsk formats
taviso_fmt: Use XOP if available. Plus a fix to sse-intrinsics.c for 32-bit
builds with SSSE3.
Format renames, unified algorithm names
Change cryptmd5 to md5crypt, and cryptsha2 to sha2crypt
-march=native fixes for linux-x86-64-gpu, linux-x86-64-opencl, linux-x86-64-
cuda
WPA-PSK fixes, OpenMP support
MySQL-fast fixes, OpenMP support
Fix breakage caused by commit aab646b42afe93ca909b
john-mpi.c: Do not include omp.h unless OMP build
Add a second test vector to PHPS_fmt_plug.c
md5cryptsse()'s uses of length[]
SSH format bugfix
KeePass format is now 1% faster!
Finally (knock on wood) the RAR cmp_all(3) bug is squatted!
undef set_key in otder to avoid a clash with OpenSSL 0.9.7's backwards
compatibility DES interface, at least on Mac OS X 10.5.8
wpa-psk big-endian fixes
john-1.7.9-jumbo-6 as released by Solar

Lukas Odzioba (22):

john-1.7.9-jumbo-5-opencl-5.diff
john-1.7.9-jumbo-5-cuda-1.diff
 john-1.7.9-jumbo-5-cuda-2.diff
magnumripper-magnum-jumbo-3279fdc-fixed.diff
Rename all opencl kernels
mscash2-cuda http://www.openwall.com/lists/john-dev/2012/03/31/15
Added WPAPSK format for CUDA and OpenCL
Fixed Makefile for CUDA builds for make -j
Added hccap2john and prepared wpapsk-cuda format for real use
Moved wpapsk* common functions and structures to wpapsk.h, prepared wpapsk-
opencl format for real use
Make django a non-plugin with checks for OpenSSL >= 1.0.0
GPU fixes from Lukas
CUDA device lister, show them with --list=cuda-devices
Fix a bunch of compiler warnings
wpa-psk CPU format added
Unify name of "local work size"
CUDA fixes
Revert to older (working) phpass-opencl for now
mscash1/2-cuda fixes
CUDA mscash formats fixes. Still not 100% OK.
phpass_fixesv2.diff
ocl_cuda_fix.diff

Samuele (16):
john-1.7.9-jumbo-5-opencl-3.diff
ssha opencl kernel update
OpenCL updates. Includes renaming rawMD5_opencl_fmt.c to opencl_rawmd5_fmt.c
and rawSHA1_opencl_fmt.c to opencl_rawsha1_fmt.c
renamed nsldaps renamed nt added mysql-sha1
added modified cl for nsldaps and nt
minor changes to cryptmd5_opencl modified nsldaps to add salt alignment as
proposes by magnum
Copyright banner modification
Starting some documentation
modified rawmd5 for the kpc test
README.opencl updated
added cryptmd5 and phpass by lukas
added more kpc and larger step in find_best_kpc
minor changes to john conf
renamed inbuffer as saved_plain
Modified Makefile with some environ variable
various test for mysqlsha1

Sayantan Datta (16):

mscash2-opencl format added
mscash2-opencl: Copyright blurb added
Warning for opencl_MSCASH2_fmt.c muted
mscash2-opencl 20% improvement
opencl-MSCASH2 13% performance improvement, added prepare() function.
opencl-mscash2 Radeon 7970/GTX 570 performance update, openMP support,
optimal work group size calculation
opencl-mscash2 performance update,better cmp_exact()
mscash2-opencl supports salts upto 19 charcters,fixed memory leaks on gpu,
added new file common_opencl_pbkdf2.c to support multiple gpus simultaneouly for
pbkdf2 step.
New format added:Blowfish OpenCL
blowfish-opencl:performance update,7970 bug fix
opencl-mscash2: added split() and salt_hash() function
mscash2-opencl: Code cleanup
opencl-mscash2:improved valid function
 bf-opencl: performance and stability update
bf-opencl: performance and stability update
bf-opencl:passed JtR TS

Tavis Ormandy (10):

New format rawsha1_sse4 by Taviso. This will need some care before including
in a Jumbo release (needs SSSE4)
* Implement buffering of passwords before crypts to reduce function call
overhead. * Write an XMM bswap that doesn't require extracting the XMM regusters. *
Use a lookup table instead of a branch in set_key(). * Implement non-sse4
comparison for those without it. * Other minor syntax changes to match rest of
code.
Only build on platforms with SSE2
Use strrchr instead of memchr
rename format name as suggested by solar.
Remove some of the overhead of a partial match by quickly eliminating obvious
non-matches.
double semicolon typo
Fix keepass2john and rawSHA1_ng compilation on gcc3.4
Try to reorder W loads closer to first use.
Add prefetch-loop-arrays optimization

bartavelle (7):
Copyright headers for md[45]-mmx.S
Added copyright statement.
Ignoring build-generated files
Fix ICC build for x86-64.S
Fix ICC build for taviso_fmt.c
Actually fix ICC build for x86-64.S
64 bit intrinsics assembly file update.

bizonix (2):
small fix for build in macosx
Update src/opencl_cryptsha512_fmt.c

Sergey V (2):
GOST format: avoid function name clashes with OpenSSL > 1.0.0
GOST format: remove unused defines FORMAT_LABEL_CP and FORMAT_NAME_CP

Your Name (1):

Reorder cmp_one/cmp_exact logic as suggested by Solar. Fix bug with
uninitialized nul byte in binary conversion. Change some tabs to spaces for
consistency. Add some more testcases.

András (1):
john-1.7.9-jumbo-5-opencl-5-macosx_10.7.diff (minus the c3_fmt stuff)

Robert Veznaver (1):

replaced mempcpy with memcpy for BSD and OS X compatibility

---
head -2 ../doc/CHANGES-jumbo > ../doc/new && git shortlog -w78,8,10 -n --no-merges
1.8.0-jumbo-1..HEAD ^origin/master | sed -r 's/^ {8}/\t/; s/([^\t]) {2,}/\1 /g; s/[
\t]+$//' >> ../doc/new && echo >> ../doc/new && perl -ne '$p++ if m/^The following
changes/; print if $p > 1' ../doc/CHANGES-jumbo >> ../doc/new && mv ../doc/new
../doc/CHANGES-jumbo
---