Major changes from 1.9.0-jumbo-1 (May 2019) in this bleeding-edge version:

- Bug fixes, most importantly in OldOffice-opencl format, which could give
  false negatives on some or many builds.  [Solar, magnum, others; 2019]

- Documentation fixes and improvements.  [Solar, magnum, others; 2019]

- Improved our portable hi-res timer for nano-second resolution on most
  archs, and using uint64_t instead of double.  This timer isn't used much
  yet, but may replace other timers in the future.  [magnum; 2019]

- Never print the "spinning wheel" unless stderr is a tty.  [magnum; 2019]

- Added new option --rules-skip-nop for when you already ran some attack
  on some slow format without rules and now want to run it *with* rules.
  [magnum; 2019]

- Mute warning about low buffer-fill in case the difference in time appears
  to correspond to less than one second.  [Solar/magnum; 2019]

- Dropped the RelaxKPCWarningCheck config option as it was obsoleted with
  the above logic  [magnum; 2019]

- If "Many salts" test exceeded 90% of target, force finish so not to get
  annoying/confusing messages.  [Solar/magnum; 2019]

- Always use fcntl() locks.  Handle spurious EAGAIN despite waiting for lock
  (likely due to ulimits or server limits).  [magnum; 2019]

- Implement some tricks for automagically decrease risk of file lock clashes
  when running a large number of nodes.  [magnum; 2019]

- Bugfix in zip2john for stream archives in certain situations.  [magnum; 2019]

- Add --fork ability to ZTEX formats.  [Denis Burykin; 2019]

- tezos2john: Added the ability to validate Tezos Seed Words.
  [LordDarkHelmet; 2019]

- Added --length=N which is a shortcut for --min-length=N --max-length=N
  [magnum; 2019]

- Add functions for querying host's total or available physical memory, if
  possible.  Among other things, this will be used for detecting when an OpenCL
  buffer that is backed by an equal size host buffer gets too big.
  [magnum; 2019]

- Single mode: Add new config option SingleMaxBufferAvailMem. If true, actual
  amount of physical memory will override SingleMaxBufferSize (it may increase
  or decrease). If that option isn't set but SingleMaxBufferSize is explicitly
  set to zero, no limit will be applied!  [magnum; 2019]

- When running "-test -mask" benchmark, don't disable self-tests.  Instead
  perform a non-mask self-test before the mask-mode benchmark.  [magnum; 2019]

- Default to benchmarking with mask.  For benchmarking without mask, use the
  new --no-mask option.  [magnum; 2019]

- Add format match symbol '#' for searching in format_name. For example,
  --format=#ipmi-opencl  will pick the RAKP-opencl format.  We already had
  the '@' symbol for searching in alorithm name.  [magnum; 2019]

- Reduce repeated log messages with fork/node/MPI.  [magnum; 2019]

- Allow wildcard for --users/groups/shells options. Also makes them case
  insensitive when wildcard is used.  [magnum; 2019]

- When ShowUIDinCracks = Y in config and format is WPAPSK, actually print
  gid instead, as that is the access point's MAC address.  [magnum; 2019]

- Show "same-salt boost" figure if applicable. Add john.conf boolean options
  ShowSaltProgress and ShowRemainOnStatus for some extra status output.
  [magnum; 2019]

- dmg2john: Recognize .backupbundle as an alias for .sparsebundle
  [magnum; 2019]

- Add option --no-keep-guessing that "turns off" FMT_NOT_EXACT just like the
  old --keep-guessing option "turns it on".  [magnum; 2019]

- Worked out a new KISS solution "FMT_BLOB" for non-hash formats, so they can
  still benefit from same-salt optimizations/exploits.  [magnum; 2019]

- Converted WPAPSK formats to the new FMT_BLOB scheme, with excellent
  improvement in performance, especially for single mode.  [magnum; 2019]

- Converted Office formats to the new FMT_BLOB scheme.  [magnum; 2019]

- Converted oldOffice CPU format to the new FMT_BLOB scheme.  The GPU version
  uses internal mask so may be hard or impossible to convert.  [magnum; 2019]

- Converted RAR3 formats to the new FMT_BLOB scheme.  [magnum; 2019]

- zip2john: Refactor and support reading central directory.  [srett; 2019]

- Added support for Enpass 6.x database files.  [dsmith, magnum; 2019]

- Single mode: Fix a buffer overflow when max. length is decreased.
  [magnum; 2020]

- Keys [<] and [>] can now be used to decrease/increase verbosity level.
  [magnum; 2020]

- At verbosity level 2, only print admin cracks to screen.  [magnum; 2020]

- Added --single-user-seed=FILE option for single mode. FILE is a wordlist
  with seeds per username (user:password[s] format).  [magnum; 2020]

- Added a workaround for some nvidia bug causing keepass-opencl to fail using
  recent drivers.  [magnum; 2020]

- Added a warning for devices only supporting OpenCL 1.0.  [Claudio; 2020]

- Added -mlc option to unique - "max. length considered".  [magnum; 2020]

- 7z formats:  Added support for BCJ and other preprocessors sometimes used
  before compression.  Before this we had false negatives for such files.
  Also updated LZMA code from upstream, to version 19.00.  [magnum; 2020]

- Upgraded Unicode stuff from version 11.0.0 to 13.0.0.  This affects the
  external modes Dumb16, Dumb32, Repeats16 and Repeats32.  It also affects
  the Easter egg option "--subsets=full-unicode" which BTW supersedes the
  external modes with far better performance.  [magnum; 2020]

- Add support for ssh new-style private keys encrypted using `aes256-ctr`
  cipher.  [vkhromov; 2020]

- Implement full checking in SSH OpenCL format, and stop it as well as the
  CPU SSH format from defaulting to --keep-guessing.  Bugs were squashed as
  well.  For any remaining problems with false-positives (if any), user can
  selectively use the --keep-guessing option.  [magnum; 2020]

- Add support for --format=LIST  [magnum; 2020]

- If command-line or config-file rules are given in UTF-8 but we're utilizing
  a legacy internal codepage, convert them to that codepage.  Mask mode already
  had the corresponding feature.  [magnum; 2020]

- Improve handling of using stdin or pipe vs. reading keystrokes.  This
  includes a new option --force-tty that will set the terminal up for
  reading status/quit keystrokes even if we're not the foreground process.
  [magnum; 2020]

- Make keypress [d] special: It will also emit a status line, but not until
  the current batch has been run among all salts.  Under some circumstances
  that may take a good while.  [magnum; 2020]

- Add command-line option --loader-dupecheck=<yes/no>, overriding config file.
  Mark the latter as deprecated in config file comments.  [magnum; 2020]

- Add alternative syntax to --salts option for loading "N most populated
  salts" as opposed to "salts having at least N hashes".  [magnum; 2020]

- Add support for Telegram Desktop >= 2.2.0/2.1.14b, CPU-only for now.
  [philsmd, magnum; 2020]

- Drop support for --fix-state-delay option, and instead do something better
  in the cracker engine - by default, and for all modes.  [magnum; 2020]

- Addition of mosquitto2john.py for cracking of Eclipse Mosquitto passwd files.
  [blackfell; 2021]

- Final fixes for zip2john selection of 1-byte or 2-byte early-reject checks,
  as well as exactly what source to use for that check, after seeing several
  issues and digging into the specs.  Note that a re-run of zip2john is needed
  for correct cracking.  [magnum; 2021]

- Allow any node range divisible by fork (or MPI) count.  [Solar, magnum; 2021]

- Fix autoconf selection of arch.h for Apple M1.  Improve ARM pseudo-intrinsics
  portability (would fail on clang).  Allow RAR formats to build on ARM where we
  detect it can handle unaligned memory access.  [magnum; 2021]

- Add options --subsets-prefer-short and --subsets-prefer-small to Subsets
  mode, affecting candidate order.  [magnum; 2021]

- Support "./configure --without-openssl" on all platforms, which excludes much
  functionality, but allows the build to complete.  (Previously, this was only
  supported on macOS and required use of Apple's CommonCrypto.)  [Solar; 2021]

- Repair "make -f Makefile.legacy linux-mic" (easy cross-compilation for first
  generation Xeon Phi) and make it not depend on zlib by default.  The OpenSSL
  and GMP dependencies are now also easy to exclude by commenting out lines in
  Makefile.legacy, for all targets.  [Solar; 2021]

- Drop support for Apple's CommonCrypto, it was defunct for many years anyway.
  [magnum; 2021]

- Improved Monero support: Now supports legacy wallets from before 2016 that are
  not using JSON format yet.  [patrickd; 2021]

- Add --without-unrar option to configure, for building without the non-free
  ClamAV unrar code (crippling the RAR v3 formats a little).  Simply deleting
  src/unrar*.[ch] will infer that option as well.  [magnum; 2021]

- Support --target-encoding with --make-charset, for generating charsets for
  formats like LM that need to be in a legacy codepage.  [magnum; 2021]

- Incremental: Allow supplying a charset file name as "mode", with no config
  entry required.  [magnum; 2021]

- Cracker: Re-transfer salt every 30 seconds even if only one salt is loaded.
  This minimizes impact if a device-side salt gets thrashed.  [magnum; 2021]

- pwsafe2john.py: Fixed parsing bug of iteration count, that led to
  uncrackable hashes in the past.  [NecroMortis; 2021]

- Python3 compatibility improvements in various *2john.py scripts
  [exploide; 2019-2021]

- Add new option --catch-up=NAME, for running a new session only until it
  reaches the candidates tried count of a different, existing and paused
  session, then exit.  [magnum; 2021]

- Add BestCrypt Volume Encryption V4 format.  [Jean-Christophe Delaunay; 2021]

- Addition of bestcryptve2john.py for cracking of BestCrypt Full Disk Encryption.
  [trounce1 / HN; 2021]

- Added missing code to the Office formats for cracking Office 2007 documents
  with a 256-bit key (apparently very uncommon).  The CPU format would bail
  with a message (hence the issue reported) but unfortunately the OpenCL format
  just produced false negatives until now.  [magnum; 2021]

- Added formats for cracking cryptoSafe vaults.  They are loaded as-is: There's
  no "cryptosafe2john" or such involved.  [magnum; 2021]

- Added sense2john.py: pfSense/OPNsense config.xml hash extract script.
  [Private Wolf; 2021]

- The Zip (WinZip) formats got several optimizations and the OpenCL version now
  handles large data sizes much better.  [magnum; 2021]


Major changes from 1.8.0-jumbo-1 (December 2014) to 1.9.0-jumbo-1 (May 2019):

- Updated to 1.9.0 core, which brought the following relevant major changes:

  - Optimizations for faster handling of large password hash files (such as
    with tens or hundreds million hashes), including loading, cracking, and
    "--show".  These include avoidance of unnecessary parsing (some of which
    creeped into the loader in prior jumbo versions), use of larger hash
    tables, optional use of SSE prefetch instructions on groups of many hash
    table lookups instead of doing the lookups one by one, and data layout
    changes to improve locality of reference.  [Solar; 2015-2017]

  - Benchmark using all-different candidate passwords of length 7 by default
    (except for a few formats where the length is different - e.g., WPA's is 8
    as that's the shortest valid), which resembles actual cracking and hashcat
    benchmarks closer.  [Solar, magnum; 2019]

  - Bitslice DES implementation supporting more SIMD instruction sets than
    before (in addition to our prior support of MMX through AVX and XOP on
    x86(-64), NEON on 32-bit ARM, and AltiVec on POWER):
    - On x86(-64): AVX2, AVX-512 (including for second generation Xeon Phi),
      and MIC (for first generation Xeon Phi).
    - On Aarch64: Advanced SIMD (ASIMD).
    [Solar, magnum; 2015-2019]

  - Bitslice DES S-box expressions using AVX-512's "ternary logic" (actually,
    3-input LUT) instructions (the _mm512_ternarylogic_epi32() intrinsic).
    [DeepLearningJohnDoe, Roman Rusakov, Solar; 2015, 2019]

    (In jumbo, we now also use those expressions in OpenCL on NVIDIA Maxwell
    and above - in fact, that was their initial target, for which they were
    implemented in both JtR jumbo and hashcat earlier than the reuse of these
    expressions on AVX-512.)

  See also:

  - https://www.openwall.com/lists/announce/2019/04/12/1 1.9.0 core release

- Added FPGA support for 7 hash types for ZTEX 1.15y boards ("./configure
  --enable-ztex", requires libusb).  Specifically, we support: bcrypt,
  descrypt (including its bigcrypt extension), sha512crypt & Drupal7,
  sha256crypt, md5crypt (including its Apache apr1 and AIX smd5 variations) &
  phpass.  As far as we're aware, several of these are implemented on FPGA
  for the very first time.  For bcrypt, our ~119k c/s at cost 5 in ~27W greatly
  outperforms latest high-end GPUs per board, per dollar, and per Watt.  For
  descrypt (where we have ~970M c/s in ~34W) and to a lesser extent for
  sha512crypt & Drupal7 and for sha256crypt, our FPGA results are comparable to
  current GPUs'.  For md5crypt & phpass our FPGA results are much worse than
  current GPUs'; we provide support for those hashes to allow for more (re)uses
  of those boards.  We also support multi-board clusters (tested by Royce
  Williams for up to 16 boards, thus 64 FPGAs, all sharing a USB 2.0 port on a
  Raspberry Pi 2 host).  For all 7 hash types, we have on-device candidate
  password generation for mask mode (and hybrid modes applying a mask on top of
  host-provided candidates from another cracking mode) and on-device hash
  comparison (of computed hashes against those loaded for cracking).  We
  provide pre-built bitstreams (5 of them, two of which support two hash types
  each due to our use of multi-threaded soft CPU cores interfacing to
  cryptographic cores) and full source project trees.  [Hardware design and
  host code by Denis Burykin, project coordination by Solar Designer, testing
  also by Royce Williams, Aleksey Cherepanov, and teraflopgroup.  2016-2019.

  See also:

  - doc/README-ZTEX, src/ztex/fpga-*/README.md
  - [List.ZTEX:Devices] and [ZTEX:*] john.conf sections
  - https://www.openwall.com/lists/john-users/2019/03/26/3 bcrypt
  - https://www.openwall.com/lists/john-users/2019/03/29/1 descrypt
  - https://www.openwall.com/lists/john-users/2019/02/03/1 sha512crypt, Drupal7
  - https://www.openwall.com/lists/john-users/2019/01/12/1 sha256crypt
  - https://www.openwall.com/lists/john-users/2019/04/01/1 md5crypt, phpass
  - https://www.techsolvency.com/passwords/ztex/ Royce Williams' cluster
  - https://www.ztex.de/usb-fpga-1/usb-fpga-1.15y.e.html board specifications

  These are old (introduced in 2011-2012), mostly ex-Bitcoin-miner boards with
  four Spartan-6 LX150 FPGAs per board.  ZTEX sold these boards for 999 EUR
  (plus EU VAT if applicable) in 2012 with the price gradually decreasing to
  349 EUR (plus VAT) in 2015, after which point the boards were discontinued.
  Used boards were commonly resold on eBay, etc. (often in significant
  quantities) in 2014 to 2016 for anywhere from $50 to 250 EUR, but are now
  unfortunately hard to find.  We support both German original and compatible
  US clones of the boards.

- Dropped CUDA support because of lack of interest.  We're focusing on OpenCL,
  which is more portable and also runs great on NVIDIA cards (in fact, much
  better than CUDA did for us before, due to our runtime auto-tuning and
  greater focus on getting OpenCL right).

- We now have 88 OpenCL formats, up from 47 in 1.8.0-jumbo-1.  (The formats may
  be listed with "--list=formats --format=opencl".)

  - Added 47 OpenCL formats: androidbackup-opencl, ansible-opencl,
    axcrypt-opencl, axcrypt2-opencl, bitlocker-opencl, bitwarden-opencl,
    cloudkeychain-opencl, dashlane-opencl, diskcryptor-aes-opencl,
    diskcryptor-opencl, electrum-modern-opencl, enpass-opencl, ethereum-opencl,
    ethereum-presale-opencl, fvde-opencl, geli-opencl, iwork-opencl,
    keepass-opencl, keystore-opencl, krb5asrep-aes-opencl, lm-opencl,
    lp-opencl, lpcli-opencl, mscash-opencl, notes-opencl, office-opencl,
    openbsd-softraid-opencl, pbkdf2-hmac-md4-opencl, pbkdf2-hmac-md5-opencl,
    pem-opencl, pfx-opencl, pgpdisk-opencl, pgpsda-opencl, pgpwde-opencl,
    raw-sha512-free-opencl, salted-sha1-opencl, sappse-opencl, sl3-opencl,
    solarwinds-opencl, ssh-opencl, sspr-opencl, telegram-opencl, tezos-opencl,
    truecrypt-opencl, vmx-opencl, wpapsk-pmk-opencl, xsha512-free-opencl.

  - Dropped 6 OpenCL formats (functionality merged into other OpenCL formats):
    odf-aes-opencl, office2007-opencl, office2010-opencl, office2013-opencl,
    ssha-opencl, sxc-opencl.

  [Dhiru Kholia, magnum, Sayantan Datta, Elena Ago, terrybwest, Ivan Freed;
  2015-2019]

- We now have 407 CPU formats, up from 381 in 1.8.0-jumbo-1 (including
  pre-defined dynamic formats), or 262 non-dynamic CPU formats, up from 194 in
  1.8.0-jumbo-1, despite having dropped many obsolete ones.  (The formats may
  be listed with "--list=formats --format=cpu".)

  - Added 80 CPU formats (not including pre-defined dynamic formats): adxcrypt,
    andotp, androidbackup, ansible, argon2, as400-des, as400-ssha1, axcrypt,
    azuread, bestcrypt, bitlocker, bitshares, bitwarden, bks, dashlane,
    diskcryptor, dominosec8, dpapimk, electrum, enpass, ethereum, fortigate256,
    fvde, geli, has-160, itunes-backup, iwork, krb5-17, krb5-3, krb5asrep,
    krb5tgs, leet, lp, lpcli, md5crypt-long, monero, money, multibit, net-ah,
    notes, nsec3, o10glogon, o3logon, oracle12c, ospf, padlock, palshop,
    pbkdf2-hmac-md4, pbkdf2-hmac-md5, pem, pgpdisk, pgpsda, pgpwde, phps2,
    plaintext, qnx, racf-kdfaes, radius, raw-sha1-axcrypt, raw-sha3, saph,
    sappse, scram, securezip, signal, sl3, snmp, solarwinds, sspr, stribog-256,
    stribog-512, tacacs-plus, tc_ripemd160boot, telegram, tezos, vdi, vmx,
    wpapsk-pmk, xmpp-scram, zipmonster.

  - Dropped 12 CPU formats (not including pre-defined dynamic formats):
    aix-smd5, efs, md4-gen, nsldap, nt2, raw-sha, raw-sha1-ng, raw-sha256-ng,
    raw-sha512-ng, sha1-gen, ssh-ng, sxc.  Their functionality is available in
    other formats - e.g., AIX smd5 hashes are now supported by our main
    md5crypt* formats.

  [Dhiru Kholia, JimF, magnum, Fist0urs, Rob Schoemaker, MrTchuss,
  Michael Kramer, Ralf Sager, bigendiansmalls, Agnieszka Bielec, Ivan Freed,
  Elena Ago, Claudio Andre, Solar; 2015-2019]

- Several old formats got support for additional underlying hash, KDF, and/or
  cipher types under their previous format names, making them more general -
  e.g., the OpenBSD-SoftRAID format now supports bcrypt-pbkdf.  [Dhiru, others]

- Several file archive formats got better support for file format variations,
  large file support, and/or more complete verification (no longer producing
  false positives, and thus no longer needing to continue running after a first
  seemingly successful guess).  [magnum, philsmd, JimF, others?]

- Added many new pre-defined dynamic format recipes.  See run/dynamic.conf.
  [Dhiru, JimF, Remi Dubois, Ivan Novikov; 2015-2018]

- Added dynamic compiler mode that can handle simple custom algorithms on CPU
  (including with automatic use of SIMD) - e.g. "sha1(md5($p).$s)" - without
  any programming - just state that very string on the command line as
  "--format=dynamic='sha1(md5($p).$s)'".  This is somewhat of a hack, but it
  has clever self-testing so if it seems to work chances are it really does.
  Available features include tens of fast hash functions (from common like MD5
  to exotic ones like Whirlpool), string concatenation, encoding/decoding,
  conversion to lowercase or uppercase, and references to the password, salt,
  username, and string constants.  See doc/DYNAMIC_EXPRESSIONS.  [JimF; 2015]

- Many formats now make better use of shared code, often with optimizations
  and/or SIMD support that was previously lacking.  [magnum, JimF; 2015-2019]

- Shared code for reversing steps in MD4/MD5/SHA-1/SHA-2, boosting several fast
  hash formats.  [magnum; 2015]

- We added a terrific "pseudo-intrinsics" abstraction layer, which lets us use
  the one same SIMD source code for many architectures and widths.  [Zhang Lei,
  magnum, JimF; GSoC 2015, 2015-2019]

- Where relevant, all SIMD formats now support AVX2, AVX-512 (taking advantage
  of AVX-512BW if present), and MIC, as well as NEON, ASIMD, and AltiVec -
  almost all of them using said pseudo-intrinsics (except for bitslice DES
  code, which comes from JtR core and uses its own pseudo-intrinsics for now).
  [magnum, Zhang Lei, JimF, Solar; GSoC 2015, 2015-2019]

- When AES-NI is available, we now use it more or less globally, sometimes with
  quite significant boost.  [magnum; 2015]

- Runtime CPUID tests for SSSE3, SSE4.1, SSE4.2, AVX2, AVX512F, and AVX512BW
  (AVX and XOP were already present from 1.8 core), making it possible for
  distros to build a full-featured fallback chain for "any" x86 CPU (including
  along with fallback from OpenMP-enabled to non-OpenMP builds when only one
  thread would be run).  See doc/README-DISTROS.  [magnum; 2015, 2017, 2018]

- Countless performance improvements (in terms of faster code, better early
  rejection, and/or things moved from host to device-side), sometimes to single
  formats, sometimes to all formats using a certain hash type, sometimes
  globally.  [magnum, Claudio, Solar, others; 2015-2019]

- Better tuning (by our team) of candidate password buffering for hundreds of
  CPU formats, as well as optional auto-tuning (on user's system, with
  "--tune=auto" and maybe also with "--verbosity=5" to see what it does) for
  all CPU formats, both with and without OpenMP.  [magnum; 2018-2019]

- Many OpenCL formats optimized and/or re-tuned to be friendly to newer NVIDIA
  and AMD GPUs, and to newer driver and OpenCL backend versions.  Some OpenCL
  formats gained generally beneficial optimizations (for older hardware too),
  and notably our md5crypt-opencl is now about twice faster on older AMD GPUs
  as well.  [Claudio, Solar, magnum; 2019]

- Many improvements to OpenCL auto-tuning (which is enabled by default), where
  we try to arrive at an optimal combination of global and local work sizes,
  including addition of a backwards pass to retry lower global work sizes in
  case the device was not yet fully warmed up to its high-performance clock
  rate when the auto-tuning started (important for NVIDIA GTX 10xx series and
  above).  [Claudio, magnum, Solar; 2015, 2019]

- When auto-tuning an OpenCL format for a real run (not "--test"), tune for the
  actually loaded hashes (as opposed to test vectors) and in some cases for an
  actual candidate password length (inferred from the requested cracking mode
  and its settings).  [magnum; 2017, 2019]

- Nearly all OpenCL formats now do all post-processing on GPU, so don't need
  more than one CPU core.  Post-processing on CPU is kept where it presumably
  wouldn't run well on a GPU (e.g. RAR or ZIP decompression), but for them we
  often have excellent early-reject - often even on device-side.  [magnum,
  Dhiru; 2018-2019]

- Graceful handling of GPU overheating - rather than terminate the process,
  JtR will now optionally (and by default) sleep until the temperature is below
  the limit, thereby adjusting the duty cycle to keep the temperature around
  the limit.  (Applies to NVIDIA and old AMD drivers.  We do not yet have GPU
  temperature monitoring for new AMD drivers.)  [Claudio, Solar; 2019]

- We've switched from 0-based to 1-based OpenCL device numbers for consistency
  with hashcat.  (We also use 1-based numbers for ZTEX FPGA boards now.)
  [Claudio, magnum, Solar; 2019]

- More efficient session interrupt/restore with many salts.  Previously, we'd
  retest the current set of buffered candidate passwords against all salts; now
  we (re)test them only against previously untouched salts.  This matters a lot
  when the candidate password buffers are large (e.g., for a GPU), target hash
  type is slow, and different salt count is large.  [JimF; 2016-2017]

- PRINCE cracking mode ("--prince[=FILE]") added due to kind contribution by
  atom of hashcat project.  It's not a rewrite but atom's original code, with
  additions for JtR session restore and some extras.  PRINCE is a wordlist-like
  mode, but it combines multiple words from a wordlist to form progressively
  longer candidate passwords.  See doc/PRINCE.  [atom, magnum; 2015]

- Subsets cracking mode added ("--subsets[=CHARSET]"), which exploits the
  weakness of having too few different characters in a password even if those
  come from a much larger set of potential characters.  A similar mode was
  already present as an external mode (originally by Solar) but the new mode is
  way faster, has full Unicode support (UTF-32 with no limitations whatsoever)
  and unlike that external mode it also supports session restore.
  See doc/SUBSETS.  [magnum; 2019]

- Hybrid external mode added.  This means external mode can produce lots of
  candidates from a single base word.  See "External Hybrid Scripting" in
  doc/EXTERNAL and "Hybrid_example", "Leet", and "Case" external modes in the
  default john.conf and the "HybridLeet" external mode in hybrid.conf.
  [JimF, Christien Rioux; 2016]

- Stacking of cracking modes improved.  Mask can now be stacked after any other
  cracking mode, referring to that other mode's output "word" as "?w" in the
  mask.  See doc/MASK.  The experimental "--regex" mode can be stacked before
  mask mode and after any other cracking mode.  [magnum, JimF; 2015-2016]

- Rules stacking.  The new option "--rules-stack" can add rules to any cracking
  mode, or after the normal "--rules" option (so you get rules x rules).
  [magnum; 2018]

- Support for what used to be hashcat-specific rules.  The ones that did not
  clash with our existing commands just work out-of-the-box.  Support for the
  ones that clash can be turned on/off at will within a rule set (using lines
  "!! hashcat logic ON" / "!! hashcat logic OFF").  See doc/RULES-hashcat.
  [JimF, magnum; 2016, 2018]

- Added third-party hashcat rule sets to run/rules/ and referenced them from
  separate sections as well as from [List.Rules:hashcat] in default john.conf,
  so "--rules=hashcat" activates most of them.  Our "--rules=all" also invokes
  these rules, but only as the last step after completing our usual rule sets.
  [magnum, individual rule set authors; 2018]

- Support for giving short rule commands directly on the command line,
  including with preprocessor, e.g. "--rules=:[luc]$[0-9]" to request
  "lowercase, uppercase, or capitalize, and append a digit" (30 rules after
  preprocessor expansion).  The leading colon requests this new feature, as
  opposed to requesting a rule set name like this option normally does.
  [JimF; 2016]

- Support for running several rule sets once after another, e.g.
  "--rules=wordlist,shifttoggle".  [JimF; 2016]

- Enhanced the "single crack" mode (which targets hashes with candidate
  passwords derived from related information such as their corresponding
  usernames) to be reasonable to use on massively-parallel devices such as
  GPUs in some cases, which was never the case before (we advised for this mode
  to always be used purely on CPU).  This is achieved through buffering of much
  larger numbers of candidate passwords per target salt (deriving them from
  application of a larger number of mangling rules) and teaching the rest of
  this mode's logic to cope up with such extensive buffering.  As part of this
  change, means were added for limiting this mode's memory usage (relevant when
  hashes having a lot of different salts are loaded for cracking), most notably
  the "SingleMaxBufferSize" setting in john.conf (4 GB by default).
  See doc/MODES.  [magnum; 2018]

- Added means for supplying global seed words for "single crack" mode, from
  command line ("--single-seed=WORD[,...]") or file ("--single-wordlist=FILE").
  [magnum; 2016]

- Wordlist mode: Better suppression of UTF-8 BOMs at a little performance cost.
  [magnum; 2016]

- Unicode support is now at version 11.0.0, and we also added a few legacy
  codepages.  [magnum; 2018]

- UTF-32 support in external modes.  This made an awesome boost to Dumb16/32
  and Repeats16/32 modes.  [magnum; 2018]

- Use our own invention "UTF-32-8" in subsets mode, for a significant boost in
  final conversion to UTF-8.  In the future we will likely make much more use
  of this trick.  [magnum; 2018]

- Full Unicode/codepage support even for OpenCL - most notably for formats like
  NT and LM.  [magnum; 2014-2019]

- Perfect hash tables for quick matching of computed against loaded hashes on
  GPU, used by many of our fast hash OpenCL formats.  So far, tested for up to
  320 million SHA-1 hashes, which used up 10 GB of GPU memory and 63 GB of host
  memory.  For comparison, when using CPU only (and bitmaps along with simpler
  non-perfect hash tables), the same hashes need 25 GB on host only, but the
  attack runs slower (than on-device mask mode, see below).  [Sayantan; 2015]

- On-device mask mode (and compare) implemented in nearly all OpenCL formats
  that need device-side mask acceleration.  Unlike most (maybe all) other
  crackers, we can do full speed cracking (or e.g. hybrid wordlist + mask)
  beyond ASCII, e.g. cracking Russian or Greek NT hashes just as easy as
  "Latin-1" - and without any significant speed penalty.  [Sayantan, Claudio,
  magnum; 2015-2019]

- Many improvements to mask mode, including incrementing lengths with
  stretching of masks (so you can say e.g. "-mask=?a -min-len=5 -max-len=7".
  [Sayantan, magnum; 2015, 2018]

- Uppercase ?W in mask mode, which is similar to ?w (takes another cracking
  mode's output "word" for construction of a hybrid mode) but toggles case of
  all characters in that "word".  [Sayantan; 2015]

- Extra (read-only) pot files that will be considered when loading hashes (such
  as to exclude hashes previously cracked on other systems, etc.)
  [magnum, JimF; 2015]

- Improved support for huge "hashes" (e.g. RAR archives) by introducing
  shortened pot entries and an alternate read line function that can read
  arbitrarily long lines.  [magnum, JimF; 2016]

- A negative figure for "--max-run-time=N" will now abort after N seconds of
  not cracking anything.  [magnum; 2016]

- Improved logging with optional full date/time stamp ("LogDateFormat",
  "LogDateFormatUTC", "LogDateStderrFormat" in john.conf).  [JimF; 2016]

- JSON interface for frontends (like Johnny the GUI) to use in the future for
  querying stuff.  [magnum, Aleksey Cherepanov; 2017, 2019]

- Many updates to *2john programs for supporting more/newer input formats and
  runtime environments (e.g., Python 3 compatibility).
  [Dhiru, magnum, philsmd, Albert Veli, others; 2015-2018]

- wpapcap2john: Support for more link types, more/newer packet types,
  more/newer algorithms e.g. 802.11n, anonce fuzzing, pcap-ng input format;
  dropped hard-coded limits in favor of dynamic allocations.
  [magnum; 2014-2018]

- More extensive self-tests with "--test-full" and optional builtin formats
  fuzzer with "--fuzz" (when built with "./configure --enable-fuzz").
  [Kai Zhao; GSoC 2015]

- "configure" options "--enable-ubsan" and "--enable-ubsantrap" for building
  with UndefinedBehaviorSanitizer (we already had "--enable-asan" for building
  with AddressSanitizer in 1.8.0-jumbo-1).  [Frank, JimF; 2015, 2017]

- "configure" options "--disable-simd" and "--enable-simd=foo" to easily build
  without SIMD support or for a particular SIMD instruction set (other than the
  build host's best).  [magnum, JimF; 2017]

- Default to not enable OpenMP for fast hash formats where OpenMP scalability
  is too poor and we strongly recommend use of "--fork" instead.  Accordingly,
  "configure" option "--disable-openmp-for-fast-formats" is replaced with its
  opposite "--enable-openmp-for-fast-formats".  [magnum; 2015]

- Lots of improvements and tweaks to our usage of autoconf.  [magnum]

- Many bug fixes, cleanups, unifications, and so on.  Many fixes initiated from
  source code, static, or runtime checking tools like ASan, UbSan, and fuzzers.
  [magnum, Frank, Claudio, Solar, Christien Rioux, others; 2015-2019]

- Many fixes for big-endian architectures and/or for those that don't allow
  unaligned access.
  [magnum, JimF, Claudio, Frank, Solar, others]

- Many improvements to documentation, although we're still lagging behind.
  [magnum, Frank, Solar, others]

- Far more extensive use of Continuous Integration (CI), where pull requests
  can't be merged until passing numerous tests on different platforms.  This is
  mostly part of our development process and not the release, although some
  CI-related files do exist in our released tree.
  [Claudio, magnum; 2015-2019]
