Monday, January 22, 2018

Reinforcement Learning & Automated Testing - part 1

2:23 PM Posted by ASM
I will be sharing through a series of blog posts our past experimentations with the use of reinforcement learning for automated testing, to both chase bugs and find vulnerabilities.

Our initial goal was to build a generic intelligent approach to identify vulnerabilities in mobile applications, targeting initially Android Java based applications and iOS LLVM bitcode based applications. Our experimentation lead us to learn about reinforcement learning and to use it for what seemed to give very interesting results.

For the unfamiliar with reinforcement learning, it is a a branch of machine learning that relies on an input loop to continuously enhance results.


Reinforcement learning was for instance used by the AlphaGo project, that used a specialized form of reinforcement learning called deep reinforcement learning and that, as the name suggests, uses deep learning.

Reinforcement learning is in reality quite simple and intuitive. An agent performs an action that he sends to an environment, he then collects information about the new state and action outcome (referred to as reward or punishment), and finally computes a new action based on that output. The input is computed using an algorithm that is referred to as a Policy.



Though the use of reinforcement learning for security testing AFAIK has never been mentioned before, except for 2 very recent academic papers that claims to be the first ones doing so; in reality reinforcement learning has existed for quite some time in some security testing tools and has already proven to produce amazing results. AFL by Michal Zalewsky is the best example that has found a staggering number of vulnerabilities.

AFL is commonly referred to as an evolutionary fuzzer. It generates a test case that gets passed to an instrumented program, then collects the execution trace and generates new inputs that aims at increasing code coverage within the tested program.

 Evolutionary fuzzing in general needs to solve 3 problems:
1 - Fast Instrumentation
2 - Smart Code Coverage Algorithm
3 - Efficient Vulnerability Identification

Though the first (Fast Instrumentation) and last (Efficient Vulnerability Identification) problems have nothing to do with reinforcement learning, I find them so intersting that I believe are worthy of getting some explanation.

Instrumentation consists simply of tracing a program execution, the principle is simple, but the execution is notoriously complex. Instrumentation can have several levels of granularity, per function call, per block or even per instruction. The higher the granularity is, the slower it gets.
There are different ways to instrument a program:
- Compile-time, which simply delegates the task to the compiler to add instrumentation instructions. Initially it was the fastest approach, as the compiler have better understanding of the program, but most importantly the compiler is able to run his optimizations on the instrumented code.

- Software-based run-time, this approach is adapted when we don't have access to the source code of the program. This is by far the slowest approach as it requires constant jumping between the instrumented code and the instrumentation code. It is also very error prone and difficult to get right.

- Hardware-based run-time, which is my favorite approach as it brings the best of both worlds; low overhead and no need for source code. Intel and ARM added support in their processors for program tracing with very low overhead and both AFL and HonggFuzz for instance have support for using hardware-based instrumentation.

The efficient vulnerability identification is yet another complex subject. Initially most fuzzers relied on the program to crash as a sign for a potential vulnerability. However crashes may not occur if for instance the overflow is too small to overwrite anything interesting.

More advanced approach is the one used by sanitizers. LLVM sanitizers perform compile time modifications to a program to make the triggering fact of a vulnerability more apparent, like the use of memory guards that wraps every memory allocation.

All of these approaches are solely adapted for low level languages hunting low level vulnerabilities, like overflows or use-after-free.

The 2nd component of evolutionary fuzzers is the algorithm to increase the code coverage and this is where the reinforcement part happens.

AFL uses genetic algorithms to generate input and relies on the block based instrumentation to identify if an input was capable of triggering a new path in the program.

Genetic algorithms aim at imitating natural selection which consists of producing input, running a set of modifications (crossover and mutation) and selecting from that generated population a subset that passes a fitness function .



In the case of AFL genetic algorithm:
- crossover operation consists for instance of block switching between inputs;
- mutation operation consists for instance of bit flipping;
- fitness function measures the discovery of a new execution path.

AFL also adds an element of curiosity or an exploration bonus by privileging input that triggers new execution path. Genetic algorithms and exploration bonuses are commonly used in modern reinforcement learning solutions.

Other approaches, that predate AFL genetic algorithm, consist of using  SMT and SAT solvers. This approach requires a highly granular instrumentation and attempts to solve complex equations to discover a new execution branch.

SMT solvers have known huge progress in the recent years, but other than the non-public SAGE, there is no fuzzer that has reported good results with this approach.

Other fuzzers tries a combination of multiple techniques to build on the strengths of both approaches. Driller for instance that won the 2nd place at the DARPA Cyber Grand challenge used both AFL, a modified Qemu and Z3 SMT solver.

In the next blog posts I will dive into some limitations of these approaches and present the use reinforcement learning for identifying high level vulnerabilities like SQLi, Command Injection and XXE.

Wednesday, January 17, 2018

Critical Attack Surface of Mobile Applications

1:24 PM Posted by ASM



LiveOverflow published an interesting video on Mobile Application Security where he tackled the Attack Surface of mobile applications and a case of a security 'researcher' who oversells the results of his work.

Mobile Applications are built on top of an environment that has attack surface reduction in mind, through the use of sandboxing, explicit permission model, automated updates and the offering of an API that tries to be secure by default.

However, some Mobile Applications do however expose a critical attack surface that requires special attention, either due to the technology stack they are based on, the kind of usages the application offers or the interaction that the application has with other components.
Another key factor to take into account is the need for these environments to add more features and functionalities. These features offers room for developers to be creative in finding new ways to use our phones, but at the same time increases the attack surface. Take for instance iOS App Extension, this functionality similar to Android intents was added later on to the iOS ecosystem.

These are few examples that we have seen in the past as security critical vulnerabilities but are very specific to the Mobile environment:

Remotely exploitable JavaScript injection in JavaScript-based applications:

Applications developed using JavaScript Framework, like Cordova, Ionic, might be vulnerable to JavaScript or HTML injection. These vulnerabilities can be leveraged into remote code injection due to the nature of the API exposed through these Frameworks.

For such a vulnerability to be considered critical, the attacker must have the capacity to send the malicious input to other users without any special interaction.

For instance, a sport's app that allows users to share their progress through a personal wall. If that wall is vulnerable to JavaScript injection (XSS), then any user who views the wall of the attacker will be compromised.

Even HTML injection vulnerabilities might be transformed into JavaScript injection via JavaScript Gadget attack.

Memory corruption in native code through untrusted input:

Several applications handles the parsing of binary formats like Audio, Video and Images using native libraries. A memory corruption vulnerability in these libraries using an untrusted input will result in a remote code execution.

For instance, if a chat application that allows sending voice recording in MP4 format, suffers from a memory corruption vulnerability, this will result in remote code execution in the context of the application.

Java, Kotlin, Objective C and Swift are all memory safe languages unless unsafe APIs are used, however linking against libraries in C and C++ opens the door for these kind of vulnerabilities.

Intent injection in browsable Activities exploited using drive-by attacks in Chrome:


Chrome allows sending intent with extra parameters to activities with the Browsable category. An application vulnerable to injection through the intent extra parameters is vulnerable to driveby exploitation.

The attacker may either entice the victim into visiting his malicious page, or serve the attack using Ads for instance. Firefox browser requires extra user interaction to trigger intent sending, while most other browsers on mobile devices do not support this feature.

Communication over clear-text traffic or using insecure TLS/SSL server certificate validation:

The impact of this vulnerability depends on the nature of the exchanged data. For instance if either the authentication phase or any session enabled action is performed over insecure channels, this will result in the compromise of the users' session.

If the application is developed in JavaScript Frameworks, the retrieval of remote JavaScript or HTML will result in a remote code execution. If the application downloads a shared library (.so, .dex, .jar) over insecure channels, this will also result in remote code execution.

An example of this vulnerability is the use of ALLOW_ALL_HOSTNAME_VERIFIER:

ALLOW_ALL_HOSTNAME_VERIFIER implementation don't perform any validation:

Session management shared between Mobile and Web application and the lack of Web related protection on the Mobile Backend:

Web Applications share the same browser with other web applications, which creates opportunities for a set of attacks that are not applicable to mobile applications, like CRSF, session hijacking through all sorts of XSS and even Clickjacking.

In general, these attack vectors are absent for Mobile Applications, therefore developers don't need to implement any security protection against these attacks.

Some Mobile Application backends do however share the session management system between the web and the mobile backend, which creates the opportunity for an attacker to link against the mobile backend from the browser and exploit these vulnerabilities.

Friday, June 16, 2017

Finding security bugs in Android applications the hard way

Ostorlab is a community effort to build a mobile application vulnerability scanner to help developers build secure mobile applications. One of the new key components of the scanner detection capabilities is a new shiny static  taint engine for Android Dalvik Bytecode that was heavily optimized for performance and low false positives.

A simple version of a Static Taint Engine computes how user controlled input propagates inside an application. Tracking the flow uses tainting of variables and attributes, hence the name 'Static Taint Engine'. This taint information serves to detect vulnerabilities in the application.

Few months after shipping the initial version of the engine and scanning over 10.000 mobile applications uploaded by users, these are some of the key results we have collected so far.

The static taint engine has detected over 600 high risk vulnerabilities, ranging from content provider SQL injection, insecure SSL/TLS server certificate validation (detected statically), command injection, insecure shared preferences, weak cryptography, hard coded keys and many other classes of vulnerabilities.

These are some examples of the vulnerabilities found using the static taint engine, I was cautious to only share examples from voluntarily insecure applications:

Content provider SQL injection:

The second parameter (1 if you count from 0) of the method android.database.sqlite.SQLiteDatabase.delete() will cause a SQL injection if user-controlled.
The parameter is exposed by the exported content provider method jakhar.aseem.diva.NotesProvider.delete(), hence making the application vulnerable to a SQL injection:

[TAINT] Parameter '1' ==*==*==*==*==>>> Sink '[u'Landroid/database/sqlite/SQLiteDatabase;', u'delete', u'(Ljava/lang/String; Ljava/lang/String; [Ljava/lang/String;)I', u'1', u'SQL_SINK']'
===========
|__Ljakhar/aseem/diva/NotesProvider;->delete(Landroid/net/Uri; Ljava/lang/String; [Ljava/lang/String;)I / 0
 |__Landroid/content/ContentResolver;->notifyChange(Landroid/net/Uri; Landroid/database/ContentObserver;)V (no childs) / 1
 |__Landroid/content/Context;->getContentResolver()Landroid/content/ContentResolver; (no childs) / 1
 |__Landroid/content/UriMatcher;->match(Landroid/net/Uri;)I (no childs) / 1
 |__Landroid/database/sqlite/SQLiteDatabase;->delete(Ljava/lang/String; Ljava/lang/String; [Ljava/lang/String;)I (no childs) / 1
 |__Landroid/net/Uri;->getLastPathSegment()Ljava/lang/String; (no childs) / 1
 |__Landroid/text/TextUtils;->isEmpty(Ljava/lang/CharSequence;)Z (no childs) / 1
 |__Ljakhar/aseem/diva/NotesProvider;->getContext()Landroid/content/Context; (no childs) / 1
 |__Ljava/lang/IllegalArgumentException;->(Ljava/lang/String;)V (no childs) / 1
 |__Ljava/lang/StringBuilder;->()V (no childs) / 1
 |__Ljava/lang/StringBuilder;->append(C)Ljava/lang/StringBuilder; (no childs) / 1
 |__Ljava/lang/StringBuilder;->append(Ljava/lang/Object;)Ljava/lang/StringBuilder; (no childs) / 1
 |__Ljava/lang/StringBuilder;->append(Ljava/lang/String;)Ljava/lang/StringBuilder; (no childs) / 1
 |__Ljava/lang/StringBuilder;->toString()Ljava/lang/String; (no childs) / 1
===========
User controlled parameter is used to construct an SQL parameter vulnerable to SQL injection
Method jakhar.aseem.diva.NotesProvider.delete():

    public int delete(android.net.Uri p8, String p9, String[] p10)
    {
        int v0;
        switch (jakhar.aseem.diva.NotesProvider.urimatcher.match(p8)) {
            case 1:
                v0 = this.mDB.delete("notes", p9, p10);
                break;
            case 2:
                String v2_6;
                int v3_0 = this.mDB;
                StringBuilder v5_1 = new StringBuilder().append("_id = ").append(p8.getLastPathSegment());
                if (android.text.TextUtils.isEmpty(p9)) {
                    v2_6 = "";
                } else {
                    v2_6 = new StringBuilder().append(" AND (").append(p9).append(41).toString();
                }
                v0 = v3_0.delete("notes", v5_1.append(v2_6).toString(), p10);
                break;
            default:
                throw new IllegalArgumentException(new StringBuilder().append("Divanotes(delete): Unsupported URI ").append(p8).toString());
        }
        this.getContext().getContentResolver().notifyChange(p8, 0);
        return v0;
    }

Command injection:

This is an example of the use of a dangerous commands that sets insecure permissive permissions using the mode '777', read write execute to user, group and other, it can't get more permissive that that :/ :

[TAINT] String '/system/bin/chmod -R 0777 F1.txt file12.txt' ==*==*==*==*==>>> Sink '[u'Ljava/lang/Runtime;', u'exec', u'([Ljava/lang/String; [Ljava/lang/String; Ljava/io/File;)Ljava/lang/Process;', u'Object', u'COMMAND_SINK']'
===========
|__Lcom/ibm/android/analyzer/test/cmdinjection/CommandInjection6;->onCreate(Landroid/os/Bundle;)V / 0
 |__Landroid/app/Activity;->onCreate(Landroid/os/Bundle;)V (no childs) / 1
 |__Landroid/content/Intent;->getStringExtra(Ljava/lang/String;)Ljava/lang/String; (no childs) / 1
 |__Lcom/ibm/android/analyzer/test/cmdinjection/CommandInjection6;->cmdRuntime(Ljava/lang/String; I)V / 1
  |__Landroid/content/Context;->getFilesDir()Ljava/io/File; (no childs) / 2
  |__Landroid/util/Log;->i(Ljava/lang/String; Ljava/lang/String;)I (no childs) / 2
  |__Ljava/io/File;->getAbsolutePath()Ljava/lang/String; (no childs) / 2
  |__Ljava/lang/Exception;->printStackTrace()V (no childs) / 2
  |__Ljava/lang/Runtime;->exec(Ljava/lang/String; [Ljava/lang/String; Ljava/io/File;)Ljava/lang/Process; (no childs) / 2
  |__Ljava/lang/Runtime;->exec(Ljava/lang/String; [Ljava/lang/String;)Ljava/lang/Process; (no childs) / 2
  |__Ljava/lang/Runtime;->exec(Ljava/lang/String;)Ljava/lang/Process; (no childs) / 2
  |__Ljava/lang/Runtime;->exec([Ljava/lang/String; [Ljava/lang/String; Ljava/io/File;)Ljava/lang/Process; (no childs) / 2
  |__Ljava/lang/Runtime;->exec([Ljava/lang/String; [Ljava/lang/String;)Ljava/lang/Process; (no childs) / 2
  |__Ljava/lang/Runtime;->exec([Ljava/lang/String;)Ljava/lang/Process; (no childs) / 2
  |__Ljava/lang/Runtime;->getRuntime()Ljava/lang/Runtime; (no childs) / 2
  |__Ljava/lang/StringBuilder;->()V (no childs) / 2
  |__Ljava/lang/StringBuilder;->append(Ljava/lang/String;)Ljava/lang/StringBuilder; (no childs) / 2
  |__Ljava/lang/StringBuilder;->toString()Ljava/lang/String; (no childs) / 2
 |__Lcom/ibm/android/analyzer/test/cmdinjection/CommandInjection6;->getIntent()Landroid/content/Intent; (no childs) / 1
 |__Ljava/lang/StringBuilder;->()V (no childs) / 1
 |__Ljava/lang/StringBuilder;->append(Ljava/lang/String;)Ljava/lang/StringBuilder; (no childs) / 1
 |__Ljava/lang/StringBuilder;->toString()Ljava/lang/String; (no childs) / 1
===========
The application executes a dangerous command
Method com.ibm.android.analyzer.test.cmdinjection.CommandInjection6.onCreate():

    protected void onCreate(android.os.Bundle p7)
    {
        super.onCreate(p7);
        android.content.Intent v2 = this.getIntent();
        String v0 = v2.getStringExtra("exec");
        if (v0 == null) {
            String v1 = v2.getStringExtra("execR");
            if (v1 == null) {
                this.cmdRuntime("/system/bin/chmod 0777 /data/data/com.ibm.android.analyzer.test/1.txt", 5);
                this.cmdRuntime("/system/bin/chmod -R 0777 F1.txt file12.txt", 5);
            } else {
                this.cmdRuntime(new StringBuilder().append("/system/bin/sh ").append(v1).toString(), 5);
            }
        } else {
            this.cmdRuntime(v0, 5);
        }
        return;
    }

Hard-coded encryption keys:

The use of hard-coded encryption keys are another example of common vulnerabilities we see in mobile applications, in the example, the string 'superSecurePassword' is used to called an encryption method:

[TAINT] String 'superSecurePassword' ==*==*==*==*==>>> Sink '[u'Ljavax/crypto/spec/SecretKeySpec;', u'', u'([B Ljava/lang/String;)V', u'0', u'CIPHER_SINK']'
===========
|__Lcom/android/insecurebankv2/MyBroadCastReceiver;->onReceive(Landroid/content/Context; Landroid/content/Intent;)V / 0
 |__Landroid/content/Context;->getSharedPreferences(Ljava/lang/String; I)Landroid/content/SharedPreferences; (no childs) / 1
 |__Landroid/content/Intent;->getStringExtra(Ljava/lang/String;)Ljava/lang/String; (no childs) / 1
 |__Landroid/content/SharedPreferences;->getString(Ljava/lang/String; Ljava/lang/String;)Ljava/lang/String; (no childs) / 1
 |__Landroid/telephony/SmsManager;->getDefault()Landroid/telephony/SmsManager; (no childs) / 1
 |__Landroid/telephony/SmsManager;->sendTextMessage(Ljava/lang/String; Ljava/lang/String; Ljava/lang/String; Landroid/app/PendingIntent; Landroid/app/PendingIntent;)V (no childs) / 1
 |__Landroid/util/Base64;->decode(Ljava/lang/String; I)[B (no childs) / 1
 |__Lcom/android/insecurebankv2/CryptoClass;->()V / 1
  |__Ljava/lang/Object;->()V (no childs) / 2
 |__Lcom/android/insecurebankv2/CryptoClass;->aesDeccryptedString(Ljava/lang/String;)Ljava/lang/String; / 1
  |__Landroid/util/Base64;->decode([B I)[B (no childs) / 2
  |__Lcom/android/insecurebankv2/CryptoClass;->aes256decrypt([B [B [B)[B / 2
   |__Ljavax/crypto/Cipher;->doFinal([B)[B (no childs) / 3
   |__Ljavax/crypto/Cipher;->getInstance(Ljava/lang/String;)Ljavax/crypto/Cipher; (no childs) / 3
   |__Ljavax/crypto/Cipher;->init(I Ljava/security/Key; Ljava/security/spec/AlgorithmParameterSpec;)V (no childs) / 3
   |__Ljavax/crypto/spec/IvParameterSpec;->([B)V (no childs) / 3
   |__Ljavax/crypto/spec/SecretKeySpec;->([B Ljava/lang/String;)V (no childs) / 3
  |__Ljava/lang/String;->([B Ljava/lang/String;)V (no childs) / 2
  |__Ljava/lang/String;->getBytes(Ljava/lang/String;)[B (no childs) / 2
 |__Ljava/io/PrintStream;->println(Ljava/lang/String;)V (no childs) / 1
 |__Ljava/lang/Exception;->printStackTrace()V (no childs) / 1
 |__Ljava/lang/String;->([B Ljava/lang/String;)V (no childs) / 1
 |__Ljava/lang/String;->toString()Ljava/lang/String; (no childs) / 1
 |__Ljava/lang/StringBuilder;->()V (no childs) / 1
 |__Ljava/lang/StringBuilder;->append(Ljava/lang/String;)Ljava/lang/StringBuilder; (no childs) / 1
 |__Ljava/lang/StringBuilder;->toString()Ljava/lang/String; (no childs) / 1
===========
The application uses a hardcoded key to encrypt the data
Method com.android.insecurebankv2.MyBroadCastReceiver.onReceive():

    public void onReceive(android.content.Context p17, android.content.Intent p18)
    {
        String v12 = p18.getStringExtra("phonenumber");
        String v10 = p18.getStringExtra("newpass");
        if (v12 == null) {
            System.out.println("Phone number is null");
        } else {
            try {
                android.content.SharedPreferences v13 = p17.getSharedPreferences("mySharedPreferences", 1);
                this.usernameBase64ByteString = new String(android.util.Base64.decode(v13.getString("EncryptedUsername", 0), 0), "UTF-8");
                String v8 = new com.android.insecurebankv2.CryptoClass().aesDeccryptedString(v13.getString("superSecurePassword", 0));
                String v2 = v12.toString();
                String v4 = new StringBuilder().append("Updated Password from: ").append(v8).append(" to: ").append(v10).toString();
                android.telephony.SmsManager v1 = android.telephony.SmsManager.getDefault();
                System.out.println(new StringBuilder().append("For the changepassword - phonenumber: ").append(v2).append(" password is: ").append(v4).toString());
                v1.sendTextMessage(v2, 0, v4, 0, 0);
            } catch (Exception v9) {
                v9.printStackTrace();
            }
        }
        return;
    }

Insecure SSL/TLS service certificate validation:

This is an example of a method using the insecure ALLOW_ALL_HOSTNAME_VERIFIER to construct an SSL/TLS certificate validation scheme:

[TAINT] Class 'Lorg/apache/http/conn/ssl/SSLSocketFactory;' ==*==*==*==*==>>> Sink '[u'Lorg/apache/http/conn/ssl/SSLSocketFactory;', u'setHostnameVerifier', u'(Lorg/apache/http/conn/ssl/X509HostnameVerifier;)V', u'Object', u'SSLTLS_SINK']'
===========
|__Lcom/ibm/android/analyzer/test/domainvalidation/InsecureApacheSSFAllowAllHostnameVerifier$1;->call()Ljava/lang/Void; / 0
 |__Landroid/util/Log;->i(Ljava/lang/String; Ljava/lang/String;)I (no childs) / 1
 |__Ljava/lang/Exception;->printStackTrace()V (no childs) / 1
 |__Ljava/net/URL;->(Ljava/lang/String;)V (no childs) / 1
 |__Ljava/net/URL;->openConnection()Ljava/net/URLConnection; (no childs) / 1
 |__Ljava/security/KeyStore;->getDefaultType()Ljava/lang/String; (no childs) / 1
 |__Ljava/security/KeyStore;->getInstance(Ljava/lang/String;)Ljava/security/KeyStore; (no childs) / 1
 |__Ljava/security/KeyStore;->load(Ljava/io/InputStream; [C)V (no childs) / 1
 |__Ljavax/net/ssl/HttpsURLConnection;->connect()V (no childs) / 1
 |__Ljavax/net/ssl/SSLContext;->getInstance(Ljava/lang/String;)Ljavax/net/ssl/SSLContext; (no childs) / 1
 |__Ljavax/net/ssl/SSLContext;->init([Ljavax/net/ssl/KeyManager; [Ljavax/net/ssl/TrustManager; Ljava/security/SecureRandom;)V (no childs) / 1
 |__Lorg/apache/http/conn/ssl/SSLSocketFactory;->(Ljava/security/KeyStore;)V (no childs) / 1
 |__Lorg/apache/http/conn/ssl/SSLSocketFactory;->setHostnameVerifier(Lorg/apache/http/conn/ssl/X509HostnameVerifier;)V (no childs) / 1
===========
Use of the insecure attribute ALLOW_ALL_HOSTNAME_VERIFIER to validate TLS certificate
Method com.ibm.android.analyzer.test.domainvalidation.InsecureApacheSSFAllowAllHostnameVerifier$1.call():

    public Void call()
    {
        try {
            android.util.Log.i(this.this$0.TAG, "1");
            javax.net.ssl.SSLContext.getInstance("TLS").init(0, 0, 0);
            java.net.URL v4_1 = new java.net.URL("https://1.www.s81c.com/i/v17/t/ibm_logo_print.png?dv1");
            android.util.Log.i(this.this$0.TAG, "2");
            javax.net.ssl.HttpsURLConnection v5_1 = ((javax.net.ssl.HttpsURLConnection) v4_1.openConnection());
            java.security.KeyStore v3 = java.security.KeyStore.getInstance(java.security.KeyStore.getDefaultType());
            v3.load(0, 0);
            android.util.Log.i(this.this$0.TAG, "3");
            new org.apache.http.conn.ssl.SSLSocketFactory(v3).setHostnameVerifier(org.apache.http.conn.ssl.SSLSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER);
            android.util.Log.i(this.this$0.TAG, "4");
            v5_1.connect();
            android.util.Log.i(this.this$0.TAG, "5");
        } catch (Exception v0) {
            android.util.Log.i(this.this$0.TAG, "exception 1!");
            v0.printStackTrace();
        }
        return 0;
    }

Key concepts:

The engine is almost a full year effort that started as a PoC in Python. Python allowed for quick prototyping, focusing on the algorithms and data structures. The current implementation uses a graph representation of the taint propagation inside a single function (see graph).


The graph is used to evaluate the taint of other functions offering a very fast and real world usable static taint engine, while at same time taking into account object oriented aspect of Dalvik Bytecode to ensure accurate taint propagation.

To generate a taint graph, a list of execution paths are compiled and evaluated singularly, then fused with a global function taint.

This approach is however limited if the function has an exponential execution path structure (see graph example), this problem is commonly known as path explosion and is a strong limitation of static analysis methods, like symbolic execution.



To remediate this limitation, we transform the problem into a 'search problem' rather then a 'brute force problem'. A path selection algorithm selects paths with the highest probability of the presence of a vulnerability, for instance if a particular path do not cross any sink function - sink functions might cause a vulnerability if called using user controlled parameters - then there is no vulnerability to look for and the execution paths are excluded.



The current implementation was rewritten in C++14  after investigating several other programming languages (Rust, Go and C) which offered over 200x gain in execution speed.

There are still room to increase performance and code coverage, but also fix several false positives due to the use of a default over-tainted graph for low level native methods.

These capabilities are already part of Ostorlab Scanner and are continuously, and silently :), being enhanced every day.

We urge you to test it and share your feedback. If there is a vulnerability that you think we are missing or a false positive that the scanner is reporting, we would love to hear from you and try to work on a ways to fix it or detect it.

Sunday, April 23, 2017

New Taint Engine ... more vulnerabilities found

6:33 AM Posted by ASM

We have been for the last few months hard at work developing a new scan engine to identify new classes of vulnerabilities. The new scan engine is capable of identifying SQL injections, intent hijacking, insecure random seed, insecure cryptography etc.

The new scan engine uses taint propagation and it was rewritten multiple times to enhance performance and ressource consumption allowing for an increased application coverage.

It is already available and all of Ostorlab users can already scan their applications for security issues to fix. The scanner is still in beta mode and more work will be poured in the next few weeks to enhance reporting and correct false positives and false negatives. We have however already had great results identifying a huge number of high risk vulnerabilities (https://www.ostorlab.co/report/scan/1/)

We will also follow up with new blog post to share our journey, from what is taint analysis and what advantages does it provide and also delve into many of the software engineering issues we faced, like path explosion, static analysis and false positives, C++ to Python binding and how to push Python performance.


Thursday, November 24, 2016

Testing the security of Cordova applications

Native vs. Hybrid:

Hybrid frameworks like Cordova offers the advantage of building one app for multiple platform (support for Android, iOS, Windows Phone, FireOS, FirefoxOS ...) . The framework is easy and fast to develop with and offers generally a single API for all platforms.

Hybrid apps suffer however from performance issues, even if the latest versions have made great enhances to the speed of the framework and the speed delay is unnoticeable on modern phones.

Cordova:

Cordova is platform to develop hybrid mobile apps using HTML5, CSS3 and Javascript, it is open-source and licensed under Apache Licence version 2.0, the core components offer a rich set of functionality in JavaScript, which can be extended using native languages.

Internals:

To review the security of a Cordova mobile application, it is important to have an initial understanding of the internals of the framework to know what to look for and where.

config.xml:

config.xml is the main configuration file that defines several aspects of the mobile application, like enabled plugins, platform specific settings and list of custom hooks. For more information on the options of config.xml file, please check the following URL: https://cordova.apache.org/docs/en/latest/config_ref/

For a security audit, it is the first thing to check to have an initial understanding of the application capabilities.

<?xml version='1.0' encoding='utf-8'?>
<widget id="io.cordova.hellocordova" version="0.0.1" xmlns="http://www.w3.org/ns/widgets" xmlns:cdv="http://cordova.apache.org/ns/1.0">
  <name>HelloCordova</name>
  <description>
      A sample Apache Cordova application that responds to the deviceready event.
  </description>
  <author email="dev@cordova.apache.org" href="http://cordova.io">
      Apache Cordova Team
  </author>
  <content src="index.html" />
  <plugin name="cordova-plugin-whitelist" spec="1" />
  <access origin="*" />
  <allow-intent href="http://*/*" />
  <allow-intent href="https://*/*" />
  <allow-intent href="tel:*" />
  <allow-intent href="sms:*" />
  <allow-intent href="mailto:*" />
  <allow-intent href="geo:*" />
  <platform name="android">
      <allow-intent href="market:*" />
  </platform>
  <platform name="ios">
      <allow-intent href="itms:*" />
      <allow-intent href="itms-apps:*" />
  </platform>
</widget>

The config.xml file resides at the root of the Cordova project folder which as the following structure:
  • hooks: allows to modify how the Cordova CLI works
  • platforms: hosts the native code for each platform. It is possible for developers to modify it to extend or alter the Cordova framework, although the recommended way is to add plugins.
  • plugins: plugins extend the JavaScript API and are stored in the current folder
  • www: this is the main folder storing the HTML, JavaScript and CSS code.

Whitelist

The whitelist plugin is an important security plugins that defines authorized URI for navigation, intent and network access. The plugins also allows the definition of a CSP (content-security-policy), a very important protection against cross-site scripting (XSS) vulnerabilities, highly dangerous for Cordova application.

For more information on CSP, I highly recommand the following links https://content-security-policy.com/ and this presentation by 2 Google folks on the new nonce-based protection https://www.youtube.com/watch?v=zlH_bBQMgkc.

<!-- Good default declaration:
    * gap: is required only on iOS (when using UIWebView) and is needed for JS->native communication
    * https://ssl.gstatic.com is required only on Android and is needed for TalkBack to function properly
    * Disables use of eval() and inline scripts in order to mitigate risk of XSS vulnerabilities. To change this:
        * Enable inline JS: add 'unsafe-inline' to default-src
        * Enable eval(): add 'unsafe-eval' to default-src
-->
<meta http-equiv="Content-Security-Policy" content="default-src 'self' data: gap: https://ssl.gstatic.com; style-src 'self' 'unsafe-inline'; media-src *">

<!-- Allow everything but only from the same origin and foo.com -->
<meta http-equiv="Content-Security-Policy" content="default-src 'self' foo.com">

<!-- This policy allows everything (eg CSS, AJAX, object, frame, media, etc) except that 
    * CSS only from the same origin and inline styles,
    * scripts only from the same origin and inline styles, and eval()
-->
<meta http-equiv="Content-Security-Policy" content="default-src *; style-src 'self' 'unsafe-inline'; script-src 'self' 'unsafe-inline' 'unsafe-eval'">

<!-- Allows XHRs only over HTTPS on the same domain. -->
<meta http-equiv="Content-Security-Policy" content="default-src 'self' https:">

<!-- Allow iframe to https://cordova.apache.org/ -->
<meta http-equiv="Content-Security-Policy" content="default-src 'self'; frame-src 'self' https://cordova.apache.org">

Plugins

Plugins defines the set of functionalities supported by the application, Cordova has a rich list of existing plugins that allows access to common functionalities like Camera, GPS, Networking and Battery. Developers can develop their own plugins (https://cordova.apache.org/docs/en/latest/guide/hybrid/plugins/index.html) or include one of the hundreds of open-source plugins available here (https://cordova.apache.org/docs/en/latest/guide/hybrid/plugins/index.html).

Cordova plugins are easy to develop,  for Android for instance, they are implemented by simply extending the class CordovaPlugin and overriding one of the execute methods.

Additional frameworks

Applications developed using Cordova usually use complementary frameworks like Sencha, Angular or use extended version of Cordova like PhonGap and Ionic. These frameworks extends Cordova functionality to make application development a lot more easier, for instance Ionic provides services such as push notifications, A/B testing, analytics, code deploys, and automated builds.
These frameworks have their own logic and configuration files.

Debugging

Debugging code either for development or for security testing is very helpful to identify bugs and vulnerabilities or to ease understanding of the application. The tools available for debugging depend on the target platform. For Android, it is possible to use Chrome remote debugging, this provides access to all the developer tools inside Cordova. For iOS, it is possible to use the Safari Web Inspector which offers a very similar set of functionality.

Cordova has become a real alternative to build mobile application. From a security perspective, Cordova introduces new challenges and an extended attack surface, here are some of the common security weaknesses and attention points:

Common security weaknesses:

Insecure Whitelist: the whitelist Cordova plugin is an important cornerstone in the security of Cordova apps. As stated earlier, the plugins allows to define authorized URIs the WebView can navigate to, authorized intents the application can request for the case of Android and the authorized networks accesses.

Another important feature of the plugin is the support of CSP policies to protect against XSS vulnerabilities. CSP is necessary as the whitelist filters do not apply to Websockets and <video> HTML5 tag.

Permissive policies like:

<!-- Don't block any requests -->
<access origin="*" />
are insecure and unfortunately common and facilities the exploitation of XSS vulnerabilities in the application. This vulnerability is checked by the our openly available security scanner.

Cross Site Scripting (XSS): From the OWASP definition, Cross-Site Scripting (XSS) attacks are a type of injection, in which malicious scripts are injected into otherwise benign and trusted web sites. XSS attacks occur when an attacker uses a web application to send malicious code, generally in the form of a browser side script, to a different end user. Flaws that allow these attacks to succeed are quite widespread and occur anywhere a web application uses input from a user within the output it generates without validating or encoding it.

XSS vulnerabilities do apply to Cordova applications and are highly dangerous as they are equivalent to full code execution in the context of the mobile application. The range of possibilities depends on the list of loaded plugins.

To illustrate the vulnerability, let's take the following example of a vulnerable application that lists all contacts with an image (the code is inspired from https://github.com/cfjedimaster/Cordova-Examples/tree/master/stealcontacts):

The application basically retrieves contacts with a photo (line 20) and display the photo and the displayName attribute. The application includes the displayName insecurely, thus introducing an XSS vulnerability.

To exploit the vulnerability, we can simply create a contact with the following displayName:



Once, we launch the application, we are greeted with the alert(1) box.


The payload is in this case harmless, but it is easily possible to replace it with a malicious one to access sensitive information for instance.

Unused plugins: Reducing the attack surface of the application is important to reduce the impact of vulnerabilities like XSS, particularly plugins with dangerous functionality, like file, camera and contact access.

Plugins could also suffer from their own class of vulnerabilities, like SQL injection, insecure filesystem access or XXE injection.

Unpatched Cordova versions: Cordova has suffered in the past from several vulnerabilities bypassing the whitelist restrictions, allowing to load arbitrary resources. Fixing the vulnerability usually requires building and publishing a new version of the application.

For a list of known vulnerabilities, check the following URL: https://www.cvedetails.com/vulnerability-list/vendor_id-45/product_id-27153/Apache-Cordova.html

Android is still Android  and iOS is still iOS: Beside the new class vulnerabilities introduced by Cordova, particularly XSS, a Cordova mobile application is still a mobile application, and classical mobile application vulnerabilities are still applicable.

Monday, July 18, 2016

Python Concurrency and Parallelism: building a custom ProcessPoolExecutor

At Ostorlab we scan hundreds of Mobile Applications each day, each scan is very resource intensive but at the same time, since the beginning,  we had to optimize the code for speed and maximize use of cloud ressources.

Afficher l'image d'origine

Without re-discussing concurrency and parallelism on Python, there are plenty of excellent ressources on the subject, my favorite is "Fluent Python" by Luciano Ramalho.

But here are some quick notes:
  • In CPython, the GIL prevents multiple native threads from executing Python bytecodes at once. This means that Threading is inappropriate for CPU-bound parallelism. I recommend these talk for more information on the subject "Understanding the Python GIL" and "Removing Python's GIL: The Gilectomy"
  • Processing is more adapted to CPU-bound parallelism, however spawning a process results in a significant overhead, if the goal is to run multiple tasks both concurrently and in parallel, running each task in a separate process is highly inefficient
To solve that issue, concurrent.futures library was introduced in Python 3.2 and back-ported to Python 2.5. The library allows spawning a number of workers, default is the number of cores on your computer, to which you can pass tasks for processing. Tasks are sent through queues after being serialized using Pickle.

This is an example from Fluent Python public Github:

"""Download flags of top 20 countries by population
ProcessPoolExecutor version
Sample run::
    $ python3 flags_threadpool.py    BD retrieved.    EG retrieved.    CN retrieved.    ...    PH retrieved.    US retrieved.    IR retrieved.    20 flags downloaded in 0.93s
"""# BEGIN FLAGS_PROCESSPOOLfrom concurrent import futures

from flags import save_flag, get_flag, show, main

MAX_WORKERS = 20

def download_one(cc):
    image = get_flag(cc)
    show(cc)
    save_flag(image, cc.lower() + '.gif')
    return cc


def download_many(cc_list):
    with futures.ProcessPoolExecutor() as executor:  # <1>        
        res = executor.map(download_one, sorted(cc_list))

    return len(list(res))


if __name__ == '__main__':
    main(download_many)
# END FLAGS_PROCESSPOOL

Playing with this, we rapidly faced a lot of limitations because of Pickle serialization limitations, the two main issues were:
For the second issue, we had to think of a way to pass arguments to tasks without going through the queue and the serialization process.

Before using the ProcessPoolExecutor, we used simple Processes which received the arguments in args parameter:

p = multiprocessing.Process(
        target=_process_worker,
        args=(self._call_queue,
              self._result_queue,
              self._initial_args,
              self._initial_kwargs))


Passing arguments this way posed no issue on the size and requires of course no serialization. Python allows to pass queues, pipes and even shared memory using the types multiprocessing.Value and multiprocessing.Array (see multiprocessing)

The particularity of our code was that the passed argument were always the same and significant in size, so we  decided to modify ProcessPoolExecutor to pass these arguments at the initialization which he then passes to the spawned working processes.

Tasks will be executed with these argument as initial parameters, here are the main changes, add initial_args and initial_kwargs attributes:

class ProcessPoolExecutor(_base.Executor):
    def __init__(self, max_workers=None, *args, **kwargs):
        """Initializes a new ProcessPoolExecutor instance.
        Args:            max_workers: The maximum number of processes that can be used to                execute the given calls. If None or not given then as many                worker processes will be created as the machine has processors.        """        _check_system_limits()

        if max_workers is None:
            self._max_workers = multiprocessing.cpu_count()
        else:
            self._max_workers = max_workers

        # Make the call queue slightly larger than the number of processes to        # prevent the worker processes from idling. But don't make it too big        # because futures in the call queue cannot be cancelled.        self._call_queue = multiprocessing.Queue(self._max_workers +
                                                 EXTRA_QUEUED_CALLS)
        self._result_queue = multiprocessing.Queue()
        self._work_ids = queue.Queue()
        self._queue_management_thread = None        self._processes = set()

        # Shutdown is a two-step process.        self._shutdown_thread = False        self._shutdown_lock = threading.Lock()
        self._queue_count = 0        self._pending_work_items = {}
        self._initial_args = args
        self._initial_kwargs = kwargs

Pass the attributes to the spawned processes:

def _adjust_process_count(self):
    for _ in range(len(self._processes), self._max_workers):
        p = multiprocessing.Process(
                target=_process_worker,
                args=(self._call_queue,
                      self._result_queue,
                      self._initial_args,
                      self._initial_kwargs))
        p.start()
        self._processes.add(p)

Use the parameters to run each task:

def _process_worker(call_queue, result_queue, args, kwargs):
    """Evaluates calls from call_queue and places the results in result_queue.
    This worker is run in a separate process.
    Args:        call_queue: A multiprocessing.Queue of _CallItems that will be read and            evaluated by the worker.        result_queue: A multiprocessing.Queue of _ResultItems that will written            to by the worker.        shutdown: A multiprocessing.Event that will be set as a signal to the            worker that it should exit when call_queue is empty.    """    while True:
        call_item = call_queue.get(block=True)
        if call_item is None:
            # Wake up queue management thread            result_queue.put(None)
            return        try:
            args += call_item.args
            kwargs.update(**call_item.kwargs)
            r = call_item.fn(*args, **kwargs)
        except BaseException:
            e = sys.exc_info()[1]
            result_queue.put(_ResultItem(call_item.work_id,
                                         exception=e))
        else:
            result_queue.put(_ResultItem(call_item.work_id,
                                         result=r))

The benefit was even greater as we removed all the extra processing due to serialization of the same object.

The same code went from executing for more than 20 mins to less than a minute thanks to the use of multiprocessing, fixed process spawning overhead and bypassing all the extra serialization.

Going further, we plan on experimenting with Go and Unikernels to see if we can gain more performance.

Friday, May 27, 2016

New in Android M and N: Runtime Permissions

9:17 PM Posted by Amine Mesbahi , , , ,
In Android, the permission system was one of the major security concerns of the platform for many reasons:
  • The user has only two choices, he either allows the application to have all the permissions during the installation without having any clue why and when the app will use them, or denies the permissions and cannot use the application at all(take it or leave it offer).
  • Once the permission is granted during the installation, the application can use it during its lifecycle without any notification and the user cannot deny the permissions unless he removes the application from the device.
  • It is not transparent for users, but also for developers. You might have a cool feature in your app which required a specific permission, but if you don't educate the user and give him context why you need it, the user can find it suspicious and decides to not install your application.
  • There is over 130 unique permissions available in Android. Even if the user checks the permissions one by one during the installation, he can quickly get confused (What is exactly BODY_SENSORS ? what is the difference between BLUETOOTH, BLUETOOTH_ADMIN, BLUETOOTH_PRIVILEGED ?)

Finally Android permission system is redesigned !
   
Starting Android (6.0 Marshmallow), application will not be granted any permission at installation time. Instead, application has to ask the user for a permission one-by-one at runtime. It is an important step to protect the users privacy by allowing them to choose what the application can access and what it can't and the user still have the option to use the application.

Developers should understand the new permission system and adapt the application to handle all the cases and give users a good user experience.

So as a developer, what should I know about the new Android's permission system:
  1. There are two kind of permissions: Normal permissions and dangerous permissions
    • Normal permissions allow the application to access external data excluding the user's private information. You can check the full list here.
    • Dangerous permissions allow the application to access the user's private information. 
  2. For all Android versions, each time you want to use a permission (both normal and dangerous permissions), you need to add the appropriate <uses-permission> tag in your app manifest. 
  3. Normal permissions are granted automatically, and you don't need to request them at runtime.
  4. If you try to access a private information, you need to explicitly require its "dangerous permission" at runtime, otherwise the method will throw a SecurityException causing a crash of the application.
  5. The user can revoke a granted permission anytime !
  6. Permissions are grouped into Permission Groups, so if the user has previously granted a permission of a specific group, you can use all the other permissions of that group without requesting them again. 
  7. To adapt your application to the new permission system, you need to follow the logic below:


    8. Since an example is worth a thousand words, let's go through some code. In this example, we will request the permission SEND_SMS (see comments):

//Unique identifier to identify each permission request
final private int REQUEST_CODE_ASK_PERMISSIONS = 999;
private void sendSMSWrapper() {
    //Check to see if the permission is available
    int hasSendSMSPermission = checkSelfPermission(Manifest.permission.SEND_SMS);
    if (hasSendSMSPermission != PackageManager.PERMISSION_GRANTED) {
     //if the permission is not granted, check if the user has already denied this request
            if (!shouldShowRequestPermissionRationale(Manifest.permission.SEND_SMS)) {
  //Explain to the user why we need to send the sms
      Snackbar.Make(layout, "Send SMS is required to share the new events with your contacts.", Snackbar.LengthIndefinite)
             .SetAction("OK", v => RequestPermissions(Manifest.permission.SEND_SMS, REQUEST_CODE_ASK_PERMISSIONS))
             .Show();
      return;
            }
 // request the permission
        requestPermissions(new String[] {Manifest.permission.SEND_SMS},
                REQUEST_CODE_ASK_PERMISSIONS);
        return;
    }
    // execute your code using the user's private information
    sendSMS();
}
 

@Override
public void onRequestPermissionsResult(int requestCode, String[] permissions, int[] grantResults) {
    switch (requestCode) {
        case REQUEST_CODE_ASK_PERMISSIONS:
            {
            // Check for SEND_SMS
            if (CheckSelfPermission(Manifest.permission.SEND_SMS) == PackageManager.PERMISSION_GRANTED
       {
                //Permission Granted
                sendSMS();
            } else {
                // Permission Denied
         var snack = Snackbar.Make(layout, "Send SMS permission is denied.", Snackbar.LengthShort);
         snack.Show();
            }
            }
            break;
        default:
            super.onRequestPermissionsResult(requestCode, permissions, grantResults);
    }
}

9. To ensure the backward compatibility, we recommend to use Support Library v4 which implements the new methods for Android API level < 23, so you can write only one code and the library will return the corresponding value based on the Android version. Otherwise, you need to check manually the Android version and apply the old permission system for Android API level < 23 and the new permission system for the newer Android API level.

if (Build.VERSION.SDK_INT >= 23) {
    // Runtime permission system
} else {
    // Old permission system
}
10. It is important to understand the new Android's permission system in order to give your users a better user experience. However, to make your code shorter and cleaner, you can handle the permissions using a 3rd party library. Our favorite is PermissionsDispatcher which provides a simple annotation-based API and covers all the basic usages.

11-To learn more about the new permission system, you can check the following references: 
 My 2 cents.

Popular Posts