ScanMe mobile

ScanMe Mobile: A Local and Cloud Hybrid Service for Analyzing APKs ∗ ∗ ∗ § ∗ ∗ Yevgeniy Cole , Hanlin Zhang , Linqiang Ge , Sixiao Wei , Wei Yu , Chao Lu , § § † Genshe Chen , Dan Shen , Erik Blasch , Khanh D. Pham ∗ † Towson Univ., USA [email protected],{lge2,hzhang4}@students.towson.edu,{wyu,clu}@towson.edu § Intelligent Fusion Technology, USA {sixiao.wei,dshen,gchen}@intfusiontech.com † Air Force Research Laboratory, USA [email protected], [email protected] ABSTRACT ket share [5]. Because of its popularity, it has piqued the interest of adversaries, who may launch malware attacks. According to Fortinet, the network security product corporation, 96.54% of mobile malware infections were found on Android [4]. From November 2013 to October 2014, Kaspersky Lab [6] found 295,539 new malicious apps. Of these apps, 12,100 were mobile banking Trojans. Existing research states that Android experienced four times more malware attacks during this period in comparison with the previous reporting period [2]. Further, it was shown that 53% of all Android attacks were SMS Trojans, designed to steal money and personal information [2]. To defend against mobile malware, static analysis [8] was developed to conduct malware detection by viewing metadata and analyzing the decompiled source code of executables. Nonetheless, static analysis is limited by the ability of code analyzers [7] and adversaries developed numerous obfuscation techniques against known static techniques. Further, the security of mobile devices is challenged by resources limitations (e.g., energy and computing capacity, etc.), while the detection of attacks requires real-time processing capability. The mounting volume of data stored in mobile devices and the high computational power requirement hinder the effectiveness of threat detection and analysis on Android systems. Therefore, as mobile malware continues to evolve and becomes more prominent, it is critical that new dynamic, efficient, and reliable tools and techniques are created to combat newer malware. In this paper, we developed a local and cloud hybrid malware detection service, denoted as ScanMe Mobile. We implemented the mobile cloud based architecture to efficiently perform real-time detection. In our system, we applied Google Cloud Messaging (GCM). The data collected from mobile devices are delivered to a detection center, which conducts threat monitoring and detection based on collected data. Our developed system, ScanMe Mobile, can conduct both static and dynamic analysis. ScanMe Mobile offers users a pre-configured sandbox environment to test and analyze mobile applications before they install them on their phones. In layman’s terms, a sandbox is “an environment, in which the actions of a process are restricted according to As mobile malware increases in numbers and sophistication, it becomes pertinent for users to have access to tools that can inform them of potentially malicious applications. In this paper, we developed a cloud based Android malware analysis service called ScanMe Mobile. The objective of this service is to allow users to learn information about Android application package (APK) files before installing them on their Android devices. With ScanMe Mobile, users can locally scan APK files on their phone’s SD (Secure Digital) memory card, compile a comprehensive report, and share the report by publishing it through a web interface. ScanMe Mobile allows users to perform both static and dynamic analysis on APK files. In addition to integrating some existing analysis tools into the system, we performed Android malware detection based on machine learning techniques. Our experimental data shows that our proposed system can effectively detect malware on the Android platform. CCS Concepts •Security and privacy → Software and application security; Software security engineering; Keywords Android Malware; Google Cloud Messaging; Machine Learning; Sandbox; Google App Engine 1. INTRODUCTION Worldwide, as of first quarter 2015, Android is the most popular smart mobile platform, controlling 78% of the marPermission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. RACS ’15, October 09-12, 2015, Prague, Czech Republic c 2015 ACM. ISBN 978-1-4503-3738-0/15/10. . . $15.00 ⃝ DOI: http://dx.doi.org/10.1145/2811411.2811483 268 a security policy” [1]. To detect malicious APKs through dynamic analysis, we integrated Droidbox to analyze the runtime behavior of scanned applications and implemented an Artificial Neural Network (ANN) based scheme to learn their behavior patterns. upload to a sandbox server. The application is also responsible for registering the device to the Google Cloud Messaging service and notifies the user when a new message is received. • Google Cloud Messaging Service (GCM): GCM is a third party service that provides data transmission between the operation center and mobile devices. When the application runs, the device will register to the GCM service. After the registration, the application sends its registration ID to the Google App Engine Datastore and Website backend, where a new device entity is created in the datastore. • Local Application Sandbox Server (LASS): In LASS, an Ubuntu 14.04 system, a VM (virtual machine) will act as an application sandbox to analyze APK files. A script was developed to automatically install and configure all necessary tools required for this environment. We adopt a plug and play solution, which allows users to install their own local sandbox systems and begin analyzing their APKs quickly. Figure 1: ScanMe System Architecture Using real-world malicious and non-malicious APK samples, we conducted experiments on Android mobile devices and evaluated the effectiveness of ScanMe Mobile in terms of detection accuracy and overhead. Our experimental data shows our proposed detection system can effectively detect malware on the Android platform. For example, our preconfigured sandbox environment can successfully capture the intents SEND, CALL, PHONE NUMBER in malware, which are all known for stolen information through SMS (Short Message Service (SMS) messaging. Our dynamic analysis malware classification system can achieve a detection rate at 85%. When it comes to overhead, our proposed detection service requires very little resources to run on an Android device. ScanMe Mobile is efficient by taking less than one minute to complete a static and dynamic analysis run. The remainder of the paper is organized as follows: We introduce the ScanMe Mobile design and implementation in detail in Section 2. In Section 3, we show performance evaluation of ScanMe Mobile system. Finally, we conclude the paper in Section 4. • Detection Module: In the local application sandbox server, ScanMe Mobile obtains the information of uploaded applications. Among the obtained information, permissions required by the application will be sent to the detection module. To show the effectiveness of the ScanMe Mobile system, we provision machine learning based schemes to carry out detection. Through the machine learning process, the trained module can be used to determine whether uploaded applications are malware or benign applications. • Google App Engine (GAE) Datastore and Website: Once the report is compiled, it is sent and stored in the Google App Engine datastore. The datastore also stores the registration IDs and messages of registered mobile devices. Google App Engine hosts the web application’s frontend as well as the backend. We used JSP technology and the Bootstrap framework to develop our website. We now introduce the workflow of ScanMe Mobile. As shown in Figure 1, a new application implemented in the Android platform first needs to register the device to the GCM with a goal of retrieving its registration identification (ID). The ID information will be stored and sent to GAE backend as a “Device” entity. The user selects and uploads APKs to the local sandbox server. Permission information is sent to the machine learning detection module. The detection result will be sent back to the sandbox. After obtaining the detection data, the APK file will be analyzed and a report will be sent to GAE. Once the sample’s report is written to the datastore or when there is an error, the backend sends a message to the registered device, and the user receives a message from the GCM. The last step for users is to navigate to the GAE website. Users need to enter the identification for the samples and then view the results. 2. SYSTEM DESIGN AND IMPLEMENTATION In this section, we first introduce the system architecture and our design of ScanMe Mobile. We then present our implementations of the Android Client, Google Cloud Messaging (GCM), Local Application Sandbox (LASS), and the Google App Engine database and web site. 2.1 System Architecture As shown in Figure 1, our system consists of five components: (i) the Android client application, (ii) a local sandbox server, (iii) the detection module, (iv) a Google cloud messaging service, and (v) the Google App engine datastore and website. Individual components are described as follow: • Android Client Application: The Android client can host an application that recursively scans its Secure Digital (SD) card for all APK files. This application allows the user to select one or more applications to 2.2 System Implementation In this subsection, we describe the implementation of ScanMe Mobile. 269 2.2.1 Android Application and receive messages, as well as have access to the backend’s API (Application Programming Interface). When a register request is made by the client application, the GCM responds to it with a unique registration ID. At this point, the client sends another request through the Endpoints API to the assigned GAE instance. This instance accepts the request and stores the device’s unique ID in the datastore. The backend can send a message to the client by calling the Endpoints’ doSendViaGCM() method. When the message is received, the client executes onMessage() method and passes the message to the MainActivity class through an intent. By default, GCM is configured to broadcast a message, which will be sent from the backend to 10 other devices found in the datastore. This functionality can be turned off and the message will only be sent to devices that upload APKs. Our ScanMe Mobile Android application is designed to be user friendly. It allows users to quickly select APKs to upload without going through menus and forms. Figure 2 shows the layout of the design of the ScanMe Application. When the application runs for the first time, it will ask the user to register to the GCM service. Users have the choice to register or continue using the application. The ScanMe application allows users to upload and view their reports locally. Users who would rather view their report online will have to register. If the user chooses to register, the registration ID will be stored locally on GAE datastore. The registration ID is stored locally before it is saved to the datastore. Therefore, if the datastore fails to create a new device entity, the user will not be able to receive messages. To deal with this problem, users can clear the cache and register again. The application only has permission to scan a SD card. If APKs are stored on an internal storage, it will fail to find them. By default, when users download applications from Google Play and not from third party markets, the APKs are stored internally, but the user can force applications to be downloaded to the SD card directly. Applications will be displayed within the scroll panel of the designed user interface. The user can choose to upload one or more applications. The system works best when only one APK is uploaded at a time because both static and dynamic analysis can be performed. The sandbox server is configured to run only one Android Virtual Device (AVD) at a time. To enable the analysis of multiple applications, users will have to turn off dynamic analysis within the BASH startScan script. For multiple applications, only static analysis is performed. In ScanMe Mobile, a PHP script, called fileUpload.php, running on LASS, is configured to communicate with ScanMe’s Android Application to allow APKs, as well as the device registration ID to be uploaded. After the script returns either a successful upload or a failed upload, a message will be displayed to the user. Figure 3: Users’ Directory 2.2.3 Local Application Sandbox (LASS) LASS is responsible for the static and dynamic analysis of APK files, the compilation and formatting of reports, and their submittal to http://1.scanmeandroid.appspot.com/. For security reasons, the sandbox runs locally and not on the cloud. A local sandbox environment gives the user more control without having to be concerned about malware propagation. When dynamic analysis is enabled for known malware, users can toggle the ability to dynamically analyze malware without being connected to a network. The script will enable networking once dynamic analysis has concluded. When analysis is complete, a compiled report can be published to a website or kept local. The ScanMe Mobile service allows users to compile reports and share them with a community by posting results on a Google App Engine powered website. By running our BASH script (i.e., InstallSandbox), the entire environment can be installed and configured automatically within 30 minutes. This allows any user with an Ubuntu 14.04 VM to set up a safe running environment and analyze malware. The script must be run with super user privileges (sudo) and it takes an argument of an existing account name. All non-system specific packages, directories and files will be written under user’s home directory. Figure 3 shows the architecture of the user’s directory. A list of InstallSandbox configurations is given below. The installation of packages is shown as follows: Figure 2: ScanmeApp 2.2.2 Google Cloud Messaging (GCM) • Apache2, PHP5, MySql 5.6. To establish a communication channel between the Android client application and a GAE instance, all clients must register to the GCM service. ScanMe Mobile utilizes Google Cloud Endpoint technology to integrate mobile clients with a Google App Engine backend. Endpoint clients can send • Python libraries and Android Analysis tool dependencies. • Various analysis tool dependencies. 270 • OpenJDK 7, Android SDK, Android API 19,18,17 and 16 system images. [3]. We implemented a script called ManifestScript, which separates identified permissions into those three categories, lists all custom permissions, features, services, receivers and activities, and writes all data to the APK’s report file. Upon writing to the report file, the script returns to startStan, which converts the APK to a jar file and decompiles it into java codes. The classes.dex houses all of the Java code for the Android application. To read the Java code within classes.dex, it must first be converted to a Jar file with dex2jar. The Jar file can then be decompiled into Java classes using jd-cmd. A custom parsing script is used to retrieve all mentioned websites, IP addresses, phone numbers, intents and email addresses. At this time, Androapkinfo, an Androguard tool, is also run to list all files, which are not located within the /res (resource) directory. Androapkinfo will generate some output files, which can be analyzed at a later time to provide more insight into the APK’s behavior. When static analysis concludes and if users have dynamic analysis enabled in startScan, an emulator will be started with DroidBox script startemu.sh. The startScan script then waits two minutes for the emulator to load. Once it is loaded, droidbox.sh will be executed. This script starts droidbox.py, which pushes and installs the APK in the emulator. DroidBox runs the application after installation. The application runs for two minutes. During this time, all file access, incoming and outgoing network data, phone calls, and SMS (Short Message Service) messages will be logged. All results are displayed in JSON format. A python script was developed to parse JSON so that all data can be appended to the APK’s report. After both static and dynamic analysis are complete, start Scan runs a jar client, which sends a request to http://1dot-scanmeandroid.appspot.com/submitsample. This client sends the registration ID of the device, as well as the compiled report. Once received, the GAE server begins a datastore transaction. If creating a new entity is successful, a message is then composed and is sent to the corresponding device using the GCM service. • Git, Maven, G++, Expect, x86 system libraries for Ubuntu 14.04, and installation and configuration of Android open source analysis tools (e.g., Apktool, Androguard, Droid−Box4.1.1, Jd−cmd, Dex2Jar, etc.). The system configuration is shown as follows: • Android SDK configuration and Android virtual device creation. • Creation of files and directory structure illustrated in Figure 3. • Apache2 configuration to allow web access to publish html within user accounts. • PHP5 configuration to allow max POST request of 40MB. The execution within user directories is shown as follows: • Change file and directory permissions to allow file uploads and write permissions to allow scan-apk to read and write to files within users’ directory. • Configuration of MySql to run as an instance under a specific user. • Symbolic links are created under HOME/bin to allow easy access to analysis tools. When the sandbox environment has been installed, an inotify BASH script must be run in the background. The script waits for new files uploaded within the /home/public html/uploads directory. After a new APK has been uploaded, the script executes the startScan BASH script. start Scan retrieves the device ID, which was written to identifiers.txt by the fileUpload.php, generates APK’s MD5 hash, retrieves APK’s file size and name, checks for native code, creates a XML element for each value and writes those values to a report file. The report’s name contains the APK’s name, MD5 hash, and a timestamp. The report is stored in the /home/AppSandbox/reports directory. To keep track of uploaded files, from which device and when, startScan creates an entry containing this information within uploadhistory.txt. After parsing the uploaded APK, static analysis will be performed next using file stored at /home/AppSandbox/apks directory. StartScan runs Androaxml, which is a utility within the Androguard suite, to convert the binary AndroidManifest.xml file to a human readable XML file. This file is then parsed by the manifestScript, which extracts all permissions found within the manifest file and matches them with permissions found in permissions.txt file. Permissions.txt lists 206 Android permissions along with their category. Android permissions are divided into three categories: normal, dangerous, and signature/system. Normal permissions allow the application to use API calls that are not harmful to the device and the user. Permissions under the dangerous category have access to potentially harmful API calls. Signature/system permissions can cause the most harm and have the most dangerous privileges. Notice that signature permissions are hard to obtain as they require certification by a manufacturer. They can also be obtained if an application is signed and installed in a special folder on the device 2.2.4 Machine Learning Based Detection Module In the Sandbox, we obtain much information related to uploaded applications. The permission information will be used for further malware detection. To demonstrate the effectiveness of our developed system, we implemented an Artificial Neural Network (ANN)-based malware detection classifier, which uses permissions to detect unknown malware. The classifier learns patterns of permissions and conduct detection. There are two processes in the detection module: offline training and online detection. In the offline training process, the ANN learns the abnormal behaviors of malware in terms of permission requests. The learned model will be further used to carry out malware classification in the online detection process. Figure 4: Workflow of Machine Learning Based Detection 271 The workflow of the system is shown in Figure 4. In the offline training process, the first step is to retrieve the permissions from Android applications. For Android applications, all permissions are included in the Android-Manifest.xml file. We obtain this file from the Sandbox. Next, we collect a set of files, where each file consists of permissions requested by one application. Figure 5 illustrates an example of permissions requested by an application. For the training process, we need to process the data and map them to the format required by the ANN. Because the ANN only accepts integers as input, we map each permission name to an integer number. In terms of values for corresponding features, if a particular permission is requested, its feature value is 1. Otherwise, its feature value is 0. We then combine multiple files and generate the input for the ANN. An example of the input is shown in Figure 6. In the example, each row is one application and each column is the value of the features. Here, 1 means this permission is requested by the application while 0 means the permission is not requested. At this point, we obtain the input that is used for offline training process. We use the learned model in the Matlab neural network tool box (http://www.mathworks.com) to conduct unknown malware classification. In the online detection process, the workflow is similar to the one described in the offline training process. To classify an application, the first step is to obtain permissions and map the permission to the format required by the ANN. Then, we use the trained ANN to determine whether a new application is either a malicious or benign application. The test file has the exactly same format as the training file, which consists of features associated with each application and values associated with each corresponding feature. The online detection process will generate the output file, which contains the classification result. The result is either +1 or -1. When the number is positive, the ANN classifies it as a benign application. When the number is negative, the ANN classifies it as malware. Table 1: Scan Results Intents Activities Emails EMAIL, BCC, CC ShowAllPics [email protected] CALL AdMobActivity [email protected] DELETE AdActivity [email protected] NEW OUTGOING CALL LoginActivity [email protected] PACKAGE ADDED GameAcitivity PACKAGE REMOVED OpenVpnSettings IP Address PHONE NUMBER Activation 12.34.43.21 DAIL Snake 119.147.7.30 PACKAGE REPLACED VideoAdActivity USER PRESENT BrowserActivity Features CLOSE SYSTEM DIALOGS SmartWallActivity software.live wallpaper INSERT hardware.touchscreen MEDIA MOUNTED other samples in the database. If it does, a message is composed and sent to the user, notifying the user how to access the existing results. If it is a new request, a new entity will be created through a transaction. When an error occurs during the middle of a transaction, the entire transaction is rolled back. The GAE frontend or website was designed with the Bootstrap framework and JSP technology. The website is mobile friendly and easy to navigate. When a APK sample has been successfully stored to the datastore, a message is sent to the mobile device. The message contains the request required by GAE to locate the target sample. For now, users can simply navigate to the website mentioned above and enter their sample’s ID or MD5 hash value. Once the APK sample is retrieved, a utility class parses the text object for the sample. Results will be then displayed in four categories: APK information, Android manifest, retrieved strings, and dynamic malware analysis. 3. PERFORMANCE EVALUATION We now show the performance evaluation results in terms of Sandbox performance, and detection accuracy. Figure 5: Example of Permissions of Android Application Figure 7: Detection Rate versus Training Set Ratio To test the effectiveness of our system, 100 malware samples and 100 benign applications were gathered from the Android Malware Genome Project (http://www.malgenomeproject. org/). The goal was to obtain the information from applications and classify them into either malware or benign applications. Results are illustrated in Tables 1 and 2. Table 1 makes note of all interesting intents, activities, emails, IP addresses, and features found. The intents SEND, CALL, PHONE NUMBER were found in malware, which are known for stolen information through SMS messaging. The intents associated with packages came from malware, which installed additional packages from a remote server. Most of activities listed came from malicious applications hosting vari- Figure 6: Example of Input for ANN 2.2.5 Google App Engine Backend and Frontend When a submitted request is made to GAE backend at http://1-dot-scanmeandroid.appspot.com/submitsample, the backend checks whether the sample’s MD5 hash matches an- 272 80%. When we increase the training set ratio to 70%, the detection rate approaches almost 86%. Figure 8 shows the relationship between the false positive rate and the training set ratio. As we can see from Figure 8, the false positive rate declines as the training set ratio increases. 4. CONCLUSION Figure 8: False Positive Rate versus Training Set Ratio ous advertisements. The activities such as .PureGirls16 and .ShowAllPics were from a variation of the GoldDream malware. Five applications contained email addresses. It is revealed that [email protected] (DroidLyzer) and [email protected] (XXShenqi:Valentine’s Day worm) are both associated with malicious applications. Table 2 lists some interesting services and websites found in the source code. For example, three of malicious applications (e.g., wall paper apps. MySendEmailService and MyMakeMessageService) were found within Troogoogle.apk. This app is a resource APK for the famous XXShenqi Valentine’s Day worm. Table 2: Scan Results Services MySendEmailService http://cn.ad.adon.vpon.com/api/webviewAdReq MyMakeMessageService http://lebar.gicp.net/more.aspx?pid=9973 .com.GoldDream.zj.zjService http://tw.ad.adon.vpon.com/api/webviewAdClick WallpaperService http://wap.casee.cn/wlogo.gif TapContextService http://www.casee.cn SystemPlus http://www.adwo.com .MainRun http://register.tapcontext.com/register/5 YahwehOrNoWay.theword http://eula.ad-market.mobi/ProtocolGW/protocol/eula OpenVpnService http://www.ad-market.mobi/1.3/getads SmsReceiver http://www.startappexchange.com/1.3/getads .InstallService http://e.admob.com/clk SnakeService http://media.admob.com/ AndroidMDKProvicer http://gamehall2.3g.qq.com/ghall/m/ liveWallpaper http://www.facebook.com/actionfiguretherapy In this paper, we designed and implemented the ScanMe Mobile application to address the security issue of mobiles. Our developed system can conduct both static analysis and dynamic analysis. ScanMe allows users to upload APK files, analyze them in the sandbox and publish reports online. In addition to integrate some existing analysis tools into the system, we developed an Android malware detection system using the machine learning based scheme. Our experimental data shows that our proposed system is efficient and effective to detect malware on the Android platform. 5. ACKNOWLEDGEMENT This material was support in part by research sponsored by Air Force Research Laboratory under agreement number FA9453-15-C-0016. This work was also supported in part by US National Science Foundation (NSF) under grants: CNS 1117175 and 1350145. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of Air Force Research Laboratory or the U.S. funding agencies. 6. REFERENCES To test the detection accuracy of our machine learning based detection, we conducted experiments using those 100 benign applications and 100 malwares from the Android Malware Genome Project. We obtained the input from the Sandbox and then passed it into our detection module. We used the Matlab Neural Network Toolbox built-in Matlab R2013a (8.1.0.604) to run the machine learning algorithm. With a larger training set, more information can be used to train the ANN classifier, leading to a higher detection accuracy. We define the training set ratio as the ratio of the number of training samples to the total number of samples. To measure the effectiveness of our detection system, we define the detection rate as the probability of correctly classifying the malware. That is, the ratio of the number of malware correctly detected to the total number of malware samples. We also define the false positive rate as the probability of falsely classifying benign applications. That is, the ratio of the number of benign applications falsely detected to the total number of benign applications. Figure 7 illustrates the relationship between the detection rate and the training set ratio. As we can see, the detection rate increases as the training set ratio grows. For example, when the training set ratio is 60%, the detection rate is 273 [1] T. Blasing, L. Batyuk, A. Schmidt, S. Camtepe, and S. Albayrak. An android application sandbox system for suspicious software detection. In Proceedings of 5th International Conference on Malicious and Unwanted Software (MALWARE), 2010. [2] K. S. Bulletin. http://securelist.com/analysis/kaspersky-securitybulletin/68010/kaspersky-security-bulletin-2014-overallstatistics-for-2014/. [3] A. P. Felt, E. Chin, S. Hanna, D. Song, and D. Wagner. Android permissions demystified. In Proceedings of the 18th ACM Conference on Computer and Communications Security (CCS), 2011. [4] Fortinet. http://www.fortinet.com/. [5] IDC. http://www.idc.com/prodserv/smartphone-osmarket-share.jsp. [6] Kaspersky. http://usa.kaspersky.com/. [7] J. Oberheide, E. Cooke, and F. Jahanian. Cloudav: N-version antivirus in the network cloud. In Proceedings of the 17th Conference on USENIX Security Symposium (SECURITY), 2008. [8] A.-D. Schmidt, R. Bye, H.-G. Schmidt, J. H. Clausen, O. Kiraz, K. Yuksel, S. A. Camtepe, and S. Albayrak. Static analysis of executables for collaborative malware detection on android. In Proceedings of the IEEE International Conference on Communications (ICC), 2009.