How to Extract Android Apps Data to Get the Best Data Scraping Results?

Components

• Android Developer Studio
• MITMProxy
• APK-MITM
• NodeJS

Instructions

Make and start the Android Virtual Mechanism: In Android Developer Studio, one needs to use an AVD Manager option from the tools menu to make an emulator. No particular settings are available to think about as you can apply any device or any Android version. Here, we have used Pixel 3 getting a preview Android version (called “R”). While finished, we would get a screen that looks like:

2. Find the APK file and Modify it with APK-MITM: After that, we want an apk file to run the emulator. Using https://apkgk.com , we can do this. There we founded and downloaded the file we needed. Once downloaded, the use of apk-mitm is very easy.

3. Making a Proxy–From the mitmproxy tools, here we have used mitmweb because we never learned any computer keyboard shortcuts. For doing this using Ubuntu, using a terminal we navigated to the place we have downloaded the tools and typed “. /mitmweb”. After getting started, you will have a screen like this.

4. Setting your AVD for using a proxy: In advanced settings about your AVD, just visit Settings->Proxy to set a proxy. You have to set it as provided here. Make use of 127.0.0.1 like a host name, 8080 like the port number as well as press apply. You may have to reboot AVD at that point — to do that, long press your power button.

5. Installing CA certificate (mitmproxy) on an AVD: Using an Android emulator, open the browser and visit http://mitm.it. Then, click the Android icon and download its certificate. It won’t get installed automatically, therefore you have to go to Settings -> Security -> Encryption & Credentials -> Install a certificate -> CA certificate for installing a certificate, which you have downloaded. You would know what you’ve made it right, mitmproxy is inside Trusted Credentials.

6. Installing an App and Start Observing Traffic: Install the app just by pulling a file, which you have created in step-2. Open it and you need to start getting some traffic in a mitmproxy screen. You will have a json file grabbing from AWS S3.