Skip to content

Instantly share code, notes, and snippets.

@chrisekelley

chrisekelley/sqlite-plugin-largeHeap-property

Last active April 6, 2021 15:30
Show Gist options
  • Save chrisekelley/0d03756e23ade8edb3dd9db432266dcd to your computer and use it in GitHub Desktop.
Save chrisekelley/0d03756e23ade8edb3dd9db432266dcd to your computer and use it in GitHub Desktop.
Download ZIP
Testing after adding android:largeHeap="true" to Cordova project
Raw
sqlite-plugin-largeHeap-property
It passed the 15MB and 25MB payload test, but failed at 35MB:
Error:
```console
04-06 10:01:00.083 2838 3034 I g.rti.tangerin: Starting a blocking GC Alloc
04-06 10:01:00.083 2838 3034 I g.rti.tangerin: Starting a blocking GC Alloc
04-06 10:01:00.133 2838 3034 I g.rti.tangerin: Alloc concurrent copying GC freed 92(18KB) AllocSpace objects, 0(0B) LOS objects, 18% free, 107MB/131MB, paused 103us total 49.774ms
04-06 10:01:00.133 2838 3034 I g.rti.tangerin: Starting a blocking GC Alloc
04-06 10:01:00.157 2838 3034 I g.rti.tangerin: Alloc concurrent copying GC freed 3(16KB) AllocSpace objects, 0(0B) LOS objects, 18% free, 107MB/131MB, paused 112us total 24.169ms
04-06 10:01:00.157 2838 3034 I g.rti.tangerin: Forcing collection of SoftReferences for 140MB allocation
04-06 10:01:00.157 2838 3034 I g.rti.tangerin: Starting a blocking GC Alloc
04-06 10:01:00.186 2838 3034 I g.rti.tangerin: Alloc concurrent copying GC freed 836(56KB) AllocSpace objects, 0(0B) LOS objects, 18% free, 107MB/131MB, paused 113us total 28.855ms
04-06 10:01:00.191 2838 3034 W g.rti.tangerin: Throwing OutOfMemoryError "Failed to allocate a 146800688 byte allocation with 25165824 free bytes and 84MB until OOM, max allowed footprint 137446152, growth limit 201326592" (VmSize 4346692 kB)
04-06 10:01:00.198 2838 3034 I g.rti.tangerin: Starting a blocking GC Alloc
04-06 10:01:00.198 2838 3034 I g.rti.tangerin: Starting a blocking GC Alloc
04-06 10:01:00.223 2838 3034 I g.rti.tangerin: Alloc concurrent copying GC freed 4(16KB) AllocSpace objects, 0(0B) LOS objects, 18% free, 107MB/131MB, paused 251us total 24.489ms
04-06 10:01:00.223 2838 3034 I g.rti.tangerin: Starting a blocking GC Alloc
04-06 10:01:00.247 2838 3034 I g.rti.tangerin: Alloc concurrent copying GC freed 4(15KB) AllocSpace objects, 0(0B) LOS objects, 18% free, 107MB/131MB, paused 92us total 24.387ms
04-06 10:01:00.247 2838 3034 I g.rti.tangerin: Forcing collection of SoftReferences for 140MB allocation
04-06 10:01:00.247 2838 3034 I g.rti.tangerin: Starting a blocking GC Alloc
04-06 10:01:00.254 2943 2943 I SKBD : alm isTosAccept false
04-06 10:01:00.276 2838 3034 I g.rti.tangerin: Alloc concurrent copying GC freed 12(16KB) AllocSpace objects, 0(0B) LOS objects, 18% free, 107MB/131MB, paused 112us total 28.004ms
04-06 10:01:00.276 2838 3034 W g.rti.tangerin: Throwing OutOfMemoryError "Failed to allocate a 146800688 byte allocation with 25165824 free bytes and 84MB until OOM, max allowed footprint 137445856, growth limit 201326592" (VmSize 4346692 kB)
04-06 10:01:00.318 2838 3034 W System.err: java.lang.OutOfMemoryError: Failed to allocate a 146800688 byte allocation with 25165824 free bytes and 84MB until OOM, max allowed footprint 137445856, growth limit 201326592
04-06 10:01:00.342 2838 3034 W System.err: at java.util.Arrays.copyOf(Arrays.java:3260)
04-06 10:01:00.342 2838 3034 W System.err: at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:125)
04-06 10:01:00.342 2838 3034 W System.err: at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:660)
04-06 10:01:00.342 2838 3034 W System.err: at java.lang.StringBuilder.append(StringBuilder.java:203)
04-06 10:01:00.342 2838 3034 W System.err: at org.json.JSONTokener.nextString(JSONTokener.java:223)
04-06 10:01:00.342 2838 3034 W System.err: at org.json.JSONTokener.nextValue(JSONTokener.java:107)
04-06 10:01:00.342 2838 3034 W System.err: at org.json.JSONTokener.readArray(JSONTokener.java:429)
04-06 10:01:00.342 2838 3034 W System.err: at org.json.JSONTokener.nextValue(JSONTokener.java:103)
04-06 10:01:00.342 2838 3034 W System.err: at org.json.JSONTokener.readArray(JSONTokener.java:429)
04-06 10:01:00.342 2838 3034 W System.err: at org.json.JSONTokener.nextValue(JSONTokener.java:103)
04-06 10:01:00.343 2838 3034 W System.err: at org.json.JSONTokener.readArray(JSONTokener.java:429)
04-06 10:01:00.343 2838 3034 W System.err: at org.json.JSONTokener.nextValue(JSONTokener.java:103)
04-06 10:01:00.343 2838 3034 W System.err: at org.json.JSONTokener.readArray(JSONTokener.java:429)
04-06 10:01:00.343 2838 3034 W System.err: at org.json.JSONTokener.nextValue(JSONTokener.java:103)
04-06 10:01:00.343 2838 3034 W System.err: at org.json.JSONArray.<init>(JSONArray.java:92)
04-06 10:01:00.343 2838 3034 W System.err: at org.json.JSONArray.<init>(JSONArray.java:108)
04-06 10:01:00.343 2838 3034 W System.err: at org.apache.cordova.CordovaPlugin.execute(CordovaPlugin.java:97)
04-06 10:01:00.343 2838 3034 W System.err: at org.apache.cordova.PluginManager.exec(PluginManager.java:132)
04-06 10:01:00.343 2838 3034 W System.err: at org.apache.cordova.CordovaBridge.jsExec(CordovaBridge.java:59)
04-06 10:01:00.343 2838 3034 W System.err: at org.apache.cordova.engine.SystemExposedJsApi.exec(SystemExposedJsApi.java:41)
04-06 10:01:00.346 2838 3034 W System.err: at android.os.MessageQueue.nativePollOnce(Native Method)
04-06 10:01:00.346 2838 3034 W System.err: at android.os.MessageQueue.next(MessageQueue.java:326)
04-06 10:01:00.346 2838 3034 W System.err: at android.os.Looper.loop(Looper.java:181)
04-06 10:01:00.346 2838 3034 W System.err: at android.os.HandlerThread.run(HandlerThread.java:65)
04-06 10:01:00.372 1016 1543 D InputTransport: Input channel destroyed: fd=384
```
Here is my test:
```js
let size = 35
let pouchDBOptions = {
adapter: 'cordova-sqlite',
location: 'default',
androidDatabaseImplementation: 2,
key:'test'
}
let pouch = new PouchDB('test', pouchDBOptions);
function formatBytes(a,b=2){if(0===a)return"0 Bytes";const c=0>b?0:b,d=Math.floor(Math.log(a)/Math.log(1024));return parseFloat((a/Math.pow(1024,d)).toFixed(c))+" "+["Bytes","KB","MB","GB","TB","PB","EB","ZB","YB"][d]}
let s = 'x'.repeat(size*1024*1024)
let len =(new TextEncoder().encode(s)).length
console.log(formatBytes(len))
let doc = {
foo:s
}
try {
let newDoc = await pouch.post(doc)
console.log("newDoc: " + JSON.stringify(newDoc))
} catch (e) {
console.log("Database error: " + e);
console.trace();
}
```
Here is the change in my Cordova project:
https://github.com/Tangerine-Community/docker-tangerine-base-image/commit/6850f247fa7977d0f72600042c66a7bc6a633bc1#diff-e757cf5099a6e072c16c8a71673917477046ddd195cb60891bd37d539a9dcce5R33
From the files generated by Cordova build:
platforms/android/gradle.properties:
```
org.gradle.daemon=true
org.gradle.jvmargs=-Xmx2048m
android.useAndroidX=false
android.enableJetifier=false
android.useAndroidX=false
android.enableJetifier=false
android.useAndroidX=true
android.enableJetifier=true
```
config.xml:
```
<?xml version='1.0' encoding='utf-8'?>
<widget xmlns:android="http://schemas.android.com/apk/res/android" id="org.rti.tangerine" version="0.0.1" xmlns="http://www.w3.org/ns/widgets"
xmlns:cdv="http://cordova.apache.org/ns/1.0" >
<name>Tangerine</name>
<description>
Mobile data collection system.
</description>
<author email="[email protected]" href="github.com/Tangerine-Community/Tangerine">
Tangerine dream team
</author>
<content src="shell/index.html" />
<icon src="res/icon.png" />
<allow-navigation href="tangerinecentral.org" />
<allow-intent href="*" />
<access origin="*" />
<chcp>
<config-file url="https://talky.ngrok.io/releases/qa/apks/group-c1180e34-c91d-494b-8ad4-37145cc3e4c5/www/chcp.json"/>
<native-interface version="1" />
<auto-download enabled="false" />
<auto-install enabled="false" />
</chcp>
<uses-sdk android:minSdkVersion="21" android:targetSdkVersion="29" />
<config-file parent="/*" target="AndroidManifest.xml">
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.ACCESS_WIFI_STATE" />
<uses-permission android:name="android.permission.CHANGE_WIFI_STATE" />
<uses-permission android:name="android.permission.SEND_SMS" />
<uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" />
<uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION" />
<uses-permission android:name="android.permission.CAMERA"/>
</config-file>
<edit-config file="app/src/main/AndroidManifest.xml" mode="overwrite" target="/manifest/application">
<application android:largeHeap="true" />
<application android:requestLegacyExternalStorage="true" />
</edit-config>
</widget>
```
the final source of truth - the AndroidManifest/xml
```
<?xml version='1.0' encoding='utf-8'?>
<manifest android:hardwareAccelerated="true" android:versionCode="1" android:versionName="0.0.1" package="org.rti.tangerine" xmlns:android="http://schemas.android.com/apk/res/android">
<supports-screens android:anyDensity="true" android:largeScreens="true" android:normalScreens="true" android:resizeable="true" android:smallScreens="true" android:xlargeScreens="true" />
<uses-permission android:name="android.permission.INTERNET" />
<application android:requestLegacyExternalStorage="true">
<activity android:configChanges="orientation|keyboardHidden|keyboard|screenSize|locale|smallestScreenSize|screenLayout|uiMode" android:label="@string/activity_name" android:launchMode="sin
gleTop" android:name="MainActivity" android:theme="@android:style/Theme.DeviceDefault.NoActionBar" android:windowSoftInputMode="adjustResize">
<intent-filter android:label="@string/launcher_name">
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
</activity>
<provider android:authorities="${applicationId}.provider" android:exported="false" android:grantUriPermissions="true" android:name="org.apache.cordova.camera.FileProvider">
<meta-data android:name="android.support.FILE_PROVIDER_PATHS" android:resource="@xml/camera_provider_paths" />
</provider>
</application>
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.ACCESS_WIFI_STATE" />
<uses-permission android:name="android.permission.CHANGE_WIFI_STATE" />
<uses-permission android:name="android.permission.SEND_SMS" />
<uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" />
<uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION" />
<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-feature android:name="android.hardware.location.gps" />
<uses-permission android:name="android.permission.BLUETOOTH" />
<uses-permission android:name="android.permission.BLUETOOTH_ADMIN" />
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
<uses-feature android:name="android.hardware.telephony" android:required="false" />
</manifest>
```
Interesting! Where is `android:largeHeap="true"`? Is is not in the application manifest:
```
<application android:requestLegacyExternalStorage="true">
```
*** Is there an issue w/ how I am setting it? ***
I am using the following syntax to set it:
```
<edit-config file="app/src/main/AndroidManifest.xml" mode="overwrite" target="/manifest/application">
<application android:largeHeap="true" />
<application android:requestLegacyExternalStorage="true" />
</edit-config>
```
Lastly:
As per this https://stackoverflow.com/a/14812811 link, I may try out this option:
```
add in your build.gradle file
dexOptions {
javaMaxHeapSize "4g"
}
```
It may be possible to do this inside the docker environment by having it set GRADLE_OPTS - `export GRADLE_OPTS=-Xmx4g`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment