Skip to content

Instantly share code, notes, and snippets.

@huylenq
Last active November 17, 2017 14:56
Show Gist options
  • Select an option

  • Save huylenq/803fc9cd99e852029151902fb2c2883a to your computer and use it in GitHub Desktop.

Select an option

Save huylenq/803fc9cd99e852029151902fb2c2883a to your computer and use it in GitHub Desktop.

Treasure Data Exercise

Mobile SDK

Basic Development

Refactor this pseudocode:

buf = new bytes[N]
// This function `read_from_socket` behaves the
// same as http://man7.org/linux/man-pages/man2/read.2.html
read_len = read_from_socket(socket, buf, N)
if (read_len < N) {
  throw IOException("Failed to read data completely from a socket: "
                    + N)
}

Possible Improvements:

Read from stream

It is possible that read returns a smaller number of bytes since it only get the available bytes right at the time. So the implementation is fundamentally wrong, it should has a loop that obtains the bytes until satisfies the requested size, or EOF from socket, or the buffer is full (which is not really happens in our case).

Provide specific failure

The richer in details of exceptions, the easier it is for debugging and troubleshooting, especially when incidents happen. read system call does support a mechanism to detect the errors by passing N = 0. That should be invoked in case of read_len < N, and the exception then should be thrown with that extra bit of valuable information.

Exit gracefully

Close the socket before exit. This really depends on the current context of the executing code, which is hard to tell from a snippet, not a complete method. But the general rule of thumb is the one that opens the resource is the one that cleans it up. So if the code above the snippet (supposingly in the same method) opens the socket, then it is also responsible for closing that socket before return or terminated by failure.

Implementation:

Since this is about general development, Python is used for its pseudocode friendliness and readability. A Java version is also provided. This example implementation also fleshes out a simple use case to have a better demonstration.

def next_message(address):
    """An artificial terminal API to fetch the next fixed-length
    message from an endpoint with the provided address.
    """
    buf = buf or bytes(MESSAGE_SIZE)
    buf_offset = actual_read = 0
    try:
        socket = HLSocket.connect(address)
        while (buf_offset < MESSAGE_SIZE):
            actual_read = read_from_socket(
                socket,
                # pointer to the buffer at offset
                offset_buffer_pointer(buf, buf_offset),
                MESSAGE_SIZE - actual_read)
            if actual_read == -1:
                # Attempts to retrieve failure reason by
                # passing 0 as the requested size
                error_code = read_from_socket(socket, None, 0)
                raise IOError(
                    f"Failed to read data completely from a socket:
                    {MESSAGE_SIZE}. Error code: {error_code})"
            elif actual_read == 0 and buf_offset < MESSAGE_SIZE:
                raise IOError(
                    f"EOF before reaching the requested size {MESSAGE_SIZE}")
            buf_offset += actual_read
    finally:
        # Release the socket since we are the one openned it
        # This can be more succint if we use Python `with` construct
        socket.close()
    return buf

iOS & Android - Mobile Development

Application Settings

How do you save and load application settings that shouldn’t be ephemeral? Could you write example code snippets to do that for both Android and iOS apps?

Each platform has its own idiomatic way. But generally, both use file-based key/value store.

iOS

Uses User Defaults System, or through Settings bundle for infrequent changes.

NSUserDefaults *defaults = [NSUserDefaults standardUserDefaults];
[defaults setString:"meter" forKey:@"Unit"];
// In Cocoa applications, UserDefaults is changed synchronously
// in the same process but asynchronously to persistent storage.
NString *storedUnit = [defaults stringForKey:@"Unit"];

Android

Through SharedPreferences

// Call from an Activity
SharedPreferences sharedPref = getActivity()
    .getPreferences(Context.MODE_PRIVATE);
SharedPreferences.Editor editor = sharedPref.edit();
editor.putString(getString(R.string.unit), "Meter");
editor.commit();
String storedUnit = sharedPref().getString(getString(R.string.unit));

Database Connections

When using SQLite in Android and iOS app, when do you open and close a database resource (database handle) in apps?

This is, in my opinion, subjective. I myself prefer on demand method, open new connections when needed and close it after done. Compare to keeping the connection around then open/close it by hooking to application’s life cycles, it is much more manageable and less error prone with multi-threading and locks.

It is sometimes over-emphasized on the overhead of constantly opening and closing connections. It’s true for some applications when database accesses are in high frequency where caching is critical or a high complexity database [fn::Especially in case of SQLite when the initial query has to parse out all the sqlite_master table] then keeping an opening connection is sensible. But for most mobile applications, it doesn’t have that much of an impact on the perceived performance. A wise choice would be going with whatever that make things more simple and straightforward (but preferably flexible to change) for the early stage of development, then improve later after actual measures and it shows it does pay off. Otherwise any optimization while increases the complexity is premature optimization.

Android Development

Let’s say you’re facing a Java compiler version problem in Android library development. You need to extract Java class files from the apk file. How do you do that?

~dex2jar~ comes to the rescue. After obtained a jar, which is typically just a ZIP format. We can just unzip to get the bytecode class files.

After you get Java class files from the apk file, you need to check JDK version which compiled the class files. How do you do that?

javap -verbose <class_name> will show the class file format version. Or one without javap but his favorite hex editor can get it from the class file signature on byte seventh and eighth (unsigned 16-bit integer, big-endian format). The class file version number can then be mapped back to JDK version.

After the above checking, you noticed you used the wrong JDK version when building the Android library. How do you adjust the JDK version to build the Android library?

Ensure you have the corresponding JDK installed on your local machine. After that, it highly varies by the tool that being used: Gradle, Maven, Leiningen, etc. But for most tools, if not configured explicitly, build with the JAVA_HOME environment variable pointing to the installed JDK location is sufficient.

iOS Development

You developed a library for iOS application and deployed it to Cocoapods repository. But a customer reported they ran into the following warnings when trying to use the library via Cocoapods.

{
  "totalWarnings":12,
  "warnings":[
    {"source":"ld",
     "message":"object file (.../libFoobarClient.a(foobar_io_sqli te3.o))
         was built for newer iOS version (9.3) than being linked (8.0)"
    },
    ...
  ]
}

The podspec file you created is like this:

Pod::Spec.new do |s|
  s.name         = "Foobar Client"
  s.platforms    = { :ios => "" }
  s.source       = { :git => "https://github.com/abcde-lab/foobar-client.git",
                     :tag => "3.1.4"}
  s.source_files = 'FoobarClient'
  s.dependency "AwesomeHttpClient", '= 3.2.28'
  ...
end

The podspec defined does not explicitly specify the iOS minimal deployment target nor platform version. Last time I checked, the default in this case would be 8.0 (at max). So it’s probably caused by the transitive dependency from AwesomeHttpClient which perhaps has the minimum deployment target set to 9.3.

You are facing a link error when developing a library for iOS application. You need to see if the static library libFoobarClient.a includes a function AllocateMemPool for both arm7 and amd64 architectures. How do you do that?

Using lipo -info on that static library file. Or you will eventually findout in XCode when linking to that static library if you enable the corresponding architecture support in your project.

Unity Development

Let’s say you need to get JSON data from a HTTP server in your application. The data is large and it takes several seconds, but the download shouldn’t affect rendering UI on your application. Write an example code to do that in C#.

Unity in specific

The idiomatic way is to leverage their UnityWebRequest which is already asynchronous by default and works seamlessly with their coroutine (which yield the result to the main thread). Below is a shamelessly ripped off example from Unity’s API documentation with some small modifications:

public class CommunicationManager : MonoBehaviour {
    void Start() {
        StartCoroutine(FetchMyBigDataFromMainThread());
    }
    IEnumerator FetchMyBigDataFromMainThread() {
        using (UnityWebRequest www =
               UnityWebRequest.Get("http://api.treasuredata.com/v4/fatty-data")) {
            // The work is offloaded to some worker threads
            yield return www.SendWebRequest();
            // The result is yielded back to main thread
            if (www.isNetworkError || www.isHttpError) {
                Debug.Log(www.error);
            } else {
                Debug.Log(www.downloadHandler.text);
            }
        }
    }
}

.NET/Mono in general

Basically we have to dispatch away from the rendering/drawing thread. If the HTTP client library we’re using is blocking, naively we could just spawn a new Thread(). A more proper way in production is to take advantage of the managed ThreadPool. The example below demonstrates using .NET’s TPL.

...
using System.Net.Http;

class CommunicationManager {
    private HttpClient httpClient = new HttpClient();

    void FetchMyBigDataFromMainThread() {
        // Possibly called from a rendering thread
        httpClient
          // System.Net.Http executes on the thread pool and
          // propages System.Threading.Tasks.Task as result
          .GetAsync("https://www.treasuredata.com/v4/fatty-data")
          .ContinueWith((Task<HttpResponseMessage> task) => {
              // Execute on a managed thread from thread pool
              if (task.Status == TaskStatus.Faulted) {
                  Console.Error.WriteLine("*Furiously blame the server guy!* " +
                                          task.Exception);
              }
              else if (task.Status == TaskStatus.RanToCompletion) {
                  Console.WriteLine("Happily handle my data: " + task.Result);
                  // At this point, we can consider it's done if the data just need
                  // to be forwarded to some non-UI tasks. Otherwise, to be able to
                  // consume this piece of data on the rendering thread or main thread
                  // in Unity, we have to maintain a job queue that accessible from
                  // Unity's main thread. Below demonstrates a simple MonoBehaviour
                  // component (which must be attached to a game object on scene hierachy)
                  UnityDispatcher.Instance.Dispatch(() => {
                      // Call Unity API as normal
                      GameObject.FindWithTag("Hero")
                                .GetComponent<Hero>()
                                .showOffTheData(task.Result);
                  });
              }
          });
    }
}

// Proper error handling and cleanup are skipped for brevity
class UnityDispatcher {
    private static UnityDispatcher _instance = null;
    private readonly Queue<Action> _queue = new Queue<Action>();
    public static UnityDispatcher Instance {
        get {
            if (_instance == null) {
                throw new Exception(
                    "Couldn't found an UnityDispatcher instance. " +
                    "Did you add the component to the scene?")
            }
            return _instance;
        }
    }
    public void Dispatch(Action action) {
        _queue.Enqueue(action);
    }
    void Awake() {
        if (_instance == null) {
            _instance = this;
        }
    }
    void Update() {
        lock (_queue) {
            // It would be better if the queued actions are coroutines.
            // So we wouldn't block the main thread and be equipped with
            // additional time-based operations (WaitUntil, WaitForSeconds, etc.)
            // at hand. But those are besides the point if this example, so
            // this queue is intentionally kept stupidly simple.
            while (_queue.Count > 0) {
                _queue.Dequeue();
            }
        }
    }
}

Java

The source code below is already shared on https://gist.github.com/huylenq/871656a2247164374e9684a91987b613 but including here for the sake of completeness.

[PDF version only]

Appendix

The source of this document is also hosted on this Gist where it is easier to grab the example code with indentations intact.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment