Enable AWS X-Ray in koa project written in async/await

Recently at work I was required to implement aws x-ray in our web service. There is official document on [AWS website] (https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-nodejs.html). However it doens’t help much because the official plugin only supports Express. What we are using is Koa.

Set up local X-Ray daemon

This is the first step for development and it’s quite friendly. After starting the daemon, I can configure xray sdk to publish packet to local udp port (127.0.0.1:2000) and the daemon will submit segment to aws using credentials in ~/.aws/credentials.

Use x-ray to trace koa request

The root cause of the lack of support on Koa is that the xray sdk cannot work with async/await. it uses cls for segment storage, which doesn’t support async/await. When you call AWSXRay.setSegment(), it will save it to a cls namespace. Later when you try to getSegment(), it returns empty. As a result we cannot use automatic mode for x-ray. We have to manually create and close segment following this github repo. By using koa middleware, we can start a segment at the beggining of processing the request and close it at the end.

const segment = new Segment(name, root, parent)
segment.addIncomingRequestData(new IncomingRequestData(ctx.req))
segment.close()

In the x-ray console, in order to see the correct url for the request, we must pass in correct IncomingRequestData.

How to work with SQL?

The xray sdk supports SQL query subsegment as shown here. The first problem I faced was that we are using sequelize as an ORM for Postgres. I found a github repo But it doesn’t work. I looked into their code and found that essentially it was simply calling AWSXRay.captureMySQL(require('pg'));. No wonder it won’t work. Even if i changed it to AWSXRay.capturePostgres(require('pg')); it didn’t work because it was using cls in the lower level. I have to manually capture subsegment.

Capture subsegment

According to document I should use AWSXRay.captureAsyncFunc to create subsegment for a function. Unfortunately it doens’t work either because it uses cls as well. From aws-xray-sdk-core/lib/capture.js line 30 I know it’s actually very easy to do it manually

const subsegment = segment.addNewSubsegment(name)
subsegment.close()

Just do these two lines before and after the function execution. If I want to do this for many functions, I’d better do it in a decorator way. Suprisingly nodejs is going to support decorator in the future. It’s only in stage II now. Our project is not using babel so I have to wrap things myself.

async wrappedFn(...arg) {
    const subsegment = segment.addNewSubsegment(name)
    let res = await originalFunction.apply(this, arg)
    subsegment.close()
    return res
}

Repeate this for the whole class

Apperently I cannot wrap all functions in our project. I decided to write a helper to wrap all functions in a class. After some trial and error, I got this

function xrayObject(obj) {
    const className = obj.constructor.name;
    for(const method of Object.getOwnPropertyNames( Object.getPrototypeOf(obj))) {
        const temp = obj[method];    *1
        const subsegmentName = `${className}.${method}`;
        if(temp.constructor.name === 'AsyncFunction') {  *2
            obj[method] = async (...args) => {
                const subsegment = currentSegment.addNewSubsegment(name);
                const res = await temp.apply(obj, args);
                subsegment.close();
                return res;
            };
        } else {
            obj[method] = (...args) => {
                const subsegment = currentSegment.addNewSubsegment(name);
                const res = temp.apply(obj, args);
                subsegment.close();
                return res;
            };
        }
    }
    return obj;
}

*1 If I want to wrap the function and keep the function name, I have to save it to temp first. Otherwise the wrap will create infinite loop. *2 I didn’t have this check before. Without it this wrapper will turn every function into async function. That is not what we want.

With this wrapper I can trace all function executions within a class, including database related ones.

Use dlib on iOS project

I started an iOS project last week and decided to use dlib for some features it provides. This blog will keep track of the process.

Compile with dlib

There is an introduction blog about this topic. Getting Started With dlib on iOS

It was very useful but while following it, I still got some problems.

0

Some error about libdlib.a not compiled for arm64.
Solution: don’t use General iOS Device which uses mac’s architecture. Use simulator or real device.

1

Undefined symbols for architecture arm64:
  "_USER_ERROR__missing_dlib_all_source_cpp_file__OR__inconsistent_use_of_DEBUG_or_ENABLE_ASSERTS_preprocessor_directives_", referenced from:
      _dlib_check_consistent_assert_usage in DlibWrapper.o
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Solution: https://stackoverflow.com/questions/37208964/error-including-dlib-library-in-c-project add dlib/all/source.cpp to your project sources list(CXXFILES)

2

jpeglib.h file not found

Initially I tried to look for where the was included. I found that in the tutorial it set preprocessor macro DLIB_JPEG_SUPPORT. And in the code, it said

#ifdef DLIB_JPEG_STATIC
#   include "../external/libjpeg/jpeglib.h"
#else
#   include <jpeglib.h>
#endif

I thought it was because that I didn’t compile dlib with this DLIB_JPEG_STATIC set. But I was wrong. Anyway, after about 10 hours try/error, I released that my dlib archive file was in x86_64 architecture. I then went back to the tutorial and read line by line.

After

cmake -G Xcode ..
cmake --build . --config Release

There is a libdlib.a in Release folder. This is not the one I suppose to use. The architecture was by default macos. I should open the Xcode project in build/dlib_build, change the project setting (not the target setting!), use architecture = arm64, create new schema for release, then run it. Now there is a new folder called /build/dlib_build/Release-iphoneos. Inside it there is a new libdlib.a. This is the one I should use. hint: at this time the error was actually symbol not found for arm64.... libdlib.a is built for architecture other than arm64... something like that.

About OpenCV2 for 2016 Macbook Pro

I recently bought the 2016 Macbook Pro. When I set up my OpenCV stuff in this new computer, I experienced great difficulty. Basically I cannot compile the OpenCV2.4.12 library due to lack of QTKit. Many answers on the internet regarding this simply doesn’t work (maybe they work for OpenCV 3).
To be clear, I have no problem installing OpenCV 2 with homebrew. Just I cannot build static *.a files from the library.
Out of no choice, I decided to copy the build folder from my old Air to my new computer. And to archive, I created a git repo for it. https://github.com/XiomarG/OpenCV2.4.12-build

I haven’t tried yet. No sure whether this solution works or not.

Use LibSVM in OpenCV

This post will elaborate how to use libsvm in OpenCV for image classification.

Image analysis always comes down to classification, and classification always comes down to svm.

Why use libsvm instead of CvSVM?

When people try to use svm with OpenCV, they always first come to OpenCV’s built-in svm class names CvSVM. However, CvSVM is developed based on a very old version of libsvm. I am not sure how much difference it makes, but it definitely lacks a very important feature: when making prediction, it only returns the class with highest probability, not the probability of all classes.

My above statement maybe not precise enough. CvSVM does provide a probability-alike prediction with a flag – but only for 2-class classification (CvSVM::predict). But mostly, people need multi-class classification, and we want probabilities for each class because we need to do some post-prediction analysis as the original prediction may not be accurate enough. After some research about tuning CvSVM, it comes to conclusion that CvSVM is incapable of that. The only solution comes to using libSVM.

How to use libsvm offline with OpenCV

I will only explain the very basic use case here. Prerequisite:

download libsvm source code, compile it ($ make), to obtain executables (svm-predict.exec, svm-scale.exec, svm-predict.exec).

LIBSVM is mostly for command line use. To use it, we need two files: one is the training set, and one is the test data. First step is to construct the training file. A training data file is a txt file, with N lines of data. Each line represents a training vector. The structure is like this:

${classLabel} ${attributeIndex}:${attribute} (repeat)   

For example, if vector [0.3, 0.5, 0.7] belongs to class 2, it’s represented in the file as

2 1:0.3 2:0.5 3:0.7

For some reasone I forgot, the attributeIndex starts from 1 instead of 0. libsvm is designed to accommodate sparse matrix, so value zero can be skipped. It means a vector of [0, 0, 0.3, 0, 0, 0.5] in class 1 is represented as

1 3:0.3 6:0.5

After the training matrix is saved as trainingData, we need to train a model. Using the following command

$ ./svm-train -b 1 trainingData

There are some parameters we can set, but here we will use default for all except -b, which is probability_estimates: whether to train a SVC or SVR model for probability estimates, 0 or 1 (default 0). This is exactly what we want from libsvm.

Then we will get the svm model trainingData.model. You can use svm-scale to scale it, but that is not our focus now.

Then create the vector to be predicted. It has the same format as in training vectors. The first value ${classLabel} is not important. But we can use it for verification. If the class label is the same as the result of prediction, it means a successful prediction. So basically we can built a test set, to verify how well the trained model works. Furthermore, we can use this test set to iterate training process to obtain the best svm parameter.

Suppose the test vector file name is testData. To predict, run

$ svm-predict -b 1 testData trainingData.model testResult

This means to use trainingData.model to predict testData and save the result in testResult. For a 5 classes classification, the testResult may look like this

3 0.1 0.1 0.9 0.3 0.4

This is quite self-explanatory. So the work of using libsvm with OpenCV offline is straghtforward.

  1. write training data matrix into a file using libsvm format
  2. run command to train the model
  3. write target vector to file using libsvm format
  4. run command to predict the class
  5. read the result file and process

This is ok if you don’t do prediction that frequently. However, if you use it a lot, as in video processing, it is still far away from feasible. We need some C++ code to do the prediction.

Run svm-predict in your code

If you google libsvm and opencv, you will probably end up with this blog. This is a very good one. It inspired me how to make it work, though its code is not exactly what I want. This code essentially loads the model and target vector from files, then processes them. We just need to make some modification so we don’t need to load target vector, but create the targe vector in code directly. Kuanting’s code illustrated how the target vector in code should look like.

So in my code, I need to predict class by a histogram vector. My prediction function looks like this:

#include "svm.h"
...
void predict(string modelPath, Mat& hist) {

    const char *MODEL_FILE = modelPath.c_str();
    if ((this->SVMModel = svm_load_model(MODEL_FILE)) == 0) {
        this->modelLoaded = false;
        fprintf(stderr, "Can't load SVM model %s", MODEL_FILE);
        return;
    }

    struct svm_node *svmVec;
    svmVec = (struct svm_node *)malloc((hist.cols+1)*sizeof(struct svm_node));
    int j;
    for (j = 0; j < hist.cols; j++) {
        svmVec[j].index = j+1;
        svmVec[j].value = hist.at<float>(0, j);
    }
    svmVec[j].index = -1; // this is quite essential. No documentation.

    double scores[8]; // suppose there are eight classes
    if(svm_check_probability_model(SVMModel)) {
        svm_predict_probability(SVMModel, svmVec, scores);
    }
}

Then scores will save the prediction probability of all 8 classes.

What can be improved

  1. since we already know how to construct the target vector, it should be easy to build the training vectors, which means we can do the training directly in code instead of save to file -> run svm-train.
  2. libsvm’s default parameter of course doesn’t guarantee good result. Grid-search can be done to find the best parameters, but I haven’t tried yet.
How to build machine independent Mac OS app with OpenCV

This blog briefly records how I built a machine-independent Mac OS app which uses OpenCV.

If you want to build an iOS app with OpenCV, you won’t even think about this question: environment dependency. The main reason is that OpenCV provides a framework, which you can easily download and import to your app project. Not to mention that cocoapod supports OpenCV. But if you think you just need to make the same amount of effort for a Mac app, you are wrong.

I had one project that required to process the camera feed in a mac app. I wrote some POC scripts in python, which is fairly easy. But when it comes to Xcode for mac, trouble begins.

Step 1: Install OpenCV

It should be fairly simple to install OpenCV as OSX is just an Unix-like system. I forgot why it took me a while to do that (could not find the right instructions). Just do followings:

  1. download source code from here (any version you like), then inside the source code folder:

  2. run the following commands in sequence:

    $ mkdir build
    $ cd build
    $ cmake ..
    $ make
    $ make install
    

    (Alert: this will cause problem in the following steps. I just want to repeat what I did step by step.)

Now inside /build/lib, you can see a bunch of dynamic libraries like libopencv_calib3d.3.1.0.dylib. Inside the Xcode project setting, inside Linked Frameworks and Libraries, add the libraries you need.

Then, in Build Setting – Search Paths, add /usr/local/include to Header Search Path, and add the absolute_path/to/opencv/build/lib to Library Search Path. Now you should be able to #include "opencv2/opencv.hpp" in your .h file.

There is no tutorial/example online (as far as I know) that guides you how to access each frame and process them. Thanks to that I have done similar things for iOS, after a couple of hours of debug, I got it working. I have a demo project for that.

Step 3: Make the app machine-independent

I forgot this draft for a few days and forgot what I wanted to add previously… anyway, here comes the core part of this article. If you just send this app to another computer and run it, it will fail. If you see the crash log, you will find something like “unable to locate /Users/clarke/Documents/opencv-2.4.12/build/lib/libopencv_calib3d.2.4.dylib (compatibility version 2.4.0, current version 2.4.12)“.

I tried many solutions, like copy all .dylib files to project folder, add them to Build Phases – Copy Bundle Resources, and removed every absolute link in project setting. Still got the same error.

Though I struggled on this for a whole weekend, when solved, it’s quite simple. First of all, the opencv libraries were built as dynamic. Even though I copied them in project setting, they still reference to that absolute location in my computer. So we need to build static libraries. Just a little change in the opencv installation step:

$ mkdir build
$ cd build
$ cmake -DBUILD_SHARED_LIBS=OFF ..
$ make
$ make install

EDIT: (2016-12-15 it seems the latest OSX doesn’t have QTKit installed. When I tried to follow these steps in my 2016 mac book pro, during make it threw error that QTKit.h cannot be found. I googled some solution but none works and they are mostly for OpenCV3. So my guess is only in OpenCV3 it’s fixed. To continue using OpenCV2, I have put my built OpenCV2 with static library to a repository)

Then instead of a bunch of .dylib files. you get .a files. Next, in Xcode project setting, embed these .a files instead of .dylib files. Problem solved? BOOM!

If you search Undefined symbols for architecture x86_64: "_gzclose", reference from ...., the first few answers will talk about the Xcode c++ compiler. In Build Setting – Apple LLVM 7.1 – language C++, change c++ compiler library to compiler default. Unfortunately, it doesn’t help.

The truth is, these libraries inside the /opencv-x.x.x/build/lib/ have dependencies on the third party libraries inside /opencv-x.x.x/build/3rdparty/lib/. Then embed these libraries into your project setting as well. Finally you got a machine-independent OpenCV app.

I didn’t elaborate every details here. So if you have any question, just comment.