【转】

TensorFlow is an open source software library for machine learning, developed by Google and currently used in many of their projects.

Aneasy, fast, and funway to get started with TensorFlow is to build an image classifier: an offline and simplified alternative toGoogle’s Cloud Vision APIwhere our Android device can detect and recognize objects from an image(or directly from the camera input).

In this article, we will create an Android app that can recognize video-game characters.

First, run the Android classifier sample

An official TensorFlow Android image classifier sample is available on the mainGitHub repository.

However, if you want to build it, it will take you some time, as you’ll need to install the NDK, Bazel, and the total build time with Android Studio will take around 40 minutes.

That’s not really compatible with my initial “easy, fast, and fun” description, so instead I’ve created a gradle standalone prebuilt fork of TensorFlow 1.0.1 atgithub.com/Nilhcem/tensorflow-classifier-androidyou can directly clone, import on Android Studio and run within 2 minutes.

OPTIONAL (begin)

If you want to port the TensorFlow official Android Sample to a prebuilt Android Studio project by yourself, you will need to:

Import the original TensorFlow Android sample sources (commit)

Create a new Android Studio project (applicationId:org.tensorflow.demo), to have an already set up gradle configuration (commit)

Move TensorFlow’s Android sample files into the new gradle project (commit)

Remove the DetectorActivity and StylizeActivity (we will only need the ClassifierActivity today) (commit)

Download the CI nightly prebuilt NDK librariesfrom this linkand place those into your gradle project (commit)

Download thepre-trained ImageNet model (inception5h)to your assets folder (commit)

OPTIONAL (end)

Once you have imported the TensorFlow Android sample, run it with Android Studio and start detecting things

Then, create your own image classifier

Right now, the TensorFlow sample uses “Inception”, a model pre-trained to detect 1000 objects fromImageNet2012 Challenge image dataset.

We are going to transfer learning, which means we are starting with a model that has been already trained on another problem. We will then be retraining it on a similar problem. Deep learning from scratch can take days, but transfer learning can be done in short order.

And for that, we’ll need some data….

1. Gather lots of images

Inception works well with a various set of images (at least 30 images, more is better).

We will create a~/tf_files/gamesfolder and place each set of jpeg images in subdirectories (such as~/tf_files/games/mario,~/tf_files/games/bombermanetc)

A quick way to download multiple images at once is to search something on Google Images, and use aChrome extension for batch download.

2. Retrain the model to learn from your images

Now that we have all our images, we will retrain the model.

We can use a docker container to have an already set up TensorFlow environment:

docker run -it -v$HOME/tf_files:/tf_files\gcr.io/tensorflow/tensorflow:latest-develcd/tensorflowgit pullgit checkout v1.0.1python tensorflow/examples/image_retraining/retrain.py\--bottleneck_dir=/tf_files/bottlenecks\--how_many_training_steps 4000\--model_dir=/tf_files/inception\--output_graph=/tf_files/retrained_graph.pb\--output_labels=/tf_files/retrained_labels.txt\--image_dir /tf_files/games

Don’t stop your container yet, you will need it again very soon.

This operation can take several minutes depending on how many images you have and how many training steps you specified.

These commands will make TensorFlow download the inception model and retrain it to detect images from~/tf_files/games.

The script will generate two files: the model in a protobuf file (retrained_graph.pb) and a label list of all the objects it can recognize (retrained_labels.txt).

For more information, the best tutorial you can find on the Internet so far is Google’sTensorFlow for Poetscodelab (I highly recommend you to do it)

3. Optimize the model

We have our model. However, if we try to import it in our Android sample, we will get an error:

Op BatchNormWithGlobalNormalization is not available in GraphDef version 21. It has been removed in version 9. Use tf.nn.batch_normalization().

We first need to optimize it first, using a tool namedoptimize_for_inference:

./configure# you can select all default valuesbazel build tensorflow/python/tools:optimize_for_inference

Building the tool should take around 20 minutes.#BePatient

OPTIONAL (begin)

At this point, a good thing to do, to save time, would be to commit your current docker process to a new image. This way, you won’t have to rebuild theoptimize_for_inferencetool anymore.

exitdocker ps -adocker commit # And then start your new container with:docker run -it -v$HOME/tf_files:/tf_files cd/tensorflow

OPTIONAL (end)

Now the tool is built, you can optimize your model:

bazel-bin/tensorflow/python/tools/optimize_for_inference\--input=/tf_files/retrained_graph.pb\--output=/tf_files/retrained_graph_optimized.pb\--input_names=Mul\--output_names=final_result

You can now exit your docker container.

This script will generate a~/tf_files/retrained_graph_optimized.pbfile you will now be able into import in your Android project.

For more information about mobile optimizations, I suggest you to read theTensorflow for Mobile Poetsarticle

4. Import the new model in your Android application

We have our retrained model.

We can now delete the previous ImageNet model from our Android app’sassetsfolder and place the new model (~/tf_files/retrained_graph_optimized.pband~/tf_files/retrained_labels.txt) instead.

Also, we should update some constants in theClassifierActivity.java, as specified in a comment in this same file:

privatestaticfinalintINPUT_SIZE=299;privatestaticfinalintIMAGE_MEAN=128;privatestaticfinalfloatIMAGE_STD=128;privatestaticfinalStringINPUT_NAME="Mul";privatestaticfinalStringOUTPUT_NAME="final_result";privatestaticfinalStringMODEL_FILE="file:///android_asset/retrained_graph_optimized.pb";privatestaticfinalStringLABEL_FILE="file:///android_asset/retrained_labels.txt";

5. Test the trained AI

We have installed the new AI model. Now, we can deploy the project on an Android device, and have fun detecting objects using our new retrained AI:

How about Android Things?

Using TensorFlow on Android Things is a little bit easier than on Android.

There is already an official gradle sample project that works out-of-the-box with the ImageNet model, and which can be deployed quickly:github.com/androidthings/sample-tensorflow-imageclassifier/.

And if you want to use your own custom model instead, you will simply need to do exactly the same steps explained in this article (Android Things is Android, after all):

Place your custom model (.pband.txtfiles) in the assets directory

Remove the.aarlib and use TensorFlow’s latest libs instead

Modify the constants in theTensorFlowImageClassifier.javafile

(I shamelessly stole this LCD picture idea from@riggaroo’s latest conference. Don’t tell her.)

You can watch the custom classifier in action, on thisYouTube video.

The LCD driver used is the1602 LCD module driver.

Conclusion (a.k.a. why would we need that?)

In this post, you saw something completely useless: using deep learning to classify video game characters, but you could do something funnier, like adab detectorshifumi game, for instance.

Better, you could create a robot that changes its behaviour and its way of talking according to who’s in front of it (a child / an adult).

You can use deep learning algorithm to identify skin cancer, or detect defective pieces and automatically stop a production line as soon as possible.

Why not even create a sign language recognizer?

Have fun!

（Original Link：http://nilhcem.com/android/custom-tensorflow-classifier）