Review: Keras sails through deep learning

Martins-Retina-MacBook:~ martinheller$ python
Python 2.7.10 (default, Oct  6 2017, 22:29:07)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.31)] on darwin
Type “help”, “copyright”, “credits” or “license” for more information.
>>> import keras
Using TensorFlow backend.
>>> from keras.models import Sequential
>>> from keras.layers import Dense, Dropout, Activation
>>> from keras.optimizers import SGD
>>>
>>> import numpy as np
>>> x_train = np.random.random((1000, 20))
>>> y_train = keras.utils.to_categorical(np.random.randint(10, size=(1000, 1)), num_classes=10)
>>> x_test = np.random.random((100, 20))
>>> y_test = keras.utils.to_categorical(np.random.randint(10, size=(100, 1)), num_classes=10)
>>>
>>> model = Sequential()
>>> model.add(Dense(64, activation=’relu’, input_dim=20))
>>> model.add(Dropout(0.5))
>>> model.add(Dense(64, activation=’relu’))
>>> model.add(Dropout(0.5))
>>> model.add(Dense(10, activation=’softmax’))
>>>
>>> sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
>>> model.compile(loss=’categorical_crossentropy’,
...               optimizer=sgd,
...               metrics=[‘accuracy’])
>>> model.fit(x_train, y_train,
...           epochs=20,
...           batch_size=128)
Epoch 1/20
1000/1000 [==============================] - 0s 319us/step - loss: 2.3804 - acc: 0.0910
Epoch 2/20
1000/1000 [==============================] - 0s 12us/step - loss: 2.3517 - acc: 0.0860
Epoch 3/20
1000/1000 [==============================] - 0s 13us/step - loss: 2.3573 - acc: 0.1000
Epoch 4/20
1000/1000 [==============================] - 0s 16us/step - loss: 2.3303 - acc: 0.0990
Epoch 5/20
1000/1000 [==============================] - 0s 15us/step - loss: 2.3177 - acc: 0.1090
Epoch 6/20
1000/1000 [==============================] - 0s 11us/step - loss: 2.3180 - acc: 0.1050
Epoch 7/20
1000/1000 [==============================] - 0s 14us/step - loss: 2.3125 - acc: 0.1310
Epoch 8/20
1000/1000 [==============================] - 0s 12us/step - loss: 2.3068 - acc: 0.1330
Epoch 9/20
1000/1000 [==============================] - 0s 11us/step - loss: 2.3066 - acc: 0.0970
Epoch 10/20
1000/1000 [==============================] - 0s 13us/step - loss: 2.2954 - acc: 0.1100
Epoch 11/20
1000/1000 [==============================] - 0s 12us/step - loss: 2.3065 - acc: 0.1110
Epoch 12/20
1000/1000 [==============================] - 0s 11us/step - loss: 2.3057 - acc: 0.1140
Epoch 13/20
1000/1000 [==============================] - 0s 11us/step - loss: 2.2993 - acc: 0.1200
Epoch 14/20
1000/1000 [==============================] - 0s 13us/step - loss: 2.2978 - acc: 0.1240
Epoch 15/20
1000/1000 [==============================] - 0s 12us/step - loss: 2.2989 - acc: 0.1230
Epoch 16/20
1000/1000 [==============================] - 0s 11us/step - loss: 2.3001 - acc: 0.1180
Epoch 17/20
1000/1000 [==============================] - 0s 12us/step - loss: 2.2892 - acc: 0.1210
Epoch 18/20
1000/1000 [==============================] - 0s 12us/step - loss: 2.2993 - acc: 0.1060
Epoch 19/20
1000/1000 [==============================] - 0s 12us/step - loss: 2.2990 - acc: 0.1110
Epoch 20/20
1000/1000 [==============================] - 0s 12us/step - loss: 2.2936 - acc: 0.1200
<keras.callbacks.History object at 0x112847990>
>>> score = model.evaluate(x_test, y_test, batch_size=128)
100/100 [==============================] - 0s 537us/step
>>> print (score)
[‘2.2815165519714355’, ‘0.1599999964237213’]

Here Keras is reporting a categorical cross-entropy loss of 2.28 and an accuracy of 16 percent from the evaluation of the model on the test data. That would be bad, except for the fact that the inputs were random—no model could fit them.

Deploying Keras

Keras models can be deployed across a great range of platforms—perhaps greater than any other deep learning framework. That includes iOS, via CoreML; Android, via the TensorFlow Android runtime; in a browser, via Keras.js and WebDNN; on Google Cloud, via TensorFlow-Serving; in a Python webapp backend; on the JVM, via DL4J model import; and on Raspberry Pi.

Keras applications, data sets, and examples

Keras supplies seven of the common deep learning sample data sets via the keras.datasets class. That includes cifar10 and cifar100 small color images, IMDB movie reviews, Reuters newswire topics, MNIST handwritten digits, MNIST fashion images, and Boston housing prices.

Keras also supplies 10 well-known models pre-trained against ImageNet: Xception, VGG16, VGG19, ResNet50, InceptionV3, InceptionResNetV2, MobileNet, DenseNet, NASNet, and MobileNetV2TK. You can use these models to predict the classification of images, extract features from them, and fine-tune the models on a different set of classes.

By the way, fine-tuning existing models is a good way to speed up training. For example, you can add layers as you wish, freeze the base layers to train the new layers, then unfreeze some of the base layers to fine-tune the training. You can freeze a layer by setting layer.trainable = False.

The Keras examples repository contains more than 40 sample models. They cover vision models, text and sequences, and generative models.

If I were starting a new deep learning project today, I would most likely do the research with Keras. Keras is really about as simple as it could be, given that the hard part of building deep neural network models is finding a network topology that fits the data as accurately as possible without overfitting.

—

Cost: Free open source under the MIT license.

Platform: Linux, MacOS, Windows, or Raspbian; TensorFlow, Theano, or CNTK back end.