TensorFlow Template for Deep Learning Beginners
Updated: Jun 3
How to Build Your First Deep Neural Network
What is Deep Learning?
Deep learning is a sub-category of machine learning models that uses neural networks. In a nutshell, neural networks connect multiple layers of nodes/neurons and each node can be considered as a mini machine learning model. The output of the model then feeds as the input of the subsequent node.
TensorFlow is a Python library that primarily focuses on providing deep learning framework.
To install and import TensorFlow library:
pip install tensorflow import tensorflow as tf
How to Build a TensorFlow Deep Neural Network?
The skeleton of a deep learning models generally follows the structure below and we can use Keras API to implement a beginner friendly deep learning model. There is a lot of variation we can add at each stage to make the model more complex.
Define the model
Compile the model
Fit the model
Evaluation and Prediction
Grab the code template from our Code Snippet section.
1. Prepare Dataset
Deep learning is fundamentally machine learning algorithms and consists of both supervised learning and unsupervised learning. For a supervised learning, it requires splitting the dataset into train and test set (sometimes also involve a validation set) as below.
from sklearn.model_selection import train_test_split X = df.drop(['user-definedlabeln'], axis=1) y = df['user-definedlabeln'] X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42)
There is a lot more you can do with the raw dataset, such as preprocessing and feature engineerings, but let’s keep it simple in this article.
2. Define the Model
One of the the simplest form of deep learning neural network is sequential model, which is composed of a single stack of layers and each layer has only one input tensor and one output tensor. We can create a sequential model by passing multiple Dense layers.
model = tf.keras.models.Sequential([ keras.layers.Dense(units=32, activation="sigmoid"), keras.layers.Dense(units=16, activation="sigmoid"), keras.layers.Dense(units=1, activation="ReLU") ])
number of layers and units: a deep learning model must have an input layer and an output layer. The number of hidden layers between input and output can vary. The number of units per layer is also a hyperparameter that we can experiment on. The article “How to Configure the Number of Layers and Nodes in a Neural Network” provides a guide on how to experiment and perform search to determine number of layers and nodes.
activation function: each layer of the model requires an activation function. You can think of it as a mini statistical model that transforms the node input into output. Activation functions contribute to the non-linearity of neural network algorithm. Hidden layers usually apply the same activation functions and output layer can have a different one depends on the prediction type.
Below are common activation functions and each has its pros and cons.
Sigmoid: ranging from 0 to 1, sigmoid is suitable for binary output
ReLU: it preserves linear behavior and solves the issue of vanishing gradient in Sigmoid and Tanh, but it can suffer from other problems like saturated or “dead” units when the input is negative
Tanh: the stronger gradient makes it more sensitive to small difference, however it has the issue of saturation and slow learning rate at extreme values
Linear: it is suitable for regression problem with continuous numeric output