Multiply these two numbers if you are really smart!
Deep learning models are really powerful models. They are called universal function estimators and in this post i will challenge them by calculating a not to much hard function; multiplication of two numbers (<100).
I do not speak much about deep learning, codes or other stuffs here, but this is a regression problem. So first i produce a dataset, then i train the model to output the the target, given two input numbers. Note that multiplication is not an easy function to learn for NNs. Actually, when i started this experiment i was not aware of the works have been done in the subject. But a bit later i found some interesting research on the topic. Heaton (2016) in his research and PhD Thesis empirically studied the subject. His thesis title is “Automated Feature Engineering for Deep Neural Networks with Genetic Programming” and in some experiment he shows that some functions are harder for NNs to be learned such as the ratio of features.
The following notebook contains the code:
We can see that a deep neural network has a hard time to learn the multiplication of two positive numbers (<100). If we repeat the experiment with the multiplication function we will reach to a very low loss with just one output layer. Whenever we face with data that the multiplication of the features can has an impact on the output, we need to help the model using feature engineering. Note that we could add a multiplication layer to the model and reduce the loss very easily.