The backend, this is where the magic happens; all the processes, from decoding the image to sending the results out, all of this is done by this backend. The backend is straightforward, it accepts the base64 encoded image and sends it to the predictor class which does all the work. What goes on in the predictor class is what this blog is about and with a little bit of computer vision (CNN only).
This is the brain of the web app, it does everything for me. The process of writing this class was pretty smooth because I like python, and I enjoyed the whole process of learning this new stuff 'Flask'. I love flask so much, it's so simple yet so powerful, it is so easy to throw together a running local host on this stuff so fast that it took me more time to restart my laptop when the server crashed than it was for me to throw together the initial 'hello world' on flask. The documentation for flask is super helpful, I would highly suggest flask to anyone trying to make a simple web app without having to learn intimidating libraries. The predictor model of the predictor class ( Canvas class in the repo located under webapp/HWES/canvas_process.py) is built with the help of fastai. Fastai provides a high-level API for quickly creating a deep learning model. I adore PyTorch, what more can a person want than the dynamic computational graphs of PyTorch, but the transfer learning part always seemed a little off to me. I was never successful at getting it to work, they were good but building the model from scratch gave me better results. Then I found out about fastai, the high level and mid-level API of fastai with its ability to seamlessly integrate any other stuff written in PyTorch is just magical. So yeah, then I trained my model on dataset provided on Kaggle by Xainano. I used resnet50 architecture here, this is a bit of an overkill but I wanted to be on the safe side, pretrained on imagenet dataset. I didn't transform the data that much other than random rotation with a max angle of 20°. The reason I didn't many transformations is that at the end the model will be predicting from a stable image i.e the image to be predicted will just be black text with a white background. The learning rate was found using lr_find(), first introduced by Leslie N. Smith in his paper Cyclical Learning Rates for Training Neural Networks, which improved the loss significantly. There are some other minor things as well but they aren't worth mentioning. I am really happy with the prediction of the model, it isn't 100% accurate but with minimal training, I cannot ask for more.
This class is pretty straightforward it just does maths, nothing more. The hardest part was integrating the integration part (pun not intended), converting the user sent string into mathematical equations. For integration I had to first change the string to LaTeX format (I am not entirely sure if it is LaTeX or not) and then perform the operations; performing simple maths was easy I just had to send the maths to eval(). However, working with eval is very not recommended as it performs anything the user enters so if the user enters __import__('os').system(rm -rf *) your project is gone. In this use case, it is fine, as the user can only input inside the canvas which cannot identify alphabets. I am happy with how the class works, also the maths class can be found under webapp/HWES/utils/math_lib.py so yeah feel free to take a look at it.
Yeah, so this is the end of the project blog. I had the most fun during my time at the backend, frontend isn't my type. I like beautiful things, but I like it more when things work. However, I am learning react.js which seems like a great JavaScript library, and Django (I love flask but Django is more easily deployable on the web, or that is what I have heard).