Deep Learning

These are my notes on refreshing my object detection knowledge. We will start with bounding boxes for localization and cover everything we need before jumping in to implement YOLO algorithms. This tutorial includes answers to the following questions: What is localization? What are a bounding box and sliding window? How to measure the success of a predicted bounding box: Intersection over the union. How to get rid of extra bounding boxes: Non-max suppression. Evaluation Metric for Object Detection: Mean average precision Check references for addresses of the images. Object Detection Object Detection is finding objects in an image and where they are located in the image. Adding localization or location on detected objects for a classification task will give us object detection. ...

This post includes my notes from the lecture “Makemore Part 3: Activations & Gradients, BatchNorm” by Andrej Karpathy. Link of the video: https://www.youtube.com/watch?v=P6sfmUTpUmc Initialization Fixing the initial Loss: Initial loss must be arranged (the value depends on the question), in our case its a uniform probability. When initializing make sure the numbers do not take extreme values (* .01) Do not initialize to 0 Having 0 and 1 in softmax (a lot of them) is really bad, since the gradient will be 0 (vanishing gradient). This is called saturated tanh. So in summary: What we did is basically just making sure initially we give the network random values such that it is varied, and not making gradients 0 (no dead neurons). In the case of tanh, when we use softmax to squash the values, if the initial values were too broad, we will get a lot of vanishing gradients due to values ending up above 1 or below -1. So we first reduce the initial values and then use softmax on them (and continue training process). Kaiming Init Okay we know how to fix initialization now, but how much should we reduce these numbers? Meaning what is the value we should scale the layers with. Here comes Kaiming init. ...