Evaluating the model Recall (R) $R = \frac{TP}{TP + FN}$ i.e. number of true values “recalled” out of total true values Precision (P) $P = \frac{TP}{TP + FP}$ i.e. How precise you were in identifying positive cases. Covid RTPCR Test has high precision and low recall. This means if the test result is Covid positive, there is a high chance that it is correct but if it says negative, nothing can said with certainty. ...
Notes on Reinforcement Learning
Reinforcement Learning can do wonders but selecting the best algorithm and then tuning it’s hyper-parameters is not an easy task. This blog post aims at making the process easier by listing out a few general suggestions and a list of hyper-parameters to tune for each algorithm.
Misc
A collection of practical tips and best practices for deep learning, covering model training, optimization, and deployment.
A quick guide on screen resolutions.
Docker can be used to quickly try out a new software stack with lot of dependencies but one is generally forced to google the essential docker commands and arguments every time a new container is to be launched. This cheat sheet aims to list the important steps to build and launch a docker container and the arguments used with it. Hope it helps!
Dockerfile Create a file in your project folder named Dockerfile with the following contents FROM nvidia/cuda:10.2-cudnn7-runtime-ubuntu18.04 ENV PATH="/root/miniconda3/bin:${PATH}" ARG PATH="/root/miniconda3/bin:${PATH}" WORKDIR /root RUN apt update && \ apt install -y \ htop \ python3-dev \ wget \ gcc \ libopenmpi-dev \ cmake \ zlib1g-dev \ libjpeg-dev \ libsdl2-dev \ xvfb RUN wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh && \ mkdir /.conda && \ sh Miniconda3-latest-Linux-x86_64.sh -b && \ rm -f Miniconda3-latest-Linux-x86_64.sh RUN conda create -y -n myenv python=3.8 ADD requirements.txt /root/requirements.txt RUN /bin/bash -c "source activate myenv \ && pip install -r requirements.txt" # Copy files in current path to workdir COPY . . # CMD is executed when docker container is started # CMD ["/bin/bash", "ping google.com"] CMD bash Create a docker image from the docker file docker build -t image_name:tag /path/to/Dockerfile/ eg: docker build -t my_python_image:1.0 /home/light/my_project FROM Can select a prebult image to build upon ...
Huang et al. showed that mixed precision training is 1.5x to 5.5x faster over float32 on V100 GPUs, and an additional 1.3x to 2.5x faster on A100 GPUs on a variety of networks. On very large networks the need for mixed precision is even more evident. Narayanan et al. reports that it would take 34 days to train GPT-3 175B on 1024 A100 GPUs (with a batch size of 1536), but it’s estimated it would take over a year using float32! ...
NAS One of the key challenges in designing deep learning models is finding the task specific architecture. This involves choosing the number of layers, the number of neurons per layer, and the connections between different layers. Typically, this is done through a process of trial and error, where the researcher trains multiple architectures to find the best among. This can be time-consuming and may not always yield the best possible results. There are various methods to automatically find a better architecture using RL, evolution techniques, Gradient Descent, etc. Google claims to use NAS in BERT, Google Pixel 6a face recognition, Waymo to get better accuracy and speed. ...