Inprogresses

Misc

A quick guide on screen resolutions.

Dockerfile Create a file in your project folder named Dockerfile with the following contents FROM nvidia/cuda:10.2-cudnn7-runtime-ubuntu18.04 ENV PATH="/root/miniconda3/bin:${PATH}" ARG PATH="/root/miniconda3/bin:${PATH}" WORKDIR /root RUN apt update && \ apt install -y \ htop \ python3-dev \ wget \ gcc \ libopenmpi-dev \ cmake \ zlib1g-dev \ libjpeg-dev \ libsdl2-dev \ xvfb RUN wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh && \ mkdir /.conda && \ sh Miniconda3-latest-Linux-x86_64.sh -b && \ rm -f Miniconda3-latest-Linux-x86_64.sh RUN conda create -y -n myenv python=3.8 ADD requirements.txt /root/requirements.txt RUN /bin/bash -c "source activate myenv \ && pip install -r requirements.txt" # Copy files in current path to workdir COPY . . # CMD is executed when docker container is started # CMD ["/bin/bash", "ping google.com"] CMD bash Create a docker image from the docker file docker build -t image_name:tag /path/to/Dockerfile/ eg: docker build -t my_python_image:1.0 /home/light/my_project FROM Can select a prebult image to build upon ...

Huang et al. showed that mixed precision training is 1.5x to 5.5x faster over float32 on V100 GPUs, and an additional 1.3x to 2.5x faster on A100 GPUs on a variety of networks. On very large networks the need for mixed precision is even more evident. Narayanan et al. reports that it would take 34 days to train GPT-3 175B on 1024 A100 GPUs (with a batch size of 1536), but it’s estimated it would take over a year using float32! ...

NAS One of the key challenges in designing deep learning models is finding the task specific architecture. This involves choosing the number of layers, the number of neurons per layer, and the connections between different layers. Typically, this is done through a process of trial and error, where the researcher trains multiple architectures to find the best among. This can be time-consuming and may not always yield the best possible results. There are various methods to automatically find a better architecture using RL, evolution techniques, Gradient Descent, etc. Google claims to use NAS in BERT, Google Pixel 6a face recognition, Waymo to get better accuracy and speed. ...