Today I Learned

A collection of concise write-ups on small things I learn across a variety of languages and technologies.

Categories

• CleanCode [3]
• Go [8]
• Miscellaneous [17]
• Python [10]
• Shell [20]
• WebDev [4]

CleanCode

• Naming Variables & Functions
  Read More 🔽

  Naming Variables & Functions

  1. The name of a variable, function, or class should answer all the big questions. It should tell you why it exists, what it does, and how it is used.
  2. Classes & Objects should have noun or noun phrase names like Student,Account etc. A class name should not be a verb.
  3. Methods should have verb or verb phrase names like getCategories(), saveTutorial().

  ---

  PS : I have been reading CleanCode for a while & logging what I learn here.

• Writing Comments
  Read More 🔽

  Writing Comments

  1. The proper use of comments is to compensate for our failure to express ourself in code.
  2. Inaccurate comments are far worse than no comments at all.
  3. Comments do not make up for the bad code. i.e if you find yourself writing comments for a code that is complex to understand. MAKE IT LESS COMPLEX.
  4. Short functions don't need much description. A well-chosen name for a small function that does one thing is usually better than a comment header.
  5. For example. In this code the comments are not needed bcoz the fucntion name describes what it is doing.

  def get_category_list():
      ''' Walk the current directory and get a list of all subdirectories at that
      level.  These are the "categories" in which there are TILs. '''
      dirs = [x for x in os.listdir('.') if os.path.isdir(x) and '.git' not in x]
      return dirs

  Summary

  Until now I thought writing comments is a good practice & we should write comments wherever possible. But know reading it is an eye opener & shocking 😱. I will try to avoid comments now.

  ---

  PS : I have been reading CleanCode for a while & logging what I learn here.

  

• Writing Functions
  Read More 🔽

  Writing Functions

  Got to learn some new points regarding functions() in CleanCode.

  1. Functions should be small.
  2. They should do one thing only.

     FUNCTIONS SHOULD DO ONE THING. THEY SHOULD DO IT WELL. THEY SHOULD DO IT ONLY.

  3. To know if a function is doing more than "one thing" see if you can extract another funtion from it with a name that is not merely a restatement of its implementation.
  4. Function arguments should NEVER be greater than 3.
  5. We should never ignore any part of code.The parts we ignore are where the bugs will hide.

  ---

  PS : I have been reading CleanCode for a while & logging what I learn here.

  

Go

• Add version info in Go projects
  Read More 🔽

  Add version info in Go projects

  Go offers a nice way to specify version information at compile time using the -ldflags flag in go build command.

  How to use it effectively ?

  Just declare a -v flag in your application using the flag package

  // yourapp.go
  import (
      "os"
      "flag"
  )
  
  var AppVersion string = "dev"
  
  func main() {
  
      Version := flag.Bool("v", false, "Prints Current AreYouOk Version")
      if *Version {
        fmt.Println(AppVersion)
        os.Exit(0)
      }
  }

  Now compile this & provide the value for AppVersion variable at compile time

  go build -ldflags="-X 'main.AppVersion=v1.0.0'"

  Test if it works by running ./yourapp -v

  

• Clearing terminal screen in Go
  Read More 🔽

  Clearing terminal screen in Go

  There are two ways I like (without any external dependency)

  Using os/exec package

  I have added some boilerplate code to make sure you see whats happening. I think this is probably the best way to do this.

  package main
   
  import (
      "os"
      "fmt"
      "os/exec"
      "time"
  )
   
  func main() {
  	fmt.Println("this is a line")
  	fmt.Println("this is a line")
  	fmt.Println("this is a line")
  	fmt.Println("this is a line")
  	fmt.Println("this is a line")
  
  	fmt.Println("Clearing Screen in 2s...")
  
  	// sleep for 2 seconds
  	duration, _ := time.ParseDuration("2s")
  	time.Sleep(duration)
      
      c := exec.Command("clear")
      c.Stdout = os.Stdout
      c.Run()
  }

  Using ANSI Escape Sequences

  Not a good way but may come in handy for some situations.

  package main
   
  import (
      "fmt"
      "time"
  )
   
  func main() {
  	fmt.Println("this is a line")
  	fmt.Println("this is a line")
  	fmt.Println("this is a line")
  	fmt.Println("this is a line")
  	fmt.Println("this is a line")
  
  	fmt.Println("Clearing Screen in 1s...")
  	dur, _ := time.ParseDuration("1s")
  	time.Sleep(dur)
  
      fmt.Print("\033[2J")
  }

  The sequence \033[2J is read as Esc[2J where "2j" clears the screen and moves the cursor to the home position (line 0, column 0).

• Convert `string` to `int` and vice-versa in Go
  Read More 🔽

  Convert string to int and vice-versa in Go

  There are basically 2-3 methods to convert integer to string and back to integer but the most easiest way is to use the Itoa and Atoi methods in the strconv package.

  Demo

  package main
  
  import (
      "fmt"
      "strconv"
  )
  
  func main() {
  	// String to Int
  	nice, _ := strconv.Atoi("69")
  	fmt.Printf("%T", nice)
  
  	//Integer to String
  	high := strconv.Itoa(420)
  	fmt.Printf("\n%T", high)
  }

  Atoi returns a second value (err ignored in this example for easier understanding)

  The above code should output:

  int
  string
  

  See online demo

  

• Creating Python's `next()` alternative using Go Closures
  Read More 🔽

  Creating Python's next() alternative using Go Closures

  If you don't know what next() in python means, the below code illustrates it.

  MCU_Movies = iter(["Iron Man", "Thor", "Captain America: The first Avenger"])
  x = next(MCU_Movies)
  print(x, end="\n")
  x = next(MCU_Movies)
  print(x, end="\n")
  x = next(MCU_Movies)
  print(x)

  So if you had guess this would print

  Iron Man
  Thor
  Captain America: The first Avenger

  The next() function is used to get the next item in an iterator.

  Go doesn't have a next method (nor the concept of iterators actually) so we will try to achieve something similar using Closures.

  • A closure is implemented through a anonymous(function with no name) function, basically closure is an instance of function.
  • In Go functions are first class citizens, meaning we can do all sort of things with them, assign them to a variable, pass as an argument to another function.

  Below is a naive implementation of how this could look in Go. Ping me if you have a better way to do this ;)

  package main
  
  import "fmt"
  
  /*
  nextIterator returns another function, which we define anonymously in the body of nextIterator.
  The returned function closes over the variable index to form a closure.
  */
  func nextIterator(array []int) func() int {
      index := -1
  
      return func() int{
          index++
          return array[index]
      }
  }
  func main() {
  
  	// an integer array
      var prices = []int{7, 1, 5}
  
      // create an instance of the anonymous function. i.e, a closue
      next := nextIterator(prices)
  
      // call the closure
      fmt.Println(next())
      fmt.Println(next())
      fmt.Println(next())
  
  }

  See this demo on Go Playground.

• Measure Exection time in Go
  Read More 🔽

  Measure Exection time in Go

  To know who how long your go code executes you can use the time.Now() and time.Since() methods in the time package.

  Demo

  package main
  
  import (
  	"fmt"
  	"time"
  )
  
  func main(){
  
  	start := time.Now()
  	dur, _ := time.ParseDuration("15ms")
  
  	// A Go Anonymous function (self-executing)
  	func (){
  		for i := 0; i < 100; i++ {
  			time.Sleep(dur)
  			fmt.Println("Bhupesh is programming in Go")
  		}
  	}()
  
  	elapsed := time.Since(start)
  
  	fmt.Printf("Execution Time : %s", elapsed)
  }

  Since() returns the time elapsed since t (start in our demo). It is shorthand for time.Now().Sub(t).

  Here is the output of the above code You can also play with the online demo

  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Bhupesh is programming in Go
  Execution Time : 750ms
  

  

• Reading & Setting Environment variables in Go
  Read More 🔽

  Reading & Setting Environment variables in Go

  Use os.Getenv() and os.Setenv() for reading and setting environment variables.

  Demo

  package main
  
  import (
  	"fmt"
  	"os"
  )
  
  func main() {
  	os.Setenv("NAME", "gopher")
  
  	fmt.Printf("My Desktop Environment : %s.\n", os.Getenv("XDG_CURRENT_DESKTOP"))
  }

  

• Splitting strings in Go
  Read More 🔽

  Splitting strings in Go

  Splitting strings in Go is done by using the Split() method. You need to import the strings standard library to use this.

  Demo

  package main 
  
  import (
  	"fmt"
  	"strings"
  )
  
  func main(){
  	var date string = "1999-03-12"
  	date_array = strings.Split(date, "-")
  
  	fmt.Println(date_array)
  }

  The split() return a Go Array, running this program should print the following:

  [1999 03 12]
  

  

• Where are my build files when I use `go run`
  Read More 🔽

  Where are my build files when I use go run

  By default, 'go run' runs the compiled binary directly. The binaries are stored in a temp work folder, to see where they are stored use the -work flag.

  Demo

  go run --work fizzbuzz.go

  Sample Output

  WORK=/tmp/go-build645222420
  [1 2 Fizz]
  

  When you run this go will not delete the temporary build when exiting. The default directory may vary with your system & GOPATH.

Miscellaneous

• Converting videos to High quality GIFs
  Read More 🔽

  Converting videos to High quality GIFs

  Converting videos to gifs using ffmpeg is a pain in the ass if you don't know what's happening. GIF size getting 10x the size of original video ? Don't worry, I got you!

  1. Always create a custom palette
  2. Don't increase/decrease file dimensions
  3. Save unnecessary frame conversion by using -t to convert video until timestamp.
  4. Experiment with fps (default value is 24)

  # Get video dimensions
  ffprobe -v error -select_streams v:0 -show_entries stream=width,height -of csv=s=x:p=0 video.mp4
  # generate a palette
  ffmpeg -i video.mp4 -vf "fps=22,scale=1024:-1:flags=lanczos,palettegen" palette.png
  # use the generated palette
  ffmpeg -t 29 -i video.mp4 -i palette.png -filter_complex "fps=22,scale=1024:-1:flags=lanczos[x];[x][1:v]paletteuse" output.gif

  Download complete script here

  Resources

  • High quality GIF from video

  

• Creating a Regex Compiler/Parser - Research
  Read More 🔽

  Creating a Regex Compiler/Parser - Research

  Making a regex parser/compiler is not simple as it sounds, here is the overview of the steps:

  1. Convert the expression to Postfix notation.

     While converting to postfix, you also need to handle Character Classes & Range Quantifiers. Tutorials on internet haven't done this. Read this for an insight on how to do this.

  2. Convert the postfix in above step to AST.
  3. Convert the AST to a state machine, preferably a NFA (non-deterministic finite automata)

  Resources

  • Regular Expression Matching Can Be Simple And Fast by Russ Cox
  • Converting Regular Expressions to Postfix Notation with the Shunting-Yard Algorithm
  • Postfix to NFA using Thompson algorithm
  • Regex Compiler: Part 2
  • Regular Expressions Based on Nondeterministic Finite Automaton
  • Using NFA to evaluate regular expressions
  • How to implement regular expression NFA with character ranges?
  • Regex under the hood: Implementing a simple regex compiler in Go

  

• Deploying to Heroku
  Read More 🔽

  Deploying to Heroku

  List of steps to follow when you are deployihg a new repository/project (Python).

  1. heroku login
  2. touch Procfile Create Procfile for deployment. For a Django Web-App the contents of Procfile would be.

  gunicorn djangoherokuapp.wsgi --log-file -
  

  3. touch runtime.txt Specify your Python version here. For example

  python-3.6.8
  

  4. heroku create herokuAPPName Before running this, Make sure to add appname.herokuapp.com in ALLOWED_HOSTS and your requirements.txt is updated.

  List of commands to run when you are deployihg a cloned repository.

  1. heroku login Login with your e-mail and password.
  2. heroku git:remote -a <app-name> Where app-name is the name of app on heroku.
  3. git push heroku master Push new changes to heroku.
• Docker 🐋 quick guide
  Read More 🔽

  Docker 🐋 quick guide

  1. Remove docker image.

     docker rmi <img-id>

  2. Remove docker container.

     docker rm <container-id>

  3. Build a docker image with a name.

     docker build -f <dockerfile-path> -t name .

  4. Run a container.

     docker run -p 3000:3000 <container-id>

  5. Stop a container.

     docker stop <container-id>

  6. Run a container in detach mode (run in background).

     docker run -d <container-id>

     For the love of God always add a -d while running a container. Speaking this from experience.

     If you don't run in detach mode, you won't be able to Ctrl+C (or exit), instead use Ctrl+PC (yes the P key and C key).

  7. List docker volumes.

     docker volume ls

  8. Remove docker volume.

     docker volume rm <volume-name>

  9. Check port mapping.

     docker port <name>

  10. Starting a docker container

      docker start <container>

      The first two letters of CONTAINER_ID can be provided as an argument too.

  11. Run a command inside container.

      docker container exec <CONTAINER> ls -la

  12. Check history of an image.

      docker history <IMAGE>

  Docker Compose

  1. Build and run containers.

     docker-compose up -d

  2. Stop containers.

     docker-compose stop

  3. Check logs/console messages.

     docker-compose logs <image name>

  4. List all containers.

     docker-compose ps

  

• Internet search tips & tricks for developers
  Read More 🔽

  Internet search tips & tricks for developers

  All of the below mentioned tips works in both DuckDuckGo & Google (i use both 😉), it should work fine in other search engines too.

  1. file:pdf golang

     Use it to search for books, presentations or specific file types

  2. inurl:docs.djangoproject.com templates

     Use it to look for occurence of some phrases in the URL of the website mentioned. inurl docs.djangoproject.com look for templates phrase.

  3. site:github.com synatx error

     Limit search results to a specific site, good for looking for bug fixes.

  4. "how to add pagination in django"

     Double quotes can be used for exact matches of the phrase (doesn't work sometimes).

  5. related:http://freecodecamp.org

     Related specifier, "related:" returns the root domain of similar websites

  6. intitle:best vim plugins

     Intitle specifier returns results that contains your searched word in the title.

  

• Killing Open Ports in Linux
  Read More 🔽

  Killing Open Ports in Linux

  I had this weird error while running Django Development Server.

  System check identified no issues (0 silenced).
  January 27, 2020 - 16:42:39
  Django version 2.2.9, using settings 'codeclassroom.settings'
  Starting development server at http://127.0.0.1:8000/
  Quit the server with CONTROL-C.
  Error: That port is already in use.
  

  Solution

  1. Run netstat -ntlp to see available open ports.

  (Not all processes could be identified, non-owned process info
   will not be shown, you would have to be root to see it all.)
  Active Internet connections (only servers)
  Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
  tcp        0      0 127.0.0.1:42405         0.0.0.0:*               LISTEN      -                   
  tcp        0      0 127.0.0.1:5940          0.0.0.0:*               LISTEN      -                   
  tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      -                   
  tcp        0      0 127.0.0.1:631           0.0.0.0:*               LISTEN      -                   
  tcp        0      0 127.0.0.1:8000          0.0.0.0:*               LISTEN      12054/python3       
  tcp6       0      0 ::1:631                 :::*                    LISTEN      -  
  

  Here is a man netstat for what it does

  netstat - Print network connections, routing tables, interface statistics, masquerade connections, and multicast memberships

  You see the python3 one? Kill 😈 this process

  2. Run kill -9 12054 Kill is used for Removing a background process or job, -9 specifies SIGKILL (Forced termination) where 12054 is the PID

  3. Run the development server again.

  

• My Vim Cheatsheet
  Read More 🔽

  My Vim Cheatsheet

                           
   _______ _______ _______ 
  |\     /|\     /|\     /|
  | +---+ | +---+ | +---+ |
  | |   | | |   | | |   | |
  | |v  | | |i  | | |m  | |
  | +---+ | +---+ | +---+ |
  |/_____\|/_____\|/_____\|
                           
  

  I have started transitioning slowly to lightweight editors, because of my low system configuration. And what can be better than vim. I will keep a log of things I learn in the process.

  Update: I started using vim "actively" from Nov 12, 2020 and it has now been 1 month complete in Vim & I don't think I am moving to another text editor in future.

  For starters I use neovim (v0.4.4).

  All my Plugins & Colorschemes are listed in my dotfiles

  Super Basic Stuff

  Some must know stuff filtered from the vast array of vim things.

  Editing

  • i insert text before the cursor position
  • a append text after the cursor position (my advice, always use this instead of i)
  • A append text at end of line
  • o open a new line after current line
  • O open a new line before current line
  • x delete character under cursor
  • D delete until the end of line
  • r replace the character under cursor
  • R replace stuff until we want
  • dd Delete current line.
  • cc delete current line and switch to insert mode.
  • C delete everything from the cursor position to the EOL.

  Basic Stuff

  Undo/Redo/Repeat

  • u : Undo latest changes in vim.
  • Ctrl + r : to redo
  • . : repeat last change in vim.

  Cut/Copy/Paste

  I felt like a rookie when I used to search this, anyways here is how you do it:

  1. Enable visual mode by pressing v.
  2. Use arrow keys to select text.
  3. Use d to Cut. OR
  4. Use y to yank (copy) text (only inside vim)

     :"+y : for yanking(copying) text to system's clipboard.

  5. Use p to paste after the cursor position or P to paste before the cursor.

     :"+p : to paste from system's clipboard

  Search & Replace

  1. Move your cursor to the desired word

  2. Use * to select all its occurrences.

  3. Hit Esc and use :%s//<replace-word>/ to replace all the selected words.

     :nohlsearch : for clearing search highlighting. Also read (:h usr_12.txt), section 12.2 for a nice overview on search.

     When in search mode instead of hitting Enter use Ctrl + g and Ctrl + t to traverse matches while still being in search mode.

  Intermediate Stuff

  1. :earlier N : Time travel in past N seconds.
  2. :later N : Time travel in future N seconds.
  3. :echo $MYVIMRC : to view location of your default .vimrc file.
  4. Use == in Visual Mode to fix line indent.
  5. When in command mode (:), use Ctrl + f to browse through your command history, live edit any command and hit enter to run it (the quick fix window).
  6. Use :resize 60 to resize windows horizontally or :vertical resize 60 for vertical resizing. Also signed values can be used like +5, -2.
  7. Use :right, :left or :center to align text. Assuming width of document is textwidth (default is 80). You can also specify arguments for e.g :center 100 will move the start of line to 100th column.
  8. To list all your active/inactive buffers, use :buffers in command mode. You can switch to a buffer by providing the buffer name, :buffer <TAB> to see all buffers.
  9. Use :verb map <key> to check which key is mapped to what operation. Useful when debugging your mappings and differentiating them from that of a plugin.

     Read help for checking key notations :h key-notation

  10. Use vim's wildignore setting to exclude searching for files and directories according to your project. For e.g for python projects this could look like

      set wildignore+=*/.git/*,*/site-packages/*,*/lib/*,*/bin/*,*.pyc

      This should exclude searching through your virtual environments [Read manual :h 'wildignore']. Another handy trick is to exclude media files from appearing in search by excluding them as well.

      set wildignore+=*.jpg,*.bmp,*.gif,*.png,*.jpeg,*.avi,*.mp4,*.mkv,*.pdf,*.odt

  11. :syntax will output all highlight groups for syntax highlighting of the current open file. It can come handy when you are writing your own colorscheme.
  12. Scrolling 2 or more windows together. When in multiple windows (or splits), you can use scrollbind. Pick one window then :set scb, pick another window :set scb for disabling use :set noscb
  13. To search for pattern in vim help text use :helpgrep or :helpg
  14. If you have spell-checking (:set spell) enabled use zg to exclude certain words from being reported as misspelled. This adds the words to your own list of words called a spellfile. On NeoVim this fill is created automatically, although you can do it manually.

      mkdir -p ~/.vim/spell/

      then in vimrc

      set spellfile=~/.vim/spell/en.utf-8.add

  15. Use q: to open command line history or Ctrl + f when already in command mode
  16. Use q/ to open search history, this will list all the things you searched using search mode /. Press i to change anything and <CR> to execute again.
  17. To quickly jump to function definition or variable assignments under cursor use gd(local declaration) or gD(global declaration)
  18. To reselect the last visual selection use gv.
  19. When in visual mode use gU to make text uppercase & gu to lowercase.

  Code Folding

  It helps you view only a selected range of text. (Read :h usr_28.txt for a quick overview)

  Quick settings to put in vimrc/init.vimrc

  set foldmethod=indent
  set foldcolumn=2

  You can also setup foldmethod based on file type

  • za: Toggle code folding.
  • zR: Open all folds.
  • zM: Close all folds.
  • zo: Open current fold.
  • zc: Close current fold.

  Navigation

  • w jump through beginning of words in a line
  • e jump to end of words in a line
  • b to move backward
  • H jump to top of text under screen (not to be confused with top of file).
  • M jump to middle
  • L jump to bottom
  • gg go to top of file
  • GG go to end of file
  • 0 go to beginning of line
  • $ go to end of current line
  • ^ go to first character in a line
  • g_ go to last character of the line
  • zb put current line at bottom of screen
  • zt put current line at top of screen
  • Ctrl+f scroll down 1 page
  • Ctrl+b scroll up 1 page

  Character Wise

  • f : find next
  • F : find backward
  • t : find next char & place cursor before
  • T : find next char & place cursor before backward
  • ; : go to the next occurrence of f/t
  • , : go to previous occurrence of f/t

  Completions

  Use Ctrl + x +

  1. f = File name completion
  2. l = Whole line completion (context aware, handy if you are copy pasting a previously typed line)
  3. i = Keywords in current & included file ("include" means when you import or #include)
  4. s = Spelling suggestions
  5. k = Keywords from dictionary. For this to work add set dictionary+=/usr/share/dict/words to your vimrc

  use :help ins-completion to see more such completions

  Registers

  Take registers as "special vim storage locations". There are exactly 21 registers which store different kind of stuff, from these 4 registers are read-only. In command mode use :di or :reg to display contents of all these registers. Do h registers to read the docs

  File Browsing

  Vim has a default file browser called netrw, below are some handy tips that will help:

  1. R : rename a file/directory.
  2. qf : Show file info.
  3. x : open file in associated program, use it open media files like images.
  4. Ctrl + l : refresh netrw, Opens a new buffer. Use :e . instead.
  5. d : Make a new directory.
  6. gh : toggle display of hidden files.
  7. D : Delete a file/directory (Doesn't work on non-empty directories).

  Ex mode

  It let's you run commands repetitively without using :.

  Use Q to enter into Ex mode, vi or visual to go back.

  The Ex mode in Vim is quite underrated in 2020 since we have a :term but learning about it can be quite helpful sometimes.

  ---

  I will only add stuff here when I start using it or use it for the first time.

  

• Record your Desktop using `ffmpeg`
  Read More 🔽

  Record your Desktop using ffmpeg

  1. Make sure you have ffmpeg installed, by checking ffmpeg -version. If not install use sudo apt-get install ffmpeg.

  2. Run the following command.

  ffmpeg -video_size 1280x1024 -framerate 25 -f x11grab -i :0.0+0,0 -c:v libx264rgb -crf 0 -preset ultrafast output.mkv

  • -video_size specifies the size of the recorded area. If you have a different screen size, use that instead of 1920x1080. If you want to record only an area of the screen, specify the area size here.

  • -framerate specifies the frame rate, i. e. how many frames of video are recorded in a second. The lowest allowed framerate is 20.

  • -f x11grab is what actually tells FFmpeg to record your screen. You shouldn't change that.

  • -i :0.0+0,0 is where you specify the x and y offset of the top left corner of the area that you want to record. For example, use :0.0+100,200 to have an x offset of 100 and an y offset of 200.

  • -c:v libx264rgb -crf 0 -preset ultrafast are encoding options. These specify a fast and lossless recording.

  Run xdpyinfo | grep 'dimensions:' to know your monitor dimensions

  Resources

  • Record-Your-Desktop-Using-FFmpeg-on-Ubuntu-Linux
• Releases on GitHub
  Read More 🔽

  Releases on GitHub

  Git tagging is generally used to release software on github. Here are some basic git commands for tagging.

  • To tag specific points of your repo. Run this when you commit something.

    git tag -a v1.4 -m "my version 1.4"

  • To lists all the tags of your repo.

    git tag

  • To tag specific commits.

    git tag -a v1.4 9fceb02

  • To push tags on GitHub.

    git push origin v1.4

  

• SEO Tools
  Read More 🔽

  SEO Tools

  Here is a list of some tools to test your website for SEO, Rich Snippets and Social Media.

  • Facebook's Sharing Debugger.
  • Twitter Card Validator.
  • LinkedIn Post Inspector
  • web.dev
  • Structured Data Testing
  • Structured Data Markup Helper
  • Rich Results Test
  • JSONLD.com

  

• Searching your way through vim
  Read More 🔽

  Searching your way through vim

  Matching a word "exactly"

  /\<hello world\>/

  \< and \> mark the start and end of a whole word resp.

  Searching between 2 words (inclusive)

  /red\&.*blue/

  This will highlight everything b/w "red" and "blue" on the same line.

  \& also called as "Branch" matches the last pattern but only if all the preceding patterns match at the same position

  Example: Find f-strings f"\&.*"

  Search between 2 words on different lines.

  hello\_.*world

  This will highlight everything between hello world.

  Example: Find try/catch block hello\_.*world

  "\_.*except" matches all text from the current position to the last occurrence of "except" in the file.

  Read :help pattern.txt for everything related to pattern searching.

  

• Semantic Versioning
  Read More 🔽

  Semantic Versioning

  • Describes how softwares are assigned version numbers.
  • We generally see 3 parts in a version number, x.y.z (MAJOR, MINOR, PATCH)
    1. x represents MAJOR part - meant for describing any major backend code changes, support of APIs etc.
    2. y represents MINOR part - meant for describing very small changes.
    3. z represents PATCH part - meant for describing bug fixes.

  Resources

  • Semantic Versioning 2.0.0
• Streaming videos, things behind the curtain
  Read More 🔽

  Streaming videos, things behind the curtain

  This is a log of things I learn on how streaming "videos" works & what are the modern ways companies do it and possibly everything about it.

  • TCP is more appropriate for serving video on demand

  • Live streaming via TCP/IP, then it would be forced to wait for dropped packets before it could continue processing newer data. That's not good because:

    1. Old data will be re-transmitted (that's probably for a frame that was already displayed and therefore worthless).
    2. New data can't arrive until after old data was re-transmitted.

  • UDP is ideal for teleconferencing.

  • QUIC protocol for transport layer over UDP. It is fast, secure and reliable. It builds on top of UDP

  • The <audio> and <video> tags are protocol agnostic, no browser currently supports anything other than HTTP without requiring plugins, although this looks set to change. Protocols other than HTTP may also be subject to blocking from firewalls or proxy servers.

  • Netflix and other streaming providers make extensive use of distributed content delivery networks (CDN), which store content in locations around the world that are much closer to users.

  • Adaptive Streaming: Quality of video is automatically chosen based on user's network and processing capabilities. (think YouTube's Auto setting. DASH)

  Protocols

  1. DASH Used by YouTube, Netflix or Amazon Prime Video (and many others). DASH’ manifest is called the Media Presentation Description (or MPD) and is at its base XML. It’s an adaptive bitrate streaming technique that enables high-quality streaming of videos over the web from conventional HTTP web servers. Via this technique, the content is made available to the viewer at different bit rates. YouTube client automatically adapts the video rendering as per the internet connection speed of the viewer thus cutting down the buffering as much as possible.
  2. HLS Developed by Apple, used by DailyMotion, Twitch.tv, and many others. The HLS manifest is called the playlist and is in the m3u8 format (which are m3u playlist files, encoded in UTF-8).
  3. Smooth Streaming Developed by Microsoft, used by multiple Microsoft products and MyCanal.

  Resources

  • TCP vs UDP on video stream
  • Live streaming web audio and video
  • How video streaming works on the web: An introduction

  

• What's a Procfile 👀
  Read More 🔽

  What's a Procfile 👀

  I recently deployed a Python application on Heroku, there I encountered a Procfile. This is what I got to know :

  • The Procfile is a simple text file that is named Procfile without a file extension. For example, Procfile.txt is not a valid Procfile.
  • It specifies the commands that are executed by the app on startup. For e.g A Django server.
  • Example: If you want to run a python script on Heroku, your Procfile content should be worker: python script.py

  Resources

  • The Procfile
• Writing Cleaner Commits - Template
  Read More 🔽

  Writing Cleaner Commits - Template

  Writing cleaner commits is hard, so I use this template which makes me a pro 😅

  # If applied, this commit will...
  # [Add/Fix/Remove/Update/Refactor/Document] [issue #id] [summary]
  
  
  # Why is it necessary? (Bug fix, feature, improvements?)
  -
  # How does the change address the issue? 
  -
  # What side effects does this change have?
  -
  
  

  OR

  # If applied, this commit will...
  # [Add/Fix/Remove/Update/Refactor/Document]
  
  # Reference any issue number here
  - This fixes #454
  # Why is it necessary? (Bug fix, feature, improvements?)
  -
  # How does the change address the issue? 
  -
  

  How ?

  You have to configure Git to use the above template file (for example, .gitmessage in your home directory), then create the template file by running.

  git config --global commit.template ~/.gitmessage
  subl ~/.gitmessage

  This will invoke sublime with the template (use code if you use VSCode) Now copy paste the above template, hit save and your are done.

  Now when commiting changes instead of using git commit -m "", Use git commit this will invoke the commit template which you already set.

  Resources

  • How to Write a Git Commit Message
  • Git commit practices your future self will thank you for
• Writing Cover letter - Tips
  Read More 🔽

  Writing Cover letter - Tips

  I am not a hiring expert but I think these tips can help someone stand ahead of the crowd. This might also help when you are cold-emailing a recruiter.

  1. Keep the letter to 2 paragraphs. The 1st para highlighting WHY do you want to work there & 2nd para highlighting HOW hiring you is going to benefit them.
  2. Do research about the company. Go through their social, blogs etc. Yes this is the time to be a stalker on internet
  3. Always add a personal touch while explaining why do you want to work with them. This is the time to be as "thoughtful" as you can.
  4. Don't use any copy-paste templates, not worth it. If a company is asking for a cover letter means that they know a Resume has limitations, they want to hear the WHY & HOW of hiring you is going to help them.
  5. Explain something exceptional/cool you did.

  This is a progressive post, I will add/delete as I learn. Suggestions welcome

  

• vim surround: quick cheatsheet
  Read More 🔽

  vim surround: quick cheatsheet

  Adding ys (you surround)

  ys takes a valid vim motion or text object as first argument

  • ysiw" : surround inner word with "
  • yss] : surround current line with ]
  • ys2aw* : add * around 2 words

  The vS mapping

  • vS" : visually surround selected text with "

  Spicy Stuff

  • yswt : prompt & surround with a html tag

  If a t or < is specified, Vim prompts for an HTML/XML tag to insert (also supports adding attributes)

  • yswf : prompt & surround with a function call

  If a f is specified, Vim prompts for an function name to insert (use F to add space around ())

  Changing cs (change surround)

  Takes 2 arguments a target to replace & a replacement character

  • cs]) : change surrounding [] with ()

  Deleting ds (delete surround)

  • ds" : delete surrounding "
  • dst : delete surrounding tag (HTML)

  Do :helpg surround for more docs.

  Thanks tpope!

  

Python

• Check indentation errors in python 🐍
  Read More 🔽

  Check indentation errors in python 🐍

  Use tabnanny in python standard library for this.

  python -m tabnanny hack-nasa.py

  

• Converting 📅⏲ datetime object to string & vice-versa
  Read More 🔽

  Converting 📅⏲ datetime object to string & vice-versa

  Sometimes only the standard lib datetime is necessary to do all date/time related tasks.

  I am logging this here because I always forget how to do this 😶 (I want to end this once and for all)

  datetime object to string

  >>> from datetime import datetime
  >>> d = datetime.now()
  >>> d
  datetime.datetime(2020, 11, 7, 20, 15, 58, 389341)
  >>> d.strftime("%d %b, %Y")
  '07 Nov, 2020'

  From string to datetime object

  >>> from datetime import datetime
  >>> date_string = "2020-06-20T08:22:54Z"
  >>> datetime.strptime(date_string, '%Y-%m-%dT%H:%M:%SZ')
  datetime.datetime(2020, 6, 20, 8, 22, 54)

  Both strptime & strftime can ofc be chained.

  >>> from datetime import datetime
  >>> date_string = "2020-06-20T08:22:54Z"
  >>> datetime.strptime(date_string, '%Y-%m-%dT%H:%M:%SZ').strftime("%d %b, %Y")

  This can be handy when you are getting date/time fields from external resource (like an API) and only want to display a part of it (like days, month etc).

  Format codes for strptime and strftime

  Directive	Meaning	Example
  %a	Weekday as locale’s abbreviated name.	Sun, Mon, …, Sat (en_US); So, Mo, …, Sa (de_DE)
  %A	Weekday as locale’s full name.	Sunday, Monday, …, Saturday (en_US); Sonntag, Montag, …, Samstag (de_DE)
  %w	Weekday as a decimal number, where 0 is Sunday and 6 is Saturday.	0, 1, …, 6
  %d	Day of the month as a zero-padded decimal number.	01, 02, …, 31
  %b	Month as locale’s abbreviated name.	Jan, Feb, …, Dec (en_US); Jan, Feb, …, Dez (de_DE)
  %B	Month as locale’s full name.	January, February, …, December (en_US); Januar, Februar, …, Dezember (de_DE)
  %m	Month as a zero-padded decimal number.	01, 02, …, 12
  %y	Year without century as a zero-padded decimal number.	00, 01, …, 99
  %Y	Year with century as a decimal number.	0001, 0002, …, 2013, 2014, …, 9998, 9999
  %H	Hour (24-hour clock) as a zero-padded decimal number.	00, 01, …, 23
  %I	Hour (12-hour clock) as a zero-padded decimal number.	01, 02, …, 12
  %p	Locale’s equivalent of either AM or PM.	AM, PM (en_US); am, pm (de_DE)
  %M	Minute as a zero-padded decimal number.	00, 01, …, 59
  %S	Second as a zero-padded decimal number.	00, 01, …, 59
  %f	Microsecond as a decimal number, zero-padded on the left.	000000, 000001, …, 999999
  %z	UTC offset in the form ±HHMM[SS[.ffffff]] (empty string if the object is naive).	(empty), +0000, -0400, +1030, +063415, -030712.345216
  %Z	Time zone name (empty string if the object is naive).	(empty), UTC, GMT
  %j	Day of the year as a zero-padded decimal number.	001, 002, …, 366
  %U	Week number of the year (Sunday as the first day of the week) as a zero padded decimal number. All days in a new year preceding the first Sunday are considered to be in week 0.	00, 01, …, 53
  %W	Week number of the year (Monday as the first day of the week) as a decimal number. All days in a new year preceding the first Monday are considered to be in week 0.	00, 01, …, 53
  %c	Locale’s appropriate date and time representation.	Tue Aug 16 21:30:00 1988 (en_US); Di 16 Aug 21:30:00 1988 (de_DE)
  %x	Locale’s appropriate date representation.	08/16/88 (None); 08/16/1988 (en_US); 16.08.1988 (de_DE)
  %X	Locale’s appropriate time representation.	21:30:00 (en_US); 21:30:00 (de_DE)
  %%	A literal '%' character.	%

  
• Cryptographically strong random string
  Read More 🔽

  Cryptographically strong random string

  One liner

  python3 -c "import secrets;print(secrets.token_urlsafe())"

  Sample Runs

  >>> import secrets
  >>> secrets.token_urlsafe()
  'noLCpWgg5bJbALwlqAKKWUcb4nZg0LvxIUFHyhyei-I'
  >>> secrets.token_urlsafe()
  '8HhV5FMm2vxfrSoO9o_v65FRy6bLbvc89POSX0fnMqk'
  >>> secrets.token_urlsafe()
  'bClPydJqA7_0GsDvUAqqShUH5ZucWzdErg0tZIGZU2k'
  >>> secrets.token_urlsafe()
  '82LSHzCKkwo5y__3NZrck27ZbDL1WiKoSYxQQki8uvA'
  >>> 

  

• Difference b/w Class & Function Based Views in Django
  Read More 🔽

  Difference b/w Class & Function Based Views in Django

  Function Based Views	Class Based Views
  More setup Less Abstraction Requires error checking Less modular	Little setup More "magic" abstraction Error handling built-in (generics) Much more modular Sane and stable generic API

  

• Fastest Python First: Tips and Tricks 🏃
  Read More 🔽

  Fastest Python First: Tips and Tricks 🏃

  Here are some tips that I learned to make Python Programs faster (WIP).

  1. Always compile regular expression, if you have to match patterns a lot of times.

  import re
  
  pattern = re.compile(some_regular_expression)
  some_text = re.sub(pattern, '', data)

  2. Use List comprehension for iterating over a list.

  import time
  
  itemlist = [23, 45, 56, 67, 1, 100, 340, 90]
  """ Normal Iteration """
  start_time = time.time()
  for item in itemlist:
  	if item % 2 == 0:
  		print(item)
  end_time = time.time()
  
  print("Without List Comprehension : " + str(end_time - start_time))
  
  """ List Comprehension """
  start_time = time.time()
  [print(item) for item in itemlist if item % 2 == 0]
  end_time = time.time()
  
  print("With List Comprehension : " + str(end_time - start_time))

  Output :

  56
  100
  340
  90
  Without List Comprehension : 0.0002067089080810547
  56
  100
  340
  90
  With List Comprehension : 0.00019121170043945312

  

• Functional Programming in Python 🐍
  Read More 🔽

  Functional Programming in Python 🐍

  Features like lambda, map, filter, reduce are generally used to perform functional programming related tasks in Python. Let's take a quick look on them.

  Lambdas

  • Anonymous functions
  • No function name,
  • They can be passed as function arguments/objects.
  • No return statment, evaluated exrpession is returned automatically.
  • Single line function.

  Example :

  double = lambda x: x*x
  print(double(34))
  
  elementList = [34, 56, 78, 90, 0, 12]
  doubleList = lambda elementList: [e*e for e in elementList]
  print(doubleList(elementList))

  Map

  • applies a function to all the items in an input list.
  • map(function_to_apply, list_of_inputs).

  Example :

  myList = ["bhupesh", "varshney", "is", "a", "developer"]
  
  capitalize = list(map(lambda x: x.upper(), myList))
  print(capitalize)

  Filter

  • creates a list of elements for which a function returns True.

  Example :

  mylist = [23, 45, 6, 8, 10, 16]
  evenList = list(filter(lambda x: x%2 == 0, mylist))
  print(evenList)

  Reduce

  • accepts a function and a sequence(list/set etc) and returns a single value calculated.
  • Initially, the function is called with the first two items from the sequence and the result is returned.
  • The function is then called again with the result obtained in step 1 and the next value in the sequence. This process keeps repeating until there are items in the sequence.

  Example :

  from functools import reduce
  
  li = [5, 8, 10, 20, 50, 100]
  
  sum = reduce((lambda x, y: x + y), li) 
  print(sum)

  

• PEP8 - the fashion 💃 police of Python
  Read More 🔽

  PEP8 - the fashion 💃 police of Python

  Well basically PEP8 is a style guide which provides guidelines and best practices for writing python code.

  How I learn?

  Well bascially the official Python docs for PEP8 seems good but I use pep8.org. It provides much more cleaner interface.

  Summary

  Below is a summary which includes some go-to rules you can memorize.

  1. Use 4 spaces per indentation level.
  2. Spaces are preferred instead of tabs (Why ?? 😥) Python disallows mixing of Tabs & Spaces (Syntax Errors). So be consistent with what you choose, I prefer tabs 😉
  3. Limit all lines to a maximum of 79 characters. Use \ to break/continue line.
  4. Never use the characters ‘l’ (lowercase letter el), ‘O’ (uppercase letter oh), or ‘I’ (uppercase letter eye) as single character variable names. These are misunderstood with numerals one and zero in some font styles.
  5. Function Names - lowercase words separated by _ .
  6. Class Names - Start each word with a capital letter. Use CamelCase E.g StudentClass

  Tools

  If use Sublime Text, you can install Python PEP8 Autoformat - it does the job for you.

• Publishing a Package on PyPI
  Read More 🔽

  Publishing a Package on PyPI

  I just published my first package on pypi 😍 I used the following steps to do it :

  1. Put your python files/classes inside the folder package-name.Also make sure your main class file has the same name package-name.

  2. Add the __init__.py file in the same folder. Use the init file like this.

  from coderunner.coderunner import Run

  3. Now make a file setup.py inside the root of your github folder. Add the following contents in it:

  import setuptools
  
  with open("README.md", "r") as fh:
      long_description = fh.read()
  
  setuptools.setup(
      name="plagcheck",
      version="0.1",
      author="Bhupesh Varshney",
      author_email="[email protected]",
      description="A Powerful Moss results scrapper",
      keywords='mosspy moss plagiarism cheat',
      long_description=long_description,
      long_description_content_type="text/markdown",
      url="https://github.com/codeclassroom/PlagCheck",
      project_urls={
          "Documentation": "https://github.com/codeclassroom/PlagCheck/blob/master/docs/docs.md",
          "Source Code": "https://github.com/codeclassroom/PlagCheck",
          "Funding": "https://www.patreon.com/bePatron?u=18082750",
          "Say Thanks!": "https://github.com/codeclassroom/PlagCheck/issues/new?assignees=&labels=&template=---say-thank-you.md&title=",
      },
      packages=setuptools.find_packages(),
      install_requires=[
          'requests',
          'mosspy',
          'beautifulsoup4',
          'lxml',
      ],
      classifiers=[
          "Programming Language :: Python :: 3",
          "License :: OSI Approved :: MIT License",
          'Topic :: Software Development :: Build Tools',
          "Operating System :: OS Independent",
      ],
      python_requires='>=3.6',
  )

  4. Now make a file setup.cfg. It is used for displaying project description on PyPi.

  [metadata]
  description-file = README.md

  5. Install the followig libraries.

  pip3 install setuptools wheel twine

  6. Run the following command.

  python3 setup.py sdist bdist_wheel

  7. Finally upload it to PyPi.

  twine upload dist/*

  This will prompt for your PyPi username and password.

  Resources

  • Packaging Python Projects
  • How to upload your python package to PyPi

  

• Specify dev dependencies in setup.py
  Read More 🔽

  Specify dev dependencies in setup.py

  # setup.py
  ...
  
  extras_require = {
      "dev": [
          "pytest>=3.7",
      ],
  }

  Test it locally

  pip install -e .[dev]

  

• Writing Unit Tests in Python ✅
  Read More 🔽

  Writing Unit Tests in Python ✅

  1. Simple and easy just import the Python 3 built-in library unittest.
  2. Wrap up tests in a Class.
  3. Use assert methods.

  import unittest
  
  class TestSum(unittest.TestCase):
  
      def test_sum(self):
          self.assertEqual(sum([1, 2, 3]), 6, "Should be 6")
  
      def test_sum_tuple(self):
          self.assertEqual(sum((1, 2, 2)), 6, "Should be 6")
  
  if __name__ == '__main__':
      unittest.main()

  Resources

  • Getting Started With Testing in Python
  • unittest — Unit testing framework

Shell

• Alternative to 'ls' commnand
  Read More 🔽

  Alternative to 'ls' commnand

  the bash builtin echo can be used to list contents of a directory.

  echo *

  List all files that start with letter 'i'.

  echo i*

  How

  The character * serves as a "wild card" for filename expansion in globbing. By itself, it matches every filename in a given directory.

  Resource

  • globbing
• Colorize Output in Terminal
  Read More 🔽

  Colorize Output in Terminal

  The ANSI escape sequences help adding color to the terminal (Doesn't work on Windows I guess)

  Here is the format:

  \033[<ForegroundColorCode>;<BackgroundColorCode>;<Style>mYour Text

  where

  <ForegroundColorCode>, <BackgroundColorCode> & <Style> are integer Color Codes. See Resources for list of colors.

  Example :

  echo -e "\033[92;1mHello\033[0m"

  Hello will be bold & green in color

  Resources

  • List of ANSI color escape sequences
  • Bash tips: Colors and formatting (ANSI/VT100 Control sequences)

  

• Copy one file to multiple files in Bash
  Read More 🔽

  Copy one file to multiple files in Bash

  for f in file{1..10}.py; do cp main.py $f; done

  this will create new files file_1.py, file_2.py etc and copy contents of main.py file to all of them.

• Extract file id from drive shareable link
  Read More 🔽

  Extract file id from drive shareable link

  I host my blog images on Google Drive sometimes, the normal shareable link is not the actual image source. Instead this is :

  https://drive.google.com/uc?export=view&id=<INSERT-ID>

  INSERT_ID is the file id (in the shareable link)which is higlighted below

  https://drive.google.com/file/d/1ifRiquoNw3awVTX6geyNoDp8FW6tafOE/view?usp=sharing

  here is a bash script to convert the link.

  #!/usr/bin/env bash
  
  str="$1"
  # remove everything after the last /
  remove_last=${str%/*}
  # get everything after the last /
  get_last=${remove_last##*/}
  echo "https://drive.google.com/uc?export=view&id=$get_last"

  Also a Python version

  You can now use it in <img> src

• Find boot-up time in linux 🐧
  Read More 🔽

  Find boot-up time in linux 🐧

  We can achieve this using the systemd service. Just run this

  systemd-analyze

  Demo:

  Startup finished in 36.655s (kernel) + 58.030s (userspace) = 1min 34.685s
  graphical.target reached after 57.709s in userspace

  The graphical.target specifies how long it took to reach to the log-in screen.

  Other tricks

  1. You can also plot service initializations in a SVG graph.

  systemd-analyze plot > demo.svg

  2. Check which service takes most of the time.

  systemd-analyze blame

  

• Find default git branch name
  Read More 🔽

  Find default git branch name

  git remote show origin | awk '/HEAD/ {print $3}'

  

• Find files changed 7 days ago
  Read More 🔽

  Find files changed 7 days ago

  To find last modified file

  find Documents/ -mtime -1

  where mtime means "Last Modification Time"

  To find files Accessed (read operation)

  find Documents/ -atime -7

  where atime means "Last Access Time"

  -7 signifies anything changed 7 days or less. +7 signifies anything changed atleast 7 days ago. 7 signifies anything changed exactly 7 days ago.

  an alternative version

  find Documents/ -newermt "7 days ago" -ls

  

• Find fonts available in Linux
  Read More 🔽

  Find fonts available in Linux

  fc-list

  

• Finding all Python Virtual Environments in your system
  Read More 🔽

  Finding all Python Virtual Environments in your system

  So if you work with Python all day, you already know about Virtual Environments. The only problem with me 😅 is that I end up creating a lot of them when using and making my side-projects. We know that there is a activate script that can be used for this purpose.

  shut up & give me answer

  Ok

  Using find

  find /home -name "*activate"

  This will list out all activate scripts in your home directory (should work fine). Only problem, it is slow.

  Sample Ouput

  /home/bhupesh/Desktop/testFind/bin/activate
  /home/bhupesh/Desktop/bits/bin/activate
  /home/bhupesh/Desktop/cc-new/bin/activate
  /home/bhupesh/Desktop/test-audio/bin/activate
  /home/bhupesh/Desktop/comp-code/bin/activate
  /home/bhupesh/Desktop/req-blog/bin/activate
  /home/bhupesh/Desktop/py-cli/bin/activate
  /home/bhupesh/Desktop/sian-env/bin/activate
  /home/bhupesh/Desktop/ques/bin/activate
  /home/bhupesh/Documents/api/bin/activate
  /home/bhupesh/Documents/defe-venv/bin/activate
  /home/bhupesh/Documents/bot-tutorial/bin/activate
  /home/bhupesh/Documents/cc/bin/activate
  /home/bhupesh/Documents/test-package/bin/activate
  /home/bhupesh/Documents/qt-lear/bin/activate
  /home/bhupesh/Documents/csv-voil/bin/activate
  /home/bhupesh/Documents/bottest/bin/activate
  /home/bhupesh/Documents/new-tutorialdb/bin/activate
  /home/bhupesh/Documents/cc2/bin/activate
  /home/bhupesh/Documents/plag/bin/activate
  find /home -name "*activate"  2.57s user 4.14s system 7% cpu 1:31.72 total
  

  Using locate

  locate -b '\activate' | grep "/home"

  Grep your home directory for all activate scripts.

  Sample Output

  /home/bhupesh/Desktop/bits/bin/activate
  /home/bhupesh/Desktop/cc-new/bin/activate
  /home/bhupesh/Desktop/comp-code/bin/activate
  /home/bhupesh/Desktop/py-cli/bin/activate
  /home/bhupesh/Desktop/ques/bin/activate
  /home/bhupesh/Desktop/req-blog/bin/activate
  /home/bhupesh/Desktop/sian-env/bin/activate
  /home/bhupesh/Desktop/test-audio/bin/activate
  /home/bhupesh/Desktop/testFind/bin/activate
  /home/bhupesh/Documents/api/bin/activate
  /home/bhupesh/Documents/bot-tutorial/bin/activate
  /home/bhupesh/Documents/bottest/bin/activate
  /home/bhupesh/Documents/cc/bin/activate
  /home/bhupesh/Documents/cc2/bin/activate
  /home/bhupesh/Documents/csv-voil/bin/activate
  /home/bhupesh/Documents/defe-venv/bin/activate
  /home/bhupesh/Documents/new-tutorialdb/bin/activate
  /home/bhupesh/Documents/plag/bin/activate
  /home/bhupesh/Documents/qt-lear/bin/activate
  /home/bhupesh/Documents/test-package/bin/activate
  locate -b '\activate'  0.45s user 0.02s system 99% cpu 0.476 total
  grep --color=auto --exclude-dir={.bzr,CVS,.git,.hg,.svn,.idea,.tox} "/home"  0.00s user 0.00s system 0% cpu 0.472 total
  

  Now you can just do source <path>.

• Finding the file creation date/time on Linux
  Read More 🔽

  Finding the file creation date/time on Linux

  1. Find inode number of file.

  $ ls -i myfile.md
  9344160 myfile.md

  2. Find name of your root partition

  $ df -h
  Filesystem      Size  Used Avail Use% Mounted on
  udev            924M     0  924M   0% /dev
  tmpfs           191M  1.4M  190M   1% /run
  /dev/sda1       146G   38G  101G  28% /
  tmpfs           954M  121M  833M  13% /dev/shm
  ...

  3. Use the inode no in stat & debugfs

  sudo debugfs -R 'stat <9344160>' /dev/sda1

  Look for crtime, that is our file creation date/time

  Here is a one liner if your filesystem is ext4

  sudo debugfs -R "stat <$(ls -i myfile.md | awk '{ print $1}')>" /dev/sda1 | grep 'crtime'

  • debugfs is a ext2, ext3, ext4 file system debugger. The -R flag causes debugfs to execute a single command. In our case that command is stat which is used to check file status. We need to run debugfs where our filesystem is mounted, i.e /dev/sda1.

  Note: this will not work on older file systems. Although this has changed with modern file systems such as ext4 & Btrfs which has been designed to store file creation time.

  

• Generate random numbers in bash
  Read More 🔽

  Generate random numbers in bash

  Easiest way is to use the $RANDOM variable.

  >> echo "$RANDOM"
  12261

  Each time this parameter is referenced, a random integer between 0 and 32767 is generated.

  A better way which I like is to use the GNU coreutil, shuf

  One Random number between 69 & 420

  shuf -i69-420 -n1

  Five Random numbers between 69 & 420

  shuf -i69-420 -n5

  

• Get System info using Shell Commands
  Read More 🔽

  Get System info using Shell Commands

  Memory Used/Total

  free -h | awk '/^Mem:/ {print $3 "/" $2}'

  Show CPU temperature

  sensors | awk '/^Core*/ {print $1$2, $3}'

  Most Memory Intensive processes

  ps axch -o cmd:15,%mem --sort=-%mem | head

  Most CPU Intensive processes

  ps axch -o cmd:15,%cpu --sort=-%cpu | head

  I wrote a small shell script to get (almost) realtime update of your system.

• Get Total System Memory using `vmstat` command
  Read More 🔽

  Get Total System Memory using vmstat command

  vmstat -s | grep "total memory" | grep -Eo '[0-9]{1,}'

  This will print the total memory (your RAM) in highlighted text.

  The command vmstat -s is usually used to print memory statistics a sample output might look like

     1882140 K total memory
      644068 K used memory
      861172 K active memory
      653200 K inactive memory
      217160 K free memory
       55140 K buffer memory
      965772 K swap cache
     2097148 K total swap
      230400 K used swap
     1866748 K free swap
      169316 non-nice user cpu ticks
        4939 nice user cpu ticks
       37944 system cpu ticks
      666678 idle cpu ticks
       53315 IO-wait cpu ticks
           0 IRQ cpu ticks
         693 softirq cpu ticks
           0 stolen cpu ticks
     2554778 pages paged in
     1429680 pages paged out
       40722 pages swapped in
      191481 pages swapped out
     3487312 interrupts
    10042547 CPU context switches
  1590932382 boot time
        9975 forks

  

• Get last commit date of file
  Read More 🔽

  Get last commit date of file

  this can be done using the git log command

  git log --follow -p -- filename

  This by default renders the diffed version history of the file over time. This can be suppressed by -q flag after that we can use awk to look for pattern Date.

  git log --follow -q -- README.md | awk '/Date/ { print $4,$3,$6 }'

  This will grab the date of each commit which modified this file!

  14 Nov 2020
  11 Nov 2020
  11 Nov 2020
  7 Nov 2020
  ...

  Here is simple script that will list last commit date of each file inside a git repo.

  #!/usr/bin/env bash
  
  # Utility to list last commit date of each file in a git repo
  
  [[ ! -d ".git" ]] && echo -e "Not a git repo" && exit 1
  
  for file in $(du --exclude='.git' -a . | awk '{ print $2 }'); do
      if [[ -f "${file:2}" ]]; then
          commit_date=$(git log --follow -q -- "${file:2}" | awk '/Date/ { print $4,$3,$6 }' | head -1)
          [[ "$commit_date" ]] && printf "%12s : %s\n" "$commit_date" "${file:2}"
      fi
  done

  A better version of the script is available in my dotfiles

  

• Line Discipline in Unix/Linux Machines
  Read More 🔽

  Line Discipline in Unix/Linux Machines

  Line discipline handles things like backspace and also generates various signals for special characters like Ctrl + C/Z etc.

  stty -a will display all these settings. To know more do man ssty.

  Demo

  Other than Ctrl+c and Ctrl+z which you already know about. Here are some other tricks.

  Fire up your terminal. And start typing....

  Keyboard Shortcut	Description
  Ctrl+?	Delete the last input character, Basically Backspace (See what I am talking about, ^? can be used in terminals which may not support the backspace key
  Ctrl+q	Erase line, works like carriage return /r
  Ctrl+a	Moves cursor to beginning of line
  Ctrl+e	Moves cursor to end of line
  Ctrl+w	Delete the last input "word"
  Ctrl+k	Erase line to the end, from current cursor position
  Ctrl+y	Paste the last erased text

  Apart from these line input specific keyboard shortcuts. We also have ...

  Multiline Input

  Use / for continuing the multiline input.

  bhupesh@dev: hello my name\
  is\
  bhupesh\
  check\
  > my boi\
  > hoooo\
  > 

  A better version

  #!/bin/bash
  
  echo -e "Enter Commit Message (Ctrl+d to stop) : "
  commit_message=$(</dev/stdin)
  
  echo -e "\n\n$commit_message"

  Make it executable and run.

  Enter Commit Message (Ctrl+d to stop) : 
  - fixed bug #454
  - Increase reponse time
  - style fixes
  
  
  - fixed bug #454
  - Increase reponse time
  - style fixes
  

  All of this is controlled by the tty driver

  Resources

  • The TTY demystified

  

• Line Editors in Linux, Tips and Tricks
  Read More 🔽

  Line Editors in Linux, Tips and Tricks

  I will log various ways through which tools like sed, cut and tr can be used.

  sed 😥

  • Print specific lines from a file using line numbers.

    # print lines 12 to 22
    sed -n '12,22p' file.txt

  • Omit first line of output.

    sed -n '1!p'

  • Omit last line of file.

    sed '$d' file.txt

  • Print everything after a pattern (inclusive).

    sed -n '/pattern/,$ p' file.txt

  • Print everything before a pattern (inclusive).

    sed -n '1,/pattern/ p' file.txt

  tr ➡️

  • Translate (or convert) all () to [] in a text file.

    tr '()' '[]'

  • Translate all occurrences of multiple spaces with a single space.

    tr -s ' '

  cut ✂️

  • Print every 4th word (or field) from a space separated STDIN.

    cut -d' ' -f4

    I don't know about you but this is pretty cool.

  awk

  • Don't print first line of file

    awk NR\>1 file.txt

  

• Monitor network (data) usage
  Read More 🔽

  Monitor network (data) usage

  The amount of data sent(uploaded) & received (downloaded) can be found out using the following bash script.

  • Only works per session, i.e stats are gathered once you power up your PC (or login).
  • Good to have if you have limited data avaiability & want to montior your data usage.

  netu() {
      # [net]work [u]sage: check network usage stats
  
      net_device=$(route | awk '/default/ {print $8}')
      TRANSMITTED=$(ifconfig "$net_device" | awk '/TX packets/ {print $6$7}')
      RECEIVED=$(ifconfig "$net_device" | awk '/RX packets/ {print $6$7}')
  
      pc_uptime=$(uptime -p | awk '{for (i=2; i<NF; i++) printf $i " "; if (NF >= 1) print $NF; }')
      printf "%s\n\n" "Network Usage since $pc_uptime"
      printf "%s\n" "$(tput bold)🔼 TRANSMITTED $(tput sgr0): $TRANSMITTED"
      printf "%s\n" "$(tput bold)🔽 RECEIVED    $(tput sgr0): $RECEIVED"
  }
  

  Demo

  

  Grab it from here

• Print lines between 2 words
  Read More 🔽

  Print lines between 2 words

  You may arrive in a situation where you may want to "extract" out text between two words. For example to view the latest changelog (where x.x.x is the latest version) in a CHANGELOG.md file.

  Using sed

  sed -n -e '/x.x.x/,/0.1.0/ p' CHANGELOG.md | sed -e '1d;$d'

  sed -e '1d;$d' removes the first & last line.

  Using awk

  awk '/x.x.x/,/0.1.0/' CHANGELOG.md | awk 'NR>2 {print last} {last=$0}'

  awk 'NR>2 {print last} {last=$0}' removes the first & last line.

  NOTE: NR means which Line number is being processed

  Resources

  • How to show lines after each grep match until other specific match?
  • What is the easiest way to remove 1st and last line from file with awk?
• Random emoji 😲 in one line
  Read More 🔽

  Random emoji 😲 in one line

  printf "%b\n" "\U1F$(shuf -i600-700 -n1)"

  PS: I am still working on a better way, this will only generate emojis in UNICODE range 1F601 to 1F700 while ignoring codepoints like 1F60A 😊. Let me know if you have a beter way (create an issue)

  

• Shell Redirections ↔ Quick Guide
  Read More 🔽

  Shell Redirections ↔ Quick Guide

  File descriptors:

  • stdin : 0
  • stdout : 1
  • stderr : 2

  Redirecting stderr

  1. Use 2>. Compatible with both bash and sh

  Redirecting both stderr & stdout

  1. With bash, use &>
  2. With sh, use > where-to-redirect 2>&1

WebDev

• Auto-complete in plain HTML
  Read More 🔽

  Auto-complete in plain HTML

  You can make a type-ahead/autocomplete like thing in plain HTML using the <datalist></datalist> tag.

  Usage

  <!DOCTYPE html>
  <html>
  <head>
  	<title></title>
  </head>
  <body>
  <h1>Type Ahead HTML</h1>
  <input list="test" placeholder="Choose an option">
  <datalist id="test">
  	<option value="C++">
  	<option value="Python">
  	<option value="Go">
  	<option value="JavaScript">
  	<option value="HTML">
  </datalist>
  </body>
  </html>

  Now as you type G it will show Go ;)

  Resources

  • tweet by Álvaro Trigo
• I learned about Open Graph protocol
  Read More 🔽

  I learned about Open Graph protocol

  What it does ?

  • Open Graph Makes your website into rich "graph" objects.
  • Now, what I understand from it is that it provides us to add additional metadata to your website which makes it more rich on social media. For e.g you see the thumbnails of links in the Telegram - courtesy of OpenGraph

  How ?

  Information is added into the <head> tags. For e.g below is the metadata of one of my blogs

  <meta property="og:description" content="A collection of C++ STL features (functions/libraries) which can be learned in 30 seconds or less" />
  <meta property="og:title" content="30 Seconds of C++" />
  <meta property="og:url" content="/30-Seconds-of-C++/" />
  <meta property="og:image" content="/images/blog5.png"/>

  Sidenotes

  Socila Media platforms like Twitter, LinkedIn, Telegram heavily use this meta info to render links, display images etc.

• Live Editing HTML
  Read More 🔽

  Live Editing HTML

  Yes !, you can live edit webpages by adding the following in the <html> tag.

  <!DOCTYPE html>
  <html contenteditable>
  <head>
  	<title>Yay! Live Editing</title>
  </head>
  <body>
  	<p>Try Editing Tis</p>
  </body>
  </html>

  Setting contenteditable="true" will make its content editable.

  So what ? Well you can use this HTML5 feature to make a motepad right into your browser.

  data:text/html, <html contenteditable <head ><title>Notepad</title></head><body style="background-color:black;color: white;"></body></html>

  

• `async` & `defer` Attributes
  Read More 🔽

  async & defer Attributes

  Usually javascript files interrupt the parsing of HTML document. To prevent this 2 special attributes of the <script> elements are used.

  async

  <script async src="script.js">

  • The async attribute is used to indicate to the browser that the script file can be executed asynchronously.
  • Therefore the HTML parser does not need to pause and wait for the JS code to load, it is intead fetched in parallel.
  • It is only available for externally located script files.

  defer

  <script defer src="script.js">

  • The defer attribute tells the browser to only execute the script file once the HTML document has been fully parsed.
  • The js file can be downloaded but it does not executes until the whole HTML is parsed.

Usage

See USAGE.md to know how I use this repository.

Author

👤 Bhupesh Varshney

☺️ Show your support

Support me by giving a ⭐️ if this project helped you! or just

📝 License

Copyright © 2020 Bhupesh Varshney.
This project is MIT licensed.

About

Original Idea/Work thoughtbot/til.