vhbui02

Parallel download and rename at the same time

https://stackoverflow.com/a/51584237/9122512

Intro

xargs reads lines of text from the STDIN or from the pipelined outputs of another command and turns them into multiple commands and executes them.

xargs usually go with find command.

Syntax

xargs [options] [command [initial-arguments]]

vhbui02

Update outdated package?

• DO NOT upgrade package in pip list --outdated, since there're no assurance that something won't be broken.

Revert old version

pip install --force-reinstall -v "<package_name>==<version>"

Use pip or pip3

• pip3 will try to install packages for python3
• pip can install packages from python, python2, ONLY IF you installed multiple version of python, otherwise it's the same as pip3

vhbui02

Pre-defined and automatic variable

RS: Recond Seperator

• AWK processes data 1 record at a time, records are seperated by RS from a whole input data stream
• By default, RS=\n

NR: Record Number

• If you're using RS=\n by default, NR will be the current input line number.

FS/OFS: Field Seperator/Output Field Seperator

• AWK splits 1 record into multiple field based on the value of FS

vhbui02

Table of contents

I create a custom GitOpenSourceApps, AppImagesApplication in home folder

Duplicate /snap/bin directory in $PATH remove

https://askubuntu.com/questions/1349900/duplicate-system-path-variable-snap-bin-snap-bin

systemctl

• Start on boot sudo systemctl enable <service>
• Disable perminent sudo systemctl disable <service>

vhbui02

95% CI of 1 CAT

prop.test(x, n, p) binom.test(x, n, p) p = 1/num. of level

Check Ratio also

table(c(x1, n1), c(x2, n2)) prop.test(table)

Z-test

vhbui02

Research later

https://www.mongodb.com/docs/atlas/atlas-search/

Pros

• perform full-text search
• support fuzzy search

Cons

https://www.mongodb.com/docs/atlas/atlas-search/limitations/

vhbui02

Create a text index, it can include any field whose value is a string or an array of string elements

A collection can only have 1 text search index, but the good news is that text search index can cover multiple fields

Cons

• does not support fuzzy search
• does not support autocomplete
• does not work well with language using diacritic marks

Using find()

vhbui02

On-demand Materialized View

• pre-computed aggregation pipeline
• results are stored on and read from disk

Actually, it simply is a normal collection which is the result of a $merge or $out stage. There is no such thing as a special data structure called Materialized View like MySQL

Examples

// create mock data

vhbui02

worth to note

• aggregation can use indexes from input Collection to improve performance => indexing is very essential in aggregation.

• if indexes are good enough, they can cover a stage, that means simply using index is enough to return all matching documents, nullify the need to read documents directly => This make the query have very high performance

$match, $sort, $group can be benefited from indexes

• index in $match query field => identify relevent document very fast.
• index on sorted field can be used to return data for $sort stage => no need to sort again
• index on grouped field that matches $sort order, can return all of the field values needed to execute the $group state (this means if the data is sorted by grouped field, all identical values of grouped field are put close together). This is a covered stage.

vhbui02

CLUSTERED COLLECTIONS

A Collection with 1 clustered index

Pros

• 1. fast query without 2nd-ary index, use clustered index key instead to make range or equality comparison.
• 2. lower storage size, very good for bulk inserts
• 3. eliminate the need of TTL index, since: