- There are three types of permissions in files and folders in unix
- Read (r)
- Write (w)
- Execute (x)
- And, there is a classification of users called UGO (explained bellow):
- U ~> User (usually, you)
# PowerShell script to keep a Windows PC awake | |
Write-Host "Keeping PC awake... (send Ctrl+C to quit)" | |
while (1) { | |
$wsh = New-Object -ComObject WScript.Shell | |
$wsh.SendKeys('+{F15}') | |
Start-Sleep -seconds 59 | |
} |
largest = None | |
smallest = None | |
while True: | |
inp = raw_input("Enter a number: ") | |
if inp == "done" : break | |
try: | |
num = float(inp) | |
except: | |
print ("Invalid input") |
import decimal | |
# for reference, the first 100 digits of pi | |
pi = decimal.Decimal('3.1415926535897932384626433832795028841971693993751058209749445923078164062862089986280348253421170679') | |
# Basic recursive factorial calculation. For large n switch to iterative. | |
def fact(n): | |
if n == 0: | |
return 1 |
# Useful references: | |
# | |
# https://superuser.com/questions/992511/emulate-a-keyboard-button-via-the-command-line | |
# https://ss64.com/vb/sendkeys.html | |
# https://social.technet.microsoft.com/Forums/windowsserver/en-US/96b339e2-e9da-4802-a66d-be619aeb21ac/execute-function-one-time-in-every-10-mins-in-windows-powershell?forum=winserverpowershell | |
# https://learn-powershell.net/2013/02/08/powershell-and-events-object-events/ | |
# | |
# Future enhancements - use events rather than an infinite loop | |
$wsh = New-Object -ComObject WScript.Shell | |
while (1) { |
Basic file formats - such as CSV, JSON or other text formats - can be useful when exchanging data between applications. When it comes to storing intermediate data between steps of an application, Parquet can provide more advanced capabilities:
- Support for complex types, as opposed to string-based types (CSV) or a limited type system (JSON only supports strings, basic numbers, booleans).
- Columnar storage - more efficient when not all the columns are used or when filtering the data.
- Partitioning - files are partitioned out of the box
- Compression - pages can be compressed with Snappy or Gzip (this preserves the partitioning)
The tests here are performed with Spark 2.0.1 on a cluster with 3 workers (c4.4xlarge
, 16 vCPU and 30 GB each).
The dplyr
package in R makes data wrangling significantly easier.
The beauty of dplyr
is that, by design, the options available are limited.
Specifically, a set of key verbs form the core of the package.
Using these verbs you can solve a wide range of data problems effectively in a shorter timeframe.
Whilse transitioning to Python I have greatly missed the ease with which I can think through and solve problems using dplyr in R.
The purpose of this document is to demonstrate how to execute the key dplyr verbs when manipulating data using Python (with the pandas
package).
dplyr is organised around six key verbs:
require(RCurl) | |
require(XML) | |
# Let's make a special class | |
NISTBeaconResponse <- function (ts) { | |
if(!is.integer(ts) & | |
!inherits(ts, "POSIXct") & | |
!inherits(ts, "POSIXlt")) { | |
stop("We expected a unix timestamp as an integer or a POSIXct or POSIXlt value") | |
} |
This snippet of code was posted in 2014 and slightly revised in 2016 and 2017. It was more of a quick'n'dirty script than a polished tool. It is made only for Linux and in Python 2, which has since become outdated.
I currently do not use it, and I suggest you avoid it as well. Please do not expect support for using this script.
🔥 If you need an alternative, @glaucocustodio has kindly suggested EasyVPN in this comment.
The rest of the README is left for historical purposed.
#Set up data partition | |
sudo mkdir /data | |
sudo chmod 777 /data | |
sudo "echo /dev/xvdb /data ext4 rw,user,exec,comment=cloudconfig 0 2 >> /etc/fstab" | |
sudo mount /data | |
#Install build environment | |
sudo sed -i "s/enabled=0/enabled=1" /etc/yum.repos.d/epel.epo | |
sudo yum -y update | |
sudo yum -y upgrade |