rayhassan

The TRIM command enables an operating system to notify the SSD of pages which no longer contain valid data. For a file deletion operation, the operating system will mark the files sectors as free for new data, then send a TRIM command to the SSD. After trimming, the SSD will not preserve any contents of the block when writing new data to a page of flash memory, resulting in less write amplification (fewer writes), higher write throughput (no need for a read-erase-modify sequence), thus increasing drive life.

Different SSDs implement the command somewhat differently, so performance can vary.[3][8]

Trim tells the SSD to mark a LBA region as invalid and subsequent reads on the region will not return any meaningful data. However, the data could still reside on the flash internally.


are my disks TRIM enabled

lsblk -o MOUNTPOINT,DISC-MAX,FSTYPE

rayhassan

Basic Disk Partition Offsets: wmic.exe

Windows can be interrogated for disk-related information via Windows Management Instrumentation (WMI). A straightforward method for obtaining partition starting offsets of Windows basic disks is this Wmic.exe command.

Command Line Syntax
wmic partition get BlockSize, StartingOffset, Name, Index

The value for Index is the same as disk number in the Disk Management Microsoft Management Console (MMC) snap-in (Diskmgmt.msc); wmic volume can also be used to map disk indexes and drive letters.

rayhassan

o get k8s cluster status

# kubectl --kubeconfig ./kubeconfig cluster-info

Kubernetes master is running at https://10.68.68.108:443
Elasticsearch is running at https://10.68.68.108:443/api/v1/namespaces/kube-system/services/elasticsearch-logging/proxy
Kibana is running at https://10.68.68.108:443/api/v1/namespaces/kube-system/services/kibana-logging/proxy
KubeDNS is running at https://10.68.68.108:443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

rayhassan

The devices were partitions that served as a SAP Sybase IQ consistency group, and mapped to an EMC VNX 5700 with 9 SSDs configured to provide 20 x 35 GB devices. The devices were mapped to dbspace files as follows:  
2 devices for shared IQ_SYSTEM_MAIN  
12 devices for shared IQ_USER_MAIN  
2 devices for shared IQ_SHARED_TEMP  
2 devices for node 1 IQ_SYSTEM_TEMP  
2 devices for node 2 IQ_SYSTEM_TEMP 
1 device (450 GB) for ext4 filesystem used to host source load data (node 1) 

brw-rw----. 1 root disk 8,  16 Jul 17 08:17 /dev/sdb

rayhassan

Flume
=====

ingest : number or sources of origin for events  / number of events (per source) <<<<* this is required 

R.o.T : one aggregator for every 4-16 agents

so using the above info work from outer tier to final inner tier - and we know final destination of events : hdfs ?