markito
/ pseudo_java_profiler.sh
Last active
August 29, 2015 14:17
— forked from mrbuk/pseudo_java_profiler.sh
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/bash | |
| if [ "$#" -ne 1 ]; then | |
| echo "Usage: $0 jps-filter-grep-expr" | |
| exit 1 | |
| fi | |
| jps_filter=$1 | |
| # extract the PID of the java process |
markito
/ gist:825f2ae981af12eb60d7
Created
July 16, 2015 22:59
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| runtime - Runtime classpath for source set 'main'. | |
| +--- org.apache.geode:gemfire-core:1.0.0-incubating-SNAPSHOT | |
| | +--- org.apache.logging.log4j:log4j-slf4j-impl:2.1 | |
| | | \--- org.apache.logging.log4j:log4j-api:2.1 | |
| | +--- org.apache.logging.log4j:log4j-jcl:2.1 | |
| | | +--- commons-logging:commons-logging:1.2 | |
| | | \--- org.apache.logging.log4j:log4j-api:2.1 | |
| | +--- org.apache.logging.log4j:log4j-jul:2.1 | |
| | | \--- org.apache.logging.log4j:log4j-api:2.1 | |
| | +--- org.springframework:spring-aop:3.2.12.RELEASE -> 4.1.6.RELEASE |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/usr/bin/python | |
| # @author markito | |
| # Python script that query running lattice containers and allows log in using wsh | |
| # Usage: ./lattice-ssh.py <PATH_TO_LATTICE> | |
| import os | |
| import subprocess | |
| import requests | |
| import sys | |
| import json |
markito
/ geodeClient.groovy
Created
August 21, 2015 04:58
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| @GrabResolver(name='asf-snapshots', root="https://repository.apache.org/content/repositories/snapshots") | |
| @Grab(group="org.apache.geode", module = "gemfire-core", version = "1.0.0-incubating-SNAPSHOT") | |
| import com.gemstone.gemfire.cache.client.ClientCacheFactory | |
| import com.gemstone.gemfire.cache.client.ClientRegionShortcut | |
| cache = new ClientCacheFactory() | |
| .addPoolLocator("localhost", 10334) | |
| .create(); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| dependencies { | |
| compile 'org.apache.geode:gemfire-core:1.0.0-incubating.M1' | |
| } | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <dependencies> | |
| <dependency> | |
| <groupId>org.apache.geode</groupId> | |
| <artifactId>gemfire-core</artifactId> | |
| <version>1.0.0-incubating.M1</version> | |
| </dependency> | |
| </dependencies> |
markito
/ SockerRule.java
Created
July 29, 2016 21:11
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| public class SocketRule implements TestRule { | |
| public static final Logger logger = LoggerFactory.getLogger(SocketRule.class); | |
| private Map<Integer, Exception> providedPorts = new HashMap<Integer, Exception>(); | |
| private int lastPort = 8000; | |
| public int findFreePort() { | |
| do { | |
| lastPort++; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| """ | |
| Fix the Sigmoid class so that it computes the sigmoid function | |
| on the forward pass! | |
| Scroll down to get started. | |
| """ | |
| import numpy as np | |
| class Node(object): |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import uuid | |
| import hashlib | |
| def hashText(text): | |
| """ | |
| Basic hashing function for a text using random unique salt. | |
| """ | |
| salt = uuid.uuid4().hex | |
| return hashlib.sha256(salt.encode() + text.encode()).hexdigest() + ':' + salt | |
markito
/ gist:a8ffcdde8cf8ebb0e69ead2363902f06
Created
September 15, 2017 14:14
Spark GC/memory settings
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| How much memory is permanently in memory vs how much is used for transformations | |
| (ratio) | |
| spark.storage.memoryFraction | |
| Suggested settings... (need to debug the logs after these settings) | |
| -XX:+UseG1GC -XX:+PrintFlagsFinal -XX:+PrintReferenceGC -verbose:gc | |
| -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintAdaptiveSizePolicy | |
| -XX:+UnlockDiagnosticVMOptions -XX:+G1SummarizeConcMark | |
| -Xms88g -Xmx88g -XX:InitiatingHeapOccupancyPercent=35 | |
| -XX:ConcGCThread=15 -XX:+AlwaysPreTouch |