ThaiDat

slenky

/*
 * Licensed to the Apache Software Foundation (ASF) under one or more
 * contributor license agreements.  See the NOTICE file distributed with
 * this work for additional information regarding copyright ownership.
 * The ASF licenses this file to You under the Apache License, Version 2.0
 * (the "License"); you may not use this file except in compliance with
 * the License.  You may obtain a copy of the License at
 *
 *    http://www.apache.org/licenses/LICENSE-2.0
 *

slenky

/*
 * Licensed to the Apache Software Foundation (ASF) under one or more
 * contributor license agreements.  See the NOTICE file distributed with
 * this work for additional information regarding copyright ownership.
 * The ASF licenses this file to You under the Apache License, Version 2.0
 * (the "License"); you may not use this file except in compliance with
 * the License.  You may obtain a copy of the License at
 *
 *    http://www.apache.org/licenses/LICENSE-2.0
 *

idiomer

'''
The path is a directory by default
'''

def hdfs_list(path, subtract_one=True):
    fs = spark._jvm.org.apache.hadoop.fs.FileSystem.get(spark._jsc.hadoopConfiguration())
    list_status = fs.listStatus(spark._jvm.org.apache.hadoop.fs.Path(path))
    # file.getPath().getName(), file.getBlockSize(), file.getLen()
    files_size = [file.getLen() for file in list_status]
    totol_size_in_MB = sum(files_size) / 1024.0 / 1024.0

tbaums

The following assumes you are using Minikube and Trow for container registry, installed using the quick install method documented here

Pull the following containers locally (docker pull):

For Kudo:

1. kudobuilder/controller:v0.9.0

From params.yaml:

1. mesosphere/cassandra-prometheus-exporter:2.2.1-0.1.1
2. mesosphere/cassandra:3.11.5-0.1.1

sticilface

Guide to PROGMEM on ESP8266 and Arduino IDE

Intro

On low memory devices like the arduino and esp8266 you do not want strings to be stored in RAM. This occurs by default on these systems. Declare a string const char * xyz = "this is a string" and it will use up RAM.

The solution on these devices is to allow strings to be stored in read only memory, in Arduino this is the PROGMEM macro. Most of my experience is with the ESP8266 which is a 32bit micros controller. This device stores PROGMEM data in flash. The macro PROGMEM on ESP8266 is simply

#define PROGMEM   ICACHE_RODATA_ATTR

nlsun

Repoxy

How to Configure

Copy this snippet and write it to a file:

{
  "id": "/repoxy",
  "cpus": 1,
  "acceptedResourceRoles": [

vemacs

# systemd unit file
# place in /etc/systemd/system
# systemctl enable gotty.service
# systemctl start gotty.service

[Unit]
Description=Gotty Web Terminal
After=network.target

[Service]

tam7t

vault.barrier.*
name="vault_barrier"
method="$1"

vault.consul.*
name="vault_consul"
method="$1"

vault.route.*.*
name="vault_route"

tobilg

Custom S3 endpoints with Spark

To be able to use custom endpoints with the latest Spark distribution, one needs to add an external package (hadoop-aws). Then, custum endpoints can be configured according to docs.

Use the hadoop-aws package

bin/spark-shell --packages org.apache.hadoop:hadoop-aws:2.7.2

SparkContext configuration