Denis M. Gabaydulin sherman

Y-Factors Formula

Здесь опубликованы коэффициенты для одного из внутренних ранжировщиков в поиске Яндекса. Это не итоговая формула, которая влияет на результат появления ссылки в серпе, т.к. судя по изученному коду – внутри используется несколько ранжировщиков и поисковых движков, результаты которых мержатся между собой и уже итоговый результат приводит к распределению ссылок по топу выдачи.

коэффициенты влияния на факторы

| Коэффициент | Название Фактора | Описание Фактора |

Deep Learning

concepts

forward and backward propagation
vanishing gradient
image convolution operation
feature map, filter/kernel
receptive field
embedding
translation invariance

Kafka 0.11.0.0 (Confluent 3.3.0) added support to manipulate offsets for a consumer group via cli kafka-consumer-groups command.

List the topics to which the group is subscribed

kafka-consumer-groups --bootstrap-server <kafkahost:port> --group <group_id> --describe

Note the values under "CURRENT-OFFSET" and "LOG-END-OFFSET". "CURRENT-OFFSET" is the offset where this consumer group is currently at in each of the partitions.

Reset the consumer offset for a topic (preview)

If you want to learn Scala, I’d suggest:

Install IntelliJ + Scala Plugin
Don’t do the Coursera courses yet.
Don’t do the “red book” Functional Programming in Scala yet.
Do: http://underscore.io/books/
- Essential Scala
- Essential Play
- Essential Slick
Do Scala for the Impatient: https://www.amazon.com/Scala-Impatient-Cay-S-Horstmann/dp/0321774094
The Neophyte’s Guide to Scala http://danielwestheide.com/scala/neophytes.html

Spark - High availability

Components in play

As a reminder, here are the components in play to run an application:

The cluster:
- Spark Master: coordinates the resources
- Spark Workers: offer resources to run the applications
The application:

Kafka Topics

List existing topics

bin/kafka-topics.sh --zookeeper localhost:2181 --list

Describe a topic

bin/kafka-topics.sh --zookeeper localhost:2181 --describe --topic mytopic

Purge a topic

bin/kafka-topics.sh --zookeeper localhost:2181 --alter --topic mytopic --config retention.ms=1000

... wait a minute ...

System Design Cheatsheet

Picking the right architecture = Picking the right battles + Managing trade-offs

Basic Steps

Clarify and agree on the scope of the system

User cases (description of sequences of events that, taken together, lead to a system doing something useful)
- Who is going to use it?
- How are they going to use it?

	package demo.cont;

	import java.lang.invoke.MethodHandle;
	import java.lang.invoke.MethodHandles;
	import java.lang.invoke.MethodType;
	import java.util.function.Consumer;

	public final class Continuation {

	private final Object delegate;

	#' @title Geolocate IP Addresses Through ip-api.com
	#' @description
	#' \code{geolocate} consumes a vector of IP addresses and geolocates them via
	#' \href{http://ip-api.com}{ip-api.com}.
	#' @param ip a character vector of IP addresses.
	#' @param lang a string to specify an output localisation language.
	#' Allowed values: en, de, es, pt-BR, fr, ja, zh-CN, ru.
	#' @param fields a string to specify which fields to return.
	#' @param delay a logical to whether or not to delay each request by 400ms.
	#' ip-api.com has a maximum threshold of 150 requests a minute. Disable it for