Skip to content

Instantly share code, notes, and snippets.

View lovasoa's full-sized avatar
🎯
Focusing

Ophir LOJKINE lovasoa

🎯
Focusing
View GitHub Profile
@lovasoa
lovasoa / Экзамен по анализу данных.md
Last active May 12, 2017 10:37
Экзамен по анализу данных

Экзамен по анализу данных

методы построения регрессионных моделей

Регрессия, это процесс оценки отношений между переменными. Для того, чтобы строить регрессионную модель, существует несколько методов.

Каждый метод анализирует введённые данные, и даёт в выводе параметры одной функции, которая приближает данные. Можно потом использовать эту функцию чтобы делать прогнозы с новыми данными, например.

Методы

@lovasoa
lovasoa / St. Petersbourg.ipynb
Created May 22, 2017 21:41
St. Petersbourg: Paradoxe de Saint-Pétersbourg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@lovasoa
lovasoa / St. Petersbourg.ipynb
Created May 22, 2017 21:41
St. Petersbourg: Paradoxe de Saint-Pétersbourg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@lovasoa
lovasoa / ipython_progressbar_iterator.py
Created June 5, 2017 17:50
ipython progressbar iterator: show a progressbar for loops inside ipython
from IPython.html.widgets import FloatProgressWidget
from IPython.display import display
def display_progress(collection):
"""
>>> l = [1,2,3]
>>> for e in display_progress(l): do_something(e)
"""
f = FloatProgressWidget(min=0, max=len(collection))
display(f)
@lovasoa
lovasoa / poisson.ipynb
Last active June 19, 2017 00:30
Как считается диапазон в функции countApprox в Spark (Распределение Пуассона)
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@lovasoa
lovasoa / mustaches.json
Last active September 18, 2017 20:58
A json file with a lot of mustaches
{"test": "{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{}}{{
@lovasoa
lovasoa / wikidata_extract_official_websites.sh
Last active October 9, 2018 12:12
Extract information from wikidata in one line of bash (with `jq`)
# Wikidata can be queried in SPARQL using https://query.wikidata.org/
# However, result size is limited. So this little script processes official wikidata dumps in order to extract information.
# This allows to make simple queries, and stream the results
curl --silent "https://dumps.wikimedia.org/wikidatawiki/entities/latest-all.json.bz2" | bunzip2 | jq --stream -c -M -r '
select(
(( .[0][1] == "labels" and .[0][2] == "en" and .[0][3] == "value" ) and length > 1) or
( .[0][1] == "claims" and .[0][2] == "P856" and .[0][6] == "value" )
) | .[1] '
@lovasoa
lovasoa / deep_json_array.py
Created October 21, 2017 19:50
Breaking python's json parser. Trying to find the smallest valid json object that python's default json module cannot parse.
#!/usr/bin/env python3
import json
# Dichotomic search of the deepest parsable json array that python can parse
(m,M) = (0, int(1e6))
while m+1<M:
middle = (m+M)//2
try:
json.loads('[' * middle + ']' * middle)
m = middle
@lovasoa
lovasoa / DamerauLevenshteinAlgorithm.java
Last active November 2, 2017 14:47
Generalized Damerau-Levenshtein algorithm implementation in java. Works with lists of any type.
package com.qwant.utils;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.stream.IntStream;
// Inspired From : https://github.com/KevinStern/software-and-algorithms
// This version has me modified to work with any element type, not only string
@lovasoa
lovasoa / BatchConsumer.java
Created November 22, 2017 08:33
Thread-safe class to implement a stream consumer that works in batch.
package com.qwant.utils;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.ScheduledFuture;
import java.util.concurrent.TimeUnit;
import java.util.function.Consumer;