Skip to content

Instantly share code, notes, and snippets.

@shadiakiki1986
shadiakiki1986 / README.md
Last active October 13, 2022 18:22
Run a Scrapy spider in a Celery Task (in django)

My first shot at fixing this was in tasks.py file below.

But then that just gave a "unhandled error in deferred", so I went on to use CrawlRunner.

That showed no output at all anymore, and didn't run as expected.

Eventually, I just settled on CELERY_WORKER_MAX_TASKS_PER_CHILD=1=1 in settings.py

Note: CELERY_WORKER_MAX_TASKS_PER_CHILD=1 is for django. Celery without django probably drops the CELERY_ prefix

@shadiakiki1986
shadiakiki1986 / README.md
Last active May 29, 2018 07:27
H2O driverless AI installation instructions

Installation scripts for H2O driverless AI

Launch order

  • install.sh (requires sudo)
    • need to exit and re-ssh into machine for "docker usage without sudo" to take effect
  • for GPU, refer to init-gpu.sh
  • dropbox.sh (optional)
  • start.sh
@shadiakiki1986
shadiakiki1986 / README.md
Last active January 5, 2018 15:16
g2-ml keras spot instance init: cpu and gpu

These are a few scripts that help me install requirements on fresh AWS EC2 instances for deep learning

@shadiakiki1986
shadiakiki1986 / README.md
Last active August 15, 2017 14:05
Steps to deep clean pymssql repository
@shadiakiki1986
shadiakiki1986 / test.py
Last active July 14, 2017 16:49
sklearn PCA requires normalization
# modified example of
# http://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html#sklearn.decomposition.PCA.transform
import numpy as np
from sklearn.decomposition import PCA
# Just the same PCA as the example
pca = PCA(n_components=2, svd_solver='full')
# same input as example
@shadiakiki1986
shadiakiki1986 / widgets.py
Created July 4, 2017 14:28
Django simplest read-only widget
# Based on the below answer on the SO question:
# In a Django form, how do I make a field readonly (or disabled) so that it cannot be edited?
# https://stackoverflow.com/a/15134622/4126114
#
# Usage of either widget classes below:
# class MyForm(forms.ModelForm):
# class Meta:
# widgets = {
# 'field': ReadOnlyWidgetSimple(),
# 'foreign_key': ReadOnlyWidgetMyModel()
@shadiakiki1986
shadiakiki1986 / example1.py
Last active June 28, 2017 16:08
Reconcile sklearn with MLJAR
"""
Reconciling results from MLJAR and sklearn.
Random forest classifier
X==y
Usage
pew new --python=python2 MLJAR_TEST
pip install mljar sklearn numpy scipy pandas
export MLJAR_TOKEN=exampleexampleexample
@shadiakiki1986
shadiakiki1986 / definitions-arabic.txt
Last active January 7, 2017 21:26
qamous al filaha
2016-12-18 عبدو الأَشقَر
١. طاب اليَد: الأَرضُ أَصبَحَت صالِحَة لِلفِلاحَة
٢. كيف العَدّان؟: كَيفَ الطّقسُ المُتَقَلِّب؟
٣. مغَطغطَة : مغَيّمَة وَصُعوبَة في الرُّؤيَة
٤. الوجاق: صوبيا تَعمَلُ عَلى الحَطَب، أَو البابور
{
"name": "root/test-phpcache-filesystem",
"require": {
"cache/filesystem-adapter": "^0.3.2"
},
"authors": [
{
"name": "Shadi Akiki",
"email": "shadiakiki1986@gmail.com"
}
@shadiakiki1986
shadiakiki1986 / awsEc2Helper.sh
Last active July 1, 2017 01:26
Helper for aws ec2 CLI
#!/bin/bash
# This is a helper for aws ec2 CLI
# https://gist.github.com/shadiakiki1986/a4bc2956eaaef7dede10
if [ $# -ne 2 ] || [ "$2" == "help" ]; then
echo "Usage:"
echo " bash $0 id '*'"
echo " bash $0 id udacity"
echo " bash $0 id '*udacity*'"
echo " bash $0 start id"