Valerio Maggio leriomaggio

Case Study: Textual Data

Now it is time to crack on with our first case study 🙌.

Assume we have some data in textual format that we want to use as input to some ML model. To make it even more fun, instead of working with standard textual data, let's imagine we have some source code listings we want to process.

Our examplar data will be two functions, written in our most favourite programming language$^5$: init_random_weights_tensor, and init_random_weights_array:

[5]: Examples adapted from Learning PyTorch with Examples tutorial.

Kubeflow Kale: from Jupyter Notebook to Complex Pipelines

Abstract

In this talk I will present a new solution to automatically scale Jupyter notebooks to complex and reproducibility pipelines based on Kubernetes and KubeFlow.

Description

Nowadays, most of the High Performance Computing (HPC) tasks are carried out in the Cloud, and this is as much

import pandas as pd
import numpy as np
from IPython.display import HTML

Network Analysis Tutorial using Python & `networkx`

MSCX Ph.D. Summer School, Salina (ME) - Italy

Author: Valerio Maggio

PostDoc Data Scientist @ FBK/MPBA

import pandas as pd
import numpy as np

	import numpy as np
	import multiprocessing as mp
	from argparse import ArgumentParser


	def get_current_seed(wid):
	s = np.random.get_state()[1][0]
	print(f"Pool Worker {wid}: Random State {s}")

	from typing import Tuple, List, Optional, Union, NewType
	from typing import Protocol, runtime_checkable
	from abc import ABC, abstractmethod
	import pandas as pd
	try:
	import torch as t
	from torch import Tensor as Tensor
	except ImportError: # fallback to Numpy, if torch not available - this is just as a POP!
	import numpy as t
	from numpy import ndarray as Tensor

	"""Python/Numpy implementation of the Borda Count"""

	import numpy as np
	from nptyping import NDArray
	from typing import Tuple, Union, List, Any

	RankingsList = Union[NDArray[(Any, Any), np.int], List[List[int]]]
	Ranking = NDArray[(Any,), np.int]
	Counts = NDArray[(Any,), np.int]
	AveragePositions = NDArray[(Any,), np.float]

	import umap
	import numpy as np
	import scipy as sp

	print('umap: ', umap.__version__)
	print('Numpy: ', np.__version__)
	print('Scipy: ', sp.__version__)

	# umap: 0.4.0
	# Numpy: 1.17.5