This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import pyarrow as pa | |
| import pyarrow.compute as pc | |
| def sort_dictionary(arr): | |
| o = pc.sort_indices(arr.dictionary) | |
| return pa.DictionaryArray.from_arrays( | |
| dictionary=pc.take(arr.dictionary, o), | |
| indices=pc.take(pc.sort_indices(o), arr.indices) | |
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| use arrow::array::{make_array, ArrayData, Int64Array}; | |
| use arrow::pyarrow::PyArrowType; | |
| use pyo3::prelude::*; | |
| fn chunked_binary_search(chunks: Vec<&[i64]>, val: i64, left: bool) -> usize { | |
| let val = if left { val } else { val + 1 }; | |
| let chunks: Vec<&[i64]> = chunks.into_iter().filter(|x| !x.is_empty()).collect(); | |
| if chunks.is_empty() { | |
| return 0; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import numpy as np | |
| def estimate_eigeinvalues(A, X, extra=False): | |
| """From estimated eigenvectors X of A, compute estimated eigenvalues of A.""" | |
| n, _ = A.shape | |
| R = np.eye(n) - X.T @ X | |
| S = X.T @ A @ X | |
| w = np.diag(S) / (1 - np.diag(R)) | |
| if extra: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import numpy as np | |
| N = 200 | |
| L = 10_000_000 | |
| p = 0.2 | |
| r = np.arange(L / p, dtype=int) | |
| s = [ | |
| r[np.random.rand(r.shape[0]) > 1 - p] | |
| for _ in range(N) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #include <algorithm> | |
| #include <iostream> | |
| #include <string> | |
| #include <thread> | |
| #include <vector> | |
| #include <future> | |
| #include <queue> | |
| #include <condition_variable> | |
| #include <mutex> | |
| #include <functional> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| from pathlib import Path | |
| import numpy as np | |
| import pandas as pd | |
| class RaggedIndexer: | |
| def __init__(self, counts): | |
| self.stops = np.r_[0, np.cumsum(counts)] | |
| self.starts = self.stops[:-1] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # https://rockylinux.org/cloud-images/ | |
| # AMI 0951577d22b8e8dd4 rocky 9.1 | |
| # Instance p3.2xlarge | |
| dnf install -y epel-release | |
| crb enable | |
| dnf config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/rhel9/x86_64/cuda-rhel9.repo | |
| dnf module install -y nvidia-driver:latest | |
| reboot |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import numpy as np | |
| from opt_einsum import contract | |
| class DRE: | |
| """Deep Regression Ensemble | |
| Parameters | |
| ---------- | |
| n_layers : int | |
| Number of hidden layers. |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import os | |
| from multiprocessing import get_context | |
| from concurrent.futures import ProcessPoolExecutor | |
| from concurrent.futures.thread import _global_shutdown_lock as L | |
| os.register_at_fork( | |
| before=L.acquire, | |
| after_in_child=L._at_fork_reinit, | |
| after_in_parent=L.release |
NewerOlder