Skip to content

Instantly share code, notes, and snippets.

View jhejderup's full-sized avatar
:octocat:

Joseph Hejderup jhejderup

:octocat:
14 followers · 9 following
  • Delft
  • 15:42 (UTC +01:00)
View GitHub Profile
@jhejderup
jhejderup / 0.buildstats.sh
Last active May 19, 2020 13:14
Präzi Analytics
## ON LIMA
cd /data/praezi/data
## Total number of processed releases
find . -maxdepth 2 -mindepth 2 -type d -printf '%f\n'
## Total number of releases
find . -name callgraph.json | wc -l
## Total number of packages represented
@jhejderup
jhejderup / cargolock2csv.py
Last active March 20, 2020 14:18
#!/usr/bin/python3
# -*- coding: utf-8 -*-
""" Generate a dependency graph from a Cargo.lock file
Run: python3 cargolock2csv.py <path-to-Cargo.lock>
Deps: pip3 install toml
Return: pkg_nodes.csv, pkg_edges.csv
"""
import sys
import toml
@jhejderup
jhejderup / easychair2acm.py
Last active March 9, 2020 16:03
#!/usr/bin/python3
# -*- coding: utf-8 -*-
""" Convert EasyChair data into ACM data for eRights generation
Run: python3 easychair2acm.py <submitted_papers.csv> <authors.csv> <accepted_papers.csv>
"""
import sys
assert len(sys.argv) == 4
@jhejderup
jhejderup / cg-stats.py
Last active March 23, 2020 15:16
Praezi
##
## dep: pip3 install kafka-python
## run python3 cg-stats.py
from kafka import KafkaConsumer, KafkaProducer, TopicPartition
from kafka.errors import KafkaError
import json
consumer = KafkaConsumer(
'praezi.callgraphs',
@jhejderup
jhejderup / Dockerfile
Last active January 13, 2020 12:10
Docker setup for parsing dependency update information from ght pull request events
FROM node:12-buster-slim
# ----
# Instal essentials
RUN apt-get update && \
apt-get install -y build-essential && \
apt-get clean && \
apt-get autoclean && \
apt-get autoremove
@jhejderup
jhejderup / ght_dependabot_csv.js
Last active January 13, 2020 11:39
GHTorrent extracting dependabot records from the events collection
db.getCollection('events').aggregate([
{
$sort: {_id: -1}
},{
$match: {
"$and": [
{"type": {$eq: "PullRequestEvent"}},
{"payload.pull_request.head.repo.language": { $eq: "Java"}},
{"actor.id": {$eq: 27856297 }},
]}
@jhejderup
jhejderup / parse_mutation_info.py
Created September 27, 2019 14:40
Parsing PITest mutations.csv
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import os.path, sys, sets
def reformat(line):
return replace_slash_dot(
split_line(
remove_first_char(
@jhejderup
jhejderup / list.txt
Last active September 17, 2019 11:51
Replication pack
AddstarMC/Minigames 0b4e9d40f5bc88b5930ac0094d7f403bde3fc55a
Adobe-Consulting-Services/acs-aem-tools c588590b149a409950b6d250db6b3583331f1688
afrunt/jach 18519dd0bd9e65772393eb186c02f1bf4713e58c
aintshy/hub 4cacc3857e095cf1b369eb65cd1dcc71649be640
airlift/slice 8a2c786718d8d7936a25feaa1a152438f7c78ba5
alex-bretet/cloudstreetmarket.com 76de2e0d31e6f8f57db87a2964b9c7aa6f2acdfe
alexvictoor/web-logback 51c01491821d5780c595ff4329ebeabbb5008c6c
akraxx/gitlab-jira-integration 7cdaa7053af0cd6a1677d725697256504ac6f46f
3wks/thundr 13ee04f4ebf0634dfbb5141bd5635b3a9587bc4a
alibaba/canal 44722fe01c5568239e190775c5c9dc7360335db1
@jhejderup
jhejderup / clean_calltrace.py
Last active August 18, 2019 14:06
Dependency Update Emulation
# coding=utf8
# the above tag defines encoding for this document and is for Python 2.x compatibility
import re
import sys
regex = r"(\<|\>)\[\d\]\[\d\](.*)\=\d+"
if __name__ == "__main__":
for line in sys.stdin:
@jhejderup
jhejderup / ast-mining.sh
Last active August 7, 2019 08:52
Data cleaning
#Go to folders
cd /data/uppdatera/github/repos
## Install and do call-mining
time cat ../tests-working.txt | parallel 'cd {}; cmd="mvn install -Dmaven.test.skip=true"; $cmd &> /dev/null; [[ $? -eq 0 ]] && echo {}' > ../build-jars.txt
## Run AST Miner
time cat ../build-jars.txt | parallel 'cd {}; cmd="java -Xss256m -Xms2048m -Xmx4096m -jar /data/uppdatera/scripts/callminer/target/callminer.jar /data/uppdatera/github/repos/{}/pom.xml"; $cmd &> /dev/null; [[ $? -eq 0 ]] && echo {}' > ../ast-mining.txt
cat ast-mining.txt | grep -v "/home/jhejderup/.m2" > ast_mining2.txt