Skip to content

Instantly share code, notes, and snippets.

View cwgem's full-sized avatar

Chris White cwgem

View GitHub Profile
@cwgem
cwgem / issue_workflow_feature.md
Created July 21, 2018 14:46
GitHub Issues Workflow Enhancement Proposal

Problem

GitHub has become a major platform for hosting code and making it more visible. Many projects hosted on GitHub rely on the Issues feature to provide a way for users to report issues with the project. Unfortunately there is a low cost to entry in posting an issue. The effect of this is a lot of noise relative to the popularity of the project. A solid example of this getting out of hand is the Visual Studio Code issue tracker with over 5000 issues total.

While a new feature was added around May to provide an initial filter, it's not a complete solution and even has a way to easily pull back to the original tracker. There is also a lack of rich metadata around issues. One can only look for labels (which even then have to be applied by someone with permissions) or attempt some sort of natural language parsing. This barrier means that automation around issue filtering is not feasible f

@cwgem
cwgem / build_guide.md
Last active November 11, 2024 18:16
Building `dockerd` and `docker-cli` from source

Introduction

This guide looks at what it will take to build dockerd and docker-cli from source with Ubuntu. Ubuntu was chosen as the OS as it uses the same apt-get calls the standard Dockerfile method uses making the experience a bit more seamless. While the official method is building docker in a container, here are some reasons why you may want to build this way:

  • Custom source modifications for testing
  • System that does not yet have a docker(d) binary release
  • You want to test things on a non-standard toolchain
  • You just want to deep dive

As you can probably tell using this method means you're delving away from support. Please don't file issues/PRs unless you can reproduce in the official build environment. In fact unless you really have one of the strong needs above you're better off building off a container since this guide is mostly a lot of copy/paste from the Dockerfile used in the official build system.

@cwgem
cwgem / docker_save_analysis.md
Created July 2, 2018 00:53
`docker save` output tar analysis

A basic look

You can use docker save to output a tar file of an image, potentially making it workable for analyzing the filesystem contents ala anti-virus scans and such:

$ docker pull microsoft/dotnet
$ docker save microsoft/dotnet --output dotnet.tar

So here we pull the official dotnet image from the docker store, then use docker save to push it to a tar file. After that taking a look shows:

@cwgem
cwgem / python_tar_fun.md
Created June 30, 2018 21:58
Fun ways to fool with python tar handling

Introduction

Python's tarfile documentation leaves the following nasty warning:

Warning Never extract archives from untrusted sources without prior inspection. It is possible that files are created outside of path, e.g. members that have absolute filenames starting with "/" or filenames with two dots "..".

Let's see what that exactly means.

@cwgem
cwgem / hcl_load_test.go
Created May 4, 2018 18:18
Messing around with loading an HCL ast via finding tf files recursively in a directory passed in via args
package main
import (
"flag"
"fmt"
"io/ioutil"
"os"
"path/filepath"
"github.com/hashicorp/hcl"
@cwgem
cwgem / git_cleanup.md
Created April 5, 2018 15:29
Fixing up a post git push miss on a non-master branch
  1. git checkout master
  2. git pull --rebase to ensure master is up to date
  3. git checkout [branch here]
  4. Fix the issue with your branch
  5. git add and git commit -m "fix" to commit the fix change
  6. git rebase -i master (if this fails you need to figure out what is conflicting with master and fix it)
  7. Find the "fix" commit and change pick to f (This will essentially merge your fix with the previous commit and pretend like the commit never happened separately)
  8. Write your commit info out with editor of choice
  9. Since you've re-written history you'll need to force push the branch. Use git push --force-with-lease origin [branch here]. This will ensure that if someone else has made changes to the branch you won't clear out their changes and bail out safely
@cwgem
cwgem / ssm_blog_doc.json
Last active July 22, 2017 16:35
SSM Automation Doc for Blog Automation
{
"schemaVersion":"0.3",
"description":"Updates AMI with Linux distribution packages and Amazon software. For details,see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/sysman-ami-walkthrough.html",
"assumeRole":"{{AutomationAssumeRole}}",
"parameters":{
"SourceAmiId":{
"type":"String",
"description":"(Required) The source Amazon Machine Image ID.",
"default":"ami-8b92b4ee"
},
@cwgem
cwgem / codepipeline_handoff.py
Created July 20, 2017 00:22
Having lambada pass off a codepipline task to an EC2 instance
import boto3
import os
def lambda_handler(event, context):
job_id = event['CodePipeline.job']['id']
s3_info = (event['CodePipeline.job']['data']
['inputArtifacts'][0]['location']['s3Location'])
client = boto3.client('ssm')
@cwgem
cwgem / filesize_based_resize_png.py
Created July 18, 2017 16:46
Some code thrown together to explore resizing PNG files to a certain filesize given the oddities the format has regarding that
# Pillow external module
from PIL import Image
# bitmath external module
from bitmath import MB
# Standard system modules
from os import stat
from io import BytesIO
from os.path import expanduser
@cwgem
cwgem / lambda_python_env.json
Created July 16, 2017 23:55
Lambda Python Environment SSM Doc
{
"schemaVersion":"0.3",
"description":"Takes a Lambda AMI and sets up Python3 + virtualenv",
"assumeRole":"{{AutomationAssumeRole}}",
"parameters":{
"SourceAmiId":{
"type":"String",
"description":"(Required) The source Lambda compliant AMI ID.",
"default":"ami-f6035893"
},