Skip to content

Instantly share code, notes, and snippets.

View abitofhelp's full-sized avatar

Michael Gardner abitofhelp

13 followers · 0 following
View GitHub Profile
@abitofhelp
abitofhelp / cs-stringextensions.cs
Last active August 14, 2018 22:01
This gist implements methods that extend the abilities of string objects.
////////////////////////////////////////////////////////////////////////////////////////////////////
// file: Extensions\StringExtensions.cs
//
// summary: Implements the string extensions class
////////////////////////////////////////////////////////////////////////////////////////////////////
using System;
using System.Linq;
using System.Security.Cryptography;
@abitofhelp
abitofhelp / go-refcntbuffer.go
Last active September 15, 2018 01:15
This gist implements a streaming process where a chunk of data is read from a file in a goroutine, the data is passed through a channel to another goroutine, which writes the data to a file. It uses a reference counting pool of buffers.
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// Copyright (c) 2018 A Bit of Help, Inc. - All Rights Reserved, Worldwide.
// Use of this source code is governed by a BSD-style license that can be found in the LICENSE file.
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// Package refcnt implements a reference count based []byte buffer.
package refcnt
import (
"errors"
@abitofhelp
abitofhelp / go-stream-file-between-goroutines-with-channel
Last active August 21, 2024 06:24
This gist implements a streaming process where a chunk of data is read from a file in a goroutine, the data is passed through a channel to another goroutine, which writes the data to a file.
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// Copyright (c) 2018 A Bit of Help, Inc. - All Rights Reserved, Worldwide.
// Use of this source code is governed by a MIT license that can be found in the LICENSE file.
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// Package main implements streaming of a file through a buffered channel where it is written to a new file.
// A data race condition arose in the for loop that reads data from the input file and sending it to the channel.
// I moved the allocation of the buffer to the top of the loop, to resolve the data race issue.
// I've created a companion gist,"go-stream-file-between-goroutines-with-pipe" that resolves the race issue using a
// FIFO pipe. It worked, but the price was that it was slower than using a channel.
@abitofhelp
abitofhelp / go-directed-acyclic-graph-etl.go
Last active July 28, 2018 20:53
This gist implements a directed acyclic graph to implement an extract, transform, and load ("ETL") process for files.
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// Copyright (c) 2018 A Bit of Help, Inc. - All Rights Reserved, Worldwide.
// Use of this source code is governed by a MIT license that can be found in the LICENSE file.
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// Package main implements an ETL process where a file is read, and its contents streamed to a fan-out consisting
// of two, parallel goroutines: One that calculates the MD5 checksum and passes it along to the fan-in stage; The
// other saves the content stream to a new file and passes the file's metadata to the fan-in stage. The fan-in stage
// is the sink for the ETL process. It merges the metadata from the fan-out operations into a JSON object, which is
// persisted to a local file.
@abitofhelp
abitofhelp / go-many-to-many-buffered-channel.go
Last active July 28, 2018 02:21
This gist shows the use of a buffered channel with multiple senders and receivers.
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// Copyright (c) 2018 A Bit of Help, Inc. - All Rights Reserved, Worldwide.
// Use of this source code is governed by a MIT license that can be found in the LICENSE file.
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// Package main is the entry point for the application
// and is responsible for configuring the environment.
package main
import (
@abitofhelp
abitofhelp / go-one-to-many-buffered-channel.go
Last active July 28, 2018 01:33
This gist shows the use of a buffered channel with a single sender and multiple receivers.
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// Copyright (c) 2018 A Bit of Help, Inc. - All Rights Reserved, Worldwide.
// Use of this source code is governed by a MIT license that can be found in the LICENSE file.
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// Package main is the entry point for the application
// and is responsible for configuring the environment.
package main
import (
@abitofhelp
abitofhelp / go-producer-consumer.go
Last active July 26, 2018 19:51
This gist shows how to implement a producer-consumer pattern in go.
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// Copyright (c) 2018 A Bit of Help, Inc. - All Rights Reserved, Worldwide.
// Use of this source code is governed by a MIT license that can be found in the LICENSE file.
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// Package main is the entry point for the application
// and is responsible for configuring the environment.
package main
import (
@abitofhelp
abitofhelp / go-read-file-chunks.go
Last active July 28, 2018 05:42
This gist shows how to read from a file using chunks and calculating the MD5 hash on-the-fly.
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// Copyright (c) 2018 A Bit of Help, Inc. - All Rights Reserved, Worldwide.
// Use of this source code is governed by a MIT license that can be found in the LICENSE file.
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// Package main is the entry point for the application
// and is responsible for configuring the environment.
package main
import (