Skip to content

Instantly share code, notes, and snippets.

View calmh's full-sized avatar
:bowtie:
I may be slow to respond.

Jakob Borg calmh

:bowtie:
I may be slow to respond.
View GitHub Profile
@calmh
calmh / http_proxy.txt
Last active October 15, 2015 01:34
proxy traces
jb@syno:~/src/github.com/syncthing/syncthing git:(master) $ git checkout syncthing/pr/2375
Note: checking out 'syncthing/pr/2375'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
@calmh
calmh / -
Created September 26, 2015 11:22
jb@syno:~/src/github.com/AudriusButkevicius/gohashcompare git:(master 1↑) 1M 7A $ go run main.go
Build: go1.5.1 darwin-amd64
testing: warning: no tests to run
PASS
SHA256-8 5000 731164 ns/op 179.26 MB/s 32 B/op 1 allocs/op
SHAKE256-8 5000 598993 ns/op 218.82 MB/s 64 B/op 1 allocs/op
FNV64-8 10000 213811 ns/op 613.03 MB/s 16 B/op 1 allocs/op
murmur3-128-8 100000 30360 ns/op 4317.24 MB/s 16 B/op 1 allocs/op
jb@syno:~/src/github.com/AudriusButkevicius/gohashcompare git:(master 1↑) 1M 7A $
@calmh
calmh / -
Created September 25, 2015 08:39
jb@syno:~/src/github.com/AudriusButkevicius/gohashcompare git:(master 1↑) 7A $ go run main.go
Build: go1.5.1 darwin-amd64
testing: warning: no tests to run
PASS
SHA256-8 3000 741730 ns/op 176.71 MB/s 32 B/op 1 allocs/op
SHAKE256-8 5000 592125 ns/op 221.36 MB/s 64 B/op 1 allocs/op
FNV-8 10000 217857 ns/op 601.64 MB/s 16 B/op 1 allocs/op
jb@syno:~/src/github.com/AudriusButkevicius/gohashcompare git:(master 1↑) 1M 7A $
@calmh
calmh / datarace.go
Last active September 21, 2015 09:46
package main
import (
"fmt"
"time"
"github.com/thejerf/suture"
)
func main() {
reporting-disabled = false
[meta]
dir = "/var/opt/influxdb/meta"
hostname = "localhost"
bind-address = ":8088"
retention-autocreate = true
election-timeout = "1s"
heartbeat-timeout = "1s"
leader-lease-timeout = "500ms"
@calmh
calmh / db_test.go
Last active September 1, 2015 10:23
package dbtest
import (
"database/sql"
"fmt"
"os"
"sync"
"testing"
"time"
// Copyright 2014 The DST Authors. All rights reserved.
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
package main
import (
"crypto/rand"
"flag"
"io"
jb@syno:/usr/local/go1.5/src ((detached) $ ./all.bash
##### Building Go bootstrap tool.
cmd/dist
##### Building Go toolchain using /usr/local/go1.4.2/.
bootstrap/internal/obj
bootstrap/asm/internal/flags
bootstrap/compile/internal/big
bootstrap/internal/gcprog
bootstrap/internal/obj/arm

Problem

We do two pass scanning to figure out our local contents;

  1. Walk the folder. For each file check if it's already in the database and is a match, otherwise hash it and add to the database. Database is indexes by file name.

  2. Iterate over the database, issuing an os.Lstat() for each item in it. If we get back an error, note the file as deleted.

This breaks when a file is renamed case-only on a case insensitive file system. In step one we find it as a new file, since the database is case

package db
import (
"os"
"testing"
"github.com/boltdb/bolt"
)
func TestBoltTransactions(t *testing.T) {