As a company grows, many teams find it necessary to collaborate across multiple locations, both within a team and across teams. This is challenging, especially for those who are in an office with a smaller presence or who work remotely full-time.
Teams that span multiple locations will require patience and flexibility as they are likely to encounter technical, interpersonal, and logistical issues. By accepting these limitations and being pragmatic, teams and individuals in this position have thrived and been successful long-term.
This document aims to suggest principles, practices, and tools for establishing an effective and inclusive workflow with remote collaborators. These ideas have been gathered from various teams that engage in these practices, and while not all of them will work out of the box, hopefully they will provide a useful starting point.
- If there are any remote people on your team, treat all conversations as if they’re entirely remote
- Anchors should guide their team in establishing effective remote collaboration principles
- There is no substitute for being in the same place as your team.
- Even if your team is not distributed, be aware of teams that are.
When interacting with individuals and teams in other offices, the practices we adopt can make a significant difference. What follows can be summarized as ‘be cognizant of individuals in other locations, and minimize overhead where possible.’
- Teams should maintain a presence in Slack. Consider having the PM and/or an interrupt pair enable Slack alerts to make your team accessible for others in the organization.
- Leadership should consider the added overhead of cross-office work when allocating less experienced developers to a project
- Anchors should consider whether the team can effectively handle remote participants.
- On-boarding a team member is hard. On-boarding a remote team member is harder.
- Travel can make a huge difference and should happen as often as is practical.
- Take into account local working hours when scheduling team interactions.
- One person from each location should arrive sufficiently early to ensure the technology is setup correctly for them.
- ensure audio/video/additional hardware/software is configured and functional
- check with other locations that all communication works before starting the meeting
- five minutes is usually sufficient for audio/video. More complicated setups may require longer
- For shorter interactions (e.g. standups, a 5-10 minute team conversation) minimizing overhead is key
- appear.in requires less time to log in than google hangouts
- headsets at the pairing stations take less time than booking a room
- Choose communication methods that are inclusive for all participants
- a shared google document is more accessible than writing on a whiteboard
- trello is more accessible than physically moving post-it notes
- a drawing tablet and shared screen is more accessible than diagrams on a whiteboard
- Remote interrupts always take precedence during a conversation
- this is especially important when there more participants in one location than another.
- Watch Dan Wendorf's Lightning Talk entitled “Facilitating for Remotes” (available on the Cloud Foundry tech talks wiki page: https://sites.google.com/a/pivotal.io/cloud-foundry/tech-talks)
- Taking breaks is important. It can be easy to forget to take breaks, and harder to ask your pair for one.
- Just as pairing is a skill, so is remote pairing. It takes time to learn and be effective. Be more patient and empathetic than you might be in person.
- Video sharing on a separate screen can help build empathy and provide visual clues as to what your pair is doing and how they are feeling
- Consider taking turns hosting the remote pairing session; it can share the technological frustrations e.g. screen resolution or lag
- Find out how that team prefers to be contacted (Slack vs. in-person)
- Treat all conversations with that team as if they’re entirely remote (be considerate and inclusive of remote members in conversations)
- When contacting other teams in Slack, try to be clear and concise with your question/issue and be respectful of their time. Jump to video chat if the conversation is too low-bandwidth
The main activities that your remote collaboration workflows should enable are:
- Remote Pair Programming
- Impromptu conversations with any number of remote team members (they should be easy to initiate)
- Meetings with remote team members (e.g. IPMs, retros)
Consider the fact that the hardware and software that is optimal for one of those activities might be different than what works best for another. As a result, remote collaboration requires a few different pieces of tech (hardware and software). Consider the following when deciding what tools to use:
- Choose the communication technologies with lowest overhead so that you are likely to have conversations
- Try a couple of different technologies before you settle on one!
- Some technologies can be buggy/glitchy, especially USB audio adapters. If you're having trouble establishing reliable connections, try another piece of hardware or software; check with others who work remotely - it's likely that someone has encountered your problem and has a solution.
- All computers used by your team would ideally have a consistent software setup (e.g. using sprout), especially when alternating who hosts (which is recommended).
With the right combination of people and technology, remote pair programming can be almost as effective as in-person pair programming. A few people have even said that they prefer pairing remotely as it helps them focus while pairing in a noisy, distraction-filled environment.
-
Sprout Wrap Uses soloist and librarian-chef for recipes
-
Pivotal workstation setup Started as a response to Sprout Wrap. Design goal was to have simple bash scripts that are easy to understand and modify.
-
Alfalfa Started as a response to Sprout Wrap and Workstation Setup. They found it hard to modify sprout-wrap, but disliked using bash scripts.
- Git Author A simple tool to add multiple authors to commit mesages.
- Git Duet Another simple tool to add multiple authors to commit mesages.
- cred-alert Scans repos for credentials and then shouts if it finds them. Add this to your git precommit hooks.
- direnv An environment switcher for the shell.
- jq jq is like sed for JSON data
- Google Hangouts (requires login, so has more overhead when used on conference machines)
- http://appear.in/ - no login required, easy to start conversations with multiple parties. Easy to set up persistent room URLs
- iChat
- Teamspeak server
- 2 or more team members can join conversation (e.g. PM can join the existing chat session that a pair is in)
- only audio
- https://github.com/pivotal-cf-experimental/teamspeak-bosh-release
- Mumble (similar to Teamspeak)
- http://perch.co - video portal
- http://zoom.us - video chat
- Slack integrations exist for some of the above services (instantly start a chat) - see https://slack.com/apps
Detailed blog post about Pivotal's remote pair audio/video setup prevalent in NYC: http://blog.pivotal.io/labs/labs/integrating-remote-developers-foolproof-audio-setups
The goal with sharing screens is to seamlessly simulate the in-person pair programming experience. See also: Screen Sharing Technologies
- High-Bandwidth options: If the host has high upload bandwidth (> 4Mbs), try these:
- Screenhero
- This works with > 2 participants
- OS X Screen Sharing
- OS X Screen Sharing or VNC Client with the Remote Pairing Release
- iChat
- An especially good option if you need to punch through a firewall without a VPN.
- Detailed blog post regarding Screen Sharing vs. iChat: http://remotepairprogramming.com/remote-pair-programming-technology
- Screenhero
- Low-Bandwidth options: if the host has a slow upload bandwidth (< 4Mbs) try these:
- https://atom.io/packages/remote-atom
- https://atom.io/packages/atom-pair
- https://teletype.atom.io/
- tmux, a multi-terminal sharing application
- VSCode Live Share: https://docs.microsoft.com/en-us/visualstudio/liveshare/use/vscode
- Other Screen Sharing Tech:
- TeamViewer -- we have been using this with success, especially for remote pair interviews. No ssh access required.
- LogMeIn -- similar to TeamViewer.
- Skype sharing only -- use as a last resort. Does not allow the remote viewer to control the screen.
- https://realtimeboard.com/
- https://www.draw.io/
- https://www.lucidchart.com/
- https://www.gliffy.com/
- Zoom whiteboard: https://support.zoom.us/hc/en-us/articles/205677665-How-Do-I-Use-Whiteboard-
- Google Slides: https://docs.google.com/presentation
- https://www.deekit.com/
- https://hackmd.io
- Headsets
- Recommended headset: Sennheiser PC 360 or Game One. It costs $299 but makes you sound like Barry White due to incredible microphone noise isolation.
- Headsets with dual 3.5mm (⅛ “) headphone/mic jacks are preferred over USB headsets (fewer driver issues, can be used with iPads).
Sometimes we want to have two headsets at one computer. This is more difficult that it seems. Here are some tips.
First, don't plug two USB headsets into the computer. This won't work without special software. Read more about how it's really hard to use two USB devices here.
Next, what is a splitter, and what works to combine a headphone/mic jack into a single "iPhone-style" combo jack? See the photos below.
The rule goes like this:
- Two rings for your two ears -- It's a headphone splitter.
- Three rings on your jack is for your two ears and your voice -- it's a headphone/mic splitter
Pay attention to the markings on the adapter. Most will indicate that they support either a headphones or microphone
There are some special adapters that make dual iPhone-style port adapters (one male TRRS to two female TRRS).
- Two headsets can be wired together as a unit (which can be plugged in to any workstation so that 2 people can use it to chat with remote people).
- 2x Sennheiser headset with ⅛” headphone and mic jacks
- 1x USB audio adapter
- If you are using a Macbook use a cable-based adapter - because single-piece units tend to block other ports
- Note, the picture above shows a Turtle Beach AMIGO II adapter, but these are not recommended, they tend to be quiet and hard to hear the remote person.
- Although they are unfortunately discontinued, the Sennheiser UUSB adapter works well if you can find one
- Currently ordering and trying out Audio Technica ATR2USB, will update with review.
- 2x stereo ⅛” splitter (1 male end, split to 2 female ends)
- Splitters with 3 metal contacts on the male connectors should work. Those with 4 will not.
- one splitter connects the mics from the two headsets to the mic input of the audio adapter
- second splitter connects the headphones from both headsets to the headphone output of the audio adapter
- Snowball mic is a good quality mic if you want to include your remote pair in local conversations
- USB audio adapters aren't always reliable; if you are having audio problems, make sure your adapter is not plugged in through a hub. Try unplugging and replugging, or try another brand of adapter (look at Chad's setup and other resources at the bottom of this page).
- When going to a meeting, bring an ipad running a chat session with your remote team member(s)
- Using a second monitor (the "tall" monitor) can be handy, especially for tmux + vim. See http://remotepairprogramming.com/post/43644494666/making-your-macs-secondary-monitor-into-the-p
- It's nice to have sidetone, which is the effect of sound picked up by the mic being played back at a low level in the earpiece of the headset. It means you can hear yourself (and your pair, if your headsets are wired together as above). Use Sidetone to add this.
- http://cheat.errtheblog.com/s/tmux
- http://www.dayid.org/os/notes/tm.html
- If using iTerm, enable 256 color mode in preferences -> profiles -> default -> terminal. Set to xterm-256color and open a new tab or window.
- iTerm2 has better mouse support for tmux: http://www.iterm2.com/#/section/home
- Learn tips such as how to setup an ssh tunnel using a SOCKS4 or 5 proxy and set a
keep-alive
to keep connections open without continually getting disconnected. http://www.symkat.com/ssh-tips-and-tricks-you-need - Learn how to setup a reverse ssh tunnel to ssh from a non-firewalled machine to one that is behind a firewall and NAT. https://brendonrobinson.wordpress.com/2011/09/11/ssh-through-firewalls-using-a-reverse-ssh-tunnel/
- sshuttle: A Poor man's VPN Over SSH https://www.unixmen.com/sshuttle-poor-mans-vpn-ssh/
- Remote Pivots may need VPN access. See Request and Use Pivot VPN
- Static IP is nice but often not needed.
- DynDNS -- you can create a free unique URL for your computer using DynDNS.
- Question: Can Pivotal provide a DNS for external computers?
- Get savvy with the advanced networking options on your home router. You will likely have to forward SSH and other traffic to your Pivotal computer.
- NAT forwarding for ssh, screen sharing. See example screenshots attached to this document.
- Skype incoming port forwarding for both home and office machine.
- Realize that the host's upload speed is key, since their upload max is the remote's download max.
- Recommend at least 5Mbs upload for hosting.
Apple Airport Extreme - Screen Sharing
Apple Airport Extreme -- SSH
Mac System Preferences - Remote Login, Remote Management
dyndns.com configuration
- 1 x "budi 25CM headphone splitter" (available at Central Computers, near the SF office). This is the splitter that should be plugged into the workstation's "headphone port".
Note that when considering alternatives (or simply why/how this works):
-
The "25CM" branding appears to be a misnomer. This device is in fact a 3.5mm AUX splitter.
-
The budi splitter does not split input/microphone one way, and output the other. Rather, the input/output are split evenly across both lines.
-
The splitter needs to be a TRRS device, with four conductors (the metal "tip", "rings" and "sleeve") in order to support stereo + microphone.
-
While the budi splitter generally works well, 1 out of 5 failed for us due to faulty construction.
-
2 x Sennheiser PC 350 SE headsets, with included "PCV 07 adaptor". The adapters should, of course, be attached to the budi splitter and the headsets.
Note that:
-
The PCV 07 adapter that ships with the headset (and is separately available and described as "Combo Audio Adaptor - Mac") works much better and more consistently than the standalone (and very similar in appearance) Sennheiser PCV 05[1]. Various trials in our setup revealed the PCV 05 to be a weak link, as it were[2].
-
Given the use of the budi splitter, there is no need for the criss-cross setup that many of us have tried with varying success[3].
-
While I prefer the PC 350 SE -- especially because of its mute-when-boom-is-up feature[4] -- the follow headsets are also know to work with this setup:
- Sennheiser Game Zero
- Sennheiser Game One
- Logitech G230
-
macOS sound preferences should be configured as follows:
- "Output" ... Name: "Headphones"; Type: "Headphone port"
- "Input" ... Name: "External Microphone"; Type: "Microphone port"
- It seems that one indictor of a "good" setup is: the iTunes app will launch :)
- Once you have your dual-headset configuration in place, leave it as is. This always-available arrangement is magical. Footnotes
[1]: While I haven't found any clear proof, it appears that the difference between these adapters may be "OMTP" vs. "CTIA" conductor arrangements.
[2]: If you need to distinguish the two, the included/working PCV 07 adapters have blue coloring, whereas the PCV 05's have gray.
[3]: Some setups start with an adapter connected to the workstation that splits the input and output separately. The idea then is that additional, similar splitters are chained, and criss-crossed to send the (now independent) input and output channels to the headsets. We've seen limited success with such arrangements.
[4]: I like to find subtle collaboration "hacks" like this. In this case, when on a video call, the physical, non-verbal action of raising and lowering the headset boom provides a natural cue of intention to those others on the call.
- Remote Pair Programming Blog http://remotepairprogramming.com/
- The Wide Teams Podcast http://www.wideteams.com/
- Laurie Williams drove deep studies on pair programming and collaborative software process. Her book (alongside R. Kessler) Pair Programming Illuminated is a reference on the matter. Part I is a complete introduction to the technique, including experiments, advantages, misconceptions etc.
- She also wrote several articles, papers and a thesis on these subjects. This article about pair programming culture is fun and a good guide to how to behave when pair programming. Everything about pair programming I learnt in Kindergarten
- Williams-Cockburn paper on Cost and benefits of pair programming summarizes other works on the economics and key advantages of this practice.
- Williams-Kessler-Cunningham-Jeffries Strengthening the case for pair programming exposes evidence (experimentation and real life examples) demystifying common pair programming misconceptions
- Different desk layouts can be tried out to improve the pair programming experience.
- Laurie William’s dissertation on Collaborative Software Process
- Joe Moore's official tech talk. First time I gave this talk, so it's not awesome http://blog.pivotal.io/pivotal-labs/tech-talks/143-remote-pair-programming-people-and-technology
- Joe Moore's Agile2012 version of the Remote Pair Programming talk. Probably my best. http://remotepairprogramming.com/post/63314033830/this-video-was-taken-at-the-agile2012-conference
- Integrating Remote Developers: Intuitive, Flexible Video Conferencing (PivotalBlabs post): http://blog.pivotal.io/labs/labs/integrating-remote-developers-foolproof-audio-setups
- Best Remote Pairing Settings (PivotalBlabs post) - http://blog.pivotal.io/labs/labs/best-remote-pairing-settings
- Best Remote Pairing Audio/Video Settings (PivotalBlabs post) - http://blog.pivotal.io/labs/labs/best-remote-pairing-audio-video-options
- Remote Pairing Headsets List: https://gist.github.com/thewoolleyman/1099721