When using FactoryGirl to create records in tests, it is necessary to be aware that you are usually creating a lot more
records than is obvious in the code. While this is usually harmless aside from slowing down tests, it is capable of creating
some mind boggling flakiness. Some scenarios where this is probable are tests that count the number of records, tests that are
expecting a certain record and assume that only it exists. Improperly set model level validations can also contribute to this.
When working with records in a way that implicitly sets timestamps with the intention of comparing them it is possible to
create a test which is dependent on code running faster or slower than a certain threshold. This is troublesome since
background processes can have an influence on execution time and running your tests on different machines (i.e. local, ci)
can also produce different execution times. This can be avoided with either explicitly setting time stamps or using Timecop
and understanding the difference between #travel and #freeze.
Any request hitting and outside service is inherently more probably to be flaky. Any time the service in question is
experiencing issues, is unavailable, or there are network issues in general the test will fail. Worst of all is that you
as a developer have no control over this which can result in long periods of time where it is impossible to test your code
properly. These kinds of requests need to be properly subbed either with vcr or direct method stubbing.
While we don't have any experience with async flakiness in RSpec, it is certainly something to keep in mind when investigating a flaky test. If there is any async code running in the flaky test it is a prime candidate for the source of flakiness.