By default when you open a file it is opened in preview mode. You can tell a tab is a preview if the text describing the file that was opened is itallicized. If you double click on the tab it will remove preview mode, resulting in the file staying open when you open another file.
You can also disable preview mode by adding the following to your setting.json:
