Skip to content

Instantly share code, notes, and snippets.

@dgodfrey206
Created October 25, 2017 01:59
Show Gist options
  • Save dgodfrey206/cb48dc0955a77a21e1152a9c5e4d3d84 to your computer and use it in GitHub Desktop.
Save dgodfrey206/cb48dc0955a77a21e1152a9c5e4d3d84 to your computer and use it in GitHub Desktop.
Removes duplicates from lines
#include <iostream>
#include <sstream>
#include <set>
#include <unordered_set>
#include<vector>
using namespace std;
int main() {
#ifdef TESTING
stringstream ss("10 yards of white lace burlap table runner\n"
"10 yards of no fray white lace burlap\n"
"10 yards of finished edge white lace burlap\n"
"10 yards of sewn white lace burlap\n"
"10 yards of sewn edge white lace burlap\n"
"10 yards of white lace burlap table runner\n"
"Sewn edges white lace burlap 10 yards\n"
"Finished edges white lace burlap 10 yards\n"
"White lace burlap no fraying\n"
"White lace burlap 10 yards no fraying\n"
"white lace burlap sashes for wedding\n"
"wedding white lace burlap sashes\n"
"wedding white lace burlap chair sashes\n");
#else
ifstream stream("file.txt");
#endif
unordered_set<string> unique_lines;
for (string line; getline(ss, line); unique_lines.insert(line)) {
if (!unique_lines.count(line)) {
cout << line << endl;
}
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment