This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| require 'nokogiri' | |
| require 'fileutils' | |
| require 'active_support/inflector' | |
| def friendly_filename(filename) | |
| filename.gsub(/[^\w\s_-]+/, '') | |
| .gsub(/(^|\b\s)\s+($|\s?\b)/, '\\1\\2') | |
| .gsub(/\s/, '_') | |
| end |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <!DOCTYPE html> | |
| <html> | |
| <head> | |
| <meta charset="UTF-8"> | |
| <!-- IF PEN IS PRIVATE --> | |
| <!-- <meta name="robots" content="noindex"> --> | |
| <!-- END --> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <!DOCTYPE html> | |
| <html> | |
| <head> | |
| <meta charset="UTF-8"> | |
| <!-- IF PEN IS PRIVATE --> | |
| <!-- <meta name="robots" content="noindex"> --> | |
| <!-- END --> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| __all__ = ['WriteToMongo'] | |
| import json | |
| from pymongo import MongoClient | |
| from apache_beam.transforms import PTransform | |
| from apache_beam.io import iobase | |
| class _MongoSink(iobase.Sink): | |
| """A :class:`~apache_beam.io.iobase.Sink`.""" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| class Author | |
| include Mongoid::Document | |
| include Mongoid::Timestamps | |
| field :first_name, type: String | |
| field :last_name, type: String | |
| field :email, type: String | |
| has_many :posts | |
| end |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ae Article.where(author_id: '5bb13233d5cc2ec29c28649b') |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| +--------------------+-----------------------------+ | |
| | Winning Plan | FETCH (7 / 7) -> IXSCAN (7) | | |
| +--------------------+-----------------------------+ | |
| | Used Indexes | author_id_1 (forward) | | |
| +--------------------+-----------------------------+ | |
| | Rejected Plans | 0 | | |
| +--------------------+-----------------------------+ | |
| | Documents Returned | 7 | | |
| +--------------------+-----------------------------+ | |
| | Documents Examined | 7 | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| package com.sandboxws; | |
| import java.util.Arrays; | |
| import org.apache.beam.sdk.Pipeline; | |
| import org.apache.beam.sdk.io.TextIO; | |
| import org.apache.beam.sdk.transforms.Count; | |
| import org.apache.beam.sdk.transforms.FlatMapElements; | |
| import org.apache.beam.sdk.transforms.MapElements; | |
| import org.apache.beam.sdk.values.KV; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Pipeline pipeline = Pipeline.create(); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| PCollection<String> csvRows = pipeline.apply("Read from CSV", TextIO.read().from("./reviews.csv")); |