This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| package models | |
| // News <model> | |
| // is used to describe article model. | |
| type News struct { | |
| ID string `json:"_id" bson:"_id"` | |
| Title string `json:"title" bson:"title"` | |
| Preamble string `json:"preamble" bson:"preamble"` | |
| TimeAdded int64 `json:"time_added" bson:"time_added"` | |
| Link string `json:"link" bson:"link"` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| package crawler | |
| import ( | |
| "fmt" | |
| "time" | |
| "github.com/gocolly/colly" | |
| ) | |
| // SecretMag <struct> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| package utils | |
| import ( | |
| "crypto/sha1" | |
| "encoding/hex" | |
| ) | |
| // MakeHash <function> | |
| // is used to create hash from string | |
| func MakeHash(s string) string { |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| package crawler | |
| import ( | |
| "time" | |
| ) | |
| // NewsCrawler <interface> | |
| // is used to describe news crawler class instance | |
| type NewsCrawler interface { | |
| Run() []models.News |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| package db | |
| import ( | |
| "context" | |
| "fmt" | |
| "time" | |
| "go.mongodb.org/mongo-driver/mongo" | |
| "go.mongodb.org/mongo-driver/mongo/options" | |
| "go.mongodb.org/mongo-driver/mongo/readpref" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| package db | |
| // FillSeedsInformation <function> | |
| // is used to fill information that needed before usage of news endpoint | |
| func (s Service) FillSeedsInformation() { | |
| s.fillNewsSources() | |
| s.fillNewsTypes() | |
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| package db | |
| import ( | |
| "go.mongodb.org/mongo-driver/bson" | |
| ) | |
| const ( | |
| newsSourcesCollection = "news_sources" | |
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| package db | |
| import ( | |
| "go.mongodb.org/mongo-driver/bson" | |
| "go.mongodb.org/mongo-driver/mongo/options" | |
| ) | |
| const ( | |
| newsCollection = "news" | |
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| package server | |
| import ( | |
| ) | |
| // Init <function> | |
| // is used to initialize server and all the corresponding services such as DB, Utils, Workers | |
| func Init() { | |
| // utils |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| package crawler | |
| import ( | |
| "time" | |
| ) | |
| // NewsCrawler <interface> | |
| // is used to describe news crawler class instance | |
| type NewsCrawler interface { | |
| Run() []models.News |