This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import ( | |
"context" | |
"encoding/json" | |
"io/ioutil" | |
"log" | |
"net/http" | |
"os" | |
"github.com/aws/aws-sdk-go/aws" | |
"github.com/aws/aws-sdk-go/aws/credentials" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
package web | |
import ( | |
"bytes" | |
"net/http" | |
"net/http/httptest" | |
"strconv" | |
"testing" | |
"time" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
1. Node changed in Drupal. | |
2. JSON payload added to drupal6_nodevents SQS queue. | |
3. Job picked up and POST request made to storage worker. | |
4. Storage worker gets raw JSON from Drupal ec-services and stores it on S3. | |
5. S3 notification triggered and published to SNS topic. | |
6. Downstream workers (right now cp-worker-format-applenews, cp-worker-cache-prewarm) get the notification pushed to their SQS queues. | |
7. Downstream workers get a POST request to their roor endpoint and do their work. | |
8. In the case of cp-worker-format-applenews, when it transforms and stores the data on S3, a notification is published to another SNS topic. | |
9. The downstream worker (cp-worker-push-applenews) gets the notification it its own SQS queue, gets the data and pushes to Apple News. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# In SNS stack CF: | |
"Outputs" : { | |
"SnsTopicName" : { | |
"Value" : {"Ref" : "SNStopic"} | |
} | |
} | |
# In worker CF: | |
"GetSNSTopicName" : { | |
"Type": "Custom::GetSNSTopicName", |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
map $arg_v $version{ | |
default "0"; | |
~^(?<v>\d{1})$ $v; | |
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
func NewQueryParams(r *http.Request) (qp QueryParams) { | |
query := r.URL.Query() | |
qp.Version = query.Get("version") | |
if qp.Version == "" { | |
qp.Version = DefaultAPIVersion | |
} | |
qp.State = query.Get("state") | |
if qp.State == "" { | |
qp.State = DefaultContentState |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
package main | |
import ( | |
"fmt" | |
"log" | |
"strings" | |
"github.com/PuerkitoBio/goquery" | |
) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{"message": "BOOM!"} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
package main | |
import ( | |
"fmt" | |
"log" | |
"strings" | |
"golang.org/x/net/html" | |
) | |
var content = `<p><!--[if gte mso 9]><xml> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
[ | |
{ | |
"name": "econ_db_count_users", | |
"help": "Current number of users in Drupal.", | |
"interval": 60, | |
"query": "SELECT COUNT(uid) FROM users", | |
"type": "gauge", | |
"datasource": "vulcan_default_datasource" | |
} | |
] |