This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| def policy_checker(name): | |
| """ | |
| Check if there is a need to take an action for a policy. | |
| Args: | |
| name: policy name | |
| Returns: | |
| """ | |
| policy = PolicyModel.query(PolicyModel.Name == name).get() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| cron: | |
| - description: "Run scheduled tasks" | |
| url: /tasks/schedule | |
| schedule: every 60 minutes synchronized | |
| target: zorya-backend |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| """Admin view """ | |
| import flask_admin | |
| from flask_admin.contrib import appengine | |
| from wtforms import validators | |
| from collections import OrderedDict | |
| from validators import GreaterEqualThan, SmallerEqualThan | |
| class LastUpdatedOrderedDict(OrderedDict): | |
| """Store items in the order the keys were last added""" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| def calc_slope(self, minuets): | |
| """ | |
| Calculate the slope of available memory change. | |
| :param: minuets how long to go back in time | |
| """ | |
| met = metrics.Metrics(self.cluster_name) | |
| series = met.read_timeseries('YARNMemoryAvailablePercentage', minuets) | |
| retlist = [] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| def calc_how_many(self): | |
| """ | |
| Calculate how many new nodes of each type we need. | |
| :return: | |
| """ | |
| # No allocated memory so we don't need any workers above the | |
| # bare minimum | |
| if self.scale_to != -1: | |
| self.total = self.min_instances |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| @app.route('/tasks/check-load') | |
| def check_load(): | |
| """Entry point for cron task that launches a task for each cluster | |
| check cluster stats""" | |
| clusters = settings.get_all_clusters_settings() | |
| for cluster in clusters.iter(): | |
| task = taskqueue.add(queue_name='shamash', | |
| url="/monitors", | |
| method='GET', | |
| params={'cluster_name': cluster.Cluster}) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| cron: | |
| - description: "check cluster load" | |
| url: /tasks/check-load | |
| schedule: every 2 minutes |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| def get_tf_record(sentence): | |
| global words | |
| # tokenize the pattern | |
| sentence_words = nltk.word_tokenize(sentence) | |
| # stem each word | |
| sentence_words = [stemmer.stem(word.lower()) for word in sentence_words] | |
| # bag of words | |
| bow = [0]*len(words) | |
| for s in sentence_words: | |
| for i, w in enumerate(words): |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # shuffle our features and turn into np.array as tensorflow takes in numpy array | |
| random.shuffle(training) | |
| training = np.array(training) | |
| # trainX contains the Bag of words and train_y contains the label/ category | |
| train_x = list(training[:, 0]) | |
| train_y = list(training[:, 1]) | |
| # reset underlying graph data | |
| tf.reset_default_graph() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| SELECT review ,(communication+price+wait+quality) as score | |
| FROM [[aviv-playground:telegrass_reviews.reviews] | |
| WHERE RAND() < 0.7 |