finance-pandas-mysql-scrapy-scikit_learn-pythorch备忘

finance-pandas-mysql-scrapy-scikit_learn-pythorch备忘

#mysql批量插入效率提升问题。#SQLAlchemy
https://towardsdatascience.com/how-to-perform-bulk-inserts-with-sqlalchemy-efficiently-in-python-23044656b97d
SQLAlchemy
https://towardsdatascience.com/a-minimalist-end-to-end-scrapy-tutorial-part-iii-bcd94a2e8bf3
https://www.tutorialspoint.com/sqlalchemy/sqlalchemy_quick_guide.htm



来源website，以及个人总结

query = ("SELECT price, house_size, year_built FROM Listing") df = pd.read_sql(query, self.conn) 
d = {'Mean': df.mean(), 'Min': df.min(), 'Max': df.max(), 'Median': df.median()} 
return pd.DataFrame.from_dict(d, dtype='int32')[["Min", "Max", "Mean", "Median"]].transpose() 

query = "SELECT price, year_built FROM Listing" df = pd.read_sql(query, self.conn) 
x = df["year_built"].tolist() y = df["price"].tolist() c = Counter(zip(x, y)) 
s = [10 * c[(xx, yy)] for xx, yy in zip(x, y)] df.plot(kind="scatter", x="year_built", y="price", s=s, color="blue") 
yy, locs = plt.yticks() ll = ['%.0f' % a for a in yy] plt.yticks(yy, ll) plt.show()