Skip to content

Instantly share code, notes, and snippets.

@zhaokuohaha
zhaokuohaha / Robots协议和robots.txt解读.md
Last active October 30, 2024 02:25
robots.txt基本语法, 以及robots协议的各家搜索引擎实现。

文件简介

简单的说 Robot 协议是用于告诉网络爬虫(主要是搜索引擎爬虫)本域名下的网页中, 那些部分是不应该爬取的,具体来说就是在网站的域名根目录下添加一份robots.txt文件, 并在文件中声明对应的规则. 由于 Url 对大小写敏感, 所以 robots.txt 文件名要求全部小写.

搜索引擎爬虫在爬取一个网站时, 首先检查是否存在 robots.txt 文件, 如果存在, 则在爬取该网站下其他网页时, 应遵循文件所定义的范围进行爬取.

需要注意的是 Robots 协议是一个共同遵循的"道德约定", 但是不具有法律效应, 同时也不保证所有搜索引擎都会遵守这个约定

内容和语法

@sketchpunk
sketchpunk / b64_to_float32Array.js
Created September 1, 2017 03:08
Base64 String to Float32Array in Javascript, good for WebGL
var blob = window.atob("AAAAAAAAAAAAAAAAAACAPwAAAAAAAAAAAAAAAAAAgD8AAAAA"), // Base64 string converted to a char array
fLen = blob.length / Float32Array.BYTES_PER_ELEMENT, // How many floats can be made, but be even
dView = new DataView( new ArrayBuffer(Float32Array.BYTES_PER_ELEMENT) ), // ArrayBuffer/DataView to convert 4 bytes into 1 float.
fAry = new Float32Array(fLen), // Final Output at the correct size
p = 0; // Position
for(var j=0; j < fLen; j++){
p = j * 4;
dView.setUint8(0,blob.charCodeAt(p));
dView.setUint8(1,blob.charCodeAt(p+1));
@jwilson8767
jwilson8767 / es6-element-ready.js
Last active February 24, 2025 08:50
Wait for an element to exist. ES6, Promise, MutationObserver
// MIT Licensed
// Author: jwilson8767
/**
* Waits for an element satisfying selector to exist, then resolves promise with the element.
* Useful for resolving race conditions.
*
* @param selector
* @returns {Promise}
*/
@FranklinYu
FranklinYu / README.markdown
Last active June 1, 2025 01:40
links for old versions of Docker for Mac (inspired by docker/for-mac#1120)

links for old versions of Docker for Mac

Deprecated

Docker provides download links in release note. They promised that

(we) will also include download links in release notes for future releases.

Note:

@ntuaha
ntuaha / getDateString.js
Last active September 20, 2021 21:36
Javascript 漂亮日期顯示 YYYYMMDDhhmmss or YYYY-MM-DD hh:mm:ss
'use strict'
function pad(v){
return (v<10)?'0'+v:v
}
function getDateString(d){
var year = d.getFullYear();
var month = pad(d.getMonth()+1);
var day = pad(d.getDate());
@kottenator
kottenator / simple-pagination.js
Created July 13, 2015 20:44
Simple pagination algorithm
// Implementation in ES6
function pagination(c, m) {
var current = c,
last = m,
delta = 2,
left = current - delta,
right = current + delta + 1,
range = [],
rangeWithDots = [],
l;
@Ivlyth
Ivlyth / js-date-format.js
Last active January 1, 2025 06:45
format javascript date to format "YYYY-mm-dd HH:MM:SS"
var d = new Date();
d = new Date(d.getTime() - 3000000);
var date_format_str = d.getFullYear().toString()+"-"+((d.getMonth()+1).toString().length==2?(d.getMonth()+1).toString():"0"+(d.getMonth()+1).toString())+"-"+(d.getDate().toString().length==2?d.getDate().toString():"0"+d.getDate().toString())+" "+(d.getHours().toString().length==2?d.getHours().toString():"0"+d.getHours().toString())+":"+((parseInt(d.getMinutes()/5)*5).toString().length==2?(parseInt(d.getMinutes()/5)*5).toString():"0"+(parseInt(d.getMinutes()/5)*5).toString())+":00";
console.log(date_format_str);
//2015-03-31 13:35:00
@kiinlam
kiinlam / regex_example.txt
Last active July 26, 2022 13:04
正则匹配
正则匹配
匹配中文字符的正则表达式: [\u4e00-\u9fa5]
评注:匹配中文还真是个头疼的事,有了这个表达式就好办了
匹配双字节字符(包括汉字在内):[^\x00-\xff]
评注:可以用来计算字符串的长度(一个双字节字符长度计2,ASCII字符计1)
匹配空白行的正则表达式:\n\s*\r
评注:可以用来删除空白行
@Swaagie
Swaagie / visibly.js
Last active December 21, 2015 00:39 — forked from addyosmani/visibly.js
Updated visibly against firefox native API
/*!
* isVis - v0.5.5 Aug 2011 - Page Visibility API Polyfill
* Copyright (c) 2011 Addy Osmani
* Dual licensed under the MIT and GPL licenses.
*/
(function () {
window.visibly = {
b: null,
q: document,
@jareware
jareware / SCSS.md
Last active May 18, 2025 18:44
Advanced SCSS, or, 16 cool things you may not have known your stylesheets could do

⇐ back to the gist-blog at jrw.fi

Advanced SCSS

Or, 16 cool things you may not have known your stylesheets could do. I'd rather have kept it to a nice round number like 10, but they just kept coming. Sorry.

I've been using SCSS/SASS for most of my styling work since 2009, and I'm a huge fan of Compass (by the great @chriseppstein). It really helped many of us through the darkest cross-browser crap. Even though browsers are increasingly playing nice with CSS, another problem has become very topical: managing the complexity in stylesheets as our in-browser apps get larger and larger. SCSS is an indispensable tool for dealing with this.

This isn't an introduction to the language by a long shot; many things probably won't make sense unless you have some SCSS under your belt already. That said, if you're not yet comfy with the basics, check out the aweso