Skip to content

Instantly share code, notes, and snippets.

@piyo7
Last active November 10, 2015 02:13
Show Gist options
  • Save piyo7/dbd7e2390a95adb53860 to your computer and use it in GitHub Desktop.
Save piyo7/dbd7e2390a95adb53860 to your computer and use it in GitHub Desktop.
スクレイピングをAPI化したい人生だった ref: http://qiita.com/piyo7/items/c84c82012bfec48f7b6a
sbt.version=0.13.8
name := "hello_api"
version := "1.0"
scalaVersion := "2.11.7"
libraryDependencies ++= Seq(
"org.seleniumhq.selenium" % "selenium-java" % "2.48.2"
)
[
"Qiita - A technical knowledge sharing platform for programmers.",
"タグ一覧",
"Qiita:Team",
"About Qiita",
"Advent Calendar",
"ログイン",
"Organization一覧",
"キータ (@Qiita) | Twitter",
"Qiitaで何があったのか - はてな匿名ダイアリー - はてラボ",
"qrank | Qiitaの人気記事ランキング",
"Qiitaってなに?よく分かってなかった方のためのQiita基礎知識|ferret ...",
"Qiita のニュース検索結果",
"週刊 Qiita"
]
> curl -X POST https://XXXXXXX.ap-northeast-1.amazonaws.com/prod/hello_api -H "x-api-key: XXXXXXX" -d "Qiita"
{"message": "Endpoint request timed out"}
import com.gargoylesoftware.htmlunit.BrowserVersion
//import org.openqa.selenium.chrome.ChromeDriver
import org.openqa.selenium.htmlunit.HtmlUnitDriver
import scala.collection.JavaConversions._
object Main {
def main(args: Array[String]) {
println(awsLambda(args.head))
}
def awsLambda(str: String): java.util.List[String] = {
//System.setProperty("webdriver.chrome.driver","C:\\path\\to\\chromedriver.exe")
//val driver = new ChromeDriver()
val driver = new HtmlUnitDriver(BrowserVersion.CHROME)
driver.get("http://www.google.com")
val query = driver.findElementByName("q")
query.sendKeys(str)
query.submit
val results = driver.findElementsByClassName("r").map(_.getText)
driver.quit()
results
}
}
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.0")
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment