Last active
December 19, 2019 06:25
-
-
Save gsscoder/fdd1ec65b1e915b1ba202c3296d92f0f to your computer and use it in GitHub Desktop.
F# script that searches the web using PickAll and counts the words of results descriptions
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| (* | |
| * pickall_count_words.fsx | |
| * - F# Script that demonstrates the use of PickAll (github.com/gsscoder/pickall) | |
| * - Searches the web and counts the words of results descriptions | |
| * - Requires PickAll.dll (0.8.0) and AngleSharp.dll (0.14.0-alpha-787) | |
| * in the script directory | |
| *) | |
| #r "PickAll.dll" | |
| open System | |
| open Microsoft.FSharp.Control | |
| open PickAll | |
| open PickAll.Searchers | |
| let alpha (s : string) = | |
| not (s.ToCharArray() | |
| |> Seq.map (fun x -> Char.IsLetterOrDigit(x) && not (Char.IsWhiteSpace(x))) | |
| |> Seq.contains false) | |
| let query = "Steve Jobs" | |
| let context = (new SearchContext()) | |
| .With<Google>() | |
| .With<DuckDuckGo>() | |
| .With<Yahoo>() | |
| let results = context.SearchAsync(query) | |
| |> Async.AwaitTask | |
| |> Async.RunSynchronously | |
| let words = results | |
| |> Seq.map (fun x -> x.Description.Split()) | |
| |> Seq.concat | |
| words |> Seq.filter alpha | |
| |> Seq.countBy id | |
| |> Seq.sortBy (fun (_, y) -> -y) | |
| |> Seq.iter (fun (x, y) -> printfn "%s %d" x y) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment