Skip to content

Instantly share code, notes, and snippets.

@alexandre
Last active September 25, 2015 18:44
Show Gist options
  • Select an option

  • Save alexandre/577590e837d3b2e7cf19 to your computer and use it in GitHub Desktop.

Select an option

Save alexandre/577590e837d3b2e7cf19 to your computer and use it in GitHub Desktop.
Wifi livre SP - Teste para provar que o contador apresenta apenas multiplos de 5/10
alexandre@moes ~ $ mkproject -p $(which python3) wifilivresp-stats
#...processo de instalação
(wifilivresp-stats)alexandre@moes ~/Development/wifilivresp-stats $ pip install flask beautifulsoup4 ipython requests
# ...a ideia é deixar esse script rodando em algum lugar para ver de tempos em tempos se isso é verdade ou não
(wifilivresp-stats)alexandre@moes ~/Development/wifilivresp-stats $ ipython
Python 3.4.3 (default, Jul 28 2015, 18:20:59)
In [1]: import requests as r
In [2]: from bs4 import BeautifulSoup as bsoup
In [3]: url = 'http://www.wifilivre.sp.gov.br/index.php'
In [4]: source = r.get(url).text
In [6]: soup = bsoup(source)
In [8]: menu = soup.select('#menu')[0]
In [9]: menu
Out[9]:
<ul id="menu">
<li class="" style="margin-top:17px;"><a href="index.php?nav=0">O Projeto</a></li>
<li class=""><a href="index.php?nav=1">Características</a></li>
<li>Regiões atendidas</li>
<li class="zonas "><a href="index.php?regiao=CEN"><span class="centro"></span>Centro</a></li>
<li class="zonas "><a href="index.php?regiao=ZN"><span class="norte"></span>Zona Norte</a></li>
<li class="zonas "><a href="index.php?regiao=ZL"><span class="leste"></span>Zona Leste</a></li>
<li class="zonas "><a href="index.php?regiao=ZS"><span class="sul"></span>Zona Sul</a></li>
<li class="zonas "><a href="index.php?regiao=ZO"><span class="oeste"></span>Zona Oeste</a></li>
<li class=""><a href="index.php?nav=2&amp;htm=noticias14.htm">Notícias</a></li>
<li class=""><a href="index.php?nav=4">Perguntas Frequentes</a></li>
<li class=""><a href="index.php?nav=3">Contatos</a></li>
</ul>
In [10]: menu.select('a')
Out[10]:
[<a href="index.php?nav=0">O Projeto</a>,
<a href="index.php?nav=1">Características</a>,
<a href="index.php?regiao=CEN"><span class="centro"></span>Centro</a>,
<a href="index.php?regiao=ZN"><span class="norte"></span>Zona Norte</a>,
<a href="index.php?regiao=ZL"><span class="leste"></span>Zona Leste</a>,
<a href="index.php?regiao=ZS"><span class="sul"></span>Zona Sul</a>,
<a href="index.php?regiao=ZO"><span class="oeste"></span>Zona Oeste</a>,
<a href="index.php?nav=2&amp;htm=noticias14.htm">Notícias</a>,
<a href="index.php?nav=4">Perguntas Frequentes</a>,
<a href="index.php?nav=3">Contatos</a>]
In [12]: menu.select('.zonas')
Out[12]:
[<li class="zonas "><a href="index.php?regiao=CEN"><span class="centro"></span>Centro</a></li>,
<li class="zonas "><a href="index.php?regiao=ZN"><span class="norte"></span>Zona Norte</a></li>,
<li class="zonas "><a href="index.php?regiao=ZL"><span class="leste"></span>Zona Leste</a></li>,
<li class="zonas "><a href="index.php?regiao=ZS"><span class="sul"></span>Zona Sul</a></li>,
<li class="zonas "><a href="index.php?regiao=ZO"><span class="oeste"></span>Zona Oeste</a></li>]
In [14]: zonas = [z.a for z in menu.select('.zonas')]
In [15]: zonas
Out[15]:
[<a href="index.php?regiao=CEN"><span class="centro"></span>Centro</a>,
<a href="index.php?regiao=ZN"><span class="norte"></span>Zona Norte</a>,
<a href="index.php?regiao=ZL"><span class="leste"></span>Zona Leste</a>,
<a href="index.php?regiao=ZS"><span class="sul"></span>Zona Sul</a>,
<a href="index.php?regiao=ZO"><span class="oeste"></span>Zona Oeste</a>]
In [17]: url
Out[17]: 'http://www.wifilivre.sp.gov.br/index.php'
In [18]: main = 'http://www.wifilivre.sp.gov.br/'
In [24]: foo.get('href')
Out[24]: 'index.php?regiao=CEN'
In [25]: zonas_code = [bsoup(r.get(main + z.get('href')).text) for z in zonas]
In [34]: zona_spots = [(z, [a for a in z.select('.conteudo, a') if 'exibe' in a.get('href','')]) for z in zonas_code]
In [36]: all_spots_link = [a for s in zona_spots for a in s[1]]
In [37]: all_spots_link
Out[37]:
[<a href="index.php?exibe=11">CENTRO CULTURAL SÃO PAULO</a>,
<a href="index.php?exibe=10">LARGO DO AROUCHE</a>,
<a href="index.php?exibe=4">LARGO DO CAMBUCI</a>,
<a href="index.php?exibe=14">LARGO DO PAISSANDÚ</a>,
<a href="index.php?exibe=17">LARGO SANTA CECILIA</a>,
<a href="index.php?exibe=18">LARGO SÃO BENTO</a>,
<a href="index.php?exibe=16">LARGO SÃO FRANCISCO</a>,
<a href="index.php?exibe=1">MASP</a>,
<a href="index.php?exibe=22">MERCADO MUNICIPAL</a>,
<a href="index.php?exibe=8">PARQUE DA ACLIMAÇÃO</a>,
<a href="index.php?exibe=3">PARQUE DA LUZ</a>,
<a href="index.php?exibe=21">PARQUE DOM PEDRO II</a>,
<a href="index.php?exibe=13">PRAÇA DA BANDEIRA</a>,
<a href="index.php?exibe=9">PRAÇA DA LIBERDADE</a>,
<a href="index.php?exibe=15">PRAÇA DA REPÚBLICA</a>,
<a href="index.php?exibe=23">PRAÇA DA SÉ</a>,
<a href="index.php?exibe=20">PRAÇA DO PATRIARCA</a>,
<a href="index.php?exibe=5">PRAÇA DOM JOSÉ GASPAR</a>,
<a href="index.php?exibe=2">PRAÇA DON ORIONE</a>,
<a href="index.php?exibe=12">PRAÇA RAMOS DE AZEVEDO</a>,
<a href="index.php?exibe=7">PRAÇA ROOSEVELT</a>,
<a href="index.php?exibe=6">PRAÇA ROTARY</a>,
<a href="index.php?exibe=19">PÁTEO DO COLÉGIO</a>,
<a href="index.php?exibe=64">LARGO DA MATRIZ</a>,
<a href="index.php?exibe=60">LARGO DO JAPONES</a>,
<a href="index.php?exibe=73">PARQUE DOMINGOS LUÍS</a>,
<a href="index.php?exibe=78">PARQUE TIETÊ</a>,
<a href="index.php?exibe=71">PRAÇA 14º ÁREA / RUA CALDAS NOVAS / RUA DOZE</a>,
<a href="index.php?exibe=67">PRAÇA DA CULTURA</a>,
<a href="index.php?exibe=65">PRAÇA DOUTOR JOÃO BATISTA VASQUES</a>,
<a href="index.php?exibe=69">PRAÇA FUHAD H. SMAIRE</a>,
<a href="index.php?exibe=70">PRAÇA JOÃO BOLDO</a>,
<a href="index.php?exibe=77">PRAÇA LOURENÇO DE BELLIS</a>,
<a href="index.php?exibe=62">PRAÇA MARCO ANTONIO PRIMON MAESTRE</a>,
<a href="index.php?exibe=66">PRAÇA MARIQUINHA SCIASCIA</a>,
<a href="index.php?exibe=63">PRAÇA NA RUA DAS IBIRAIARAS</a>,
<a href="index.php?exibe=74">PRAÇA NOSSA SENHORA DOS PRAZERES</a>,
<a href="index.php?exibe=76">PRAÇA NOVO MUNDO</a>,
<a href="index.php?exibe=75">PRAÇA OSCAR DA SILVA</a>,
<a href="index.php?exibe=72">PRAÇA VEREADOR ANTONIO SAMPAIO / AV ULTRAMARINO</a>,
<a href="index.php?exibe=68">PRAÇA VIGÁRIO JOÃO G. DE LIMA (PRAÇA DO SAMBA)</a>,
<a href="index.php?exibe=61">TERMINAL CASA VERDE</a>,
<a href="index.php?exibe=26">LARGO DA CONCÓRDIA</a>,
<a href="index.php?exibe=49">LARGO DO ROSÁRIO</a>,
<a href="index.php?exibe=41">PARQUE CHICO MENDES</a>,
<a href="index.php?exibe=58">PARQUE ECOLÓGICO DA VILA PRUDENTE</a>,
<a href="index.php?exibe=48">PARQUE LINEAR TIQUATIRA</a>,
<a href="index.php?exibe=45">PARQUE RAUL SEIXAS</a>,
<a href="index.php?exibe=39">PARQUE SANTA AMÉLIA</a>,
<a href="index.php?exibe=46">PRAÇA AUGUSTO DOMINGUES ALVES MAIA</a>,
<a href="index.php?exibe=44">PRAÇA BRASIL (PRAÇA MÃE MENININHA DO GANTOIS)</a>,
<a href="index.php?exibe=38">PRAÇA CECÍLIA MARQUES DE ARAÚJO</a>,
<a href="index.php?exibe=27">PRAÇA CIRO PONTES / AO LADO DO SENAI</a>,
<a href="index.php?exibe=54">PRAÇA CRAVEIRO DO CAMPO</a>,
<a href="index.php?exibe=50">PRAÇA DA CONQUISTA (PRAÇA VEREADOR JOÃO APARECIDO DE PAULA)</a>,
<a href="index.php?exibe=35">PRAÇA DA ESTAÇÃO VILA MARA</a>,
<a href="index.php?exibe=47">PRAÇA DILVA GOMES MARTINS (COHAB 1)</a>,
<a href="index.php?exibe=33">PRAÇA DO 65 / POMBAS URBANAS</a>,
<a href="index.php?exibe=34">PRAÇA DO TERMINAL METALÚRGICOS</a>,
<a href="index.php?exibe=52">PRAÇA FELISBERTO FERNANDES DA SILVA / LARGO SÃO MATEUS</a>,
<a href="index.php?exibe=56">PRAÇA FORTUNATO DA SILVEIRA / UNICSUL</a>,
<a href="index.php?exibe=25">PRAÇA GENERAL HUMBERTO DE SOUSA MELLO</a>,
<a href="index.php?exibe=36">PRAÇA GIOVANI FANI</a>,
<a href="index.php?exibe=40">PRAÇA JAGUAMITANGA</a>,
<a href="index.php?exibe=37">PRAÇA JESUS TEIXEIRA DA COSTA</a>,
<a href="index.php?exibe=28">PRAÇA KANTUTA</a>,
<a href="index.php?exibe=43">PRAÇA NA RUA GREGÓRIO RAMALHO</a>,
<a href="index.php?exibe=51">PRAÇA OSLEI FRANCISCO BORGES</a>,
<a href="index.php?exibe=53">PRAÇA OSVALDO LUÍS DA SILVEIRA</a>,
<a href="index.php?exibe=55">PRAÇA PADRE ALEIXO (DO FORRÓ)</a>,
<a href="index.php?exibe=57">PRAÇA PADRE DAMIÃO</a>,
<a href="index.php?exibe=42">PRAÇA PROFESSORAS (NA AV. DAS ALAMANDAS)</a>,
<a href="index.php?exibe=32">PRAÇA SAMPAIO VIDAL</a>,
<a href="index.php?exibe=29">PRAÇA SILVIO ROMERO</a>,
<a href="index.php?exibe=30">PRAÇA SÃO JOÃO VICENZOTTO</a>,
<a href="index.php?exibe=24">PRAÇA SÃO LUIS DO CURU</a>,
<a href="index.php?exibe=59">PRAÇA TORQUATO PLAZA</a>,
<a href="index.php?exibe=31">PRAÇA VICENTE FALCETTA</a>,
<a href="index.php?exibe=115">ALDEIA TENONDE-PORÃ</a>,
<a href="index.php?exibe=102">CALÇADÃO CULTURAL DO GRAJAÚ</a>,
<a href="index.php?exibe=100">CAMPO DO PALMEIRINHA - PARAISÓPOLIS</a>,
<a href="index.php?exibe=118">CDC CAMPO BELO</a>,
<a href="index.php?exibe=108">CENTRO DE CONVIVÊNCIA EDUCATIVO E CULTURAL DE HELIÓPOLIS</a>,
<a href="index.php?exibe=94">LARGO DE MOEMA</a>,
<a href="index.php?exibe=96">LARGO DONA ANA ROSA</a>,
<a href="index.php?exibe=107">LARGO SANTA ANGELA</a>,
<a href="index.php?exibe=121">PARQUE DA INDEPENDÊNCIA</a>,
<a href="index.php?exibe=111">PARQUE DO NABUCO</a>,
<a href="index.php?exibe=99">PARQUE SANTO DIAS</a>,
<a href="index.php?exibe=112">POLO CULTURAL DO JARDIM ÂNGELA</a>,
<a href="index.php?exibe=104">PRAIA DO SOL</a>,
<a href="index.php?exibe=106">PRAÇA ALEXANDRE FRANCISCO REBOUÇAS</a>,
<a href="index.php?exibe=110">PRAÇA ALFREDO EGYDIO DE SOUZA ARANHA (PARQUE LINA E PAULO RAIA)</a>,
<a href="index.php?exibe=105">PRAÇA BACHAREL FERNANDO BRAGA PEREIRA DA ROCHA</a>,
<a href="index.php?exibe=109">PRAÇA DIRCEU DE CASTRO FONTOURA</a>,
<a href="index.php?exibe=114">PRAÇA DO FEIRÃO SÃO LUIS</a>,
<a href="index.php?exibe=113">PRAÇA DO LARGO DE PIRAPORINHA</a>,
<a href="index.php?exibe=101">PRAÇA ESCOLAR</a>,
<a href="index.php?exibe=120">PRAÇA FLORIANO PEIXOTO</a>,
<a href="index.php?exibe=117">PRAÇA JOÃO ADÃO E PRAÇA JACÓ REIMBERG FILHO</a>,
<a href="index.php?exibe=103">PRAÇA JOÃO BEIÇOLA</a>,
<a href="index.php?exibe=98">PRAÇA JOÃO TADEU PRIOLLI (PRAÇA DO CAMPO LIMPO)</a>,
<a href="index.php?exibe=116">PRAÇA JÚLIO CÉSAR DE CAMPOS</a>,
<a href="index.php?exibe=97">PRAÇA LUISA P DOS SANTOS (PRAÇA ARARIBA)</a>,
<a href="index.php?exibe=95">PRAÇA SANTA RITA DE CÁSSIA</a>,
<a href="index.php?exibe=119">PRAÇA TUNEY ARANTES</a>,
<a href="index.php?exibe=93">LARGO DA BATATA</a>,
<a href="index.php?exibe=86">PARQUE ALFREDO VOLPI</a>,
<a href="index.php?exibe=87">PARQUE RAPOSO TAVARES</a>,
<a href="index.php?exibe=91">PRAÇA ARLINDO ROSSI</a>,
<a href="index.php?exibe=92">PRAÇA BENEDITO CALIXTO</a>,
<a href="index.php?exibe=79">PRAÇA CONDE FRANCISCO MATARAZZO</a>,
<a href="index.php?exibe=82">PRAÇA CORNÉLIA</a>,
<a href="index.php?exibe=90">PRAÇA DO PÔR-DO-SOL</a>,
<a href="index.php?exibe=85">PRAÇA ELIS REGINA</a>,
<a href="index.php?exibe=89">PRAÇA ENG. NORIWUKI YAMAMOTO</a>,
<a href="index.php?exibe=81">PRAÇA GEN. PORTO CARREIRO</a>,
<a href="index.php?exibe=80">PRAÇA GENERAL GUIMARÃES</a>,
<a href="index.php?exibe=88">PRAÇA WILSON MOREIRA DA COSTA</a>,
<a href="index.php?exibe=83">PRAÇA ZILDA NATEL</a>]
In [41]: all_spots_source = [bsoup(r.get(main + a.get('href')).text).select('.detalhes')[0] for a in all_spots_link]
In [45]: all_spots_source[0].select('td, span')
Out[45]:
[<td> <strong><img src="imgs/icoRegiao.gif">Região:</img></strong><span class="centro"> CEN</span></td>,
<td> <strong><img src="imgs/icoLote.gif">Lote: </img></strong><span class="centro">LOTE1</span></td>,
<td> <strong><img src="imgs/icoAcessos.gif">Acessos:</img></strong> <span class="centro">250</span></td>,
<td>
<!-- \ ************** <strong><img src="imgs/icoCriticidade.gif" >Criticidade:</strong> <span class='centro'>%</span> -->
</td>,
<span class="centro"> CEN</span>,
<span class="centro">LOTE1</span>,
<span class="centro">250</span>]
In [46]: [x for x in all_spots_source[0].select('td, span') if x.text.isdigit()]
Out[46]: [<span class="centro">250</span>]
In [47]: [x.text for x in all_spots_source[0].select('td, span') if x.text.isdigit()]
Out[47]: ['250']
In [48]: [x.text for item in all_spots_source for x in item.select('td, span') if x.text.isdigit()]
Out[48]:
['250', '250', '50', '50', '75', '100', '100', '250', '250', '150', '250', '250', '100', '250', '250', '150',
'100', '150', '50', '100', '250', '100', '50', '125', '100', '125', '250', '50', '75', '100', '75', '75', '50',
'75', '75', '100', '50', '100', '75', '50', '75', '100', '150', '50', '100', '125', '100', '100', '100', '50',
'150', '50', '50', '50', '75', '50', '50', '150', '150', '100', '100', '75', '50', '50', '100', '125', '75', '50',
'50', '150', '100', '75', '50', '100', '50', '50', '50', '50', '50', '150', '150', '50', '125', '125', '75', '75',
'250', '100', '125', '50', '75', '50', '75', '75', '75', '75', '75', '100', '150', '50', '75', '150', '50', '100',
'50', '50', '125', '75', '75', '100', '250', '50', '75', '100', '75', '50', '75', '50', '75', '50']
In [49]: len([x.text for item in all_spots_source for x in item.select('td, span') if x.text.isdigit()])
Out[49]: 120
# Estranho, não?!
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment