Skip to content

Instantly share code, notes, and snippets.

@halfak

halfak/example_revscoring_profile.md

Created November 15, 2019 15:52
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save halfak/b2a968e23d70887d29a6ec6b6a0cdeaa to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save halfak/b2a968e23d70887d29a6ec6b6a0cdeaa to your computer and use it in GitHub Desktop.
Download ZIP
Example profile from running extract on our article quality model dataset for Enwiki
Raw
example_revscoring_profile.md

Extracting 36 values:

  • feature.wikitext.revision.chars
  • feature.wikitext.revision.content_chars
  • feature.wikitext.revision.ref_tags
  • feature.(wikitext.revision.ref_tags / max(wikitext.revision.content_chars, 1))
  • feature.wikitext.revision.wikilinks
  • feature.(wikitext.revision.wikilinks / max(wikitext.revision.content_chars, 1))
  • feature.wikitext.revision.external_links
  • feature.(wikitext.revision.external_links / max(wikitext.revision.content_chars, 1))
  • feature.wikitext.revision.headings_by_level(2)
  • feature.(wikitext.revision.headings_by_level(2) / max(wikitext.revision.content_chars, 1))
  • feature.wikitext.revision.headings_by_level(3)
  • feature.(wikitext.revision.headings_by_level(3) / max(wikitext.revision.content_chars, 1))
  • feature.enwiki.revision.image_links
  • feature.(enwiki.revision.image_links / max(wikitext.revision.content_chars, 1))
  • feature.enwiki.revision.category_links
  • feature.(enwiki.revision.category_links / max(wikitext.revision.content_chars, 1))
  • feature.(enwiki.revision.shortened_footnote_templates + wikitext.revision.ref_tags)
  • feature.((enwiki.revision.shortened_footnote_templates + wikitext.revision.ref_tags) / max(wikitext.revision.content_chars, 1))
  • feature.(enwiki.revision.cite_templates + enwiki.revision.shortened_footnote_templates)
  • feature.((enwiki.revision.cite_templates + enwiki.revision.shortened_footnote_templates) / max(wikitext.revision.content_chars, 1))
  • feature.((enwiki.revision.cite_templates + enwiki.revision.shortened_footnote_templates) / max((enwiki.revision.shortened_footnote_templates + wikitext.revision.ref_tags), 1))
  • feature.max(((enwiki.revision.shortened_footnote_templates + wikitext.revision.ref_tags) - (enwiki.revision.cite_templates + enwiki.revision.shortened_footnote_templates)), 0)
  • feature.(max(((enwiki.revision.shortened_footnote_templates + wikitext.revision.ref_tags) - (enwiki.revision.cite_templates + enwiki.revision.shortened_footnote_templates)), 0) / max(wikitext.revision.content_chars, 1))
  • feature.enwiki.revision.non_cite_templates
  • feature.(enwiki.revision.non_cite_templates / max(wikitext.revision.content_chars, 1))
  • feature.enwiki.revision.infobox_templates
  • feature.(enwiki.revision.cn_templates + 1)
  • feature.(enwiki.revision.cn_templates / max(wikitext.revision.content_chars, 1))
  • feature.(enwiki.revision.who_templates + 1)
  • feature.(enwiki.revision.who_templates / max(wikitext.revision.content_chars, 1))
  • feature.enwiki.main_article_templates
  • feature.(enwiki.main_article_templates / max(wikitext.revision.content_chars, 1))
  • feature.(english.stemmed.revision.stems_length / max(wikitext.revision.content_chars, 1))
  • feature.log((enwiki.revision.paragraphs_without_refs_total_length + 1))
  • feature.len(<datasource.english.words_to_watch.revision.matches>)
  • feature.(len(<datasource.english.words_to_watch.revision.matches>) / max(len(<datasource.wikitext.revision.words>), 1))

Batch size: 50

stat value
batch_extractions 20
total_time 950.398
min_time 28.175
max_time 85.468
mean_time 47.52
median_time 43.58

Features

name executions min max mean median
feature.0 1000 0 0.002 0 0
feature.wikitext.revision.ref_tags 1000 0 0.005 0 0
feature.(enwiki.revision.image_links / max(wikitext.revision.content_chars, 1)) 1000 0 0.004 0 0
feature.wikitext.revision.headings_by_level(3) 1000 0 0.001 0 0
feature.(len(<datasource.english.words_to_watch.revision.matches>) / max(len(<datasource.wikitext.revision.words>), 1)) 1000 0 0.002 0 0
feature.max(wikitext.revision.content_chars, 1) 1000 0 0.002 0 0
feature.wikitext.revision.templates 1000 0 0 0 0
feature.(enwiki.revision.category_links / max(wikitext.revision.content_chars, 1)) 1000 0 0 0 0
feature.enwiki.revision.paragraphs_without_refs_total_length 1000 0 0.006 0 0
feature.(enwiki.revision.who_templates + 1) 1000 0 0.005 0 0
feature.enwiki.revision.non_cite_templates 1000 0 0 0 0
feature.log((enwiki.revision.paragraphs_without_refs_total_length + 1)) 1000 0 0.004 0 0
feature.max(len(<datasource.wikitext.revision.words>), 1) 1000 0 0.003 0 0
feature.wikitext.revision.wikilinks 1000 0 0.004 0 0
feature.enwiki.revision.shortened_footnote_templates 1000 0 0.006 0 0
feature.((enwiki.revision.cite_templates + enwiki.revision.shortened_footnote_templates) / max((enwiki.revision.shortened_footnote_templates + wikitext.revision.ref_tags), 1)) 1000 0 0.004 0 0
feature.len(<datasource.wikitext.revision.words>) 1000 0 0.012 0 0
feature.wikitext.revision.chars 1000 0 0.006 0 0
feature.((enwiki.revision.shortened_footnote_templates + wikitext.revision.ref_tags) / max(wikitext.revision.content_chars, 1)) 1000 0 0.007 0 0
feature.(enwiki.revision.paragraphs_without_refs_total_length + 1) 1000 0 0.001 0 0
feature.(wikitext.revision.ref_tags / max(wikitext.revision.content_chars, 1)) 1000 0 0.006 0 0
feature.(enwiki.revision.cn_templates + 1) 1000 0 0.002 0 0
feature.(enwiki.revision.non_cite_templates / max(wikitext.revision.content_chars, 1)) 1000 0 0.006 0 0
feature.enwiki.revision.category_links 1000 0 0.002 0 0
feature.enwiki.main_article_templates 1000 0 0.009 0 0

Datasources

name executions min max mean median
datasource.english.stemmed.revision.stems 1000 0.011 1.029 0.194 0.15
datasource.enwiki.revision.revision.paragraphs_sentences_and_whitespace 1000 0.007 1.945 0.169 0.123
datasource.wikitext.revision.wikicode 1000 0.003 5.013 0.164 0.085
datasource.tokenized(datasource.revision.text) 1000 0.003 0.974 0.115 0.086
datasource.english.words_to_watch.revision.matches 1000 0.004 0.64 0.11 0.081
datasource.wikitext.revision.content 1000 0 1.053 0.02 0.011
datasource.wikitext.revision.node_class_map 1000 0.001 0.759 0.018 0.013
datasource.enwiki.revision.paragraphs 1000 0 0.053 0.009 0.006
datasource.wikitext.revision.words 1000 0 0.05 0.009 0.006
datasource.enwiki.revision.paragraphs_without_refs 1000 0 0.11 0.005 0.002
datasource.wikitext.revision.tag_names 1000 0 0.075 0.002 0.001
datasource.map(<built-in function len>, <datasource.english.stemmed.revision.stems>) 1000 0 0.013 0.001 0.001
datasource.wikitext.revision.template_names 1000 0 0.013 0.001 0
datasource.wikitext.revision.wikilink_titles 1000 0 0.011 0.001 0.001
datasource.wikitext.revision.tags 1000 0 0.003 0 0
datasource.wikitext.revision.headings 1000 0 0 0 0
datasource.wikitext.revision.wikilinks 1000 0 0.001 0 0
datasource.revision.text 1000 0 0.007 0 0
datasource.wikitext.revision.headings_by_level(3) 1000 0 0.007 0 0
datasource.wikitext.revision.templates 1000 0 0.004 0 0
datasource.wikitext.revision.wikilink_titles_matching(Category:) 1000 0 0.007 0 0
datasource.wikitext.revision.template_names_matching(Main) 1000 0 0.006 0 0
datasource.wikitext.revision.template_names_matching(Who) 1000 0 0.008 0 0
datasource.wikitext.revision.wikilink_titles_matching(File Image:) 1000 0 0.009 0
datasource.wikitext.revision.tag_names_matching(ref) 1000 0 0.012 0 0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment