Skip to content

Instantly share code, notes, and snippets.

@AdamNaj

AdamNaj/Find-MostExecutedSolrQuery.ps1

Last active June 2, 2024 18:34
Download ZIP

Revisions

  1. AdamNaj revised this gist Jun 2, 2024. 1 changed file with 31 additions and 9 deletions.
    40 changes: 31 additions & 9 deletions Find-MostExecutedSolrQueryOnDiskDrive.ps1
    View file
    Original file line number Diff line number Diff line change
    @@ -1,6 +1,6 @@
    Clear-Host
    # Gets the latest 30 log files, filters only solr query lines from them, groups them by query content and orders by most executed,
    # Letting you find things that are most effective to cache.
    # Gets large number of large log files, filters only Solr query lines from them, saves it to a file
    # later takes that file and counts each queries more memory effectively

    $logFileFilter = "azure.log.*.*"
    $logfileCountToAnalyse = 200
    @@ -11,11 +11,33 @@ $files = Get-ChildItem $SitecoreLogFolder -Recurse -Filter $logFileFilter `
    | Sort -Property LastWriteTime -Descending `
    | Select -first $logfileCountToAnalyse

    $i = 0;
    $count = $files.Count

    $files | Get-Content `
    | ? {$_.Contains("INFO Solr Query - ?") } `
    | % { $_.Substring($_.IndexOf("Solr Query - ?")+13) } `
    | Group-Object `
    | ? { $_.Count -gt $minQueryOccurence } `
    | Sort-Object -Property Count -Descending `
    | ft Count, Name
    $files | %{
    $i = $i +1;
    Write-Progress "$_.Name" -PercentComplete ($i/$count*100);
    $_ }
    | Get-Content `
    | ? {$_.Contains("INFO Solr Query - ?") } `
    | % { $_.Substring($_.IndexOf("Solr Query - ?")+13) }
    | Set-Content "$SitecoreLogFolder.txt"

    $queries = @{}

    Get-Content "$SitecoreLogFolder.txt" |
    ForEach-Object {
    if($queries.Contains($_)){
    $queries[$_] = $queries[$_]+1
    } else {
    $queries[$_] = 1;
    }
    }

    $queries. Keys |
    ForEach-Object { [PSCustomObject]@{Query = $_; Count=$queries[$_] } } |
    Sort-Object -Property Count,Query -Descending |
    Where-Object { $_.Count -gt 100} |
    Format-Table Count, Query |
    Out-String |
    Set-Content "$SitecoreLogFolder.QueryContent.txt"
  2. AdamNaj revised this gist Jun 2, 2024. 1 changed file with 21 additions and 0 deletions.
    21 changes: 21 additions & 0 deletions Find-MostExecutedSolrQueryOnDiskDrive.ps1
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,21 @@
    Clear-Host
    # Gets the latest 30 log files, filters only solr query lines from them, groups them by query content and orders by most executed,
    # Letting you find things that are most effective to cache.

    $logFileFilter = "azure.log.*.*"
    $logfileCountToAnalyse = 200
    $minQueryOccurence = 100
    $SitecoreLogFolder = "C:\Projects\CD Logs\20240522"

    $files = Get-ChildItem $SitecoreLogFolder -Recurse -Filter $logFileFilter `
    | Sort -Property LastWriteTime -Descending `
    | Select -first $logfileCountToAnalyse


    $files | Get-Content `
    | ? {$_.Contains("INFO Solr Query - ?") } `
    | % { $_.Substring($_.IndexOf("Solr Query - ?")+13) } `
    | Group-Object `
    | ? { $_.Count -gt $minQueryOccurence } `
    | Sort-Object -Property Count -Descending `
    | ft Count, Name
  3. AdamNaj created this gist May 22, 2024.
    19 changes: 19 additions & 0 deletions Find-MostExecutedSolrQuery.ps1
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,19 @@
    # Gets the latest 30 log files, filters only solr query lines from them, groups them by query content and orders by most executed,
    # Letting you find things that are most effective to cache.

    Set-HostProperty -HostWidth 2000

    $logFileFilter = "azure.log.*"
    $logfileCountToAnalyse = 30
    $minQueryOccurence = 20

    Get-ChildItem $SitecoreLogFolder -Recurse -Filter $logFileFilter `
    | Sort -Property LastWriteTime -Descending `
    | Select -first $logfileCountToAnalyse `
    | Get-Content `
    | ? {$_.Contains("INFO Solr Query - ?") } `
    | % { $_.Substring($_.IndexOf("Solr Query - ?")+13) } `
    | Group-Object `
    | ? { $_.Count -gt $minQueryOccurence } `
    | Sort-Object -Property Count -Descending `
    | ft Count, Name