Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Select an option

  • Save PanosGreg/75840344df8e2fd3e8dbd3848c44fa41 to your computer and use it in GitHub Desktop.

Select an option

Save PanosGreg/75840344df8e2fd3e8dbd3848c44fa41 to your computer and use it in GitHub Desktop.
A more real-world example on how to do ser/des with Google's Protobuf in PowerShell
function Get-ProtoC {
<#
.SYNOPSIS
Download latest protoc.exe binary from github
The protoc tool is the protocol buffer compiler
Source: https://github.com/protocolbuffers/protobuf/releases
.EXAMPLE
Get-ProtoC -Path C:\temp
#>
[OutputType([System.IO.FileInfo])]
[cmdletbinding()]
param (
[string]$Path = 'C:\Temp'
)
#Requires -Modules Microsoft.PowerShell.Utility, Microsoft.PowerShell.Archive, Microsoft.PowerShell.Management
# get the download url for the zip file
$repo = 'protobuf'
$org = 'protocolbuffers'
$releases = "https://api.github.com/repos/$org/$repo/releases"
$all = Invoke-RestMethod -Uri $releases -Verbose:$false
$win64 = $all[0].assets | where name -like *win64.zip # <-- the first one in the list is the latest version (that's the [0] index)
$zipfile = Join-Path $Path $win64.name
# download the zip file
if (Test-Path $zipfile) {
Write-Warning "The zip file $zipfile already exists, it will be overwritten"
Remove-Item -Path $zipfile -Force -Verbose:$false
}
$params = @{
Uri = $win64.url
Headers = @{Accept='application/octet-stream'}
OutFile = $zipfile
Verbose = $false
ErrorAction = 'Stop'
}
Invoke-WebRequest @params
if (-not (Test-Path $zipfile)) {
Write-Warning "Cannot find the downloaded file $zipfile, maybe the download had an error"
return
}
# uncompress it
$UnzipPath = $zipfile.Replace('.zip',$null)
if (Test-Path $UnzipPath) {Write-Warning "The path $UnzipPath already exists" ; return}
(Expand-Archive -Path $zipfile -DestinationPath $UnzipPath -Verbose:$false -EA Stop) 4>$null
if (-not (Test-Path $UnzipPath)) {
Write-Warning "Cannot find the unziiped folder $UnzipPath, maybe the zip extraction had errors"
return
}
# get the .exe binary
$exe = Join-Path $UnzipPath 'bin\protoc.exe'
if (-not (Test-Path $exe)) {
Write-Warning "Cannot find the .exe file $exe, you'll need to get it manually"
return
}
$FinalExe = (Join-Path $Path 'protoc.exe')
if (Test-Path $FinalExe) {
Write-Verbose "The .exe file $FinalExe already exists, it will be overwritten"
}
Copy-Item -Path $exe -Destination $Path -Verbose:$false -Force -ErrorAction Stop
# remove the unzipped folder
Remove-Item -Path $UnzipPath -Recurse -Force -Verbose:$false -ErrorAction Stop
if (Test-Path $UnzipPath) {
Write-Warning "Could not remove the unzipped folder $UnzipPath, you'll need to delete it manually"
}
# remove the zip file
Remove-Item -Path $zipfile -Force -Verbose:$false -ErrorAction Stop
# finally show the output
Get-Item $FinalExe
}
## Protobuf Serialization & Deserialization
# This is the process to serialize and deserialize data into/from protobuf
# Protobuf is supposed to be much smaller than JSON or XML and that's its main advantage
# Date: 09-Jun-2026
# you will need to following thing:
# - nuget tool (ex. scoop install nuget)
# the following was done in PowerShell 7.6.2 on Windows 11
# which means we are running on a .NET v10.0 runtime (that is PS v7.6)
# This example uses the "service.proto" sample file as the protocol buffer message definition
## ** Initial Setup **
# go to a sample folder (create one if need be)
cd (md C:\temp\ProtoService)
# download the protoc tool (that's the proto compiler)
# get it from: https://github.com/protocolbuffers/protobuf/releases
. C:\temp\Get-ProtoC.ps1
Get-ProtoC -Path C:\temp\ProtoService
# generate the .cs file from that .proto file
$DST_DIR = 'C:\temp\ProtoService'
.\protoc --proto_path=. --csharp_out="$DST_DIR" .\service.proto
# Note: this will generate the C# file c:\temp\proto\MyCustomProto\Service.cs
# download the Google.Protobuf library
cd (md ProtobufLibs)
nuget install Google.Protobuf
cd ..
# load the google library
$gDll = (dir 'ProtobufLibs\Google*\lib\net5.0\*.dll').FullName
Add-Type -Path $gDll
if (-not 'Google.Protobuf.IMessage' -as [type]) {Write-Warning 'Cannot find the Protobuf classes!!'}
# now load our library
$ref = [Google.Protobuf.MessageParser].Assembly.FullName
Add-Type -Path .\Service.cs -ReferencedAssemblies $ref -IgnoreWarnings -WarningAction Ignore
## ** Serialization **
# we'll do an example with the following windows services: WinRM,Spooler,RPCSS,DcomLaunch
# let's use our library
$ProtoSvcList = [ProtoServiceProcess.ServiceList]::new()
Get-Service WinRM,Spooler,RPCSS,DcomLaunch | foreach {
$obj = [ProtoServiceProcess.Service]@{
BinaryPathName = $_.BinaryPathName
DelayedAutoStart = $_.DelayedAutoStart
Description = $_.Description
DisplayName = $_.DisplayName
ServiceName = $_.ServiceName
ServiceType = $_.ServiceType.ToString() -as [ProtoServiceProcess.ServiceTypeEnum]
StartType = $_.StartType.ToString() -as [ProtoServiceProcess.StartTypeEnum]
Status = $_.Status.ToString() -as [ProtoServiceProcess.StatusEnum]
UserName = $_.UserName
}
if ($_.RequiredServices) {$obj.RequiredServices.AddRange([string[]]($_.RequiredServices.Name))}
if ($_.ServicesDependedOn) {$obj.ServicesDependedOn.AddRange([string[]]($_.ServicesDependedOn.Name))}
# Note: you cant set a value to the repeated properties (RequiredServices,ServicesDependedOn)
# you can only mutate them in place, by using their methods, for ex. .Add() or .AddRange(), etc..
$ProtoSvcList.Services.Add($obj)
}
# convert the data into a byte array
$MemStream = [System.IO.MemoryStream]::new()
$ProtoStream = [Google.Protobuf.CodedOutputStream]::new($MemStream)
$ProtoSvcList.WriteTo($ProtoStream)
$ProtoStream.Flush()
$SerializedBytes = $MemStream.ToArray()
# save the byte array into a file
$Path = Join-Path $PWD service.dat
$File = [System.IO.File]::Create($Path)
$File.Write($SerializedBytes,0,$SerializedBytes.Length)
$File.Close()
# clean up (order matters here)
$ProtoStream.Dispose()
$MemStream.Dispose()
$File.Dispose()
# you can now send that data anywhere you want
<# by now you have the following files in the c:\temp\proto folder
- Service.cs
- service.dat
- service.proto
- protoc.exe
And the \ProtobufLibs sub directory
#>
## ** Deserialization **
# To properly check the deserialization, we can open up a new powershell console
# to simulate a different environment
# load the google library
cd C:\temp\ProtoService
$gDll = (dir 'ProtobufLibs\Google*\lib\net5.0\*.dll').FullName
Add-Type -Path $gDll
# load up our class
$ref = [Google.Protobuf.MessageParser].Assembly.FullName
Add-Type -Path .\Service.cs -ReferencedAssemblies $ref -IgnoreWarnings -WarningAction Ignore
# Get the parser for the service message
$Parser = [ProtoServiceProcess.ServiceList]::Parser
# get the byte array from our data
$Path = 'C:\temp\ProtoService\service.dat'
$ByteData = [System.IO.File]::ReadAllBytes($Path) # <-- this needs a full path, not relative path
# Deserialize the byte array back into a service object
$DeserializedServices = $Parser.ParseFrom($ByteData)
### === Size Comparison
$list = Get-Service WinRM,Spooler,RPCSS,DcomLaunch | foreach {
@{
BinaryPathName = $_.BinaryPathName
DelayedAutoStart = $_.DelayedAutoStart
Description = $_.Description
DisplayName = $_.DisplayName
ServiceName = $_.ServiceName
ServiceType = $_.ServiceType
StartType = $_.StartType
Status = $_.Status
UserName = $_.UserName
RequiredServices = $_.RequiredServices.Name
ServicesDependedOn = $_.ServicesDependedOn.Name
}
}
($list | ConvertTo-Json -Compress).Length
# returns: 3049 bytes
# The size difference is not that big, because if we compress the json string (from $list) with gzip (native .net library)
# then the 2 sizes (the Protobuf and the Zipped Json) won't be that different
# compression details:
$json = $list | ConvertTo-Json -Compress
# compress the json string
$ms = [System.IO.MemoryStream]::new()
$mode = [System.IO.Compression.CompressionMode]::Compress
$gz = [System.IO.Compression.GZipStream]::new($ms, $mode)
$sw = [System.IO.StreamWriter]::new($gz)
$sw.Write($json)
$sw.Close()
$bytes = $ms.ToArray()
$OutStr = [System.Convert]::ToBase64String($bytes)
$ms.Close() # MemoryStream
$gz.Close() # GZipStream
$OutStr.Length # <-- returns 1472 bytes, which is smaller than the Protocol Buffer size
$OutStr | Out-File Base64CompressedJson.txt -Encoding UTF8
# decompression details (to get back the json string)
$InputString = Get-Content Base64CompressedJson.txt -Raw -Encoding UTF8
$data = [System.Convert]::FromBase64String($InputString)
$ms = [System.IO.MemoryStream]::new()
$ms.Write($data, 0, $data.Length)
$ms.Seek(0,0) | Out-Null
$mode = [System.IO.Compression.CompressionMode]::Decompress
$gz = [System.IO.Compression.GZipStream]::new($ms, $mode)
$sr = [System.IO.StreamReader]::new($gz)
$OutStr = $sr.ReadToEnd()
$sr.Close() # StreamReader
$ms.Close() # MemoryStream
$gz.Close() # GZipStream
$OutStr | ConvertFrom-Json # <-- you get back the custom service objects
## ** Verdict **
<#
- You cannot use Protobuf with adhoc objects, because you have to write up the .proto file in advance in order to
get a.cs and then load up the classes from the .cs file
Which means you need to know before hand the objects you are going to emit, and their corresponding properties
which makes the solution quite difficult to use on-the-fly. It's probably only good for pre-defined workflows
like a CICD pipeline where you know what kind of data you send/receive.
- We cant use the already thousands of .NET Framework objects (ex. Diagnostics.Process, ServiceController, etc)
because we have to supply/write a .proto file for each object we are going to serialize/deserialize
Again, this makes the solution quite cumbersome.
- The size benefits do not seem that great, at least when tested on a small scale (4 windows services)
Actually the zipped json string was smaller than the serialized protobuf data (which supposedly is compressed as well)
- We need an automated way (a script or function perhaps) that can take a .Net object and generate the .proto file for it
with all the relevant protobuf messages and enums.
Because the process to write the .proto file for each class and their properties, and also the enums required, is quite time consuming.
(essentially that would be a transpiler in a way, just like the protoc tool which takes a proto file and converts it to a C sharp file)
Date: 09-Jun-2026
#>
syntax = "proto3";
package windowsservice;
option csharp_namespace = "ProtoServiceProcess";
message ServiceList {
repeated Service Services = 1;
}
message Service { // <-- System.Service.ServiceController
string DisplayName = 1;
string ServiceName = 2;
repeated string ServicesDependedOn = 3;
repeated string RequiredServices = 4;
string UserName = 5;
string Description = 6;
string BinaryPathName = 7;
bool DelayedAutoStart = 8;
ServiceTypeEnum ServiceType = 9;
StartTypeEnum StartType = 10;
StatusEnum Status = 11;
}
enum ServiceTypeEnum { // <-- System.ServiceProcess.ServiceType
SERVICE_TYPE_ENUM_UNSPECIFIED = 0;
KernelDriver = 1;
FileSystemDriver = 2;
Adapter = 3;
RecognizerDriver = 4;
Win32OwnProcess = 5;
Win32ShareProcess = 6;
InteractiveProcess = 7;
}
enum StartTypeEnum { // <-- System.ServiceProcess.ServiceStartMode
START_TYPE_ENUM_UNSPECIFIED = 0;
Boot = 1;
System = 2;
Automatic = 3;
Manual = 4;
Disabled = 5;
}
enum StatusEnum { // <-- System.ServiceProcess.ServiceControllerStatus
STATUS_ENUM_UNSPECIFIED = 0;
Stopped = 1;
StartPending = 2;
StopPending = 3;
Running = 4;
ContinuePending = 5;
PausePending = 6;
Paused = 7;
}

Verdict

  • You cannot use Protobuf with adhoc objects, because you have to write up the .proto file in advance in order to get a.cs and then load up the classes from the .cs file

    Which means you need to know before hand the objects you are going to emit, and their corresponding properties which makes the solution quite difficult to use on-the-fly. It's probably only good for pre-defined workflows like a CICD pipeline where you know what kind of data you send/receive.

  • We cant use the already thousands of .NET Framework objects provided by MS (ex. Diagnostics.Process, ServiceController, etc) because we have to supply/write a .proto file for each object we are going to serialize/deserialize

    Again, this makes the solution quite cumbersome.

  • The size benefits do not seem that great, at least when tested on a small scale (4 windows services). Actually the zipped json string was smaller than the serialized protobuf data (which supposedly is compressed as well)

  • We need an automated way (a script or function perhaps) that can take a .Net object and generate the .proto file for it with all the relevant protobuf messages and enums.

    Because the process to write the .proto file for each class and their properties, and also the enums required, is quite time consuming.

    (essentially that would be a transpiler in a way, just like the protoc tool which takes a proto file and converts it to a C sharp file)

Date: 09-Jun-2026
Author: Panos Grigoriadis

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment