PlutoLogging.jl

PlutoLogging.jl

### A Pluto.jl notebook ###
# v0.11.3

using Markdown
using InteractiveUtils

# ╔═╡ 7154a0e6-d7a3-11ea-2657-db1e00c80460
md"Define a function which will do some logging. Note that this could just as well be in an external library"

# ╔═╡ 514ac9ea-d7a0-11ea-1512-e99df979d240
md"Let's collect some logs using `with_logger`, and `Test.TestLogger` for a quick and dirty way to demonstrate this (`TestLogger` is a semi-internal API). In practice, Pluto.jl may want its own logger."

# ╔═╡ 86794d1c-d7a0-11ea-393b-f540f7fd608b
md"Format the logs as a tabular view.

In practice this would presumably be part of the Pluto cell UI somehow; perhaps folded away but available in a popup.

Obviously there's a lot of options for how to do this nicely, and it would be cool if there was UI to do some light filtering, and perhaps an option to map variables to column of the table, and to display Julia values richly in a similar way to the default Pluto output.

The example here is a bit of a formatting disaster, really!
"

# ╔═╡ 16cb8602-d79f-11ea-1230-55a5ece049fc
md"A function to create a tabular view of logging info"

# ╔═╡ f6ebcfb8-d79e-11ea-00e5-79a5d6e0d812
md"Primitive styling experiments. Pardon the mess, I know only enough HTML/CSS to be dangerous :)"

# ╔═╡ 9a565fe6-d796-11ea-01ef-5f2e3c3bbbd0
html"""
<table style="width:100%">
<tr><th style="width:10%">Level</th><th>Module</th><th>Message</th></tr>
<tr style="background-color:#DDDDFF;"><td>Debug</td><td title="Hover!">Base</td><td>"Debugging!"</td></tr>
<tr style="background-color:#DDDDDD;"><td>Info</td><td>Base</td><td>"Some information"</td></tr>
<tr style="background-color:#FFEEDD;"><td>Warn</td><td>MyPkg</td><td>"Uh oh"</td></tr>
<tr style="background-color:#FFDDDD;"><td>Error</td><td>MyPkg</td><td>"Bad things"</td></tr>

</table>
"""

# ╔═╡ e1859b80-d7a4-11ea-2bcd-ed8a0cbf4c7d
html"""
<detail style="display: block;">
<summary>
A summary
</summary>
<p>
Hi blah blah
</p>
</detail>
"""

# ╔═╡ 176456f8-d7ad-11ea-0796-158017db0c16
md"Trying to make use of Pluto's builtin value formatting... Almost, but not quite.  Some styling thing is going wrong here!"

# ╔═╡ 47652cb6-d79d-11ea-33c4-c175255a6101
using Logging, Test, TerminalLoggers, Markdown

# ╔═╡ 86d5c12c-d7a6-11ea-2733-f7e48ab50543
using Pluto

# ╔═╡ 5239d6cc-d7a3-11ea-03a6-5d92a5ef9538
function some_logging(x)
	@debug "Debugging some julia code! `x^2`" x x^2
	for i=1:3
		@info "Iterating" i
	end
end

# ╔═╡ 2cb8772e-d79d-11ea-1e80-9fb5faaa2b63
logs = begin
	# May not want debug level by default... but it's here for demonstration
	t = Test.TestLogger(min_level=Logging.Debug)

	with_logger(t) do
		x = rand(UInt8, 2,2)
		some_logging(x)
		@warn """
			Uh oh, something weird happened and we needed to warn you about it. You could try:
		
			* foo
			* bar
			"""
		try
			1÷0
		catch ex
			@error "An error" exception=(ex,catch_backtrace())
		end
	end

	t.logs
end

# ╔═╡ 96d92068-d79d-11ea-2eb4-1d4fbe697824
level_colors = Dict(
	Logging.Debug=>"#DDDDFF",
	Logging.Info=>"#DDDDDD",
	Logging.Warn=>"#FFEEDD",
	Logging.Error=>"#FFDDDD"
)

# ╔═╡ a673f174-d79d-11ea-3290-f76c71cb8cba
function make_logs_table(logs)
	rows = []
	for log in logs
		# The message should be formatted as markdown
		message = if log.message isa AbstractString
			sprint() do io
				show(io, MIME"text/html"(), Markdown.parse(log.message))
			end
		else
			sprint(show, log.message)
		end
		if !isempty(log.kwargs)
			# Hooking into Pluto's output system would be nice,
			#   but I've busted the formatting somehow...
			# vars = first(Pluto.PlutoRunner.format_output(log.kwargs))
			ctx = IOContext(devnull, :displaysize=>(200,100))
			var_rows = TerminalLoggers.format_key_value_pairs(log.kwargs, ctx)
			vars_detail = join([' '^r[1]*r[2] for r in var_rows], "\n")
			vars = """
			<pre>$vars_detail</pre>
			"""
		else
			vars = ""
		end
		# Drop the Pluto cell from the file name
		file,cell = split(string(log.file), "#==#")
		push!(rows,
			"""
			<tr style="background-color:$(level_colors[log.level]);">
				<td title="$(log.level) @ $(log._module) — $(file):$(log.line)">
					$message</td>
				<td>$vars</td>
			</tr>
			""")
	end
	HTML("""
		<table>
		<tr>
			<th>Message</th>
			<th  style="width:40%">Variables</th>
		</tr>
		$(join(rows, "\n"))
		</table>
		""")
end

# ╔═╡ a42f5718-d79e-11ea-0335-7f2ba6bbf61f
make_logs_table(logs)

# ╔═╡ 98455c7c-d7a6-11ea-17b7-15bb344ad833
HTML("""<pluto-output class="inline_output "><div>$(first(Pluto.PlutoRunner.format_output(logs[1]))))</div></pluto-output>""")

# ╔═╡ ba0f02a0-d7a5-11ea-2f91-75041665e3e5
# Trying to make use of Pluto's builtin value formatting... Almost, but not quite.  Some styling thing is going wrong here!
HTML(sprint(show, MIME"application/vnd.pluto.tree+xml"(), Dict(string(k)=>v for (k,v) in logs[1].kwargs)))

# ╔═╡ a0469142-d7ae-11ea-009a-79e8b45cb010
logs[1]

# ╔═╡ Cell order:
# ╟─7154a0e6-d7a3-11ea-2657-db1e00c80460
# ╠═5239d6cc-d7a3-11ea-03a6-5d92a5ef9538
# ╟─514ac9ea-d7a0-11ea-1512-e99df979d240
# ╠═2cb8772e-d79d-11ea-1e80-9fb5faaa2b63
# ╟─86794d1c-d7a0-11ea-393b-f540f7fd608b
# ╠═a42f5718-d79e-11ea-0335-7f2ba6bbf61f
# ╠═96d92068-d79d-11ea-2eb4-1d4fbe697824
# ╟─16cb8602-d79f-11ea-1230-55a5ece049fc
# ╠═a673f174-d79d-11ea-3290-f76c71cb8cba
# ╠═47652cb6-d79d-11ea-33c4-c175255a6101
# ╟─f6ebcfb8-d79e-11ea-00e5-79a5d6e0d812
# ╠═9a565fe6-d796-11ea-01ef-5f2e3c3bbbd0
# ╠═e1859b80-d7a4-11ea-2bcd-ed8a0cbf4c7d
# ╠═86d5c12c-d7a6-11ea-2733-f7e48ab50543
# ╟─176456f8-d7ad-11ea-0796-158017db0c16
# ╠═98455c7c-d7a6-11ea-17b7-15bb344ad833
# ╠═ba0f02a0-d7a5-11ea-2f91-75041665e3e5
# ╠═a0469142-d7ae-11ea-009a-79e8b45cb010

But not sure what would happen if @distributed or @threads is in the block.

fonsp/Pluto.jl#437

Let's keep things simple for now

agree to stick to simplicity in order to be able to maintain things...
But how do you plan to define @info etc ? Via a PlutoLogger <:AbstractLogger pushing data to your channel ?

ok I see it in cell 2cb8772e ....

But then I think the re-routing could be even simpler once the PlutoLogger is already running.
There might be a @from_stdout loglevel which could control this.

macro log_stdout(block)
    quote
	original_stdout = stdout
        out_rd, out_wr = redirect_stdout()
        function myread(out_rd)
            while !eof(out_rd)
                @info readline(out_rd)
            end
            nothing
        end
        reader = @async myread(out_rd)
         $(esc(block))
	redirect_stdout(original_stdout)
        close(out_wr)
 end

If not in Pluto this could also go into PlutoUI instead of the current idea. I would wait for fonsp/Pluto.jl#437 and try this out.

Yep! We will use Base.@info, and define our own logger! I got it working today:

Great! Curious where this leads us!

☺

@fonsp I'm happy to help with integrating this as much as you need. I think the tooling on the Julia side is actually almost all there, it's the UI which seems like a lot of work to me ;-)

route stdout lines to @info messages

@j-fu note that Logging2.jl library already has a way to do this. It's nascent but I can release a version if it's useful.

I see. Indeed it seems that Logging2.jl has what I would love to see, with a more profound way to implement this.
IMHO worth to be merged into Logging.jl ...

Yeah, that's the plan — it's basically a staging area for Logging. The problem with Logging is that — like all other stdlibs — its release cycle is tied to releases of Julia. So for now Logging2 is separate.