### A Pluto.jl notebook ###
# v0.20.21

using Markdown
using InteractiveUtils

# ╔═╡ cf42c692-dc36-11f0-b2a3-27f4334c30bd
md"""# Meta-Programming in Julia
#### *The code writes itself*

Lab Workshop – Nicolas Jacquin
"""

# ╔═╡ e6e3f304-6b91-4850-a1b8-87dbd6158afe
md"""
### Section 1: Why?
Sometimes, we need to work with the code syntax directly, treating it as data. This comes up often for logging, benchmarking and monitoring (think of `@time`` or @btime` from `BenchmarkTools` and `@showprogress` from `ProgressMeter`), adding extra features to existing code without rewriting it (think of `@threads`, `@async`), changing the meaning of some code structures (with @assert to make something a unit test) or cheeky compiler tricks ([@turbo](https://juliasimd.github.io/LoopVectorization.jl/latest/api/#LoopVectorization.@turbo), which promises the compiler your code doesn't index out of bounds, iterate over empty collections, order doesn't matter, etc...)

We want to write things like this: ```@log_step x + y * z```

And get outputs like this: ```[LOG] x + y * z = 42```

This is behaviour that's difficult to emulate with a function. Behold, an abberation:
"""

# ╔═╡ 999c7979-2804-4eb4-aedb-367894a9e5bf
begin
	function log_step(expr, value)
	    println("[LOG] ", expr, " = ", value)
	    value
	end
	a = 2; b = 3; c = 4
	log_step("a + b * c", a + b * c)
end

# ╔═╡ 04ee0e97-4ff6-4e88-8db0-9668f63a03ca
md"""
This is worse than useless. I need to pay attention to it every time I refactor the code to make sure the expression stays up to date with what the code represents, meaning the string can lie. An error is pretty much guaranted to happen eventually.

Fortunately, julia allows us to manipulate the code's syntax tree to create macros, such as `@time` or `@assert` to treat code as data. Let's use another macro to take a look at what `@assert`, as in this workshop, we will learn how to do exactely that.
"""

# ╔═╡ bcabeb33-99f6-4eda-9200-c0469fecfd06
@macroexpand(@assert 1+1==2)

# ╔═╡ b3cdf287-ccef-4595-9d7e-d204f85cf49c
begin
	@assert 1+1 == 2
	@assert 1+1 == 3
end

# ╔═╡ 52c6e07c-3ee3-43ea-a45f-90fb700e45c2
md"""
### Section 2: Code is data

In julia, code itself is a data structure of type `Expr`. You can explicitely instantiate `Expr` objects using the : (quote) operator. This is called "quoting", it tells julia that this code should not be ran (yet), but represented
"""

# ╔═╡ 93ce6b10-781b-4647-bc8c-e604d4627ac2
begin
	ex = :(a + b * c)
	println(typeof(ex))
end

# ╔═╡ d51a0088-d7d0-4215-862d-6f056ceae37d
md"""
We can use the `dump` function to look at how the `Expr` struct is actually organised, you'll see the code itself is represented within the struct."""

# ╔═╡ 7dd2eda9-c0bf-481f-9606-c0ec2a75075f
dump(ex)

# ╔═╡ de4938b4-8ab6-4f70-8e69-37c7457294a8
md"""
To analyse an `Expr`, julia (and many other languages), organises it into a syntax tree, a way to systematically describe the code structure."""

# ╔═╡ 2735268b-715e-4e5c-97d6-ad6ff02147ba
Meta.show_sexpr(ex)

# ╔═╡ addd90b9-dd18-45e9-a4bc-9c6de77f77ca
md"""
If you're familiar with Lisp, this syntax should be familiar to you. The root node is `:call`, `:+` is and argument and the function to call, `:a` and `(:call, :*, :b, :c)` are arguments to `:+`, with the second one also being an expression by itself.
We can isolate these elements specifically
"""

# ╔═╡ 398735a0-23d2-4ef3-95ce-95a93a0bc505
begin
	println(ex.head)
	println(ex.args)
end

# ╔═╡ ce6b6465-ab29-4553-ad85-086107560f53
md"""
We can do `Expr` "blocks" to prepare ourselves to write macros
"""

# ╔═╡ 4ec8af12-5d0a-4699-9b5e-ffd66c87c52e
begin
	blk = quote
	    x = 1
	    y = 2
	    x + y
	end
	
	dump(blk)
end

# ╔═╡ 9ef93c45-8a29-4db6-81e7-1fd2d3f34c86
md"""#### Section 3: Running quoted code
We know we can write code as data, but none of that is useful if we can't *run* said data (code).

- quoting (:) is generating code (an `Expr`)
- evaluating (eval) is running said code
"""

# ╔═╡ 1e06d46e-76ac-4130-8aea-caaf9142f9e5
begin 
	x = 10
	println("quoted code expression: ", :(x + 1))
	println("evaluated code expression: ", eval(:(x + 1)))
end

# ╔═╡ 008d7154-f8b5-4e64-9518-7b4a65188ddc
md"""
Altough `eval` is usually not something you want inside a macro as it runs in global scope and breaks precompilation
"""

# ╔═╡ 40b6567d-8d0d-47d5-8b9f-be7b9aea4c65
md"""#### Section 4: Interpolation
We can represent code as `Expr` and run said code, but how do we actually make it useful programatically?

With interpolation, you can insert expressions into quoted code, allowing for a programmatic approach to `Expr` generation
"""

# ╔═╡ 131945b5-dc95-453a-8df9-a982365440d2
begin
	y = :a
	println(:(y + 1))
	println(:($y + 1))
end

# ╔═╡ c59836d9-caa7-4303-8d6d-ac3c921ebfff
md"""Interpolation is tree splicing (not just string concatenation), which allows you to generate structured code programatically (in a way that's sometimes not possible with functions)"""

# ╔═╡ ec279071-8aea-4cc6-b991-f2a1108e28f9
Meta.show_sexpr(:($y + 1))

# ╔═╡ 5f3d9367-c9be-4e5e-b519-484b9118e04c
md"""When you treat code as data, you can manipulate it in every way you would standard data, weaving whatever you want into the syntax tree the same way you would with actual values"""

# ╔═╡ 0c794ddf-722e-4dcf-a27d-4b30ad90ba18
begin 
	fields = [:a, :b, :c]
	
	exp = quote
	    $(fields[1]) + $(fields[2])
	end
	println(exp)
end

# ╔═╡ 4c032c9c-70e9-470e-93c1-92ad8731ca78
md"""
You can even do nested interpolation expression building
"""

# ╔═╡ 4ae4eee2-88b9-49b4-a56d-7506019017b4
begin
	println(:( $(Expr(:call, :+, :a, :b)) * c ))
end

# ╔═╡ d678d3d8-4f65-41b2-8e2c-46222b53b1fa
md"""# Section 5 - Macro Building
now that we know how to generate live code programatically, we can look at practical examples of macro building. Weaving around the syntax tree of our code to add useful bits.

Let's write a macro that adds timing around a block (essentially `@time`)
"""

# ╔═╡ 55d5695e-e7f9-47b3-994b-db1630a8eb8f
macro my_time(ex)
    quote
        t0 = time()
        result = $(ex)
        println("Elapsed: ", time() - t0)
        result
    end
end

# ╔═╡ 845934d5-25e1-4ffa-b094-4822ff07ca76
@my_time sum(rand(10^6))

# ╔═╡ 464f04f5-f9dd-4736-a6dc-728e8864132a
md"""
using the `macro` block, the quoted code will be evaluated when the macro is executed, and the `@my_macro` syntax can be used, where the first parameter of the macro is the `Expr` the macro is executed on

However, unlike a function, by default a macro does not have its own scope. This means we can create variable capture issues."""

# ╔═╡ 626bdd75-5c7a-49b7-bb39-670eee41a4f1
begin
	t0 = 100
	println(@my_time t0 + 1)
end

# ╔═╡ 299547c3-2ab5-475a-9167-074d74b99231
md"""
It is however quite easy to fix. `esc` tells julia "this code belongs to the caller, not the macro", essentially separating the macro's scope from the `Expr` passed to the macro"""

# ╔═╡ ce81a6a9-9b89-4826-ac12-5d790753b6d2
begin
	macro my_time_fixed(ex)
	    quote
	        t0 = time()
	        result = $(esc(ex))
	        println("Elapsed: ", time() - t0)
	        result
	    end
	end
end

# ╔═╡ e066b39c-4026-42ff-8d67-52bf89514039
begin
	@my_time println(t0 + 1)
	@my_time_fixed println(t0 + 1)
end

# ╔═╡ 5a136329-53b6-45e5-9f89-fcb68b68dcb3
md"""As a rule of thumb, you should always use `esc` to isolate the macro's code from the interpolated part (unless variable capture is the wanted behaviour, which can happen)."""

# ╔═╡ 1285eeed-292a-4b86-9a2e-50bc74026b55
md"""
#### Section 6 - Generated Functions
This is gonna get a bit more funky, but you can go a step further by generating function code at compile time, dependant of the argument types. This is done with the `@generated` macro. Step by step, it looks like this:

- Julia sees a call: f(x)
- Julia infers the types of x
- Julia sees f is @generated
- Julia calls the generator with the inferred types
- The generator returns an `Expr`
- That expression is compiled as the method body
- Runtime execution happens later

Generated functions allow you to create new functions programatically. This is similar to what you do with function parametrization using type overloading, but with a lot more control. 

Demonstration:

"""

# ╔═╡ 95a917ec-6548-48b6-a5ac-d989d64fec53
begin
	function sum_tuple(t::NTuple{N,Int}) where N
	    s = 0
	    for i in 1:N
	        s += t[i]
	    end
	    s
	end
	println(sum_tuple((1, 2, 3)))
end

# ╔═╡ f6e11500-caa2-4b5b-8c85-870f91446fca
md"""
This is simple, and it works, however it's not guaranted the compiler will unroll the loop (serializing it as a series of sum).
"""

# ╔═╡ d29caee3-ca6c-41d9-85c1-e672c8c0aa21
println(@code_typed sum_tuple((1, 2, 3)))

# ╔═╡ 856171ab-2cf2-4a74-a438-15f79272d1c6
md""" Those gotos are the loop, in compiler language. Let's build a version that explicitely unrolls the loop."""

# ╔═╡ 8dc83209-309e-4582-a6e7-cfeec1e042a3
begin
	@generated function sum_tuple_generated(t::NTuple{N,Int}) where N
    terms = [:(t[$i]) for i in 1:N]
    reduce((a,b) -> :($a + $b), terms)
	end
	println(sum_tuple_generated((1, 2, 3)))
end

# ╔═╡ d355ab9c-d262-4a2c-a72e-a9c5f7373939
md"""We now programatically generate code that sums all the elements in the tuple (in other word, from the tupe we litterally generate `t[1] + t[2] + t[3]...`)
If we inspect the code, you'll see that **there's no loop**"""

# ╔═╡ bff7a1d0-5697-43b9-a0c8-f7e72115c9c4
println(@code_typed sum_tuple_generated((1, 2, 3)))

# ╔═╡ 031833d1-302b-4848-bf3f-fdbbc66bfe95
md"""
Metaprogamming tools are useful to manipulate the code itself, however it should be used sparingly (as it is not very intuitive to read), do not use it when: 
- A function works
- Multiple dispatch works
- Clarity suffers too much

Thank you for your attention :D
"""


# ╔═╡ 00000000-0000-0000-0000-000000000001
PLUTO_PROJECT_TOML_CONTENTS = """
[deps]
"""

# ╔═╡ 00000000-0000-0000-0000-000000000002
PLUTO_MANIFEST_TOML_CONTENTS = """
# This file is machine-generated - editing it directly is not advised

julia_version = "1.12.3"
manifest_format = "2.0"
project_hash = "71853c6197a6a7f222db0f1978c7cb232b87c5ee"

[deps]
"""

# ╔═╡ Cell order:
# ╟─cf42c692-dc36-11f0-b2a3-27f4334c30bd
# ╟─e6e3f304-6b91-4850-a1b8-87dbd6158afe
# ╠═999c7979-2804-4eb4-aedb-367894a9e5bf
# ╠═04ee0e97-4ff6-4e88-8db0-9668f63a03ca
# ╠═bcabeb33-99f6-4eda-9200-c0469fecfd06
# ╠═b3cdf287-ccef-4595-9d7e-d204f85cf49c
# ╟─52c6e07c-3ee3-43ea-a45f-90fb700e45c2
# ╠═93ce6b10-781b-4647-bc8c-e604d4627ac2
# ╟─d51a0088-d7d0-4215-862d-6f056ceae37d
# ╠═7dd2eda9-c0bf-481f-9606-c0ec2a75075f
# ╟─de4938b4-8ab6-4f70-8e69-37c7457294a8
# ╠═2735268b-715e-4e5c-97d6-ad6ff02147ba
# ╟─addd90b9-dd18-45e9-a4bc-9c6de77f77ca
# ╠═398735a0-23d2-4ef3-95ce-95a93a0bc505
# ╟─ce6b6465-ab29-4553-ad85-086107560f53
# ╠═4ec8af12-5d0a-4699-9b5e-ffd66c87c52e
# ╟─9ef93c45-8a29-4db6-81e7-1fd2d3f34c86
# ╠═1e06d46e-76ac-4130-8aea-caaf9142f9e5
# ╟─008d7154-f8b5-4e64-9518-7b4a65188ddc
# ╟─40b6567d-8d0d-47d5-8b9f-be7b9aea4c65
# ╠═131945b5-dc95-453a-8df9-a982365440d2
# ╟─c59836d9-caa7-4303-8d6d-ac3c921ebfff
# ╠═ec279071-8aea-4cc6-b991-f2a1108e28f9
# ╟─5f3d9367-c9be-4e5e-b519-484b9118e04c
# ╠═0c794ddf-722e-4dcf-a27d-4b30ad90ba18
# ╟─4c032c9c-70e9-470e-93c1-92ad8731ca78
# ╠═4ae4eee2-88b9-49b4-a56d-7506019017b4
# ╟─d678d3d8-4f65-41b2-8e2c-46222b53b1fa
# ╠═55d5695e-e7f9-47b3-994b-db1630a8eb8f
# ╠═845934d5-25e1-4ffa-b094-4822ff07ca76
# ╟─464f04f5-f9dd-4736-a6dc-728e8864132a
# ╠═626bdd75-5c7a-49b7-bb39-670eee41a4f1
# ╟─299547c3-2ab5-475a-9167-074d74b99231
# ╠═ce81a6a9-9b89-4826-ac12-5d790753b6d2
# ╠═e066b39c-4026-42ff-8d67-52bf89514039
# ╠═5a136329-53b6-45e5-9f89-fcb68b68dcb3
# ╟─1285eeed-292a-4b86-9a2e-50bc74026b55
# ╠═95a917ec-6548-48b6-a5ac-d989d64fec53
# ╟─f6e11500-caa2-4b5b-8c85-870f91446fca
# ╠═d29caee3-ca6c-41d9-85c1-e672c8c0aa21
# ╟─856171ab-2cf2-4a74-a438-15f79272d1c6
# ╠═8dc83209-309e-4582-a6e7-cfeec1e042a3
# ╟─d355ab9c-d262-4a2c-a72e-a9c5f7373939
# ╠═bff7a1d0-5697-43b9-a0c8-f7e72115c9c4
# ╟─031833d1-302b-4848-bf3f-fdbbc66bfe95
# ╟─00000000-0000-0000-0000-000000000001
# ╟─00000000-0000-0000-0000-000000000002