Expected performance

[dfdx]

Recently I've rewritten Yota's tracer from Cassette to JuliaInterpreter which I found more flexible and much easier to debug. However, the cost of this move is extreme slowdown - a simple neural network which usually takes ~1 second now takes almost 10 minutes in tracer, and most of this time is due to JuliaInterpreter (I can provide exact code and timings if needed).

I see that a lot of work is being done to improve the situation, but in the end what slowdown should we expect? Is it 5x, 10x or 100x compared to runtime of normal call (including compilation time to make it fair)?

[KristofferC]

We are at a point where the current strategy doesn't have any real low hanging fruits left (at least not at the order of magnitude level). More drastic things like #309 will likely improve the situation but it is not clear by how much and if (or at all) that will ever be implemented.

[dfdx]

Got it, thanks!
Just for my own reference in future, here's a simple snippet to test my use case:

using JuliaInterpreter
using BenchmarkTools

linear(W, b, x) = W * x .+ b
relu(x) = max.(x, 0)
net(W1, b1, W2, b2, W3, b3, x) = linear(W3, b3, linear(W2, b2, linear(W1, b1, x)))
W = rand(100, 100)
b = rand(100)

@btime net(W, b, W, b, W, b, x)
#  225.147 μs (12 allocations: 469.22 KiB)

@btime @interpret net(W, b, W, b, W, b, x)
  16.585 s (46406343 allocations: 1.59 GiB)