How to get the equations for TemplateExpressionSpec and how to use it to define a custom objective function

[cristinarg9]

Hi!!
I am new to PySR and am trying to use it to create symbolic equations within a larger predefined expression:
$$y=\alpha_1 * f(x_1, x_2) + \alpha_2 * g(x_1, x_2, x_3)$$
Here is the code I am using:

template = TemplateExpressionSpec(
    expressions=["f", "g"],
    variable_names=['x1', 'x2', 'x3', 'x4'],
    parameters={"p1": 1, "p2": 1},
    combine="p1[1] * f(x1, x2) + p2[1] * g(x1, x3, x4)",
)

I have two questions regarding the implementation of this problem:

1. Can I export the equations to LaTeX when using TemplateExpressionSpec with multiple expressions? I am encountering the following error:

model.latex()
ValueError: `expression_spec=<pysr.expression_specs.TemplateExpressionSpec object at 0x000002AF10F06810>` does not support latex export.

Also, I am using Spyder and cannot visualize the variable I saved with the equations as: equations = model.equations_.

2. I would like to perform optimization by adjusting a loss function structured as follows:

$$F =(y_{real}-y_{predicted})+\beta_1 * (f_{real}-f_{predicted}) + \beta_2 * (g_{real}-g_{predicted})$$
where $\beta_1$ and $\beta_2$ are weight coefficients that I will define. I am unsure how to incorporate the expressions f and g into the loss function.

P.S. I want to thank Miles for his work on the development of PySR. It is an excellent tool, very well documented, and a pleasure to use so far.

[MilesCranmer]

1. Correct, template expressions do not support exporting to latex. (Let me know if you have any suggestions to make the error message a bit more clear? I wasn't sure how I could put this in an error message to make it clear that it is not supported.)
2. For this I would add the additional labels as features:

beta_1 = 1.0  # SET ME
beta_2 = 2.0  # SET ME

template = TemplateExpressionSpec(
    expressions=["f", "g"],
    variable_names=["x1", "x2", "x3", "x4", "y_real", "f_real", "g_real"]
    parameters={"p": 2},
    combine=f"""
        f_predicted = f(x1, x2)
        g_predicted = g(x1, x3, x4)
        y_predicted = p[1] * f_predicted + p[2] * g_predicted
        
        (y_real - y_predicted)^2 + {beta_1} * (f_real - f_predicted)^2 + {beta_2} * (g_real - g_predicted)^2
    """,
)

model = PySRRegressor(expression_spec=template, elementwise_loss="L1DistLoss()", **kwargs)

# Set the target to be 0:
model.fit(X, np.zeros(X.shape[0]))

I'm assuming you want to take the mean-squared error (?) which is why I wrote these out as ^2.

I use L1DistLoss() here so that we don't apply the square twice (regular is L2DistLoss()).

Note that we would need to pass y_real, f_real, and g_real as additional features of X. Note also that we set the y target to simply be 0.0, as a workaround because we have the template expression outputting 0.0 for all rows if the loss is perfect.

The tricky thing about a loss like this is that you can't use model.predict, because you have repurposed the template as a loss function. If you need that it may be better to use the Julia backend directly (allows more flexibility in the objects you pass - you can just fit on 3-tuples directly), or write a custom loss_function_expression (see other examples on the forum, like from @gm89uk).

---

Thanks for the kind message!

[MilesCranmer]

Btw, for 1, you can just do list(model.equations_.equation) to get the string representations for each expression

[cristinarg9]

Hi Miles!
Thanks a lot for your detailed answer, really appreciated! 😊

1 - I was just wondering if the LaTeX export was being developed, but it’s totally fine if it’s not. I’ll manage with model.equations_.
2 - The option of including predictions in X seems like a viable approach for my problem. If I need to push further, I’ll explore the Julia backend. In any case, it’s very helpful to know how far I can go with Python.

[MilesCranmer]

Hi @cristinarg9,

I don't know anybody who is currently working on this. I don't think it should be too bad to add if you are interested?

Another option is to integrate Latexify.jl with the backend which would make this all easier. I started this here: MilesCranmer/SymbolicRegression.jl#280 but haven't had time to finish.

Cheers,
Miles

[cristinarg9]

Hi Miles,

Thanks for the info! There’s no need to add it for my case, I can manage with what’s currently available.

Cheers,
Cristina