mismatch between the streaming logs that we get from azure-ai-ml and the log file we download at the end

[miguelgfierro]

Description

Related to #2222 (comment)

Related Issues

References

Checklist:

• I have followed the contribution guidelines and code style for this project.
• I have added tests covering my contributions.
• I have updated the documentation accordingly.
• I have signed the commits, e.g. git commit -s -m "your commit message".
• This PR is being made to staging branch AND NOT TO main branch.

[SimonYansenZhao]

After I changed azureml-related packages to older versions:

the logs became valid:

[SimonYansenZhao]

The latest log downloaded via azure mlflow in the run triggered last week

is the same as the log 3 months ago

[miguelgfierro]

After I changed azureml-related packages to older versions:

@SimonYansenZhao but where is the problem coming from? Is it azure-ai-ml or mlfow?

I'm going to try a few combinations:

• azure-ai-ml>1,<=1.23 and azureml-mlflow>1,<2 -> failed , it needs marshmallow
• azure-ai-ml>1,<=1.18.0 and azureml-mlflow>1,<2 -> failed, it needs marshmallow
• azure-ai-ml>1,<=1.18.0 and azureml-mlflow>1,<2 and marshmallow>3,<5 -> failed with marshmallow 4.0
• azure-ai-ml>1,<=1.18.0 and azureml-mlflow>1,<2 and marshmallow>3,<4 -> failed with marshmallow 3.26.1
• azure-ai-ml>1,<=1.18.0 and azureml-mlflow>1,<2 and marshmallow>3,<=3.18 -> the azureml works but the test fails because of [BUG] Issue with Surprise and numpy #2224
• azure-ai-ml>1,<2 and azureml-mlflow>1,<2 and marshmallow>3,<=3.18 -> the azureml works but the test fails because of [BUG] Issue with Surprise and numpy #2224
• azure-ai-ml>1,<2 and azureml-mlflow>1,<2 and marshmallow>3,<=3.22 ->the azureml works but the test fails because of [BUG] Issue with Surprise and numpy #2224
• azure-ai-ml>1,<2 and azureml-mlflow>1,<2 and marshmallow>3,<=3.25.1 -> the azureml works but the test fails because of [BUG] Issue with Surprise and numpy #2224
• azure-ai-ml>1,<2 and azureml-mlflow>1,<2 and marshmallow>3,<5 -> while I was doing the tests, there was a new version azure-ai-ml==1.26.4 now azureml works but the test fails because of [BUG] Issue with Surprise and numpy #2224
• azure-ai-ml>1,<=1.26.3 and azureml-mlflow>1,<2 and marshmallow>3,<5 -> while I was doing the tests, there was a new version azure-ai-ml==1.26.4 the azureml works but the test fails because of [BUG] Issue with Surprise and numpy #2224
• azure-ai-ml>1,<2 and azureml-mlflow>1,<2 -> the azureml works but the test fails because of [BUG] Issue with Surprise and numpy #2224
• azure-ai-ml>1,<=1.24.0 and azureml-mlflow>1,<2 -> it failed, it needs marshmallow
• azure-ai-ml>1,<=1.25.0 and azureml-mlflow>1,<2 -> it failed, it needs marshmallow
• azure-ai-ml>1,<=1.26.0 and azureml-mlflow>1,<2 -> it failed, it needs marshmallow
• azure-ai-ml>1,<=1.26.2 and azureml-mlflow>1,<2 ->it failed, it needs marshmallow
• azure-ai-ml>1,<=1.26.3 and azureml-mlflow>1,<2 -> it failed, it needs marshmallow
• azure-ai-ml>1,<=1.26.4 and azureml-mlflow>1,<2 -> the azureml works but the test fails because of [BUG] Issue with Surprise and numpy #2224

[SimonYansenZhao]

I don't think the problem is from mlflow because mlflow only defines the universal interfaces for azure-ai-ml to use. mlflow doesn't know how to get the logs from AzureML workspace but azure-ai-ml knows.

[miguelgfierro]

@SimonYansenZhao please review the tests I've made. At the end, the solution I found it works is: pip install "azure-ai-ml>=1.26.4,<2" "azureml-mlflow>1,<2"
See https://github.com/recommenders-team/recommenders/actions/runs/14619686582/job/41018376740

Let me know if you think it is ok, and we can merge this. Then there are a few other bugs we need to fix on a different PR.

[SimonYansenZhao]

@miguelgfierro OK, let's merge it.

By looking at what you changed, I find the latest azure-ai-ml is used (i.e., 1.26.4, released just yesterday). It looks like the mismatched log issue is from the AzureML side, and they were aware of the issue and fixed it yesterday.

[SimonYansenZhao]

The new issues, I think, may have something to do with the out-dated surprise package.

[miguelgfierro]

@miguelgfierro OK, let's merge it.

ok merging